Information display device

Abstract

When the wearer of the HMD approaches an obstacle or exceeds a set boundary, the information display device outputs a warning to allow the wearer to recognize the limit of action.SOLUTION: An information display device comprising: a processor, wherein the processor moves a self-area set by a user in accordance with movement of the user, and determines presence of an obstacle to be warned based on the self-area.EFFECT: There is provided an information display device that enables a wearer to more suitably perceive the possibility of contact with an object in the real world while considering the wearer's sense of immersion. In addition, even in a case where the situation changes due to the movement of the wearer, it is possible to cope with the situation in the same manner.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information display device. [Background technology]

[0002] For example, virtual reality (VR) spaces are used in various fields, such as games, education, and tourism. To experience these virtual spaces (hereinafter sometimes referred to as VR spaces), a head-mounted display (HMD), an example of an information display device, is used. A head-mounted display (hereinafter sometimes referred to as HMD) is, for example, a device worn on the head and displays a virtual space image on a goggle-like display. This device is equipped with, for example, multiple sensors, such as a camera, a sensor for measuring distance to an object, and a position measurement sensor, as well as a CPU for image processing and a battery. When wearing this HMD and experiencing a VR space, depending on the content, the wearer may move around in the VR space. However, the actual space in which the wearer is located contains various objects (obstacles), such as walls and desks, limiting the areas in which the wearer can move around. Therefore, for safety reasons, a warning is issued when the HMD wearer approaches an obstacle. Patent Document 1 (Patent Document 1) discloses a technology of this type that enables appropriate identification of nearby objects that should be warned as obstacles. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-002290 Summary of the Invention [Problem to be solved by the invention]

[0004] The wearer can set an activity area to avoid obstacles, i.e., a boundary within which they can move around safely, and then experience the space displayed by the information display device. When the wearer approaches or crosses the set boundary, the information display device can output a warning to make the wearer aware of the limits of their actions.

[0005] Here, if a boundary is set that makes the warning area wider than necessary, for example, a warning may be output even when there is no actual possibility of a collision, which may result in a loss of immersion. On the other hand, if the warning area is inappropriate, a warning may not be output even when there is a possibility of a collision. Therefore, it is considered important to suppress the output of unnecessary warnings that damage the immersion and to set an area in which necessary warnings are output. [Means for solving the problem]

[0006] According to the present invention, there is provided an information display device as follows: The information display device includes a processor, which moves a user-defined self-region in accordance with the user's movement, and determines the presence of an obstacle for which a warning should be issued based on the user-defined self-region. [Effects of the Invention]

[0007] According to the present invention, an information display device is provided that allows the wearer to more appropriately perceive the possibility of contact with an object in the real world while taking into consideration the wearer's sense of immersion, and is also capable of responding similarly to situations where the situation changes as the wearer moves. [Brief explanation of the drawings]

[0008] [Figure 1A] FIG. 1 is a top view showing an example of an information display device. [Figure 1B] FIG. 1 is a front view showing an example of an information display device. [Figure 1C] FIG. 1 is a side view showing an example of an information display device. [Figure 2A]FIG. 2 is a top view showing an example of an information display device, illustrating an example of an imaging range of an external camera. [Figure 2B] FIG. 2 is a front view showing an example of an information display device, illustrating an example of an imaging range of an external camera. [Figure 2C] FIG. 2 is a side view showing an example of an information display device, illustrating an example of an imaging range of an external camera. [Figure 3] FIG. 2 is a block diagram showing an example of a hardware configuration of an information display device. [Figure 4] FIG. 2 is a functional block diagram for explaining an example of a function of the information display device. [Figure 5A] FIG. 2 is a front view showing an example of a control device. [Figure 5B] FIG. 2 is a side view showing an example of a control device. [Figure 6] FIG. 2 is a block diagram showing an example of a hardware configuration of a control device. [Figure 7A] 10 is a flowchart illustrating an example of the overall process in self-region setting. [Figure 7B] FIG. 10 is a diagram for explaining details of an example of floor detection processing. [Figure 7C] FIG. 10 is a diagram for explaining details of an example of a process for generating a trajectory. [Figure 7D] FIG. 10 is a diagram for explaining an example of a self region based on a generated trajectory. [Figure 8] 10 is a flowchart illustrating an example of a warning process. [Figure 9A] FIG. 10 is a diagram for explaining an example of the state of the user's own area when the user moves; [Figure 9B] FIG. 10 is a diagram for explaining an example of the state of the user's own area when the user moves; [Figure 9C] FIG. 10 is a diagram for explaining an example of the state of the user's own area when the user moves; [Figure 9D] FIG. 10 is a diagram for explaining an example of processing when a warning is displayed. [Figure 9E] FIG. 10 is a diagram for explaining an example of processing when a warning is displayed. [Figure 9F] FIG. 10 is a diagram for explaining an example of processing when a warning is displayed. [Figure 10A]FIG. 10 is a diagram illustrating an example of setting a self region according to a moving speed. [Figure 10B] FIG. 10 is a diagram for explaining an example of setting a self-region according to a user's posture. [Figure 11A] FIG. 10 is a diagram for explaining an example of a process for updating and enlarging a self region. [Figure 11B] FIG. 10 is a diagram for explaining an example of a process for updating and enlarging a self region. [Figure 12] FIG. 10 is a diagram for explaining an example of a process for updating a specific range of a self area. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, examples of embodiments of the present invention will be described with reference to the drawings. Similar components throughout the drawings will be designated by the same reference numerals, and redundant explanations may be omitted.

[0010] First Embodiment A first embodiment will be described with reference to Figs. 1 to 9. First, an overview of the information display device will be described with reference to Figs. 1 to 2. Figs. 1A to 1C are external views showing the information display device in a worn state. Figs. 2A to 2C are diagrams showing the imaging ranges of the external cameras provided on the information display device. Note that Xh, Yh, and Zh are coordinate axes of a coordinate system based on the information display device. Xh indicates the left-right direction of the information display device. Yh indicates the up-down direction of the information display device. Zh indicates the front-to-back direction of the information display device.

[0011] As shown in FIGS. 1A-1C, the information display device 100 (HMD) is goggle-shaped and is worn on the head of a wearer (user U1). A touch sensor 122L used by the user U1 for input operations is provided on the side of the information display device 100. In this example, a touch sensor (not shown) is also provided on the right side, so that touch sensors are provided on both the left and right sides. However, the touch sensor may be provided on only one side. Furthermore, when the user U1 performs input operations on the information display device 100 using a control device described later, the touch sensor may be omitted.

[0012] The information display device 100 is also provided with operation keys (not shown in FIGS. 1A-1C) for power, volume, etc. The operation keys can be provided in an appropriate position, for example, on the side of the information display device 100. The operation keys may be, for example, switches that can switch on / off operations such as power and audio output. Adjustment knobs for adjusting volume, etc. may also be provided.

[0013] Display units (131L, 131R) are provided inside the information display device 100. In this example, a left display unit 131L is provided in a position visible to the left eye of user U1, and a right display unit 131R is provided in a position visible to the right eye of user U1. The left and right display units (131L, 131R) are configured using non-transmissive displays. These displays display created VR space images, real space images captured by an external camera, and the like. Note that one display unit may divide its display area and display images for the left eye and right eye side by side. In this case, the other display unit may be omitted. In this case, the display unit may be disposed in the center in the left-right direction (Xh axis direction).

[0014] The information display device 100 has multiple external cameras installed on its front side. In this example, an external camera 133LU is installed at the upper left corner, an external camera 133LD is installed at the lower left corner, an external camera 133RU is installed at the upper right corner, and an external camera 133RD is installed at the lower right corner. These external cameras (133LU, 133LD, 133RU, 133RD) can capture images of the real space around the wearer (user U1) and capture images of real objects. The information display device 100 can also measure the distance to an object based on parallax information from these external cameras. Therefore, the external cameras can be used as distance measuring devices. The information display device 100 may also be provided with separate imaging devices and distance measuring devices, and imaging and distance measuring may be performed based on separate devices. For example, an external camera may be used as the imaging device, and a distance sensor may be used as the distance measuring device.

[0015] An audio output unit 141L is provided on a side of the information display device 100. In this example, only the left audio output unit 141L is shown, but an audio output unit (not shown) is also provided on the right side, so that audio output units are provided on both the left and right sides. Each audio output unit can be configured as a stereo speaker, and the wearer (user U1) can hear the audio output from each audio output unit. The information display device 100 also includes an audio input unit that inputs the voice of the wearer (user U1). This audio input unit can be configured using a monaural microphone.

[0016] The information display device 100 is also provided with an expansion interface unit. The expansion interface unit is an interface used for charging, wired communication, etc. Here, a terminal connection port for connecting a terminal may be formed on, for example, a side surface of the information display device 100.

[0017] Next, the shooting range of each external camera will be described. As shown in Fig. 2A, in the XhZh plane, the shooting angle of view of external camera 133RU is set to A1RU, and the shooting angle of view of external camera 133LU is set to A1LU. Although not shown in the figure, in the XhZh plane, the shooting angle of view of external camera 133RD is set to A1RD, and the shooting angle of view of external camera 133LD is set to A1LD.

[0018] 2B, in the XhYh plane, the angle of view of the external camera 133RU is set to A2RU, the angle of view of the external camera 133LU is set to A2LU, the angle of view of the external camera 133RD is set to A2RD, and the angle of view of the external camera 133LD is set to A2LD.

[0019] 2C, in the YhZh plane, the angle of view of the external camera 133LU is set to A3LU, and the angle of view of the external camera 133LD is set to A3LD. Although not shown in the figure, in the YhZh plane, the angle of view of the external camera 133RU is set to A3RU, and the angle of view of the external camera 133RD is set to A3RD.

[0020] Then, by performing stereo photography using external camera 133LU and external camera 133RU, it becomes possible to measure distance in the Yh-axis direction (up and down direction).Similarly, by performing stereo photography using external camera 133LD and external camera 133RD, it becomes possible to measure distance in the Yh-axis direction (up and down direction).

[0021] In addition, by performing stereo photography using any combination of the external cameras (133LU, 133LD, 133RU, 133RD), distance measurement in the Zh axis direction (forward) becomes possible.

[0022] Furthermore, by performing stereo photography using the external cameras 133LU and 133LD, it becomes possible to measure distances in the Xh-axis direction (left and right direction). By performing stereo photography using the external cameras 133RU and 133RD, it becomes possible to measure distances in the Xh-axis direction (left and right direction).

[0023] Next, an example of the hardware configuration of the information display device will be described with reference to Fig. 3. The information display device 100 includes a main control unit 101 (processor), a RAM 103, a storage unit 110, an operation input unit 120, a video processing unit 130, an audio processing unit 140, a position information acquisition unit 150, a sensor unit 160, a communication unit 170, and an expansion I / F unit 180 (expansion interface unit). These are connected via a bus 102, which is a transmission / reception path for exchanging various data, commands, etc. The information display device 100 also includes a battery (not shown) that serves as a power source.

[0024] The main control unit 101 functions as a main processor of the information display device 100. The main control unit 101 can be configured using, for example, a CPU (Central Processing Unit), but may also be configured using other semiconductor devices. The main control unit 101 may also be configured as an MCU (Micro Controller Unit).

[0025] The RAM 103 is a main storage device used by the main control unit 101 during data processing. The storage unit 110 stores data such as operation programs and can be configured using an appropriate auxiliary storage device. The storage unit 110 can be configured using a non-volatile storage medium such as a flash ROM, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an SSD (Solid State Drive), or an HDD (Hard Disk Drive). The RAM 103 and the storage unit 110 form a storage unit that stores data.

[0026] The operation input unit 120 is a user operation interface, and includes, for example, operation keys 121 and a touch sensor 122. The operation keys 121 are keys used for user operation, and can include, for example, a power key that serves as a power switch and volume keys related to volume operation. The touch sensor 122 is used to input operation details of the wearer. For example, appropriate information may be output to the display unit 131, and the wearer (user) may input corresponding to that information via the touch sensor 122. However, as described above, when input operations are performed using a control device described later, the touch sensor 122 may be omitted.

[0027] The image processing unit 130 is used for image processing and includes, for example, a display unit 131, an image signal processing unit 132, a first image input unit 133, and a second image input unit 134. The display unit 131 is used to present appropriate information and, in this embodiment, is configured using a non-transmissive display. The image signal processing unit 132 is an image (video) signal processor and has the function of processing image signals. The first image input unit 133 constitutes an imaging element that converts image information into an electrical signal. Data acquired by external cameras (133LU, 133LD, 133RU, 133RD) that capture images of the wearer (user)'s surroundings is input to the first image input unit 133. The second image input unit 134 constitutes an imaging element that converts image information into an electrical signal. Data acquired by an internal camera (not shown) that detects the wearer's (user's) line of sight is input to the second image input unit 134.

[0028] The audio processing unit 140 is used for audio processing, and includes, for example, an audio output unit 141, an audio signal processing unit 142, and an audio input unit 143. As described above, the audio output unit 141 outputs audio and is configured using a speaker. The audio signal processing unit 142 is an audio signal processor and has the function of processing audio signals. The audio input unit 143 is used for audio input and is configured using an appropriate microphone.

[0029] The position information acquisition unit 150 can be configured using, for example, a GPS receiver, and acquires the position information of the information display device 100. However, the information display device 100 may acquire the position information by other appropriate methods. For example, the main control unit 101 may extract feature points in a space by analyzing a peripheral image acquired by the first image input unit 133, and acquire relative position information based on these feature points, thereby allowing the information display device 100 to acquire the position information. In this way, when the information display device 100 acquires the position information without relying on the position information acquisition unit 150, the position information acquisition unit 150 may be omitted.

[0030] The sensor unit 160 can be configured using, for example, a gyro sensor 161, a geomagnetic sensor 162, and an acceleration sensor 163, and can be used to grasp data such as the state of the wearer (user) and the orientation of the information display device 100. However, the sensors listed here are only examples, and as long as they can execute the predetermined processing, the listed sensors may be omitted as appropriate, or other types of sensors may be included. The information display device 100 may be provided with, for example, a distance sensor that can detect the distance to an object, and this distance sensor may be used to grasp the position, size, etc. of surrounding objects.

[0031] The communication unit 170 is configured as an interface for performing appropriate communication, and includes, for example, a BT communication unit 171 and an NFC communication unit 172. The BT communication unit 171 is an interface used for short-range wireless communication using Bluetooth (registered trademark). The NFC communication unit 172 is an interface used for short-range wireless communication using NFC (Near Field Communication). Note that the communication unit 170 may also include an interface used for other types of communication. For example, the communication unit 170 may include an interface for performing mobile communication, and may use this interface to communicate with other information terminals (for example, appropriate mobile devices or wearable devices). Furthermore, the communication unit 170 may also include, for example, an interface for performing wireless LAN communication.

[0032] The expansion I / F unit 180 is used to expand the functions of the information display device 100. As described above, the expansion I / F unit 180 may include an interface used for charging and an interface used for wired communication. The expansion I / F unit 180 may also include an interface for connecting an external storage device (e.g., a USB flash drive). The information display device 100 may then input and output data to and from the external storage device via this interface.

[0033] Next, functions of the information display device 100 will be described with reference to FIG. 4. As shown in FIG. 4, the storage unit 110 stores a basic operation program 1001, which is a basic program such as an OS, and an application 1002 for executing predetermined functions. The storage unit 110 also includes a various data storage area 1009, which stores various operation setting values ​​and various information (such as video to be played, still images to be displayed, and audio data to be output when experiencing a VR space, etc.). The programs and applications stored in the storage unit 110 are loaded into the RAM 103 by the main control unit 101, and the main control unit 101 executes the programs and applications loaded into the RAM 103 to realize predetermined functions related to this embodiment. The applications 1002 stored in the storage unit 110 may include, for example, an application for playing appropriate video, an application for playing VR video, an application for displaying images, or other appropriate applications. The main control unit 101 may then load these applications into the RAM 103 to realize predetermined functions.

[0034] The main control unit 101 loads the programs stored in the storage unit 110 into the RAM 103, and realizes, as an example, the functions (1101, 1111 to 1117) described below. However, the description here is an example, and the main control unit 101 may realize basic operations and applications other than these.

[0035] The basic operation function unit 1101 controls the basic operations of the information display device 100 .

[0036] The position / orientation information detection unit 1111 acquires position information and orientation information (such as direction and tilt) of the information display device 100 in real space. Here, the main control unit 101 may acquire position information based on a position information signal acquired by the position information acquisition unit 150. The main control unit 101 may extract feature points in real space based on an image acquired by the first image input unit 133, estimate a relative position from the extracted feature points, and acquire position information. The main control unit 101 can acquire orientation information based on data acquired by the sensor unit 160. For example, the main control unit 101 may acquire the direction (orientation) of the information display device 100 based on data from a geomagnetic sensor 162. For example, the main control unit 101 may acquire the tilt of the information display device 100 based on data from a gyro sensor 161 and an acceleration sensor 163.

[0037] The control device information acquisition unit 1112 acquires information about the control device by transmitting and receiving data to and receiving from the control device described later via the communication unit 170. Here, the acquired information about the control device may include information input by the user to the control device, attitude information about the control device, etc.

[0038] The image acquisition / distance measurement unit 1113 acquires image information of the periphery of the information display device 100 via the first image input unit 133. The image acquisition / distance measurement unit 1113 also calculates distance information to the photographed object based on the acquired image information. When a distance measurement device such as a distance sensor is used, the image acquisition / distance measurement unit 1113 may calculate distance information to the object based on information acquired by the distance measurement device. The main control unit 101 may estimate the direction of the photographed object based on the position of the object in the image sensor and the angle of view of the external camera that captures the image.

[0039] The floor position detection unit 1114 detects the floor position of a space (room, etc.) in which a user wearing the information display device 100 is located. The floor position detection unit 1114 detects floor position information based on, for example, position information and posture information of the information display device 100 detected by the position / posture information detection unit 1111, posture information of the control device acquired by the control device information acquisition unit 1112, distance information acquired by the image information acquisition / distance measurement unit 1113, etc.

[0040] The self-region setting unit 1115 sets a self-region for the wearer (user) of the information display device 100 to safely experience VR. That is, the wearer of the information display device 100 experiences a VR space or the like in a real space. During the experience, the wearer may move in various directions depending on the content output by the information display device 100. However, various obstacles may exist in the real space in which the wearer is located. For this reason, when the wearer moves or performs an activity such as moving their hands during the experience, they need to avoid these objects. Therefore, the wearer sets a region in which they can safely move and act without coming into contact with these objects. Note that, for example, during a VR experience, a VR space image is displayed, and these objects cannot be recognized.

[0041] The self-region setting unit 1115 sets the self-region of the user wearing the information display device 100. The self-region setting unit 1115 sets the self-region based on, for example, the position information and posture information of the information display device 100 detected by the position / posture information detection unit 1111, the posture information of the control device acquired by the control device information acquisition unit 1112, the distance information acquired by the image information acquisition / distance measurement unit 1113, etc.

[0042] The obstacle detection unit 1116 analyzes image information of the periphery of the information display device 100 captured by an external camera and acquired via the first image input unit 133, detects objects / people, etc. around the information display device 100, and calculates distance information to the detected objects / people. Furthermore, the obstacle detection unit 1116 displays a warning when the detected object / people is within the self region set by the self region setting unit 1115. Note that the obstacle detection unit 1116 may use data output by the image acquisition / distance measurement unit 1113 for processing.

[0043] The display control unit 1117 controls the display of information on the display unit 131. For example, the display control unit 1117 controls the display of an image generated by an application. The display control unit 1117 also controls the video-through display of image information around the information display device 100 captured by the first image input unit 133 (i.e., the external cameras 133LU / 133RU / 133LD / 133RD). The display control unit 1117 also controls the display of the self-region set by the self-region setting unit 1115, a warning display generated by the obstacle detection unit 1116, and the like on the display unit 131. The display control unit 1117 may display the self-region display and the warning display in an appropriate manner. For example, the display control unit 1117 may display information related to the self-region and the warning by superimposing it on the currently displayed image.

[0044] The RAM 103 also has a temporary storage area 1199. The temporary storage area 1199 is a temporary storage area for various types of information created or acquired by an application, for example.

[0045] Next, an example of the configuration of the control device will be described with reference to Figs. 5A, 5B, and 6. The control device 200 can communicate with the information display device 100, and the wearer (user) can use the control device 200 as a controller. As will be described later, the control device 200 may also be used in setting a user's own area. The control device 200 can be held in the wearer's hand for use. One control device may be provided, and the wearer may use it by holding it in either the left or right hand. Alternatively, two control devices may be provided, and the wearer may use them by holding them in each of the left and right hands.

[0046] As shown in FIGS. 5A and 5B, the control device 200 has a shape that can be held by a user, and is formed with an end portion 200a that protrudes in the front-rear direction. The control device 200 also has a plurality of operation keys (221a to 221c), including a first operation key 221a, a second operation key 221b, and a third operation key 221c. The control device 200 also has an interface (not shown in FIGS. 5A and 5B) for connecting a charging terminal. In FIGS. 5A and 5B, Xc, Yc, and Zc are coordinate systems based on the control device 200, with Xc indicating the left-right direction of the control device 200, Yc indicating the up-down direction of the control device 200, and Zc indicating the front-rear direction of the control device 200.

[0047] Next, an example of the hardware configuration of the control device 200 will be described. As shown in Fig. 6, the control device 200 includes a main control unit 201, a RAM 203, a storage unit 210, an operation input unit 220, a sensor unit 260, a communication unit 270, and an expansion I / F unit 280 (expansion interface unit). These are connected via a bus 202, which is a transmission / reception path for exchanging various data, commands, etc. The control device 200 also includes a battery (not shown) that serves as a power source.

[0048] The main control unit 201 functions as the main processor of the control device 200. The main control unit 201 can be configured using, for example, a CPU, but may also be configured using other semiconductor devices. The main control unit 201 may also be configured as an MCU.

[0049] The RAM 203 is a main storage device used by the main control unit 201 during data processing. The storage unit 210 stores data such as operating programs and can be configured using an appropriate auxiliary storage device. The storage unit 210 can be configured using a non-volatile storage medium such as a flash ROM, an EEPROM, an SSD, or an HDD.

[0050] The operation input unit 220 is a user operation interface, and includes, for example, operation keys 221. The operation keys 221 are keys used for user operations, and can include, for example, the above-described first operation key 221a, second operation key 221b, and third operation key 221c.

[0051] The sensor unit 260 can be configured using, for example, a gyro sensor 261, a geomagnetic sensor 262, and an acceleration sensor 263, and can be used to grasp data such as the attitude and orientation of the control device 200. However, the sensors listed here are only examples, and other types of sensors may be included.

[0052] The communication unit 270 is configured as an interface for performing appropriate communication, and includes, for example, a BT communication unit 271 and an NFC communication unit 272. The BT communication unit 271 is an interface used for short-range wireless communication by Bluetooth (registered trademark). The NFC communication unit 272 is an interface used for short-range wireless communication by NFC (Near Field Communication).

[0053] The expansion I / F unit 280 is used to expand the functions of the control device 200. The expansion I / F unit 280 may include, for example, an interface used for charging and an interface used for wired communication.

[0054] Next, an example of the self-region initial setting process will be described with reference to Figures 7A to 7D. As shown in Figure 7A, when this process starts, the main control unit 101 of the information display device 100 first executes floor surface detection process S101. The floor surface detection process S101 relates to calibration.

[0055] First, the user places the control device 200 on the floor and tilts his / her head (or neck) to capture an image of the control device 200 with the external camera.

[0056] Then, the main control unit 101 of the information display device 100 executes the position / posture information detection unit 1111 to acquire information on the position and posture of the information display device 100 (S151). The main control unit 101 also executes the image acquisition / distance measurement unit 1113 to acquire image information of the surroundings captured by the external camera from the first image acquisition unit 133 (S152), and further calculates the distance to the control device 200 placed on the floor using the acquired image information (S153).

[0057] Furthermore, the main control unit 101 executes the control device information acquisition unit 1112 to acquire the attitude information of the control device 200, which the control device 200 acquires using the sensor unit 260 (S154).

[0058] The main control unit 101 executes the floor surface position detection unit 1114 and performs processing using the acquired position information and attitude information of the information display device 100, the calculated distance information to the control device 200, and the acquired attitude information of the control device 200. Based on this processing, the main control unit 101 calculates the floor surface position with the information display device 100 as the reference (S155). Then, the main control unit 101 stores the calculated floor surface information in the storage unit 110 (S156).

[0059] Here, the floor detection process S101 will be described in more detail with reference to Fig. 7B. In Fig. 7B, Xs, Ys, and Zs are coordinate axes of a coordinate system in real space S, where Xs indicates the latitude direction in real space, Ys indicates the direction of gravity in real space, and Zs indicates the longitude direction in real space.

[0060] The user U1 places the control device 200 on the floor of a room or the like where the floor surface is to be detected so that the Ys axis of the real space S coincides with the Yc axis of the control device. Then, the user U1 tilts his / her head (tilts his / her neck) and captures an image of the control device 200 with the external camera of the information display device 100.

[0061] (1) The main control unit 101 obtains the elevation angle of the orientation direction (Zh axis direction) of the information display device 100 with respect to the XsZs plane of the real space S from the orientation information of the information display device 100 obtained by the position / orientation information detection unit 1111. That is, the angle between the Zh direction of the information display device 100 and the XsZs plane of the real space S is obtained. Furthermore, (2) the main control unit 101 analyzes image information captured by the external camera and acquired by the image acquisition / ranging unit 1113. Through this analysis, the main control unit 101 obtains the elevation and depression angles of the position direction of the control device 200 with respect to the XhZh plane with the information display device 100 as the reference, from the imaging position of the control device 200 in the imaging element constituting the first image input unit 133 and the imaging angle of view of the first image input unit 133. That is, the angle between the XhZh plane of the information display device 100 and the position direction of the control device 200 with respect to the position of the information display device 100 as the reference is obtained. Then, (3) based on the results of each angle, the main control unit 101 calculates the angle between the straight line connecting the information display device 100 and the control device 200 and the vertical direction (Ys-axis direction) of real space S. Explaining (3) in detail, the main control unit 101 uses the elevation angle calculated in (1) above to calculate the angle of the position direction of the control device 200 with respect to the XsZs plane in real space from the angle of the position direction of the control device 200 with respect to the XhZh plane calculated in (2) above. The main control unit then calculates the angle between the straight line connecting the information display device and the control device and the vertical direction of real space S using this angle based on the XsZs plane in real space.

[0062] Furthermore, (4) the main control unit 101 calculates the position information of the control device 200 in the real space S. Here, the main control unit 101 calculates the position information of the control device 200 from the position information of the information display device 100 in the real space S acquired by the position / posture information detection unit 1111, the distance information from the information display device 100 to the control device 200 calculated by the image acquisition / distance measurement unit 1113, and the angle calculated in (3).

[0063] Furthermore, (5) the main control unit 101 corrects the position information of the control device 200 calculated in (4) above with information about the size of the control device 200. Then, the main control unit 101 calculates position information of the floor surface in the real space S from information about the distance from the information display device 100 to the control device 200 based on the corrected position information of the control device 200 and the angle calculated in (3). The main control unit 101 calculates, as the position information of the floor surface, information in the Ys-axis direction that mainly indicates height (i.e., information about the length from the information display device 100 to the floor surface). In the floor surface detection process, the main control unit 101 may calculate the height (information in the Ys-axis direction) using trigonometric ratios, with the distance from the information display device 100 to the control device 200 as the hypotenuse.

[0064] The main control unit 101 acquires the attitude information of the control device 200 in the real space and determines whether the coordinate system of the real space matches the coordinate system of the control device 200. If there is a difference between these coordinate systems (for example, if there is a difference between the Ys-axis direction and the Yc-axis direction), there is a possibility that accurate processing is not being performed, so the main control unit 101 outputs a warning and prompts the user to relocate the control device 200.

[0065] Alternatively, the floor surface detection process S101 may be performed by the information display device 100 alone, without using the control device 200. In this case, the user U1 sets the information display device 100 to floor surface detection mode (calibration mode) by operating an instruction. Furthermore, the information display device 100 operating in floor surface detection mode (calibration mode) can be removed from the head and placed on the floor in a predetermined posture. In this state, the main control unit 101 of the information display device 100 corrects the posture information of the information display device 100 (mainly the position on the Ys axis in the real space S) acquired from the position / posture information detection unit 1111 with information related to the size of the information display device 100, thereby enabling floor surface position information to be calculated.

[0066] Returning to the flowchart of FIG. 7A, an example of processing after the floor detection processing will be described. The user U1 holds the control device 200 in his / her hand and performs an operation to request the start of the self-region setting processing. Here, the main control unit 101 of the information display device 100 executes the self-region setting unit 1115 to check whether or not the user U1 has performed an operation to request the start of the self-region setting processing (S102). Here, if the start request operation has been performed, the processing proceeds to S103. On the other hand, if the start request operation has not been performed, the processing ends. Note that the start request operation may be performed using the touch sensor 122 of the information display device 100. On the other hand, the start request operation may be performed by the user U1 operating the operation input unit 220 of the control device 200.

[0067] The main control unit 101 executes the position / posture information detection unit 1111 to acquire position information and posture information of the information display device 100 in real space from the position information acquisition unit 150 and the sensor unit 160 (S103).

[0068] The main control unit 101 executes the floor position detection unit 1114 and confirms the floor position information based on the information regarding the floor position using the information display device 100 as the position reference read from the storage unit 110 and the position information of the information display device 100 acquired in the processing of S103 (S104).

[0069] The main control unit 101 executes the image acquisition / distance measurement unit 1113 to analyze the image information acquired by the first image input unit 133 and calculate the distance information from the information display device 100 to the control device 200 (S105).

[0070] The main control unit 101 executes the control device information acquisition unit 1112 to acquire the attitude information of the control device in real space via the communication unit 170 (S106).

[0071] The main control unit 101 executes the self region setting unit 1115 to calculate position information of the intersection C between the floor surface and an extension line EX extending from the control device 200 in the Zc-axis direction. The main control unit 101 may, for example, calculate the position information of the intersection C between the floor surface and the extension line EX extending in the Zc-axis direction, using the end 200a that is the tip of the control device 200 in the Z-axis direction as a reference. Here, the main control unit 101 calculates the position information of the intersection based on, for example, the acquired position information and posture information of the information display device 100, the confirmed position information of the floor surface, the calculated distance information from the information display device 100 to the control device 200, and the acquired posture information of the control device 200 (S107).

[0072] When the user U1 moves the control device 200 in the real space, the above-mentioned processes of S103 to S107 are repeatedly executed, and the main control unit 101 executes the self region setting unit 1115 to obtain a trajectory LC made up of a collection of the calculated intersection points C (S108).

[0073] Then, the main control unit 101 executes the self-region setting unit 1115 to check whether the trajectory LC has closed around the user U1 (S109). If the trajectory LC has closed around the user U1, the process proceeds to S110. If not, the process returns to S103, and the main control unit 101 repeats the processes of S103 to S108.

[0074] Next, the main control unit 101 executes the self-region setting unit 1115 and sets the self-region RG based on the trajectory LC (S110). Then, the main control unit 101 executes the display control unit 1117 and displays the self-region RG set in the processing of S110 on the display unit 131 (S111). Here, the user U1 can check the displayed self-region RG.

[0075] Then, the main control unit 101 executes the self-area setting unit 1115 and stores in the storage unit 110 information on the area shape, with the position information of the information display device 100 acquired in the processing of S103 as the position reference, for the self-area RG set in the processing of S110 (S112).

[0076] Here, the process of generating a trajectory will be described in more detail with reference to Fig. 7C. In Fig. 7C, Xs, Ys, and Zs are coordinate axes of a coordinate system in real space, similar to Fig. 7B.

[0077] The user U1 holds the control device 200 in his / her hand and points the end 200a of the control device 200 toward the floor. The user U1 then draws the trajectory he / she wants to generate on the surrounding floor. When the user U1 operates the control device 200 in this way, the main control unit 101 performs the processing described below.

[0078] (1) The main control unit 101 obtains the elevation angle of the orientation direction (Zh axis direction) of the information display device 100 with respect to the XsZs plane of the real space S from the orientation information of the information display device 100 obtained by the position / orientation information detection unit 1111. That is, the angle between the Zh direction of the information display device 100 and the XsZs plane of the real space S is obtained. Furthermore, (2) the main control unit 101 analyzes image information captured by the external camera and acquired by the image acquisition / ranging unit 1113. Through this analysis, the main control unit 101 obtains the elevation and depression angles of the position direction of the control device 200 with respect to the XhZh plane with the information display device 100 as the reference, from the imaging position of the control device 200 in the imaging element constituting the first image input unit 133 and the imaging angle of view of the first image input unit 133. That is, the angle between the XhZh plane of the information display device 100 and the position direction of the control device 200 with respect to the position of the information display device 100 as the reference is obtained. Then, (3) the main control unit 101 calculates the angle between the straight line connecting the information display device 100 and the control device 200 and the vertical direction (Ys-axis direction) of real space S based on the results of each angle. Explaining (3) in detail, the main control unit 101 uses the elevation angle calculated in (1) above to calculate the angle of the position direction of the control device 200 with respect to the XsZs plane in real space from the angle of the position direction of the control device 200 with respect to the XhZh plane calculated in (2) above. Then, the main control unit 101 calculates the angle between the straight line connecting the information display device 100 and the control device 200 and the vertical direction of real space S using this angle based on the XsZs plane in real space.

[0079] Furthermore, (4) the main control unit 101 calculates the position information of the control device 200 in the real space S. The main control unit 101 calculates the position information of the control device 200 from the position information of the information display device 100 in the real space S acquired by the position / posture information detection unit 1111, the distance information from the information display device 100 to the control device 200 calculated by the image acquisition / distance measurement unit 1113, and the angle calculated in (3).

[0080] Furthermore, (5) the main control unit 101 calculates trajectory information of an extension line EX extending from the tip of the control device 200 in the Zc-axis direction, based on the position information of the control device 200 calculated in (4) above and the attitude information of the control device 200 acquired by the control device information acquisition unit 1112. Also, (6) the main control unit 101 calculates position information of the floor surface in the real space S, using the position information of the information display device 100 in the real space S acquired by the position / attitude information detection unit 1111. The main control unit 101 can calculate information in the Ys-axis direction that mainly indicates height (i.e., information about the length from the information display device 100 to the floor surface) as the position information of the floor surface. Note that the main control unit 101 may also use information about the position of the floor surface relative to the information display device 100, which is stored in the storage unit 110.

[0081] Then, (7) the main control unit 101 calculates the position information of the intersection C between the extension line EX extending from the tip of the control device 200 in the Zc-axis direction and the floor surface from the trajectory information of the extension line EX and the position information of the floor surface obtained in (6) (mainly height information in the Ys-axis direction).

[0082] The user U1 operates the control device 200 and moves the tip of the control device 200 toward the surrounding floor surface. Then, the information display device 100 generates trajectory information of an extension line EX around the user U1, and generates a trajectory LC consisting of a collection of intersections C.

[0083] Next, an example of a self-region based on the generated trajectory will be described with reference to Fig. 7D. As shown in Fig. 7D, the main control unit 101 generates a self-region RG in a three-dimensional space, which is a shape obtained by extending the trajectory surrounding the user U1 in the vertical direction (Ys-axis direction) of the real space S, for example. Here, the height of the self-region RG can be, for example, higher than the height of the tip of the user U1's hand when the user U1 raises his or her arm. Alternatively, the self-region RG may be a shape extending up to the ceiling in the vertical direction (Ys-axis direction) of the real space S.

[0084] In addition to the method of generating a self region based on a trajectory LC constructed on the floor surface using the control device 200 described in the procedure above, the self region may also be generated by the following method. For example, if a request to start the self region setting process is made in the process of S102, a circle of a predetermined radius centered on the position of the user U1 (the position of the information display device 100) is temporarily set on the floor surface in place of the trajectory LC, and the control device 200 is used to deform the circle on the floor surface as necessary. After this, the deformed circle can be extended in the vertical direction (Ys-axis direction) of the real space S to generate a self region in three-dimensional space. Note that the shape temporarily set on the floor surface in place of the trajectory LC is not limited to a circle, and may be a rectangle, pentagon, hexagon, etc.

[0085] Next, an example of the warning display process will be described with reference to Fig. 8. As shown in Fig. 8, the main control unit 101 of the information display device 100 executes the position / posture information detection unit 1111 to acquire position information and posture information of the information display device 100 in the real space S from the position information acquisition unit 150 and the sensor unit 160 (S201).

[0086] The main control unit 101 executes the floor position detection unit 1114 to confirm the floor position information based on the information regarding the floor position using the information display device 100 as the position reference read from the storage unit 110 and the position information of the information display device 100 acquired in S201 (S202).

[0087] The main control unit 101 executes the self-region setting unit 1115 and analyzes the image information acquired by the first image input unit 133. The main control unit 101 checks, through this analysis, whether the environment allows the self-region RG to be set. Then, the main control unit 101 sets the self-region based on the acquired position information of the information display device 100 and the position information of the floor surface confirmed in S202 (S203).

[0088] After setting the self region in S203, the main control unit 101 executes the image acquisition / distance measurement unit 1113 to acquire an image of the periphery of the information display device 100 from the first image input unit 133 (S204).

[0089] The main control unit 101 analyzes the image of the area around the information display device 100 acquired in the process of S204, and checks whether any objects or people other than the user are captured in the image information (S205). The main control unit 101 may check not only whether any objects / people are present in front, behind, to the left, or right of the user U1, but also below the user U1. The main control unit 101 may also check whether the height of the floor surface is significantly different from the floor surface position information confirmed in the process of S202, and may check, for example, whether there are any stairs in the image information.

[0090] In S206, if the main control unit 101 determines that an object / person exists in the vicinity, the process proceeds to S207. On the other hand, if the main control unit 101 determines that no object / person exists in the vicinity, the process returns to S201.

[0091] The main control unit 101 executes the image acquisition / distance measurement unit 1113 to analyze the image of the surroundings of the information display device 100 acquired in the process of S204, and calculates distance information to objects or people other than the user (S207).

[0092] The main control unit 101 determines whether an object or person other than the user is present within the self region (S208). Here, the main control unit 101 makes this determination based on information about the self region set in the process of S203 and information about the distance to the object or person other than the user calculated in the process of S207. If an object or person other than the user is present within the self region, the process proceeds to S209. On the other hand, if an object or person other than the user is not present within the self region, the process returns to S201. Then, the main control unit 101 repeats the processes of S201 to S207.

[0093] The main control unit 101 executes the display control unit 1117 to display the warning display generated by the obstacle detection unit 1116 on the display unit 131 so that it overlaps with the direction in which the object or person was confirmed (S209). The warning display may be superimposed on the image of the running application. The warning display may be displayed in an appropriate manner using a text message, a symbol, an illustration, or the like. The warning display may also be displayed by visualizing the self-region (or the boundary line of the self-region). The main control unit 101 may also display the warning by superimposing the acquired image of the object or person.

[0094] Next, the state of the user's own area when the user moves will be described with reference to Figures 9A to 9C. Figure 9A shows an example of the user's own area before the user moves. In this example, user U1 is located at position P0, and the user's own area RG0 is set.

[0095] FIG. 9B shows a self-region RGn1 in a coordinate system based on the information display device 100 when the user moves MV1 from position P0 to position Pn. When the user U1 moves from position P0 to position Pn, the self-region RGn1 is set using the position Pn after the movement as a position reference. The shape of this self-region RGn1 is maintained in the coordinate system (XhZh) based on the information display device 100 before and after the movement of the user U1. FIG. 9C shows a self-region RGn2 in a coordinate system in the real space S when the user U1 moves MV1 from position P0 to Pn. As in the case of FIG. 9B, when the user U1 moves from position P0 to position Pn, the self-region RGn2 is set using the position Pn after the movement as a position reference. The shape of this self-region RGn2 is maintained in the coordinate system in the real space S before and after the movement of the user U1.

[0096] Next, with reference to Figs. 9D-9F, an example of processing when a warning is displayed will be described. Fig. 9D shows the situation before a warning is displayed. In this situation, the user U1 is located at position P0, and a self region RG0 is set. In this case, the obstacle region RO is not within the self region RG0, so no warning is displayed. Then, as shown in Fig. 9E, when the user U1 moves MV1 from position P0 to position Pn, and at least a portion of the obstacle region RO enters the self region RGn1, which is the self region at position Pn, the main control unit 101 displays a warning (exclamation mark) on the display unit 131. The obstacle region RO is a region that the main control unit 101 sets in advance near the obstacle OBJ.

[0097] 9F, when the user U1 moves to a position Pn and the obstacle region RO enters the own region, the main control unit 101 updates the own region to RGn3 so that the shape does not overlap with the obstacle region RO. Then, the main control unit 101 displays the updated shape of the own region RGn3 on the display unit 131 to warn the user U1.

[0098] According to the first embodiment, an information display device 100 is provided that allows the wearer to more appropriately perceive the possibility of contact with an object in the real world while taking into consideration the sense of immersion of the wearer. Furthermore, even when the situation changes as the wearer moves, the information display device 100 can similarly respond.

[0099] Second Embodiment Next, a second embodiment will be described with reference to Fig. 10A. In the second embodiment, the main control unit 101 performs processing to set a self-region according to the moving speed. Note that the same description as that already described may be omitted.

[0100] 10A, when user U1 moves MV2 from position P0 to position Pm2 via position Pm1, a self-region is set at position Pm1 during the movement according to the moving speed. Here, as an example, the self-region RGm1 can be a region having a shape similar to the initial shape (i.e., the self-region at position P0) and expanded around the periphery according to the moving speed. That is, the main control unit 101 may set a self-region in which the front, back, left, and right of the self-region are changed by an equal expansion rate.

[0101] On the other hand, as an example, the self region may not be similar to the initial shape (i.e., the self region at position P0), but may be a region having a shape expanded in accordance with the moving speed. That is, the main control unit 101 may set the self region by changing the front, rear, left, and right of the self region at an appropriate expansion rate. For example, when the user U1 moves forward, the main control unit 101 may set the self region based on an expansion rate of 2 times forward, 1.5 times left and right, and 1 time backward, taking into consideration the viewpoint of paying closer attention to obstacles present ahead.

[0102] The main control unit 101 can calculate the moving speed as appropriate. For example, the main control unit 101 may estimate the moving speed based on a change over time in the position information of the information display device 100. As described above, the main control unit 101 can acquire the position information by using, for example, the position information acquisition unit 150 or feature points included in the image information.

[0103] According to the second embodiment, it is possible to adjust the shape of the self region to suit the movement situation of the user U1 based on the movement speed of the user U1.

[0104] Third Embodiment Next, a third embodiment will be described with reference to Fig. 10B. In the third embodiment, the main control unit 101 performs processing to set a self-region according to the user's posture. Note that descriptions similar to those already described may be omitted.

[0105] 10B, when the user U1 extends the hand (arm) holding the control device 200 forward at position P0, a self-region RGh is set that is enlarged in the direction (forward) of the outstretched hand (arm). Here, as an example, the self-region RGh can be a region having a shape that is enlarged only in the direction of the outstretched hand (arm) compared to its initial shape. That is, the main control unit 101 may set the self-region RGh that is changed at an appropriate enlargement rate in the direction in which the hand (arm) is outstretched.

[0106] 10B has been described with reference to a case where the hand (arm) is stretched forward, but if the user U1 stretches the hand (arm) holding the control device 200 to the side, for example, the main control unit 101 may set a self-region with a shape expanded to the side. Also, if the user U1 stretches the hand (arm) holding the control device 200 diagonally forward, for example, the main control unit 101 may set a self-region expanded in the diagonal forward direction.

[0107] The main control unit 101 can acquire appropriate information and perform processing. The main control unit 101 may acquire position information of the control device 200 based on image information captured by an external camera, for example, and set the self-region based on the acquired information. Similarly, the main control unit 101 may acquire position information of the hand or arm of the user U1 based on the image information, and set the self-region. A similar self-region enlargement process is possible when the user U1 extends the hand (arm) that is not holding the control device 200 in any direction, or when the user U1 stretches out his or her leg, for example.

[0108] According to the third embodiment, it is possible to adjust the shape of the self-region to suit the posture of the user U1.

[0109] <Fourth embodiment> Next, a fourth embodiment will be described with reference to Figures 11A and 11B. In the fourth embodiment, the main control unit 101 performs processing to update and expand the self-region. Note that descriptions similar to those already described may be omitted.

[0110] 11A, when user U1 moves MV3, a self-region RG0 is set at time T=0, a self-region RG1 is set at time T=1, a self-region RG2 is set at time T=2, a self-region RG3 is set at time T=3, and a self-region RG4 is set at time T=4 (present). Note that in FIG. 11A, the self-regions (RG0, RG1, RG2, RG3) at times T=0 to 3 are indicated by dashed lines, and the self-region RG4 at time T=4 is indicated by a solid line.

[0111] The main control unit 101 may then perform a process of updating the self region to a new self region RGXa by adding the self region set in accordance with the movement MV3 of the user U1. That is, the main control unit 101 may set a self region (RG0) at time T=0, update the self region to a self region (RG1) by adding the shape of the self region (RG0) at time T=1, and update the self region to a self region (RG2) by adding the shapes of the self regions (RG0, RG1) at time T=2. The main control unit may also update the self region to a self region (RG3) by adding the shapes of the self regions (RG0, RG1, RG2) at time T=3. The main control unit may then update the self region to a self region (RGXa) by adding the shapes of the self regions (RG0, RG1, RG2, RG3) to the self region (RG4) at time T=4. Note that the self region (RGXa) is indicated by a dotted line in FIG. 11A.

[0112] In this way, the main control unit 101 updates the self area to one in which the self area at the past time and the self area at the current time overlap. However, when updating the self area as described using Fig. 11A, the size of the self area may become excessive. Therefore, the main control unit 101 may update the self area as described below using Fig. 11B.

[0113] As shown in Fig. 11B, when user U1 moves MV3 in the same manner as in Fig. 11A above, a self-region RG0 is set at time T = 0, a self-region RG1 is set at time T = 1, a self-region RG2 is set at time T = 2, a self-region RG3 is set at time T = 3, and a self-region RG4 is set at time T = 4 (present). Note that in Fig. 11B, the self-regions (RG0, RG1, RG2, RG3) from time T = 0 to T = 3 are indicated by dashed lines, and the self-region RG4 at time T = 4 is indicated by a solid line.

[0114] The main control unit 101 may then perform a process of updating the self area RGXb to a new self area RGXb that includes the self area set in accordance with the movement MV3 of the user U1. However, in this process, the main control unit 101 deletes information about the self area that is older than a predetermined time and updates it. In other words, the main control unit 101 updates the self area to one that overlaps the self area at the current time with the self area at a past time up to a predetermined time before the present.

[0115] That is, the main control unit 101 may set a self region (RG0) at time T=0, update the self region to a self region (RG1) by adding the shape of the self region (RG0) to the self region (RG1) at time T=1, and update the self region to a self region (RG2) by adding the shape of the self region (RG0, RG1) to the self region (RG2) at time T=2. Then, the main control unit 101 may update the self region to a self region (RG3) by adding the shape of the self region (RG1, RG2) to the self region (RG3) at time T=3. Furthermore, the main control unit 101 may update the self region to a self region (RGXb) by adding the shape of the self region (RG2, RG3) to the self region (RG4) at time T=4. Note that in FIG. 11B, the self region (RGXb) is indicated by a dotted line.

[0116] According to the fourth embodiment, the main control unit 101 can update the own area to an expanded one at a predetermined time (T=1 to 4), store the updated own area in the storage unit, and perform warning processing using the updated own area. When updating the own area, the main control unit 101 may store the own area set at a predetermined time in the storage unit and perform processing. Furthermore, the main control unit 101 may delete information about unnecessary own areas that have been in use for a predetermined time or more from the storage unit.

[0117] Fifth Embodiment Next, a fifth embodiment will be described with reference to Fig. 12. In the fifth embodiment, the main control unit 101 performs a process of updating the own area only within a specific range. Note that the same description as that already described may be omitted.

[0118] 12, when user U1 is at a certain position and facing a certain direction, the range recognizable by the external camera of information display device 100 is limited to the area ahead, including the hatched area surrounded by a solid line in the figure. Therefore, when updating the set self region, main control unit 101 may update only the area recognizable by the external camera. In other words, main control unit 101 may update only the area RGs, which is the shooting range of the external camera, within the set self region.

[0119] Furthermore, as in the fourth embodiment, when user U1 moves and the main control unit 101 updates the self-area set at a predetermined time, the main control unit 101 may perform a process of enlarging and updating the self-area with respect to the range of the area RGs recognizable by the external camera.

[0120] According to the fifth embodiment, the self region can be updated only with respect to the range that can be recognized by the external camera, and the photographing range of the external camera and the range of the self region can be made to correspond to each other.

[0121] According to the above embodiment, an information display device (e.g., HMD) that can properly grasp the surrounding situation is provided, which can contribute to, for example, the ninth Sustainable Development Goal (SDG) advocated by the United Nations - building infrastructure for industrial and technological innovation.

[0122] Although the embodiments of the present invention have been described above, it goes without saying that the configurations for realizing the technology of the present invention are not limited to the above-described embodiments, and various modifications are possible. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. All of these fall within the scope of the present invention. Furthermore, numerical values, messages, etc. appearing in the text and figures are merely examples, and the effects of the present invention will not be impaired even if different ones are used.

[0123] It is sufficient if the predetermined processing can be executed, and for example, the programs used in each processing example may be independent programs, or multiple programs may constitute a single application program. Also, the order in which each processing is performed may be changed.

[0124] Some or all of the functions of the above-described embodiments may be implemented in hardware, for example, by designing them as integrated circuits. Alternatively, they may be implemented in software by a general-purpose processor, a special-purpose processor, a microprocessor unit, a CPU, or the like interpreting and executing an operating program that implements each function. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. Furthermore, the scope of software implementation is not limited, and hardware and software may be used together. Alternatively, some or all of the functions may be implemented by a server. Note that the server may be any type of server, as long as it can execute functions in cooperation with other components via communications. For example, the server may be a local server, a cloud server, an edge server, an online service, or the like. Information such as programs, tables, and files that implement each function may be stored in a memory, a recording device such as a hard disk or solid-state drive (SSD), or a recording medium such as an IC card, SD card, or DVD, or may be stored in a device on a communications network.

[0125] Furthermore, the control lines and information lines shown in the diagram are those considered necessary for explanation, and do not necessarily represent all of the control lines and information lines on the product. In reality, it can be assumed that almost all components are interconnected.

[0126] In S208 to S209, an example has been described in which a warning is displayed when an object or the like is present in the self-region, but the main control unit 101 may also issue a warning when an obstacle region enters the self-region, as described above with reference to Fig. 9F. The main control unit 101 may also issue a warning when the boundary of the self-region approaches an obstacle region.

[0127] The information display device 100 may process a gesture of the user U1 captured by an external camera as an operation input. Here, for example, the storage unit 110 may store data associating the operation content with the gesture, and the main control unit 101 may refer to the data and perform processing according to the operation content.

[0128] The information display device 100 can be suitably used, for example, in experiencing a VR space. However, the information display device 100 can also be used to experience a space related to MR (Mixed Reality) based on a virtual space.

[0129] In the second embodiment, an example has been described in which the user's own area is enlarged according to the moving speed, but the main control unit 101 may, for example, perform a process of switching the display according to the moving speed. For example, when the main control unit 101 determines that the user is moving at a predetermined speed or faster, the main control unit 101 may switch the image currently displayed on the display unit 131 to an image captured by an external camera. [Explanation of symbols]

[0130] 100 Information display device 101 Main control unit 103 RAM 110 Storage Department 131 Display section 133 First image input unit 150 Location information acquisition section

Claims

1. An information display device, a processor; The processor: a user area set by the user of the information display device is moved in accordance with the user's movement; determining the presence of an obstacle to be warned about based on the self region; An information display device characterized by:

2. 2. The information display device according to claim 1, The processor: When the user moves, the shape of the self-region is changed in accordance with the moving speed of the user. An information display device characterized by:

3. 2. The information display device according to claim 1, The processor: changing the shape of the self-region in accordance with the posture of the user; An information display device characterized by:

4. 2. The information display device according to claim 1, The processor: When the user moves, the self area is updated to one in which the self area at the past time and the self area at the current time are overlapped. An information display device characterized by:

5. 2. The information display device according to claim 1, The processor: When the user moves, the self area is updated to one in which the self area at the past time from the present to a predetermined time ago and the self area at the current time are overlapped. An information display device characterized by:

6. The information display device according to claim 1, The processor: When an obstacle enters the self-region, it is determined that an obstacle to be warned about exists. An information display device characterized by:

7. 2. The information display device according to claim 1, The processor: An obstacle area is set in advance around the obstacle; When at least a part of the obstacle region is within the self region, it is determined that an obstacle to be warned about is present. An information display device characterized by:

8. The information display device according to claim 1, Equipped with an external camera that captures real space, The processor: updating the self-region in a range overlapping with the photographing range of the external camera; An information display device characterized by:

9. The information display device according to claim 1, a location information acquisition unit configured using a GPS receiver; The processor: estimating a moving speed of the user based on a time change in the location information acquired by the location information acquisition unit; changing the shape of the self-region based on the estimated moving speed; An information display device characterized by:

10. The information display device according to claim 1, Equipped with an external camera that captures real space, The processor: extracting feature points from image information captured by the external camera and acquiring position information relative to the extracted feature points; estimating a moving speed of the user based on a change over time in the acquired position information; changing the shape of the self-region based on the estimated moving speed; An information display device characterized by:

11. The information display device according to claim 1, Equipped with an external camera that captures real space, The processor: acquiring position information of the user's hand or arm based on image information acquired by the external camera; changing the shape of the self-region based on the acquired position information of the user's hand or arm; An information display device characterized by:

12. The information display device according to claim 1, A storage unit is provided, The processor: storing a self-region at a predetermined time during the user's movement in the storage unit; When the user moves, the self area at the past time stored in the storage unit and the self area at the current time are updated to an overlapping self area. An information display device characterized by:

13. The information display device according to claim 1, A storage unit is provided, The processor: storing a self-region at a predetermined time during the user's movement in the storage unit; When the user moves, the self area stored in the storage unit is updated to an overlapping self area at the current time and the self area at the past time up to a predetermined time before the present time. An information display device characterized by:

14. The information display device according to claim 13, The processor: deleting from the storage unit the own area stored in the storage unit that is older than the predetermined time; An information display device characterized by:

15. The information display device according to claim 1, A distance measuring device; A display unit; Equipped with The processor: acquiring distance information to an obstacle based on an output from the distance measuring device; Further using the distance information to determine the presence of an obstacle that should be warned about; When it is determined that there is an obstacle that requires a warning, a message to that effect is output to the display unit. An information display device characterized by:

Images