Information display device and information processing system

Abstract

This information display device is mounted on the head of a user. The information display device comprises a processor, and an antenna that receives multipath-propagating radio waves. The processor analyzes multipath propagation information for the radio waves received by the antenna. When the presence of a moving body has been detected from the analysis result, the processor alerts the user.

Description

Information display device, information processing system

[0001] The present invention relates to an information display device and an information processing system.

[0002] A technology for experiencing virtual reality (VR) by generating a virtual space and displaying it on a display is known. Also known is an augmented reality (AR) technology that superimposes virtual objects created using computer graphics (CG) or the like on an image of real space. Another known technology is mixed reality (MR) that combines real space and virtual reality to generate a space in which the two interact with each other in real time. These technologies are widely used, and wearable devices worn by users have become widespread. One example of such a technology is a head-mounted display (HMD) worn by a user on the head. HMDs can be used, for example, for gaming experiences, simulations, and the like.

[0003] When a user is immersed in an HMD or the like, it becomes difficult for the user to grasp the situation around them. As a result, for example, the user may not be aware of the presence of other people. Patent Document 1 is known as a technology for providing information to a user using an HMD. Patent Document 1 discloses a technology for notifying the user of the presence of an object, such as a person, when the user is immersed in a VR space.

[0004] Specifically, Patent Document 1 states, for example, "The head-mounted display is a head-mounted display for use in virtual space. The head-mounted display includes a display, a camera, a distance detection unit, an image generation unit, a memory unit, and a control unit. The display displays an image. The camera captures images of real space. The distance detection unit detects the distance to an object that exists in real space. The image generation unit generates an image to be displayed on the display. The memory unit stores type conditions and distance conditions for the object to be displayed. The control unit then recognizes the type of object from the image captured by the camera, extracts objects that match the type conditions and distance conditions, and superimposes an image showing the extracted object on an image of the virtual space and displays it on the display."

[0005] International Patent Publication No. 2023 / 105653

[0006] The technology of Patent Document 1 can detect objects within the detection range of a camera or the like, but cannot detect objects outside this detection range. Therefore, for example, when a user is immersed in an experience, the user may not notice an intrusion of a person outside the detection range. Therefore, there is a challenge in better achieving both a sense of immersion and safety for the user.

[0007] According to a first aspect of the present invention, there is provided the following information display device. This information display device is an information display device worn on a user's head. The information display device includes a processor and an antenna that receives radio waves propagating in multiple waves. The processor analyzes multiple wave propagation information of the radio waves received by the antenna. Then, when the presence of a moving object is detected from the analysis result of the multiple wave propagation information, the processor issues a warning to the user.

[0008] According to a second aspect of the present invention, there is provided an information processing system as follows. The information processing system includes a computer device and an information display device. The computer device analyzes received information of radio waves propagating in multiple waves, and when the presence of a moving object is detected through the analysis of the received information, transmits detection information to the information display device. The information display device is capable of communicating with the computer device and is worn on the user's head. When the information display device receives the detection information from the computer device, it issues a warning to the user.

[0009] According to the present invention, a technology is provided that can better achieve both a sense of immersion and safety for the user. Note that problems, configurations, and effects other than those described above will become apparent from the following description of the preferred embodiment of the invention.

[0010] 10 is a diagram showing an example of a moving object detection range in relation to a detection method using a sensor mounted on an HMD. FIG. 11 is a diagram showing an example of a moving object detection range in relation to an embodiment. FIG. 12 is a diagram for explaining the principle of Wi-Fi sensing. FIG. 13 is a diagram showing an example of the configuration of an HMD. FIG. 14 is a diagram showing an example of the configuration of an HMD. FIG. 15 is a flowchart for explaining an example of a process related to an attention call in the first embodiment. FIG. 16 is a diagram for explaining an example of setting an urgency level. FIG. 17 is a flowchart for explaining an example of a process related to an attention call in the second embodiment. FIG. 18 is a diagram showing an example of classification of detection targets. FIG. 19 is a flowchart for explaining an example of a process related to an attention call in the third embodiment. FIG. 19 is a diagram showing an example of classification of detection areas. FIG. 19 is a flowchart for explaining an example of a process related to an attention call in the fourth embodiment. FIG. 19 is a diagram for explaining a determination example of S101. FIG. 19 is a diagram for explaining a determination example of S102 and S202. FIG. 19 is a diagram for explaining a determination example of S103 and S203. FIG. 19 is a diagram for explaining an example of an attention call method in relation to S105. FIG. 19 is a diagram for explaining an example of transmission information in relation to S105. FIG. 19 is a diagram for explaining a determination example of S107. FIG. 19 is a diagram showing an example of a situation during VR viewing. FIG. 10 is a diagram illustrating an example of a warning message displayed.

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment is an example for explaining the present invention, and for clarity of explanation, appropriate omissions and simplifications have been made. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0012] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0013] Although various types of information may be described using expressions such as "table," "list," and "queue" as examples, the various types of information may also be expressed using data structures other than these. For example, various types of information such as "XX table," "XX list," and "XX queue" may also be expressed as "XX information." When describing identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable.

[0014] When there are multiple components with the same or similar functions, they may be described using the same reference numeral with different subscripts. When there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0015] In the embodiments, processing performed by executing a program may be described. Here, a computer executes the program using a processor (e.g., a CPU or a GPU) and performs processing defined by the program using storage resources (e.g., a memory) and interface devices (e.g., a communication port). Therefore, the entity that executes the program and performs the processing may be the processor. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. Similarly, the entity that executes the program and performs the processing may be a controller, device, system, computer, or node that has a processor. The entity that executes the program and performs the processing may be any processing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit is, for example, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a complex programmable logic device (CPLD), or the like.

[0016] A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in an embodiment, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0017] First, an example of a moving object detection method using a sensor mounted on an HMD will be described. As shown in FIG. 1 , when a camera, a distance measurement sensor, or the like mounted on an HMD is used, a moving object detection range 50p, which is a range within which a moving object can be detected, corresponds to the detection range of the sensor. For example, when a user 51 uses an HMD in a room in a house, the entire room in which the user 51 is present becomes the moving object detection range 50p. Alternatively, if the detection range of the sensor does not cover the entire room, the moving object detection range becomes a partial area of ​​the room centered on the user, such as only in front of the user. Therefore, this method cannot detect a person 52 (e.g., a family member) in another room or a person 53 (e.g., a suspicious person) entering the house. As a result, the user 51 cannot notice the presence of a person outside the detection range of the sensor.

[0018] Therefore, for example, when the user 51 is immersed in the HMD and the user 51's vision and hearing are restricted, the following case may occur. That is, for example, if a suspicious person 53 breaks into a house, the user 51 will not be able to notice the theft in a room where the user 51 is not present. Also, if the suspicious person 53 harms the user 51, the user 51 will not be able to notice the presence of the suspicious person 53 until the suspicious person 53 enters the moving object detection range of a sensor mounted on the HMD. In another example, the user 51 will not be able to notice the presence of a family member 52 when the family member 52 returns home and enters the house, or when the family member 52 is in another room, or even when the family member 52 is outside the detection range of a sensor mounted on an HMD in the same room.

[0019] In the following embodiments, a technology for better achieving both immersion and safety for the user 51 will be described. In the following embodiments, an HMD will be described as an example of an information display device. Furthermore, the following description will be given using Wi-Fi radio waves as radio waves transmitted and received between an HMD and an external device or server and propagating in multiple waves, Wi-Fi Channel State Information (CSI) as radio wave multiplicity propagation information, and Wi-Fi sensing as wireless communication sensing using multiplicity propagation information. However, the present invention is not limited to these examples. Other radio waves may be used as long as they are transmitted and received between devices and propagating in multiple waves. For example, short-range wireless communication such as Bluetooth may be used, or base station communication such as a mobile phone network may be used. Note that multiplicity propagation refers to the reception of radio waves from a transmitter to a receiver via multiple different paths due to the effects of reflection, diffraction, and the like in wireless communication. The multiplexed received information includes information affected by each of the propagation paths. For example, when a person moves in a space where radio waves are transmitted and received, the received radio waves are affected.

[0020] First Embodiment FIG. 2 shows an example of a motion detection range in an embodiment. In this embodiment, a user 51 uses an HMD in a room in a house. Wi-Fi sensing is performed, and, as an example, the entire house becomes the motion detection range 50. Wi-Fi sensing, which is being standardized by IEEE P802.11bf, uses CSI transmitted and received over multiple paths between devices communicating using Wi-Fi, thereby expanding the motion detection coverage to, for example, the entire house. Wi-Fi sensing can perform motion detection using commonly available communication radio waves without using special devices such as sensors, cameras, or wearable devices. Wi-Fi is a registered trademark and a wireless communication standard. Wi-Fi uses radio waves in the 2.4 GHz, 5 GHz, and 6 GHz frequency bands.

[0021] Wi-Fi sensing will now be described in more detail with reference to FIG. 3. (a) shows a case where a mobile object is not present within the network, while (b) shows a case where a mobile object is present within the network. In the example of (a), a Wi-Fi router 61, a smartphone 62, a first computer 63, and a second computer 64 are placed in a space, and these devices communicate via Wi-Fi via a network (e.g., a local area network). At this time, Wi-Fi radio waves undergo multipath propagation, in which radio waves arrive via multiple paths. For example, in addition to direct waves that arrive between the transmitter and receiver, radio waves that undergo multipath propagation also include delayed waves that arrive while being affected by buildings, structures, mobile objects, etc. The influences referred to here refer to attenuation of radio wave intensity and phase shifts of radio wave waveforms due to reflection and diffraction of radio waves caused by interference with reflected waves from buildings, structures, and mobile objects. Here, as shown in (b), when a mobile object 65 is present in the space where these devices are placed, the Wi-Fi radio waves are subject to the aforementioned influences, resulting in displacement. As a result, waveform distortion occurs in the waveforms being communicated between the devices. This waveform distortion contains information about the mobile unit 65.

[0022] Wi-Fi sensing can identify a moving object by analyzing information on radio waves propagating in multiple waves (in other words, CSI information). Here, by performing known analyses in Wi-Fi sensing, it is possible to identify the location of the moving object, identify the operating mode of the moving object, identify the type of moving object, and so on. Furthermore, for example, if the moving object is a person, it is also possible to perform individual recognition (i.e., to identify the individual), as well as state recognition such as breathing recognition and fall recognition. Note that Wi-Fi sensing may perform analysis using a machine learning model (e.g., a convolutional neural network) or analysis using AI.

[0023] Next, an example of the configuration of an HMD will be described with reference to FIGS. 4 and 5. In this embodiment, a fully immersive HMD capable of providing a VR experience is used. FIG. 4 shows the appearance of this HMD. As shown in the figure, in this example, the HMD 1 performs tracking using an inside-out method by using a sensor (camera, etc.) mounted on the headset. Therefore, tracking is possible without the need for a separate external sensor, and the user can, for example, enjoy a VR experience in a cable-free state.

[0024] The HMD 1 is formed with wearing housings 18a and 18b. When viewed from the perspective of a user wearing the HMD 1, the left side is considered left and the right side is considered right, and the wearing housing 18b of the HMD 1 is provided with a left display 102a for the left eye and a right display 102b for the right eye. By displaying an image for the left eye on the left display 102a and an image for the right eye on the right display 102b, the HMD 1 can, for example, allow the user to recognize a three-dimensional image. The displays may be configured to use other methods capable of three-dimensional display, for example. The HMD 1 (specifically, the displays) may also be provided with lenses used for adjusting the display position, etc.

[0025] The HMD 1 also includes cameras and distance sensors. In this example, a left front camera 112L, a right front camera 112R, a left front distance sensor 167, and a right front distance sensor 168 are arranged on the front side of the HMD 1. These components can be used to acquire information about the area in front of the HMD 1. A left side camera (not shown), a right side camera 114R, a left side distance sensor (not shown), and a right side distance sensor 172 are arranged on the left and right sides of the HMD 1. These components can be used to acquire information about the areas to the side of the HMD 1. The HMD 1 also includes a left rear camera 113L, a right rear camera 113R, a left rear distance sensor 169, and a right rear distance sensor 170. These components can be used to acquire information about the area behind the HMD 1. The number, arrangement, and angle of view of the cameras and distance sensors can be set as appropriate. The HMD 1 also includes a speaker 122 that outputs audio to the user. As an example, the speakers 122 may be arranged on both the left and right sides of the HMD 1 .

[0026] 5 shows an example of a specific configuration of hardware, etc. The HMD 1 includes a display 102, a data processing unit 110, a front camera 112, a rear camera 113, and a side camera 114.

[0027] The display 102 displays, for example, an image of VR content. In this embodiment, as described above, the display 102 includes a left display 102L for the left eye and a right display 102R for the right eye. However, the structure of the display may be changed as appropriate.

[0028] The data processing unit 110 includes, for example, a processor 101, a ROM (Read Only Memory) 103, a RAM (Random Access Memory) 104, and a storage 105. The processor 101 is configured using a CPU (Central Processing Unit) and the like, and is connected to various components via a bus 106. The ROM 103 is an auxiliary storage device and is configured by a flash ROM or the like. The RAM 104 is a main storage device. The processor 101 temporarily stores data such as programs in the RAM 104 and executes data processing. The storage 105 is an auxiliary storage device and is configured by a non-volatile storage medium.

[0029] The auxiliary storage device may store various types of data such as programs. For example, it may store a basic operation program related to an OS (Operating System), a VR viewing program used for viewing VR content, and an alert program that alerts the user, which will be described in detail later. Note that these programs may each be separate programs. However, this is not limited to this, and a single program may include a basic operation function, a VR viewing function, and an alert function and provide these functions. Alternatively, different divided programs may provide the above functions.

[0030] As described above, the front camera 112 is used to obtain information about the front, and includes, for example, a left front camera 112L and a right front camera 112R. As described above, the rear camera 113 is used to obtain information about the rear, and includes, for example, a left rear camera 113L and a right rear camera 113R. As described above, the side camera 114 includes, for example, a left side camera 114L and a right side camera 114R. Note that the camera may include an infrared camera, an infrared light illuminator, or the like, to improve infrared light sensitivity in consideration of taking pictures in dark areas.

[0031] The sensor group 160 may include, for example, a GNSS sensor (GPS 161 in the figure), a gyro sensor 162, a geomagnetic sensor 163, an acceleration sensor 164, an illuminance sensor 165, and a proximity sensor 166. Note that the sensor group 160 may include sensors of different types than those described above. Furthermore, the sensor group 160 may be configured such that the above sensors are omitted as appropriate. Furthermore, the sensor group 160 may be omitted from the HMD 1.

[0032] The GNSS sensor is a device that receives signals from GNSS (Global Navigation Satellite System) satellites in the sky and is used to detect the current position of the HMD 1. The HMD 1 can use the GNSS sensor to detect the position of the user wearing the HMD 1.

[0033] The acceleration sensor 164 detects acceleration, which is a change in speed per unit time, and can detect movement, vibration, shock, and the like. The acceleration sensor 164 can detect the tilt and orientation of the HMD 1 worn by the user. The gyro sensor 162 detects angular velocity in the rotational direction and can detect vertical, horizontal, and diagonal orientations. Therefore, the acceleration sensor 164 and the gyro sensor 162 can be used to detect the orientation of the HMD 1, such as the tilt and orientation. The geomagnetic sensor 163 detects the Earth's magnetic force and detects the direction in which the HMD 1 is facing. A three-axis type sensor that detects geomagnetic fields in the up-down direction in addition to the front-back and left-right directions can be used to detect the movement of the HMD 1 by capturing changes in geomagnetic field in response to the movement of the HMD 1. These sensors make it possible to detect the posture of the user wearing the HMD 1.

[0034] The illuminance sensor 165 is a sensor that detects the ambient brightness. The illuminance sensor 165 includes, for example, a light-receiving element such as a phototransistor or a photodiode as a sensor element, converts the received light into an electric current, and outputs the electric current corresponding to the illuminance. The HMD 1 can, for example, automatically adjust the backlight of the display 102 based on the output of the illuminance sensor 165. This can reduce power consumption and improve the visibility of the screen.

[0035] The proximity sensor 166 is a sensor that detects the approach of an object, a person, etc. without contact. The proximity sensor 166, for example, emits energy such as light, receives the energy reflected from the object using a light-receiving element, etc., and converts the energy into an electric current. The proximity sensor 166 then detects the approach of an object, a person, etc. based on the change in energy. The HMD 1 can detect the approach of an object, a person, etc. using the proximity sensor 166.

[0036] As described above, the HMD 1 is equipped with distance measurement sensors, for example, a left front distance measurement sensor 167, a right front distance measurement sensor 168, a left rear distance measurement sensor 169, a right rear distance measurement sensor 170, a left side distance measurement sensor 171, and a right side distance measurement sensor 172.

[0037] The ranging sensor is a sensor that can measure the distance from the HMD 1 to an object, the object's position, and capture the three-dimensional shape of the object. Examples of ranging sensors include LiDAR (Light Detection and Ranging), which irradiates an object with laser light such as infrared light and measures the scattered light that bounces back; TOF (Time of Flight) sensors, which measure distance by measuring the reflection time of pulsed light irradiated on the object for each pixel; and millimeter-wave radar, which emits millimeter-wave radio waves and captures the reflected waves. The ranging sensor may also be a sensor that performs measurements based on the angle at which light reflected by the object is received. That is, the ranging sensor may be a triangulation sensor. Furthermore, in the HMD 1, the ranging sensor may be configured as a stereo camera that performs measurements based on parallax images.

[0038] The HMD 1 also includes a gaze detection sensor 173. The gaze detection sensor 173 may include a left eye gaze sensor that detects the gaze of the left eye and a right eye gaze sensor that detects the gaze of the right eye, and can detect the movement and direction of the left and right eyes to capture the user's gaze point. The HMD 1 may perform eye tracking using the gaze detection sensor 173 and, for example, acquire the detection results as input information. The HMD 1 may then execute a predetermined process based on this input information.

[0039] The HMD 1 also includes a microphone 121, a speaker 122, a vibrator 174, and a battery 175. The microphone 121 collects the user's own voice, external sounds, and the like, and converts them into audio data. The user may vocalize instructions such as input operations, and the HMD 1 may acquire the audio data collected by the microphone 121 as instruction information for input operations and execute predetermined processing. The speaker 122 outputs audio based on the audio data. The speaker 122 outputs, for example, audio (sound) of VR content played during a VR experience. The speaker 122 can also notify the user of various notification information by audio. The vibrator 174 is a device that generates vibrations under control of the processor 101, and converts, for example, notifications sent to the user by the HMD 1 into vibrations. The battery 175 is configured to supply power to the HMD 1. The processor 101 may control the amount of power output by the battery 175 based on, for example, the operating status of the HMD 1 (such as the calculation load status of the HMD 1 and the dimming status of the display 102). However, the HMD 1 may be configured such that the processor 101 does not control the battery 175.

[0040] The HMD 1 includes an operation I / F 130. The operation I / F 130 is an interface used by the user for operation input, and information that the user wishes to input is input via the operation I / F 130. The operation I / F 130 is an interface that realizes input means using, for example, gaze, hands, gestures, a pointer, and even operation devices separate from the HMD 1, such as a keyboard, key buttons, and touch keys. The HMD 1 can be configured to be connectable to these operation devices via a wired or wireless connection.

[0041] The HMD 1 also includes a communication I / F 141, a wireless transmission I / F 142, and a wireless reception I / F 143. The communication I / F 141 is a communication interface that performs wireless communication with an external server or the like via an external network using base station communication or the like. The communication I / F can receive various information stored in the server. Note that long-distance wireless communication such as W-CDMA (Wideband Code Division Multiple Access) or GSM (Global System for Mobile communications) (registered trademark) can be used as base station communication.

[0042] The wireless transmission I / F 142 and the wireless reception I / F 143 are communication interfaces that perform short-range wireless communication with devices within a range where short-range wireless communication is possible. Short-range wireless communication is performed using, for example, but not limited to, an electronic tag. If the HMD 1 is located near a device that is capable of at least wireless communication, short-range wireless communication may be performed using Bluetooth (registered trademark), IrDA (Infrared Data Association, registered trademark), Zigbee (registered trademark), HomeRF (Home Radio Frequency, registered trademark), or a wireless LAN (IEEE 802.11a, IEEE 802.11b, IEEE 802.11g).

[0043] The processor 101 may also be directly connected to an external device via the wireless transmission I / F 142 and the wireless reception I / F 143 to transmit and receive data. The processor 101 may also be directly connected to a content distribution device via the wireless transmission I / F 142 and the wireless reception I / F 143 to receive VR content and transmit and receive data. Here, the content distribution device is, for example, a computer such as a personal computer, a home game console, a portable game console, or a smartphone.

[0044] Note that the arithmetic processing of the processor 101 may be executed in part or in whole by an external server, or in part or in whole by a device within a range where short-range wireless communication is possible, such as a content distribution device.

[0045] The HMD 1 also includes a network sensing unit 180. When an external device or an external server analyzes multipath propagation information (e.g., Wi-Fi CSI information) of radio waves transmitted and received between devices, the network sensing unit 180 receives the results of the external processing, and the processor 101 controls alerting the user based on the received results. In this case, the network sensing unit 180 may include, for example, a communication I / F 141, or a wireless transmission I / F 142 and a wireless reception I / F 143.

[0046] When the HMD 1 analyzes multipath propagation information (e.g., Wi-Fi CSI information) of radio waves transmitted and received between devices, for example, the network sensing 180 receives radio waves (e.g., Wi-Fi radio waves) transmitted from the devices, and the processor 101 extracts multipath propagation information from the received radio waves, analyzes the extracted multipath propagation information, and controls alerting the user based on the results. In this case, the network sensing 180 may be configured to include, for example, a device (transmitter and antenna) that transmits and receives Wi-Fi radio waves. When the analysis is performed using multipath propagation information of radio waves using base station communication, the network sensing 180 may be configured to include a device (transmitter and antenna) that transmits and receives radio waves using base station communication. The transmitting and receiving device may also be equipped with multiple antennas, which allows for efficient capture of radio waves from multiple paths.

[0047] The HMD 1 may acquire position information, for example, based on the following method. In this case, the GPS 161 may be omitted.

[0048] The HMD 1 may perform position detection based on the reception timing between devices using FTM (Fine Timing Measurement) of radio waves transmitted and received between devices. For example, in FTM using Wi-Fi radio waves, distance is calculated by measuring the round-trip time of radio waves between devices. However, this method can only measure the distance to devices capable of transmitting and receiving Wi-Fi radio waves. The HMD 1 may also perform position detection using multipath propagation information of radio waves transmitted and received between devices. For example, by analyzing CSI information within a Wi-Fi area, it is possible to perform position detection even for mobile objects that do not have information terminals (e.g., smartphones) capable of transmitting and receiving Wi-Fi radio waves. The HMD 1 may perform position detection and distance detection by combining FTM, CSI, and the like.

[0049] The HMD 1 may determine the current position by analyzing, for example, an image captured by a camera and / or three-dimensional point cloud data measured by a distance measurement sensor. That is, the HMD 1 may determine the current position based on SLAM (Simultaneous Localization and Mapping).

[0050] Also, there is known a technology such as a visual positioning service (VPS) that identifies a position by analyzing a camera image. The HMD 1 may acquire position information by the VPS using an image captured by the camera. The HMD 1 may transmit the image captured by the camera to an external device, an external server, or the like. The HMD 1 may then acquire position information based on the VPS from the external device, the external server, or the like.

[0051] The HMD 1 may also acquire position information by Pedestrian Dead Reckoning (PDR), for example, using a sensor. The PDR may use a gyro sensor 162, an acceleration sensor 164, or the like. The HMD 1 may perform position detection and distance detection by combining a plurality of these methods. This enables position detection that compensates for errors and shortcomings of various detection methods.

[0052] The position information processed by the HMD 1 may be world coordinate system information based on the real space, or may be local coordinate system information based on the position of the HMD 1. It may also be a coordinate system based on something other than the HMD 1. For example, this includes a local coordinate system centered on a suspicious person.

[0053] Next, an example of a process related to a warning in this embodiment will be described with reference to Fig. 6. First, data is registered in a device that performs Wi-Fi sensing (S100).

[0054] Before viewing VR, the user uses the HMD 1 to set (map) an area to be detected (detection target area). As an example, mapping is performed by the user wearing the HMD 1 moving the detection target area. As a result, the HMD 1 acquires map data of the detection target area. The detection target area is an area that is the target of Wi-Fi sensing, and is, for example, the entire indoor space where the HMD is used. For example, if the detection target area is indoors, this map data includes information such as indoor corridors and the range and position of each room. However, the user may also set (map) the detection target area so that it includes outdoors. The mapping of the detection target area may be performed by the user operating a controller, or may be performed using an image captured by an external camera built into the HMD 1 worn by the user.

[0055] Map data of the detection target area is stored in a device that performs Wi-Fi sensing, and when the HMD 1 performs Wi-Fi sensing, the HMD 1 stores the map data of the detection target area in the storage 105. On the other hand, when Wi-Fi sensing is performed outside the HMD 1, the HMD 1 transmits the map data of the detection target area to an external device, an external server, or the like, and storage of the external device, external server, or the like stores the map data of the detection target area. The map data of the detection target area is registered and used in the Wi-Fi sensing process.

[0056] Furthermore, a device that performs Wi-Fi sensing processing acquires and registers position information of Wi-Fi access points. When the HMD 1 performs Wi-Fi sensing, the HMD 1 stores and registers the position information of the Wi-Fi access points in the storage 105. On the other hand, when an external device, an external server, or the like performs Wi-Fi sensing, the external device, the external server, or the like stores and registers the position information of the Wi-Fi access points in the storage.

[0057] After the advance data registration in S100, the device that performs Wi-Fi sensing determines whether to activate the detection function by Wi-Fi sensing (S101). As an example, the user can switch ON / OFF activation of the detection function of Wi-Fi sensing by an appropriate means.

[0058] For example, the user inputs information on whether the Wi-Fi sensing detection function is activated or deactivated from the operation I / F 130, and the HMD 1 acquires the user's input. When Wi-Fi sensing is performed outside the HMD 1, the HMD 1 may transmit the acquired input to an external device, an external server, or the like. Furthermore, when an information display device such as the HMD 1 displays an image of the surroundings of the HMD 1 captured by an external camera on the display, or when the image does not impair the user's vision by a certain threshold or more, the device may determine that the user's vision is not being inhibited and may determine not to execute Wi-Fi sensing. Here, when the user's vision is not impairing by a threshold or more, the image is displayed in a low occupancy range, for example, 20% or less, allowing the user to see their surroundings, assuming that a state in which an image such as VR content is displayed entirely on the HMD 1's display is 100%. Alternatively, if the speaker of the HMD 1 does not output sound at a volume above a certain threshold, it may be determined that the user's hearing is not suppressed, and it may be determined not to execute Wi-Fi sensing. Similarly, if it is determined that the user's five senses, such as sight and hearing, are not suppressed, it may be determined not to execute Wi-Fi sensing, since the user can sense danger themselves. This allows for omitting sensing of moving objects that the user is aware of, thereby reducing the processing load on the system.

[0059] If the setting in S101 is ON, the device performing Wi-Fi sensing executes Wi-Fi sensing while the user is viewing VR and determines whether a moving object is present (S102).The device performing Wi-Fi sensing also determines whether a moving object is present within the detection target area (S103).

[0060] Note that when the user maps the detection target area in S100, a moving object already present in the detection target area may be treated as a moving object that was present from the beginning. A moving object that was present from the beginning is likely to be known by the user, and a moving object that is newly detected during the VR experience is considered to be less dangerous.

[0061] Therefore, when the HMD 1 performs Wi-Fi sensing, the HMD 1 may acquire feature data related to the CSI of a moving object (a moving object that exists from the beginning) in the map data by Wi-Fi sensing, for example, when registering the map data in S100. Then, the HMD 1 may exclude the moving object having the acquired feature data from the processing targets in S102 and thereafter. On the other hand, when Wi-Fi sensing is performed outside the HMD 1, an external device, an external server, or the like may detect a moving object in the map data and acquire feature data of the CSI of the moving object, for example, when registering the map data. Then, the moving object having the feature data acquired at this timing may be excluded from the processing targets in S102 and thereafter. Here, "exclusion" includes exclusion from detection targets, exclusion from urgency level determination targets, exclusion from display targets, exclusion from attention alert targets, and exclusion from transmission information. This eliminates the need to process moving objects that existed before S101, thereby reducing the processing load on the HMD 1 and preventing unnecessary information from being displayed to the user, thereby not interfering with immersion in the VR space.

[0062] For example, the HMD 1 may determine an urgency level (S104) and issue a warning in S106 (described later). Here, the urgency level is a value related to the level of warning. By considering the level of urgency, warnings can be issued based on the level of urgency, ranging from simple warnings such as displaying a message to warnings that stop the VR and prompt the user to become more aware. However, if warnings are issued without considering the level of urgency, S104 may be omitted. Furthermore, as described above, if it is determined that the user's five senses, such as sight and hearing, are not suppressed, the user may be able to sense danger themselves. Therefore, control may be exercised so as not to determine the urgency level or issue a warning to the user. This prevents unnecessary notifications targeting moving objects that the user is aware of. Furthermore, if it is determined that the user's sight and hearing are suppressed due to the progress of content or a change in content, the HMD 1 may switch to determining the urgency level or issuing a warning to the user.

[0063] For example, as shown in FIG. 7 (more specifically, the left part of FIG. 7 ), a user can set the urgency level based on the classification of moving objects. In this example, the urgency levels are set to three levels, from Level 0 to Level 3. Level 0 is the lowest level of urgency, and the higher the number from Level 1, the higher the urgency, with Level 3 being the most urgent. When setting the urgency level, a user first registers familiar objects, such as their family, acquaintances, animals (pets) owned by the user, and machines (autonomously mobile robots) owned by the user, in a database. Note that this database is constructed so that a device performing Wi-Fi sensing can refer to it during Wi-Fi sensing.

[0064] The database also manages familiar objects registered by the user in a predetermined classification. In the example of Fig. 7, the database manages the registered objects of the user in the classifications of "family," "acquaintances," "non-human (animals)," and "non-human (machines)."

[0065] Furthermore, the database manages the user's registration target and the feature data of the CSI of the registration target in association with each other. Here, when the user registers the registration target, the device performing Wi-Fi sensing selects the corresponding registration target from the targets detected by Wi-Fi sensing, thereby acquiring the feature data of the CSI of the registration target.

[0066] The user can then set an urgency level for each classification of moving objects managed in the database and for each classification of moving objects not managed in the database. Therefore, familiar objects are registered in the database, and the urgency level for the classification of moving objects managed in the database is set low, while the urgency level for the classification of moving objects not registered as familiar objects is set high. This prevents excessive alerts to familiar objects and provides appropriate alerts to non-familiar moving objects. In other words, it is possible to effectively prevent a decrease in the user's sense of immersion due to excessive alerts based on the detection of familiar objects (i.e., highly safe objects). Although not shown in the figure, moving objects managed in the database may be excluded from alert targets and configured not to be notified to the user.

[0067] Furthermore, the user can set the urgency level based on the classification of the detection area, as shown in FIG. 7 (more specifically, the upper part of FIG. 7 ). For example, the detection area is classified as indoors and outdoors, and the user can set an indoor urgency level and an outdoor urgency level. For example, Wi-Fi radio waves transmitted between devices reach not only indoors but also some outdoors, and radio waves can be transmitted and received outside the entrance or around the house. Therefore, even if the device is not installed outdoors, radio waves propagated by multiple waves due to the influence of moving objects outdoors can be detected. Furthermore, indoors, the floor is classified as the same floor as the floor where the user uses the HMD (where the z-axis value of the detected position is approximately the same) and a different floor (where the z-axis value of the detected position is different), and the user can set an urgency level for the same floor and a different floor. The risk of contact with the detected object increases when the user and the detected object are on the same floor in three-dimensional positional relationship rather than in linear distance. Furthermore, the same floor is divided into the space used by the user when using the HMD (sometimes called the same space), other spaces close to the same space, and other spaces far from the same space, and the user can set an urgency level for the same floor. Here, the same space corresponds to the space (play area) set as the range within which the user moves and / or acts to have a VR experience, for example.

[0068] For example, a user can set the urgency level so that the indoor urgency level tends to be higher than the outdoor urgency level. Furthermore, a user can set the urgency level so that the urgency level on the same floor tends to be higher than the urgency level on another floor. Furthermore, a user can set the urgency level so that the urgency level of a space closer to the user tends to be higher than the urgency level of a space farther from the user on the same floor. By setting the urgency level in this manner, excessive alerts based on detections that pose little risk to the user can be suppressed, and appropriate alerts can be issued for detections that pose a significant risk. Therefore, a decrease in the user's sense of immersion due to excessive alerts based on detections that pose little risk to the user can be effectively suppressed.

[0069] In the above example, the urgency level is set separately for the classification of the detection target and the classification of the detection area. However, as shown in the entirety of FIG. 7 , the urgency level may be set by associating the detection target with the detection area. In other words, the urgency level may be set for a combination of the classification of the detection target and the classification of the detection area. In this case, detailed settings that take both into consideration are possible.

[0070] In the example of FIG. 7 , the urgency level is set taking into account minor classifications. However, for example, the urgency level may be set taking into account medium classifications. For example, the urgency level may be set for detection targets classified into human, non-human (animal), non-human (machine), human not in the database, non-human (animal), and non-human (machine). Furthermore, even for a person not in the database, the urgency level may be different depending on whether the person is detected together with a person registered in the database or whether the person not in the database is detected alone. Furthermore, the urgency level may be set for detection areas classified into same floor, different floor, and outdoors. The classification of detection targets and detection areas in this example is merely an example and is not limited to this. More detailed classifications or fewer classifications may also be used.

[0071] The user may register the database, for example, at the timing of S100 before viewing VR content. The user may also set the urgency level, for example, at the timing of S100 before viewing VR content. Here, the HMD 1 may display a setting menu on the display 102, and the user may set the urgency level using the operation I / F 130. The user may also select and set from among pre-set urgency level settings. With regard to this setting, the previous setting information is saved, and if the user does not set it, the past setting information may be automatically set.

[0072] When the HMD 1 performs Wi-Fi sensing, information regarding the user setting of the urgency level may be stored in the storage 105. When Wi-Fi sensing is performed by an external device, an external server, or the like, the HMD 1 transmits information regarding the user setting of the urgency level, and this information may be stored in storage referenced by the external device, the external server, or the like.

[0073] In the determination of S104, the Wi-Fi sensing device can identify a moving object based on the characteristics of the multipath propagation information (CSI) contained in radio waves affected by the moving object through Wi-Fi sensing. Then, in S104, the Wi-Fi sensing device references the above-described urgency level setting to acquire the urgency level corresponding to the identified moving object. Here, the urgency level may be set based on information from the sensors of the HMD 1 in addition to the multipath propagation information. For example, if a detection target exists in the same space, the accuracy of target identification and position detection can be improved by also referencing information from the camera and ranging sensor of the HMD 1. Furthermore, the results of detection of surrounding sounds (such as surrounding noises, door opening and closing sounds, and footsteps) from the microphone of the HMD 1 may be reflected.

[0074] Returning to FIG. 6 , the description will continue. The HMD 1 selects a method of alerting the user to the moving object and communication information (S105). When S104 is executed, the level of urgency is taken into consideration. When Wi-Fi sensing is executed outside the HMD 1, an external device, an external server, or the like may execute S105 and transmit the selection result of the urgency level to the HMD 1. An example of the determination in S105 will be described in detail later.

[0075] Based on the result of S105, the HMD 1 alerts the user of the HMD 1 regarding the detection of a moving object (S106). Then, the HMD 1 determines whether to end the alert (S107), and ends the alert according to the result of S107 (S108). Note that an external device, an external server, or the like may execute S107 and may transmit the determination result of the end of the alert to the HMD 1. An example of the determination in S107 will be described in detail later.

[0076] According to the first embodiment, by detecting a moving object using Wi-Fi sensing and issuing a warning, it is possible to achieve both a sense of immersion and safety for the user.

[0077] Second Embodiment In the process of the first embodiment, an example was described in which, when a moving object is simply detected within a detection target area (detection target area), a warning about the moving object is issued. However, by making all detection targets the target of warning, unnecessary warnings are issued, which reduces the user's sense of immersion. For example, it is considered that highly safe people such as family members and acquaintances may be excluded from the target of warning.

[0078] Therefore, in the second embodiment, an example will be described in which it is determined in S202 shown in Fig. 8 whether a moving object is a target for a warning, and the warning is issued based on the result. Note that the description of the content that has already been explained may be omitted. Also, as in the above explanation, when issuing a warning without considering the urgency level, S104 may be omitted, and the detection area determination S103 may be omitted.

[0079] As with the urgency levels described above, a user first registers familiar objects such as their family, acquaintances, animals (pets) owned by the user, and machines (autonomously mobile robots) owned by the user in a database. This database is constructed so that a device that performs wireless communication sensing (e.g., Wi-Fi sensing) using multipath propagation information can refer to the database during Wi-Fi sensing.

[0080] The database also manages familiar objects registered by the user in a predetermined classification. In the example of Fig. 9, the database manages the registered objects of the user in the classifications of "family," "acquaintances," "non-human (animals)," and "non-human (machines)."

[0081] Furthermore, the database manages the user's registration target and characteristic data of the registration target's multipath propagation information (CSI) in association with each other. Here, when the user registers the registration target, a device performing Wi-Fi sensing detects the registration target through Wi-Fi sensing, thereby acquiring characteristic data of the CSI of the registration target. Furthermore, the device performing Wi-Fi sensing may detect the registration target using information received by another device, or may receive and use information of the registration target detected by another device performing Wi-Fi sensing.

[0082] Then, in S202, when the device performing Wi-Fi sensing detects a moving object through Wi-Fi sensing, it determines whether or not this moving object is a detection target.

[0083] Here, a device performing Wi-Fi sensing acquires CSI feature data of a mobile object through Wi-Fi sensing, for example, and checks whether corresponding CSI feature data exists in a database. If the CSI feature data exists in the database, the mobile object having this CSI feature data is excluded from the targets of alerts. On the other hand, if the corresponding CSI feature data does not exist in the database, the mobile object having this CSI feature data becomes the target of alerts. Note that if no CSI feature data exists, the mobile object may be checked against past history data to see if the CSI feature data has ever been acquired. Even if no feature data with CS1 exists, if there is any corresponding data among previously detected data, the risk of alerts is reduced. If unregistered feature data corresponding to the past history exists, unnecessary notifications can be prevented by arbitrarily excluding the mobile object from the targets of alerts.

[0084] The user may register the database at the timing of S100 before VR viewing, for example. Here, the HMD 1 may display a setting menu on the display 102, and the user may perform settings using the operation I / F 130. Also, at the timing of S100, the detected target may be registered in the database, or the user may select from the detected targets and register them in the database.

[0085] According to the second embodiment, as an example, a highly safe moving object that the user does not need to detect is registered in a database, and the moving object is excluded from targets for alerts, thereby suppressing a decrease in the user's sense of immersion. Furthermore, a decrease in the user's sense of immersion can also be suppressed by registering a moving object that exists at a certain point in time in a database and excluding the moving object from targets for alerts. Note that, although an example of excluding an object already registered in a database from targets for alerts has been described here, a similar effect can be obtained even when a determination example described later is applied.

[0086] Third Embodiment In the first embodiment, an example was described in which a warning about a moving object is issued when the moving object is simply detected within a detection target area. However, it is also possible to identify the area within the detection target area in which the moving object is detected and issue a warning in a manner corresponding to the area in which the moving object is detected. Furthermore, for example, it is also possible that the warning conveys to the user the direction in which the moving object was detected relative to the user itself, the direction of the area in which the moving object was detected, information about the area in which the moving object is located, and the like. Physical information about the moving object, such as its height, may also be conveyed.

[0087] Therefore, in the third embodiment, an example will be described in which an area where a moving object exists is determined in S203 shown in Fig. 10 and a warning is issued based on the determination result. Note that the description of the content that has already been described may be omitted.

[0088] A device that performs wireless communication sensing (e.g., Wi-Fi sensing) using multipath wave propagation information detects a moving object through Wi-Fi sensing. Here, as shown in Fig. 11 , for example, the device that performs Wi-Fi sensing references map data of the detection target area created in S100 and determines whether the detection area for the moving object is, for example, the same floor indoors, a different floor indoors, or an outdoor area.

[0089] Furthermore, a device that performs Wi-Fi sensing may acquire information about the play area and determine whether the detection area of ​​the moving object is the same space as the user, a different space from the user (close distance), or a different space from the user (long distance).

[0090] The device performing Wi-Fi sensing may determine the detection area of ​​a moving object based on the position information of the Wi-Fi router (i.e., information about the access point). The device performing Wi-Fi sensing may also acquire the position information of the HMD 1 and the position information of the Wi-Fi router, and perform processing by linking the respective position information (coordinate systems) in Wi-Fi sensing. The device may also acquire the position information using multipath wave propagation information communicated between the detected moving object and an information terminal owned by the detected moving object.

[0091] The Wi-Fi sensing device may acquire the position of the HMD 1 and the position of the moving object, and determine the detection area based only on the distance to the moving object. In this case, map data may not be used. The presence of a wall or other obstacle may be determined by utilizing the attenuation of signals transmitted and received between the user and the detected moving object.

[0092] Furthermore, the Wi-Fi sensing device may acquire the position of the HMD 1 and the position of the moving object, and determine, based only on the distance to the moving object, whether the detection area of ​​the moving object is the same space as the user, a separate space from the user (close distance), or a separate space from the user (long distance). In this case, the play area information does not need to be used.

[0093] According to this embodiment, by determining the detection area of ​​a moving object, it is possible to issue a warning in a manner appropriate to the detection area. For example, by issuing a weak warning for a detection area where there is little need for a warning or issuing a warning after the content has ended, excessive warnings can be suppressed and a decrease in the user's sense of immersion can be suppressed. Furthermore, by issuing a weak warning for detection of a moving object in a detection area far from the user, excessive warnings can be suppressed and a decrease in the user's sense of immersion can be suppressed. Furthermore, for example, for detection of a moving object in a detection area close to the user, a warning may be issued that conveys the distance and direction to the moving object. Note that detection areas and non-detection areas may be preset by the user. For example, by not performing detection in areas such as a toilet or bathroom, or by detecting only entry and exit to the area and not detecting subtle movements within the area, invasion of privacy can be prevented. Furthermore, as described above, if it is determined that the user's five senses, such as sight and hearing, are not suppressed, the user can also sense danger themselves, so the detection area settings may be revised. For example, it may be determined that the user can recognize a moving object in the same space as the user, and therefore the detection of the moving object, the determination of the urgency level, and the user's attention may not be performed. Furthermore, unnecessary notifications targeting moving objects recognized by the user may be prevented by performing a process such as lowering the urgency level for the entire detection area including the same space as the user, or for all detection areas other than the same space as the user. Furthermore, if it is determined that the user's vision or hearing is impaired due to the progress of content or a change in content, the urgency level may be returned to the original setting, or the user's attention may be reissued.

[0094] <Fourth embodiment> A fourth embodiment will be described with reference to Fig. 12. Note that the description of the content already described may be omitted. As shown in Fig. 12, the process of S202 described in the second embodiment and the process of S203 described in the third embodiment may be executed in combination. This enables more appropriate attention to be drawn.

[0095] <Specific Example of Processing> Next, a specific example of processing will be described with reference to Fig. 13 to Fig. 18. First, a specific example of the detection function activation determination (S101) will be described with reference to Fig. 13. Furthermore, when one or more of the following examples are satisfied, the detection function is activated via wireless communication.

[0096] As shown in FIG. 13 , a device that performs wireless communication sensing (e.g., Wi-Fi sensing) using multipath propagation information activates a detection function using Wi-Fi sensing when the use of sensing information (i.e., CSI information) via Wi-Fi radio waves is configured (Example 1). Here, as described above, a user can switch the activation of the Wi-Fi sensing detection function ON / OFF using appropriate means. That is, a user can switch the activation related to Example 1 ON / OFF using appropriate means. This reduces the need to notify the user when something non-threatening, such as a family member, is simultaneously present, and therefore switches ON / OFF to prevent unnecessary notifications from disrupting the immersive experience.

[0097] When a device that performs Wi-Fi sensing receives sensing information via Wi-Fi radio waves, it activates the Wi-Fi sensing detection function (Example 2). In Example 2, the Wi-Fi sensing detection function is automatically activated regardless of user operation. When only one user is present and is experiencing VR, it is necessary to notify the user of danger posed by a suspicious person. In such cases, danger can be avoided by automatically activating the detection function. The detection function may also be activated automatically if the HMD camera and sensors cannot detect a person in the same space, or if a moving object registered in the database at the time of activation cannot be detected.

[0098] The Wi-Fi sensing device activates its detection function when a moving object detected by Wi-Fi radio wave sensing is closer to the user than a specified distance. On the other hand, when a moving object detected by Wi-Fi radio wave sensing is farther from the user than a specified distance, the device automatically stops its detection function (Example 3). This detection distance may be changed depending on the size of the building in which the user is experiencing VR, or may be set arbitrarily by the user.

[0099] When a Wi-Fi sensing device detects a sound above a specified level (above a specified volume) around the user, it automatically activates the detection function (Example 4). This allows a user who is hearing impaired to be notified of high-risk danger information, such as a suspicious person breaking a window or a family member screaming.

[0100] Here, when the device performing Wi-Fi sensing is the HMD 1, the HMD 1 may detect and process ambient sounds using a microphone 121 or a sound sensor appropriately provided in the HMD 1. Alternatively, for example, a device provided within the detection target area may detect sounds and transmit the detection results to the HMD 1. The direction from which the sound is emitted may also be used in conjunction with Wi-Fi sensing detection.

[0101] When the device performing Wi-Fi sensing is an external device, an external server, etc., the HMD 1 may detect surrounding sounds using a microphone 121 or a sound sensor appropriately provided in the HMD 1 and transmit the detection results to the external device, external server, etc. Also, for example, a device provided within the detection target area may detect sounds and transmit the detection results to the external device, external server, etc. Also, the direction of sounds detected by multiple devices may be used in combination with Wi-Fi sensing detection.

[0102] When a device that performs Wi-Fi sensing (in this example, an external server) receives an activation command from an external terminal (such as the HMD 1 or an information terminal owned by a family member), it activates the detection function (Example 5). Here, the user can switch the activation related to Example 5 ON / OFF using an appropriate means. For example, when information such as information about a family member's return home or information about a suspicious person entering the house is sent from a family member's terminal or another device, the detection function can be activated to receive notifications even during the VR experience.

[0103] Next, specific examples of determining the presence of a moving body (S102) and determining the detection target (S202) will be described with reference to Fig. 14. Note that in Examples 3, 4, 6, and 7, a device that performs wireless communication sensing (e.g., Wi-Fi sensing) using multipath propagation information may manage information (such as CSI information) of moving bodies determined as detection targets in a database and perform this processing by referring to the database.

[0104] For example, if one or more of the following examples are met, the process proceeds to the next step: If none of the following examples are met, the device performing Wi-Fi sensing waits until it detects a moving object.

[0105] As shown in Example 1, when a device performing Wi-Fi sensing detects a moving object within the detection range of Wi-Fi sensing, it determines the presence of the moving object (S102).

[0106] As shown in Example 2, a device performing Wi-Fi sensing determines that a moving object detected within the detection range of Wi-Fi sensing is a moving object for which a warning should be issued if the moving object is not a detection exclusion target (S202). Here, a detection exclusion target is a moving object for which a warning should be issued, and the user can set a detection exclusion target before S202, such as S100. If no particular setting is made, a detection exclusion target previously set by the user may be used, or a detection exclusion target may be selected and used from recommended settings previously set on the HMD, an external server, or an external device.

[0107] As shown in Example 3, when a moving object detected within the Wi-Fi sensing detection range is different from an object detected previously, the device that performs Wi-Fi sensing determines that the moving object is one requiring attention (S202). Here, based on the similarity with the object detected previously, if the similarity is equal to or greater than a specified value, the device may determine that the moving object is the same as the object detected previously.

[0108] As shown in Example 4, if a mobile object detected within the Wi-Fi sensing detection range is not in a pre-registered database, the device that performs Wi-Fi sensing determines that the mobile object is one requiring a warning (S202). Here, based on the similarity with a previously pre-registered database, if the similarity is equal to or greater than a specified value, the device may determine that the mobile object is the same as the object in the previously pre-registered database.

[0109] As shown in Example 5, the device performing Wi-Fi sensing determines that a moving object detected within the detection range of Wi-Fi sensing is a moving object that requires a warning if the moving object is a moving object that has been newly detected after the startup in S101 (S102, S202). That is, moving objects detected within the detection target area during user mapping may be excluded from the targets of warning, and moving objects that are newly detected after the startup in S101 may be the targets of warning.

[0110] As shown in Example 6, when a device performing Wi-Fi sensing detects multiple moving objects within the detection range of Wi-Fi sensing and a previously detected moving object and a moving object different from the previously detected moving object are detected simultaneously within a certain range, the device determines that the moving object different from the previously detected moving object is a moving object that requires attention (S202).

[0111] As shown in Example 7, when a plurality of moving objects are detected within the detection range of Wi-Fi sensing, and a moving object that is in a pre-registered database and a moving object that is not in the pre-registered database are simultaneously detected within a certain range, the device that performs Wi-Fi sensing determines that the moving object that is not in the pre-registered database is a moving object that requires a warning (S202). However, if there are multiple simultaneous detection histories, the moving object may be registered in the database and excluded from the targets of warnings, for example, as a companion whose safety is guaranteed.

[0112] As shown in Example 8, when a moving object detected within the Wi-Fi sensing detection range is identified as a human, the device that performs Wi-Fi sensing determines that the moving object is one for which a warning is required (S202). If the detected moving object is an animal or machine other than a human, the degree of danger is low, and therefore the urgency of issuing a notification is low, so the moving object may be excluded from the moving objects for which a warning is required. Furthermore, the user may set whether to limit the targets of notification to humans or to include other moving objects as well.

[0113] Next, a specific example of the detection area determination (S103, S203) will be described with reference to FIG.

[0114] As shown in Example 1, when a device performing Wi-Fi sensing detects a moving object within the detection range of Wi-Fi sensing, it determines that the detection area of ​​the moving object is within the detection range of Wi-Fi sensing (S103). Typically, the communication area of ​​Wi-Fi is set to cover the entire building, and communication is possible even in the area surrounding the building if it is close by. Therefore, it can be said that the entire communication area is the detection area.

[0115] As shown in Example 2, when a device performing Wi-Fi sensing detects a moving object in a specific area within the detection range of the Wi-Fi sensing, the device determines that the moving object has been detected (S203). For example, when a moving object is detected in a room different from the room in which the user is present, the device performing Wi-Fi sensing determines that the moving object has been detected in this room. As an example, by setting the entrance as the detection area for sensing, it is possible to detect a moving object entering through the entrance and notify the user of the detected information.

[0116] As shown in Example 3, when a Wi-Fi sensing device detects a moving object outside of pre-registered map data, it determines that the detection area of ​​the moving object is outside the detection target area (S103, S203). Moving objects outside the detection target area may be excluded from alert notifications. As described above, because radio waves used for communication extend beyond the building's exterior, in some cases, they may detect information about moving objects within adjacent buildings, or even moving objects such as people or vehicles traveling outdoors. Therefore, a detection target area for moving objects may be set in advance, and the device may be configured to not detect moving object detection information in that area, or not notify even if detected. Alternatively, the detection target area may be set to only include the area around the building entrance or the building's exterior, such as a garden or warehouse.

[0117] As shown in Example 4, when a device performing Wi-Fi sensing detects a moving object within pre-registered map data, it determines that the detection area of ​​the moving object is within the detection target area (S103, S203). The presence of a moving object within pre-registered map data indicates that the moving object has entered the detection area, which poses a risk of contact with the user, and it becomes necessary to determine whether to notify the user.

[0118] As shown in Example 5, when a device performing Wi-Fi sensing detects a moving object outside a detection exclusion area set within the detection range of Wi-Fi sensing, the device may determine the area in which the moving object was detected as the detection area (S203). Here, the detection exclusion area is an area in which warnings about the moving object are excluded, and the user can set a detection exclusion area within the detection target area before S203, such as S100. For example, by not performing detection in areas such as toilets and bathrooms, or by detecting only entry and exit to the area and not detecting detailed movements within the area, it is possible to prevent privacy violations.

[0119] As shown in Example 6, if a Wi-Fi sensing device detects a moving object continuously for a specified time or longer in a predetermined area set within the Wi-Fi sensing detection range, the device may determine that area as the detection area (S203). Here, prior to S203, such as S100, the user may set the predetermined area to be used in Example 6 within the Wi-Fi sensing detection range. Wi-Fi sensing detects moving objects by analyzing multipath propagation information, but measurement errors can lead to false detection of a moving object. Therefore, by determining that a moving object has been detected only if it has been detected for a specified time or longer, it is possible to prevent notifications due to false detection. Furthermore, notifications due to moving objects passing by a front door can be prevented by specifying the time until detection.

[0120] As shown in Example 7, when a Wi-Fi sensing device detects that a mobile object has entered the map from an entrance / exit other than those registered in the map data, it may determine that the area into which the mobile object has entered is the detection area (S203). If a mobile object has entered from an entrance / exit other than a normally set entrance / exit (such as a front door), there is a high possibility that it is an intrusion by a suspicious person. Therefore, when a mobile object has entered from such an area, the level of urgency is high and a notification is sent to the user. Here, the user can set the target entrance / exit in Example 7 before S203, such as S100.

[0121] As shown in Example 8, when a device performing Wi-Fi sensing detects a moving object within a specified distance from a user, the device may determine that the area in which the moving object is detected is a detection area (S203). Even when a moving object is detected, if the distance from the user is sufficiently far, the risk is considered low. Therefore, by performing detection only when a moving object enters within a certain distance, it is possible to prevent frequent notification information from being sent to the user.

[0122] As shown in Example 9, when a Wi-Fi sensing device detects a moving object within a play area set by a user, it may determine that the play area where the moving object was detected is the detection area (S203). Typically, a moving object entering the play area is detected using the cameras and sensors provided in the HMD 1. However, there are cases where the cameras and sensors provided in the HMD 1 do not detect an area covering 180 degrees around the user, but only target the area in front of the user. In such cases, by using Wi-Fi sensing information in a combined manner, it is possible to compensate for detection in areas not covered by the detection area of ​​the HMD 1.

[0123] Next, a specific example of the selection of the attention-attracting method and the communication information (S105) will be described with reference to FIGS. 16 and 17. FIG.

[0124] FIG. 16 shows examples of alert methods. As shown in Example 1, a device performing Wi-Fi sensing may determine an alert method in which information about a moving object is displayed to a user when the content the user is viewing ends. Alerting a user can, to some extent, disrupt the user's immersion in the content they are experiencing. Therefore, if it is determined that the level of urgency is not high, it is better to transmit information when the content ends.

[0125] As shown in Example 2, the Wi-Fi sensing device may determine an alert method for displaying information about a moving object at a timing set in advance by the user. The user may set the timing of the alert to match the content being experienced, allowing the alert to be issued at any timing. For example, when playing a highly immersive VR game, the device may be configured to notify alerts only when the content ends, except for alerts with a high level of urgency. When performing office work in a less immersive VR space, the device may be configured to issue alerts at any time.

[0126] As shown in Example 3, a device that performs Wi-Fi sensing may determine an attention method for displaying information about a moving object while a user is viewing content. If the urgency level is higher than a predetermined threshold, the user needs to be alerted while the content is being viewed. The attention may be issued while the content is being viewed in a situation where the user's input frequency is below a predetermined value, such as on a setting screen. Since situations where the user's input frequency is high are likely to result in a high level of immersion, avoiding such situations as much as possible can prevent disruption of immersion.

[0127] As shown in Example 4, the Wi-Fi sensing device may determine an alert method that interrupts the content the user is viewing and displays information about the moving object. In situations with a higher level of urgency, it may be necessary to provide an alert even if it disrupts the user's immersion. In such cases, the content may be paused or ended to provide the alert.

[0128] As shown in Example 5, the device performing Wi-Fi sensing may switch to an external camera mode and determine an attention-drawing method for displaying information about a moving object. That is, the device performing Wi-Fi sensing may determine an attention-drawing method for displaying information by superimposing information about the moving object on an image in real space. Note that the image of real space may be acquired by a camera provided in the HMD 1. Furthermore, the image of real space may be acquired by, for example, a camera provided within or around the detection target area.

[0129] As shown in Example 6, the device performing Wi-Fi sensing may determine a warning method for displaying information about a moving object using a message if the user has not previously turned off message display. Here, the user may set the warning method using the operation I / F 130.

[0130] As shown in Example 7, a device performing Wi-Fi sensing may determine how to provide a user with an alert that interrupts the content they are viewing.

[0131] As shown in Example 8, a device performing Wi-Fi sensing may determine the alert method to transition to see-through mode.

[0132] When determining the level of urgency (S104), the device performing Wi-Fi sensing may determine the method of issuing an alert based on the level of the urgency.

[0133] For example, when there is no urgency level (it is 0), the device that performs Wi-Fi sensing may determine an alert method to issue an alert when content ends or at a timing set by the user, as shown in Examples 1 and 2. Furthermore, when there is no urgency level (it is 0), the device that performs Wi-Fi sensing may determine not to issue an alert to users connected to the same network, or may reduce the processing load by setting the device not to perform detection at all.

[0134] For example, when there is no or a low urgency level, the device performing Wi-Fi sensing may determine an alert method in which the user of the HMD 1 (i.e., the immersed user) is not alerted, but the alert is given to nearby companions who are not immersed. Here, the device performing Wi-Fi sensing may determine to output alert information to smartphones of nearby companions.

[0135] For example, when the urgency level is high, as shown in Examples 7 and 8, the device performing Wi-Fi sensing may decide to issue an alert by forcibly stopping the content or forcibly switching to see-through mode, rather than simply issuing an alert by notification as shown in Examples 1 to 6.

[0136] Furthermore, the device that performs Wi-Fi sensing may determine a warning method when it determines that a moving object is a detection target and is a target for warning.

[0137] Furthermore, the device performing Wi-Fi sensing may determine an attention method depending on the detection area. For example, the device performing Wi-Fi sensing may determine an attention method such that, when a moving object is detected in an area far from the user, the device issues an attention at a timing set by the user in advance, and when a moving object is detected in an area close to the user, the device issues an attention method such that the device issues an attention in the middle of content.

[0138] 17 shows examples of transmission information. As shown in Example 1, a device performing Wi-Fi sensing may decide to transmit the type of moving object identified by Wi-Fi sensing. Note that if past detection history and moving object information (such as CSI feature data) are managed in a database, the device performing Wi-Fi sensing may specifically identify the moving object by referring to the database.

[0139] As shown in Example 2, a device performing Wi-Fi sensing may decide to transmit information about the distance to a moving object detected by Wi-Fi sensing (information about the distance between the user and the detected object). This allows the user to accurately know the distance between themselves and the detected moving object. This distance may be the straight-line distance or the distance along the shortest path that takes into account walls and other obstacles.

[0140] As shown in Example 3, a device that performs Wi-Fi sensing may decide to transmit information about the orientation of a moving object detected by Wi-Fi sensing (orientation information of the detected object relative to the user). Here, in addition to simply notifying the orientation, the device may also notify the angle at which the detected moving object is located horizontally relative to the front of the user.

[0141] As shown in Example 4, a device that performs Wi-Fi sensing may decide to transmit positioning information (position information of a moving object) of a moving object detected by Wi-Fi sensing. The positioning information may be world coordinate system information based on real space, or may be local coordinate system information based on the position of the HMD 1. Furthermore, it may be local coordinate system information centered on the detected moving object.

[0142] As shown in Example 5, a device performing Wi-Fi sensing may decide to transmit intrusion route information of a mobile object detected by Wi-Fi sensing. Here, the intrusion route information is information about the route of the mobile object after it has entered a detection target area. The device performing Wi-Fi sensing can, for example, acquire the intrusion route information from the positioning history of the mobile object detected by Wi-Fi sensing. This makes it possible to determine the risk level of the mobile object based not only on information about the mobile object at a certain point in time, but also on information about the routes it has taken in the past. Furthermore, it is also possible to know information about the location from which it entered.

[0143] As shown in Example 6, a device that performs Wi-Fi sensing may decide to transmit information about the danger level of a moving object detected by Wi-Fi sensing (i.e., the magnitude of the urgency level acquired for the moving object in S104). This allows the danger level of the detected moving object to be visually known, and therefore, even when multiple moving objects are detected, it is possible to identify moving objects that require attention.

[0144] As shown in Example 7, a device performing Wi-Fi sensing may decide to transmit behavioral information of a mobile object detected by Wi-Fi sensing. Here, the behavioral information relates to behavioral analysis results. That is, the behavioral information is information regarding the motion of the mobile object acquired by analyzing the CSI in Wi-Fi sensing. Note that, if CSI feature data and the like are managed in a database, the device performing Wi-Fi sensing may refer to the database to acquire behavioral information of the detected mobile object. It is known that wireless communication sensing using multi-wave propagation information, such as Wi-Fi sensing, can detect not only the presence of a mobile object but also its motion. By utilizing this, not only intrusion into the detection area but also behavior within the detection area can be grasped, and dangerous behavior such as searching a room can be detected and notified. Furthermore, by using technology that utilizes wireless communication sensing to grasp a person's fall or breathing state, a user who is immersed in the activity can be notified if a suspicious person is harming someone nearby or if a family member collapses inside a building.

[0145] As shown in Example 8, a device performing Wi-Fi sensing may decide to transmit location information on a map of a moving object detected by Wi-Fi sensing. That is, the device performing Wi-Fi sensing may decide to transmit the location of the moving object on a registered map (map data). This allows the positional relationship between the user and the detected object to be understood two-dimensionally, or three-dimensionally if multiple layers are involved in the detection object.

[0146] As shown in Example 9, a device performing Wi-Fi sensing may determine to transmit information on the number of moving objects detected by Wi-Fi sensing. The device performing Wi-Fi sensing may, for example, determine to transmit information on the number of all moving objects detected simultaneously. Furthermore, for example, if multiple moving objects are detected simultaneously and determined to be humans, the device performing Wi-Fi sensing may determine to transmit information on the number of moving objects determined to be humans (information on the number of people). Furthermore, when multiple moving objects are detected, priorities may be set and displayed. For example, the moving objects that the user should be aware of may be displayed in order of increasing urgency, or by displaying only moving objects with an urgency level above a certain level, or by erasing previously detected objects when a newly detected object appears.

[0147] As shown in Example 10, a device performing Wi-Fi sensing may decide to transmit detection signal level information of a moving object detected by Wi-Fi sensing. Here, the detection signal level information is information that determines the level of detection accuracy from the level of the detection signal. If the detection signal level is lower than a predetermined value, processing such as not performing detection may be performed.

[0148] As shown in Example 11, a device performing Wi-Fi sensing may decide to simply transmit information about the detection of a moving object through Wi-Fi sensing. For example, the device may simply display a pop-up message such as "A moving object has been detected" without displaying detailed information such as the type, direction, or urgency level of the moving object.

[0149] Next, an example of determining the alert termination condition (S107) will be described with reference to FIG.

[0150] As shown in Example 1, a device performing Wi-Fi sensing may determine that the warning should be terminated when a moving object can no longer be detected within the Wi-Fi sensing detection range (or detection target area).

[0151] As shown in Example 2, the Wi-Fi sensing device may terminate the warning when the danger level of the moving object is eliminated. The Wi-Fi sensing device may terminate the warning regarding the moving object, for example, when the urgency level of the moving object falls below a predetermined value. Hysteresis may be provided between the urgency level at which display starts and the urgency level at which display ends, to prevent the display from starting and ending intermittently.

[0152] As shown in Example 3, a device performing Wi-Fi sensing may determine that the warning should be terminated when a moving object detected within the detection range of the Wi-Fi sensing moves out of the detection range, or when a moving object detected within the detection target area moves out of the detection target area.

[0153] As shown in Example 4, the device performing Wi-Fi sensing may determine the end of the alert when the alert setting is turned off. The device performing Wi-Fi sensing may determine the end of the detection information display when the alert setting is turned off, for example. Here, the user may perform the setting using the operation I / F 130. However, if a moving object with an urgency level equal to or higher than a predetermined level is present within the detection area, the device may continue Wi-Fi sensing without accepting user operation.

[0154] Next, an example of a warning display will be described with reference to Figures 19 and 20. As shown in Figure 19, when a user 51 watches VR content 71, the user 51 is immersed in the VR content 71, and the user 51's vision is restricted by the display screen and the user's hearing is restricted by the speaker. At this time, it is difficult for the user 51 to grasp the situation around them.

[0155] When a moving object is detected by Wi-Fi sensing in such a state where some of the five senses are suppressed, the HMD 1 may display various information to alert the user to the moving object, as shown in FIG. 20 . In other words, the HMD 1 has a state in which content is played back and a state in which the user is alerted to the presence of the moving object, and when the presence of the moving object is detected, the HMD 1 alerts the user to the presence of the moving object by switching some or all of the content playback states depending on the urgency level of the detected moving object. Here, switching some of the states means, for example, displaying a message or interrupt information on a portion of the display where the content is displayed. Furthermore, switching all of the states means, for example, pausing or ending the display of the content, or displaying an alert on the entire display screen.

[0156] For example, as shown in (a), the HMD 1 may display a pop-up message 72 indicating the detection of a moving object in the VR content 71 that the user is viewing. (a) is an alert based on the above-mentioned example 3 in the alert method and example 11 in the transmitted information, but an alert based on another combination may also be performed.

[0157] Furthermore, the display mode may be changed depending on the urgency level of the moving object, the type of the moving object, the detection area of ​​the moving object, etc. For example, the HMD 1 may determine the size of the message to be displayed based on the urgency level, the type of the moving object, the detection area, etc., and display the pop-up message 72. The HMD 1 may also determine whether to light up, flash, or blink the pop-up message 72 based on the urgency, the type of the moving object, the detection area, etc., and display the pop-up message 72 in the determined mode. The HMD 1 may also determine the flashing or blinking cycle based on the urgency, the type of the moving object, the detection area, etc., and display the pop-up message 72 at the determined cycle. The HMD 1 may also determine the color of the pop-up message 72 based on the urgency, the type of the moving object, the detection area, etc., and display the pop-up message 72 in the determined color.

[0158] As shown in (b), the transparency of the pop-up message 72 may be taken into consideration. That is, the HMD 1 may set the transparency based on the urgency level, the type of moving object, the detection area, etc., and display the pop-up message 72. Here, the lower the transparency, the more difficult it becomes to see the image of the VR content 71 in the portion overlapping the pop-up message 72.

[0159] As shown in (c), the HMD 1 may superimpose an image of real space on the VR content 71 being viewed by the user 51, and display a real image 73 or a CG (Computer Graphics) image of the detected moving object. For example, if a camera is installed at the entrance and a moving object enters through the entrance, the HMD 1 may acquire image data acquired by the camera and superimpose the moving object on an image based on the image data. (c) shows an alert based on Example 5 of the above-mentioned alert method and Examples 1 and 11 of the transmitted information, but alerts based on other combinations may also be performed.

[0160] The HMD 1 may also change the ratio of the VR image and display it in the empty display area. For example, the HMD 1 may display an image of real space and a real image or CG image of the detected moving object in this area. The HMD 1 may set the ratio of the VR image depending on the urgency, the type of moving object, the detection area, etc. For example, as shown in (d), the HMD 1 may form a message display area within the VR content 71 and display a message in that area.

[0161] For example, as shown in (e), the HMD 1 may call attention to the direction of the moving object by moving a pop-up message 72 in the direction in which the moving object is detected in real space. The HMD 1 may also call attention to the direction of the moving object, the distance to the moving object, etc. by displaying the direction of the moving object, the distance to the moving object, etc. The HMD 1 may also display a real image or a CG image of the moving object when the user faces the direction in which the moving object is detected.

[0162] For example, as shown in (f), the HMD 1 may display a map 74 based on map data in the VR content 71. (f) is an alert based on the above-mentioned example 3 of the alert method and example 8 of the examples of communication information, but an alert based on other combinations may also be performed.

[0163] Here, in order to present the positional relationship between the HMD 1 itself and the detected moving object, the map 74 may include, for example, information indicating the position of the user 51 in addition to information on the position of the moving object. The HMD 1 may indicate the position of the moving object and the position of the user 51 on the map data using, for example, a symbol, a mark, a sign, or the like. Furthermore, if the floor on which the user 51 is located is different from the floor on which the moving object is located, the HMD 1 may display the map 74 of the floor on which the moving object is located and omit displaying the map 74 of the floor on which the user 51 is located.

[0164] As another display example, the HMD 1 may display a graphic with a radar chart structure, in which a reference point (center point) corresponds to the position of the user 51, each vertex corresponds to a respective direction, and the distance from the reference point to the vertex indicates the distance from the user 51 to the moving object. In this case, the user 51 can easily grasp the relative positional relationship with the moving object.

[0165] Although the embodiments have been described above, the present invention is not limited to the above-described embodiments and includes various modifications and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, for example, other configurations may be added, deleted, or replaced with part of the configuration of the embodiment.

[0166] While the HMD 1 has been described as an example of an information display device, the present invention is not limited thereto, and may be applied to other information display devices that temporarily or over time suppress vision and hearing. For example, hearing is suppressed even when using AR glasses, which may result in the user failing to notice the presence of a suspicious person, family member, or other person. Therefore, the information display device may be, for example, AR glasses. Furthermore, since vision and hearing are similarly suppressed when using MR glasses, the information display device may be, for example, an HMD that uses MR or MR glasses. The information display device may also be an HMD to which a smartphone, tablet, or the like can be attached. Alternatively, hearing is similarly suppressed when using smart glasses (wearable devices) that only have an audio function and no video display function, which may result in the user failing to notice the presence of a suspicious person, family member, or other person. Even when immersed in playing a game or watching a video on a smartphone or tablet, vision is suppressed, and hearing is also suppressed when wearing headphones, etc. Therefore, the information display device may be a smartphone or smart glasses without a video display function that transmits information via audio.

[0167] The user may also be alerted by sound or vibration. For example, the user may be alerted by sound output from the speaker 122. The user may also be alerted by a vibration pattern of the vibrator 174.

[0168] According to the above, the following information processing system is provided as an example. This information processing system includes a computer device (external device, external server, etc.) that analyzes the CSI of Wi-Fi radio waves, and an information display device that can communicate with the computer device and is worn on the user's head. When the information display device acquires information that a moving object has been detected from the computer device, it alerts the user.

[0169] 1 Head-mounted display 101 Processor 180 Network sensing

Claims

1. An information display device worn on a user's head, comprising: a processor; and an antenna for receiving radio waves propagating in multiple waves, wherein the processor analyzes the multiple wave propagation information of the radio waves received by the antenna, and when the presence of a moving object is detected from the analysis results of the multiple wave propagation information, the information display device issues a warning to the user.

2. An information display device according to claim 1, characterized in that the processor issues a warning to the user when the presence of a moving object is detected within a detection target area previously set by the user.

3. An information display device according to claim 2, characterized in that the processor does not issue a warning about moving objects present within the detection target area when the detection target area is set by the user.

4. An information display device according to claim 1, characterized in that the processor refers to an urgency level relating to the degree of warning and determines the manner of warning regarding the detected moving object.

5. An information display device according to claim 4, wherein the urgency level can be set for a combination of a detection target classification and a detection area classification.

6. An information display device according to claim 1, characterized in that the processor does not issue a warning about a moving object registered by the user.

7. An information display device according to claim 1, wherein the processor issues a warning in a manner that corresponds to the detection area of ​​the moving object.

8. An information display device to be worn on a user's head, comprising: a processor; and an antenna for receiving radio waves propagating in multiple waves, wherein the control states of the processor include a determination state for determining the presence of a moving object using multi-wave propagation information of the radio waves received by the antenna; and a first state for alerting the user to the presence of the moving object.

9. An information display device according to claim 8, wherein the control state of said processor further includes a state for analyzing multi-wave propagation information of radio waves received by said antenna.

10. An information display device according to claim 8, wherein the multipath wave propagation information includes information analyzed by an external device.

11. The information display device according to claim 8 further comprises a camera, and in the determination state, determines the presence of a moving object using multi-wave propagation information of radio waves received by the antenna and the captured image acquired by the camera.

12. An information display device according to claim 8, wherein the control state of the processor further includes a second state in which content is played back, and wherein the information display device switches between the first state and the second state.

13. The information display device described in claim 8 further comprises a camera and a display, and in the first state, when the display does not display a photographed image of the surroundings of the information display device acquired by the camera, the information display device is characterized in that it alerts the user to the presence of the moving object.

14. The information display device described in claim 8 further comprises a speaker, and in the first state, when the speaker outputs a sound at a volume equal to or greater than a predetermined threshold, the information display device alerts the user to the presence of the moving object.

15. An information processing system comprising: a computer device; and an information display device that can communicate with the computer device and is worn on a user's head, wherein the computer device analyzes received information of radio waves that propagate in multiple waves, and when the presence of a moving object is detected by analyzing the received information, transmits detection information to the information display device, and when the information display device obtains the detection information from the computer device, it issues a warning to the user.

Images