Environmental status notification device, environmental status notification method, and program

Abstract

[Problem] To provide a technology advantageous for improving user safety. [Solution] An environment state notification device is provided with: an environment information acquisition unit that is attached to a user and that acquires environment information data indicating information about the environment around the user; an environment state recognition unit that recognizes a state of the environment around the user on the basis of the environment information data; and a notification unit that notifies the user in accordance with the state of the environment around the user recognized by the environment state recognition unit.

Description

Technical Field

[0001] The present disclosure relates to an environmental condition notification device, an environmental condition notification method, and a program.

Background Art

[0002] In recent years, various wearable devices that can be worn by users have been proposed.

[0003] Patent Document 1 discloses a wearable terminal equipped with a camera. In this wearable terminal, the angle of view of the camera is controlled according to the user's behavior state.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] With the increasing attention to the automatic driving of vehicles, the safety driving support devices mounted on vehicles have also developed by leaps and bounds. On the other hand, the safety driving support devices are still in the development stage, and further improvement of the safety functions is required. In addition, a considerable number of vehicles without safety driving support devices are still running on public roads. Therefore, pedestrians still need to walk while paying sufficient attention to vehicles.

[0006] In addition, pedestrians need to pay sufficient attention not only to vehicles but also to other entities such as suspicious persons to ensure safety. Furthermore, avoiding approaching and entering the areas where pedestrians need to be cautious in advance is very effective from the perspective of improving pedestrian safety.

[0007] The present disclosure has been made in view of the above circumstances, and provides a technology that is advantageous for improving user safety. [Means for solving the problem]

[0008] One aspect of this disclosure relates to an environmental status notification device comprising: an environmental information acquisition unit attached to a user and acquiring environmental information data indicating information about the user's surrounding environment; an environmental status recognition unit that recognizes the state of the surrounding environment for the user based on the environmental information data; and a notification unit that provides a notification to the user according to the state of the surrounding environment for the user recognized by the environmental status recognition unit.

[0009] The environmental information acquisition unit may include a wave sensor that receives at least one of electromagnetic waves and sound waves from the user's surrounding environment to acquire environmental information data.

[0010] The wave sensor may acquire images of the user's surrounding environment.

[0011] The environmental information acquisition unit includes an electromagnetic wave emission unit that emits detected electromagnetic waves or detected sound waves, and the wave sensor may receive the reflected detected electromagnetic waves or detected sound waves.

[0012] The environmental information acquisition unit may include a location information acquisition unit that acquires the user's location information, and a map information matching unit that acquires environmental information data based on the location information and map information.

[0013] The location information acquisition unit may acquire the user's location information from a satellite positioning system.

[0014] The environmental status notification device includes a communication unit for data communication, and the environmental status recognition unit may recognize the state of the surrounding environment for the user based on environmental information data acquired by the environmental information acquisition unit and environmental information data sent from the first external device via the communication unit.

[0015] The environmental status notification device includes a user status information acquisition unit that acquires user status data indicating the user's status, and the environmental status recognition unit may recognize the state of the surrounding environment for the user based on the environmental information data and the user status data.

[0016] The user state information acquisition unit may include a sensor that detects at least one of the user's movement speed and acceleration.

[0017] The environmental state recognition unit may predict the user's movement path based on user state data and recognize the state of the surrounding environment relative to the user based on the predicted user movement path.

[0018] The environmental state recognition unit may acquire a dynamic map of the user's surrounding environment and recognize the state of the surrounding environment for the user based on the environmental information data and the dynamic map.

[0019] The notification unit may be attached to the user.

[0020] The notification unit may provide notifications to the user through at least one of the following senses: hearing, sight, and touch.

[0021] The notification unit may, in accordance with the state of the surrounding environment for the user as recognized by the environmental state recognition unit, notify a second external device that is present in the user's surrounding environment but is not attached to the user.

[0022] The second external device may be a vehicle.

[0023] The environmental status notification device includes a communication unit for data communication, and the communication unit may send data indicating the state of the surrounding environment for the user, as recognized by the environmental status recognition unit, to a third external device.

[0024] The environmental condition notification device includes an attachment state detection unit that detects the attachment state of the environmental information acquisition unit for the user, and the notification unit may notify the user according to the detection result of the attachment state detection unit.

[0025] Another aspect of the present disclosure relates to an environmental condition notification method including: a step of obtaining environmental information data indicating information on the user's surrounding environment by an environmental information acquisition unit attached to the user; a step of an environmental condition recognition unit recognizing the state of the surrounding environment for the user based on the environmental information data; and a step of a notification unit notifying the user according to the state of the surrounding environment for the user recognized by the environmental condition recognition unit.

[0026] Another aspect of the present disclosure relates to a program for causing a computer to execute: a procedure of obtaining environmental information data indicating information on the user's surrounding environment by an environmental information acquisition unit attached to the user; a procedure of an environmental condition recognition unit recognizing the state of the surrounding environment for the user based on the environmental information data; and a procedure of a notification unit notifying the user according to the state of the surrounding environment for the user recognized by the environmental condition recognition unit.

Brief Description of Drawings

[0027] [Figure 1] FIG. 1 is a diagram showing an example of a wearable device functioning as an environmental condition notification device. [Figure 2] FIG. 2 is a diagram showing an example of the functional configuration of the wearable device. [Figure 3] FIG. 3 is a diagram showing an example of the functional configuration of the environmental condition notification device. [Figure 4] FIG. 4 is a diagram showing another example of the functional configuration of the environmental condition notification device. [Figure 5] FIG. 5 is a diagram showing another example of the functional configuration of the environmental condition notification device. [Figure 6] FIG. 6 is a diagram showing another example of the functional configuration of the wearable device. [Figure 7] FIG. 7 is a diagram showing another example of the functional configuration of the wearable device. [Figure 8] Figure 8 is a flowchart showing an example of an environmental condition notification method. [Figure 9] Figure 9 shows another example of the functional configuration of an environmental status notification device. [Figure 10] Figure 10 shows another example of the functional configuration of an environmental status notification device. [Figure 11] Figure 11 shows another example of the functional configuration of an environmental status notification device. [Figure 12] Figure 12 shows another example of the functional configuration of an environmental status notification device. [Figure 13] Figure 13 shows another example of the functional configuration of an environmental status notification device. [Modes for carrying out the invention]

[0028] Figure 1 shows an example of a wearable device 11 that functions as an environmental condition notification device 10. Figure 2 shows an example of the functional configuration of the wearable device 11.

[0029] The wearable device 11 shown in Figure 1 is attached to the user's clothing.

[0030] The wearable device 11 is not limited to being attached to clothing, and can be attached to user 1 in any manner. Typically, wearable device 11 can be worn by user 1 in the form of a wristwatch, neck strap, glasses, earphones (including headphones), or ring.

[0031] The wearable device 11 shown in Figure 2 comprises a mounting section 21, a control section 22, a memory 23, an environmental information acquisition section 24, an environmental state recognition section 25, and a notification section 26.

[0032] The mounting portion 21 is used to attach the wearable device 11 to the user 1. The specific structure of the mounting portion 21 is not limited, nor is the manner in which the wearable device 11 is attached to the user 1.

[0033] As shown in Figure 1, when the wearable device 11 is attached to the user 1's clothing, the attachment part 21 is equipped with a magnet, and the wearable device 11 can be attached to the clothing detachably by using magnetic force to clamp onto the clothing. However, the attachment part 21 may also be in the form of a clip or other to attach the wearable device 11 to the user 1's clothing.

[0034] In the case of a wristwatch-type wearable device 11, the attachment part 21 can be made by a belt (wristband).

[0035] Thus, the mounting portion 21 has a configuration and shape suitable for the mounting method of the wearable device 11 to the user 1.

[0036] The control unit 22 controls each functional component of the wearable device 11. The control unit 22 typically includes various processors such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit).

[0037] The specific configuration of the control unit 22 is not limited. The control unit 22 may be composed of a single device, and this control unit 22 may integrally control multiple functional components included in the wearable device 11. Alternatively, the control unit 22 may be composed of multiple devices. For example, each functional component of the wearable device 11 may have a corresponding control device (control unit 22) built in.

[0038] Memory 23 stores various types of data (including programs). Memory 23 includes at least one of a non-volatile storage medium and a volatile storage medium. Memory 23 can be configured, for example, as EEPROM (Electrically Erasable Programmable Read Only Memory) or RAM (Random Access Memory). Memory 23 can also be configured as a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device.

[0039] Each functional component (including the control unit 22) of the wearable device 11 may read various data from the memory 23, store new data in the memory 23, or update data stored in the memory 23.

[0040] The environmental information acquisition unit 24 is attached to user 1 and acquires environmental information data that shows information about user 1's surrounding environment. The specific configuration of the environmental information acquisition unit 24 and the specific information included in the environmental information data are not limited.

[0041] Typically, the environmental information acquisition unit 24 includes a wave sensor that operates under the control of the control unit 22. The wave sensor receives at least one of electromagnetic waves and sound waves (e.g., ultrasound) from the environment surrounding the user 1 and acquires environmental information data.

[0042] The electromagnetic waves referred to here may include light such as visible light and radio waves such as millimeter waves. Therefore, the wave sensor may be configured as an image sensor that acquires images (still images and / or videos) of the environment surrounding user 1, or as a sensor that receives / detects radio waves or light reflected by objects around user 1.

[0043] Specifically, the environmental information acquisition unit 24 may include a camera (imaging device), LiDAR (Light Detection and Ranging), ToF (Time of Flight), millimeter-wave radar, and / or ultrasonic sensors. The camera can employ various imaging methods, such as a stereo camera, a monocular camera, and an infrared camera.

[0044] The environmental information acquisition unit 24 may have a sensor fusion unit (not shown). This sensor fusion unit performs sensor fusion processing to obtain new information from a combination of multiple different types of sensor data (e.g., image data and radar detection data). Sensor fusion processing may include, for example, a process of integrating, fusing, or combining multiple types of sensor data.

[0045] The environmental state recognition unit 25 recognizes the state of the surrounding environment for user 1 based on the environmental information data acquired by the environmental information acquisition unit 24.

[0046] The term "the state of the surrounding environment for User 1" as used here may include the state of the surrounding environment's safety for User 1. For example, whether or not a moving object such as a vehicle or bicycle is approaching User 1, and whether or not User 1's current location or predicted future location is a location requiring caution, may be included in "the state of the surrounding environment for User 1." In addition, whether or not a moving object such as a vehicle or a living being such as a person located near User 1 is exhibiting suspicious behavior may also be included in "the state of the surrounding environment for User 1."

[0047] Furthermore, "the state of the surrounding environment for User 1" may include the state of the surrounding environment that could have effects on User 1 other than safety.

[0048] When the environmental state recognition unit 25 analyzes environmental information data, the specific analysis method used by the environmental state recognition unit 25 is not limited.

[0049] For example, the environmental state recognition unit 25 is equipped with AI (Artificial Intelligence) and can analyze environmental information data using trained data obtained through deep learning. Here, the trained data may include trained parameters and an inference program incorporating those trained parameters. The trained parameters and inference program are obtained, for example, as a result of learning using raw data and / or a training dataset obtained from raw data.

[0050] The environmental state recognition unit 25 in this example may use trained data in which "environmental information data and / or data derived from environmental information data" are associated with "data relating to the state of the surrounding environment for user 1". By applying "environmental information data and / or data derived from environmental information data" as input to the trained data, the environmental state recognition unit 25 can obtain "data relating to the state of the surrounding environment for user 1" as output.

[0051] The notification unit 26 sends a notification to user 1 according to the "state of the surrounding environment for user 1" recognized by the environmental state recognition unit 25.

[0052] The method by which the notification unit 26 notifies user 1 is not limited. The notification unit 26 attached to user 1 can typically notify user 1 through user 1's senses (e.g., at least one of hearing, sight, and touch).

[0053] For example, the notification unit 26 can notify user 1 of the "state of the surrounding environment to user 1" through audio or music provided to user 1 via a speaker or earphones. In particular, notifications via user 1's hearing enable quick and intuitive notification to user 1 and do not impair user 1's visual information.

[0054] Recently, various audio technologies, such as 3D audio technology, have been used to provide voice and music to the user 1. The notification unit 26 can bring about new beneficial effects, such as the recognition of the location and direction of objects requiring attention and the induction of reflexive safety actions by the user 1, by providing notifications to the user 1 based on such various audio technologies. Examples of the use of 3D audio technology in the notification unit 26 will be described later.

[0055] Furthermore, the notification unit 26 may notify user 1 of the "state of the surrounding environment for user 1" through images provided to user 1 via glasses (e.g., AR (Augmented Reality) glasses). For example, the notification unit 26 can display images on the glasses that show the state of the surrounding environment for user 1. The notification unit 26 may also turn off the image display on the glasses to allow user 1 to see the surrounding environment if an object requiring attention is approaching user 1 or user 1 is approaching an area requiring attention.

[0056] Furthermore, the notification unit 26 can notify user 1 of the "state of the surrounding environment to user 1" through tactile changes such as vibration provided to user 1 via a haptics device (e.g., a vibration device or a pressure variable device).

[0057] Figure 3 shows an example of the functional configuration of the environmental status notification device 10.

[0058] In the example shown in Figure 3, the environmental information acquisition unit 24 includes an imaging device 31 that acquires images of the environment surrounding user 1, and has an image sensor (such as a CMOS (Complementary Metal Oxide Semiconductor) sensor) as a wave sensor.

[0059] The imaging device 31 transmits captured images of the surrounding environment as environmental information data D11 to the environmental state recognition unit 25. The environmental state recognition unit 25 is equipped with an image analysis device 32 that analyzes the captured images sent from the imaging device 31.

[0060] The image analysis device 32 recognizes the "state of the surrounding environment for user 1" by analyzing the captured image. For example, the image analysis device 32 can recognize whether the captured image includes moving objects such as vehicles approaching user 1, unsafe areas such as roadways, and / or other objects that pedestrians should pay attention to.

[0061] The specific image analysis method performed by the image analysis device 32 is not limited. For example, the image analysis device 32 may recognize the types of objects surrounding user 1 by performing recognition processing such as semantic segmentation.

[0062] Furthermore, the image analysis device 32 may use techniques such as SLAM (Simultaneous Localization and Mapping) based on the image data acquired by the imaging device 31 to estimate its own position and create an environmental map (local map).

[0063] Local maps can be created, for example, as 3D high-precision maps and occupancy grid maps. 3D high-precision maps can be created, for example, as point cloud maps (described later). Occupancy grid maps can be created by dividing the 3D or 2D space around user 1 into grids of a predetermined size, and showing the occupancy status of objects on a grid-by-grid basis. The occupancy status of objects here is indicated, for example, by the presence or absence of an object or its probability of existence.

[0064] The image analysis device 32 then determines, based on the analysis results of the captured image, whether or not it is necessary to alert user 1.

[0065] For example, if the captured image includes a moving object approaching user 1, an unsafe area, and / or other objects that pedestrians should pay attention to, the image analysis device 32 may determine that it is necessary to alert user 1. On the other hand, if the captured image does not include objects that pedestrians should pay attention to, the image analysis device 32 may determine that it is not necessary to alert user 1.

[0066] The image analysis device 32 transmits a notification instruction signal D12 to the notification unit 26 depending on whether or not it is necessary to alert user 1.

[0067] For example, if the image analysis device 32 determines that it is necessary to alert user 1, it transmits a notification instruction signal D12 indicating this determination to the notification unit 26. On the other hand, if the image analysis device 32 determines that it is not necessary to alert user 1, it may transmit a notification instruction signal D12 indicating this determination to the notification unit 26, or it may not transmit the notification instruction signal D12 itself to the notification unit 26.

[0068] Thus, the image analysis device 32 may transmit the notification instruction signal D12 to the notification unit 26 whether or not it is necessary to alert user 1. Alternatively, the image analysis device 32 may transmit the notification instruction signal D12 to the notification unit 26 only when it is necessary to alert user 1.

[0069] The notification unit 26, based on the notification instruction signal D12 sent from the image analysis device 32, provides notifications to user 1 according to the state of the surrounding environment.

[0070] The notification unit 26 in this example is equipped with an audio device, a vibration device, and / or a video device.

[0071] For example, when the notification unit 26 receives a notification instruction signal D12 that is sent when it is necessary to draw the attention of user 1, it issues a notification (voice, vibration, and / or video) to user 1 to draw the attention of user 1. On the other hand, when the notification unit 26 receives a notification instruction signal D12 that is sent when it is not necessary to draw the attention of user 1, it does not have to issue a notification to user 1, or it may issue a notification to user 1 indicating that it is not necessary to draw the attention of user 1. Furthermore, the notification unit 26 does not have to issue a notification to user 1 while it is not receiving a notification instruction signal D12 from the image analysis device 32.

[0072] As described above, the environmental condition notification device 10 in this example can notify the user 1 of the presence of objects requiring attention in the surrounding environment based on the captured image data acquired by the imaging device 31 included in the wearable device 11. Based on such notifications, the user 1 can understand the state of the surrounding environment and take appropriate action as needed.

[0073] Thus, the environmental condition notification device 10 in this example is configured as a pedestrian safety protection device that improves the safety of pedestrians (user 1).

[0074] Figure 4 shows another example of the functional configuration of the environmental status notification device 10.

[0075] In the example shown in Figure 4, the environmental information acquisition unit 24 includes an emitter (emitting unit) 35 that emits detected electromagnetic waves or detected sound waves, and a receiver (wave sensor) 36 that receives the reflected detected electromagnetic waves or detected sound waves.

[0076] The detected electromagnetic wave can be either light or radio waves.

[0077] For example, when the emitter 35 and receiver 36 are configured as LiDAR or ToF, the emitter 35 emits light as a detection electromagnetic wave toward the surrounding environment, and the receiver 36 receives the light reflected from the surrounding environment.

[0078] On the other hand, when the emitter 35 and receiver 36 are configured as a millimeter-wave radar, the emitter 35 emits millimeter-wave radio waves as detection electromagnetic waves toward the surrounding environment, and the receiver 36 receives the radio waves reflected in the surrounding environment.

[0079] The environmental information acquisition unit 24 transmits the surrounding environment information, based on the reflected electromagnetic waves or reflected sound waves received by the receiver 36, as environmental information data D11 to the environmental state recognition unit 25.

[0080] The environmental state recognition unit 25 is equipped with a received data analysis device 37. The received data analysis device 37 recognizes the "state of the surrounding environment for user 1" by analyzing the surrounding environment information based on the detection results (reflected electromagnetic waves or reflected sound waves) from the receiver 36 sent from the environmental information acquisition unit 24. Based on the analysis results of the surrounding environment information, the received data analysis device 37 decides whether or not it is necessary to alert user 1.

[0081] Furthermore, the received data analysis device 37 is capable of recognizing the same objects as the image analysis device 32 (see Figure 3) described above, and can determine whether or not it is necessary to alert user 1 using the same criteria as the image analysis device 32. For example, the received data analysis device 37 may perform self-position estimation and environmental map creation using technologies such as SLAM based on the detection results of the receiver 36.

[0082] The receiving data analysis device 37 transmits a notification instruction signal D12 to the notification unit 26 depending on whether or not it is necessary to alert user 1. Based on the notification instruction signal D12 sent from the receiving data analysis device 37, the notification unit 26 provides user 1 with a notification corresponding to the state of the surrounding environment.

[0083] As described above, the environmental status notification device 10 in this example can notify the user 1 of the presence of an object requiring attention in the surrounding environment based on the surrounding environment information acquired by the emitter 35 and receiver 36.

[0084] Figure 5 shows another example of the functional configuration of the environmental status notification device 10.

[0085] In the example shown in Figure 5, the environmental information acquisition unit 24 includes a location information acquisition unit 40 and a map information matching unit 41.

[0086] The location information acquisition unit 40 acquires the location information of user 1. The location information acquisition unit 40 can acquire the location information of user 1 (more precisely, the wearable device 11) from, for example, a GNSS (Global Navigation Satellite System) 42. The location information acquisition unit 40 may also acquire the location information of user 1 by means other than GNSS 42, for example, by obtaining the location information of user 1 via a beacon.

[0087] The map information matching unit 41 acquires environmental information data D11 based on the location information of user 1 acquired by the location information acquisition unit 40 and the map information. In this example, the map information matching unit 41 acquires map information from the map information providing device 43.

[0088] The map information providing device 43 may be provided as part of the wearable device 11, or it may be provided separately from the wearable device 11. For example, the map information matching unit 41 may read map information from the memory 23 (see Figure 2) of the wearable device 11 acting as the map information providing device 43. Alternatively, map information may be sent to the map information matching unit 41 from a map information providing device 43 (e.g., a server) connected to the wearable device 11 via a network such as the Internet.

[0089] The map information matching unit 41 may, for example, acquire map information for a limited range determined based on the location information of user 1 acquired by the location information acquisition unit 40 from the map information providing device 43. The map information matching unit 41 may also send information necessary for acquiring such limited range map information (for example, the location information of user 1) to the map information providing device 43. The map information providing device 43 may transmit the map information for the limited range to the map information matching unit 41 based on the information sent from the map information matching unit 41.

[0090] The map information acquired by the map information matching unit 41 may be a three-dimensional high-precision map, or it may be a high-precision map such as a dynamic map, point cloud map, or vector map. A dynamic map is a map that includes four layers, such as dynamic information, semi-dynamic information, semi-static information, and static information, as described later. A point cloud map is a map composed of point clouds (point cloud data). A vector map is a map that associates traffic information, such as the location of lanes and traffic lights, with a point cloud map.

[0091] The point cloud map and vector map may be provided by an external device such as a server, or they may be created by the wearable device 11 (for example, the environmental information acquisition unit 24 or the environmental state recognition unit 25) based on environmental information data acquired by the environmental information acquisition unit 24.

[0092] The environmental information acquisition unit 24 transmits the environmental information data D11 acquired in this manner (for example, information data in which the location of user 1 is reflected on the map information) to the environmental state recognition unit 25.

[0093] The environmental state recognition unit 25 is equipped with a location information analysis device 44 that analyzes the environmental information data D11 sent from the environmental information acquisition unit 24.

[0094] The location information analysis device 44 recognizes the "state of the surrounding environment for user 1" by analyzing environmental information data D11 based on the location information and map information of user 1. As an example, the recognition of whether or not user 1 is located in an area requiring attention (e.g., a roadway or near a roadway) may be included in the recognition of the "state of the surrounding environment for user 1" as referred to here.

[0095] The location information analysis device 44 then determines, based on the analysis results of the environmental information data D11, whether or not it is necessary to alert user 1.

[0096] The location information analysis device 44 may determine, for example, that it is necessary to alert user 1 if user 1 is located in an area requiring attention. On the other hand, if user 1 is not located in an area requiring attention, the location information analysis device 44 may determine that it is not necessary to alert user 1.

[0097] The location information analysis device 44 transmits a notification instruction signal D12 to the notification unit 26 depending on whether or not it is necessary to alert user 1.

[0098] The notification unit 26 provides user 1 with notifications corresponding to the state of the surrounding environment based on the notification instruction signal D12 sent from the received data analysis device 37.

[0099] As described above, the environmental status notification device 10 in this example can notify user 1 of the presence of objects requiring attention in the surrounding environment based on user 1's location information and map information.

[0100] Figure 6 shows another example of the functional configuration of the wearable device 11.

[0101] The wearable device 11 shown in Figure 6 includes, in addition to the mounting unit 21, control unit 22, memory 23, environmental information acquisition unit 24, environmental state recognition unit 25, and notification unit 26 described above, a mounting state detection unit 27.

[0102] The mounting status detection unit 27 detects the mounting status of the environmental information acquisition unit 24 for user 1 under the control of the control unit 22. That is, the mounting status detection unit 27 can detect the mounting status of the environmental information acquisition unit 24 based on the environmental information data acquired by the environmental information acquisition unit 24 and / or the state of the surrounding environment for user 1 recognized by the environmental state recognition unit 25.

[0103] If the environmental information data acquired by the environmental information acquisition unit 24 falls within the data range expected under normal conditions, the mounting status detection unit 27 may detect that the environmental information acquisition unit 24 is properly installed for user 1. On the other hand, if the environmental information data acquired by the environmental information acquisition unit 24 does not fall within the data range expected under normal conditions, the mounting status detection unit 27 may detect that the environmental information acquisition unit 24 is not properly installed for user 1.

[0104] Similarly, if the state of the surrounding environment for user 1, as recognized by the environmental state recognition unit 25, falls within the data range expected under normal conditions, the mounting state detection unit 27 may detect that the environmental information acquisition unit 24 is properly installed for user 1. On the other hand, if the state of the surrounding environment for user 1, as recognized by the environmental state recognition unit 25, falls outside the data range expected under normal conditions, the mounting state detection unit 27 may detect that the environmental information acquisition unit 24 is not properly installed for user 1.

[0105] The mounting condition detection unit 27 may further detect the type (category) of mounting defects based on environmental information data acquired by the environmental information acquisition unit 24 and / or the state of the surrounding environment for user 1 recognized by the environmental state recognition unit 25.

[0106] For example, the mounting status detection unit 27 may detect a defect in the installation location of the environmental information acquisition unit 24. That is, if the data obtained from the environmental information acquisition unit 24 and the environmental status recognition unit 25 indicates that the installation location of the environmental information acquisition unit 24 is unsuitable for acquiring environmental information data, the mounting status detection unit 27 may detect a defect in the installation location of the environmental information acquisition unit 24.

[0107] For example, if the data obtained from the environmental information acquisition unit 24 and the environmental state recognition unit 25 indicates that the environmental information acquisition unit 24 (e.g., a sensor) is obstructed and therefore unsuitable for acquiring environmental information data, a defect in the installation location of the environmental information acquisition unit 24 may be detected.

[0108] As another example, if the data obtained from the environmental information acquisition unit 24 and the environmental state recognition unit 25 indicates that the direction (orientation) of detection by the environmental information acquisition unit 24 (e.g., a sensor) is inappropriate, a defect in the installation location of the environmental information acquisition unit 24 may be detected. For example, if the environmental information acquisition unit 24 is supposed to detect the surrounding environment in front of user 1, but is actually detecting the surrounding environment behind user 1, a defect in the installation location of the environmental information acquisition unit 24 may be detected.

[0109] Whether the detection direction of the environmental information acquisition unit 24 is inappropriate may be detected, for example, based on information indicating the progress of user 1 derived from the environmental information data D11 (images, etc.) acquired by the environmental information acquisition unit 24.

[0110] The notification unit 26 provides a notification to user 1 according to the detection result of the mounting status detection unit 27.

[0111] In other words, if the mounting status detection unit 27 detects that the mounting status of the environmental information acquisition unit 24 is inappropriate, the notification unit 26 issues a notification to user 1 indicating that the mounting status of the environmental information acquisition unit 24 is inappropriate. On the other hand, if the mounting status detection unit 27 detects that the mounting status of the environmental information acquisition unit 24 is appropriate, the notification unit 26 may issue a notification to user 1 indicating that the mounting status of the environmental information acquisition unit 24 is appropriate, or it may choose not to issue a notification at all.

[0112] As described above, the environmental status notification device 10 in this example can notify user 1 of the installation status of the environmental information acquisition unit 24, thereby encouraging user 1 to properly install the environmental information acquisition unit 24.

[0113] Figure 7 shows another example of the functional configuration of the wearable device 11.

[0114] The wearable device 11 shown in Figure 7 includes, in addition to the mounting unit 21, control unit 22, memory 23, environmental information acquisition unit 24, environmental state recognition unit 25, notification unit 26, and mounting state detection unit 27 described above, a user state information acquisition unit 28.

[0115] The user status information acquisition unit 28 acquires user status data that indicates information about the status of user 1.

[0116] The specific information included in user state data is not limited, but it includes information about user 1's state that may affect the state of the surrounding environment for user 1. For example, information about user 1's movement speed and / or acceleration may be included in user state data.

[0117] Therefore, the user state information acquisition unit 28 may include a sensor that detects at least one of the speed and acceleration of user 1's movement. The sensors included in the user state information acquisition unit 28 are not limited. For example, the user state information acquisition unit 28 may include a speed sensor, an acceleration sensor, an angular velocity sensor (gyro sensor), and an inertial measurement unit (IMU) that integrates them. The user state information acquisition unit 28 may also include a functional component that acquires other information indicating user 1's movement (see, for example, the position information acquisition unit 40 and the map information matching unit 41 shown in Figure 5).

[0118] The environmental state recognition unit 25 recognizes the state of the surrounding environment for user 1 based on the environmental information data acquired by the environmental information acquisition unit 24 and the user state data acquired by the user state information acquisition unit 28.

[0119] For example, the environmental state recognition unit 25 may predict the user's movement path based on user state data and recognize the state of the surrounding environment for user 1 based on the predicted movement path of user 1 and environmental information data. For instance, the environmental state recognition unit 25 can recognize whether the predicted movement path of user 1 approaches or passes through an area or object that user 1 requires attention.

[0120] Thus, the environmental state recognition unit 25 can recognize not only whether the current situation of user 1 could threaten user 1's safety, but also whether the predicted future situation of user 1 could threaten user 1's safety.

[0121] The notification unit 26 sends a notification to user 1 according to the "state of the surrounding environment for user 1" recognized by the environmental state recognition unit 25.

[0122] As described above, the environmental status notification device 10 in this example can notify user 1 of the presence of objects in the surrounding environment that require attention, both currently and / or in the future, based on user status data.

[0123] Figure 8 is a flowchart showing an example of an environmental condition notification method.

[0124] In the example shown in Figure 8, first, the environmental information acquisition unit 24 starts acquiring environmental information data that shows information about the surrounding environment of user 1, and the user status information acquisition unit 28 starts acquiring user status data that shows the status of user 1 (S11 in Figure 8).

[0125] The mounting status detection unit 27 then detects the mounting status of the wearable device 11, including the environmental information acquisition unit 24 (S12). Specifically, the mounting status detection unit 27 determines whether the installation position of the wearable device 11 (especially the environmental information acquisition unit 24) is appropriate (S13).

[0126] If an abnormality is detected in the mounting state of the wearable device 11 including the environmental information acquisition unit 24 (Y in S13), a warning is issued from the notification unit 26 to inform the user 1 of the abnormality in the mounting state of the wearable device 11 including the environmental information acquisition unit 24 (S14).

[0127] The specific method of issuing a warning from the notification unit 26 is not limited. For example, the notification unit 26 can issue a warning to user 1 through sound, video and / or vibration, similar to the notification to user 1 described above.

[0128] This allows user 1 to notice inappropriate situations such as the wearable device 11 (especially the environmental information acquisition unit 24) being hidden by the body or clothing, the environmental information acquisition unit 24 being oriented incorrectly, or the wearable device 11 being worn in an inappropriate position.

[0129] After the notification unit 26 issues a warning regarding an abnormality in the mounting status of the wearable device 11, including the environmental information acquisition unit 24, steps S12 and S13 described above are repeated.

[0130] On the other hand, if no abnormality is detected in the mounting state of the wearable device 11 including the environmental information acquisition unit 24 (N in S13), the environmental status notification device 10 proceeds to the next step (i.e., S15).

[0131] In this way, user 1 is prompted to properly attach the wearable device 11, and the environmental status notification device 10 will not proceed to the next step S15 until the wearable device 11 is properly attached to user 1.

[0132] In step S15, the environmental state recognition unit 25 recognizes the state of the surrounding environment for user 1 based on the environmental information data and user state data.

[0133] In this example, the environmental state recognition unit 25 recognizes the state of the environment surrounding user 1 and predicts user 1's path based on environmental information data and user state data. More specifically, information regarding the movement of a moving object such as a vehicle (including location information) and information regarding areas in the surrounding environment that require attention are acquired as environmental information data. However, other arbitrary information may also be included in the environmental information data.

[0134] The environmental state recognition unit 25 then determines whether or not the moving object is approaching the user 1's predicted path (for example, the user 1's predicted position) (S16).

[0135] If it is determined that a moving object is approaching User 1's predicted path (User 1's predicted position) (Y in S16), the notification unit 26 issues a notification to User 1 warning of the approaching object (S17). This allows User 1 to recognize the approaching object and take appropriate action.

[0136] After the notification unit 26 issues a warning to user 1 about the approach of a moving object, in this example, the detection and determination of the attachment status of the wearable device 11 to user 1 (S12 and S13) is performed again. Alternatively, after the notification unit 26 issues a warning to user 1 about the approach of a moving object, the recognition of the surrounding environment and estimation of the user's predicted path (S15) may be performed again without the detection and determination of the attachment status of the wearable device 11.

[0137] On the other hand, if it is determined that the moving object will not approach the predicted path of user 1 (predicted position of user 1) (N in S16), the environmental state recognition unit 25 determines whether or not user 1 is approaching an area requiring attention (S18).

[0138] If it is determined that user 1 is approaching an area requiring attention (Y in S18), the notification unit 26 issues a notification to user 1 warning of user 1's approach to the area requiring attention (S19).

[0139] After the notification unit 26 issues a warning to user 1 about approaching an area requiring attention, the environmental status notification device 10 in this example returns to step S12 described above. However, the environmental status notification device 10 may return to step S15 instead of returning to step S12.

[0140] On the other hand, if it is determined that user 1 does not approach the area requiring attention (N in S18), no notification is sent from the notification unit 26 to user 1.

[0141] In this manner, the notification unit 26 notifies user 1 according to the state of the surrounding environment for user 1 as recognized by the environmental state recognition unit 25.

[0142] The above-described series of processes (S11-S19) may be repeated. Alternatively, some of the above-described series of processes (for example, S12-S19 or S15-S19) may be repeated.

[0143] Furthermore, some of the functional components included in the example of the wearable device 11 described above may be provided separately from the wearable device 11.

[0144] Figure 9 shows another example of the functional configuration of the environmental status notification device 10.

[0145] In the example shown in Figure 9, the wearable device 11 does not include the environmental state recognition unit 25, but includes the communication unit 29.

[0146] The communication unit 29 communicates data with the external communication unit 55 of the external device 51. In addition to the external communication unit 55, the external device 51 includes an environmental state recognition unit 25.

[0147] The method of communication between the communication unit 29 and the external communication unit 55 is not limited. Furthermore, multiple communication methods conforming to two or more communication standards may be used to communicate between the communication unit 29 and the external communication unit 55.

[0148] The communication unit 29 can communicate with the external communication unit 55 based on wireless communication methods such as 5G (fifth-generation mobile communication system), LTE (Long Term Evolution), and DSRC (Dedicated Short Range Communications). Examples of external networks with which the communication unit 29 communicates include the internet, cloud networks, or carrier-specific networks. The communication method used by the communication unit 29 to the external network is not particularly limited, as long as it is a wireless communication method that enables digital two-way communication at a predetermined communication speed and over a predetermined distance.

[0149] The communication unit 29 can communicate with communication devices located near the communication unit 29 (i.e., the wearable device 11) using P2P (Peer To Peer) technology. Communication devices located near the wearable device 11 include, for example, communication devices mounted on mobile objects such as vehicles and bicycles, communication devices that are fixed in place and do not move in stores, etc., and MTC (Machine Type Communication) terminals.

[0150] The communication unit 29 may receive electromagnetic waves transmitted by road traffic information communication systems (VICS (Vehicle Information and Communication System) (registered trademark)) such as radio beacons, optical beacons, and FM multiplex broadcasting.

[0151] Alternatively, the communication unit 29 and the external communication unit 55 may be directly connected via wireless communication intended for relatively short-range communication, such as Near Field Communication (NFC) or Bluetooth®.

[0152] The other configurations of the wearable device 11 in this example are the same as those of the wearable device 11 shown in Figure 7 above.

[0153] In this example, the environmental information data D11 acquired by the environmental information acquisition unit 24 and the user status data D13 acquired by the user status information acquisition unit 28 are sent to the environmental status recognition unit 25 via the communication unit 29 and the external communication unit 55.

[0154] The environmental state recognition unit 25 recognizes the state of the surrounding environment for user 1 based on the received environmental information data D11 and user state data D13.

[0155] The environmental state recognition unit 25 then sends a notification instruction signal D12 corresponding to the state of the surrounding environment for the recognized user 1 to the notification unit 26 via the external communication unit 55 and the communication unit 29.

[0156] The notification unit 26 provides notification to user 1 in response to the received notification instruction signal D12. The notification instruction signal D12 may be sent directly to the notification unit 26 from the external device 51 via the communication unit 29, or it may be sent to the notification unit 26 via another functional component of the wearable device 11 (e.g., the control unit 22).

[0157] Figure 10 shows another example of the functional configuration of the environmental status notification device 10.

[0158] The configuration of the wearable device 11 shown in Figure 10 is basically the same as that of the wearable device 11 shown in Figure 7 above.

[0159] However, in this example, the notification unit 26 notifies external devices 52 (second external devices) that are present in the environment surrounding user 1 but are not attached to user 1, according to the state of the surrounding environment for user 1 recognized by the environment state recognition unit 25.

[0160] The external device 52 is typically a moving object such as a vehicle (including passengers), but is not limited to that.

[0161] The method of notification from the notification unit 26 to the external device 52 is not limited. The notification unit 26 can notify the external device 52 by, for example, emitting sound or light (e.g., a warning sound or warning light).

[0162] The notification unit 26 may also send a notification signal to the external device 52 via the communication unit of the wearable device 11 (not shown in Figure 10). If the external device 52 is a moving object such as a vehicle, the network between the notification unit 26 and the external device 52 can be configured, for example, by CAN (Controller Area Network), LIN (Local Interconnect Network), or LAN (Local Area Network). Such a network can also be configured by an in-vehicle communication network or bus network conforming to FlexRay®, Ethernet®, or digital bidirectional communication standards.

[0163] The notification signal sent from the notification unit 26 to the external device 52 may include arbitrary information. For example, warning information, user 1's location information, environmental information data, user status data, and / or map information may be sent from the notification unit 26 to the external device 52.

[0164] In this example, the wearable device 11 (environmental status notification device 10) provides notifications to both the user 1 and the external device 52 via the notification unit 26, thereby further enhancing the safety of the user 1.

[0165] Figure 11 shows another example of the functional configuration of the environmental status notification device 10.

[0166] The wearable device 11 shown in Figure 11 includes a communication unit 29 that communicates data with an external device (third external device) 53.

[0167] The other configurations of the wearable device 11 in this example are the same as those of the wearable device 11 shown in Figure 7 above.

[0168] In this example, the communication unit 29 sends the surrounding environment status data D14, which indicates the state of the surrounding environment for user 1 as recognized by the environment state recognition unit 25, to the external device 53. This allows the external device 53 to understand the state of the surrounding environment for user 1.

[0169] The external device 53 is not limited. For example, the external device 53 can be configured as a mobile terminal owned by the user 1's guardian and / or a terminal device owned by a company (e.g., an insurance company or a security company). In this case, the guardian or company can be notified of the surrounding environment for user 1, further improving user 1's safety.

[0170] Figure 12 shows another example of the functional configuration of the environmental status notification device 10.

[0171] The configuration of the wearable device 11 shown in Figure 12 is basically the same as the wearable device 11 shown in Figure 11 above.

[0172] However, in this example, the communication unit 29 communicates data with an external device (first external device) 54 and receives environmental information data D11 and / or user status data D13 sent from the external device 54.

[0173] For example, if the external device 54 is a moving object such as a vehicle located near user 1, the external device 54 may send information regarding the attributes of the moving object to the communication unit 29 as environmental information data D11. The attributes of the moving object referred to here may include, for example, information regarding whether the external device 54 is a moving object equipped with a safety support device capable of recognizing user 1, and other information that the external device 54 can provide.

[0174] The environmental state recognition unit 25 recognizes the state of the surrounding environment for user 1 based on environmental information data acquired by the environmental information acquisition unit 24 and environmental information data and / or user state data sent from an external device 54 via the communication unit 29.

[0175] As described above, the environmental status notification device 10 in this example allows the status of the surrounding environment for user 1 to be recognized based on environmental information data acquired by the wearable device 11 and the external device 54. Furthermore, the status of the surrounding environment for user 1 can be recognized based on the environmental information data and user status data acquired by the wearable device 11 and the user status data acquired by the external device 54. Therefore, it is possible to recognize the status of the surrounding environment for user 1 more accurately, and the safety of user 1 is further improved.

[0176] Figure 13 shows another example of the functional configuration of the environmental status notification device 10.

[0177] Although the mounting section 21, control unit 22, and memory 23 are not shown in Figure 13, the wearable device 11 shown in Figure 13 has a mounting section 21, a control unit 22, and a memory 23.

[0178] The wearable device 11 in this example includes an external sensor 61 and an internal sensor 63.

[0179] The external sensor 61 detects the state of the environment surrounding user 1. Therefore, the external sensor 61 may include, for example, a camera, LiDAR, ToF and / or millimeter-wave radar.

[0180] The internal sensor 63 detects the state of user 1. Therefore, the internal sensor 63 may include, for example, GNSS and / or IMU.

[0181] Data detected by the external sensor 61 is analyzed by the external information analysis unit 62, and information about the user 1's surrounding environment is acquired by the external information analysis unit 62. The information about the user 1's surrounding environment obtained in this way is sent from the external information analysis unit 62 to the environment state recognition unit 25.

[0182] Meanwhile, the data detected by the internal sensor 63 is analyzed by the internal information analysis unit 64, and information regarding the state of user 1 is obtained by the internal information analysis unit 64. The internal information analysis unit 64 calculates the direction and / or distance of movement of user 1 from the data detected by the internal sensor 63, for example, based on PDR (Pedestrian Dead Reckoning), and can obtain information regarding the movement path of user 1. The information regarding the state of user 1 obtained in this way is sent from the internal information analysis unit 64 to the environmental state recognition unit 25.

[0183] Information received by the communication unit 29 from an external device of the wearable device 11 is analyzed by the communication information analysis unit 65, and communication analysis information is acquired by the communication information analysis unit 65. The communication analysis information obtained in this way is sent from the communication information analysis unit 65 to the environmental state recognition unit 25.

[0184] Furthermore, if the information received by the communication unit 29 does not require analysis, the communication unit 29 may transmit such information to the environmental state recognition unit 25 via the communication information analysis unit 65, or it may skip the communication information analysis unit 65 and transmit it directly to the environmental state recognition unit 25.

[0185] The external devices that communicate with the communication unit 29 are not limited. The communication unit 29 may communicate data with, for example, servers, infrastructure equipment (e.g., base stations and access points), and / or mobile devices (e.g., vehicle control devices). The information that the communication unit 29 receives is also not limited; the communication unit 29 may receive, for example, map information, infrastructure information, and vehicle information.

[0186] The communication information analysis unit 65 may send and receive data with the external information analysis unit 62 and the internal information analysis unit 64.

[0187] For example, the communication information analysis unit 65 may transmit the map information received via the communication unit 29 to the internal information analysis unit 64. The internal information analysis unit 64 may, for example, use map matching technology based on the map information received from the communication information analysis unit 65 to obtain information about the state of user 1. Alternatively, the internal information analysis unit 64 may send information about the state of user 1 to the communication information analysis unit 65, and the communication information analysis unit 65 may use map matching technology to obtain information about the state of user 1. The highly accurate information about the state of user 1 obtained in this way is sent to the environment state recognition unit 25.

[0188] In this example, the environment state recognition unit 25 obtains a dynamic map of the user 1's surrounding environment from the dynamic map provision unit 66.

[0189] The dynamic map providing unit 66 is located outside the wearable device 11 and can be configured, for example, by a server.

[0190] A dynamic map is a database-like map that includes high-precision 3D geospatial information (basic map information) and additional map information, and is sometimes referred to as a local dynamic map. A dynamic map typically includes multiple layers depending on the frequency of information updates. For example, a dynamic map may include a static information data layer (Permanent static data), a quasi-static information data layer (Transient static data), a quasi-dynamic information data layer (Transient dynamic data), and a dynamic information data layer (Highly dynamic data).

[0191] The environmental state recognition unit 25 recognizes the state of the surrounding environment for user 1 based on the environmental information data, user state data, and dynamic map obtained as described above.

[0192] The notification unit 26 then notifies user 1 according to the state of the surrounding environment for user 1 as recognized by the environmental state recognition unit 25.

[0193] Furthermore, information regarding the state of the surrounding environment for user 1, as recognized by the environmental state recognition unit 25, may be sent to an external device via the communication unit 29.

[0194] [Example of notification content from the notification section] As described above, the notification unit 26 provides notifications to user 1 according to the state of the surrounding environment recognized by the environmental state recognition unit 25, but the content of the notifications is not limited.

[0195] Typically, the notification unit 26 can notify the user 1 of the state of the surrounding environment.

[0196] For example, the notification unit 26 can notify user 1 that a moving object such as a vehicle or bicycle is approaching user 1. Furthermore, if a moving object exhibiting suspicious behavior is near user 1, the notification unit 26 can notify user 1 of the presence of such an object. The notification unit 26 can also notify user 1 of the attributes of a vehicle located near user 1.

[0197] For example, if a vehicle near user 1 is capable of recognizing pedestrians beside it via sensors or communication, the notification unit 26 does not need to notify user 1. On the other hand, if a vehicle near user 1 is not capable of recognizing pedestrians beside it via sensors or communication (for example, a vehicle not equipped with a V2X communication device), the notification unit 26 may notify user 1 that the vehicle requires attention.

[0198] Furthermore, if a person exhibiting suspicious behavior is near user 1, the notification unit 26 can notify user 1 of the presence of such a suspicious person. In this case, as shown in Figure 11 above, it is also effective to transmit information indicating the presence of a suspicious person (i.e., surrounding environment status data D14) to the external device 53.

[0199] As another typical example, the notification unit 26 may notify user 1 of the "state of the surrounding environment for user 1" related to user 1's actions.

[0200] For example, the notification unit 26 can notify user 1 that user 1 is approaching or entering an area requiring attention.

[0201] Areas requiring such caution may include not only areas that are inherently dangerous to User 1 (such as roadways), but also areas that are potentially dangerous to User 1. For example, areas where utility poles or tall walls (fences) that easily create blind spots for vehicle drivers are located near User 1 may be considered potentially dangerous areas for User 1.

[0202] The environmental state recognition unit 25 can determine whether user 1 is approaching or entering an area requiring attention based on the detection results of the sensors in the environmental information acquisition unit 24. Furthermore, the environmental state recognition unit 25 can determine whether user 1 is approaching or entering an area requiring attention based on the map information acquired by the environmental information acquisition unit 24 and the location information of user 1.

[0203] Furthermore, the notification unit 26 can notify user 1 if user 1 is violating traffic rules. Whether or not user 1 is violating traffic rules can be determined from the detection results of the sensor of the environmental information acquisition unit 24.

[0204] For example, by capturing a signal using the image sensor of the environmental information acquisition unit 24, and by the environmental state recognition unit 25 analyzing the captured image to understand the state of the signal, it is possible to determine whether or not user 1 is violating traffic rules. Furthermore, based on the detection results of the sensor of the environmental information acquisition unit 24, the environmental state recognition unit 25 can determine whether or not the location where user 1 is located is a roadway, thereby determining whether or not user 1 is violating traffic rules. Additionally, based on the map information acquired by the environmental information acquisition unit 24 and the location information of user 1, the environmental state recognition unit 25 can determine whether or not the location where user 1 is located is a roadway, thereby determining whether or not user 1 is violating traffic rules.

[0205] Furthermore, the notification unit 26 can notify user 1 if user 1 has not taken sufficient safety checks, such as failing to stop when crossing the road.

[0206] Furthermore, the notification unit 26 can notify user 1 if user 1 is taking actions that are difficult for the vehicle driver to predict (for example, sudden acceleration, sudden deceleration, or swerving).

[0207] Whether user 1 has taken sufficient safety checks or whether user 1 is taking unpredictable actions can be determined based on the detection results of the sensors in the environmental information acquisition unit 24 and / or the detection results of the sensors in the user state information acquisition unit 28.

[0208] For example, based on the detection results of sensors that detect the speed and / or acceleration of user 1, it is possible to determine whether user 1 is taking sufficient safety checks or whether user 1 is taking unpredictable actions. Furthermore, based on the map information and user 1's location information acquired by the environmental information acquisition unit 24, it is possible to determine whether user 1 is taking sufficient safety checks or whether user 1 is taking unpredictable actions.

[0209] Furthermore, the notification unit 26 can notify user 1 of any abnormalities in the mounting status of the wearable device 11 (particularly the environmental information acquisition unit 24 and / or the user status information acquisition unit 28) for user 1.

[0210] For example, if the environmental information acquisition unit 24 and / or the user status information acquisition unit 28 are unable to acquire appropriate information, the notification unit 26 can notify that the wearable device 11 is in an abnormal mounting state. Also, if the information acquired by the environmental information acquisition unit 24 and / or the user status information acquisition unit 28 determines that the direction of detection by the environmental information acquisition unit 24 is inappropriate, the notification unit 26 can notify that the wearable device 11 is in an abnormal mounting state.

[0211] [Stereophonic function] When the state of the surrounding environment for user 1 requires user 1 to quickly grasp the state of the surrounding environment or to take swift evasive action, it is desirable for the notification unit 26 to provide user 1 with a quick and intuitive notification.

[0212] To achieve such rapid and intuitive notifications, the notification unit 26 can, for example, provide notifications based on 3D audio technology.

[0213] 3D audio technology is a technology that provides users with three-dimensional sound effects, such as a technology that simulates a three-dimensional sound source. The specific methods and equipment configurations for realizing 3D audio technology are not limited; for example, the technology and equipment referred to as "360 REALITY AUDIO" (registered trademark) are also based on 3D audio technology.

[0214] For example, if user 1 is about to be placed in an unsafe situation, and the notification unit 26 simply issues a warning to user 1, user 1 may not be able to instantly recognize the situation and take appropriate action.

[0215] On the other hand, the notification unit 26 provides notifications to user 1 using three-dimensional sound, which can encourage user 1 to instantly recognize the situation or induce reflexive actions, thus contributing to improved user safety.

[0216] The following describes specific examples of 3D audio functionality.

[0217] [First example of spatial audio functionality (enhanced realism)] The notification unit 26 may provide directionality to the degree of emphasis on notification sounds (such as alarm sounds) depending on the state of the surrounding environment for user 1.

[0218] For example, the notification unit 26 can notify user 1 using a sound that emphasizes the direction from an object requiring attention (such as a moving object like a vehicle or an obstacle like a utility pole) toward user 1.

[0219] The notification unit 26 can notify user 1 by acquiring the positional relationship between the object requiring attention and user 1 and creating a three-dimensional sound corresponding to that positional relationship. The notification unit 26 may also change the volume, tempo, and / or other characteristics of the emphasized sound provided to user 1 depending on the distance between the object requiring attention and user 1.

[0220] The notification unit 26 may obtain information regarding the positional relationship between the object requiring attention and user 1 from the control unit 22 or an external device, or it may derive this information based on environmental information data and / or user status data.

[0221] Furthermore, if the notification sound (including the emphasis sound) provided to user 1 from the notification unit 26 is determined by a functional component other than the notification unit 26 (for example, the control unit 22), information regarding the positional relationship between the object requiring attention and user 1 is sent to that other functional component. In this case, information regarding the positional relationship between the object requiring attention and user 1 does not need to be sent to the notification unit 26.

[0222] In this example, user 1 can quickly and intuitively grasp the spatial relationship between the object requiring attention and user 1 by listening to the sound (particularly emphasized sound) emitted from the notification unit 26.

[0223] [Second example of 3D audio functionality (user behavior control)] The notification unit 26 may provide the user 1 with sounds that induce reflexive actions from the user 1, based on spatial sound technology, depending on the state of the surrounding environment for the user 1.

[0224] For example, when User 1 approaches an area requiring attention (such as a roadway, building, or other obstacle), the notification unit 26 can notify User 1 using a sound that is emphasized in the direction from the area requiring attention towards User 1 (for example, a sound that evokes danger).

[0225] This can induce a reflexive action from User 1 to move away from the area requiring attention, thereby further improving User 1's safety.

[0226] For example, the notification unit 26 can trigger a user 1 to crouch down by sending a notification that sounds like a bat swinging suddenly coming from above.

[0227] In this example as well, the notification unit 26 can notify user 1 by acquiring the positional relationship between the object requiring attention and user 1, and creating a three-dimensional sound corresponding to that positional relationship. Alternatively, another functional component other than the notification unit 26 can acquire the positional relationship between the object requiring attention and user 1, create three-dimensional sound data corresponding to that positional relationship, and send that three-dimensional sound data to the notification unit 26 to notify user 1.

[0228] [Third example of 3D sound functionality (music fusion experience)] The notification unit 26 may provide notification sounds to user 1, fused with music, depending on the state of the surrounding environment for user 1.

[0229] For example, the music provided to User 1 from the notification unit 26 may be changed depending on User 1's distance to the object and area requiring attention, and their relative approach speed.

[0230] For example, the notification unit 26 may change the melody, instrument sounds, tempo, and / or volume (e.g., the volume of a specific instrument sound) of the music provided to user 1 from the notification unit 26, according to the relationship between user 1 and the object and area requiring attention. The notification unit 26 may also give directionality to the degree of emphasis on the music according to the state of the surrounding environment for user 1.

[0231] This allows User 1 to quickly and intuitively grasp the state of their surrounding environment and experience music that changes according to the place, time, and surrounding conditions they are actually walking in.

[0232] [Differentiation] It should be noted that the embodiments and modifications disclosed herein are illustrative in all respects and should not be construed restrictively. The embodiments and modifications described above may be omitted, substituted, and modified in various ways without departing from the scope and spirit of the appended claims. For example, the embodiments and modifications described above may be combined in whole or in part, and other embodiments may be combined with the embodiments or modifications described above. Furthermore, the effects described herein are illustrative, and other effects may result.

[0233] The technical categories that embody the above-described technical concept are not limited. For example, the above-described technical concept may be embodied by a computer program that causes a computer to execute one or more steps included in a method for manufacturing or using the above-described device. Alternatively, the above-described technical concept may be embodied by a computer-readable, non-transitory recording medium on which such a computer program is recorded.

[0234] This disclosure may also take the following configuration.

[0235] [Item 1] An environmental information acquisition unit that is attached to the user and acquires environmental information data indicating information about the user's surrounding environment, An environmental state recognition unit recognizes the state of the surrounding environment for the user based on the aforementioned environmental information data, A notification unit that notifies the user according to the state of the surrounding environment for the user recognized by the environmental state recognition unit, An environmental condition notification device equipped with the following features.

[0236] [Item 2] The environmental status notification device according to item 1, wherein the environmental information acquisition unit includes a wave sensor that receives at least one of electromagnetic waves and sound waves from the user's surrounding environment and acquires the environmental information data.

[0237] [Item 3] The wave sensor is an environmental state notification device as described in item 2, which acquires images of the environment surrounding the user.

[0238] [Item 4] The environmental information acquisition unit includes an emission unit that emits detected electromagnetic waves or detected sound waves, The wave sensor is an environmental condition notification device according to item 2 or 3, which receives the reflected detected electromagnetic wave or the detected sound wave.

[0239] [Item 5] The aforementioned environmental information acquisition unit is: A location information acquisition unit that acquires the user's location information, A map information matching unit that acquires the environmental information data based on the location information and map information, An environmental condition notification device, including any of the items 1 to 4.

[0240] [Item 6] The location information acquisition unit is an environmental status notification device as described in item 5, which acquires the user's location information from a satellite positioning system.

[0241] [Item 7] Equipped with a communication unit for data transmission, The environmental state recognition unit recognizes the state of the surrounding environment for the user based on the environmental information data acquired by the environmental information acquisition unit and the environmental information data sent from the first external device via the communication unit, as described in any of items 1 to 6.

[0242] [Item 8] The system includes a user status information acquisition unit that acquires user status data indicating the user's status, The environmental state recognition unit recognizes the state of the surrounding environment for the user based on the environmental information data and the user state data, and is an environmental state notification device according to any one of items 1 to 7.

[0243] [Item 9] The environmental state notification device according to item 8, wherein the user state information acquisition unit includes a sensor that detects at least one of the speed and acceleration of the user's movement.

[0244] [Item 10] The environmental state recognition unit predicts the user's movement path based on the user state data, and recognizes the state of the surrounding environment for the user based on the predicted movement path of the user, according to item 8 or 9 of the environmental state notification device.

[0245] [Item 11] The environmental state recognition unit acquires a dynamic map of the user's surrounding environment and recognizes the state of the surrounding environment for the user based on the environmental information data and the dynamic map, as described in any of items 1 to 10.

[0246] [Item 12] The notification unit is an environmental status notification device described in any of items 1 to 11, which is attached to the user.

[0247] [Item 13] The notification unit provides notifications to the user through at least one of hearing, sight, and touch, as described in item 12.

[0248] [Item 14] The notification unit provides notifications to a second external device that is present in the user's surrounding environment but is not attached to the user, according to the state of the surrounding environment for the user as recognized by the environmental state recognition unit, as described in any of items 1 to 13.

[0249] [Item 15] The second external device is a vehicle, an environmental condition notification device as described in item 14.

[0250] [Item 16] Equipped with a communication unit for data transmission, The communication unit sends data indicating the state of the surrounding environment for the user, as recognized by the environmental state recognition unit, to a third external device, as described in any of items 1 to 15, which is an environmental state notification device.

[0251] [Item 17] The system includes an installation status detection unit that detects the installation status of the environmental information acquisition unit for the user, The notification unit provides notification to the user according to the detection result of the mounting status detection unit, as described in any of items 1 to 16.

[0252] [Item 18] The process involves acquiring environmental information data indicating information about the user's surrounding environment using an environmental information acquisition unit attached to the user, The environmental state recognition unit recognizes the state of the surrounding environment for the user based on the environmental information data, The notification unit provides a notification to the user according to the state of the surrounding environment for the user as recognized by the environmental state recognition unit. A method for notifying environmental conditions, including the following.

[0253] [Item 19] On the computer, A procedure for acquiring environmental information data indicating information about the user's surrounding environment using an environmental information acquisition unit attached to the user, The procedure for the environmental state recognition unit to recognize the state of the surrounding environment for the user based on the environmental information data, The notification unit provides a notification to the user according to the state of the surrounding environment for the user as recognized by the environmental state recognition unit. A program to execute. [Explanation of Symbols]

[0254] 1 user 10. Environmental status notification device 11 Wearable Devices 21 Mounting part 22 Control Unit 23 memory 24 Environmental Information Acquisition Department 25 Environmental state recognition unit 26 Notification Department 27 Installation status detection unit 28 User Status Information Acquisition Unit 29 Communications Department 31 Imaging device 32 Image analysis device 35 Emitter 36 Receivers 37. Received data analysis device 40 Location information acquisition section 41 Map Information Verification Unit 43 Map Information Provisioning Device 44 Location information analysis device 51~54 External device 55 External Communications Department 61. External Sensors 62 External World Information Analysis Department 63 Internal Sensors 64 Inner World Information Analysis Department 65 Communication Information Analysis Department 66 Dynamic Map Provision Department D11 Environmental Information Data D12 Notification instruction signal D13 User Status Data D14 Surrounding Environment Condition Data

Claims

1. An environmental information acquisition unit that is attached to the user and acquires environmental information data indicating information about the user's surrounding environment, An environmental state recognition unit recognizes the state of the surrounding environment for the user based on the aforementioned environmental information data, The system includes a notification unit that notifies the user according to the state of the surrounding environment for the user as recognized by the environmental state recognition unit, The notification unit, when the user approaches an object requiring attention, provides a notification using three-dimensional sound that is emphasized in the direction from the object requiring attention towards the user. The notification unit provides a notification using three-dimensional sound emphasized in the direction toward the user, using a sound that evokes danger and induces the user to take a reflexive action away from the object requiring attention. Environmental condition notification device.

2. The environmental state notification device according to claim 1, wherein the notification unit acquires the positional relationship between the object requiring attention and the user, and notifies the user of the spatial relationship.

3. The environmental state notification device according to claim 1, wherein the environmental information acquisition unit includes a wave sensor that receives at least one of electromagnetic waves and sound waves from the user's surrounding environment and acquires the environmental information data.

4. The environmental state notification device according to claim 3, wherein the wave sensor acquires an image of the environment surrounding the user.

5. The environmental information acquisition unit includes an emission unit that emits detected electromagnetic waves or detected sound waves, The environmental condition notification device according to claim 3, wherein the wave sensor receives the reflected detected electromagnetic wave or the detected sound wave.

6. The aforementioned environmental information acquisition unit is: A location information acquisition unit that acquires the user's location information, A map information matching unit that acquires the environmental information data based on the location information and map information, The environmental status notification device according to claim 1, including the above.

7. The environmental status notification device according to claim 6, wherein the location information acquisition unit acquires the user's location information from a satellite positioning system.

8. Equipped with a communication unit for data transmission, The environmental state recognition unit recognizes the state of the surrounding environment for the user based on the environmental information data acquired by the environmental information acquisition unit and the environmental information data sent from the first external device via the communication unit, according to claim 1, the environmental state notification device.

9. The system includes a user status information acquisition unit that acquires user status data indicating the user's status, The environmental state recognition unit recognizes the state of the surrounding environment for the user based on the environmental information data and the user state data, as described in claim 1.

10. The environmental state notification device according to claim 9, wherein the user state information acquisition unit includes a sensor that detects at least one of the speed and acceleration of the user's movement.

11. The environmental state notification device according to claim 9, wherein the environmental state recognition unit predicts the user's movement path based on the user state data, and recognizes the state of the surrounding environment for the user based on the predicted movement path of the user.

12. The environmental state recognition unit acquires a dynamic map of the user's surrounding environment and recognizes the state of the surrounding environment for the user based on the environmental information data and the dynamic map, as described in claim 1.

13. The environmental status notification device according to claim 1, wherein the notification unit is attached to the user.

14. The environmental status notification device according to claim 13, wherein the notification unit provides notification to the user through at least one of hearing, sight, and touch.

15. The environmental state notification device according to claim 1, wherein the notification unit provides notification to a second external device that is present in the user's surrounding environment but is not attached to the user, in accordance with the state of the surrounding environment for the user recognized by the environmental state recognition unit.

16. The environmental condition notification device according to claim 15, wherein the second external device is a vehicle.

17. Equipped with a communication unit for data transmission, The environmental status notification device according to claim 1, wherein the communication unit sends data indicating the state of the surrounding environment for the user, as recognized by the environmental status recognition unit, to a third external device.

18. The system includes an installation status detection unit that detects the installation status of the environmental information acquisition unit for the user, The environmental status notification device according to claim 1, wherein the notification unit provides notification to the user in accordance with the detection result of the mounting status detection unit.

19. The process involves acquiring environmental information data indicating information about the user's surrounding environment using an environmental information acquisition unit attached to the user, The environmental state recognition unit recognizes the state of the surrounding environment for the user based on the environmental information data, The notification unit includes the step of notifying the user according to the state of the surrounding environment for the user as recognized by the environmental state recognition unit, The notification unit, when the user approaches an object requiring attention, provides a notification using three-dimensional sound that is emphasized in the direction from the object requiring attention towards the user. The notification unit provides a notification using three-dimensional sound emphasized in the direction toward the user, using a sound that evokes danger and induces the user to take a reflexive action away from the object requiring attention. Method for notifying environmental conditions.

20. On the computer, A procedure for acquiring environmental information data indicating information about the user's surrounding environment using an environmental information acquisition unit attached to the user, The procedure for the environmental state recognition unit to recognize the state of the surrounding environment for the user based on the environmental information data, A program for causing a notification unit to execute a procedure for notifying the user according to the state of the surrounding environment for the user as recognized by the environmental state recognition unit, The notification unit, when the user approaches an object requiring attention, provides a notification using three-dimensional sound that is emphasized in the direction from the object requiring attention towards the user. The notification unit provides a notification using three-dimensional sound emphasized in the direction toward the user, using a sound that evokes danger and induces the user to take a reflexive action away from the object requiring attention. program.

Images