Warning device, warning system, warning method, and warning program
By integrating human position estimation, movement prediction, peripheral condition provision and state estimation functions in the warning device, combining the data of peripheral sensors and human sensors, we judge and adjust the mobile prediction area, and solve the problem of frequent false warnings in the prior art, and improve the The reliability of the warning system.
Patent Information
- Application Number
- CN202280094020.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-05
- Publication Date
- 2024-11-08
AI Technical Summary
Existing warning devices are prone to false warnings in environments such as indoors where objects are placed in disorderly manner or in construction sites, resulting in a reduced reliability of warnings.
Using human position estimation, movement prediction, peripheral status provision, state estimation and risk estimation components, the object's position, movement information, status and surrounding environment information are obtained through peripheral sensors and human sensors, the mobile prediction area is judged and adjusted, and the warning signal is output. To improve the reliability of warnings.
It realizes high reliability warnings in complex environments, reduces false warnings, and improves the reliability of warning systems.
Smart Images

Figure CN118922872A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a warning device, a warning system, a warning method and a warning program. Background Art
[0002] A device has been proposed, which includes: a region information acquisition unit that acquires image information from an imaging device; a storage unit that stores detection information; a detection unit that detects a predetermined condition in a predetermined region based on the image information and the detection information; a notification unit that notifies the predetermined condition when it is detected; an estimation unit that estimates the cause of the predetermined condition; and an update unit that updates the detection information when information indicating the cause of the predetermined condition is not included in the detection information (see, for example, Patent Document 1). The device detects the predetermined condition based on the image information and the detection information stored in the storage unit, and issues a warning to a person.
[0003] Prior art literature
[0004] Patent Literature
[0005] Patent Document 1: Japanese Patent Application Publication No. 2021-76934 (see, for example, claims 1, Figure 6 ) Summary of the invention
[0006] Problems to be solved by the invention
[0007] However, the above-mentioned conventional devices only predict that a person will come into contact with an object. Therefore, in a room where objects are cluttered or at a construction site, etc., there is a problem that the reliability of the warning is reduced due to frequent occurrence of false warnings.
[0008] An object of the present invention is to provide a warning device, a warning system, a warning method, and a warning program that can issue a warning with high reliability.
[0009] Means for solving problems
[0010] The warning device of the present invention is characterized in that the warning device comprises: a human position estimation unit, which estimates movement information including the position, movement direction and movement speed of the object person based on a first detection signal output from a peripheral sensor that senses the periphery of the object person; a movement prediction unit, which estimates an area predicted to be passed by the object person, namely, a movement prediction area, based on the movement information; a peripheral condition providing unit, which provides peripheral condition information including the position of an obstacle based on at least one of the first detection signal and pre-acquired map information; a state estimation unit, which estimates the state of the object person based on a second detection signal output from a human sensor that senses at least one of the state and sound of the object person; and a risk estimation unit, which determines whether the obstacle is a warning object based on the peripheral condition information, the movement prediction area and the state of the object person, and outputs a warning signal if the obstacle is a warning object, and the movement prediction unit adjusts the movement prediction area according to the state of the object person.
[0011] The warning method of the present invention is executed by a warning device, and is characterized in that the warning method has the following steps: estimating movement information including the position, moving direction and moving speed of the object person based on a first detection signal output from a peripheral sensor that senses the periphery of the object person; estimating an area predicted to be the movement prediction area through which the object person will pass based on the movement information; providing surrounding condition information including the position of an obstacle based on at least one of the first detection signal and pre-acquired map information; estimating the state of the object person based on a second detection signal output from a human sensor that senses at least one of the state and sound of the object person; and judging whether the obstacle is a warning object based on the surrounding condition information, the movement prediction area and the state of the object person, outputting a warning signal if the obstacle is a warning object, and in the step of predicting the movement prediction area, adjusting the movement prediction area based on the state of the object person.
[0012] Effects of the Invention
[0013] According to the present invention, a highly reliable warning can be issued. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 This is a block diagram schematically showing the configuration of the warning device and the warning system according to the first embodiment.
[0015] Figure 2 This is a diagram showing an example of the hardware configuration of the warning device and the warning system according to the first embodiment.
[0016] Figure 3 This is a flowchart showing an example of the operation of the movement prediction unit of the warning device according to the first embodiment.
[0017] Figure 4 (A) and (B) are explanatory diagrams showing the operation of the movement prediction unit according to the degree of shaking.
[0018] Figure 5 (A) and (B) are explanatory diagrams showing the operation of the movement prediction unit according to the distribution of the gaze point.
[0019] Figure 6 It is an explanatory diagram showing the operation of the motion prediction unit corresponding to the free space.
[0020] Figure 7 This is a flowchart showing an example of the operation of the risk estimation unit of the warning device according to the first embodiment.
[0021] Figure 8 (A) to (C) are explanatory diagrams showing determinations made by the risk estimation unit.
[0022] Fig. 9 This is a block diagram schematically showing the configuration of a warning device and a warning system according to the second embodiment.
[0023] Fig.10 This is a block diagram schematically showing the configuration of a warning device and a warning system according to a third embodiment.
[0024] Fig.11 This is a block diagram schematically showing the configuration of a warning device and a warning system according to a fourth embodiment.
[0025] Fig.12 This is a block diagram schematically showing the configuration of a warning device and a warning system according to the fifth embodiment.
[0026] Fig.13 This is a block diagram schematically showing the configuration of a warning device and a warning system according to a sixth embodiment.
[0027] Fig.14 This is a block diagram schematically showing the configuration of a warning device and a warning system according to a seventh embodiment. DETAILED DESCRIPTION
[0028] Hereinafter, a warning device, a warning system, a warning method, and a warning program according to an embodiment will be described with reference to the drawings. The following embodiments are merely examples, and the embodiments can be appropriately combined and each embodiment can be appropriately changed.
[0029] Implementation Method 1
[0030] Figure 11 is a block diagram schematically showing the configuration of a warning device 1 and a warning system 10 according to Embodiment 1. The warning device 1 is a device capable of implementing the warning method according to Embodiment 1. The warning device 1 is, for example, a computer capable of executing the warning program according to Embodiment 1.
[0031] The warning system 10 includes a warning device 1, a peripheral sensor 101, a human sensor 111, and a warning device 117. The warning system 10 may also include an actuator 118 that acts on the body of the subject. The warning system 10 issues a highly reliable warning for preventing a stumble or fall to a subject who is walking in a target area such as a factory facility or a construction site.
[0032] The surrounding sensor 101 is a device for sensing the surroundings of the subject. The surrounding sensor 101 is, for example, a LiDAR (Light Detection and Ranging), a camera, or both. The surrounding sensor 101 may also have a microphone as a sound receiving unit. The surrounding sensor 101 is set at a position that looks around the entire surroundings of the subject. The surrounding sensor 101 is configured at a position that can sense the surrounding environment of the subject. Preferably, the surrounding sensor 101 is set at a position that looks around the entire surroundings of the subject, such as the top of the subject's head, the neck, or both.
[0033] The human sensor 111 is a device that senses the movement of the subject. The human sensor 111 is, for example, a wearable sensor worn by the subject. The human sensor 111 can include, for example, a sensor that detects the movement and acceleration of the top of the head, waist, or feet of the person (for example, a 6-axis acceleration sensor). The human sensor 111 can include a device that detects the line of sight by photographing the eyes of the person (for example, a glasses-type device). The human sensor 111 can include, for example, a microphone that is set near the corner of the subject's mouth and detects the subject's voice.
[0034] The warning device 117 notifies the subject of a warning by one or more of sound, light (e.g., lighting of a lamp), display (e.g., a display), and vibration based on the received warning signal. The warning device 117 may also include a vibration device provided on a helmet worn by the subject.
[0035] The actuator 118 is, for example, an exoskeleton device or a muscle stimulation device that is worn on the subject's body and restricts the subject's body movement. The actuator 118 is a device that applies force or stimulation to the subject in order to make the subject stay away from obstacles.
[0036] like Figure 1As shown, the warning device 1 includes a person position estimation unit 105, a movement prediction unit 114, a surrounding condition providing unit 102, a state estimation unit 201, and a risk estimation unit 116. In the first embodiment, the warning device 1 may also include a noise detection unit 103 and a free space estimation unit 104. In addition, in the first embodiment, the state estimation unit 201 includes a walking estimation unit 112, a line of sight estimation unit 113, and an emotion estimation unit 115.
[0037] The person position estimation unit 105 estimates the movement information D5 including the position, movement direction and movement speed of the subject person based on the first detection signal D1 output from the surrounding sensor 101 that senses the surroundings of the subject person. The person position estimation unit 105 analyzes the three-dimensional point group information obtained by LiDAR, the image information obtained by the camera, or both, and estimates the position of the subject person in the surrounding map by a method such as SLAM (Simultaneous Localization and Mapping). In addition, the person position estimation unit 105 can also calculate the movement direction and movement speed of the subject person's position based on the temporal change of the subject person's position.
[0038] The movement prediction unit 114 estimates a movement prediction area D14 which is an area predicted to be passed by the subject person based on the movement information D5 .
[0039] The surrounding condition providing unit 102 provides surrounding condition information D2 including the position of obstacles (e.g., steps, holes, slopes, etc.) based on at least one of the first detection signal D1 output from the surrounding sensor 101 and the map information acquired and stored in advance. The surrounding condition providing unit 102 analyzes, for example, the three-dimensional point group information obtained by the LiDAR of the surrounding sensor 101, or the image information obtained by the camera of the surrounding sensor 101, or both the three-dimensional point group information and the image information, and detects the warning target of obstacles (e.g., steps, holes, slopes, etc.) that may cause the subject to fall or fall. In addition, as information indicating obstacles that may cause the subject to fall or fall, the map information acquired in advance may also be used.
[0040] The state estimation unit 201 estimates the state of the subject person based on the second detection signal D11 output from the human sensor 111 that senses at least one of the state and the sound of the subject person.
[0041] The risk estimation unit 116 determines whether the obstacle is a warning target based on the surrounding situation information D2, the predicted movement area D14, and the state of the subject output from the state estimation unit 201, and outputs a warning signal D16 if the obstacle is a warning target.
[0042] The movement prediction unit 114 adjusts the movement prediction area D14 based on the state of the subject person output from the state estimation unit 201 .
[0043] In the first embodiment, the walking estimation unit 112 of the state estimation unit 201 estimates the shaking degree D12a indicating the shaking degree of the subject when walking based on the second detection signal D11. For example, the walking estimation unit 112 can compare the behavior of each part of the human body when walking obtained by the acceleration sensor with the normal behavior information of the walking estimation unit 112 itself, and calculate the shaking degree based on the size of the difference between them. Preferably, the larger the shaking degree D12a is, the larger the movement prediction unit 114 expands the movement prediction area D14.
[0044] In the first embodiment, the walking estimation unit 112 estimates the lifting state D12b of the subject's foot when walking based on the second detection signal D11. For example, the walking estimation unit 112 calculates the lifting state of the foot when walking based on the output signal of the acceleration sensor attached to the subject's foot as the human sensor 111. Preferably, the risk estimation unit 116 determines whether the obstacle is a warning target based on the surrounding condition information D2, the movement prediction area D14, and the lifting state D12b of the subject's foot.
[0045] The walking estimation unit 112 may execute only one of the estimation of the shaking degree D12a of the subject when walking or the estimation of the lifting degree D12b of the subject's foot when walking.
[0046] The sight line estimation unit 113 estimates the gaze point D13 indicating the position at which the subject is looking based on the second detection signal D11 output from the human sensor 111. For example, the sight line estimation unit 113 calculates which current area the subject is looking at, that is, the position being looked at by the subject, that is, the gaze point, based on the detection result of the sight line by the human sensor 111. Preferably, the smaller the distribution of the gaze point D13 is, the smaller the movement prediction unit 114 reduces the movement prediction area D14. In addition, when determining whether an obstacle is a warning target, the risk estimation unit 116 can not regard an obstacle overlapping with the gaze point D13 as a warning target. That is, the risk estimation unit 116 can not regard an obstacle located at the same position as the gaze point D13 as a warning target.
[0047] The emotion estimation unit 115 estimates the emotion level D15 indicating the excitement of the subject based on the voice signal based on the voice of the subject in the second detection signal D11 output from the human sensor 111. For example, the emotion estimation unit 115 calculates the current emotional state of the subject (e.g., the level of anger or the level of impatience, etc.) based on the voice obtained by the microphone of the human sensor 111. The emotion level can be estimated based on the volume of the person's voice, the pitch (frequency) of the voice, etc. Preferably, when the emotion level D15 exceeds a preset threshold level, the risk estimation unit 116 regards all obstacles in the movement prediction area D14 as warning targets. For example, when the emotion level D15 exceeds a preset threshold level, the risk estimation unit 116 may also regard all obstacles in the movement prediction area D14 including obstacles overlapping with the gaze point D13 as warning targets.
[0048] The noise detection unit 103 detects the noise signal D3 obtained by the microphone in the second detection signal D11 output from the human sensor 111, and calculates the size of the surrounding noise. The risk estimation unit 116 can determine the warning target based on the surrounding situation information D2, the movement prediction area D14, the state of the subject output from the state estimation unit 201, and the noise level.
[0049] The free space estimation unit 104 estimates the free space indicating the area where the subject person can move based on the first detection signal D1 output from the surrounding sensor 101. That is, the free space estimation unit 104 detects a flat ground and stairs where a person can go up and down, etc., based on the three-dimensional point group information obtained by the LiDAR of the surrounding sensor 101, the image information obtained by the camera of the surrounding sensor 101, or both the three-dimensional point group information and the image information, and calculates the free space, which is the movable area where the person can move by walking. The movement prediction unit 114 can adjust the movement prediction area D14 based on the free space.
[0050] The movement prediction unit 114 calculates the movement prediction area D14 indicating the predicted position of the future subject based on the movement information D5 including the position, movement direction and movement speed of the subject obtained from the person position estimation unit 105. In addition, if the degree of shaking of the subject is large, the error of the predicted position is also large (wide range), and if the shaking is small, the error of the predicted position is small (narrow range). Based on the assumption that the person moves in the direction being gazed, the gaze point D13 obtained from the line of sight estimation unit 113 is used, and the distribution information of the gaze point D13 is used. If the distribution of the gaze point D13 is wide, the error of the predicted position is large, and if the distribution of the gaze point D13 is narrow, the error of the predicted position is small. In addition, based on the free space D4 output from the free space estimation unit 104, when the information of the movable area of the person becomes a shape such as a channel, the movement prediction unit 114 estimates that the person moves along the channel, so it is preferred that the movement prediction unit 114 corrects the result of the movement prediction.
[0051] The risk estimation unit 116 determines whether the predicted movement area D14 of the person obtained from the movement prediction unit 114 overlaps with the surrounding obstacles obtained by the surrounding condition providing unit 102. In the case where obstacles such as steps and caves overlap with the gaze point obtained from the line of sight estimation unit 113, the subject has already recognized the obstacle, so the risk estimation unit 116 excludes the visually recognized obstacle from the warning object. The obstacle has been visually recognized, for example, when the gaze point overlaps with the same obstacle for a period of more than a predetermined set time. In addition, overlap includes not only a completely consistent situation, but also a partial overlap.
[0052] In addition, when determining whether an obstacle is a warning target, the risk estimation unit 116 may also refer to the surrounding noise obtained by the noise detection unit 103. It is generally known that in a noisy environment, the attention of a person in the field of vision is reduced (for example, the effective field of vision becomes narrower). When the noise level obtained from the noise signal D3 output from the noise detection unit 103 exceeds a preset threshold value, that is, a noise setting value, the risk estimation unit 116 may also regard obstacles not only within the central field of vision of the subject person as warning targets, but also obstacles existing in the peripheral field of vision of the subject person as warning targets, regardless of the position of the gaze point.
[0053] In addition, when determining whether an obstacle is a warning target, the risk estimation unit 116 may also refer to the emotion level D15 output from the emotion estimation unit 115. It is known that when a person's emotional state is anger and impatience, the attention and cognitive ability are reduced compared to normal times. In addition, the emotion level of a person can be estimated based on the person's voice. Therefore, when the emotion level D15 exceeds a preset threshold level, the risk estimation unit 116 may also regard the obstacle in the movement prediction area as a warning target regardless of the position of the gaze point.
[0054] Figure 2 1 is a diagram showing an example of the hardware structure of the warning device 1 and the warning system 10 of embodiment 1. The warning device 1 is, for example, an information processing device, i.e., a computer. The warning device 1 has a processor 15 and a memory 16 as a volatile storage device, and the processor 15 has a receiving unit 17, a processing unit 18, and a control unit 19. The warning device 1 may also have a non-volatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and a communication device for communicating with an external device. The memory 16 is, for example, a semiconductor memory such as a RAM (Random Access Memory). The warning device 1 may also be composed of a processor that executes a software program, i.e., a warning program, installed from a recording medium or via a communication line and stored in the memory 16.
[0055] Each function of the warning device 1 may also be realized by a processing circuit. The processing circuit may be dedicated hardware, or may be a processor 15 that executes a warning program stored in the memory 16. The processor 15 may be any of a processing device, a computing device, a microprocessor, a microcomputer, and a DSP (Digital Signal Processor).
[0056] When the processing circuit is dedicated hardware, the processing circuit is, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).
[0057] In addition, the warning device 1 may be implemented partially by dedicated hardware and implemented in other parts by software or firmware. In this way, the processing circuit can implement the above functions by hardware, software, firmware or any combination thereof.
[0058] Figure 3 1 is a flowchart showing an example of the operation of the movement prediction unit 114 of the warning device 1 according to Embodiment 1. First, the movement prediction unit 114 receives movement information D5 including the position, movement direction, and movement speed of the subject from the person position estimation unit 105 (step S101), receives the shaking degree D12a of the subject from the walking estimation unit 112 (step S102), and calculates a movement prediction area based on the movement information D5 (step S103).
[0059] Next, the motion prediction unit 114 adjusts the motion prediction region according to the shake degree D12a (step S104). Figure 4(A) and (B) are explanatory diagrams showing the operation of the movement prediction unit 114 according to the degree of shaking. Figure 4 As shown in (B), the greater the shaking degree, the larger the movement prediction area is, such as Figure 4 As shown in (A), the smaller the shaking degree is, the smaller the movement prediction area is.
[0060] Next, the movement prediction unit 114 receives the gaze point D13 from the sight line estimation unit 113 (step S105 ), and adjusts the movement prediction region according to the distribution range and size of the gaze point and the position degree D12a of the gaze point (step S106 ). Figure 5 (A) and (B) are explanatory diagrams showing the operation of the movement prediction unit 114 corresponding to the distribution of the gaze point. Figure 5 As shown in (A), the wider the distribution range of the gaze point, the larger the motion prediction area is, such as Figure 4 As shown in (B), the narrower the distribution range of the attention point, the smaller the movement prediction area is.
[0061] Next, the movement prediction unit 114 receives the free space D4 indicating the movable area of the person from the free space estimation unit 104 (step S107 ), and adjusts the movement prediction area according to the movable area (step S108 ). Figure 6 1 is an explanatory diagram showing the operation of the movement prediction unit 114 corresponding to the free space. When the information on the movable area of a person is in the shape of a passage (narrow shape), the movement prediction unit 114 estimates that the person moves along the passage based on the free space D4 output from the free space estimation unit 104, and thus the movement prediction unit 114 can reduce the movement prediction area.
[0062] Figure 7 This is a flowchart showing an example of the operation of the risk estimation unit 116 of the warning device 1 according to the first embodiment. Figure 8 (A) to (C) are explanatory diagrams showing the determination of a warning target or a warning non-target by the risk estimation unit 116 of the warning device 1 .
[0063] The risk estimation unit 116 receives the movement prediction area D14 from the movement prediction unit 114 (step S201), receives the surrounding condition information D2 from the surrounding condition providing unit 102 (step S202), receives the foot lifting condition D12b, the gaze point D13, and the emotion level D15 as the state of the subject from the state estimation unit 201 (steps S203 to S205), and receives the noise level from the noise detection unit (step S206). If the emotion level is below the predetermined threshold level and the noise level is below the noise setting value (step S207: No, step S209: No), as shown in FIG. Figure 8 As shown in (C), obstacles that exist in the predicted movement area and are not visually recognized are set as warning targets (step S211).
[0064] In the case where the emotion level D15 exceeds the predetermined threshold level (step S207: Yes), Figure 8 As shown in (A) of FIG. 2 , the risk estimation unit 116 sets obstacles existing in the predicted movement area as warning targets (step S208 ).
[0065] When the noise indicated by the noise signal D3 exceeds the predetermined noise setting value (step S209: Yes), Figure 8 As shown in (B) of FIG. 2 , the risk estimation unit 116 sets obstacles existing in the predicted movement area as warning targets (step S208 ).
[0066] As described above, according to embodiment 1, in order to achieve high-precision prediction of human movement, in addition to information such as the position and movement speed of the person, the free space D4 representing the movable area, the shaking degree D12a, and the gaze point D13 are also used, thereby improving the accuracy of the predicted area of human movement.
[0067] In addition, according to embodiment 1, in determining whether an obstacle is a warning target, the state of the person (e.g., the lift status of the foot D12b, the gaze point D13, the emotional level D15) and the noise level are used. Therefore, the obstacle that is a warning target can be appropriately determined, and excessive warnings can be avoided.
[0068] Implementation Method 2
[0069] Fig. 9 2 is a block diagram schematically showing the configuration of a warning device 2 and a warning system 20 according to Embodiment 2. The warning device 2 is a device capable of implementing the warning method according to Embodiment 2. Fig. 9 In, with Figure 1 The same or corresponding structures as shown are marked with Figure 1 The same reference numerals are used as shown. The state estimation unit 202 of the warning device 2 includes the walking estimation unit 112. The warning device 2 is different from the warning device 1 of the first embodiment in that it does not include the sight estimation unit and the emotion estimation unit. Except for this point, the second embodiment is the same as the first embodiment.
[0070] Implementation 3
[0071] Fig.10 1 is a block diagram schematically showing the configuration of the warning device 3 and the warning system 30 according to the third embodiment. The warning device 3 is a device capable of implementing the warning method according to the third embodiment. Fig.10 In, with Figure 1The same or corresponding structures as shown are marked with Figure 1 The same reference numerals are used as shown. The state estimation unit 203 of the warning device 3 includes the walking estimation unit 112 and the sight estimation unit 113. The warning device 3 differs from the warning device 1 of the first embodiment in that the warning device 3 does not include the emotion estimation unit. Except for this point, the third embodiment is the same as the first embodiment.
[0072] Implementation 4
[0073] Fig.11 4 is a block diagram schematically showing the configuration of a warning device 4 and a warning system 40 according to a fourth embodiment. The warning device 4 is a device capable of implementing the warning method according to the fourth embodiment. Fig.11 In, with Figure 1 The same or corresponding structures as shown are marked with Figure 1 The state estimation unit 204 of the warning device 4 includes the walking estimation unit 112 and the emotion estimation unit 115. The warning device 4 is different from the warning device 1 of the first embodiment in that it does not include the sight line estimation unit. Except for this point, the fourth embodiment is the same as the first embodiment.
[0074] Implementation method 5
[0075] Fig.12 1 is a block diagram schematically showing the configuration of a warning device 5 and a warning system 50 according to a fifth embodiment. The warning device 5 is a device capable of implementing the warning method according to the fifth embodiment. Fig.12 In, with Figure 1 The same or corresponding structures as shown are marked with Figure 1 The same reference numerals are used as those shown. The state estimation unit 205 of the warning device 5 includes the sight estimation unit 113. The warning device 5 is different from the warning device 1 of the first embodiment in that it does not include the walking estimation unit and the emotion estimation unit. Except for this point, the fifth embodiment is the same as the first embodiment.
[0076] Implementation 6
[0077] Fig.13 6 is a block diagram schematically showing the configuration of a warning device 6 and a warning system 60 according to a sixth embodiment. The warning device 6 is a device capable of implementing the warning method according to the sixth embodiment. Fig.13 In, with Figure 1 The same or corresponding structures as shown are marked with Figure 1 The same reference numerals are used as shown. The state estimation unit 206 of the warning device 6 includes the emotion estimation unit 115. The warning device 6 differs from the warning device 1 of the first embodiment in that it does not include the walking estimation unit and the sight estimation unit. Except for this point, the sixth embodiment is the same as the first embodiment.
[0078] Implementation 7
[0079] Fig.14 7 is a block diagram schematically showing the configuration of a warning device 7 and a warning system 70 according to a seventh embodiment. The warning device 7 is a device capable of implementing the warning method according to the seventh embodiment. Fig.14 In, with Figure 1 The same or corresponding structures as shown are marked with Figure 1 The same reference numerals are used as those shown. The state estimation unit 207 of the warning device 7 includes a sight estimation unit 113 and an emotion estimation unit 115. The warning device 7 differs from the warning device 1 of the first embodiment in that the warning device 7 does not include a walking estimation unit. Except for this point, the seventh embodiment is the same as the first embodiment.
[0080] Description of symbols
[0081] 1~7: warning device; 10, 20, 30, 40, 50, 60, 70: warning system; 101: peripheral sensor; 102: peripheral condition providing unit; 103: noise detection unit; 104: free space estimation unit; 105: human position estimation unit; 111: human sensor; 112: walking estimation unit; 113: line of sight estimation unit; 114: movement prediction unit; 115: emotion estimation unit; 116: risk estimation unit; 117: warning device; 118: actuator; 201~207: state estimation unit.
Claims
1. A warning device, characterized in that: The warning device comprises: a person position estimating unit for estimating movement information including a position, a moving direction, and a moving speed of the subject person based on a first detection signal outputted from a surrounding sensor for sensing the surroundings of the subject person; a movement prediction unit that estimates, based on the movement information, a movement prediction area that is predicted to be passed by the subject person; a surrounding condition providing unit for providing surrounding condition information including the position of an obstacle based on at least one of the first detection signal and pre-acquired map information; a state estimation unit that estimates the state of the subject person based on a second detection signal output from a human sensor that senses at least one of the state and sound of the subject person; as well as a risk estimation unit that determines whether the obstacle is a warning target based on the surrounding condition information, the predicted movement area, and the state of the subject person, and outputs a warning signal if the obstacle is a warning target. The movement prediction unit adjusts the movement prediction area according to the state of the subject person.
2. The warning device according to claim 1, characterized in that: The state estimation unit includes a walking estimation unit that estimates a shaking degree indicating a magnitude of shaking of the subject when walking based on the second detection signal. The movement prediction unit enlarges the movement prediction area as the shaking degree increases.
3. The warning device according to claim 2, characterized in that: The walking estimation unit estimates the lifting of the foot of the subject when walking based on the second detection signal. The risk estimation unit makes the determination based on the surrounding situation information, the predicted movement area, and the lifting state of the foot of the subject.
4. The warning device according to claim 1, characterized in that: The state estimation unit includes a walking estimation unit that estimates, based on the second detection signal, how the subject's foot is lifted when walking. The risk estimation unit makes the determination based on the surrounding situation information, the predicted movement area, and the lifting state of the foot of the subject.
5. The warning device according to any one of claims 1 to 4, characterized in that: The state estimation unit includes a sight line estimation unit that estimates a gaze point indicating a position at which the subject person is gazing based on the second detection signal. The movement prediction unit reduces the movement prediction area as the degree of spread of the distribution of the attention points decreases.
6. The warning device according to claim 5, characterized in that: In the determination, the risk estimation unit does not regard the obstacle overlapping the attention point as the warning target.
7. The warning device according to any one of claims 1 to 4, characterized in that: The state estimation unit includes a sight line estimation unit that estimates a gaze point indicating a position at which the subject person is gazing based on the second detection signal. In the determination, the risk estimation unit does not regard the obstacle overlapping the attention point as the warning target.
8. The warning device according to any one of claims 1 to 5, characterized in that: The second detection signal includes a voice signal based on the voice of the subject, The state estimation unit includes an emotion estimation unit that estimates an emotion level indicating the excitement of the subject based on the voice signal. When the emotion level exceeds a preset threshold level, the risk estimation unit regards all the obstacles in the predicted movement area as the warning targets.
9. The warning device according to claim 6 or 7, characterized in that: The second detection signal includes a voice signal based on the voice of the subject, The state estimation unit includes an emotion estimation unit that estimates an emotion level indicating the excitement of the subject based on the voice signal. When the emotion level exceeds a preset threshold level, the risk estimation unit regards all the obstacles in the predicted movement area, including the obstacle overlapping the gaze point, as the warning targets.
10. The warning device according to any one of claims 1 to 9, characterized in that: The warning device further includes a noise detection unit that detects a noise signal from the second detection signal. The risk estimation unit determines the warning target based on the noise level indicated by the noise signal, the surrounding condition information, the predicted movement area, the state of the subject person, and the noise level.
11. The warning device according to any one of claims 1 to 10, characterized in that: The warning device further includes a free space estimation unit that estimates a free space indicating an area in which the subject person can move based on the first detection signal. The motion prediction unit adjusts the motion prediction area according to the free space.
12. A warning system, characterized in that: The warning system has: The peripheral sensor; the human sensor; The warning device according to any one of claims 1 to 11; and A warning device issues a warning to the subject person based on the warning signal.
13. The warning system according to claim 12, characterized in that The human sensor includes a wearable sensor worn by the subject, The peripheral sensor includes a camera or LiDAR.
14. A warning method performed by a warning device, characterized in that: The warning method has the following steps: estimating movement information including a position, a movement direction, and a movement speed of the subject person based on a first detection signal output from a surrounding sensor that senses the surroundings of the subject person; estimating, based on the movement information, an area predicted to be passed by the subject person, that is, a movement prediction area; providing surrounding condition information including the location of the obstacle based on at least one of the first detection signal and pre-acquired map information; estimating the state of the subject person based on a second detection signal output from a human sensor that senses at least one of the state and sound of the subject person; as well as determining whether the obstacle is a warning target based on the surrounding condition information, the predicted movement area, and the state of the subject person, and outputting a warning signal if the obstacle is a warning target; In the step of predicting the predicted movement region, the predicted movement region is adjusted according to the state of the subject person.
15. A warning program, characterized in that: The warning program causes the computer to perform the following steps: estimating movement information including a position, a movement direction, and a movement speed of the subject person based on a first detection signal output from a surrounding sensor that senses the surroundings of the subject person; estimating, based on the movement information, an area predicted to be passed by the subject person, that is, a movement prediction area; providing surrounding condition information including the location of the obstacle based on at least one of the first detection signal and pre-acquired map information; estimating the state of the subject person based on a second detection signal output from a human sensor that senses at least one of the state and sound of the subject person; determining whether the obstacle is a warning target based on the surrounding condition information, the predicted movement area, and the state of the subject person, and outputting a warning signal if the obstacle is a warning target; as well as When predicting the predicted movement area, the predicted movement area is adjusted according to the state of the subject person.
Citation Information
Patent Citations
Dangerous action prevention device, dangerous action prevention system, dangerous action prevention method, and dangerous action prevention program
JP2021076934A
