WARNING DEVICE, WARNING SYSTEM, WARNING PROCEDURES AND WARNING PROGRAM

The warning device uses advanced sensors and analysis to assess a person's movement and environmental conditions, ensuring reliable warnings by accurately identifying potential hazards and minimizing false alarms.

DE112022006587T5Pending Publication Date: 2024-11-21MITSUBISHI ELECTRIC CORP
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
DE112022006587
Authority / Receiving Office
DE · DE
Patent Type
Applications
Current Assignee / Owner
Filing Date
2022-04-05
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Conventional warning devices frequently issue false warnings due to the disorderly placement of objects, reducing the reliability of warnings in environments like rooms or construction sites.

Method used

A warning device that includes a person position detection unit, movement anticipation unit, environmental situation providing unit, state determination unit, and risk determination unit to assess the movement and condition of a target person, determining whether obstacles are warning objects based on environmental and personal factors.

Benefits of technology

Enables warnings to be issued with high reliability by accurately identifying potential hazards and reducing false alarms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

A warning device (1) comprises a person position detection unit (105) that determines motion information (D5) indicating the position, direction of movement, and speed of movement of a target person based on a first detection signal (D1) emitted by an environment sensor (101); a motion anticipation unit (114) that determines a motion anticipation area (D14) based on the motion information (D5); an environment situation provision unit (102) that provides environment situation information (D2) indicating the position of an obstacle based on at least one of the first detection signal (D1) and map information; a condition determination unit (201) that determines the condition of the target person based on a second detection signal (D11) emitted by a person sensor (111); and a risk assessment unit (116).which issues a warning signal (D16) when it is assessed that the obstacle is a warning object, based on the environmental situation information (D2), the movement anticipation area (D14), and the target person's condition. The movement anticipation unit (114) adjusts the movement anticipation area (D14) based on the target person's condition.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a warning device, a warning system, a warning method and a warning program. STATE OF THE ART

[0002] A device has been proposed comprising an area information acquisition unit that acquires image information provided by an image acquisition device, a storage unit that stores detection information as information for detection, a detection unit that detects a predetermined situation in a predetermined area based on the image information and the detection information, a notification unit that notifies of the detection of the predetermined situation when the predetermined situation is detected, an inference unit that derives a cause of the predetermined situation, and an update unit that updates the detection information when the detection information does not include information indicating the cause of the predetermined situation (see, for example, Patent Reference 1).This device detects the predetermined situation based on the image information and the detection information stored in the storage unit and issues a warning to a person. PRIOR ART PATENT REFERENCES

[0003] Patent Reference 1: Japanese Patent Application Publication No. 2021-76934 (see, for example, claim 1 and Fig. 6). SUMMARY OF THE INVENTION TASK TO BE SOLVED BY THE INVENTION

[0004] However, the conventional device described above only anticipates the occurrence of contact between a person and an object and therefore has the problem that the reliability of the warning is reduced by frequently issuing false warnings in a room, construction site, or the like where objects are placed in a disorderly manner.

[0005] The aim of the present disclosure is to provide a warning device, a warning system, a warning method and a warning program that enable warnings to be issued with high reliability. MEANS TO SOLVE THE PROBLEM

[0006] A warning device in the present disclosure includes a person position detection unit that detects movement information including a position, a movement direction, and a movement speed of a target person based on a first detection signal output from an environment sensor that performs scanning with respect to an environment of the target person; a movement anticipation unit that detects a movement anticipation area as an area through which the target person is anticipated to move based on the movement information; an environment situation provision unit that provides environment situation information including a position of an obstacle based on at least one of the first detection signal and previously acquired map information; a state detection unit;which determines a condition of the target person based on a second detection signal output from a human sensor that samples at least one of the condition and the voice of the target person; and a risk determination unit that judges whether the obstacle is a warning object based on the surrounding situation information, the movement anticipation range, and the condition of the target person, and outputs a warning signal if the obstacle is a warning object. The movement anticipation unit adjusts the movement anticipation range based on the condition of the target person.

[0007] A warning method in the present disclosure is a method to be executed by a warning device. The warning method includes a step of determining movement information including a position, a movement direction, and a movement speed of a target person based on a first detection signal output from an environmental sensor that performs scanning with respect to an environment of the target person; a step of determining a movement anticipation area as an area through which the target person is anticipated to move based on the movement information; a step of providing environmental situation information including a position of an obstacle based on at least one of the first detection signal and previously acquired map information;a step of determining a state of the target person based on a second detection signal output from a human sensor that samples at least one of the state and the voice of the target person; and a step of making a judgment as to whether or not the obstacle is a warning object based on the surrounding situation information, the movement anticipation range, and the state of the target person, and outputting a warning signal if the obstacle is a warning object. The movement anticipation range is adjusted based on the state of the target person in the step of anticipating the movement anticipation range. EFFECT OF THE INVENTION

[0008] According to the present disclosure, warnings can be issued with high reliability. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram schematically showing the configuration of a warning device and a warning system according to a first embodiment. Fig. 2 is a diagram showing an example of the hardware configuration of the warning device and the warning system according to the first embodiment. Fig. 3 is a flowchart showing an operation example of a movement anticipation unit of the warning device according to the first embodiment. The Fig. 4A and Fig. 4B are explanatory diagrams showing the operation of the motion anticipation unit depending on a fluctuation level. The Fig. 5A and Fig. 5B are explanatory diagrams showing the operation of the motion anticipation unit depending on a distribution of a fixation point. Fig. 6 is an explanatory diagram showing the operation of the motion anticipation unit depending on a free space. Fig. 7 is a flowchart showing an operation example of a risk determination unit of the warning device according to the first embodiment. The Fig. Figures 8A to 8C are illustrative diagrams showing an assessment by the risk assessment unit. Fig. 9 is a block diagram schematically showing the configuration of a warning device and a warning system according to a second embodiment. Fig. 10 is a block diagram schematically showing the configuration of a warning device and a warning system according to a third embodiment. Fig. 11 is a block diagram schematically showing the configuration of a warning device and a warning system according to a fourth embodiment. Fig. 12 is a block diagram schematically showing the configuration of a warning device and a warning system according to a fifth embodiment. Fig. 13 is a block diagram schematically showing the configuration of a warning device and a warning system according to a sixth embodiment. Fig. 14 is a block diagram schematically showing the configuration of a warning device and a warning system according to a seventh embodiment. MODE FOR CARRYING OUT THE INVENTION

[0009] A warning device, a warning system, a warning method, and a warning program according to each embodiment will be described below with reference to the drawings. The following embodiments are only examples, and it is possible to combine embodiments as appropriate and modify each embodiment as appropriate. First embodiment

[0010] Fig. 1 is a block diagram schematically showing the configuration of a warning device 1 and a warning system 10 according to a first embodiment. The warning device 1 is a device capable of executing a warning method according to the first embodiment. The warning device 1 is, for example, a computer capable of executing a warning program according to the first embodiment.

[0011] The warning system 10 includes the warning device 1, an environmental sensor 101, a person sensor 111, and a warning issuing device 117. The warning system 10 may include an actuator 118 that acts on the body of a target person. The warning system 10 issues a warning with high reliability to prevent a fall or stumble of the target person, such as a person walking in an object area such as a factory or construction site.

[0012] The environmental sensor 101 is a device that scans the environment of the target person. The environmental sensor 101 is, for example, a LiDAR (Light Detection and Ranging), a camera, or a combination of LiDAR and camera. The environmental sensor 101 can have a microphone as a sound detection unit. The environmental sensor 101 is mounted at a position from which the entire environment of the target person can be viewed. The environmental sensor 101 is arranged at a position from which the surrounding environment of the target person can be scanned. The environmental sensor 101 should, if possible, be placed at a position from which the entire environment of the target person can be viewed, for example, in the region of the crown or on the neck of the target person, or both in the crown region and on the neck.

[0013] The human sensor 111 is a device that senses the movement of the target person. The human sensor 111 is, for example, a wearable sensor attached to the target person. The human sensor 111 may include, for example, a sensor (e.g., a six-axis acceleration sensor) that detects the movement and acceleration of a person's vertex region, waist, foot, or the like. The human sensor 111 may include a device (e.g., a pair of glasses) that detects a person's line of sight by photographing the person's eyes. The human sensor 111 may include, for example, a microphone that is arranged in the vicinity of the target person's mouth and detects the target person's voice.

[0014] The warning issuing device 117 notifies the target person of a warning through one or more of the following means: sound, light (e.g., illuminating a lamp), indications (e.g., a display), and vibration based on a received warning signal. The warning issuing device 117 may include a vibration device provided on a helmet worn by the target person.

[0015] The actuator 118 is, for example, an exoskeleton device or a muscle stimulation device attached to the body of the target person and regulating the movement of the target person's body. The actuator 118 is a device that applies force or stimulation to the target person to separate the person from an obstacle.

[0016] As in Fig. As shown in Figure 1, the warning device 1 includes a person position detection unit 105, a movement anticipation unit 114, an environmental situation provision unit 102, a state detection unit 201, and a risk detection unit 116. In the first embodiment, the warning device 1 may include a sound detection unit 103 and a clearance detection unit 104. In the first embodiment, the state detection unit 201 further includes a walking detection unit 112, a line-of-sight detection unit 113, and an emotion detection unit 115.

[0017] The person position determination unit 105 determines movement information D5 including a position, a movement direction, and a movement speed of the target person based on a first detection signal D1 output from the environmental sensor 101 that scans the surroundings of the target person. The person position determination unit 105 determines the position of the target person in a surrounding area map by analyzing the three-dimensional point cloud information obtained from the LiDAR, the image information obtained from the camera, or both the three-dimensional point cloud information and the image information, and using a method such as SLAM (Simultaneous Localization and Mapping). It is also possible for the person position determination unit 105 to calculate the movement direction and the movement speed of the position of the target person based on the change in the position of the target person in a time series.

[0018] A movement anticipation unit 114 determines a movement anticipation area D14 as an area through which the target person is anticipated to move, based on the movement information D5.

[0019] The surrounding situation providing unit 102 provides surrounding situation information D2 including the position of an obstacle (e.g., a step, a hole, a slope, or the like) based on at least one of the first detection signals D1 output from the surrounding sensor 101 and map information previously acquired and stored. The surrounding situation providing unit 102 detects a warning object such as an obstacle (e.g., a step, a hole, a slope, or the like) that may cause a fall or a fall of the target person by analyzing the three-dimensional point cloud information obtained by the LiDAR of the surrounding sensor 101, the image information obtained by the camera of the surrounding sensor 101, or both the three-dimensional point cloud information and the image information.It is also possible to use the previously obtained map information as information indicating the obstacle that may cause the target person to fall or fall.

[0020] The state determination unit 201 determines a state of the target person based on a second detection signal D11 output from the human sensor 111 that senses at least one of the state and the voice of the target person.

[0021] The risk determination unit 116 makes a judgment as to whether the obstacle is a warning object or not based on the surrounding situation information D2, the movement anticipation range D14, and the state of the target person output from the state determination unit 201, and outputs a warning signal D16 when the obstacle is a warning object.

[0022] The motion anticipation unit 114 adjusts the motion anticipation range D14 based on the state of the target person output from the state determination unit 201.

[0023] In the first embodiment, the walking detection unit 112 of the state detection unit 201 determines, based on the second detection signal D11, a sway level D12a representing the degree of sway of the target person while walking. For example, the walking detection unit 112 may compare the behavior of human body parts obtained from the acceleration sensor with the normal-time behavior information stored by the walking detection unit 112 itself, and calculate the sway level based on the degree of difference between the behavior and the normal-time behavior information. It is desirable for the movement anticipation unit 114 to expand the movement anticipation range D14 as the sway level D12a increases.

[0024] In the first embodiment, the walking determination unit 112 determines a foot height level D12b of the target person while walking based on the second detection signal D11. For example, the walking determination unit 112 calculates the foot height level while walking based on an output signal of the acceleration sensor attached to a foot of the target person as the human sensor 111. It is desirable for the risk determination unit 116 to judge whether the obstacle is a warning object based on the surrounding situation information D2, the movement anticipation range D14, and the foot height level D12b of the target person.

[0025] It is also permissible if the running detection unit 112 performs only one of the estimation of the sway level D12a of the target person when running and the estimation of the foot height level D12b of the target person when running.

[0026] The line-of-sight determination unit 113 determines a fixation point D13 indicating a position where the target person is looking, based on the second detection signal D11 output from the human sensor 111. For example, the line-of-sight determination unit 113 calculates which existing region the target person is currently looking at, that is, the fixation point as the position the target person is currently looking at, based on the result of line-of-sight detection by the human sensor 111. It is desirable for the movement anticipation unit 114 to narrow the movement anticipation range D14 as the degree of dispersion of the distribution of the fixation point D13 decreases. Furthermore, the risk determination unit 116 may consider an obstacle that overlaps with the fixation point D13 as not being a warning object when judging whether the obstacle is a warning object.That is, the risk determination unit 116 may regard an obstacle located at the same position as the fixation point D13 as not a warning object.

[0027] The emotion determination unit 115 determines an emotion level D15 indicating the degree of agitation of the target person based on a voice signal corresponding to the target person's voice in the second detection signal D11 output from the human sensor 111. For example, the emotion determination unit 115 calculates the current emotional state (e.g., the emotion level, such as an anger level or an impatience level) of the target person based on the voice received from the microphone of the human sensor 111. The emotion level can be determined based on the volume of the person's voice, the pitch (frequency) of the voice, or the like. It is desirable for the risk determination unit 116 to consider all obstacles in the movement anticipation area D14 as warning objects when the emotion level D15 exceeds a preset threshold.When the emotion level D15 exceeds the previously set threshold level, the risk determination unit 116 may regard all obstacles in the movement anticipation area D14, including the fixation point D13 overlapping with the obstacle, as warning objects.

[0028] The noise detection unit 103 calculates the ambient noise level by detecting a noise signal D3 received from the microphone in the second detection signal D11 output from the human sensor 111. The risk determination unit 116 is capable of determining the warning objects based on the ambient situation information D2, the movement anticipation range D14, the condition of the target person output from the condition determination unit 201, and the noise level.

[0029] The free space determination unit 104 determines a free space, which is an area where the target person can move, based on the first detection signal D1 output from the surrounding sensor 101. That is, the free space determination unit 104 calculates the free space as a movable region where the person can move by walking, by detecting a flat floor, a step that the person can climb and descend, etc., based on the three-dimensional point cloud information obtained by the LiDAR of the surrounding sensor 101, the image information obtained by the camera of the surrounding sensor 101, or both the three-dimensional point cloud information and the image information. The movement anticipation unit 114 is capable of adjusting the movement anticipation area D14 based on the free space.

[0030] The motion anticipation unit 114 calculates the motion anticipation range D14, which indicates the anticipated position of the target person in the future, based on the motion information D5 acquired by the person position detection unit 105, including the position, movement direction, and movement speed of the target person. Furthermore, the errors in the anticipated positions become large (wide distribution) when the fluctuation level of the target person is high, and become small (narrow distribution) when the fluctuation level is low. Assuming that the person is moving in the line-of-sight direction, distribution information regarding the fixation point D13 is obtained using the fixation points D13 acquired by the line-of-sight detection unit 113. The errors in the anticipated positions are large when the distribution of the fixation point D13 is wide, and small when the distribution of the fixation point D13 is narrow.When the information about the movable region of the person indicates a shape of a passage based on the clearance D4 output from the clearance determination unit 104, it can be inferred that the person is moving along the passage, and therefore, it is desirable for the movement anticipation unit 114 to correct the result of the movement anticipation.

[0031] The risk determination unit 116 judges whether the person's movement anticipation range D14 acquired by the movement anticipation unit 114 overlaps with an obstacle in the surroundings acquired by the surrounding situation providing unit 102. If an obstacle such as a step or a hole overlaps with the fixation point acquired by the line-of-sight determination unit 113, the target person has already recognized the obstacle, so the risk determination unit 116 excludes the visually recognized obstacle from the warning objects. If the obstacle has already been visually recognized, for example, it means that the fixation point has overlapped with the same obstacle for a period longer than or equal to a predetermined set time. Overlapping with something can mean not only completely overlapping with something but also partially overlapping with something.

[0032] In addition, when judging whether the obstacle is a warning object, the risk determination unit 116 may refer to the ambient noise obtained by the noise detection unit 103. It is well known that in a noisy environment, attention in a person's visual field deteriorates (for example, the effective visual field narrows). When the noise level obtained based on the noise signal D3 output from the noise detection unit 103 exceeds a noise target value as a previously set threshold, the risk determination unit 116 may consider not only obstacles in the central visual field of the target person but also obstacles present in the peripheral visual field of the target person as warning objects, regardless of the position of the fixation point.

[0033] Furthermore, when judging whether the obstacle is a warning object, the risk determination unit 116 may refer to the emotion level D15 output by the emotion determination unit 115. It is known that when the person's emotional state is anger, impatience, or the like, attention and cognitive abilities deteriorate compared to normal times. Furthermore, the person's emotion level may be determined based on the person's voice. Thus, the risk determination unit 116 may consider obstacles in the movement anticipation area as warning objects, regardless of the position of the fixation point, when the emotion level D15 exceeds the previously set threshold level.

[0034] Fig. 2 is a diagram showing an example of the hardware configuration of the warning device 1 and the warning system 10 according to the first embodiment. The warning device 1 is, for example, an information processing device, namely a computer. The warning device 1 includes a processor 15 having a receiving unit 17, a processing unit 18, and a control unit 19, as well as a memory 16 as a volatile storage device. The warning device 1 may include a non-volatile storage device such as a hard disk drive (HDD) or a solid-state drive (SSD) and a communication device that performs communication with external devices. The memory 16 is, for example, a semiconductor memory such as a RAM (Random Access Memory).The warning device 1 may be equipped with a processor that executes the warning program as a software program installed from a recording medium or via a communication line and stored in the memory 16.

[0035] The functions of warning device 1 can be implemented by a processing circuit. The processing circuit can be either dedicated hardware or the processor 15, which executes the warning program stored in memory 16. The processor 15 can be a processing device, an arithmetic device, a microprocessor, a microcomputer, or a DSP (digital signal processor).

[0036] If the processing circuit is dedicated hardware, the processing circuit is, for example, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or similar.

[0037] It is also possible to implement part of the warning device 1 using dedicated hardware and another part using software or firmware. As described above, the processing circuitry can implement the functions described above using hardware, software, firmware, or a combination of some of these means.

[0038] Fig. 3 is a flowchart showing an operation example of the movement anticipation unit 114 of the warning device 1 according to the first embodiment. First, the movement anticipation unit 114 receives the movement information D5 including the position, movement direction, and movement speed of the target person from the person position detection unit 105 (step S101), receives the fluctuation level D12a of the target person from the walking detection unit 112 (step S102), and calculates the movement anticipation range based on the movement information D5 (step S103).

[0039] Subsequently, the motion anticipation unit 114 adjusts the motion anticipation range based on the fluctuation level D12a (step S104). Fig. 4A and Fig. 4B are explanatory diagrams showing the operation of the motion anticipation unit 114 depending on the fluctuation level. For example, the motion anticipation unit 114 enlarges the motion anticipation range with the increase of the fluctuation level, as shown in Fig. 4B, and reduces the movement anticipation range with the decrease of the fluctuation level, as in Fig. 4A shown.

[0040] Subsequently, the motion anticipation unit 114 receives the fixation point D13 from the line-of-sight detection unit 113 (step S105) and adjusts the motion anticipation range based on the distribution range of the fixation point and its width, the position of the fixation point, and the fluctuation level D12a. Fig. 5A and Fig. 5B are explanatory diagrams showing the operation of the motion anticipation unit 114 depending on the distribution of the fixation point. For example, the motion anticipation unit 114 enlarges the motion anticipation range with the increase in the width of the distribution range of the fixation point, as shown in Fig. 5A, and reduces the movement anticipation range with the decrease in the width of the distribution range of the fixation point, as in Fig. 4B shown.

[0041] Subsequently, the movement anticipation unit 114 receives the free space D4 representing the movable region of the person from the free space determination unit 104 (step S107) and adjusts the movement anticipation range based on the movable region (step S108). Fig. 6 is an explanatory diagram showing the operation of the movement anticipation unit 114 depending on the clearance. When the information about the person's movable region based on the clearance D4 output by the clearance determination unit 104 indicates a shape such as a passage (a shape with a small width), it can be inferred that the person is moving along the passage, and thus the movement anticipation unit 114 can reduce the movement anticipation range.

[0042] Fig. 7 is a flowchart showing an operation example of the risk determination unit 116 of the warning device 1 according to the first embodiment. Fig. 8A to 8C are explanatory diagrams showing a judgment as to whether an obstacle is a warning object or not, made by the risk determination unit 116 of the warning device 1.

[0043] The risk determination unit 116 receives the movement anticipation area D14 from the movement anticipation unit 114 (step S201), receives the environmental situation information D2 from the environmental situation providing unit 102 (step S202), receives the foot height level D12b, the fixation point D13, and the emotion level D15 as the state of the target person from the state determination unit 201 (steps S203-S205), receives the noise level from the noise detection unit (step S206), and, when the emotion level is less than or equal to the predetermined threshold level and the noise level is less than or equal to the set noise value (NO in step S207 and NO in step S209), considers obstacles that exist in the movement anticipation area and are not visually recognized as the warning objects, as shown in Fig. 8C (step S211).

[0044] If the emotion level D15 exceeds the predetermined threshold level (YES in step S207), the risk determination unit 116 considers the obstacles present in the movement anticipation area as the warning objects, as shown in Fig. 8A (step S208).

[0045] When the noise represented by the noise signal D3 exceeds the predetermined set noise value (YES in step S209), the risk determination unit 116 regards the obstacles present in the movement anticipation area as the warning objects, as shown in Fig. 8B (step S208).

[0046] As described above, according to the first embodiment, the free space D4 constituting the movable region, the fluctuation level D12a, and the fixation point D13 are used in addition to the information such as the position and movement speed of the person to increase the accuracy of the person's movement anticipation, whereby the accuracy of the person's movement anticipation range can be increased.

[0047] Furthermore, according to the first embodiment, the state of the person (for example, the foot height level D12b, the fixation point D13 and the emotion level D15) and the noise level are used to judge whether the obstacle is a warning object or not, so that obstacles can be appropriately determined as warning objects and the occurrence of excessively frequent warnings can be avoided. Second embodiment

[0048] Fig. Fig. 9 is a block diagram schematically showing the configuration of a warning device 2 and a warning system 20 according to a second embodiment. The warning device 2 is a device capable of executing a warning method according to the second embodiment. Fig. 9 is assigned to each component that is Fig. 1 shown component is the same as or corresponds to the same reference numeral as in Fig. 1. In the warning device 2, a state determination unit 202 includes the walking determination unit 112. The warning device 2 differs from the warning device 1 according to the first embodiment in that it does not have a line-of-sight determination unit and an emotion determination unit. Except for these features, the second embodiment is the same as the first embodiment. Third embodiment

[0049] Fig. Fig. 10 is a block diagram schematically showing the configuration of a warning device 3 and a warning system 30 according to a third embodiment. The warning device 3 is a device capable of executing a warning method according to the third embodiment. Fig. 10 is any component that is connected to a component in Fig. 1 is identical or corresponds to it, the same reference symbol as in Fig. 1. In the warning device 3, a state determination unit 203 includes the walking determination unit 112 and the line-of-sight determination unit 113. The warning device 3 differs from the warning device 1 according to the first embodiment in that it does not have an emotion determination unit. Apart from these features, the third embodiment is the same as the first embodiment. Fourth embodiment

[0050] Fig. Fig. 11 is a block diagram schematically showing the configuration of a warning device 4 and a warning system 40 according to a fourth embodiment. The warning device 4 is a device capable of executing a warning method according to the fourth embodiment. Fig. 11 is assigned to each component that is Fig. 1 shown component is the same as or corresponds to the same reference numeral as in Fig. 1. In the warning device 4, a state determination unit 204 includes the walking determination unit 112 and the emotion determination unit 115. The warning device 4 differs from the warning device 1 according to the first embodiment in that it does not have a line-of-sight determination unit. Apart from these features, the fourth embodiment is the same as the first embodiment. Fifth embodiment

[0051] Fig. Fig. 12 is a block diagram schematically showing the configuration of a warning device 5 and a warning system 50 according to a fifth embodiment. The warning device 5 is a device capable of executing a warning method according to the fifth embodiment. Fig. 12 is any component that is identical to one in Fig. 1 shown component or corresponds to it, the same reference numeral as in Fig. 1. In the warning device 5, a state determination unit 205 includes the line-of-sight determination unit 113. The warning device 5 differs from the warning device 1 according to the first embodiment in that it does not have a walking determination unit and an emotion determination unit. Apart from these features, the fifth embodiment is the same as the first embodiment. Sixth embodiment

[0052] Fig. Fig. 13 is a block diagram schematically showing the configuration of a warning device 6 and a warning system 60 according to a sixth embodiment. The warning device 6 is a device capable of executing a warning method according to the sixth embodiment. Fig. 13 was assigned to each component that is connected to a Fig. 1 is identical or corresponds to the component shown, the same reference numeral as in Fig. 1. In the warning device 6, a state determination unit 206 includes the emotion determination unit 115. The warning device 6 differs from the warning device 1 according to the first embodiment in that it does not have a walking determination unit and a line-of-sight determination unit. Apart from these features, the sixth embodiment is the same as the first embodiment. Seventh embodiment

[0053] Fig. Fig. 14 is a block diagram schematically showing the configuration of a warning device 7 and a warning system 70 according to a seventh embodiment. The warning device 7 is a device capable of executing a warning method according to the seventh embodiment. Fig. 14 is assigned to each component that is Fig. 1 shown component is the same as or corresponds to the same reference numeral as in Fig. 1. In the warning device 7, a state determination unit 207 includes the line-of-sight determination unit 113 and the emotion determination unit 115. The warning device 7 differs from the warning device 1 according to the first embodiment in that it does not have a walking determination unit. Except for these features, the seventh embodiment is the same as the first embodiment. DESCRIPTION OF REFERENCE SYMBOLS

[0054] 1 - 7: Warning device, 10, 20, 30, 40, 50, 60, 70: Warning system, 101: Environmental sensor, 102: Environmental situation providing unit, 103: Sound detection unit, 104: Clear space detection unit, 105: Person position detection unit, 111: Person sensor, 112: Walking detection unit, 113: Line of sight detection unit, 114: Movement anticipation unit, 115: Emotion detection unit, 116: Risk detection unit, 117: Warning issuing device, 118: Actuator, 201- 207: Condition detection unit. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] 2021-76934

[0003]

Claims

[1] Warning device, comprising: a person position detecting unit for detecting movement information including a position, a moving direction, and a moving speed of a target person based on a first detection signal output from an environmental sensor that performs scanning with respect to an environment of the target person; a movement anticipation unit for determining a movement anticipation area as an area through which the target person is anticipated to move, based on the movement information; an environmental situation providing unit for providing environmental situation information including a position of an obstacle based on at least one of the first detection signal and previously acquired map information; a state detecting unit for detecting a state of the target person based on a second detection signal output from a person sensor that senses at least one of the state and the voice of the target person; and a risk determination unit for making a judgment as to whether the obstacle is a warning object or not based on the surrounding situation information, the movement anticipation range and the condition of the target person, and for issuing a warning signal if the obstacle is a warning object, wherein the motion anticipation unit adjusts the motion anticipation range based on the state of the target person. [2] Warning device according to claim 1, wherein the state determining unit comprises a walking determining unit that determines, on the basis of the second detection signal, a fluctuation level representing the degree of fluctuation of the target person when walking, and the movement anticipation unit expands the movement anticipation range with an increase in the fluctuation level. [3] Warning device according to claim 2, wherein the running detection unit determines a foot height level of the target person while running based on the second detection signal, and the risk assessment unit makes the assessment based on the environmental situation information, the movement anticipation range and the foot height level of the target person. [4] Warning device according to claim 1, wherein the state detection unit comprises a walking detection unit that detects a foot height level of the target person when walking based on the second detection signal, and the risk assessment unit makes the assessment based on the environmental situation information, the movement anticipation range and the foot height level of the target person. [5] Warning device according to one of claims 1 to 4, wherein the state determination unit comprises a line-of-sight determination unit that determines a fixation point indicating a position at which the target person is looking based on the second detection signal, and the movement anticipation unit narrows the movement anticipation area with a decrease in the degree of spread of the fixation point distribution. [6] A warning device according to claim 5, wherein the risk determination unit does not regard the obstacle intersecting with the fixation point as a warning object when making the judgment. [7] Warning device according to one of claims 1 to 4, wherein the state determination unit comprises a line-of-sight determination unit that determines a fixation point indicating a position at which the target person is looking based on the second detection signal, and the risk assessment unit considers the obstacle that overlaps with the fixation point as not a warning object when making the assessment. [8] Warning device according to one of claims 1 to 5, wherein the second detection signal comprises a voice signal based on the voice of the target person, the state determination unit has an emotion determination unit that determines an emotion level indicating a degree of excitement of the target person based on the voice signal, and If the emotion level exceeds a preset threshold level, the risk assessment unit considers all obstacles in the movement anticipation area as warning objects. [9] Warning device according to claim 6 or 7, wherein the second detection signal comprises a voice signal based on the voice of the target person, the state determination unit has an emotion determination unit that determines an emotion level indicating a degree of excitement of the target person based on the voice signal, and When the emotion level exceeds a preset threshold level, the risk detection unit considers all obstacles in the movement anticipation area, including the fixation point overlapping with the obstacle, as warning objects. [10] The warning device according to any one of claims 1 to 9, further comprising a sound detection unit for detecting a sound signal in the second detection signal, wherein the risk determination unit determines the warning objects based on a sound level represented by the sound signal, the environmental situation information, the movement anticipation range, the state of the target person and the sound level. [11] The warning device according to any one of claims 1 to 10, further comprising a free space determining unit that determines, based on the first detection signal, a free space representing a region in which the target person can move, wherein the movement anticipation unit adjusts the movement anticipation range based on the free space. [12] Warning system, comprising: the environmental sensor; the person sensor; the warning device according to one of claims 1 to 11; and a warning issuing device for issuing a warning to the target person based on the warning signal. [13] Warning system according to claim 12, wherein the person sensor comprises a wearable sensor attached to the target person, and the environmental sensor has a camera or a LiDAR. [14] Warning procedure to be carried out by a warning device, comprising: a step of acquiring movement information including a position, a movement direction, and a movement speed of a target person based on a first detection signal output from an environment sensor that performs scanning with respect to an environment of the target person; a step of determining a movement anticipation area as an area through which the target person is anticipated to move, based on the movement information; a step of providing environmental situation information including a position of an obstacle based on at least one of the first detection signal and previously acquired map information; a step of determining a state of the target person based on a second detection signal output from a person sensor that senses at least one of the state and the voice of the target person; and a step of making a judgment as to whether the obstacle is a warning object or not based on the environmental situation information, the movement anticipation range and the condition of the target person, and issuing a warning signal if the obstacle is a warning object, wherein the movement anticipation area is adjusted based on the state of the target person in the step of anticipating the movement anticipation area. [15] Warning program that causes a computer to execute: a step of acquiring movement information including a position, a movement direction, and a movement speed of a target person based on a first detection signal output from an environment sensor that performs scanning with respect to an environment of the target person; a step of determining a movement anticipation area as an area through which the target person is anticipated to move, based on the movement information; a step of providing environmental situation information including a position of an obstacle based on at least one of the first detection signal and previously acquired map information; a step of determining a state of the target person based on a second detection signal output from a human sensor that senses at least one of the state and the voice of the target person; a step of making a judgment as to whether the obstacle is a warning object or not based on the surrounding situation information, the movement anticipation range, and the state of the target person, and outputting a warning signal if the obstacle is a warning object; and a step of adjusting the movement anticipation area based on the state of the target person when the movement anticipation area is anticipated.

Citation Information

Patent Citations

  • Dangerous action prevention device, dangerous action prevention system, dangerous action prevention method, and dangerous action prevention program

    JP2021076934A

  • vehicle motion control device

    DE112005003266T5

  • JP002021076934A

  • Apparatus for preventing fall

    KR1020160015463A

  • Autonomous vehicle providing safety zone to persons in distress

    WO2018017060A1