Watching system, watching device, watching method and watching program

Abstract

To watch over a watching target efficiently.SOLUTION: When a millimeter wave sensor 10 detects a person (S1), the millimeter wave sensor transmits a detection signal to a management device 20 (S2). The management device 20 calculates a moving speed and a moving acceleration of the detected person using a received detection signal, and uses this calculated value and a threshold value to determine whether the detected person is a resident (S3). The management device 20 determines whether the detected person who has been determined to be the resident is in a dangerous state based on her behavior (S4). If the management device determines that the resident is in the dangerous state, the management device transmits an abnormality notification to an employee terminal 40 (S5).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a monitoring system, a monitoring device, a monitoring method, and a monitoring program that can efficiently monitor a target to be monitored. [Background technology]

[0002] In the past, nursing care facilities and other facilities needed to quickly detect when residents were falling over or out of bed or wheelchairs. They also needed to understand the health status of residents by detecting their daily activities, such as whether they were lying down, getting out of bed, walking, entering or leaving the room.

[0003] For this reason, monitoring systems for monitoring the status of residents are known. Infrared light is generally used as a detection means in these systems because it is necessary to be able to detect at night, and a TOF (Time Of Flight) distance image sensor (TOF sensor) capable of acquiring depth images may also be used to determine dangerous situations such as tripping or falling (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6708980 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when a TOF sensor is used as in Patent Document 1, if a resident is in a blind spot of the TOF sensor due to furniture, a wheelchair, or the like, the behavior of the resident cannot be detected, and the resident cannot be monitored. Also, because the infrared image of the TOF sensor clearly captures the behavior of the resident, there is a possibility that the privacy of the resident may be compromised. Furthermore, because TOF sensors are expensive, there is also the problem that introducing them to many rooms will result in high costs.

[0006] Therefore, it is an important issue to realize a monitoring system that is not restricted by blind spots caused by the resident's equipment, and that is efficient in terms of protecting the resident's privacy and cost. This issue arises not only for residents of nursing homes, but also when monitoring children in educational facilities and pets kept in animal farms.

[0007] The present invention has been made to solve the problems (issues) of the above-mentioned conventional technology, and aims to provide a monitoring system, monitoring device, monitoring method, and monitoring program that can efficiently monitor the person being monitored. [Means for solving the problem]

[0008] In order to solve the above problem, the present invention provides a monitoring system for monitoring a target located in a specified area, comprising: a millimeter wave sensor attached so as to be directed toward an internal space within the area; a specific processing unit that detects the presence of a moving object and identifies behavioral information of the moving object based on a detection signal from the millimeter wave sensor; a judgment unit that determines whether the moving object is the target to be monitored based on the behavioral information of the moving object identified by the judgment processing unit; and a monitoring control unit that performs monitoring control of the target to be monitored if the judgment unit determines that the moving object is the target to be monitored.

[0009] In the present invention, the identification processing unit identifies the behavior information including a moving speed or a moving acceleration of the moving object based on a detection signal of the millimeter wave sensor.

[0010] In addition, in the above invention, the present invention is characterized in that the identification processing unit identifies the movement trajectory of the moving object or the behavioral information including the stay location and stay time based on the detection signal of the millimeter wave sensor.

[0011] In the present invention, the identification processing unit identifies the behavior information including the walking state or posture of the moving object based on a detection signal from the millimeter wave sensor.

[0012] In addition, in the above invention, the present invention is characterized in that the monitoring control unit includes a danger state determination unit that determines whether the monitored object is in a predetermined danger state based on the behavioral information of the moving body, and an alarm processing unit that issues an alarm when the danger state determination unit determines that the monitored object is in the danger state.

[0013] In addition, the present invention is characterized in that, in the above invention, it further comprises a vital information acquisition unit that acquires vital information of the person being watched over based on the detection signal of the millimeter wave sensor.

[0014] The present invention also provides a monitoring device that monitors a target to be monitored that is located in a specified area, comprising: a millimeter wave sensor attached so as to be directed toward the internal space within the area; a specific processing unit that detects the presence of a moving object and identifies behavioral information of the moving object based on a detection signal from the millimeter wave sensor; a judgment unit that determines whether the moving object is the target to be monitored based on the behavioral information of the moving object identified by the judgment processing unit; and a monitoring control unit that performs monitoring control of the target to be monitored if the judgment unit determines that the moving object is the target to be monitored.

[0015] The present invention also provides a monitoring method for a monitoring system in which a millimeter wave sensor is attached so as to be directed toward an internal space within a predetermined area and the millimeter wave sensor is used to monitor a target to be monitored, and is characterized in that it includes a identification processing step of detecting the presence of a moving object and identifying behavioral information of the moving object based on a detection signal from the millimeter wave sensor, a determination step of determining whether the moving object is the target to be monitored based on the behavioral information of the moving object identified in the identification processing step, and a monitoring control step of performing monitoring control of the target to be monitored if the determination step determines that the moving object is the target to be monitored.

[0016] The present invention also provides a monitoring program executed by a monitoring device that has a millimeter-wave sensor attached so as to be directed toward an internal space within a specified area and that monitors a target to be monitored using the millimeter-wave sensor, and is characterized in that the program causes a computer to execute a specific processing procedure that detects the presence of a moving object and identifies behavioral information of the moving object based on a detection signal from the millimeter-wave sensor, a determination procedure that determines whether the moving object is the target to be monitored based on the behavioral information of the moving object identified in the specific processing procedure, and a monitoring control procedure that performs monitoring control of the target to be monitored if the moving object is determined to be the target to be monitored by the determination procedure. [Effects of the Invention]

[0017] According to the present invention, it is possible to efficiently watch over a person to be watched over. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is an explanatory diagram of an overview of a monitoring system according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating the system configuration of the monitoring system according to the first embodiment. [Figure 3] FIG. 3 is a functional block diagram showing the configuration of the management device shown in FIG. [Figure 4] FIG. 4 is a diagram showing an example of the sensor data, threshold data, detection signal data, and watching over target data shown in FIG. [Figure 5] FIG. 5 is a diagram showing an example of the vital data, walking speed data, and walking acceleration data shown in FIG. [Figure 6] FIG. 6 is a diagram showing the characteristics of a millimeter wave sensor. [Figure 7] FIG. 7 is a diagram illustrating an example of human detection according to the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of the watching target determination according to the first embodiment. [Figure 9] FIG. 9 is a flowchart illustrating a processing procedure related to monitoring control according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a hardware configuration according to the first embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of human detection according to the first modification. [Figure 12] FIG. 12 is a diagram illustrating an example of detection based on a stay location and a time according to the first modification. [Figure 13] FIG. 13 is a diagram (part 1) showing an example of human detection according to the second modification. [Figure 14] FIG. 14 is a diagram (part 2) showing an example of human detection according to the second modification. [Figure 15] FIG. 15 is an explanatory diagram of an overview of a watching system according to the second embodiment. [Figure 16] FIG. 16 is a flowchart illustrating a processing procedure related to monitoring control according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] [Embodiment 1] Hereinafter, a monitoring system, a monitoring device, a monitoring method, and a monitoring program according to the first embodiment will be described in detail with reference to the drawings.

[0020] <Outline of the monitoring system according to the first embodiment> First, an outline of the monitoring system according to the present embodiment 1 will be described. Fig. 1 is an explanatory diagram for explaining the outline of the monitoring system according to the present embodiment 1.

[0021] As shown in Fig. 1, the monitoring system according to the first embodiment detects people using a millimeter wave sensor 10. The millimeter wave sensor 10 is a sensor that detects objects using a frequency of 60 GHz, and has the property of being able to pass through cloth and plastic. This property makes it possible to detect the movements and detailed behavior of a person even if they are wearing clothes.

[0022] When the millimeter wave sensor 10 detects a person (S1), it notifies the management device 20 of a detection signal (S2). The management device 20 uses the received detection signal to calculate the movement speed and movement acceleration of the detected person, and determines whether the person is a resident using the calculated values ​​and a threshold value (S3).

[0023] For example, if two people are detected, and the movement speed or movement acceleration of one of them is above a threshold, it is determined that the person is not a resident. If the movement speed and movement acceleration of the other person (e.g., an employee) are below a threshold, it is determined that the person is a resident. This is because resident targets include elderly people and people with some difficulty walking.

[0024] The management device 20 determines whether the person determined to be a resident is in a dangerous state based on their behavior (S4). If it determines that the person is in a dangerous state, it notifies the employee terminal 40 of an abnormality (S5).

[0025] In this way, the monitoring system of this embodiment 1 is configured to determine whether or not a person detected by a millimeter wave sensor is a resident, and if it is determined that the person is a resident, determine whether or not the resident is in a dangerous state based on their behavior, and if it is determined that the resident is in a dangerous state, send an abnormality notification to the employee terminal, thereby enabling efficient monitoring of the person being monitored.

[0026] <System configuration according to the first embodiment> Next, a description will be given of the system configuration of the monitoring system according to the present embodiment 1. Fig. 2 is a diagram showing the system configuration of the monitoring system according to the present embodiment 1. As shown in Fig. 2, millimeter wave sensors 10 are installed at multiple locations within the facility.

[0027] The millimeter wave sensor 10 is connected to a communication line, which is a network within the facility. A management device 20 and a wireless router 30 are connected to the communication line. The wireless router 30 is connected to an employee terminal 40 via wireless communication such as Wi-Fi (registered trademark) so as to be able to communicate with the employee terminal 40.

[0028] The millimeter-wave sensor 10 is a sensor that detects objects using a frequency of 60 GHz. The millimeter-wave sensor 10 sequentially transmits 60 GHz radio waves in a beam-like shape within its irradiation range and receives the reflected waves. It then calculates the position information (distance, horizontal angle, elevation angle) and speed of each reflected wave, identifies the reflection point, and notifies the management device 20 of a detection signal that includes all reflection points within the irradiation range.

[0029] When the management device 20 receives a detection signal from the millimeter wave sensor 10, it stores the detection signal data and detects the same reflecting object as a person based on the position information of the reflection point included in the detection signal.The management device 20 then assigns a unique person number to the detected person and stores the person in the monitoring target data.

[0030] Furthermore, the management device 20 calculates the pulse rate and respiratory rate from the change in the velocity of the reflection point associated with the person number, and stores them in the vital data in association with the person number.

[0031] The management device 20 also calculates the walking speed of the person from the change in speed of the reflection point associated with the person number, and stores the calculated walking speed data in association with the person number. The management device 20 calculates the walking acceleration for each person number using the walking speed data, and stores the calculated walking acceleration data in association with the person number.

[0032] Furthermore, when the walking speed in the walking speed data is equal to or greater than the speed threshold, the management device 20 deletes the corresponding person number from the watching over target data. When the walking acceleration in the walking acceleration data is equal to or greater than the acceleration threshold, the management device 20 deletes the corresponding person number from the watching over target data.

[0033] In addition, the management device 20 determines whether the person associated with the person number stored in the monitored person data is in a dangerous state using the detection signal data and vital data, and if it determines that the person is in a dangerous state, it notifies the employee terminal 40 of an abnormality.

[0034] The wireless router 30 is a device that communicates with the employee terminal 40 using wireless communication such as Wi-Fi (registered trademark). The wireless router 30 is connected to a communication line within the facility and relays communication between the management device 20 and the employee terminal 40.

[0035] The employee terminal 40 is a mobile terminal such as a smartphone carried by an employee. When the employee terminal 40 receives an abnormality notification from the management device 20, the employee terminal 40 displays the abnormality notification. When displaying the abnormality notification, an alarm sound may be sounded to call attention.

[0036] <Configuration of management device 20> Next, a description will be given of the configuration of the management device 20 shown in Fig. 2. Fig. 3 is a functional block diagram showing the configuration of the management device 20 shown in Fig. 2. As shown in Fig. 3, the management device 20 is connected to a display unit 21 and an input unit 22, and has a communication unit 24, a storage unit 25, and a control unit 26.

[0037] The display unit 21 is a display device such as a liquid crystal panel or a display device. The input unit 22 is an input device such as a keyboard or a mouse. The communication unit 24 is an interface unit for communicating data with the millimeter wave sensor 10 and the wireless router 30 via a communication line within the facility.

[0038] The memory unit 25 is a memory device such as a hard disk drive or non-volatile memory, and stores sensor data 25a, threshold data 25b, detection signal data 25c, monitoring target data 25d, vital data 25e, walking speed data 25f, and walking acceleration data 25g.

[0039] The sensor data 25a is data related to the millimeter wave sensor 10 installed in the facility. The threshold data 25b is data indicating a speed threshold and an acceleration threshold. The detection signal data 25c is data indicating a detection signal received from the millimeter wave sensor 10.

[0040] The vital data 25e is data indicating the pulse rate and respiratory rate of the detected person. The walking speed data 25f is data indicating the walking speed of the detected person. The walking acceleration data 25g is data indicating the walking acceleration of the detected person.

[0041] Control unit 26 is a control unit that performs overall control of management device 20, and includes detection unit 26a, vital information acquisition unit 26b, speed calculation unit 26c, acceleration calculation unit 26d, monitoring target determination unit 26e, dangerous state determination unit 26f, and notification unit 26g. In practice, by loading and executing these programs into a CPU (Central Processing Unit), processes corresponding to detection unit 26a, vital information acquisition unit 26b, speed calculation unit 26c, acceleration calculation unit 26d, monitoring target determination unit 26e, dangerous state determination unit 26f, and notification unit 26g are executed.

[0042] The detection unit 26a is a processing unit that detects people using the detection signal received from the millimeter wave sensor 10. When the detection unit 26a receives a detection signal from the millimeter wave sensor 10, it stores the signal in detection signal data 25c, and detects identical reflecting objects as people based on the position information of the reflection points included in the detection signal. The detection unit 26a then assigns a unique person number to the detected person and stores the person in the monitoring target data 25d. Note that if the detected people are divided into multiple people based on the position information of the reflection points, a unique person number is assigned to each person. For example, the people detected on the same day are sequentially assigned numbers such as "001" and "002," and the numbers are assigned again starting from "001" the next day.

[0043] Furthermore, when the detection unit 26a assigns a person number based on a received detection signal, if the person detected by the next received detection signal is determined to be the same person as the person to whom a person number has already been assigned based on the location information, the detection unit 26a continues to use this person number.

[0044] The vital information acquisition unit 26b is a processing unit that acquires vital information of the person detected by the detection unit 26a. The vital information acquisition unit 26b extracts reflection points associated with the person numbers detected by the detection unit 26a from the detection signal data 25c in time series. The vital information acquisition unit 26b then calculates the pulse rate and respiratory rate from changes in the speed of the extracted reflection points and stores them in vital data 25e in association with the person numbers.

[0045] The speed calculation unit 26c is a processing unit that calculates the walking speed of the person detected by the detection unit 26a. The speed calculation unit 26c extracts reflection points associated with the person number detected by the detection unit 26a in time series from the detection signal data 25c. Then, the speed calculation unit 26c calculates the walking speed of the person associated with the person number from changes in the speed of the extracted reflection points, and stores the calculated walking speed in walking speed data 25f in association with the person number.

[0046] The acceleration calculation unit 26d is a processing unit that calculates the walking acceleration of the person detected by the detection unit 26a. The acceleration calculation unit 26d calculates the walking acceleration for each person number using the walking speed data 25f, and stores the calculated walking acceleration in walking acceleration data 25g in association with the person number.

[0047] The watching target determination unit 26e is a processing unit that determines whether or not the person detected by the detection unit 26a is a resident to be watched over. When the walking speed in the walking speed data 25f is equal to or greater than the speed threshold in the threshold data 25b, the watching target determination unit 26e deletes the number of the person whose walking speed is equal to or greater than the speed threshold from the watching target data 25d.

[0048] Furthermore, when the walking acceleration in the walking acceleration data 25g is equal to or greater than the acceleration threshold of the threshold data 25b, the watching target determination unit 26e deletes the person number whose walking acceleration is equal to or greater than the acceleration threshold from the watching target data 25d.

[0049] The dangerous state determination unit 26f is a processing unit that determines whether or not a resident who is the target of monitoring is in a dangerous state. The dangerous state determination unit 26f determines whether or not a person associated with a person number stored in the monitoring target data 25d is in a dangerous state using the detection signal data 25c and the vital sign data 25e.

[0050] Specifically, reflection points related to the person number of the person being watched are extracted in chronological order from the detection signal data 25c. If the outline of a reflection point extracted at one point in time suddenly becomes smaller than the outline of a reflection point extracted at the next point in time, it is determined that a dangerous situation has occurred. This is because in this case, it is estimated that the person being watched has suddenly crouched down.

[0051] Furthermore, the pulse rate and respiratory rate associated with the person number of the person being watched over are extracted in time series from the vital data 25e, and if the extracted pulse rate or respiratory rate changes suddenly, it is determined that the person is in a dangerous state.

[0052] The notification unit 26g is a processing unit that, when it is determined that a dangerous state exists, notifies the employee terminal 40 of an abnormality notification. When the dangerous state determination unit 26f determines that the person being watched over is in a dangerous state, the notification unit 26g notifies the employee terminal 40 of an abnormality notification.

[0053] Next, an example of data stored in the storage unit 25 of the management device 20 shown in Fig. 3 will be described. Fig. 4 and Fig. 5 are diagrams showing examples of the sensor data 25a, threshold data 25b, detection signal data 25c, watching over target data 25d, vital data 25e, walking speed data 25f, and walking acceleration data 25g shown in Fig. 3.

[0054] The sensor data 25a shown in FIG. 4(a) associates the sensor ID "ABC001" with a state in which the installation location is "lobby," and associates the sensor ID "ABC002" with a state in which the installation location is "dining room."

[0055] The threshold data 25b shown in FIG. 4(b) has a velocity threshold of "1.0 m / s" and an acceleration threshold of "1.3 m / s" 2" indicates a state in which

[0056] The detection signal data 25c shown in Figure 4(c) associates the time "9:20:10.0" with a state in which the horizontal angle is "30" degrees, the elevation angle is "40" degrees, the distance is "3.5" m, and the speed is "0.1" m / s, the time "9:20:10.0" with a state in which the horizontal angle is "31" degrees, the elevation angle is "40" degrees, the distance is "3.5" m, and the speed is "0.1" m / s, and the time "9:20:10.2" with a state in which the horizontal angle is "30" degrees, the elevation angle is "40" degrees, the distance is "3.5" m, and the speed is "0.2" m / s.

[0057] The watching-over target data 25d shown in FIG. 4(d) indicates a state in which the person numbers of the people to be watched over are "002" and "004".

[0058] The vital data 25e shown in Figure 5(a) associates person number "001" with a pulse rate of "60" beats / minute and a respiratory rate of "15" beats / minute, and person number "002" with a pulse rate of "75" beats / minute and a respiratory rate of "18" beats / minute.

[0059] The walking speed data 25f shown in Figure 5(b) associates the person number "001" with a walking speed of "0.8" m / s at time "9:20:00", "0.9" m / s at time "9:20:05", "1.2" m / s at time "9:20:10", and "1.0" m / s at time "9:20:15".

[0060] Furthermore, walking speed data 25f associates person number "002" with a walking speed of "0.5" m / s at time "9:20:00", "0.6" m / s at time "9:20:05", "0.5" m / s at time "9:20:10", and "0.4" m / s at time "9:20:15".

[0061] The walking acceleration data 25g shown in FIG. 5(c) indicates that the walking acceleration of person number "001" at time "9:20:00" is "0.5" m / s 2 and the walking acceleration at time "9:20:05" is "1.0" m / s 2 and the walking acceleration at time "9:20:10" is "1.5" m / s 2 and the walking acceleration at time "9:20:15" is "0.5" m / s 2 The state in which

[0062] In addition, the walking acceleration data 25g indicates that the walking acceleration of person number "002" at time "9:20:00" is "0.2" m / s 2 and the walking acceleration at time "9:20:05" is "0.3" m / s 2 and the walking acceleration at time "9:20:10" is "0.3" m / s 2 and the walking acceleration at time "9:20:15" is "0.2" m / s 2 The state in which

[0063] <Characteristics of the millimeter wave sensor 10> Next, a description will be given of the characteristics of the millimeter wave sensor 10. FIG.

[0064] The millimeter-wave sensor 10 is a sensor that detects objects using a frequency of 60 GHz. The millimeter-wave sensor 10 sequentially transmits 60 GHz radio waves in a beam-like manner within its irradiation range and receives the waves reflected from the reflecting object. It then acquires the position information (distance, horizontal angle, elevation angle) of the reflecting object for each reflected wave. Furthermore, if the reflecting object moves or moves, the Doppler effect causes a change in frequency of the reflected wave, making it possible to calculate the moving speed of the reflecting object.

[0065] 6(a), the millimeter wave sensor 10 emits radio waves with a wavelength of several mm, which have the property of penetrating through cloth, plastic, etc. For example, if a person is wearing a thin blanket or clothing, the millimeter wave sensor 10 can detect the person by penetrating the blanket or clothing.

[0066] 6(b), the millimeter wave sensor 10 can calculate the speed of movement or motion of a reflecting object using the Doppler effect, and can therefore detect even slight movements of a person due to breathing. This characteristic allows the detection of pulse rate and respiratory rate.

[0067] <Example of person detection according to embodiment 1> Next, a description will be given of an example of human detection according to the present embodiment 1. Fig. 7 is a diagram showing an example of human detection according to the present embodiment 1.

[0068] Position information (distance, horizontal angle, elevation angle) is added to the reflection points contained in the detection signal notified by the millimeter wave sensor 10. When the reflection points are arranged based on this position information, the resulting image is as shown in Fig. 7(a). In this figure, it can be seen that there are clusters of reflection points in the upper right and lower left.

[0069] If we visualize the outline of this cluster of reflection points using dashed lines, it can be shown as in Figure 7(b). The detected person is within the frame of these two dashed lines.

[0070] <An example of monitoring target determination according to embodiment 1> Next, a description will be given of an example of the watching target determination according to the present embodiment 1. Fig. 8 is a diagram showing an example of the watching target determination according to the present embodiment 1.

[0071] As shown in Figure 8(a), when the relationship between a person's walking speed and time is represented by a graph, if a data point on the graph exceeds a threshold, the person is excluded from the monitoring targets. For example, person A is excluded from the monitoring targets because his walking speed exceeds the threshold at time t. Person B is included in the monitoring targets because his walking speed is continuously below the threshold.

[0072] As shown in Figure 8(b), when the relationship between a person's walking acceleration and time is represented by a graph, if a data point on the graph exceeds a threshold, the person is excluded from the monitoring targets. For example, person A is excluded from the monitoring targets because his walking acceleration exceeds the threshold at time t. Person B is a monitoring target because his walking acceleration continues to be below the threshold.

[0073] <Processing procedure for monitoring control according to the first embodiment> Next, a description will be given of a processing procedure relating to the monitoring control according to the present embodiment 1. Fig. 9 is a flowchart showing the processing procedure relating to the monitoring control according to the present embodiment 1.

[0074] 9, the management device 20 stores the detection signal received from the millimeter wave sensor 10 in the detection signal data 25c, and detects people and the number of people using the detection signal (step S101). Then, the management device 20 assigns a person number to the detected people.

[0075] Data relating to the person number of the detected person is extracted in time series from the detection signal data 25c, and the walking speed and walking acceleration are calculated using the extracted data relating to the detection signal (step S102).The calculated walking speed is then stored as walking speed data 25f, and the walking acceleration is stored as walking acceleration data 25g.

[0076] Using the walking speed data 25f and the walking acceleration data 25g, it is determined whether or not to exclude the person assigned the person number from the monitoring targets (step S103). Specifically, if the walking speed in the walking speed data 25f is equal to or greater than the speed threshold, or if the walking acceleration in the walking acceleration data 25g is equal to or greater than the acceleration threshold, the corresponding person is excluded from the monitoring targets.

[0077] If it is determined that the person assigned the person number should be excluded from the monitoring targets (step S103; Yes), the process proceeds to step S101. If it is not determined that the person assigned the person number should be excluded from the monitoring targets (step S103; No), the monitoring of the person assigned the person number continues (step S104).

[0078] Using the detection signal data 25c and the vital data 25e, it is determined whether or not the person associated with the person number is in a dangerous state (step S105). If it is determined that the person associated with the person number is not in a dangerous state (step S105; No), the process proceeds to step S103.

[0079] If it is determined that the person associated with the person number is in a dangerous state (step S105; Yes), an abnormality notification is sent to the employee terminal 40 (step S106), and the process ends.

[0080] <Example of hardware configuration according to the first embodiment> Next, a description will be given of the correspondence between the management device 20 of the watching system according to the present embodiment and the main hardware configuration of a computer. Fig. 10 is a diagram showing an example of the hardware configuration according to the present embodiment 1.

[0081] Generally, a computer has a configuration in which a CPU 81, a ROM 82, a RAM 83, a nonvolatile memory 84, etc. are connected via a bus 85. A hard disk drive may be provided instead of the nonvolatile memory 84. For the sake of convenience of explanation, only the basic hardware configuration is shown.

[0082] Here, the ROM 82 or non-volatile memory 84 stores programs required to start the operating system (hereinafter simply referred to as "OS"), and the CPU 81 reads and executes the OS program from the ROM 82 or non-volatile memory 84 when the power is turned on.

[0083] On the other hand, various application programs executed on the OS are stored in non-volatile memory 84, and the CPU 81 executes the application programs while using RAM 83 as the main memory, thereby executing processes corresponding to the applications.

[0084] The monitoring program of the management device 20 of the monitoring system according to the first embodiment is also stored in the nonvolatile memory 84 or the like, like other application programs, and the CPU 81 loads and executes this program. In the case of the management device 20 of the monitoring system according to the first embodiment, a monitoring program including routines corresponding to the detection unit 26a, vital information acquisition unit 26b, speed calculation unit 26c, acceleration calculation unit 26d, monitoring target determination unit 26e, dangerous state determination unit 26f, and notification unit 26g shown in Fig. 3 is stored in the nonvolatile memory 84 or the like. When the CPU 81 loads and executes the monitoring program, monitoring processes corresponding to the detection unit 26a, vital information acquisition unit 26b, speed calculation unit 26c, acceleration calculation unit 26d, monitoring target determination unit 26e, dangerous state determination unit 26f, and notification unit 26g are generated.

[0085] As described above, the monitoring system according to the first embodiment is configured to determine whether a person detected by a millimeter wave sensor is a resident, and if it is determined that the person is a resident, determine whether the resident is in a dangerous state based on their behavior, and if it is determined that the resident is in a dangerous state, send an abnormality notification to an employee terminal, thereby enabling efficient monitoring of the person being monitored.

[0086] In the first embodiment, a configuration in which a person is detected using a millimeter wave sensor has been described, but the present invention is not limited to this. A configuration in which a person is detected using both a millimeter wave sensor and a TOF sensor is also possible.

[0087] In the first embodiment, the walking speed and walking acceleration are calculated from the detection signal of the millimeter wave sensor to determine whether or not the person is a resident, but the present invention is not limited to this. It is also possible to calculate the walking speed, walking acceleration, walking distance, and posture from the detection signal of the infrared 3D sensor to determine whether or not the person is a resident.

[0088] In addition, in the above-described first embodiment, a configuration has been described in which walking speed and walking acceleration are calculated from the detection signal of the millimeter wave sensor to determine whether or not a person is a resident, but the present invention is not limited to this.Vital information acquired while the resident is lying in bed can be stored in advance, and the vital information of the person acquired by the millimeter wave sensor can be compared with the previously stored vital information to determine whether or not the person is a resident.

[0089] [Variation 1] In the first embodiment, the walking speed and walking acceleration are calculated from the detection signal of the millimeter wave sensor to determine whether or not the person is a resident, but the present invention is not limited to this. It is also possible to detect the movement trajectory, location and time of stay of a person from the detection signal of the millimeter wave sensor to determine whether or not the person is a resident.

[0090] In this first modification, a monitoring system will be described that detects a person's movement trajectory, place of stay, and time from the detection signal of a millimeter wave sensor to determine whether or not the person is a resident.

[0091] <Example of person detection according to Modification 1> An example of human detection according to Modification 1 will be described below. Fig. 11 is a diagram showing an example of human detection according to Modification 1.

[0092] As shown in Figure 11, the monitoring system of this variant example 1 uses detection signals from millimeter wave sensors 10 installed at various locations within the facility to detect a person's movement trajectory, location, and time of stay, and uses the detection results to determine whether or not the person is a resident.

[0093] For example, person C is determined not to be a resident because his / her movement path from the hallway to the staff room was detected. Person D is determined to be a resident because his / her movement path from Room 101 to the hallway was detected. Person E is determined to be a resident because he / she has not moved from the chair installed in the recreation room for a predetermined time (for example, 5 minutes) or more.

[0094] Next, a description will be given of an example of detection based on a place of stay and a time according to Modification 1. Fig. 12 is a diagram showing an example of detection based on a place of stay and a time according to Modification 1.

[0095] As shown in Figure 12, there are places within the facility where residents can sit or lie down, so setting these places and times can be used as criteria for determining whether or not a person is a resident.

[0096] For example, the criterion for judgment is whether or not someone has stayed in a place where a sofa is located in a room for 5 minutes or more, and whether or not someone has stayed in a place where a bed is located for 10 minutes or more.

[0097] As described above, the monitoring system of this variant example 1 is configured to detect a person's movement trajectory, location, and time of stay from the detection signal of the millimeter wave sensor and determine whether or not the person is a resident, thereby enabling efficient monitoring of the person being monitored.

[0098] [Variation 2] In the first embodiment, the walking speed and walking acceleration are calculated from the detection signal of the millimeter wave sensor to determine whether or not the person is a resident, but the present invention is not limited to this. It is also possible to configure the system to detect the posture of a person while walking from the detection signal of the millimeter wave sensor to determine whether or not the person is a resident.

[0099] In this second modification, a monitoring system will be described that detects the posture of a person while walking from a detection signal of a millimeter wave sensor and determines whether or not the person is a resident.

[0100] <Example of person detection according to Modification 2> A description will be given of an example of human detection according to Modification 2. Figures 13 and 14 are diagrams showing an example of human detection according to Modification 2.

[0101] In the monitoring system according to the present modification 2, a detection signal relating to a walking person is acquired by the millimeter wave sensor 10. The reflection points of this detection signal are plotted, and the outline of the signal is detected to estimate the posture of the walking person.

[0102] For example, in the case of the person shown in Figure 13(a), the contour of the reflection points of the detection signal is almost vertical, and the person is not in a bent posture, so it is determined that the person is not a resident. In the case of the person shown in Figure 13(b), the contour of the reflection points of the detection signal is bent diagonally from the center, and it is estimated that the person is in a bent posture, so it is determined that the person is a resident.

[0103] In addition, the reflection points of the detected signals are plotted, and the skeleton of a person is estimated within the cluster of reflection points. From this skeleton state, the posture of a person walking can also be estimated.

[0104] For example, the person shown in Figure 14(a) is determined not to be a resident because the estimated skeleton is almost vertical, while the person shown in Figure 14(b) is determined to be a resident because the estimated skeleton is bent at the waist.

[0105] As described above, the monitoring system of this variant example 2 is configured to detect a person's walking posture from the detection signal of the millimeter wave sensor and determine whether or not the person is a resident, thereby enabling efficient monitoring of the person being monitored.

[0106] In the above-described second modification, a configuration has been described in which a person's posture while walking is detected by plotting reflection points of a detection signal from a millimeter wave sensor and detecting the outer shape or estimating the person's skeleton. However, the present invention is not limited to this. A configuration may also be used in which a person's posture while walking is detected by estimating the person's back muscles, lateral sway, and stride length from the situation in which the reflection points of the detection signal from the millimeter wave sensor are plotted. A configuration may also be used in which a person's posture while walking is estimated using a model that has been machine-learned from the situation in which the reflection points of the detection signal from the millimeter wave sensor are plotted.

[0107] [Embodiment 2] In the first embodiment, a configuration is described in which it is determined whether a resident is in danger based on the detection signal of the millimeter wave sensor, but the present invention is not limited to this. When determining whether a resident is in danger using an image captured by a camera, there is a risk that it will not be possible to determine whether a resident is in danger if the resident is in the shade of something.

[0108] Millimeter-wave sensors have the ability to penetrate cloth and plastic and are cheaper than cameras, so it is possible to configure a system using both cameras and millimeter-wave sensors to determine whether a resident is in danger.

[0109] In this second embodiment, we will explain a monitoring system that determines whether a resident is in danger or not using images captured by a camera and detection signals from a millimeter-wave sensor. This monitoring system is suitable for cases where detection accuracy is more important than protecting the privacy of the resident and cost.

[0110] <Outline of the monitoring system according to the second embodiment> An outline of the monitoring system according to the second embodiment will be described below. Fig. 15 is an explanatory diagram showing the outline of the monitoring system according to the second embodiment.

[0111] As shown in FIG. 15, the monitoring system according to the second embodiment estimates the posture of a resident using a camera 90 and a millimeter wave sensor 10, and determines whether or not the resident is in a dangerous state.

[0112] Specifically, the posture of the resident is estimated using the image captured by the camera 90, but if the posture of the resident cannot be estimated properly because the resident is in a shadow, the posture of the resident is estimated using the detection signal from the millimeter wave sensor 10.

[0113] For example, if a resident sitting in a wheelchair falls off the wheelchair for some reason and enters the blind spot of the camera 90, making it impossible to estimate the resident's posture, the posture of the resident can be estimated using the detection signal from the millimeter wave sensor 10. This is because the millimeter wave sensor 10 can detect the resident by passing through the cloth and plastic parts of the wheelchair.

[0114] <Processing procedure for monitoring control according to the second embodiment> Next, a description will be given of a processing procedure relating to the monitoring control according to the present embodiment 2. Fig. 16 is a flowchart showing a processing procedure relating to the monitoring control according to the present embodiment 2.

[0115] 16, in the monitoring system according to the second embodiment, the posture of the resident is estimated using the video captured by the camera 90 (step S201). Once this posture estimation is completed (step S202; Yes), it is determined whether the resident is in danger (step S204).

[0116] If the posture estimation in step S201 has not been completed (step S202; No), the posture of the resident is estimated using the detection signal from the millimeter wave sensor 10 (step S203), and it is determined whether the resident is in a dangerous state (step S204).

[0117] If it is determined that the resident is not in danger (step S204; No), the process ends. If it is determined that the resident is in danger (step S204; Yes), an abnormality notification is issued (step S205), and the process ends.

[0118] As described above, the monitoring system according to the second embodiment is configured to determine whether a resident is in danger or not using the video captured by the camera and the detection signal from the millimeter wave sensor, thereby enabling efficient monitoring of the person being monitored.

[0119] In the second embodiment, a configuration is described in which an image captured by a camera and a detection signal from a millimeter-wave sensor are used to determine whether a resident is in danger, but the present invention is not limited to this. The camera and millimeter-wave sensor may be installed in the same housing or in separate housings. If they are installed in separate housings, the camera and millimeter-wave sensor may be installed in close proximity or at separate locations.

[0120] Furthermore, the configurations illustrated in the above embodiments are merely functional schematics and do not necessarily have to be physically configured as shown. In other words, the distribution and integration of each device is not limited to that illustrated, and all or part of the devices can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. [Industrial Applicability]

[0121] The monitoring system, monitoring device, monitoring method, and monitoring program according to the present invention are suitable for efficiently monitoring a target to be monitored. [Explanation of symbols]

[0122] 10, 10a, 10b, 10c, 10d, 10e Millimeter wave sensor 20 Management device 21 Display section 22 Input section 24 Communications Department 25 Memory section 25a Sensor Data 25b Threshold Data 25c Detection signal data 25d Data on monitored individuals 25e Vital Data 25f Walking speed data 25g walking acceleration data 26 Control Unit 26a Detector 26b Vital information acquisition unit 26c Speed ​​calculation section 26d Acceleration calculation section 26e Monitoring target determination unit 26f Dangerous condition determination unit 26g Notification section 30 Wireless Router 40 employee terminals 81 CPU 82 ROM 83 RAM 84 Non-volatile memory 85 Bus 90 Camera

Claims

1. A monitoring system that monitors a monitoring target located in a predetermined area, a millimeter wave sensor attached so as to be directed to an interior space within the region; an identification processing unit that detects the presence of a moving object and identifies behavior information of the moving object based on a detection signal from the millimeter wave sensor; a determination unit that determines whether the moving object is the target of monitoring based on the behavior information of the moving object identified by the identification processing unit; a monitoring control unit that performs monitoring control of the target to be monitored if the determination unit determines that the moving object is the target to be monitored; A monitoring system comprising:

2. The specific processing unit The monitoring system according to claim 1, wherein the behavior information including the moving speed or the moving acceleration of the moving object is identified based on the detection signal of the millimeter wave sensor.

3. The specific processing unit The monitoring system according to claim 1, characterized in that the behavioral information including the movement trajectory of the moving object or the place and duration of stay is identified based on the detection signal of the millimeter wave sensor.

4. The specific processing unit The monitoring system according to claim 1, wherein the behavioral information including the walking state or posture of the mobile object is identified based on a detection signal from the millimeter wave sensor.

5. The monitoring control unit a danger state determination unit that determines whether the target of monitoring is in a predetermined danger state based on the behavior information of the moving object; a notification processing unit that issues a notification when the dangerous state determination unit determines that the dangerous state exists; The monitoring system according to any one of claims 1 to 4, characterized in that it comprises:

6. The monitoring system according to claim 1, further comprising a vital information acquisition unit that acquires vital information of the monitored subject based on the detection signal of the millimeter wave sensor.

7. A monitoring device that monitors a monitoring target located in a predetermined area, a millimeter wave sensor attached so as to be directed to an interior space within the region; an identification processing unit that detects the presence of a moving object and identifies behavior information of the moving object based on a detection signal from the millimeter wave sensor; a determination unit that determines whether the moving object is the target of monitoring based on the behavior information of the moving object identified by the identification processing unit; a monitoring control unit that performs monitoring control of the target to be monitored if the determination unit determines that the moving object is the target to be monitored; A monitoring device comprising:

8. A monitoring method in a monitoring system in which a millimeter wave sensor is attached so as to be directed toward an internal space within a predetermined area, and a target to be monitored is monitored using the millimeter wave sensor, an identification processing step of detecting the presence of a moving object and identifying behavior information of the moving object based on a detection signal from the millimeter wave sensor; a determination step of determining whether the moving object identified in the identification processing step is the monitoring target based on the behavior information of the moving object; a monitoring control step of performing monitoring control of the object to be monitored if the moving object is determined to be the object to be monitored by the determination step; A monitoring method comprising:

9. A monitoring program executed by a monitoring device that monitors a target to be monitored using a millimeter wave sensor attached so as to be directed toward an internal space within a predetermined area, an identification process step of detecting the presence of a moving object and identifying behavior information of the moving object based on a detection signal from the millimeter wave sensor; a determination step of determining whether or not the moving object identified in the identification processing step is the monitoring target based on the behavior information of the moving object; a monitoring control procedure for performing monitoring control of the object to be monitored if the moving object is determined to be the object to be monitored by the determination procedure; A monitoring program characterized by causing a computer to execute the above.

Images