Monitoring device and monitoring system

Abstract

This technology provides a method for detecting abnormal conditions using millimeter-wave sensors. [Solution] A monitoring device 100 is provided, comprising a millimeter-wave sensor 120 and a control unit 110 that determines whether or not there is an abnormality based on the data received from the millimeter-wave sensor. The control unit stops determining whether or not there is an abnormality when it recognizes a predetermined gesture from the data received from the millimeter-wave sensor.

Description

Technical Field

[0001] The present invention relates to a technology for monitoring the elderly, disabled people, and the like.

Background Art

[0002] Conventionally, technologies for monitoring the elderly, disabled people, and the like in various places have been known. For example, Japanese Patent Application Laid-Open No. 2021-149923 (Patent Document 1) discloses a monitoring device, a monitoring system, a monitoring method, and a program. According to Patent Document 1, in the monitoring system, the monitoring device includes a sensor information acquisition unit that acquires sensor information detected by a sensor provided in a toilet space, and an abnormality estimation unit that estimates the degree of abnormality in a monitoring target person in the toilet space using the sensor information and a learned model. The learned model is a model that estimates the degree of abnormality in a monitoring target person in the space corresponding to the input sensor information from the input sensor information, and the toilet space is formed by a first space that is a space provided in front of the entrance to the toilet seat and a second space that is a space provided with the toilet seat. At least one sensor is provided in each of the first space and the second space.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present invention is to provide a technology for detecting an abnormal state using a millimeter-wave sensor.

Means for Solving the Problems

[0005] According to one aspect of this invention, a monitoring device is provided comprising a millimeter-wave sensor and a control unit that determines whether or not there is an abnormality based on data received from the millimeter-wave sensor, wherein the control unit stops determining whether or not there is an abnormality when it recognizes a predetermined gesture from the data received from the millimeter-wave sensor. [Effects of the Invention]

[0006] As described above, the present invention provides a technology for detecting abnormal conditions using a millimeter-wave sensor. [Brief explanation of the drawing]

[0007] [Figure 1] This is an illustrative diagram showing the overall configuration and operation overview of the monitoring system 1 according to the first embodiment. [Figure 2] This is an illustrative diagram showing a room in which the monitoring device according to the first embodiment is installed. [Figure 3] This is an illustrative diagram showing what happens when the person in charge according to the first embodiment performs the first gesture. [Figure 4] This is an illustrative diagram showing what happens when the person in charge according to the first embodiment performs the second gesture. [Figure 5] This is a block diagram showing the configuration of the monitoring device according to the first embodiment. [Figure 6] This is a flowchart illustrating the information processing of the monitoring device according to the first embodiment. [Figure 7] This is a block diagram showing the server configuration according to the first embodiment. [Figure 8] This is an illustrative diagram showing the device information data according to the first embodiment. [Figure 9] This is a flowchart illustrating the information processing of the server according to the second embodiment. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below with reference to the drawings. In the following description, identical parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions of them will not be repeated. <First Embodiment> <Overall configuration and operation overview of monitoring system 1>

[0009] First, the overall configuration of the monitoring system 1 according to this embodiment will be described with reference to Figure 1. The monitoring system 1 according to this embodiment mainly consists of monitoring devices 100 installed on the ceiling or walls of each room where the person requiring care stays, a server 300 that acquires the detection results from each monitoring device 100, 100..., and communication terminals 500, 500... held by the caregivers corresponding to each monitoring device 100, 100....

[0010] Each of the monitoring devices 100 according to this embodiment is placed in a room in a nursing home or hospital, as shown in Figure 2. Each monitoring device 100 uses a millimeter-wave sensor to recognize what kind of actions the person in the room is taking. For example, based on the received data acquired using the millimeter-wave sensor, the monitoring device 100 recognizes whether a person requiring care in the room has fallen or is crouching, and determines whether the state is abnormal or normal. The monitoring device 100 uploads the recognition results and judgment results to a predetermined server 300 via an access point such as a router 200, the internet 400, or a carrier network.

[0011] Server 300 acquires recognition results from each monitoring device 100 and recognizes that an abnormality has occurred in the person requiring care. Server 300 notifies the communication terminal 500 of the person in charge of the person requiring care or the communication terminal 500 of the facility manager, etc., of the abnormality and its location. Server 300 is implemented by multiple devices on the cloud. The communication terminal 500 can be a smartphone, tablet, personal computer, speaker, etc., and can communicate with Server 300 via the internet or carrier network.

[0012] In this embodiment in particular, as shown in Figure 3, when a person in charge of a facility where the monitoring device 100 is installed performs a predetermined gesture, the monitoring device 100 interrupts its detection and judgment of abnormalities. For example, when a person in charge is cleaning a room, they make an "X" shape with both arms towards the monitoring device 100 on the ceiling of the room. As a result, the monitoring device 100 recognizes the "X" shape and interrupts its normal processing, such as detecting and judging abnormalities.

[0013] Then, when cleaning is finished and the normal monitoring mode is to begin, as shown in Figure 4, the person in charge makes a circular shape with both arms towards the monitoring device 100 on the ceiling of the room. The monitoring device 100 recognizes this circular shape and resumes normal processing such as detecting and judging abnormalities.

[0014] The following details the specific configuration of monitoring system 1, which is designed to implement these functions. <Configuration of monitoring device 100>

[0015] Referring to Figure 5, one embodiment of the monitoring device 100 that constitutes the monitoring system 1 will be described. For example, the monitoring device 100 includes, as its main components, a CPU 110, a memory 115, a millimeter-wave sensor 120, a status display unit 130, a first abnormality detection warning unit 131, a second abnormality detection warning unit 132, a data communication interface 160, a wireless antenna 161, a power supply 150, and an operation user interface unit 140.

[0016] The CPU 110 controls each part of the monitoring device 100 by executing a program stored in a memory (not shown). For example, in the present embodiment, the CPU 110 realizes the millimeter-wave analysis unit 111 by executing a control program of the memory. For example, the CPU 110 realizes the abnormal behavior determination unit 112 and the gesture determination unit 113 by executing a control program of the memory.

[0017] The memory 115 is realized by various RAMs, various ROMs, and the like. The memory 115 stores various information such as the control program of the CPU 110, data generated by the execution of the program, data received from the server 300, and the like.

[0018] The millimeter-wave sensor 120 includes a plurality of millimeter-wave oscillation antennas and a plurality of millimeter-wave reception antennas. Then, in response to a command from the CPU 110, while outputting millimeter waves of different wavelengths from the plurality of millimeter-wave oscillation antennas, the millimeter waves received by the plurality of millimeter-wave reception antennas are passed to the CPU 110. In the present embodiment, in order to recognize that the care recipient has fallen or the gesture of the caregiver, etc., the millimeter-wave sensor 120 includes about 30 to 70 millimeter-wave oscillation antennas with different frequencies from each other and about 10 to 50 millimeter-wave reception antennas.

[0019] Thereby, the CPU 110 as the abnormal behavior determination unit 112 determines whether or not an abnormal situation has occurred with respect to the care recipient based on signals with different frequencies received by the plurality of reception antennas of the millimeter-wave sensor 120 in the monitoring mode. For example, it recognizes that the care recipient has fallen, is hunched over, or is vomiting.

[0020] The CPU 110, acting as a gesture discrimination unit 113, determines in monitoring mode whether a predetermined first gesture has been recognized based on signals of different frequencies received by multiple receiving antennas of the millimeter-wave sensor 120. When the gesture discrimination unit 113 recognizes the first gesture, it interrupts the monitoring mode.

[0021] In the interruption mode, the gesture discrimination unit 113 determines whether a predetermined second gesture has been recognized based on signals of different frequencies received by multiple receiving antennas of the millimeter-wave sensor 120. If the gesture discrimination unit 113 recognizes the second gesture, it resumes the monitoring mode.

[0022] The status display unit 130 displays images and text based on data from the CPU 110.

[0023] The user interface unit 140 consists of a pointing device and switches, and inputs various commands from the user to the CPU 110. The monitoring device 100 may also have a touch panel that includes a status display unit 130 and the user interface unit 140.

[0024] The first anomaly detection and warning unit 131 is implemented by a speaker or the like. Based on a signal from the CPU 110, it outputs an audio message indicating that an anomaly has occurred.

[0025] The second abnormality detection and warning unit 132 is implemented by an LED light or the like. Based on a signal from the CPU 110, it emits light indicating that an abnormality has occurred.

[0026] The communication interface 160 and wireless antenna 161 send and receive data with other devices such as the server 300 via the router 200, the internet 400, and the carrier network. For example, the CPU 110 sends information to the server 300 indicating that there is an abnormality in the person requiring care, that a staff member is performing a predetermined gesture, or that a mode has been switched. <Information processing in monitoring device 100>

[0027] Next, with reference to Figure 6, the information processing of the monitoring device 100 according to this embodiment will be described. The CPU 110 of the monitoring device 100 periodically performs the following processing according to the program in the memory 115.

[0028] First, the CPU 110 outputs millimeter waves of a specific frequency from multiple transmitting antennas of the millimeter-wave sensor 120, and analyzes the received millimeter waves by acquiring the reflected waves with multiple receiving antennas (step S102). Based on the data received by the multiple receiving antennas, the CPU 110 creates two-dimensional image data that allows for the recognition of the object's position.

[0029] The CPU 110 determines whether the 2D data created by the millimeter-wave analysis contains a specific gesture for interrupting the anomaly detection process (step S104).

[0030] If a specific gesture is detected (YES in step S104), the CPU 110 sets the abnormality detection to stop mode (step S112).

[0031] The CPU 110 notifies the surroundings that it is in stop mode (step S114). For example, the CPU 110 changes the LED light of the second abnormality detection warning unit 132 from green to red. The CPU 110 then outputs an audio message from the first abnormality detection warning unit 131 indicating that it has switched to stop mode.

[0032] The CPU 110 notifies the server 300 via the communication interface 160 of the ID of the monitoring device 100 itself and that it has entered stop mode (step S116).

[0033] The CPU 110 determines whether the elapsed time since the start of the stop mode has reached a predetermined value, such as 30 minutes (step S118). If the predetermined time has not elapsed (the answer in step S118 is NO), the CPU 110 repeats the process from step S114.

[0034] If a predetermined time has elapsed (if the answer is YES in step S118), the CPU 110 will release the stop mode (step S120).

[0035] The CPU 110 notifies the surroundings that it is in monitoring mode (step S122). For example, the CPU 110 changes the LED light of the second abnormality detection warning unit 132 from red to green. The CPU 110 then causes the first abnormality detection warning unit 131 to output an audio message indicating that it has switched to monitoring mode.

[0036] The CPU 110 notifies the server 300 via the communication interface 160 of the ID of the monitoring device 100 itself and that it has entered monitoring mode (step S124).

[0037] On the other hand, if the specific gesture for stop mode is not included (the answer is NO in step S104), the CPU 110 determines whether the 2D data created by millimeter-wave analysis includes the specific gesture for restarting the anomaly detection (step S106).

[0038] If a specific gesture for resuming is included (if the answer is YES in step S106), the CPU 110 executes the process from step S120.

[0039] If no specific gesture for resuming is included (the answer is NO in step S106), the CPU 110 determines whether the image created by millimeter-wave analysis contains any abnormal behavior of the person requiring care (step S132). If no abnormal behavior is included (the answer is NO in step S132), the CPU 110 repeats the process from step S102.

[0040] If abnormal behavior is detected (if the answer is YES in step S132), the CPU 110 determines whether or not it is currently in stop mode (step S134). If it is in stop mode (if the answer is YES in step S134), the CPU 110 repeats the process from step S102 without outputting any abnormal behavior.

[0041] If the system is not in stop mode (the answer is NO in step S134), the CPU 110 notifies the surroundings that an abnormal situation has occurred (step S136). For example, the CPU 110 causes the LED light of the second abnormal detection warning unit 132 to illuminate green, and outputs an audio message from the first abnormal detection warning unit 131 indicating that an abnormal situation has occurred.

[0042] The CPU 110 notifies the server 300 via the communication interface 160 of the ID of the monitoring device 100 itself and that an abnormal situation has occurred (step S138).

[0043] As a result, the server 300 notifies the communication terminal 500 of the person in charge of the monitoring device 100 via the communication interface 360 ​​that an abnormal situation has occurred. The communication terminal 500 outputs an audio message or displays a message indicating that an abnormal situation has occurred.

[0044] CPU110 will terminate this process. <Server Configuration>

[0045] Next, one aspect of the configuration of the server 300 that constitutes the monitoring system 1 according to this embodiment will be described. Referring to Figure 7, the server 300 includes, as its main components, a CPU (Central Processing Unit) 310, a memory 320, an operation unit 340, and a communication interface 360.

[0046] The CPU 310 controls various parts of the server 300 by executing programs stored in memory 320. For example, the CPU 310 executes programs stored in memory 320 and references various data to perform various processes described later.

[0047] The memory 320 is implemented by various types of RAM, various types of ROM, etc., and may be embedded in the server 300, or it may be detachable from various interfaces of the server 300, or it may be a recording medium of another device accessible from the server 300. The memory 320 stores programs executed by the CPU 310, data generated by the execution of programs by the CPU 310, input data, and databases used for other services according to this embodiment.

[0048] For example, memory 320 stores device information data 121 as shown in Figure 8. The device information data 121 stores correspondences between the ID of the monitoring device 100, the ID of the facility where it is installed, the ID of the room where it is installed, the ID of the person in charge of that room, the ID of the person in charge's mobile terminal, the current operating mode, and history data of abnormality detection.

[0049] Returning to Figure 7, the operation unit 340 receives commands from the service administrator and others, and inputs those commands to the CPU 310.

[0050] The communication interface 360 ​​receives various information from the monitoring device 100 and passes it to the CPU 310. Conversely, it transmits data from the CPU 310 to the communication terminal 500 of the person in charge via the internet, carrier network, router, etc. More specifically, the CPU 310 receives information from the monitoring device 100 that an abnormal situation has occurred, refers to the device information data 121, and notifies the communication terminal 500 of the person in charge corresponding to the monitoring device 100 of this information. As a result, the communication terminal 500 of the person in charge can notify the person in charge of the abnormal situation by outputting the abnormal situation information from a display or speaker. The CPU 310 also stores the ID of the monitoring device 100 and information that an abnormal situation has occurred in the memory 320 as historical information. <Second Embodiment>

[0051] In the above embodiment, the monitoring device 100 made a determination of whether there was an abnormality in the person requiring care and recognized the caregiver's gestures based on the received millimeter waves. However, the data received by the millimeter wave sensor 120 may be sent to the server 300, allowing the server 300 to make a determination of whether there was an abnormality in the person requiring care and recognize the caregiver's gestures.

[0052] For more details, please refer to Figure 9 to describe the information processing of the server 300 according to this embodiment. The CPU 310 of the server 300 performs the following processing for each of the multiple monitoring devices 100 according to the program in the memory 320.

[0053] First, the CPU 310 analyzes the received millimeter waves by acquiring the received data from the millimeter wave sensor 120 from the monitoring device 100 via the communication interface 360 ​​(step S202). For example, it may create two-dimensional image data from the received data or determine whether an object of a predetermined shape is captured in the image.

[0054] The CPU 310 determines whether the 2D data created by the millimeter-wave analysis contains a specific gesture for interrupting the anomaly detection process (step S204).

[0055] If a specific gesture is detected (if the answer is YES in step S204), the CPU 310 sends a command to the target monitoring device 100 via the communication interface 360 ​​to switch to the abnormal detection stop mode (step S212). At the same time, the CPU 310 stores in memory 320 that the target monitoring device 100 is in stop mode.

[0056] This causes the monitoring device 100 to notify that it is in stop mode. For example, the monitoring device 100 changes the LED light of the second abnormality detection warning unit 132 from green to red. The monitoring device 100 then outputs an audio message from the first abnormality detection warning unit 131 indicating that it has switched to stop mode.

[0057] The CPU 310 determines whether the elapsed time since the start of the stop mode has reached a predetermined value, such as 30 minutes (step S218). If the predetermined time has elapsed (if the answer in step S218 is YES), the CPU 310 sends a command to the target monitoring device 100 via the communication interface 360 ​​to cancel the stop mode (step S220). At the same time, the memory 320 stores that the monitoring device 100 is in monitoring mode.

[0058] This causes the monitoring device 100 to notify that it is in monitoring mode. For example, the monitoring device 100 changes the LED light of the second abnormality detection warning unit 132 from red to green. The monitoring device 100 then outputs an audio message from the first abnormality detection warning unit 131 indicating that it has switched to monitoring mode.

[0059] On the other hand, if the specific gesture for stop mode is not included (the answer is NO in step S204), the CPU 310 determines whether the image created by the millimeter-wave analysis includes the specific gesture for restarting the anomaly detection (step S206).

[0060] If a specific gesture for resuming is included (if the answer is YES in step S206), the CPU 310 executes the process from step S220.

[0061] If no specific gesture for resuming is included (the answer is NO in step S206), the CPU 310 determines whether the image created by millimeter-wave analysis contains any abnormal behavior of the person requiring care (step S232). If no abnormal behavior is included (the answer is NO in step S232), the CPU 310 repeats the process from step S202.

[0062] If abnormal behavior is detected (if the answer is YES in step S232), the CPU 310 determines whether or not it is currently in stop mode (step S234). If it is in stop mode (if the answer is YES in step S234), the CPU 310 repeats the process from step S202.

[0063] If the system is not in stop mode (the answer is NO in step S234), the CPU 310 notifies the target monitoring device 100 via the communication interface 360 ​​that an abnormal situation has occurred (step S236). As a result, the monitoring device 100 illuminates the LED light of the second abnormality detection warning unit 132 in green and outputs an audio message from the first abnormality detection warning unit 131 indicating that an abnormal situation has occurred.

[0064] At this time, the CPU 310 notifies the communication terminal 500 of the person in charge of the monitoring device 100 via the communication interface 360 ​​that an abnormal situation has occurred.

[0065] The CPU 310 terminates the current processing related to the target monitoring device 100. <Third Embodiment>

[0066] Some or all of the roles of each device in the monitoring system 1 of the above embodiment may be performed by other devices. For example, some or all of the roles of the monitoring device 100 and the server 300 may be taken over by other devices, or some or all of the roles of each of those devices may be shared among multiple devices. For example, the role of the server 300 may be realized by multiple devices on the cloud, some of the roles of the application program of the monitoring device 100 may be realized by the server 300, or some or all of the roles of the server 300 may be realized by the application program of the communication terminal 500. <Summary>

[0067] In the above embodiment, a monitoring device is provided which includes a millimeter-wave sensor and a control unit that determines whether or not there is an abnormality based on the data received from the millimeter-wave sensor, and the control unit stops determining whether or not there is an abnormality when it recognizes a predetermined gesture from the data received from the millimeter-wave sensor.

[0068] Preferably, the monitoring device includes a communication interface for communicating with the server, and the control unit transmits information to the server via the communication interface when it detects an abnormality.

[0069] In the above embodiment, a monitoring system is provided that includes a plurality of monitoring devices, a plurality of communication terminals, and a server that receives information from the monitoring devices and transmits the information to the communication terminals corresponding to the monitoring devices.

[0070] In the above embodiment, a server is provided which includes a communication interface for communicating with a device having a millimeter-wave sensor, and a control unit that determines whether or not there is an abnormality by acquiring received data from the millimeter-wave sensor from the device via the communication interface, and the control unit stops determining whether or not there is an abnormality when it recognizes a predetermined gesture from the received data from the millimeter-wave sensor from the device.

[0071] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims rather than by the foregoing description, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of Symbols]

[0072] 1: System 100: Monitoring device 110: CPU 111: Millimeter-wave analysis unit 112: Abnormal behavior identification unit 113: Gesture Recognition Unit 115: Memory 120: Millimeter-wave sensor 121: Device Information Data 130: Status display section 131: First abnormality detection warning unit 132: Second anomaly detection and warning unit 140: Operation User Interface Section 150: Power supply 160: Communication Interface 161: Wireless antenna 300: Server 310: CPU 320: Memory 340:Operation unit 360: Communication Interface 500: Communication terminal

Claims

1. Millimeter-wave sensor and, The system includes a control unit that determines whether or not there is an abnormality based on the received data from the millimeter-wave sensor, The control unit is a monitoring device that stops determining whether or not there is an abnormality when it recognizes a predetermined gesture from the received data of the millimeter-wave sensor.

2. A communication interface for communicating with the server, The monitoring device according to claim 1, wherein the control unit, when it detects the abnormality, transmits information to that effect to the server via the communication interface.

3. Multiple monitoring devices according to claim 2, Multiple communication terminals, A monitoring system comprising: a server that receives the information from the monitoring device and transmits the information to a communication terminal corresponding to the monitoring device.

4. A communication interface for communicating with a device having a millimeter-wave sensor, The system includes a control unit that determines whether or not there is an abnormality by acquiring received data from the millimeter-wave sensor from the device via the communication interface, The control unit is a server that stops determining whether or not there is an abnormality when it recognizes a predetermined gesture from the data received from the millimeter-wave sensor from the device.

Images