A monitoring system, a method for controlling the monitoring system, and a program.

Abstract

This technology provides the ability to more accurately detect people using millimeter-wave sensors. [Solution] The monitoring system comprises a millimeter-wave sensor that detects the space inside a room as the detection target space, a control unit that detects a person using the measurement data from the millimeter-wave sensor, and a storage unit that stores pre-acquired data (data acquired in advance by the millimeter-wave sensor when no person is present in the room). The control unit repeatedly executes a first detection process to detect a person moving in the room based on multiple data acquired by the millimeter-wave sensor at multiple points in time within a short period (S13, S14, S21). If a person that was previously detected in the repeatedly executed first detection process is no longer detected (No in step S14 and Yes in step S15), a second detection process is executed to detect a person present in the room by performing difference processing between the pre-acquired data and the data acquired within the short period (S16).

Description

Technical Field

[0001] The present invention relates to a monitoring system for monitoring a person in a room and technologies related thereto.

Background Art

[0002] There is a monitoring system for monitoring a person in a room. In the monitoring system, for example, a person in the room is detected by a TOF (Time Of Flight) camera or the like, and the behavior of the person is monitored.

[0003] In the monitoring system, in order to detect the presence and posture (body position (lying position, sitting position, standing position, etc.)) of a person in the room, a TOF camera may be used as a sensor capable of obtaining three-dimensional information of the person in the space. For example, considering the availability at night and the like, a TOF camera having an infrared image sensor is used.

[0004] According to the TOF camera, depth information (depth information) of each point in the captured image can be obtained. Then, based on the image information of the captured image and the depth information, human skeleton information is acquired as three-dimensional information, and it is possible to estimate the body posture of the person based on the human skeleton information.

[0005] On the other hand, from the viewpoints of cost and ensuring personal privacy, etc., technologies using a millimeter-wave sensor instead of a TOF camera (TOF sensor) in the monitoring system have also been studied (see Patent Document 1, etc.).

[0006] According to the millimeter-wave sensor, the distance and azimuth to each point on an object (person, etc.) in the detection target space are detected, and three-dimensional position information of each point on the object is obtained. As a result, information on the surface of the object (person, etc.) is obtained as a set of points (point cloud). However, as will be described later, it is difficult to distinguish a person from other articles in a room where various articles (bed, chair, etc.) are arranged.

[0007] Furthermore, the information obtained from millimeter-wave sensors consists of point cloud position information and does not include image information such as the contours of individual parts of a person (face, shoulders, arms, torso, legs, etc.), which are included in image information obtained from image sensors. In addition, the spatial resolution (spatial resolution related to point clouds) of millimeter-wave sensors is generally relatively coarse. Therefore, the amount of information obtained from millimeter-wave sensors regarding a person in a room is less than the amount of information obtained from a TOF camera. Consequently, skeletal estimation using millimeter-wave sensors is not easy. [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2023-131563 [Overview of the Initiative] [Problems that the invention aims to solve]

[0009] In monitoring systems using millimeter-wave sensors, the behavior of a person in a room can be understood based on the data acquired by the millimeter-wave sensor. However, it is difficult to distinguish a person from other objects in a room where various items (beds, chairs, etc.) are placed.

[0010] For example, distinguishing and recognizing a moving person from other objects can be achieved by performing differential processing on multiple data points acquired sequentially in real time by a millimeter-wave sensor. In other words, it is possible to detect a person moving within a room based on multiple data points acquired at multiple points in time within a short period of time by a millimeter-wave sensor.

[0011] Specifically, based on the difference between data acquired by a millimeter-wave sensor at a certain point in time and data acquired a small time before that point, it is possible to detect a person moving within a room as a collection of points (a small area) (a point cloud). More specifically, the collection of points (a point cloud) corresponding to this difference is detected as data representing the position of the person moving within the room. By performing difference processing on both sets of data, information about objects (stationary items) present at the same location in both sets of data is canceled out, making it possible to obtain a point cloud of a moving person.

[0012] However, when using such moving person detection technology, the point cloud corresponding to the difference will not be detected when the person is stationary (or nearly stationary). Therefore, it becomes difficult even to determine whether or not a person is present in the room.

[0013] Furthermore, even when a person is moving slightly, if the person is located far from the millimeter-wave sensor, or if the person is not directly facing the direction of the millimeter-wave emission from the sensor (for example, if the millimeter-wave is emitted from diagonally above towards a person lying on the floor), the intensity of the reflected waves from the person may be relatively weak. As a result, a sufficient number of point clouds indicating the presence of a person may not be detected, making it difficult to even determine that a person is present.

[0014] Therefore, the object of the present invention is to provide a technology that can more appropriately detect people using a millimeter-wave sensor. [Means for solving the problem]

[0015] To solve the above problems, the present invention provides a monitoring system for monitoring a person in a room, comprising: a millimeter-wave sensor that uses the space in the room as the detection target space; a control unit that detects a person in the room using data acquired by the millimeter-wave sensor; and a storage unit that stores pre-acquired data, which is data acquired in advance by the millimeter-wave sensor when no person is present in the room. The control unit repeatedly executes a first detection process to detect a person moving in the room based on a plurality of data acquired by the millimeter-wave sensor at a plurality of points in time within a minute period, and if a person that was once detected in the repeatedly executed first detection process is no longer detected, it performs a second detection process to detect a person present in the room by performing a difference processing between the pre-acquired data and the data acquired by the millimeter-wave sensor within the minute period.

[0016] The control unit may execute the second detection process using the area in which a person was detected in the first detection process as the target area.

[0017] If the control unit determines that there is no person present in the room, it may update the data acquired by the millimeter-wave sensor as the previously acquired data.

[0018] If no object is detected by the differential processing of the second detection process, the control unit may determine that there is no person in the room and update the data acquired by the millimeter-wave sensor within the short period as the pre-acquired data.

[0019] If the control unit determines in the second detection process that a person is present in the room, it may further determine the posture of the person based on the point cloud detected by the difference process as a point cloud indicating the location of the person.

[0020] When it is determined in the second detection process that a person exists in the living room, based on the ratio of the horizontal length to the vertical length of the point cloud detected by the difference process as the point cloud indicating the position of the person, the control unit may determine whether the posture of the person is standing, sitting, or lying down.

[0021] When the control unit determines that the posture of the person is standing, based on the vertical position of the point cloud, the control unit may further determine whether the person is standing on the floor or elsewhere.

[0022] When the control unit determines that the posture of the person is sitting, based on the vertical position of the point cloud, the control unit may further determine whether the person is sitting on the floor.

[0023] When the control unit determines that the posture of the person is lying down, based on the vertical position of the point cloud, the control unit may further determine whether the person is lying on the bed or elsewhere.

[0024] When the control unit determines that the posture of the person is either sitting or lying down, based on the occupied area of the point cloud in the horizontal plane, the control unit may further determine whether the person is curled up.

[0025] In order to solve the above problems, a control method for a monitoring system according to the present invention includes: a) storing, as pre-acquired data, data pre-acquired by a millimeter-wave sensor that sets the space in the living room as a detection target space when no person is present in the living room in a storage unit of the monitoring system; b) repeatedly executing a first detection process for detecting a person moving in the living room based on a plurality of data acquired at a plurality of time points within a short period by the millimeter-wave sensor; and c) when a person once detected in the repeatedly executed first detection process is no longer detected, executing a second detection process for detecting a person present in the living room by performing a difference process between the pre-acquired data and the data acquired within the short period by the millimeter-wave sensor.

[0026] In order to solve the above problems, a program according to the present invention causes a computer provided in a control unit of a monitoring system that monitors a person in a living room to: a) store, as pre-acquired data, data pre-acquired by a millimeter-wave sensor that sets the space in the living room as a detection target space when no person is present in the living room in a storage unit of the monitoring system; b) repeatedly execute a first detection process for detecting a person moving in the living room based on a plurality of data acquired at a plurality of time points within a short period by the millimeter-wave sensor; and c) when a person once detected in the repeatedly executed first detection process is no longer detected, execute a second detection process for detecting a person present in the living room by performing a difference process between the pre-acquired data and the data acquired within the short period by the millimeter-wave sensor. The program may be a program for causing the computer to execute the above steps.

Effect of the Invention

[0027] According to the present invention, it is possible to more appropriately detect a person using a millimeter-wave sensor.

Brief Description of the Drawings

[0028] [Figure 1]This diagram shows a monitoring system that keeps an eye on people inside a room. [Figure 2] This is a functional block diagram showing the general configuration of the detection device. [Figure 3] This is a flowchart showing the controller's processing. [Figure 4] This figure shows a part of the processing shown in Figure 3. [Figure 5] This figure shows some of the measurement data obtained from a millimeter-wave sensor. [Figure 6] This figure shows the data sequentially acquired by the millimeter-wave sensor. [Figure 7] This is a diagram showing the first detection process, etc. [Figure 8] This is a diagram showing the second detection process, etc. [Figure 9] This figure shows an example of point cloud detection related to a person (a person lying on the floor). [Figure 10] This figure shows another detection example (an example of detecting a person lying on a bed). [Figure 11] This figure shows a modified example of measurement data from a millimeter-wave sensor. [Modes for carrying out the invention]

[0029] Embodiments of the present invention will be described below with reference to the drawings.

[0030] <1. System Overview> Figure 1 shows a monitoring system 1 that monitors a person (specifically, the person's behavior, etc.) in a living room 90. As shown in Figure 1, the monitoring system 1 comprises multiple detection devices 10 and multiple terminal devices 70, 80. Terminal device 70 is also called a management device 70, and terminal device 80 is also called a portable terminal device 80. The monitoring system 1 detects the presence or absence of a person to be monitored (such as a person receiving care) in the living room 90 and the person's posture, etc. (such as "fallen state").

[0031] Each detection device 10 and each terminal device 70, 80 are connected to each other via a network 108. The network 108 consists of a LAN (Local Area Network) and the Internet, etc. The connection to the network 108 may be wired or wireless. For example, the management device 70 may be wired to the network 108, and each detection device 10 and each mobile terminal device 80 may be wirelessly connected to the network 108. Alternatively, all devices 10, 70, and 80 may be wirelessly connected to the network 108.

[0032] In the monitoring system 1, various input / output processes (such as operation input processing and display processing) related to each detection device 10 are performed using the management device 70 and the portable terminal device 80. In other words, the management device 70 and the portable terminal device 80 each function as terminal devices for multiple detection devices 10.

[0033] The following primarily provides examples of how the monitoring system 1 may be used in nursing care facilities. However, it is not limited to these examples, and the monitoring system 1 may also be used in nursing facilities (hospitals, etc.) or in private homes.

[0034] <2. Detection device 10> Each detection device 10 is placed in the room 90 of each person being monitored (in this case, the person receiving care (resident)) (for example, in each person's individual room). Each detection device 10 is a device that detects the posture of the person being monitored ("fallen state," etc.) based on measurement data related to the person being monitored (measurement data from the millimeter-wave sensor 20), as described later.

[0035] Figure 2 is a functional block diagram showing the schematic configuration of the detection device 10.

[0036] As shown in Figure 2, the detection device 10 comprises a millimeter-wave sensor 20 and a processing unit 30.

[0037] The millimeter-wave sensor 20 transmits radio waves in the millimeter-wave band (transmitted waves) and receives radio waves (received waves) reflected from an object. The millimeter-wave sensor 20 is a radar sensor that measures the distance (r) from the millimeter-wave sensor 20 to an object, and the orientation of the object (two orientation angles (θ,φ) in mutually different directions (orthogonal directions)). It is preferable that the coordinate values ​​indicating the position (3D position) of the object acquired by the millimeter-wave sensor 20 are converted from a spherical coordinate system (r,θ,φ) based on the position of the millimeter-wave sensor 20 to a Cartesian coordinate system (X,Y,Z) fixed in the living room 90 (based on a predetermined position within the living room 90) before use.

[0038] The millimeter-wave sensor 20 is installed on the ceiling or a side wall near the ceiling of the living room 90 (see Figure 1). The millimeter-wave sensor 20 sets the space within the living room 90 (preferably the entire space, but not limited to the entire space, and may also be a part of the space) as the detection target space and detects the presence and posture of a person within the detection target space. Specifically, the millimeter-wave sensor 20 is capable of acquiring time-series data indicating the three-dimensional position of each minute part of the surface of an object (person, etc.) within the living room 90. By using the measurement data from the millimeter-wave sensor 20, it is possible to detect the presence or absence of a person and their behavior (posture, etc.) within the living room 90.

[0039] The millimeter-wave sensor 20 incorporates an antenna unit 20a that transmits a transmission wave and receives a reception wave, and an internal controller 20b that performs transmission and reception processing of the transmission and reception waves, as well as analysis processing of both waves (all housed within the millimeter-wave sensor 20). The millimeter-wave sensor 20 may, but is not limited to, detect the presence and posture of a person in the target space by calculation processing using only the internal controller 20b. For example, the millimeter-wave sensor 20 may work in cooperation with an external controller (for example, the controller 31 of the processing unit 30 (described later)) to detect the presence and posture of a person in the target space. In this case, the millimeter-wave sensor 20 works in cooperation with the processing unit 30 (controller 31) to detect a person in the target space based on data acquired by the millimeter-wave sensor 20 (measurement data from the millimeter-wave sensor 20). Specifically, the measurement data from the millimeter-wave sensor 20 is input to the controller 31 of the processing unit 30, and the presence or absence of a person and their posture are ultimately detected by calculation processing by the controller 31.

[0040] Furthermore, the processing unit 30 includes a controller (also called a control unit) 31, a storage unit 32, a communication unit 34, and an operation unit 35 (Figure 2).

[0041] The controller 31 is a control device built into the processing unit 30 that controls the detection device 10.

[0042] The controller 31 is configured as a computer system equipped with a CPU (Central Processing Unit) (also referred to as a microprocessor or hardware processor, etc.). The controller 31 performs various processes by executing a predetermined software program (hereinafter simply referred to as a program) stored in a storage unit (ROM and / or a non-volatile storage unit such as a hard disk) 32 using the CPU. The program (more specifically, a group of program modules) may be recorded on a portable recording medium such as a USB memory stick and read from the recording medium to be installed on the detection device 10. Alternatively, the program may be downloaded via a communication network or the like and installed on the detection device 10.

[0043] The controller 31 recognizes the presence and posture of the person being monitored based on the data measured (acquired) by the millimeter-wave sensor 20. If the controller 31 determines that the person is in a dangerous state (or a state equivalent to a dangerous state), it issues a warning (specifically, a warning signal (or caution signal)). The warning (warning signal, etc.) regarding the person being monitored is transmitted from the controller 31 to the terminal devices 70 and 80 and received by the terminal devices 70 and 80. The terminal devices 70 and 80 then output a warning (audio output and / or display output) based on the warning signal, etc. In this way, the warning is notified to the caregiver via the terminal devices 70 and 80.

[0044] The memory unit (also called the storage unit) 32 is composed of a storage device such as a hard disk drive (HDD) or a solid-state drive (SSD). The memory unit 32 stores (stores) time-series data of the three-dimensional position of the person being monitored.

[0045] The communication unit 34 is capable of performing network communication via the network 108. Various protocols, such as TCP / IP (Transmission Control Protocol / Internet Protocol), are used in this network communication. By using this network communication, the detection device 10 can exchange various types of data with a desired partner (for example, terminal devices 70, 80).

[0046] The operation unit 35 includes an operation input unit 35a that receives operation inputs for the detection device 10, and a display unit 35b that displays and outputs various information. In this system 1, the terminal devices 70 and 80 mainly perform the input / output functions (operation input reception function and output function (display function, etc.)) related to the detection device 10. Therefore, the operation unit 35 may be of a type that is attached to the processing unit 30 only during maintenance, etc.

[0047] <3. Terminal devices 70, 80> Of the terminal devices 70 and 80, the management device 70 is a device that manages the entire monitoring system 1 and is mainly operated by the administrator. On the other hand, the portable terminal device 80 functions as a display device that shows various information in the monitoring system 1. The portable terminal device 80 is carried by the person who provides care to the person receiving care (caregiver) and displays various information about the person receiving care. The management device 70 also functions as a display device that shows various information in the monitoring system 1.

[0048] Terminal devices 70 and 80 are information input / output terminal devices (information processing devices) capable of network communication with other devices (such as 10). Terminal devices 70 and 80 are configured as smartphones, tablet terminals, or personal computers (which can be fixed (stationary) or portable). For example, terminal device 70 is a fixed personal computer, and terminal device 80 is a portable device (mobile terminal device), more specifically, a smartphone.

[0049] Terminal devices 70 and 80 each have the same hardware configuration as processing unit 30. Management device 70 includes a controller, storage unit, communication unit, operation unit, etc., and mobile terminal device 80 also includes a controller, storage unit, communication unit, operation unit, etc.

[0050] <4.Operation> Figures 3 and 4 are flowcharts showing the processing of the controller 31. Figure 4 shows the processing of a part of Figure 3 (step S30). Figure 5 shows some of the measurement data from the millimeter-wave sensor 20, and Figure 6 shows the data acquired sequentially (in real time). Figure 7 shows the first detection process, etc., and Figure 8 shows the second detection process, etc.

[0051] In Figure 7, for illustrative purposes, the person gradually transitions from a normal standing posture to a lying posture (fallen posture (and stationary)) from time t1 to time t4, and only four data points are acquired during this period. However, in reality, many more data points are often acquired during this period. Also, in Figures 7 and 8, only the bed 92 is shown among the items in the room 90, and other items are omitted. In the left column of Figure 7 (and Figure 8), the object surface detected by the millimeter-wave sensor 20 is represented as a collection of points (small circles). These points (small circles) are shown larger than they actually are for illustrative purposes.

[0052] The following explanation will describe the person detection process performed by the controller 31, referring to Figure 3 and other figures.

[0053] First, in step S11, the controller 31 acquires (measures) data D0 (measurement data when no person is present) using the millimeter-wave sensor 20 at a time when no person is present in the room 90 (for example, time t0 (see the top row of the left column in Figure 7)). The controller 31 then stores this data D0 in the storage unit (memory unit) 32 as pre-acquired data Dp (data previously acquired (measured) by the millimeter-wave sensor 20 when no person is present in the room 90).

[0054] The data D0 may be acquired, for example, in response to instructions from the operator (caregiver, etc.) of the monitoring system 1. More specifically, when the operator determines that there is no person in the room 90, the operator gives instructions to the controller 31 via the portable terminal device 80, and the data D0 is acquired (measured) under the control of the controller 31 in response to those instructions. Alternatively, the data D0 may be acquired automatically by the monitoring system 1. For example, when the absence of a person in the room 90 is detected by various sensors (such as motion sensors installed in the room 90), the controller 31 may automatically perform a measurement using the millimeter-wave sensor 20 and acquire the data D0.

[0055] As described above, the millimeter-wave sensor 20 transmits radio waves in the millimeter-wave band (transmitted waves) and receives radio waves reflected from an object (received waves). Based on the transmitted and received waves, the millimeter-wave sensor 20 measures the distance (r) from the sensor 20 to the object, and the orientation of the object (two azimuth angles (θ,φ) in different directions (orthogonal directions)).

[0056] As shown in Figure 5, for example, the measurement data from the millimeter-wave sensor 20 is measured as data indicating the intensity of the received wave at each distance r in each of multiple directions (intensity change with respect to distance r). Figure 5 shows a situation where the signal intensity is maximum at a certain distance r1 for a received wave from a certain direction. At this time, it is detected that an object in that direction is located at a distance r1 (from the millimeter-wave sensor 20). In other words, the distance r (distance r from the millimeter-wave sensor 20) to a minute area (also called a "point area" or simply a "point") of the object located in that direction is measured. Using this measurement principle, the three-dimensional positions (r, θ, φ) of minute areas on the surfaces of various objects (people, beds, chairs, tables, wheelchairs, etc.) in the living room 90 are detected.

[0057] The measurement data (data D0, etc.) from the millimeter-wave sensor 20 is obtained as data (a collection of data) showing the intensity of the received wave at each distance r in each of multiple directions, and is stored in the storage unit 32. Such data is also called RAW data.

[0058] The measurement data from the millimeter-wave sensor 20 is stored in the storage unit 32 as raw data, for example. However, it is not limited to this, and the measurement data may be converted into a collection of point cloud data (also referred to as point cloud data) of positional information (3D positional information) of point clouds (multiple points) on the surface of objects in the room 90 and stored. Furthermore, this positional information may be expressed as coordinate values ​​in a spherical coordinate system (r,θ,φ) based on the position of the millimeter-wave sensor 20, or it may be converted into coordinate values ​​in a Cartesian coordinate system (X,Y,Z) based on a predetermined position in the room 90 and expressed as such.

[0059] In particular, data D0 is data acquired (measured) by the millimeter-wave sensor 20 at a time when no person is present in the room 90 (when no person is present). As shown in the top row of the left column of Figure 7, data D0 includes information (3D position information) of objects other than people (items such as beds) in the room 90, but does not include information (3D position information) of people.

[0060] In step S11, this data D0 is first acquired as pre-acquired data Dp (data acquired in advance by the millimeter-wave sensor 20 when no person is present in the room 90) and stored in the storage unit 32.

[0061] Subsequently, when predetermined conditions are met, data measurement by the millimeter-wave sensor 20 is started (step S12). Examples of these predetermined conditions include the detection by another sensor (such as a motion sensor) that a person has returned to the room 90, or the elapsed of a predetermined time (for example, 3 minutes) since the determination that the person is absent from the room 90.

[0062] In step S13, the first detection process is performed. The first detection process is a process to detect a person (a person in motion) that is currently moving within the living space 90 (also referred to as the moving person detection process). The first detection process is performed based on multiple data acquired by the millimeter-wave sensor 20 at multiple different points in time within a short period of time in the vicinity of the present.

[0063] Figure 6 shows the data D1, D2, D3, ... acquired (measured) in time series by the millimeter-wave sensor 20.

[0064] In the first detection process, for example, two data sets D1 and D2 acquired at multiple (different) time points t1 and t2 within a small period of time in the vicinity of the present are used as multiple data acquired at multiple points in time in the vicinity of the present. Specifically, data D1 acquired at time t1 (see also the second row from the top in the left column of Figure 7) and data D2 acquired at time t2, one frame period (e.g., 1 / 10 second) after time t1 are used (see also the third row from the top in the left column of Figure 7). In other words, data D1 and D2 from two consecutive frames are used. Then, based on the difference data D21 between data D1 and data D2 (see the top row in the right column of Figure 7), a person moving within the room is detected as a collection of points (trace regions) (point cloud). More specifically, the collection of points (point cloud) corresponding to the difference between both data sets (e.g., RAW data) is detected as data representing the position of a person moving within the room. By performing a difference process between the two sets of data being differed (also referred to as the first difference process), information about objects (stationary items) at the same location in both sets of data is canceled out, making it possible to obtain a point cloud of a moving person. The difference process may be performed by taking the difference between RAW data, but is not limited to this; it may also be performed by taking the difference between point cloud data (deleting point clouds at the same location in the two sets of data being differed), etc.

[0065] For example, points detected at the same location in both data D1 and data D2 (small circles shown as dashed lines in Figure 7) (i.e., stationary objects) are not detected in the differentially processed data D21. More specifically, points corresponding to bed 92 and the floor are not detected in the differential data D21. On the other hand, points detected in only one of data D1 or data D2 (points not detected at the same location between the two data sets) (small circles shown as solid lines in Figure 7) are detected based on the differential data D21. In other words, moving objects (and by extension, moving people) are detected based on the differential data D21. In the middle of Figure 7, it is shown that both points corresponding to a part of a person standing in data D1 and points corresponding to a part of a person with their upper body tilted in data D2 are detected based on the differential data D21. If a certain number of such points are detected, it is determined (detected) that a moving person is present. Furthermore, the range of the point cloud detected based on the differential data D21 is detected as the area where a moving person is present. In Figures 7 and 8, detected point clouds are represented by solid circles, while undetected point clouds are represented by dashed circles.

[0066] In step S14, branching processing is performed based on the processing result of the first detection process. If a moving person is detected in the first detection process (if it is detected that a moving person is present), the process proceeds from step S14 to step S21. In step S21, it is determined that a person is present in the room 90 (occupancy determination). Also in step S21, the controller 31 sets flag F, which indicates that a moving person in the room 90 has been detected, to "1" (on). After that, the process returns to step S13. It is assumed that flag F was set to "0" (off) in step S12.

[0067] In step S13, based on the difference data D32 between the following data D3 and data D2 (see the second row from the bottom in the right column of Figure 7), a person moving within the room is detected as a collection of points (trace regions) (point cloud).

[0068] Then, the process returns to step S13 via step S21 from step S14, and the same operation is repeated. If the person is in movement (continuing normal walking, in the process of falling, etc.), the first detection process (step S13) is repeatedly executed via steps S14 and S21 in this manner.

[0069] In this way, differential processing (first detection processing) between multiple data acquired sequentially in real time by the millimeter-wave sensor 20 is repeatedly performed. In other words, based on multiple data acquired by the millimeter-wave sensor 20 at multiple points in time within a short period, the process of detecting a person moving in the room (first detection processing) is repeatedly performed. According to the first detection processing, it is possible to distinguish and recognize a moving person from a stationary object (item, etc.).

[0070] However, if only the first detection process is performed, the following problems may arise. Specifically, as shown in Figure 7, if a person transitions to a stationary state (time t4) due to falling or other reasons, the first detection process will no longer detect the point cloud corresponding to the difference between data D4 and data D3 (see the bottom row of the right column in Figure 7) for the difference data D43 between data D4 and data D3 (the previous data). In other words, a person who is actually present in the room 90 (a stationary person) will no longer be detected as a collection of points (trace regions) (see the small circles in the "dashed line"). Furthermore, even when a person is almost stationary (when the person's movement is very small), it becomes difficult to detect the point cloud corresponding to the difference with the previous data, and as a result, a sufficient number of point clouds indicating the presence of a person will not be detected, making it impossible to detect a person who is actually present in the room 90.

[0071] Furthermore, even when a person is moving to a certain extent, if the person is located far from the millimeter-wave sensor 20, or if the person is not directly facing the direction of the millimeter-wave emission from the millimeter-wave sensor 20 (for example, if millimeter-waves are emitted from diagonally above towards a person lying on the floor), the intensity of the reflected waves from the person may be relatively weak. In such cases, the difference from the previous data becomes small in the first detection process, making it difficult to detect the point cloud corresponding to that difference. As a result, a sufficient number of point clouds indicating the presence of a person may not be detected, making it impossible to detect the presence of a person.

[0072] Thus, even though a person is actually present (e.g., has fallen in room 90), the first detection process alone may not detect that person (the system may be unable to detect the person).

[0073] Furthermore, if a person who was present in room 90 (a person who was initially detected) exits room 90 through the door, the moving person within room 90 will no longer be detected as a collection of points (trace areas) (point cloud) in the differential data calculated in the first detection process.

[0074] Therefore, the first detection process alone makes it difficult to distinguish whether a person is present in the room 90 (in a stationary state, etc.) or not. For example, the first detection process alone makes it difficult to reliably distinguish between a situation where a person is stationary (especially near the door) (e.g., after falling) and a situation where a person has left the room 90.

[0075] Therefore, in this embodiment, the second detection process (described later) is executed following the first detection process. Specifically, if a person who was previously detected in the repeatedly executed first detection process is no longer detected (in the repeatedly executed first detection process) (if No in step S14 and Yes in step S15), the second detection process is executed (step S16).

[0076] In detail, if a moving person is not detected in the first detection process (step S13), the process proceeds from step S14 to step S15 to determine whether or not the moving person was detected at one point. If the moving person was detected at one point in the first detection process (if Yes in step S15), the second detection process is executed (step S16). In this way, the second detection process is executed when a person who was detected at one point in the repeatedly executed first detection process is no longer detected (if No in step S14 and Yes in step S15). Specifically, if no person is detected in step S13 and flag F is "1", the second detection process (person detection process using pre-acquired data Dp) is executed.

[0077] On the other hand, if the moving person has not yet been detected in the first detection process (step S13) (if the state of not being detected continues) (if No in step S14 and No in step S15), the controller 31 does not execute the second detection process and determines that the person is absent from the room 90 (step S17). In step S17, the controller 31 maintains the flag F, which indicates that the moving person in the room 90 was detected, at "0" (off). After that, the process returns from step S17 to step S13. As long as the person remains absent, the processes in steps S13, S14, S15, and S17 are repeated. However, the process is not limited to this, and the process in Figure 3 may be temporarily stopped. In that case, for example, when the predetermined conditions described above are met, the process from data measurement by the millimeter-wave sensor 20 (step S12) onwards should be resumed.

[0078] The conditions under which the second detection process (step S16) is executed (the conditions under which a person who was initially detected in the repeatedly executed first detection process (a moving person) is no longer detected) include, as described above, situations in the room where the person is stationary, almost stationary (a situation where the person's movement is very small), where the person is located far from the millimeter-wave sensor 20, and / or where the person is not facing directly toward the direction of emission of millimeter waves. Furthermore, the conditions under which the second detection process is executed also include situations where no person is present in the room. Thus, in these various situations, it is determined that a person who was initially detected in the first detection process is no longer detected, and the second detection process is executed. In other words, the second detection process is executed when the first detection process can no longer (reliably) detect a moving person (including when their presence or absence is unknown) (when it is no longer possible to confirm that a moving person is present).

[0079] The second detection process (step S16) is a process (stationary person detection process) that detects a person currently present in the room 90 by performing a difference process (also called the second difference process) between previously acquired data Dp and the current nearby data Dc. The previously acquired data Dp is data acquired in advance by the millimeter-wave sensor 20 at a time before the present when no person was present in the room 90. The current nearby data Dc is measurement data from the millimeter-wave sensor 20, and is data acquired (measured) within a short period (nearby period) including the acquisition time (measurement time) of the data to be processed in the first detection process (especially the last data acquired in the first detection process). The current nearby data Dc is, for example, the last data acquired in the first detection process (D4, etc.). However, it is not limited to this, and the current nearby data Dc may be data acquired immediately after the last data acquired in the first detection process (within the said short period) (for example, D5 (, D6)), etc.

[0080] In the second detection process, for example, the process shown in Figure 8 is performed. Specifically, the controller 31 calculates the difference data D40 between the currently nearby data D4 and the previously acquired pre-obtained data Dp (D0, etc.). Based on this difference data D40, the presence or absence of a person in the room is detected.

[0081] In this second detection process (see Figure 8), the difference between a certain data (currently nearby data) and pre-acquired data Dp is calculated. On the other hand, in the first detection process (see Figure 7), the difference between the same data (currently nearby data) and the data immediately preceding it (previous data) is calculated. The first detection process attempts to detect minute differences (differences) with the previous data (of the same person in motion), while the second detection process attempts to detect relatively large differences (differences) with the pre-acquired data Dp (of the absence of a person). Therefore, the difference (difference) in the second detection process is detected as a larger difference than the difference (difference) in the first detection process.

[0082] In the second detection process, if a larger-than-predetermined number of point clouds are detected based on the difference data (D40) between the previously acquired data Dp and the currently nearby data Dc (D4), it is detected (determined) that a person is present in the room 90 (step S18 → step S22). In other words, the second detection process detects that a person (such as a stationary person) that could no longer be detected in the first detection process is still present in the room 90. More specifically, the set of points (point cloud) corresponding to the difference between the two sets of data (for example, RAW data) is detected as a person currently present in the room 90. Point clouds (and corresponding information) present in both data D4 and data D0 are canceled out in the difference processing of the two sets of data. As a result, point clouds (and corresponding information) representing stationary objects (objects other than people) that were present in data D0 do not remain in the difference data D40 (Figure 8). On the other hand, point clouds that exist in data D4 but not in data D0 (data showing only "objects other than people") when no people are present are detected as point clouds of objects that are not "objects other than people," i.e., point clouds of people (stationary people, etc.) based on the difference data D40. Then the process proceeds from step S22 to step S30 (described later).

[0083] On the other hand, if the point cloud detected based on the difference data (also called D90) between the pre-acquired data Dp and the current nearby data Dc (D4b (also called D9) (not shown) which is different from D4) is less than a predetermined amount (if almost no point cloud is detected), it is determined that there is no person in the room 90 (step S18 → step S23). More specifically, if no object is detected by the difference processing of the second detection process, it is determined that there is no person in the room 90. In other words, the second detection process detects that the person who could no longer be detected in the first detection process has left the room 90 and is no longer inside the room 90 (no person in the room). More specifically, if the point clouds present in both the pre-acquired data Dp and the data D9, which was last acquired in the first detection process immediately before the second detection process, cancel each other out in the difference processing of the two data sets, and if the point clouds present in only one of the data sets, either data D9 or pre-acquired data Dp, are not detected (at all or almost), then the absence of a person in the room is detected (step S23).

[0084] Thus, if a person who was previously detected in the repeatedly executed first detection process is no longer detected, it means either the person has left room 90 (there is no person in room 90) or the person is still in room 90. If an object is detected in the second detection process, it is determined that the object is a person and that the person is still in room 90 (step S22). On the other hand, if no object is detected in the second detection process, it is determined that the situation is the same as when the pre-acquired data Dp was acquired. That is, it is determined that the person is not in room 90 (and therefore the person has left room 90) (step S23).

[0085] If no person (object) is detected by the second detection process (step S16), the controller 31 determines that there is no person in the room 90 (step S23). Then, the process proceeds to step S24. In step S24, the flag F, which indicates that a person moving in the room 90 was detected, is set to "0" (off) (reset), and the previously acquired data Dp is updated with data D9. Data D9 is data acquired by the millimeter-wave sensor 20 at the time when there was no longer a person in the room 90 (the most recent time).

[0086] In step S24, the controller 31 updates the data D9 acquired by the millimeter-wave sensor 20 within a short period of time (for example, within the most recent frame period) as pre-acquired data Dp. The updated pre-acquired data Dp is used in subsequent second detection processes. By using the updated pre-acquired data Dp, it is possible to perform the second detection process based on the latest (or relatively recent) state of the room 90. Specifically, for example, if a person moves a chair in the room 90 and then leaves the room 90, the pre-acquired data Dp is updated in step S24. When the process in Figure 3 is executed again after the person returns to the room 90, the second detection process is performed based on the pre-acquired data Dp that reflects the position of the chair after the move. Therefore, in the second detection process, it is possible to correctly reflect the position of the chair after the move and detect the presence or absence of the person.

[0087] On the other hand, as described above, if a person (object) is detected by the second detection process (step S16), the controller 31 determines that a person is present in the room 90 (step S22). Then, the process proceeds to step S30.

[0088] In step S30, a process is executed to determine the posture of the person in the room 90. The process in step S30 will be explained below with reference to Figure 4. As described above, if the second detection process detects (step S22) that a person is present in the room 90 in a stationary or nearly stationary state, the process proceeds to step S30. In step S30, it is further detected (determined) whether the person is lying on the bed, lying on the floor, standing on the floor (simply standing still), standing up on the bed, sitting on the floor, or sitting in a chair, etc.

[0089] First, in step S31 (see Figure 4), the controller 31 determines whether the person's posture (also referred to as standing, sitting, or lying down) is standing, sitting, or lying down.

[0090] In detail, based on the difference data (D40, etc.) in the second detection process, the millimeter-wave sensor 20 detects points 97 (see Figure 9) that represent minute areas on the surface of the person, and the person is detected as a collection of these points 97. The smallest rectangular parallelepiped (a rectangular parallelepiped with a base parallel to the floor) 98 (a virtual rectangular parallelepiped) that encompasses the collection of these points 97 is determined to be the space in which the person exists. Figure 9 is a diagram showing an example of point cloud detection related to a person by the millimeter-wave sensor 20. Figure 10 is similar to Figure 9. Figure 9 shows an example of detection related to a person lying on the floor, while Figure 10 shows another detection example (an example of detection of a person lying on a bed). In addition, the upper part of Figure 10 shows a projection of the point cloud within the rectangular parallelepiped 98 onto the XY plane (horizontal plane), and the lower part of Figure 10 shows a projection of the point cloud within the rectangular parallelepiped 98 onto the XZ plane (vertical plane).

[0091] Furthermore, based on the ratio value (ratio) R (=Zh / L) of the height (vertical length) Zh of the rectangular parallelepiped 98 (also refer to the lower part of FIG. 10) to the length L of the diagonal on the bottom surface of the rectangular parallelepiped 98, the body position of the person is determined. Whether the value R is large, medium, or small is discriminated by two threshold values V1 and V2 (V1 > V2). When the value R is large (R > V1), it is determined that the person has a standing position. When the value R is small (R < V2), it is determined that the person has a lying position. Also, when the value R is medium (V2 < R < V1), it is determined that the person has a sitting position. Note that instead of the value R (=Zh / L), another value R2 = Zh / E etc. using the area E of the bottom surface of the rectangular parallelepiped 98 may be used.

[0092] Thus, when it is determined in the second detection process that a person (such as a stationary person) exists in the living room 90, the controller 31 further determines the body posture of the person based on the point group detected by the difference process as the point group indicating the existence position of the person (step S31). Specifically, based on the ratio R etc. of the horizontal length L and the vertical length Zh of the point group (distribution range) detected by the difference process as the point group indicating the existence position of the person, it is determined whether the body posture of the person is a standing position, a sitting position, or a lying position. According to this, it is possible to determine the body position (standing position / sitting position / lying position) of the person by effectively using data (point group data) with a small amount of information (compared to image information).

[0093] In the next step S32, branching processing is performed according to the determination result of the body posture of the person (the determination result of step S31).

[0094] When it is determined that the body posture of the person is a standing position, the process proceeds to step S33. Also, when it is determined that the body posture of the person is a sitting position, the process proceeds to step S34, and when it is determined that the body posture of the person is a lying position, the process proceeds to step S35.

[0095] In step S33, a process for detecting standing up on a surface other than the floor is performed. Specifically, it is determined whether the person is standing on the floor (normal state) or standing on a surface other than the floor (on a chair, bed, etc.) (dangerous state). In other words, it is detected whether the person is standing on the floor or standing on a surface other than the floor (standing on a chair, bed, etc.). That is, it is detected whether the person is in a dangerous position (also called a dangerous posture). Specifically, it is determined whether the person is standing on the floor or not based on the minimum value Zmin (see Figure 10, lower panel) of the vertical position of the point cloud indicating the person's location.

[0096] More specifically, if the minimum value Zmin is less than a predetermined threshold Th1 (for example, 30 cm (the smaller of the chair seat height and the bed height)), it is determined that a person is standing on the floor. If the minimum value is greater than the predetermined threshold Th1, it is determined that a person is standing somewhere other than the floor (such as on a chair or on a bed). When it is determined that a person is standing somewhere other than the floor (standing on a chair or on a bed, etc.), the controller 31 issues a warning. This warning is notified to the caregiver via terminal devices 70 and 80. When it is determined that a person is standing on the floor, the controller 31 continues to monitor the person being cared for without issuing a warning.

[0097] In this case, whether or not a person is standing on the floor is determined based on the minimum vertical position Zmin of the point cloud, but this is not limited to this method. For example, whether or not a person is standing on the floor may also be determined based on the average vertical position Zave of the point cloud (see Figure 10, bottom).

[0098] After step S33, the process returns to step S13 (Figure 3).

[0099] In step S34, a floor-sitting detection process is performed. Specifically, when it is determined that the person is sitting, the controller 31 determines whether the person is sitting on the floor (dangerous state) or sitting somewhere other than the floor (on a chair, bed, etc.) (normal state). In other words, it further determines whether the person is sitting on the floor (dangerous state). Specifically, it determines whether the person is sitting on the floor or not based on the average value of the vertical position of the point cloud, Zave (see Figure 10, lower panel).

[0100] More specifically, if the average value Zave is less than a predetermined threshold Th2 (for example, 60 cm), it is determined that the person is sitting on the floor. If the average value Zave is greater than the predetermined threshold Th2, it is determined that the person is sitting somewhere other than the floor (e.g., on a chair or bed). When it is determined that the person is sitting on the floor, the controller 31 issues a warning. This warning is notified to the caregiver via terminal devices 70 and 80. When it is determined that the person is sitting somewhere other than the floor, the controller 31 does not issue a warning.

[0101] In this case, whether or not a person is sitting on the floor is determined based on the average value Zave of the vertical position of the point cloud. However, this is not limited to this method; for example, whether or not a person is sitting on the floor may also be determined based on the minimum value Zmin of the vertical position of the point cloud.

[0102] In step S35, a process for detecting whether the person is lying down outside of the bed is performed. Specifically, when it is determined that the person is lying down, the controller 31 determines whether the person is lying on the bed or somewhere other than the bed (on the floor). In other words, it further determines whether the person is lying on the bed (normal state) or somewhere other than the bed (dangerous state). Specifically, it is determined whether the person is lying on the bed or not based on the average value Zave of the vertical position of the point cloud (see Figure 10, lower panel).

[0103] More specifically, if the average value Zave is greater than a predetermined threshold Th3 (for example, 40 cm (bed height)), it is determined that a person is lying on the bed. If the average value Zave is less than the predetermined threshold Th3, it is determined that a person is lying somewhere other than on the bed (on the floor) (dangerous situation). When it is determined that a person is lying somewhere other than on the bed (on the floor), the controller 31 issues a warning. This warning is notified to the caregiver via terminal devices 70 and 80. When it is determined that a person is lying on the bed, the controller 31 does not issue a warning.

[0104] In this case, whether or not a person is lying on the bed is determined based on the average value Zave of the vertical position of the point cloud. However, this is not the only way to determine whether or not a person is lying on the bed; for example, it may also be determined based on the minimum value Zmin of the vertical position of the point cloud (see Figure 10, bottom panel).

[0105] In the next step S36 following step S34, a crouching detection process is performed. In the next step S37 following step S35, a crouching detection process is also performed. In other words, when it is determined that the person's posture is either sitting or lying down, the controller 31 further determines whether the person is crouching (in a dangerous or dangerous state).

[0106] In this application, "crouching" means the state in which the entire body is curled up into a small ball. The state of "crouching" includes "a state in which the body is curled up and squatting (a state in which the hip and knee joints are flexed while sitting)" (a form of sitting), and "a state in which the body is curled up and the legs are bent while lying on one's side (a state in which the hip and knee joints are flexed while lying on one's side)" (a form of lying down (lateral lying)). A state of crouching while sitting is presumed to be a state in which there is a high possibility that physical discomfort is occurring, and is considered a so-called dangerous state. Similarly, a state of crouching while lying down is also a state in which there is a high possibility that physical discomfort is occurring (dangerous state), or a state in which there is a high possibility that physical discomfort will occur (state requiring attention). When sleeping in a crouched (curled) state, the joints are compressed and lymphatic flow is impaired. By detecting this supine position (a form of lateral recumbent position) early as a state requiring attention (or a dangerous state), it becomes possible to avoid the situation worsening through actions such as changing the position of the person being cared for by the caregiver.

[0107] In the crouching detection process, it is determined whether a person is crouching (in a dangerous state, etc.) based on a comparison between the area occupied by the point cloud on the horizontal plane of the living space 90 (horizontal occupied area) (for example, the area of ​​the horizontal plane (bottom surface, etc.) of the rectangular parallelepiped 98 (Figure 9)) E2 and a predetermined threshold Th5.

[0108] More specifically, if the occupied area E2 is smaller than a predetermined threshold Th5, it is determined that the person is crouching, and if the occupied area E2 is larger than the predetermined threshold Th5, it is determined that the person is not crouching. When it is determined that the person is crouching, the controller 31 issues a warning (or a warning). This warning is notified to the caregiver via terminal devices 70 and 80. When it is determined that the person is not crouching, the controller 31 does not issue a warning. After steps S36 and S37, the process returns to step S13 (Figure 3).

[0109] Thus, in step S30, the following processes are executed: detection of standing up from a surface other than the floor (step S33), detection of sitting down on the floor (step S34), detection of lying down from a surface other than the bed (step S35), and detection of crouching (steps S36, S37). This makes it possible to appropriately detect a person's dangerous state (or state requiring attention).

[0110] After returning to step S13 from step S30, monitoring of the person being cared for by the millimeter-wave sensor 20 continues. Specifically, as long as the person remains stationary, (flag F is not reset) the processes in steps S13, S14, S15, S16, S18, S22, and S30 are repeated. If the person then starts moving again, the processes in steps S13, S14, and S21 are repeated. If the person who has started moving again leaves the room 90, the processes in S13, S14, S15, S16, S18, S23, and S24 are executed.

[0111] <5. Effects of the Embodiment> According to the above configuration, it is possible to detect people more appropriately using the millimeter-wave sensor 20.

[0112] In detail, if a person who was previously detected (a person who was moving) is no longer detected in the repeatedly executed first detection process (step S13), the second detection process is executed (steps S14, S15, S16). The second detection process involves processing the difference between pre-acquired data Dp, which was acquired in advance when no person was present in the room 90, and the current nearby measurement data Dc (D4, etc.). This makes it possible to detect the presence or absence of a person in the room by the second detection process, even if the presence or absence of a person cannot be accurately determined by the first detection process alone. In detail, the second detection process can detect the presence of a person in the room. For example, it is possible to more appropriately detect a person who is stationary in the room, a person who is moving very little, a person who is far from the millimeter-wave sensor 20, and / or a person who is not facing the millimeter-wave sensor 20. Alternatively, the second detection process can detect the absence of a person in the room (no person in the room).

[0113] Furthermore, the fact that the second detection process is performed immediately after the first detection process (in other words, the first detection process is performed immediately before the second detection process) confirms that a person was detected at some point. If only the second detection process is performed (without the first detection process), it is unclear whether the object detected based on the difference data (D40, etc.) of the second difference process is a person or an object (item) other than a person. On the other hand, if the first detection process is performed immediately before the second detection process, it can be appropriately inferred that the object detected in the second detection process immediately following the first detection process is the person that was moving just before, based on the fact that a moving person was detected at some point in the first detection process.

[0114] Furthermore, if the second detection process determines that a person is present in the room, the person's posture is further determined based on the point cloud detected by the second difference process, which represents the person's location (steps S16, S18, S22, S30). Therefore, not only the presence or absence of a person but also their posture can be determined (detected).

[0115] Furthermore, if the second detection process determines that no person is present in the room, the data currently acquired in the vicinity by the millimeter-wave sensor 20 is updated as pre-acquired data Dp (steps S16, S18, S23, S24). This allows more suitable data to be prepared as pre-acquired data Dp in preparation for subsequent second detection processes.

[0116] For example, consider a scenario where a person moves an object (a chair) to another location within room 90, then leaves room 90, and subsequently re-enters room 90. The person, initially detected within room 90 during the first detection process, is no longer detected, and the second detection process is executed. In this case, the pre-acquired data Dp is updated with data acquired in step S24, immediately after the object is moved and before re-entry. In this situation, if the pre-acquired data Dp (e.g., D0) before the update is used in the second detection process (after re-entry), not only the person but also the moved object (chair) may be detected in the second detection process. Conversely, if the second detection process is executed using the updated pre-acquired data Dp under these circumstances, it is possible to appropriately detect the point cloud corresponding to the person (without detecting the point cloud corresponding to the moved object). The updated pre-acquired data Dp includes the position information of the moved object. Therefore, the differential processing in the second detection process cancels out the information about the moved object (information contained in both sets of data being differentiated), making it possible to avoid the effects of the object's movement. Consequently, it becomes possible to more accurately detect the presence or absence and posture of a person.

[0117] The same applies when other items (beds, tables, shelves, etc.) within room 90 are moved due to changes in the room's layout. The updated pre-acquired data Dp includes location information of the moved items (items after the layout change). Therefore, it is possible to avoid the effects of item movement.

[0118] Furthermore, it is preferable that the pre-acquired data Dp be updated as frequently as possible (for example, each time it is confirmed that no one is present in the room). Also, since the movement of items within a room is often carried out by the person in the room (before that person leaves), it is efficient for the pre-acquired data Dp to be updated when it is determined that a person who was detected in the room has left the room (step S24). In other words, if no object is detected by the differential processing (second differential processing) of the second detection processing (step S16) after the first detection processing, it is preferable that it is determined that no person is present in the room (step S23), and that the pre-acquired data Dp is updated at that time (step S24).

[0119] <6. Variations, etc.> The embodiments of this invention have been described above, but this invention is not limited to those described above.

[0120] For example, in the above embodiments, a configuration is shown in which the process returns from step S17 to step S13, but the process is not limited to this, and for example, the process may proceed from step S17 to step S24.

[0121] Furthermore, in the above embodiments, the pre-acquired data Dp is updated (step S24) when no person (object) is detected by the second detection process (step S16), but this is not limited to this. For example, the pre-acquired data Dp may be updated periodically. Specifically, the presence or absence of a person in the living room 90 is determined periodically (by visual inspection by a caregiver, etc., or automatically by the controller 31, etc.), and if a determination result is obtained that no person is present, the pre-acquired data Dp may be updated with the measurement data (measurement data from the millimeter-wave sensor 20) acquired at the time of the determination.

[0122] Furthermore, in the above embodiments, the presence or absence of a person is determined in the second detection process based on the difference data D40 calculated over the entire detection target area of ​​the millimeter-wave sensor 20. However, the present invention is not limited thereto. For example, the second detection process may be executed by narrowing down the detection target area to the area where a person was found to exist once detected in the repeatedly executed first detection process (such as the columnar spatial area on the floor where the person was found (3D spatial area), or the spatial area of ​​the rectangular parallelepiped 98 described above). In short, the second detection process may be executed using a portion of the entire detection target area of ​​the millimeter-wave sensor 20 (the area where a person was detected in the first detection process) as the target area. In this case, the target area of ​​the second detection process is narrowed down to the area where a person was found to exist in the first detection process, making it possible to more reliably exclude stationary objects other than people. For example, even if the position of a stationary object (such as a chair) at the time of acquiring the pre-acquired data Dp differs from the actual (current) position of the stationary object at the time of the second detection process, if the stationary object exists outside the area where a person was detected in the first detection process (i.e., the stationary object is not in the target area of ​​the second detection process), it is possible to exclude the stationary object and detect only the person.

[0123] In this modified example, it is preferable that the area in which a person who was initially detected in the repeatedly executed first detection process existed immediately before the person was no longer detected (the area in which the person was last detected in the first detection process) is set as the target area for the second detection process.

[0124] Furthermore, in the above embodiments, the first detection process is performed based on two sets of measurement data (for example, D1 and D2 (see Figure 6)) consisting of measurement data acquired by the millimeter-wave sensor 20 at one point in time within a period corresponding to one frame and measurement data acquired by the millimeter-wave sensor 20 at one point in time within the next period corresponding to one frame. However, the invention is not limited to this.

[0125] For example, as shown in Figure 11, data from multiple (e.g., 32) subframes may be measured by the millimeter-wave sensor 20 within a period corresponding to each frame, and the average data of these multiple (32) subframes may be acquired as the data for each frame. In other words, data based on data measured at an even higher frequency (subframe data) may be acquired as each frame data (measurement data Di from the millimeter-wave sensor 20, etc.). Figure 11 shows a modified example of the measurement data from the millimeter-wave sensor 20.

[0126] The second detection process is similar. Both the pre-acquired data Dp and the current nearby measurement data Dc may be the average data of multiple (e.g., 32) subframes measured within a period corresponding to each frame. The same applies to the latest data (D4, etc.) used to update the pre-acquired data Dp.

[0127] Furthermore, in the above embodiments, the first detection process is performed based on two measurement data (e.g., D1, D2) acquired by the millimeter-wave sensor 20 at two points in time within a short period of time (corresponding to a period of 2 frames) in the immediate vicinity, but is not limited to this.

[0128] For example, the first detection process may be performed based on 32 measurement data (subframe data d1, d2, ..., d32) (see Figure 11) acquired by the millimeter-wave sensor 20 at 32 points in time within a short period of time (a period corresponding to one frame) in the immediate vicinity. More specifically, the average data of the 32 measurement data d1, d2, ..., d32 may be calculated, 32 difference data between the average data and each subframe data may be calculated, and the point cloud for that frame (and / or the point cloud for each subframe) may be detected as a point cloud of a moving person based on these 32 difference data.

[0129] Furthermore, in each of the above embodiments, the processing in step S30 is performed immediately after the second detection processing (step S16) based on the measurement data acquired in the second detection processing, but is not limited to this. For example, the processing in step S30 may be performed immediately after the first detection processing (step S13) (step S21, etc.) based on the measurement data acquired in the first detection processing (measurement data from the millimeter-wave sensor 20). [Explanation of Symbols]

[0130] 1. Monitoring System 10 Detection device 20 mm wave sensor 30 processing units 31 Controllers 32 Storage Unit 70,80 Terminal devices 90 Room 92 beds Dp Pre-acquired data F flag

Claims

1. It is a monitoring system that keeps an eye on the person inside the room. A millimeter-wave sensor that uses the space within the aforementioned living room as the detection target space, A control unit that detects a person in the room using data acquired by the millimeter-wave sensor, A storage unit for storing pre-acquired data, which is data previously acquired by the millimeter-wave sensor at a time when no person is present in the room, Equipped with, The control unit, Based on multiple data acquired by the millimeter-wave sensor at multiple points in time within a short period, a first detection process is repeatedly executed to detect a person moving within the room. A monitoring system characterized in that, if a person who was previously detected in the first detection process, which is repeatedly executed, is no longer detected, a second detection process is performed to detect a person present in the room by performing a difference processing between the previously acquired data and the data acquired by the millimeter-wave sensor within the minute period.

2. The monitoring system according to claim 1, characterized in that the control unit performs the second detection process with the area in which a person was detected in the first detection process as the target area.

3. The monitoring system according to claim 1, characterized in that when the control unit determines that there is no person in the room, it updates the data acquired by the millimeter-wave sensor as the previously acquired data.

4. The monitoring system according to claim 3, characterized in that, if no object is detected by the differential processing of the second detection process, the control unit determines that there is no person in the room and updates the data acquired by the millimeter-wave sensor within the minute period as the pre-acquired data.

5. The monitoring system according to claim 1, characterized in that, when the control unit determines in the second detection process that a person is present in the room, it further determines the posture of the person based on the point cloud detected by the difference processing as a point cloud indicating the location of the person.

6. The monitoring system according to claim 5, characterized in that, when the control unit determines in the second detection process that a person is present in the room, it determines whether the person is standing, sitting, or lying down based on the ratio of the horizontal length to the vertical length of the point cloud detected by the difference processing as a point cloud indicating the location of the person.

7. The monitoring system according to claim 6, characterized in that when the control unit determines that the person is standing, it further determines whether the person is standing on the floor or somewhere other than the floor, based on the vertical position of the point cloud.

8. The monitoring system according to claim 6, characterized in that when the control unit determines that the person's posture is seated, it further determines whether the person is sitting on the floor based on the vertical position of the point cloud.

9. The monitoring system according to claim 6, characterized in that when the control unit determines that the person is lying down, it further determines whether the person is lying on a bed or somewhere other than a bed, based on the vertical position of the point cloud.

10. The monitoring system according to claim 6, characterized in that when the control unit determines that the person's posture is either sitting or lying down, it further determines whether the person is crouching or not based on the area occupied by the point cloud on the horizontal plane.

11. A control method for a monitoring system that keeps an eye on a person inside a room, a) A step of storing data previously acquired by a millimeter-wave sensor that detects the space within the room as the detection target space, at a time when no person is present in the room, as pre-acquired data in the storage unit of the monitoring system, b) The step of repeatedly performing a first detection process that detects a person moving within the room based on multiple data acquired by the millimeter-wave sensor at multiple points in time within a short period of time, c) If a person who was previously detected in the repeatedly executed first detection process is no longer detected, the second detection process is performed to detect a person present in the room by performing a difference processing between the previously acquired data and the data acquired by the millimeter-wave sensor within the specified short period. A control method for a monitoring system, characterized by comprising the following features.

12. A computer installed in the control unit of a monitoring system that keeps an eye on people inside a room, a) A step of storing data previously acquired by a millimeter-wave sensor that detects the space within the room as the detection target space, at a time when no person is present in the room, as pre-acquired data in the storage unit of the monitoring system, b) The step of repeatedly performing a first detection process that detects a person moving within the room based on multiple data acquired by the millimeter-wave sensor at multiple points in time within a short period of time, c) If a person who was previously detected in the repeatedly executed first detection process is no longer detected, the second detection process is performed to detect a person present in the room by performing a difference processing between the previously acquired data and the data acquired by the millimeter-wave sensor within the specified short period. A program to execute.

Images