Monitoring system, information processing device, program, and monitoring method

Abstract

To provide a monitoring system 101 capable of acquiring biometric information with high accuracy regardless of the state of a person to be monitored. The monitoring system 101 of the present disclosure comprises a head position estimation unit 22 that estimates the head position of the person being monitored 1, a measured location estimation unit 23 that estimates the position of a measured location 2 of the person being monitored 1 based on information on the head position estimated by the head position estimation unit 22, and a biometric information acquisition unit 25 that acquires biometric information of the person being monitored 1 at the measured location 2 estimated by the measured location estimation unit 23.

Description

[Technical field]

[0001] The present disclosure relates to a monitoring system, an information processing device, a program, and a monitoring method. [Background technology]

[0002] In recent years, monitoring systems have been developed that can acquire biological information of monitored subjects in nursing care facilities, elderly care facilities, hospitals, elderly people living alone, etc., without the subjects having to wear any equipment. For example, Patent Document 1 discloses a method of identifying a displacement site on the surface of a living body that is displaced by the pulsation of the heart or blood vessels using an infrared sensor, and detecting the pulsation of the heart or blood vessels at the identified displacement site using a millimeter wave sensor. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO2022 / 196469 publication Summary of the Invention [Problem to be solved by the invention]

[0004] The infrared sensor of Patent Document 1 uses infrared rays to identify the displaced part without contact. However, infrared rays are easily absorbed by clothes, bedding, etc., so there is a disadvantage that it is not possible to capture the displacement of the surface of the living body hidden by clothes, bedding, etc. Therefore, the method of Patent Document 1 has a problem that it is not possible to identify the displaced part depending on the state of the subject, such as when the subject is covered with a bedding, and it is not possible to acquire biometric information with high accuracy.

[0005] The present disclosure has been made to solve the problems described above, and aims to provide a monitoring system that can acquire biometric information with high accuracy regardless of the condition of the person being monitored. [Means for solving the problem]

[0006] The monitoring system according to the present disclosure is a monitoring target of a person in which at least a part of a measured part is hidden. An optical sensor that acquires information and a person being watched over based on the information acquired by the optical sensor A head position estimation unit that estimates a head position, and information on the head position estimated by the head position estimation unit Using first distance information indicating a distance between a predetermined head and a measured part of the person being watched over, and second distance information indicating a distance between the person being watched over and the optical sensor, a measured location estimation unit that estimates the location of a measured location of a person being watched over; and a biological information acquisition unit that acquires biological information of the person being watched over at the measured location estimated by the measured location estimation unit. The measured point estimation unit corrects the information on the first distance using the information on the second distance, and estimates the position of the measured point using the corrected information on the first distance. do.

[0007] The information processing device according to the present disclosure includes: The light sensor captures At least a part of the measured area of ​​the person being watched is hidden Based on the information, A head position estimation unit that estimates a head position, and information on the head position estimated by the head position estimation unit Using first distance information indicating a distance between a predetermined head and a measured part of the person being watched over, and second distance information indicating a distance between the person being watched over and the optical sensor, a measured location estimation unit that estimates the position of the measured location, and a biological information acquisition unit that acquires biological information of the person being watched over at the measured location estimated by the measured location estimation unit. The measured point estimation unit corrects the information on the first distance using the information on the second distance, and estimates the position of the measured point using the corrected information on the first distance. do.

[0008] The program according to the present disclosure is installed on a computer. The light sensor captures At least a part of the measured area of ​​the person being watched is hidden Based on the information, A head position estimation function that estimates the position of the head, and information on the head position estimated by the head position estimation function Using first distance information indicating a distance between a predetermined head and a measured part of the person being watched over, and second distance information indicating a distance between the person being watched over and the optical sensor, A program for realizing a measurement location estimation function for estimating the position of a measurement location, and a biometric information acquisition function for acquiring biometric information of a person being watched at the measurement location estimated by the measurement location estimation function. the measured point estimation function is a program for correcting information on the first distance using information on the second distance, and estimating the position of the measured point using the corrected information on the first distance. It is.

[0009] The monitoring method according to the present disclosure includes: The light sensor captures At least a part of the measured area of ​​the person being watched is hidden Based on the information, A head position estimation step of estimating a head position, and information on the head position estimated in the head position estimation step. Using first distance information indicating a distance between a predetermined head and a measured part of the person being watched over, and second distance information indicating a distance between the person being watched over and the optical sensor,a measured location estimating step of estimating a position of the measured location, and a biological information acquiring step of acquiring biological information of the person being watched over at the measured location estimated in the measured location estimating step. The measured point estimating step includes correcting information on the first distance using information on the second distance, and estimating a position of the measured point using the corrected information on the first distance. do. Effect of the Invention

[0010] According to the monitoring system, information processing device, program, and monitoring method disclosed herein, it is possible to acquire biological information with high accuracy regardless of the state of the person being watched over. [Brief description of the drawings]

[0011] [Figure 1] 1 is a schematic diagram illustrating a configuration example of a monitoring system according to a first embodiment of the present disclosure. [Diagram 2] 1 is a system block diagram showing a monitoring system according to a first embodiment of the present disclosure. [Diagram 3] FIG. 11 is a schematic diagram showing a first distance. [Figure 4] 1 is a hardware configuration diagram showing a hardware configuration of an information processing device provided in a monitoring system according to embodiment 1 of the present disclosure. [Diagram 5] 4 is a flowchart showing the procedure of a monitoring method by the monitoring system according to the first embodiment of the present disclosure. [Figure 6] 5 is a flowchart showing a processing procedure of a measured point estimation step performed by the monitoring system according to the first embodiment of the present disclosure. [Figure 7] 1 is a schematic diagram showing an irradiation position of millimeter waves by a monitoring system according to a first embodiment of the present disclosure. [Figure 8] 1 is a system block diagram showing a monitoring system according to a first modified example of the first embodiment of the present disclosure. [Figure 9] 13 is a flowchart showing a processing procedure of a measured part estimation step performed by a monitoring system according to a first modification of the first embodiment of the present disclosure. [Figure 10]FIG. 2 is a system block diagram showing a monitoring system according to a second modification of the first embodiment of the present disclosure. [Figure 11] 13 is a flowchart showing a processing procedure of a measured part estimation step performed by a monitoring system according to a second modification of the first embodiment of the present disclosure. [Figure 12] FIG. 11 is a system block diagram showing a monitoring system according to a third modification of the first embodiment of the present disclosure. [Figure 13] 13 is a flowchart showing a processing procedure of a measured part estimation step performed by a monitoring system according to a third modification of the first embodiment of the present disclosure. [Figure 14] FIG. 11 is a system block diagram showing a monitoring system according to a fourth modified example of the first embodiment of the present disclosure. [Figure 15] 13 is a flowchart showing a processing procedure of a measured part estimation step performed by a monitoring system according to a fourth modification of the first embodiment of the present disclosure. [Figure 16] 1 is a schematic diagram showing the irradiation position of a millimeter wave when the position of a measured point is estimated using directional information of two measured points. FIG. [Figure 17] FIG. 11 is a system block diagram showing a monitoring system according to a second embodiment of the present disclosure. [Figure 18] 11 is a flowchart showing the procedure of a monitoring method by a monitoring system according to embodiment 2 of the present disclosure. [Figure 19] FIG. 11 is a system block diagram showing a monitoring system according to a first modified example of the second embodiment of the present disclosure. [Figure 20] FIG. 11 is a system block diagram showing a monitoring system according to a third embodiment of the present disclosure. [Figure 21] 13 is a flowchart showing the steps of a monitoring method performed by a monitoring system according to embodiment 3 of the present disclosure. [Figure 22] FIG. 13 is a system block diagram showing a monitoring system according to a first modified example of the third embodiment of the present disclosure. [Diagram 23] FIG. 13 is a system block diagram showing a monitoring system according to a second modification of the third embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, examples of a monitoring system, an information processing device, a program, and a monitoring method according to the present disclosure will be described with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals, and descriptions thereof will not be repeated.

[0013] Embodiment 1 The first embodiment of the present disclosure relates to a monitoring system 101 that estimates the head position of a person being monitored and estimates the position of a measured part based on information on the estimated head position. The first embodiment of the present disclosure also relates to an information processing device 201 that executes processing related to the monitoring system 101, a program 301 that causes a computer to execute processing related to the monitoring system 101, and a monitoring method using the monitoring system 101.

[0014] <Configuration of First Embodiment> The configuration of a monitoring system 101 according to the first embodiment of the present disclosure will be described with reference to Fig. 1, Fig. 2, and Fig. 3. Fig. 1 is a schematic diagram showing a configuration example of the monitoring system 101, and Fig. 2 is a system block diagram of the monitoring system 101. Fig. 3 is a schematic diagram showing a first distance l.

[0015] As shown in FIG. 1, the monitoring system 101 includes an optical sensor 10, an information processing device 201, and a millimeter wave sensor 40. The monitoring system 101 is a system that estimates the position of a measured part 2 of the person being watched 1 regardless of the state of the person being watched 1, and acquires biological information of the person being watched 1 with high accuracy, using the optical sensor 10 and the millimeter wave sensor 40. Here, the measured part 2 is a part where the biological surface is displaced due to breathing, heartbeat, etc. FIG. 1 shows an example in which the chest is set as the measured part 2. The measured part 2 is not limited to the chest, and may be any part where the biological surface is displaced due to breathing, heartbeat, etc., such as the neck, side, chest, back, etc. In the following, for simplicity of explanation, an example in which the chest is set as the measured part 2 will be explained.

[0016] As shown in Fig. 1(a), at least the optical sensor 10 and the millimeter wave sensor 40 of the monitoring system 101 are installed in a room R of the person being watched over 1. Note that, as shown in Fig. 1(b), in addition to the optical sensor 10 and the millimeter wave sensor 40, an information processing device 201 may also be installed in the room R of the person being watched over 1.

[0017] The optical sensor 10 and the millimeter wave sensor 40 transmit and receive various information to and from the information processing device 201. The communication between the optical sensor 10 and the millimeter wave sensor 40 and the information processing device 201 may be performed via a signal line or wireless communication such as WiFi.

[0018] The optical sensor 10 and the millimeter wave sensor 40 can acquire information on the position of an object, and the optical sensor 10 and the millimeter wave sensor 40 are calibrated in advance for alignment so that there is no discrepancy in the information on the position of the object acquired by each of the optical sensor 10 and the millimeter wave sensor 40. Specifically, conversion equations for the coordinate systems of each of the optical sensor 10 and the millimeter wave sensor 40 are stored in advance in a storage unit 30 described below, and the conversion equations are used to convert from the coordinates of the optical sensor 10 to the coordinates of the millimeter wave sensor 40, thereby aligning the information acquired by the optical sensor 10 with the information used by the millimeter wave sensor 40.

[0019] The optical sensor 10 is a sensor that acquires information on the person being watched 1 in order to estimate the position of the head 3 of the person being watched 1. The optical sensor 10 is, for example, a visible camera, an infrared camera, a stereo camera, a LiDAR (Light Detection And Ranging), etc., and acquires information on the person being watched 1 using visible light or infrared light. The information on the person being watched 1 acquired by the optical sensor 10 is, for example, information on the shape of the person being watched 1, information on the skeleton, information on the movement, etc. In the present embodiment 1, a case will be described in which the optical sensor 10 acquires information on the shape of the person being watched 1 as the information on the person being watched 1. The information on the shape of the person being watched 1 is information on the outline of the head 3 or the torso 4, information on the features of the face, etc. It should be noted that the information on the shape of the person being watched 1 includes at least information on the shape of the head 3 of the person being watched 1. That is, the optical sensor 10 acquires information on the shape of the person being watched 1 including the outline of the head 3 of the person being watched 1 or information on the features of the face.

[0020] The millimeter wave sensor 40 is a sensor that detects displacement at a measurement location 2 of the person being watched over 1 in order to acquire bio-information of the person being watched over 1. Micro-displacements of about several tens of microns to several millimeters are observed on the surface of a living body due to breathing, heartbeat, etc. The millimeter wave sensor 40 detects the micro-displacements on the surface of the living body due to breathing, heartbeat, etc., to acquire bio-information such as respiratory rate, pulse rate, blood pressure, etc.

[0021] The millimeter wave sensor 40 uses millimeter waves to detect displacement of the biological surface at the measured location 2 of the person being watched over 1. In detail, millimeter waves are irradiated to the measured location 2 of the person being watched over 1 and reflected waves from the measured location 2 are received to detect displacement of the biological surface at the measured location 2 of the person being watched over 1. Since millimeter waves pass through clothing, bedding, etc., the millimeter wave sensor 40 can capture displacement of the biological surface hidden by clothing, bedding, etc.

[0022] Moreover, the millimeter wave sensor 40 is controlled by a millimeter wave sensor control unit 24 described later to aim at the measured location 2 and irradiate millimeter waves. Note that in order to obtain the displacement with high accuracy by the millimeter wave sensor 40, the driving method of the millimeter wave sensor 40 or the signal processing method of the millimeter wave sensor 40 may be adjusted to improve the S / N ratio. Methods for improving the S / N ratio include, for example, a driving method of the millimeter wave sensor 40 that performs beam forming on the measured location 2, a driving method of the millimeter wave sensor 40 that focuses the transmission beam scan on the measured location 2, coherent integration which is a signal processing method of the millimeter wave sensor 40, and combinations thereof.

[0023] The information processing device 201 is a computer that performs processing for estimating the position of the measured location 2 of the person being watched over 1 based on information obtained from the optical sensor 10, and processing for acquiring biological information of the person being watched over 1 based on information obtained from the millimeter wave sensor 40. As shown in FIG. 2, the information processing device 201 includes a processing unit 20 and a storage unit 30.

[0024] As shown in FIG. 2, the processing unit 20 includes an optical sensor control unit 21, a head position estimation unit 22, a measured location estimation unit 23, a millimeter wave sensor control unit 24, and a biological information acquisition unit 25. The optical sensor control unit 21 controls the optical sensor 10 to acquire shape information of the watching target 1. The head position estimation unit 22 estimates the position of the head 3 of the watching target 1 based on the shape information of the watching target 1 acquired by the optical sensor 10. The measured location estimation unit 23 estimates the position of the measured location 2 using the information on the position of the head 3 estimated by the head position estimation unit 22 and information on a first distance l set in advance. Here, the information on the first distance l is information indicating the distance between the head 3 and the measured location 2 as shown in FIG. 3. The information on the first distance l is set in advance in a database 31 to be described later, for example. The millimeter wave sensor control unit 24 controls the millimeter wave sensor 40 to detect a displacement at the position of the measured location 2 estimated by the measured location estimation unit 23. When the millimeter wave sensor 40 detects the estimated displacement at the measurement location 2, the biological information acquiring unit 25 acquires biological information of the person being watched over 1 based on the detected displacement.

[0025] 2, the storage unit 30 stores a program 301 and a database 31. The information processing device 201 reads out and executes the program 301 stored in the storage unit 30 to estimate the position of the measured part 2 of the person being watched over 1 and to acquire biological information of the person being watched over 1.

[0026] The program 301 is a program for causing a computer to function as the information processing device 201 according to embodiment 1 and to execute the monitoring method according to embodiment 1. In detail, the program 301 causes the computer to realize a head position estimation function for executing the function of the head position estimation unit 22, a measured part estimation function for executing the function of the measured part estimation unit 23, and a biological information acquisition function for executing the function of the biological information acquisition unit 25.

[0027] The database 31 stores information used when estimating the positions of the head 3 and the measured location 2 of the person being watched over 1 and performing processing to acquire biometric information of the person being watched over 1. In detail, the database 31 stores information on the shape of the head 3 of the person being watched over 1, information on a first distance l indicating the distance between the head 3 of the person being watched over 1 and the measured location 2, and a conversion formula for a coordinate system for calibrating the optical sensor 10 and the millimeter wave sensor 40.

[0028] The watching system 101 may include a display device (not shown). The display device is a device that displays the biometric information of the watching target 1 acquired by the biometric information acquiring unit 25.

[0029] Next, the hardware configuration of the information processing device 201 included in the monitoring system 101 will be described with reference to Fig. 4. Fig. 4 is a hardware configuration diagram of the information processing device 201.

[0030] 4, the information processing device 201 includes a calculation device 211, a storage device 212, an input device 213, an auxiliary storage device 214, and an output device 215. The calculation device 211, the storage device 212, the input device 213, the auxiliary storage device 214, and the output device 215 are connected to each other via a signal line 216.

[0031] The arithmetic device 211 is a device that realizes each function of the processing unit 20 shown in Fig. 2. The arithmetic device 211 realizes each function of the processing unit 20 of the information processing device 201 by reading out a necessary program 301 from the auxiliary storage device 214 and executing processing. The arithmetic device 211 is, for example, a processor, and the processor is an IC (Integrated Circuit) that performs arithmetic processing. Specific examples of the processor are, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit). The arithmetic device 211 may also be a personal computer, a microcomputer board, or an FPGA (Field Programmable Gate Array) board.

[0032] The storage device 212 is a main storage device of the information processing device 201. The main storage device temporarily stores calculations of the processing performed by the arithmetic device 211. The storage device 212 is, for example, a RAM (Random Access Memory).

[0033] The input device 213 is an input interface of the information processing device 201. The input device 213 inputs, for example, information on the shape of the person being watched over 1 acquired by the optical sensor 10, or information on the displacement at the measured location 2 of the person being watched over 1 detected by the millimeter wave sensor 40, to the arithmetic device 211.

[0034] 2, and is an auxiliary storage device of the information processing device 201. The auxiliary storage device 214 stores a program 301 required to realize each function of the processing unit 20 of the information processing device 201, and a database 31 that stores information used when estimating the position of the measured part 2 of the person being watched over 1 and performing processing to acquire biological information of the person being watched over 1. The auxiliary storage device 214 is, for example, a read only memory (ROM), a hard disk drive (HDD), or a solid state drive (SSD).

[0035] The output device 215 is an output interface of the information processing device 201. The output device 215 outputs, for example, the biometric information of the watching target 1 acquired by the biometric information acquisition unit 25 to a display device (not shown).

[0036] A signal line 216 is a transmission line for transmitting and receiving data between the components shown in FIG.

[0037] <Monitoring method by the monitoring system according to the first embodiment> Next, the procedure of estimating the positions of the head 3 and the measured parts 2 of the person being watched over 1 and acquiring the biological information of the person being watched over 1, which are performed by the watching system 101 according to the first embodiment, will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the procedure of the watching method by the watching system 101 according to the first embodiment.

[0038] As shown in FIG. 5, the monitoring method by the monitoring system 101 includes a subject information acquiring step S100, a head position estimating step S200, a measured part estimating step S300, a displacement detecting step S400, and a biological information acquiring step S500.

[0039] The target person information acquisition step S100 is a step in which the optical sensor 10 acquires shape information of the watching target person 1. In detail, first, the optical sensor control unit 21 of the information processing device 201 controls the optical sensor 10 to acquire shape information of the watching target person 1. Next, the optical sensor 10 controlled by the optical sensor control unit 21 acquires the shape information of the watching target person 1. Then, the input device 213, which is an input interface of the information processing device 201, receives the shape information of the watching target person 1 acquired by the optical sensor 10.

[0040] The head position estimation step S200 is a step of estimating the position of the head 3 of the person being watched over 1 based on the shape information of the person being watched over 1 acquired in the person information acquisition step S100. The head position estimation step S200 is executed by the head position estimation unit 22 of the information processing device 201. The head position estimation unit 22 estimates the position of the head 3 using the shape information of the person being watched over 1 acquired by the optical sensor 10. In detail, the head position estimation unit 22 identifies the head 3 by collating the shape information of the head 3 of the person being watched over 1 acquired by the optical sensor 10 with the shape information of the head 3 of the person being watched over 1 previously set in the database 31, thereby estimating the position of the head 3. Note that the head position estimation unit 22 is not limited to using the shape information of the head 3 of the person being watched over 1, and may estimate the position of the head 3 by extracting facial features of the person being watched over 1 using a face recognition system from the shape information of the person being watched over 1 acquired by the optical sensor 10, and recognizing the head 3 based on the extracted facial feature information. Here, information on the amount of facial features is, for example, information indicating the positions of facial feature points such as the eyes, nose, and mouth, the position of the face region, or the size of the face region.

[0041] The measured part estimating step S300 is a step of estimating the position of the measured part 2 using information on the position of the head 3 estimated in the head position estimating step S200 and information on a preset first distance l. The measured part estimating step S300 is executed by the measured part estimating unit 23 of the information processing device 201.

[0042] In the measured point estimation step S300, if the measured point estimation unit 23 obtains information on the position of the head 3, information on the first distance l, and also information on the direction of the measured point 2 relative to the head 3, the position of the measured point 2 can be identified as a single location. On the other hand, since the optical sensor 10 obtains shape information of the person being watched over 1 using visible light or infrared light that is easily absorbed by clothes, bedding, etc., for example, when the person being watched over 1 is covered with a bedding, the optical sensor 10 cannot obtain information on the part covered with the bedding. In other words, when the person being watched over 1 is covered with a bedding, the optical sensor 10 can obtain only shape information of the head 3, but cannot obtain shape information of the torso 4. In this case, the processing unit 20 cannot calculate the direction of the torso 4 (measured part 2) relative to the head 3 using the shape information of the torso 4, so it is necessary to estimate the position of the measured part 2 using only the position information of the head 3 and the information of the first distance l, without using information on the direction of the measured part 2 relative to the head 3.

[0043] 6 and 7, a process procedure of the measured part estimation unit 23 when estimating the position of the measured part 2 using only the information on the position of the head 3 and the information on the first distance l will be described. FIG. 6 is a flowchart showing the process procedure of the measured part estimation step S300 by the watching system 101 according to the first embodiment. FIG. 7 is a schematic diagram showing the irradiation position of the millimeter wave of the millimeter wave sensor 40, and illustrates a case where the person being watched over 1 is covered with a futon. Note that, when estimating the position of the measured part 2 using only the information on the position of the head 3 and the information on the first distance l, the measured part estimation step S300 is executed by the measured part estimation unit 23 and the millimeter wave sensor control unit 24 of the information processing device 201.

[0044] 6 and 7, the measured part estimation unit 23 sets a position that is a first distance l away from the estimated position of the head 3 as the irradiation position 41 (step S310). The set irradiation positions 41 are, for example, a plurality of positions on a circumference of a circle having a radius of the first distance l and centered on the estimated position of the head 3.

[0045] Next, the millimeter wave sensor control unit 24 controls the millimeter wave sensor 40 to irradiate millimeter waves to one irradiation position 41a of the set irradiation positions 41 (step S320). The millimeter wave sensor 40 irradiates the irradiation position 41a with millimeter waves and receives a reflected wave from the irradiation position 41a to detect a displacement of the biological body surface at the irradiation position 41a. The measured part estimation unit 23 determines whether the millimeter wave sensor 40 has detected a displacement at the irradiation position 41a (step S330). If the millimeter wave sensor 40 has not detected a displacement at the irradiation position 41a (No in step S330), the millimeter wave sensor control unit 24 controls the millimeter wave sensor 40 to irradiate millimeter waves to an unirradiated irradiation position 41b, which is one of the set irradiation positions 41 (step S340), and the measured part estimation unit 23 determines whether the millimeter wave sensor 40 has detected a displacement at the irradiation position 41b (step S330). The processes of S330 and S340 are repeated until the millimeter wave sensor 40 detects a displacement (Yes in step S350). When the millimeter wave sensor 40 detects a displacement, the measured part estimation unit 23 estimates that the irradiation position 41 at that time is the position of the measured part 2 (step S350).

[0046] Incidentally, irradiation position 41 need not necessarily be a position spaced apart from the estimated head position by first distance 1, but may be a position spaced apart from the estimated head position by a distance several centimeters longer than first distance 1, or a distance several centimeters shorter than first distance 1.

[0047] As described above, even when the person being watched over 1 is covered with a blanket, for example, the measured location estimation unit 23 can estimate the measured location 2 with high accuracy by setting the irradiation position 41 using information on the position of the head 3 and information on the preset first distance l and having the millimeter wave sensor 40 detect the displacement at the irradiation position 41. In other words, even when only information on the position of the head 3 is obtained by the optical sensor 10 and the head position estimation unit 22, the measured location 2 can be estimated by detecting the displacement at the irradiation position 41, which is likely to be the measured location 2, using the millimeter wave sensor 40.

[0048] After estimating the position of the measured location 2 in the measured location estimation step S300, a displacement detection step S400 is performed as shown in Fig. 5. The displacement detection step S400 is a step in which the millimeter wave sensor 40 detects the displacement at the measured location 2 estimated in the measured location estimation step S300, and is performed by the millimeter wave sensor 40 and the millimeter wave sensor control unit 24 of the information processing device 201. In detail, first, the millimeter wave sensor control unit 24 controls the millimeter wave sensor 40 to detect the displacement at the estimated measured location 2. Next, the millimeter wave sensor 40 controlled by the millimeter wave sensor control unit 24 detects the displacement at the measured location 2. Then, the input device 213, which is an input interface of the information processing device 201, receives information on the displacement at the measured location 2 detected by the millimeter wave sensor 40.

[0049] The biological information acquisition step S500 is a process of acquiring biological information of the person being watched over 1 based on the displacement detected in the displacement detection step S400, and is performed by the biological information acquisition unit 25 of the information processing device 201. The biological information acquisition unit 25 acquires biological information such as the respiratory rate, pulse rate, blood pressure, etc. of the person being watched over 1 based on the displacement of the body surface at the measurement location 2 detected by the millimeter wave sensor 40.

[0050] According to the above process, the position of the head 3 of the person being watched over 1 is estimated using the optical sensor 10, and the position of the measured part 2 can be estimated using information on the estimated position of the head 3 and information on a first distance l indicating a preset distance between the head 3 and the measured part 2. Even if only information on the position of the head 3 is obtained by the optical sensor 10 and the head position estimation unit 22, the measured part 2 can be estimated by detecting a displacement at the irradiation position 41, which is likely to be the measured part 2, using the millimeter wave sensor 40. In this way, the measured part 2 can be estimated regardless of the state of the person being watched over 1, and therefore biological information can be acquired with high accuracy.

[0051] <Effects of the First Embodiment> Next, the operation and effects of the watching system 101 according to the first embodiment of the present disclosure will be described.

[0052] The monitoring system 101 according to embodiment 1 of the present disclosure comprises a head position estimation unit 22 which estimates the head position of the person being watched 1, a measured location estimation unit 23 which estimates the position of the measured location 2 of the person being watched 1 based on information on the head position estimated by the head position estimation unit 22, and a biometric information acquisition unit 25 which acquires biometric information of the person being watched 1 at the measured location 2 estimated by the measured location estimation unit.

[0053] An information processing device 201 relating to embodiment 1 of the present disclosure includes a head position estimation unit 22 that estimates the head position of the person being watched over 1, a measured location estimation unit 23 that estimates the position of a measured location 2 of the person being watched over 1 based on information on the head position estimated by the head position estimation unit 22, and a biometric information acquisition unit 25 that acquires biometric information of the person being watched over 1 at the measured location 2 estimated by the measured location estimation unit 23.

[0054] The program 301 relating to embodiment 1 of the present disclosure enables a computer to realize a head position estimation function that estimates the head position of the person being watched over 1, a measured location estimation function that estimates the position of the measured location 2 based on information on the head position estimated by the head position estimation function, and a biometric information acquisition function that acquires biometric information of the person being watched over 1 at the measured location 2 estimated by the measured location estimation function.

[0055] The monitoring method according to embodiment 1 of the present disclosure includes a head position estimation step S200 for estimating the head position of the person being watched 1, a measured location estimation step S300 for estimating the position of the measured location 2 based on information on the head position estimated in the head position estimation step S200, and a biometric information acquisition step S500 for acquiring biometric information of the person being watched 1 at the measured location 2 estimated in the measured location estimation step S300.

[0056] According to the monitoring system 101, the information processing device 201, the program 301, and the monitoring method of the first embodiment of the present disclosure, the position of the head of the person being watched 1 is estimated, and the position of the measured part 2 is estimated based on the information of the estimated head position, so that the position of the measured part 2 can be estimated regardless of the state of the person being watched 1. Specifically, when the optical sensor 10 cannot directly identify the measured part 2, for example, even when the person being watched 1 is covered with a futon, the millimeter wave sensor 40 detects the displacement of the irradiation position 41 that is likely to be the measured part 2, so that the position of the measured part 2 can be estimated. Furthermore, by acquiring the displacement of the biological surface at the estimated position of the measured part 2 by the millimeter wave sensor 40, biological information can be acquired with high accuracy regardless of the state of the person being watched 1.

[0057] Variation 1. Variation 1 of embodiment 1 will be described. In embodiment 1, a monitoring system 101 that estimates the position of the measured location 2 using only information on the position of the head 3 and information on the first distance l has been described. In variation 1, a monitoring system 101a that estimates the position of the measured location 2 using information on the direction of the measured location 2 relative to the head 3 in addition to information on the position of the head 3 and information on the first distance l will be described.

[0058] The monitoring system 101a according to the first modification of the first embodiment differs from the monitoring system 101 according to the first embodiment in that information on the direction of the measured part 2 relative to the head 3 is used to estimate the measured part 2. The information on the direction of the measured part 2 is information indicating in which direction the measured part 2 is located as viewed from the head 3. For example, when the measured part 2 is the chest, the information on the direction of the measured part 2 indicates the direction from the head 3 to the torso 4.

[0059] The configuration of a watching system 101a according to the first modification of the first embodiment will be described with reference to Fig. 8. Fig. 8 is a system block diagram of the watching system 101a. 8, the monitoring system 101a includes an optical sensor 10 and a millimeter wave sensor 40 similar to those of the monitoring system 101 according to the first embodiment, and an information processing device 201a. The information processing device 201a includes a processing unit 20a and a storage unit 30a. The processing unit 20a includes a measured part estimation unit 23a that estimates the measured part 2 using information on the direction of the measured part 2 relative to the head 3, and other configurations are similar to those of the processing unit 20 according to the first embodiment. The memory unit 30a includes a program 301a for causing a computer to function as an information processing device 201a relating to variant example 1 of embodiment 1 and for executing a monitoring method relating to variant example 1 of embodiment 1, and a database 31 similar to that of embodiment 1.

[0060] The measured location estimation unit 23a estimates the direction of the measured location 2 using information on the shape of the person being watched over 1 acquired by the optical sensor 10, and estimates the position of the measured location 2 using the information on the estimated direction of the measured location 2. The procedure for estimating the measured location 2 by the measured location estimation unit 23a will be described with reference to FIG. 9. FIG. 9 is a flowchart showing the processing procedure of the measured location estimation step S300 by the watching system 101a. The watching method according to the first modification of the first embodiment is similar to the watching method according to the first embodiment shown in FIG. 5, except for the processing procedure in the measured location estimation step S300.

[0061] As shown in FIG. 9, in the measurement location estimation step S300 by the monitoring system 101a, the measurement location estimation unit 23a first determines whether the shape information of the person being watched 1 acquired by the optical sensor 10 includes not only the head 3 but also the contour information of the torso 4 or the facial feature information (step S301). If it is determined that the optical sensor 10 has acquired the contour information of the torso 4 or the facial feature information (Yes in step S301), the direction of the measurement location 2 is calculated based on the contour information of the head 3 and torso 4 acquired by the optical sensor 10 or the facial feature information (step S302). When calculating the direction of the measurement location 2 based on the facial feature information, the measurement location estimation unit 23a calculates the direction of the measurement location 2 based on the facial feature information acquired by the optical sensor 10, that is, the position information of the facial feature points. For example, the measured part estimation unit 23a extracts the positions of the eyes, nose, and mouth, and calculates the direction of the measured part 2 relative to the head 3 from the extracted positions of the eyes and mouth. Then, the position of the measured part 2 is estimated using information on the position of the head 3 estimated in the head position estimation step S200 in FIG. 5 and information on the direction of the measured part 2 calculated in step S302 (step S351). The information on the position of the measured part 2 estimated in step S351 is transmitted to the millimeter wave sensor control unit 24, and the displacement of the measured part 2 estimated by the millimeter wave sensor control unit 24 is detected (step S400 in FIG. 5). On the other hand, if the optical sensor 10 cannot obtain information on the outline of the torso part 4 or information on the feature amount of the face (No in step S301), the same process as in the first embodiment may be performed. That is, as in step S310 in FIG. 6, a position away from the position of the head 3 by the first distance 1 is set as the irradiation position 41, and the irradiation position 41 is irradiated with millimeter waves to estimate the position of the measured part 2. The processing procedure after the irradiation position 41 is set is the same as that of the watching system 101 according to the first embodiment (see FIG. 6).

[0062] According to the monitoring system 101a according to the first modification of the first embodiment, the direction of the measured part 2 is estimated using the shape information of the person being watched 1 acquired by the optical sensor 10, and the information on the direction of the measured part 2 can be used to estimate the measured part 2, so that the accuracy of estimating the measured part 2 can be improved. Specifically, as described above, the position of the measured part 2 can be specified to one location by using the information on the direction of the measured part 2 to estimate the measured part 2, and the displacement of the biological surface at the specified position of the measured part 2 is acquired by the millimeter wave sensor 40, so that biological information can be acquired with high accuracy regardless of the state of the person being watched 1. In addition, by using the information on the direction of the measured part 2 to estimate the measured part 2, it is not necessary to repeat the process of irradiating the irradiation position 41 with millimeter waves until the millimeter wave sensor 40 detects the displacement as shown in FIG. 6, so that the processing procedure of the measured part estimation step S300 can be simplified.

[0063] Variation 2. Variation 2 of the first embodiment will be described. Variation 1 describes a monitoring system 101a that estimates the position of the measured location 2 using information on the direction of the measured location 2. Variation 2 describes a monitoring system 101b that estimates the position of the measured location 2 using information on the first distance l corrected using information on the second distance L.

[0064] The monitoring system 101b according to the second modification of the first embodiment differs from the monitoring system 101 according to the first embodiment in that the monitoring system 101b uses information on the first distance l corrected using information on the second distance L to estimate the measured location 2. Here, the information on the second distance L is information indicating the distance between the person being watched over 1 and the optical sensor 10.

[0065] The information of the first distance l used by the monitoring system 101 of the first embodiment is the actual distance between the head 3 of the person being watched over 1 and the measured location 2. In contrast, the distance between the head 3 of the person being watched over 1 and the measured location 2 acquired by the optical sensor 10 is not the actual size, but an apparent distance that is smaller by the distance between the person being watched over 1 and the optical sensor 10. The apparent distance of the person being watched over 1 acquired by the optical sensor 10 changes depending on the second distance L indicating the distance between the person being watched over 1 and the optical sensor 10. Therefore, the monitoring system 101b according to the second modification corrects the information of the first distance l using the information of the second distance L, and estimates the position of the measured location 2 using the corrected information of the first distance l, thereby improving the estimation accuracy of the measured location 2.

[0066] The configuration of a watching system 101b according to the second modification of the first embodiment will be described with reference to Fig. 10. Fig. 10 is a system block diagram of the watching system 101b. 10, the monitoring system 101b includes an optical sensor 10 and a millimeter wave sensor 40 similar to those of the monitoring system 101 according to the first embodiment, and an information processing device 201b. The information processing device 201b includes a processing unit 20b and a storage unit 30b. The processing unit 20b includes a measured location estimation unit 23b that corrects information on the first distance l using information on the second distance L and estimates the position of the measured location 2 using the corrected information on the first distance l. The other configurations are the same as those of the processing unit 20 according to the first embodiment. The memory unit 30b includes a program 301b for causing a computer to function as an information processing device 201b relating to variant example 2 of embodiment 1 and for executing a monitoring method relating to variant example 2 of embodiment 1, and a database 31 similar to that of embodiment 1.

[0067] The procedure for estimating the measured points 2 by the measured point estimation unit 23b will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the procedure for the measured point estimation step S300 by the monitoring system 101b. Note that the measured point estimation step S300 in the second modification of the first embodiment is similar to the procedure for the measured point estimation step S300 by the monitoring system 101 according to the first embodiment shown in Fig. 6, except for the process of step S310.

[0068] As shown in FIG. 11, in the measured location estimation step S300 by the monitoring system 101b, the measured location estimation unit 23b first determines whether the optical sensor 10 can acquire information on the second distance L (step S303). If the millimeter wave sensor 40 is installed near the optical sensor 10, the millimeter wave sensor 40 may be used for measurement. Next, when it is determined that the information on the second distance L is acquired by the optical sensor 10 or the millimeter wave sensor 40 (Yes in step S303), the measured location estimation unit 23b corrects the first distance l using the measured second distance L (step S304). Then, a position that is away from the position of the head 3 estimated in the head position estimation step S200 in FIG. 5 by the corrected first distance l is set as the irradiation position 41 (step S311). If it is determined that information on the second distance L has not been acquired by the optical sensor 10 or the millimeter wave sensor 40 (No in step S303), a position that is the first distance l away from the position of the head 3 is set as the irradiation position 41 (step S310), similarly to the first embodiment. The processing procedure after setting the irradiation position 41 is similar to that of the monitoring system 101 according to the first embodiment (see FIG. 6).

[0069] According to the monitoring system 101b relating to variant example 2 of embodiment 1, the first distance l can be corrected using the second distance L acquired by the optical sensor 10 or the millimeter wave sensor 40, and information on the corrected first distance l can be used, thereby improving the estimation accuracy of the position of the measured location 2.

[0070] Variation 3. Modification 3 of the first embodiment will be described. Modification 2 has been described as to a monitoring system 101b that estimates the measured location 2 using the first distance l corrected using the second distance L in order to improve the estimation accuracy of the location of the measured location 2. Modification 3 will be described as to a monitoring system 101c that estimates the measured location 2 using the first distance l corrected using information on the position of furniture stored in a storage unit.

[0071] First, the configuration of a watching system 101c according to Modification 3 of Embodiment 1 will be described with reference to Fig. 12. Fig. 12 is a system block diagram of the watching system 101c. 12, the watching system 101c includes an optical sensor 10c capable of acquiring not only information on the shape of the person being watched 1 but also information on the shape of furniture arranged in the room R of the person being watched 1, a millimeter wave sensor 40 similar to that of the watching system 101 according to the first embodiment, and an information processing device 201c. The information processing device 201c includes a processing unit 20c and a storage unit 30c.

[0072] The storage unit 30c includes a program 301c for making the computer function as the information processing device 201c according to the third modification of the first embodiment and execute the monitoring method according to the third modification of the first embodiment, and a database 31c. In addition to the information stored in the database 31 of the monitoring system 101, the database 31c stores furniture shape information indicating the shape of furniture (not shown) placed in the room R of the person being watched 1, and furniture position information linked to the furniture shape information. Note that there is no particular restriction on the type of furniture, but furniture with a fixed installation position, such as a bed or a desk, is preferable because it reduces the number of times that the furniture position information can be updated.

[0073] The processing unit 20c includes a measured location estimation unit 23c that corrects the first distance l using the information on the position of the furniture stored in the storage unit 30c and estimates the measured location 2 using the corrected first distance l, and other configurations are the same as those of the processing unit 20 according to the first embodiment. In detail, when the person being watched over 1 is in the vicinity of the furniture, the measured location estimation unit 23c compares the information on the shape of the furniture acquired by the optical sensor 10c with the information on the shape of the furniture stored in the storage unit 30c, and acquires the information on the position of the furniture linked to the information on the shape of the furniture from the storage unit 30c. Then, the measured location estimation unit 23c corrects the information on the first distance l using the information on the position of the furniture acquired from the storage unit 30c, and estimates the position of the measured location 2 using the information on the corrected first distance l.

[0074] The procedure for estimating the measured points 2 by the measured point estimation unit 23c will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the procedure for the measured point estimation step S300 by the monitoring system 101c. Note that the measured point estimation step S300 in the third modification of the first embodiment is similar to the procedure for the measured point estimation step S300 by the monitoring system 101 according to the first embodiment shown in Fig. 6, except for the process of step S310.

[0075] 13, in the measured location estimation step S300 by the watching system 101c, first, the measured location estimation unit 23c judges whether or not the optical sensor 10c acquires information on the shape of the furniture near the watching target 1 (step S305). In detail, based on the information on the shape of the watching target 1 and the furniture acquired by the optical sensor 10c, the head position estimation unit 22 or the measured location estimation unit 23c estimates the position of the watching target 1 and the position of the furniture, and judges whether or not the watching target 1 is present near the position of the furniture.

[0076] If the person being watched over 1 is near the furniture (Yes in step S305), the information on the shape of the furniture acquired by the optical sensor 10c is compared with the information on the shape of the furniture stored in the database 31c of the storage unit 30c to identify the furniture (step S306). When the furniture is identified, the measured part estimation unit 23c reads and acquires the information on the position of the furniture linked to the information on the shape of the identified furniture from the database 31c of the storage unit 30c (step S307). Here, when step S307 is executed, since it is determined in step S305 that the person being watched over 1 is near the furniture, the position of the furniture acquired in step S307 can be approximated to the position of the person being watched over 1 near the furniture. That is, the information on the position of the furniture acquired in step S307 can be used as approximately the same value as the information on the second distance L indicating the distance between the person being watched over 1 and the optical sensor 10c. Therefore, by using the information on the furniture position acquired in step S307 instead of the information on the second distance L, the information on the first distance l can be corrected (step S308). Then, a position away from the information on the position of the head 3 estimated in the head position estimation step S200 in Fig. 5 by the first distance l corrected in step S308 is set as the irradiation position 41 (step S311).

[0077] If the person being watched over 1 is not near any furniture (No in step S305), a position that is a first distance 1 away from the position of the head 3 is set as the irradiation position 41 (step S310), similar to the first embodiment. The processing procedure after setting the irradiation position 41 is the same as that of the watching system 101 according to the first embodiment (see FIG. 6).

[0078] According to the monitoring system 101c of the third modification of the first embodiment, by using information on the position of the furniture stored in the memory unit 30c, it is possible to correct the first distance l without measuring the second distance L by the optical sensor 10c or the millimeter wave sensor 40. That is, according to the monitoring system 101c of the third modification of the first embodiment, even when it is not possible to obtain information on the second distance L by the optical sensor 10c or the millimeter wave sensor 40, it is possible to correct the information on the first distance l, thereby improving the estimation accuracy of the position of the measured location 2.

[0079] Variation 4. Variation 4 of the first embodiment will be described. Variation 1 has been described regarding the monitoring system 101a that calculates the measured location 2 using information on the direction of the measured location 2 in order to improve the estimation accuracy of the location of the measured location 2. Variation 4 will be described regarding the monitoring system 101d that estimates the location of the measured location 2 using information on the direction of the measured location 2 stored in a storage unit.

[0080] First, the configuration of a watching system 101d according to Variation 4 of Embodiment 1 will be described with reference to Fig. 14. Fig. 14 is a system block diagram of the watching system 101d. 14, the watching system 101d includes an optical sensor 10c capable of acquiring not only information on the shape of the person being watched over 1 but also information on the shape of furniture arranged in the room R of the person being watched over 1, similar to the watching system 101c according to Modification 3, a millimeter wave sensor 40 similar to the watching system 101 according to Embodiment 1, and an information processing device 201d. The information processing device 201d includes a processing unit 20d and a storage unit 30d.

[0081] The storage unit 30d includes a program 301d for causing a computer to function as the information processing device 201d according to the fourth modification of the first embodiment and to execute the monitoring method according to the fourth modification of the first embodiment, and a database 31d. In addition to the information stored in the database 31 of the monitoring system 101, the database 31d stores information on the shape of the bed placed in the room R of the person being watched over 1, and information on the direction of the measured part of the person being watched over when the person being watched over is on a bed associated with the information on the shape of the bed.

[0082] The processing unit 20d includes a measured location estimation unit 23d that estimates the measured location 2 using information on the direction of the measured location 2 stored in the memory unit 30d, and other configurations are the same as those of the processing unit 20 according to the first embodiment. In detail, when the person being watched over 1 is on the bed, the measured location estimation unit 23d compares the information on the shape of the bed acquired by the optical sensor 10c with the information on the shape of the bed stored in the memory unit 30d, and acquires information on the direction of the measured location 2 linked to the information on the shape of the bed from the memory unit 30d. Then, the measured location estimation unit 23d estimates the position of the measured location 2 using the information on the direction of the measured location 2 acquired from the memory unit 30d.

[0083] The procedure for estimating the measured location 2 by the measured location estimation unit 23d will be described with reference to Fig. 15. Fig. 15 is a flowchart showing the processing procedure of the measured location estimation step S300 by the monitoring system 101d. Note that the monitoring method according to the fourth modification of the first embodiment is similar to the monitoring method according to the first embodiment shown in Fig. 5, except for the processing procedure in the measured location estimation step S300.

[0084] 15, in the measured location estimation step S300 by the watching system 101d, first, similarly to the watching system 101c according to the modification 3, the measured location estimation unit 23d judges whether or not information on the shape of furniture in the vicinity of the watching target 1 is acquired by the optical sensor 10 (step S305). In detail, the head position estimation unit 22 or the measured location estimation unit 23d estimates the position of the watching target 1 and the position of the furniture based on the information on the shape of the watching target 1 and the furniture acquired by the optical sensor 10. Based on the estimated position of the watching target 1 and the position of the furniture, it is judged whether or not furniture is present in the vicinity of the watching target 1.

[0085] If the person being watched over 1 is near the furniture (Yes in step S305), the information on the shape of the furniture acquired by the optical sensor 10 is compared with the information on the shape of the bed stored in the database 31d of the storage unit 30d, and the optical sensor 10c identifies whether the furniture acquired is a bed (step S306a). If the furniture acquired by the optical sensor 10 is identified as a bed (Yes in step S306a), the measured location estimation unit 23d determines whether the person being watched over 1 is on the identified bed (S306b). In detail, the measured location estimation unit 23d determines whether the person being watched over 1 is on the bed based on the estimated positions of the person being watched over 1 and the bed, similar to step S305. If the person being watched over 1 is on the bed (Yes in step S306b), the measured part estimation unit 23d reads and acquires information on the direction of the measured part 2 of the person being watched over 1 when the person being watched over 1 is on the bed associated with the identified bed shape information from the database 31d in the storage unit 30d (step S309). Then, the position of the measured part 2 is estimated using the information on the position of the head 3 estimated in the head position estimation step S200 in Fig. 5 and the information on the direction of the measured part 2 calculated in step S309 (step S351).

[0086] If the person being watched over 1 is not near any furniture (No in step S305), if the furniture detected by the optical sensor 10c is identified as not being a bed (No in step S306a), or if the person being watched over 1 is not on the bed (No in step S306b), the process may be the same as that of the first embodiment. That is, as in step S310 in Fig. 6, a position at a first distance 1 from the position of the head 3 is set as the irradiation position 41, and the position of the measured part 2 is estimated by irradiating the irradiation position 41 with millimeter waves. The process procedure after the irradiation position 41 is set is the same as that of the watching system 101 according to the first embodiment (see Fig. 6).

[0087] According to the monitoring system 101d of the fourth modification of the first embodiment, by using the information on the direction of the measured part 2 stored in the storage unit 30d, it is possible to obtain the direction of the measured part 2 without calculating the direction of the measured part 2 from the information obtained by the optical sensor 10c. That is, according to the monitoring system 101d of the fourth modification, even when the optical sensor 10 cannot obtain information on the shape of the torso or the feature amount of the face, it is possible to obtain information on the direction of the measured part 2, and therefore it is possible to improve the estimation accuracy of the measured part 2.

[0088] Note that Modification 1, Modification 2, Modification 3, or Modification 4 may be implemented in appropriate combination. For example, when the monitoring system 101a of Modification 1 and the monitoring system 101d of Modification 4 are implemented in combination, two pieces of information on the direction of the measured location 2 can be obtained by two different measured location estimation steps S300. The reliability of the information on the direction of the measured location 2 can be determined depending on whether the two pieces of information on the direction of the measured location 2 match.

[0089] Furthermore, by implementing a combination of the monitoring system 101a of Modification 1 and the monitoring system 101d of Modification 4, it is possible to simplify the estimation of the position of the measured location 2. Here, a case where the monitoring system 101a of Modification 1 and the monitoring system 101d of Modification 4 are implemented in combination and information on the directions of two non-coincident measured locations 2 is obtained will be described with reference to Fig. 16. Fig. 16 is a schematic diagram showing the millimeter wave irradiation position 41 set in the measured location estimation step S310 when information on the directions of two non-coincident measured locations 2 is obtained.

[0090] As shown in FIG. 16, when information on the directions of two non-matching measured locations 2 is acquired, the number of millimeter wave irradiation positions 41 set in step S310 is two. Compared to when the irradiation positions 41 are set without using information on the directions of the measured locations 2 (see FIG. 7), the number of irradiation positions 41 is smaller when information on the directions of the measured locations 2 is used (see FIG. 16). That is, by implementing the monitoring system 101a of the first modification and the monitoring system 101d of the fourth modification in combination, the number of irradiation positions 41 can be narrowed down, and the number of times that millimeter waves are irradiated for estimating the measured locations 2 can be reduced. Furthermore, by implementing the monitoring system 101a of the first modification in combination with the monitoring system 101d of the fourth modification, the estimation accuracy of the measured locations 2 can be improved compared to estimating the measured locations 2 without acquiring information on the directions of the measured locations 2.

[0091] In the above, the optical sensor 10 has been described as acquiring shape information of the person being watched over 1 as the information of the person being watched over 1. The optical sensor 10 may also acquire skeletal information or movement information as the information of the person being watched over 1. In this case, the optical sensor 10 acquires skeletal information or movement information of the person being watched over 1 in the person information acquisition step S100, and estimates the position of the head 3 of the person being watched over 1 based on the skeletal information or movement information of the person being watched over 1 in the head position estimation step S200. The estimation of the position of the head 3 using the skeletal information or movement information of the person being watched over 1 can be realized in the same way as when the shape information of the person being watched over 1 is used, or by applying other known techniques. In addition, even when skeletal information or movement information is used as the information of the person being watched over 1, the same action and effect as when the shape information of the person being watched over 1 is used can be obtained.

[0092] Embodiment 2 In the first embodiment of the present disclosure, a monitoring system 101 has been described in which an optical sensor 10 acquires information on the shape of the person being watched over 1 as information on the person being watched over 1 in order to estimate the position of the head 3 of the person being watched over 1. In the second embodiment, a monitoring system 102 will be described in which an infrared sensor 11, which is a type of optical sensor 10, acquires information on the thermal energy radiated by the person being watched over 1 as information on the person being watched over 1. In the second embodiment, the same components as those in the first embodiment of the present disclosure are designated by the same reference numerals, and descriptions of the same or corresponding parts are omitted. Hereinafter, a monitoring system 102 according to the second embodiment will be described with reference to the drawings. The second embodiment of the present disclosure relates to the monitoring system 102, an information processing device 202 that executes processing related to the monitoring system 102, a program 302 that causes a computer to execute processing related to the monitoring system 102, and a monitoring method using the monitoring system 102.

[0093] <Configuration of the Second Embodiment> The configuration of the watching system 102 according to the second embodiment of the present disclosure will be described with reference to Fig. 17. Fig. 17 is a system block diagram of the watching system 102.

[0094] As shown in FIG. 17, the monitoring system 102 according to the second embodiment includes an infrared sensor 11 which is a type of optical sensor 10, an information processing device 202 corresponding to the control of the infrared sensor 11, and a millimeter wave sensor 40 similar to that of the first embodiment.

[0095] The infrared sensor 11 is a type of optical sensor 10 that acquires information on the person being watched over 1 in order to estimate the position of the head 3 of the person being watched over 1, as in the first embodiment. The infrared sensor 11 is a sensor that can detect infrared rays in the mid-infrared region of 3-5 μm or the far-infrared region of 8-14 μm, and acquires information on the thermal energy radiated by the person being watched over 1 as information on the person being watched over 1. The infrared sensor 11 acquires, for example, information on the thermal energy radiated by the person being watched over 1 as a thermal image. In general, when a thermal image of a human body is acquired using the infrared sensor 11, the head 3 shows the maximum luminance value. Therefore, the information processing device 202 described later can estimate the location showing the maximum luminance value in the acquired image as the head 3 by using the infrared sensor 11. Since the infrared sensor 11 can acquire the thermal energy radiated by the person being watched over 1, the position of the head 3 of the person being watched over 1 can be estimated regardless of day or night.

[0096] The infrared sensor 11 can also obtain shape information of the person being watched over 1 using information on the thermal energy radiated by the person being watched over 1. The infrared sensor 11 can obtain shape information of the person being watched over 1 because, when the infrared sensor 11 captures the person being watched over 1, whose temperature is higher than the surrounding area, the infrared sensor 11 can obtain a thermal image showing a temperature distribution that matches the shape of the human body. Furthermore, information on the movements of the person being watched over 1 can be obtained by recording the changes over time in the thermal image obtained by the infrared sensor 11. The information processing device 202, which will be described later, may estimate the position of the head 3 using information on the shape or movements of the person being watched over 1 obtained by the infrared sensor 11.

[0097] In order to improve the accuracy of estimating the position of the head 3, the watching system 102 may further include an optical sensor other than the infrared sensor 11 as a sensor for acquiring information on the person being watched over 1. The other optical sensor may be, for example, a visible camera, an infrared camera, a stereo camera, a LiDAR, or the like.

[0098] The information processing device 202 includes a processing unit 220 and a storage unit 230. The processing unit 220 includes an optical sensor control unit 221 that controls the infrared sensor 11, and a head position estimation unit 222 that estimates the position of the head 3 based on information on the thermal energy radiated by the watching target 1 acquired by the infrared sensor 11, and other configurations are the same as those of the information processing device 201 of the first embodiment. The storage unit 230 includes a program 302 for causing a computer to function as the information processing device 202 according to the second embodiment and to execute the watching method according to the second embodiment, and a database 31 similar to that of the first embodiment.

[0099] <Monitoring method by the monitoring system according to the second embodiment> Next, a procedure for estimating the position of the measured part 2 of the person being watched over 1 and acquiring biological information of the person being watched over 1, which are performed by the watching system 102 according to the second embodiment, will be described with reference to Fig. 18. Fig. 18 is a flowchart showing the procedure of the watching method by the watching system 102 according to the second embodiment.

[0100] 18, the monitoring method by the monitoring system 102 includes a subject information acquiring step S100a, a head position estimating step S200a, a measured part estimating step S300, a displacement detecting step S400, and a biological information acquiring step S500. The measured part estimating step S300, the displacement detecting step S400, and the biological information acquiring step S500 are the same as those in the first embodiment.

[0101] The target person information acquisition step S100a is a step in which the infrared sensor 11 acquires information on the thermal energy radiated by the watching target person 1. In detail, first, the optical sensor control unit 221 of the information processing device 202 controls the infrared sensor 11 to acquire information on the thermal energy radiated by the watching target person 1. Next, the infrared sensor 11 controlled by the optical sensor control unit 221 acquires information on the thermal energy radiated by the watching target person 1. Then, the input device 213, which is an input interface of the information processing device 202, receives the information on the thermal energy radiated by the watching target person 1 acquired by the infrared sensor 11.

[0102] The head position estimation step S200a is a step of estimating the position of the head 3 of the watching target 1 based on the information of the thermal energy radiated by the watching target 1 acquired in the target information acquisition step S100a. The head position estimation step S200a is executed by the head position estimation unit 222 of the information processing device 202. The head position estimation unit 222 estimates the location showing the maximum brightness value as the position of the head 3 from the acquired thermal energy information.

[0103] In addition, in the head position estimation step S200a, a region of interest may be set to estimate the position of the head 3 of the person being watched over 1. The region of interest can be set from information on the shape or movement of the person being watched over 1. For example, information on the shape or movement of the person being watched over 1 can be acquired from a thermal image acquired by the infrared sensor 11, and the region of interest can be set. Specifically, a moving object in the thermal image acquired by the infrared sensor 11 that has higher thermal energy than the surroundings is assumed to be the person being watched over 1, and the region of interest is set. In addition, if the monitoring system 102 includes an optical sensor such as a visible camera in addition to the infrared sensor 11, a bed or the like may be identified by the optical sensor such as a visible camera, and the periphery of the bed may be set as the region of interest. In the head position estimation step S200a, by setting a region of interest and estimating the position of the head 3, the accuracy of estimating the position of the head 3 can be further improved.

[0104] The procedure of the monitoring method after the position of the head 3 is estimated in the head position estimating step S200a is the same as in the first embodiment.

[0105] <Effects of the Second Embodiment> Next, the operation and effects of the watching system 102 according to the second embodiment of the present disclosure will be described.

[0106] A monitoring system 102 according to embodiment 2 of the present disclosure includes an infrared sensor 11 that acquires information on thermal energy radiated by the person being watched 1, a head position estimation unit 222 that estimates the position of the head 3 of the person being watched 1 based on the information on the thermal energy radiated by the person being watched 1 acquired by the infrared sensor 11, a measured location estimation unit 23 that estimates the position of the measured location 2 based on the information on the position of the head 3 estimated by the head position estimation unit 222, a millimeter wave sensor 40 that detects displacement at the measured location 2 estimated by the measured location estimation unit 23, and a biometric information acquisition unit 25 that acquires biometric information of the person being watched 1 based on the displacement detected by the millimeter wave sensor 40.

[0107] An information processing device 202 according to embodiment 2 of the present disclosure includes a head position estimation unit 222 that estimates the position of the head 3 of the person being watched over 1 based on information of thermal energy radiated by the person being watched over 1 acquired by the infrared sensor 11, a measured location estimation unit 23 that estimates the position of the measured location 2 based on the information of the position of the head 3 estimated by the head position estimation unit 222, and a biometric information acquisition unit 25 that, when the millimeter wave sensor 40 detects a displacement at the measured location 2 estimated by the measured location estimation unit 23, acquires biometric information of the person being watched over 1 based on the detected displacement.

[0108] The program 302 according to the second embodiment of the present disclosure enables a computer to realize a head position estimation function that estimates the position of the head 3 of the person being watched over 1 based on information on the thermal energy radiated by the person being watched over 1 acquired by the infrared sensor 11, a measured location estimation function that estimates the position of the measured location 2 based on information on the position of the head 3 estimated by the head position estimation function, and a biometric information acquisition function that, when the millimeter wave sensor 40 detects a displacement at the measured location 2 estimated by the measured location estimation function, acquires biometric information of the person being watched over 1 based on the detected displacement.

[0109] The monitoring method according to embodiment 2 of the present disclosure includes a subject information acquisition step S100a in which an infrared sensor 11 acquires information on thermal energy radiated by the person being watched 1, a head position estimation step S200a in which a position of the head 3 of the person being watched 1 is estimated based on the information on the thermal energy radiated by the person being watched 1 acquired in the subject information acquisition step S100a, a measured location estimation step S300 in which a position of a measured location 2 is estimated based on the information on the position of the head 3 estimated in the head position estimation step S200a, a displacement detection step S400 in which a millimeter wave sensor 40 detects a displacement at the measured location 2 estimated in the measured location estimation step S300, and a biometric information acquisition step S500 in which biometric information of the person being watched 1 is acquired based on the displacement detected in the displacement detection step S400.

[0110] According to the monitoring system 102, information processing device 202, program 302, and monitoring method of the second embodiment of the present disclosure, the position of the head 3 of the person being watched over 1 is estimated based on information on the thermal energy radiated by the person being watched over 1 acquired by the infrared sensor 11, and therefore the position of the head 3 can be estimated with high accuracy even in a state without a light source. Therefore, according to the monitoring system 102, information processing device 202, program 302, and monitoring method of the second embodiment of the present disclosure, the position of the head 3 can be estimated regardless of day or night, and further, by using information on the estimated position of the head 3 and information on the preset first distance l, the position of the measured point 2 can be estimated regardless of day or night.

[0111] Moreover, according to the monitoring system 102, information processing device 202, program 302, and monitoring method of the second embodiment of the present disclosure, the location showing the maximum brightness value can be estimated as the position of the head 3 from the information on the thermal energy acquired by the infrared sensor 11. That is, according to the configuration of the second embodiment, the location showing the maximum brightness value can be easily estimated directly as the position of the head 3, and therefore the calculation load of the information processing device 202 can be reduced compared to the first embodiment in which the position of the head 3 is estimated based on information on the shape of the person being watched over 1.

[0112] Variation 1. Variation 1 of embodiment 2 will be described. In embodiment 2, a monitoring system 102 including an infrared sensor 11 as an optical sensor 10 for acquiring information on a person 1 being watched over is described. Variation 1 will be described in which a monitoring system 102a is further provided with an infrared irradiating device for irradiating infrared light to which the infrared sensor 11 is sensitive. In addition, in the first modification of the second embodiment, the same reference numerals are used for the same components as those in the second embodiment of the present disclosure, and the description of the same or corresponding parts is omitted. Hereinafter, a monitoring system 102a according to the first modification of the second embodiment will be described with reference to the drawings. The first modification of the second embodiment of the present disclosure relates to the monitoring system 102a, an information processing device 202a that executes processing related to the monitoring system 102a, a program 302a that causes a computer to execute processing related to the monitoring system 102a, and a monitoring method using the monitoring system 102a.

[0113] The configuration of a watching system 102a according to a first modification of the second embodiment of the present disclosure will be described with reference to Fig. 19. Fig. 19 is a system block diagram of the watching system 102a.

[0114] 19, the watching system 102a according to the first modification of the second embodiment further includes an infrared irradiating device 50 in addition to the configuration of the second embodiment. The watching system 102a also includes an information processing device 202a including a processing unit 220a further including an infrared irradiating device control unit 27 in addition to the configuration of the processing unit 220 of the second embodiment, and a storage unit 230a storing a program 302a for executing the functions of the processing unit 220a and a database 31.

[0115] The infrared irradiation device 50 is a device that irradiates infrared rays to which the infrared sensor 11 has sensitivity. As in the second embodiment, the infrared sensor 11 is a sensor that can detect infrared rays in the mid-infrared region of 3-5 μm or the far-infrared region of 8-14 μm. Therefore, the infrared irradiation device 50 is a device that irradiates infrared rays in the mid-infrared region of 3-5 μm or the far-infrared region of 8-14 μm to which the infrared sensor 11 has sensitivity. The irradiation of the infrared irradiation device 50 is controlled by the infrared irradiation device control unit 27 of the information processing device 202a.

[0116] The monitoring system 102a according to the first modification of the second embodiment further includes an infrared irradiator 50 to obtain the second distance L by the infrared sensor 11. The infrared sensor 11 alone cannot obtain the distance to the target, that is, the second distance L. The infrared sensor 11 can obtain information on the second distance L indicating the distance between the monitoring target 1 and the infrared sensor 11 by detecting infrared light irradiated from the infrared irradiator 50 and reflected by the head 3. The information on the second distance L obtained by the infrared irradiator 50 and the infrared sensor 11 is used to correct the information on the first distance l. The procedure for correcting the information on the first distance l using the information on the second distance L is as shown in the measured location estimation step S300a of the first modification of the first embodiment or the measured location estimation step S300c of the third modification (see FIG. 8 and FIG. 10).

[0117] The operation and effect of the watching system 102a according to the first modification of the second embodiment of the present disclosure will be described.

[0118] According to the monitoring system 102a, the information processing device 202a, the program 302a, and the monitoring method of the first modification of the second embodiment of the present disclosure, since information on the second distance L can be acquired by the infrared irradiating device 50 and the infrared sensor 11, the first distance l can be corrected using the information on the second distance L. By calculating the position of the measured location 2 using the corrected first distance l, the calculation accuracy of the position of the measured location 2 is improved, and the estimation accuracy of the measured location 2 can be improved.

[0119] Embodiment 3 In the second embodiment of the present disclosure, a description has been given of monitoring systems 102 and 102a in which an infrared sensor 11 acquires information on thermal energy radiated by the person being watched over 1 and estimates the position of the head 3 of the person being watched over 1. In the third embodiment, a description will be given of a monitoring system 103 in which an infrared sensor 11 acquires information on thermal energy radiated by the person being watched over 1 and estimates the position of the head 3 of the person being watched over 1 and acquires the temperature of the head 3. In addition, in the third embodiment, the same components as those in the first or second embodiment of the present disclosure are designated by the same reference numerals, and a description of the same or corresponding parts will be omitted. Hereinafter, a monitoring system 103 according to the third embodiment will be described with reference to the drawings. The third embodiment of the present disclosure relates to the monitoring system 103, an information processing device 203 that executes processing related to the monitoring system 103, a program 303 that causes a computer to execute processing related to the monitoring system 103, and a monitoring method using the monitoring system 103.

[0120] <Configuration of the Third Embodiment> The configuration of a watching system 103 according to the third embodiment of the present disclosure will be described with reference to Fig. 20. Fig. 20 is a system block diagram of the watching system 103.

[0121] As shown in FIG. 20, the monitoring system 103 of embodiment 3 includes an infrared sensor 11 and a millimeter wave sensor 40 similar to those of embodiment 2, and an information processing device 203 that estimates the position of the head 3 of the person being monitored 1 based on the thermal energy information acquired by the infrared sensor 11 and acquires the temperature of the head 3.

[0122] The information processing device 203 includes a processing unit 320 and a storage unit 330. The processing unit 320 includes an optical sensor control unit 321 instead of the optical sensor control unit 221 in the processing unit 220 of the second embodiment, and further includes a determination unit 26 and a head temperature acquisition unit 28.

[0123] The determination unit 26 determines whether the biometric information of the person being watched over 1 acquired by the biometric information acquisition unit 25 is normal or abnormal. The determination unit 26 determines that the acquired biometric information, for example, the respiratory rate, pulse rate, blood pressure, etc., is normal when the value is within a predetermined range set in advance, and determines that the biometric information is abnormal when it is outside the predetermined range. The predetermined range is a range that is set in advance as a normal value based on medical knowledge, or a range that is set in advance based on the value of the biometric information of the person being watched over 1 under normal circumstances. The predetermined range is set in advance in the database 31, for example. The determination unit 26 may determine whether the biometric information is normal or abnormal based on the time change of the acquired biometric information without comparing it with a preset value. In detail, when the value of the biometric information of the person being watched over 1 acquired by the biometric information acquisition unit 25 shows an irregular change or when the value fluctuates widely, the determination unit 26 may determine that the biometric information is abnormal.

[0124] In addition to the functions of the optical sensor control unit 221 of the second embodiment, the optical sensor control unit 321 changes the infrared sensor 11 to a high-precision driving method capable of acquiring information on thermal energy with high precision when the determination unit 26 determines that there is an abnormality. The high-precision driving method is, for example, a driving method in which the S / N ratio is improved by increasing frame integration. The high-precision driving method may be a noise reduction driving method in which output variations are suppressed by performing pixel calibration using a shutter whose temperature and emissivity are known immediately before temperature measurement, or a signal strength increase driving method in which temperature sensitivity is improved by increasing the driving voltage. When the determination unit 26 determines that there is an abnormality, the infrared sensor 11 is changed to the high-precision driving method by the control of the optical sensor control unit 321, and acquires information on thermal energy of the head 3 with high precision.

[0125] The head temperature acquisition section 28 acquires the temperature of the head 3 based on information about thermal energy acquired when the infrared sensor 11 is driven by the high-precision driving method.

[0126] The storage unit 330 stores the program 303 and the database 31. The program 303 is a computer program for causing the processing unit 320 to execute the functions.

[0127] <Monitoring method by the monitoring system according to the third embodiment> Next, the procedure of the watching method performed by the watching system 103 according to the third embodiment will be described with reference to Fig. 21. Fig. 21 is a flowchart showing the procedure of the watching method performed by the watching system 103 according to the third embodiment.

[0128] As shown in FIG. 21, the monitoring method by monitoring system 103 includes, in addition to the same steps as in the second embodiment, a determination step S600 and a head temperature acquisition step S700.

[0129] 21, the procedure from subject information acquisition step S100a to biological information acquisition step S500 is the same as in the second embodiment, and first, the position of the measured part 2 of the person being watched over 1 is estimated, and biological information of the person being watched over 1 is acquired. Then, after the biological information of the person being watched over 1 is acquired in biological information acquisition step S500, the process proceeds to determination step S600.

[0130] The judgment step S600 is a step for judging whether the biometric information of the person being watched over 1 acquired in the biometric information acquisition step S500 is normal or abnormal. The judgment step S600 is executed by the judgment unit 26 of the information processing device 204. For example, the judgment unit 26 judges the acquired biometric information of the person being watched over 1 to be normal when the value is within a predetermined range set in advance, and judges the biometric information to be abnormal when the value is outside the predetermined range.

[0131] If the determination is made normal in determination step S600 (Yes in S600), the processing unit 320 ends the processing of the monitoring system 103 (END). If the determination is made abnormal in determination step S600 (No in S600), the head temperature acquisition step S700 is executed. The head temperature acquisition step S700 is a step of acquiring the temperature of the head 3 by the infrared sensor 11 when the determination is made abnormal in determination step S600. The head temperature acquisition step S700 is executed by the optical sensor control unit 321, the infrared sensor 11, and the head temperature acquisition unit 28 of the information processing device 203.

[0132] In head temperature acquisition step S700, first, optical sensor control section 321 changes infrared sensor 11 to a high-precision drive mode capable of acquiring heat energy information with high accuracy. Infrared sensor 11 is changed to the high-precision drive mode under the control of optical sensor control section 321, and acquires heat energy information of head 3 with high accuracy. Then, based on the heat energy information acquired by infrared sensor 11, head temperature acquisition section 28 acquires the temperature of head 3.

[0133] The temperature of the head 3 acquired in the head temperature acquisition step S700 may be displayed on a display device (not shown) together with the biological information acquired in the biological information acquisition step S500. Similarly, the determination result in the determination step S600 may be displayed on a display device (not shown).

[0134] <Effects of the Third Embodiment> The operation and effects of the watching system 103 according to the third embodiment of the present disclosure will be described.

[0135] According to the monitoring system 103, information processing device 203, program 303, and monitoring method of the third embodiment of the present disclosure, biological information can be acquired by the millimeter wave sensor 40 regardless of the state of the person being watched over 1, and further, when an abnormality occurs, the temperature of the head 3 can be acquired by the infrared sensor 11. That is, according to the monitoring system 103, information processing device 203, program 303, and monitoring method of the third embodiment of the present disclosure, when an abnormality occurs, body temperature can be acquired in addition to biological information such as respiratory rate, pulse rate, and blood pressure.

[0136] Moreover, according to the monitoring system 103, the information processing device 203, the program 303, and the monitoring method of the third embodiment of the present disclosure, the infrared sensor 11 is changed to the high-precision driving method only when an abnormality occurs, and therefore power consumption can be reduced more than when the infrared sensor 11 is always driven by the high-precision driving method. That is, according to the monitoring system 103, the information processing device 203, the program 303, and the monitoring method of the third embodiment of the present disclosure, various types of bioinformation can be acquired while reducing power consumption.

[0137] Variation 1. Variation 1 of embodiment 3 will be described. In embodiment 3, a configuration has been described in which the optical sensor control unit 321 changes the infrared sensor 11 to a high-precision drive method when an abnormality occurs, thereby acquiring the temperature of the head 3 of the person being watched over 1. Variation 1 will be described in which an infrared irradiator 50 is further provided, and when an abnormality occurs, the infrared irradiator 50 irradiates infrared light to which the infrared sensor 11 is sensitive, thereby acquiring the temperature of the head 3 of the person being watched over 1. In addition, in the first modification of the third embodiment, the same components as those in the third embodiment of the present disclosure are designated by the same reference numerals, and the description of the same or corresponding parts is omitted. Hereinafter, a monitoring system 103a according to the first modification of the third embodiment will be described with reference to the drawings. The first modification of the third embodiment of the present disclosure relates to the monitoring system 103a, an information processing device 203a that executes processing related to the monitoring system 103a, a program 303a that causes a computer to execute processing related to the monitoring system 103a, and a monitoring method using the monitoring system 103a.

[0138] The configuration of a watching system 103a according to Modification 1 of Embodiment 3 of the present disclosure will be described with reference to Fig. 22. Fig. 22 is a system block diagram of the watching system 103a.

[0139] As shown in Fig. 22, a monitoring system 103a according to Variation 1 of Embodiment 3 includes an infrared sensor 11 and a millimeter wave sensor 40 similar to those of Embodiment 2 or 3, an information processing device 203a, and an infrared irradiator 50. The information processing device 203a includes a processing unit 320a and a storage unit 330a. The processing unit 320a includes a head temperature acquirer 28a instead of the head temperature acquirer 28 in the processing unit 320 of Embodiment 3, and further includes an infrared irradiator control unit 27. The storage unit 330a stores a program 303a and a database 31. The program 303a is a computer program for executing the functions of the processing unit 320a.

[0140] The infrared irradiation device 50 is a device that irradiates infrared rays to which the infrared sensor 11 has sensitivity. As in the first modification of the second embodiment, the infrared irradiation device 50 is a device that irradiates infrared rays in the mid-infrared region of 3-5 μm or the far-infrared region of 8-14 μm to which the infrared sensor 11 has sensitivity. The infrared irradiation device 50 is activated by the infrared irradiation device control unit 27 when the determination unit 26 determines that an abnormality has occurred.

[0141] The infrared sensor 11 detects infrared rays irradiated from the infrared irradiation device 50 and reflected by the head 3. The head temperature acquisition unit 28a calculates the emissivity ε of the head 3 based on the infrared rays detected by the infrared sensor 11, and acquires the temperature of the head 3 by correcting the information on the thermal energy based on the calculated emissivity ε.

[0142] A method for correcting the thermal energy information by the head temperature acquisition unit 28a will be described. Consider a case where infrared rays are irradiated by the infrared irradiation device 50 and the infrared sensor 11 detects the infrared rays reflected by the target. If the luminance value detected by the infrared sensor 11 is P r If the intensity of the light emitted by the infrared irradiation device 50 is P0, the reflectance of the person being watched over 1 is R, and the distance between the person being watched over 1 and the infrared sensor 11 is L, then the luminance value P detected by the infrared sensor 11 is r is expressed by the following formula: TIFF0007637863000001.tif14150 Here, the intensity P0 of the light emitted by the infrared irradiation device 50 is a known value. Also, the distance L between the person being watched over 1 and the infrared sensor 11, i.e., information on the second distance L, can be measured by the infrared sensor 11 or the millimeter wave sensor 40. Therefore, by measuring the second distance L in advance, the luminance value P r From this, the reflectance R can be calculated. Also, according to Kirchhoff's law, the reflectance R and the emissivity ε can be expressed by the following formula: TIFF0007637863000002.tif6150 According to Kirchhoff's law, if the reflectance R is known, the emissivity ε can be calculated.

[0143] Here, the target temperature when the infrared sensor 11 calculates the temperature is T obj , the reference temperature such as the shutter temperature is T s , the compensation temperature due to temperature sensitivity, etc. is T c Then, the target temperature T obj , corrected temperature T c is given by the following formula: TIFF0007637863000003.tif16155 TIFF0007637863000004.tif16153 Reference temperature T S Since is a known value, the target temperature T obj That is, by using the emissivity ε, the accuracy of temperature measurement by the infrared sensor 11 can be improved.

[0144] According to the monitoring system 103a, information processing device 203a, program 303a, and monitoring method of Modification 1 of Embodiment 3 of the present disclosure, biological information can be acquired by the millimeter wave sensor 40 regardless of the state of the person being watched over 1, and further, when an abnormality occurs, the emissivity ε can be calculated by the infrared irradiating device 50 and the infrared sensor 11, and the temperature of the head 3 can be acquired with high accuracy. That is, according to the monitoring system 103a, information processing device 203a, program 303a, and monitoring method of Modification 1 of Embodiment 3 of the present disclosure, when an abnormality occurs, in addition to biological information such as respiratory rate, pulse rate, and blood pressure, the body temperature can also be acquired with high accuracy.

[0145] Moreover, according to the monitoring system 103a, the information processing device 203a, the program 303a, and the monitoring method of the first modification of the third embodiment of the present disclosure, the infrared irradiating device 50 is started only when an abnormality occurs, and therefore power consumption can be reduced more than if the infrared irradiating device 50 is constantly started. That is, according to the monitoring system 103a, the information processing device 203a, the program 303a, and the monitoring method of the first modification of the third embodiment of the present disclosure, various types of biological information can be acquired with high accuracy while suppressing power consumption.

[0146] Variation 2. Next, a second modification of the third embodiment will be described. In the first modification, an infrared irradiator 50 is further provided, and in the event of an abnormality, the infrared irradiator 50 irradiates infrared light to which the infrared sensor 11 is sensitive, thereby acquiring the temperature of the head 3 of the person being watched over 1. In the second modification, a reference temperature device 60 is provided instead of the infrared irradiator 50, and in the event of an abnormality, the infrared sensor 11 detects reference light generated by the reference temperature device 60, thereby acquiring the temperature of the head 3 of the person being watched over 1. In addition, in the second modification of the third embodiment, the same reference numerals are used for the same components as those in the third embodiment of the present disclosure or the first modification thereof, and the description of the same or corresponding parts is omitted. Hereinafter, a monitoring system 103b according to the second modification of the third embodiment will be described with reference to the drawings. The second modification of the third embodiment of the present disclosure relates to the monitoring system 103b, an information processing device 203b that executes processing related to the monitoring system 103b, a program 303b that causes a computer to execute processing related to the monitoring system 103b, and a monitoring method using the monitoring system 103b.

[0147] The configuration of a watching system 103b according to Modification 2 of Embodiment 3 of the present disclosure will be described with reference to Fig. 23. Fig. 23 is a system block diagram of the watching system 103b.

[0148] As shown in Fig. 23, a monitoring system 103b according to Variation 2 of Embodiment 3 includes an infrared sensor 11 and a millimeter wave sensor 40 similar to those of Embodiment 2 or 3, an information processing device 203b, and a reference temperature device 60. The information processing device 203b includes a processing unit 320b and a storage unit 330b. The processing unit 320b includes a head temperature acquisition unit 28b instead of the head temperature acquisition unit 28 in the processing unit 320 of Embodiment 3, and further includes a reference temperature device control unit 29. The storage unit 330b stores a program 303b and a database 31. The program 303b is a computer program for executing the functions of the processing unit 320b.

[0149] The reference temperature device 60 is a device, such as a blackbody furnace, that generates reference light for the infrared sensor 11. The reference temperature device 60 is activated by the reference temperature device control unit 29 when the determination unit 26 determines that an abnormality has occurred.

[0150] The infrared sensor 11 detects infrared rays radiated by the person being watched over 1 and reference light generated by the reference temperature device 60. The head temperature acquisition unit 28b corrects the temperature sensitivity of the infrared sensor 11 based on the reference light detected by the infrared sensor 11, that is, corrects information on the thermal energy radiated by the person being watched over 1, thereby acquiring the temperature of the head 3.

[0151] According to the monitoring system 103b, information processing device 203b, program 303b, and monitoring method of Modification 2 of Embodiment 3 of the present disclosure, biological information can be acquired by the millimeter wave sensor 40 regardless of the state of the person being watched over 1, and further, in the event of an abnormality, the temperature sensitivity of the infrared sensor 11 is corrected by the reference temperature device 60, and the temperature of the head 3 can be acquired with high accuracy. That is, according to the monitoring system 103b, information processing device 203b, program 303b, and monitoring method of Modification 2 of Embodiment 3 of the present disclosure, in addition to biological information such as respiratory rate, pulse rate, and blood pressure, body temperature can also be acquired with high accuracy in the event of an abnormality.

[0152] Furthermore, according to the monitoring system 103b, the information processing device 203b, the program 303b, and the monitoring method of the second modification of the third embodiment of the present disclosure, the reference temperature device 60 is started only when an abnormality occurs, and therefore power consumption can be reduced more than if the reference temperature device 60 were constantly started. That is, according to the monitoring system 103b, the information processing device 203b, the program 303b, and the monitoring method of the second modification of the third embodiment of the present disclosure, it is possible to acquire biological information with high accuracy while reducing power consumption.

[0153] Although the present disclosure has been described based on each embodiment, the present disclosure is not limited to each embodiment. Furthermore, appropriate combinations, modifications, omissions, etc. of each embodiment are also included in the scope of the technical idea of ​​the present disclosure. [Explanation of symbols]

[0154] 1. Person to be monitored 2. Measurement points 3 head 4. Body 10. Optical Sensor 11 Infrared Sensor 20, 220, 320 Processing section 21, 221, 321 Optical sensor control unit 22, 222 Head position estimation part 23 Measurement location estimation section 24 Millimeter wave sensor control unit 25 Biometric information acquisition unit 26 Judgment section 27 Infrared irradiation device control section 28 Head temperature acquisition section 29 Reference temperature device control section 30, 230, 330 storage section 31 Database 40 mmWave Sensor 50 Infrared irradiation device 60 Reference temperature device 101, 102, 103 Monitoring system 201, 202, 203 Information processing device 301, 302, 303 Programs l first distance L second distance

Claims

1. A head position estimation unit that estimates the position of the head of a person being watched over whose measured location is at least partially hidden; a measurement location estimation unit that estimates the position of the measurement location based on information about the head position estimated by the head position estimation unit; a biological information acquisition unit that acquires biological information of the person being watched over at the measured location estimated by the measured location estimation unit; A monitoring system equipped with

2. Further comprising an optical sensor for acquiring information about the person being watched over, the measured location estimation unit estimates the location of the measured location using information on the position of the head, information on a first distance indicating a predetermined distance between the head and a measured location of the person being watched over, and information on a second distance indicating a distance between the person being watched over and the optical sensor; The monitoring system according to claim 1 .

3. Further comprising a millimeter wave sensor that detects the displacement at the measurement location estimated by the measurement location estimation unit, the measured location estimation unit sets a position that is the first distance away from the head position estimated by the head position estimation unit as an irradiation position, and the millimeter wave sensor irradiates the irradiation position with millimeter waves; and when a displacement is detected at the irradiation position, estimates the irradiation position as the measured location. The monitoring system according to claim 1 .

4. the information about the person being watched over is information about the shape of the person being watched over, The measured part estimation unit estimates a direction of the measured part relative to the head using information about the shape of the person being watched over acquired by the optical sensor, and estimates a position of the measured part using information about the estimated direction of the measured part. The monitoring system according to claim 2 .

5. the measured point estimation unit corrects the information on the first distance using the information on the second distance, and estimates the position of the measured point using the corrected information on the first distance. The monitoring system according to claim 2 .

6. a storage unit that stores information about the shape of furniture arranged in the room of the person being watched over and information about the position of the furniture linked to the information about the shape of the furniture; Furthermore, The optical sensor is capable of acquiring information about the shape of the furniture, When the person being watched over is located near the furniture, the measured location estimation unit compares the shape information of the furniture acquired by the optical sensor with the shape information of the furniture stored in the memory unit, acquires position information of the furniture linked to the shape information of the furniture from the memory unit, corrects the information of the first distance using the acquired information of the furniture position, and estimates the position of the measured location using the corrected information of the first distance. The monitoring system according to claim 2 .

7. a storage unit that stores information about the shape of a bed placed in the room of the person being watched over, and information about the direction of the measured location relative to the head of the person being watched over, which information is linked to the information about the shape of the bed; Furthermore, The optical sensor is capable of acquiring information about the shape of the bed, When the person being watched over is on the bed, the measured location estimation unit compares information on the shape of the bed acquired by the optical sensor with information on the shape of the bed stored in the memory unit, acquires information on the direction of the measured location linked to the information on the shape of the bed from the memory unit, and estimates the position of the measured location using the acquired information on the direction of the measured location. The monitoring system according to claim 2 .

8. the optical sensor is an infrared sensor that detects infrared rays, the information about the person being watched over includes information about thermal energy radiated by the person being watched over, the head position estimation unit estimates the position of the head of the person being watched over based on the information on the thermal energy. The information processing device according to claim 2 .

9. further comprising an infrared irradiation device that irradiates infrared light to which the infrared sensor is sensitive; the infrared sensor detects infrared rays irradiated from the infrared irradiating device and reflected by the head, and acquires the second distance based on the detected infrared rays; the measured point estimation unit corrects the information on the first distance using information on a second distance indicating a distance between the person being watched over and the infrared sensor, and estimates the position of the measured point using the corrected information on the first distance. The monitoring system according to claim 8.

10. a determination unit that determines whether the biological information of the person being watched over acquired by the biological information acquisition unit is normal or abnormal; an optical sensor control unit that changes the infrared sensor to a high-precision driving method that can acquire information about the thermal energy with high precision when the determination unit determines that an abnormality has occurred; a head temperature acquisition unit that acquires the temperature of the head based on the information on the thermal energy acquired by the infrared sensor; Furthermore, The monitoring system according to claim 8.

11. an infrared irradiation device that irradiates infrared light to which the infrared sensor is sensitive; a determination unit that determines whether the biological information of the person being watched over acquired by the biological information acquisition unit is normal or abnormal; an infrared irradiation device control unit that activates the infrared irradiation device when the determination unit determines that an abnormality has occurred; a head temperature acquisition unit that calculates the emissivity of the head by detecting infrared rays irradiated from the infrared irradiating device and reflected by the head with the infrared sensor, and acquires the temperature of the head by correcting the thermal energy information based on the calculated emissivity; and Furthermore, The monitoring system according to claim 8.

12. a reference temperature device that generates a reference light for the infrared sensor; a determination unit that determines whether the biological information of the person being watched over acquired by the biological information acquisition unit is normal or abnormal; a reference temperature device control unit that activates the reference temperature device when the determination unit determines that an abnormality has occurred; a head temperature acquisition unit that acquires the temperature of the head based on information about the thermal energy acquired by the infrared sensor detecting the infrared rays radiated by the person being watched over and the reference light generated by the reference temperature device; and Furthermore, The monitoring system according to claim 8.

13. A head position estimation unit that estimates the position of the head of a person being watched over when at least a part of the measurement location is hidden; a measurement location estimation unit that estimates the position of the measurement location based on information about the head position estimated by the head position estimation unit; a biological information acquisition unit that acquires biological information of the person being watched over at the measured location estimated by the measured location estimation unit; An information processing device comprising:

14. On the computer, a head position estimation function for estimating the position of the head of a person being watched over whose measured location is at least partially hidden; a measurement location estimation function that estimates the position of the measurement location based on information about the head position estimated by the head location estimation function; a biological information acquisition function that acquires biological information of the person being watched over at the measured location estimated by the measured location estimation function; A program to achieve this.

15. A head position estimation step of estimating the position of the head of a person being watched over whose measured location is at least partially hidden; a measurement location estimating step of estimating the position of the measurement location based on information about the head position estimated in the head location estimating step; a biological information acquiring step of acquiring biological information of the person being watched over at the measurement location estimated in the measurement location estimating step; A monitoring method that includes:

Images