Monitoring system, information processing device, program, and monitoring method

Abstract

Provided is a monitoring system 101 with which it is possible to highly accurately acquire biological information irrespective of the state of a person being monitored.　A monitoring system 101 according to the present disclosure comprises: a head position estimation unit 22 that estimates the position of the head of a person 1 being monitored; a measurement target site estimation unit 23 that, on the basis of information pertaining to the head position estimated by the head position estimation unit 22, estimates the position of a measurement target site 2 of the person 1 being monitored; and a biological information acquisition unit 25 that acquires biological information of the person 1 being monitored at the measurement target site 2 estimated by the measurement target site estimation unit 23.

Description

Monitoring system, information processing device, program, and monitoring method

[0001] The present disclosure relates to a monitoring system, an information processing device, a program, and a monitoring method.

[0002] In recent years, monitoring systems have been developed that can acquire biometric information of monitored persons without the need for any equipment to be worn by the monitored persons in nursing care facilities / elderly care facilities, hospitals, elderly people living alone, etc. For example, Patent Literature 1 discloses a method in which an infrared sensor is used to identify a displacement site on the surface of a living body that is displaced by the pulsation of the heart or blood vessels, and a millimeter-wave sensor is used to detect the pulsation of the heart or blood vessels at the identified displacement site.

[0003] WO2022 / 196469 publication

[0004] The infrared sensor of Patent Document 1 uses infrared rays to identify the displacement site without contact. However, infrared rays are easily absorbed by clothing, bedding, etc., and therefore have the disadvantage of being unable to detect displacement on the surface of a living body hidden by clothing, bedding, etc. Therefore, the method of Patent Document 1 has the problem that it is unable to identify the displacement site depending on the state of the person being monitored, such as when the person is covered with a bedding, and is therefore unable to acquire biometric information with high accuracy.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a monitoring system that can acquire biometric information with high accuracy regardless of the condition of the person being monitored.

[0006] The monitoring system according to the present disclosure comprises a head position estimation unit that estimates the head position of the person being monitored, a measured location estimation unit that estimates the position of the measured location of the person being monitored based on information about the head position estimated by the head position estimation unit, and a biometric information acquisition unit that acquires biometric information of the person being monitored at the measured location estimated by the measured location estimation unit.

[0007] The information processing device according to the present disclosure includes a head position estimation unit that estimates the head position of the person being watched over, a measured location estimation unit that estimates the position of the measured location of the person being watched over based on the head position information estimated by the head position estimation unit, and a biometric information acquisition unit that acquires biometric information of the person being watched over at the measured location estimated by the measured location estimation unit.

[0008] The program of the present disclosure is a program for enabling a computer to realize a head position estimation function that estimates the head position of the person being monitored, a measured location estimation function that estimates the position of the measured location of the person being monitored based on information about the head position estimated by the head position estimation function, and a biometric information acquisition function that acquires biometric information of the person being monitored at the measured location estimated by the measured location estimation function.

[0009] The monitoring method of the present disclosure comprises a head position estimation step of estimating the head position of the person being monitored, a measured location estimation step of estimating the position of the measured location of the person being monitored based on information about the head position estimated in the head position estimation step, and a biometric information acquisition step of acquiring biometric information of the person being monitored at the measured location estimated in the measured location estimation step.

[0010] According to the monitoring system, information processing device, program, and monitoring method disclosed herein, it is possible to acquire biometric information with high accuracy regardless of the state of the person being watched over.

[0011] 1 is a schematic diagram showing a configuration example of a watching system according to a first embodiment of the present disclosure. FIG. 2 is a system block diagram showing a watching system according to the first embodiment of the present disclosure. FIG. 3 is a schematic diagram showing a first distance. FIG. 4 is a hardware configuration diagram showing a hardware configuration of an information processing device included in the watching system according to the first embodiment of the present disclosure. FIG. 5 is a flowchart showing the procedure of a watching method by the watching system according to the first embodiment of the present disclosure. FIG. 6 is a flowchart showing the processing procedure of a measured location estimation step by the watching system according to the first embodiment of the present disclosure. FIG. 7 is a schematic diagram showing an irradiation position of millimeter waves by the watching system according to the first embodiment of the present disclosure. FIG. 8 is a system block diagram showing a watching system according to a first modification of the first embodiment of the present disclosure. FIG. 9 is a flowchart showing the processing procedure of a measured location estimation step by the watching system according to the first modification of the first embodiment of the present disclosure. FIG. 10 is a system block diagram showing a watching system according to a second modification of the first embodiment of the present disclosure. FIG. 11 is a flowchart showing the processing procedure of a measured location estimation step by the watching system according to the second modification of the first embodiment of the present disclosure. FIG. 12 is a system block diagram showing a watching system according to a third modification of the first embodiment of the present disclosure. FIG. 13 is a flowchart showing the processing procedure of a measured location estimation step by the watching system according to the third modification of the first embodiment of the present disclosure. FIG. 14 is a system block diagram showing a watching system according to a fourth modification of the first embodiment of the present disclosure. 1 is a flowchart showing the processing procedure of a measured location estimation step by a monitoring system according to a fourth modified example of the first embodiment of the present disclosure. FIG. 2 is a schematic diagram showing an irradiation position of millimeter waves when estimating the position of a measured location using directional information of two measured locations. FIG. 3 is a system block diagram showing a monitoring system according to a second embodiment of the present disclosure. FIG. 4 is a flowchart showing the procedure of a monitoring method by a monitoring system according to the second embodiment of the present disclosure. FIG. 5 is a system block diagram showing a monitoring system according to a first modified example of the second embodiment of the present disclosure. FIG. 6 is a system block diagram showing a monitoring system according to a third embodiment of the present disclosure. FIG. 7 is a flowchart showing the procedure of a monitoring method by a monitoring system according to the third embodiment of the present disclosure. FIG. 8 is a system block diagram showing a monitoring system according to a first modified example of the third embodiment of the present disclosure.FIG. 10 is a system block diagram showing a monitoring system according to a second variant of the third embodiment of the present disclosure.

[0012] Hereinafter, examples of a monitoring system, an information processing device, a program, and a monitoring method according to the present disclosure will be described with reference to the accompanying drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals, and descriptions thereof will not be repeated.

[0013] Embodiment 1. Embodiment 1 of the present disclosure relates to a monitoring system 101 that estimates the head position of a person being monitored and estimates the position of a measured location based on information about the estimated head position. Also, embodiment 1 of the present disclosure relates to an information processing device 201 that executes processing related to the monitoring system 101, a program 301 that causes a computer to execute processing related to the monitoring system 101, and a monitoring method using the monitoring system 101.

[0014] <Configuration of First Embodiment> The configuration of a monitoring system 101 according to a first embodiment of the present disclosure will be described with reference to Fig. 1, Fig. 2, and Fig. 3. Fig. 1 is a schematic diagram showing an example configuration of the monitoring system 101, and Fig. 2 is a system block diagram of the monitoring system 101. Fig. 3 is a schematic diagram showing a first distance 1.

[0015] As shown in FIG. 1 , the monitoring system 101 includes an optical sensor 10, an information processing device 201, and a millimeter-wave sensor 40. The monitoring system 101 uses the optical sensor 10 and the millimeter-wave sensor 40 to estimate the position of a measurement location 2 of the person being watched over 1 regardless of the state of the person being watched over 1, and acquires biometric information of the person being watched over 1 with high accuracy. Here, the measurement location 2 is a location where the biological surface is displaced due to breathing, heartbeat, etc. FIG. 1 shows an example in which the chest is set as the measurement location 2. The measurement location 2 is not limited to the chest, and may be any location where the biological surface is displaced due to breathing, heartbeat, etc., such as the neck, temporal region, chest, or back. For simplicity of explanation, an example in which the chest is set as the measurement location 2 will be described below.

[0016] As shown in Fig. 1(a), at least the optical sensor 10 and the millimeter wave sensor 40 of the monitoring system 101 are installed in a room R of the person being watched over 1. Note that, as shown in Fig. 1(b), in addition to the optical sensor 10 and the millimeter wave sensor 40, an information processing device 201 may also be installed in the room R of the person being watched over 1.

[0017] The optical sensor 10 and the millimeter wave sensor 40 transmit and receive various information to and from the information processing device 201. Communication between the optical sensor 10 and the millimeter wave sensor 40 and the information processing device 201 may be performed via a signal line or wireless communication such as Wi-Fi.

[0018] The optical sensor 10 and the millimeter wave sensor 40 can acquire information on the position of an object, and the optical sensor 10 and the millimeter wave sensor 40 are calibrated in advance for alignment so that there is no discrepancy in the information on the position of an object acquired by each of the optical sensor 10 and the millimeter wave sensor 40. Specifically, conversion formulas for the coordinate systems of each of the optical sensor 10 and the millimeter wave sensor 40 are stored in advance in a storage unit 30 described below, and the conversion formulas are used to convert from the coordinates of the optical sensor 10 to the coordinates of the millimeter wave sensor 40, thereby aligning the information acquired by the optical sensor 10 with the information used by the millimeter wave sensor 40.

[0019] The optical sensor 10 is a sensor that acquires information about the person being watched over 1 in order to estimate the position of the head 3 of the person being watched over 1. The optical sensor 10 is, for example, a visible light camera, an infrared camera, a stereo camera, or a LiDAR (Light Detection and Ranging) camera, and acquires information about the person being watched over 1 using visible light or infrared light. The information about the person being watched over 1 acquired by the optical sensor 10 includes, for example, shape information, skeletal information, and movement information about the person being watched over 1. In the first embodiment, a case will be described in which the optical sensor 10 acquires shape information about the person being watched over 1 as the information about the person being watched over 1. The shape information about the person being watched over 1 includes information about the outline of the head 3 or torso 4, information about facial features, and the like. It is assumed that the shape information about the person being watched over 1 includes at least shape information about the head 3 of the person being watched over 1. That is, the optical sensor 10 acquires information about the shape of the watching over target person 1, including the shape of the outline of the head 3 of the watching over target person 1 or information about the facial feature amount.

[0020] The millimeter wave sensor 40 is a sensor that detects displacement at a measurement location 2 of the person being watched over 1 in order to acquire biometric information of the person being watched over 1. Minute displacements of the order of several tens of microns to several millimeters are observed on the surface of a living body due to breathing, heartbeat, etc. The millimeter wave sensor 40 detects minute displacements on the surface of a living body due to breathing, heartbeat, etc., to acquire biometric information such as respiratory rate, pulse rate, blood pressure, etc.

[0021] The millimeter wave sensor 40 uses millimeter waves to detect displacement of the biological surface at the measurement location 2 of the person being watched over 1. In particular, millimeter waves are irradiated onto the measurement location 2 of the person being watched over 1, and reflected waves from the measurement location 2 are received, thereby detecting displacement of the biological surface at the measurement location 2 of the person being watched over 1. Because millimeter waves can pass through clothing, bedding, etc., the millimeter wave sensor 40 can detect displacement of the biological surface hidden by clothing, bedding, etc.

[0022] Furthermore, the millimeter wave sensor 40 is controlled by a millimeter wave sensor control unit 24 (described later) to aim at the measurement location 2 and irradiate the measurement location 2 with millimeter waves. Note that, in order to acquire displacement with high precision using the millimeter wave sensor 40, the drive method of the millimeter wave sensor 40 or the signal processing method of the millimeter wave sensor 40 may be adjusted to improve the S / N ratio. Methods for improving the S / N ratio include, for example, a drive method of the millimeter wave sensor 40 that performs beamforming on the measurement location 2, a drive method of the millimeter wave sensor 40 that focuses the transmission beam scan on the measurement location 2, coherent integration as a signal processing method of the millimeter wave sensor 40, and combinations thereof.

[0023] The information processing device 201 is a computer that performs processing to estimate the position of the measured location 2 of the person being watched over 1 based on information obtained from the optical sensor 10, and processing to acquire biological information of the person being watched over 1 based on information obtained from the millimeter wave sensor 40. As shown in FIG. 2 , the information processing device 201 includes a processing unit 20 and a storage unit 30.

[0024] As shown in FIG. 2 , the processing unit 20 includes an optical sensor control unit 21, a head position estimation unit 22, a measured location estimation unit 23, a millimeter-wave sensor control unit 24, and a biological information acquisition unit 25. The optical sensor control unit 21 controls the optical sensor 10 to acquire shape information of the watching over target person 1. The head position estimation unit 22 estimates the position of the head 3 of the watching over target person 1 based on the shape information of the watching over target person 1 acquired by the optical sensor 10. The measured location estimation unit 23 estimates the position of the measured location 2 using the information on the position of the head 3 estimated by the head position estimation unit 22 and information on a preset first distance l. Here, the information on the first distance l is information indicating the distance between the head 3 and the measured location 2, as shown in FIG. 3 . The information on the first distance l is set in advance in, for example, a database 31 described later. The millimeter-wave sensor control unit 24 controls the millimeter-wave sensor 40 to detect displacement at the position of the measured location 2 estimated by the measured location estimation unit 23. When the millimeter wave sensor 40 detects a displacement at the estimated measurement location 2, the biological information acquisition unit 25 acquires biological information of the person being watched over 1 based on the detected displacement.

[0025] 2 , the storage unit 30 stores a program 301 and a database 31. The information processing device 201 reads out and executes the program 301 stored in the storage unit 30 to estimate the position of the measured location 2 of the person being watched over 1 and to perform processing for acquiring biological information of the person being watched over 1.

[0026] Program 301 is a program for causing a computer to function as information processing device 201 according to embodiment 1 and to execute the monitoring method according to embodiment 1. In detail, program 301 causes the computer to realize a head position estimation function for executing the function of head position estimation unit 22, a measured location estimation function for executing the function of measured location estimation unit 23, and a biological information acquisition function for executing the function of biological information acquisition unit 25.

[0027] The database 31 stores information used when estimating the positions of the head 3 and the measured location 2 of the person being watched over 1 and performing processing to acquire biometric information of the person being watched over 1. In detail, the database 31 stores information on the shape of the head 3 of the person being watched over 1, information on a first distance 1 indicating the distance between the head 3 of the person being watched over 1 and the measured location 2, and a transformation formula for a coordinate system used to calibrate the optical sensor 10 and the millimeter wave sensor 40.

[0028] The watching system 101 may include a display device (not shown). The display device is a device that displays the biometric information of the watching target 1 acquired by the biometric information acquisition unit 25.

[0029] Next, the hardware configuration of the information processing device 201 included in the monitoring system 101 will be described with reference to Fig. 4. Fig. 4 is a diagram showing the hardware configuration of the information processing device 201.

[0030] 4, the information processing device 201 is configured with an arithmetic unit 211, a storage device 212, an input device 213, an auxiliary storage device 214, and an output device 215. The arithmetic unit 211, the storage device 212, the input device 213, the auxiliary storage device 214, and the output device 215 are connected via a signal line 216.

[0031] The arithmetic device 211 is a device that realizes each function of the processing unit 20 shown in FIG. 2 . The arithmetic device 211 realizes each function of the processing unit 20 of the information processing device 201 by reading a necessary program 301 from the auxiliary storage device 214 and executing the process. The arithmetic device 211 is, for example, a processor, and the processor is an IC (Integrated Circuit) that performs arithmetic processing. Specific examples of the processor are, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit). The arithmetic device 211 may also be a personal computer, a microcomputer board, an FPGA (Field Programmable Gate Array) board, or the like.

[0032] The storage device 212 is a main storage device of the information processing device 201. The main storage device temporarily stores calculations of the processes performed by the arithmetic device 211. The storage device 212 is, for example, a RAM (Random Access Memory).

[0033] The input device 213 is an input interface of the information processing device 201. The input device 213 inputs, for example, information about the shape of the person being watched over 1 acquired by the optical sensor 10 or information about the displacement at the measured location 2 of the person being watched over 1 detected by the millimeter wave sensor 40 to the calculation device 211.

[0034] 2 and is an auxiliary storage device of the information processing device 201. The auxiliary storage device 214 stores a program 301 necessary to realize each function of the processing unit 20 of the information processing device 201, and a database 31 that stores information used when estimating the position of the measured location 2 of the person being watched over 1 and performing processing to acquire biological information of the person being watched over 1. The auxiliary storage device 214 is, for example, a read-only memory (ROM), a hard disk drive (HDD), or a solid state drive (SSD).

[0035] The output device 215 is an output interface of the information processing device 201. The output device 215 outputs, for example, the biometric information of the watching target 1 acquired by the biometric information acquisition unit 25 to a display device (not shown).

[0036] A signal line 216 is a transmission path for transmitting and receiving data between the components shown in FIG.

[0037] <Monitoring method by the watching system according to the first embodiment> Next, the procedure for estimating the positions of the head 3 and the measured location 2 of the person being watched over 1 and acquiring biological information of the person being watched over 1, which is performed by the watching system 101 according to the first embodiment, will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the procedure of the watching method by the watching system 101 according to the first embodiment.

[0038] As shown in Figure 5, the monitoring method using the monitoring system 101 includes a subject information acquisition step S100, a head position estimation step S200, a measured location estimation step S300, a displacement detection step S400, and a biometric information acquisition step S500.

[0039] The target person information acquisition step S100 is a step in which the optical sensor 10 acquires shape information of the watching target person 1. In detail, first, the optical sensor control unit 21 of the information processing device 201 controls the optical sensor 10 to acquire shape information of the watching target person 1. Next, the optical sensor 10 controlled by the optical sensor control unit 21 acquires shape information of the watching target person 1. Then, the input device 213, which is an input interface of the information processing device 201, receives the shape information of the watching target person 1 acquired by the optical sensor 10.

[0040] The head position estimation step S200 is a step of estimating the position of the head 3 of the watching target 1 based on the shape information of the watching target 1 acquired in the target information acquisition step S100. The head position estimation step S200 is executed by the head position estimation unit 22 of the information processing device 201. The head position estimation unit 22 estimates the position of the head 3 using the shape information of the watching target 1 acquired by the optical sensor 10. In detail, the head position estimation unit 22 identifies the head 3 and estimates the position of the head 3 by comparing the shape information of the head 3 of the watching target 1 acquired by the optical sensor 10 with shape information of the head 3 of the watching target 1 previously set in the database 31. Note that the head position estimation unit 22 is not limited to using the shape information of the head 3 of the watching target 1. It may also be configured to estimate the position of the head 3 by extracting facial features of the watching target 1 using a face recognition system from the shape information of the watching target 1 acquired by the optical sensor 10 and recognizing the head 3 based on the extracted facial feature information. Here, information on facial feature amounts is, for example, information indicating the positions of feature points such as the eyes, nose, and mouth of the face, the position of the face area, or the size of the face area.

[0041] The measured location estimation step S300 is a step of estimating the position of the measured location 2 using information on the position of the head 3 estimated in the head position estimation step S200 and information on the preset first distance 1. The measured location estimation step S300 is executed by the measured location estimation unit 23 of the information processing device 201.

[0042] In the measured location estimation step S300, if the measured location estimation unit 23 obtains information on the direction of the measured location 2 relative to the head 3 in addition to information on the position of the head 3 and information on the first distance l, the position of the measured location 2 can be identified as a single location. On the other hand, the optical sensor 10 obtains shape information of the watched over person 1 using visible light or infrared light that is easily absorbed by clothing, bedding, etc., and therefore, for example, if the watched over person 1 is covered with a bedding, it cannot obtain information on the parts of the body covered by the bedding. In other words, if the watched over person 1 is covered with a bedding, the optical sensor 10 can only obtain shape information of the head 3, but not shape information of the torso 4. In this case, the processing unit 20 cannot calculate the direction of the torso 4 (measured location 2) relative to the head 3 using information on the shape of the torso 4. Therefore, it is necessary to estimate the position of the measured location 2 using only information on the position of the head 3 and information on the first distance l, without using information on the direction of the measured location 2 relative to the head 3.

[0043] 6 and 7 , the processing procedure of the measured location estimation unit 23 when estimating the position of the measured location 2 using only information on the position of the head 3 and information on the first distance l will be described. FIG. 6 is a flowchart showing the processing procedure of the measured location estimation step S300 performed by the monitoring system 101 according to embodiment 1. FIG. 7 is a schematic diagram showing the irradiation position of millimeter waves from the millimeter wave sensor 40, illustrating the case where the person being watched over 1 is covered with a futon. Note that when estimating the position of the measured location 2 using only information on the position of the head 3 and information on the first distance l, the measured location estimation step S300 is executed by the measured location estimation unit 23 and the millimeter wave sensor control unit 24 of the information processing device 201.

[0044] 6 and 7 , the measured part estimation unit 23 sets a position that is a first distance 1 away from the estimated position of the head 3 as the irradiation position 41 (step S310). The set irradiation positions 41 may be, for example, a plurality of positions on a circumference of a circle having a radius equal to the first distance 1 and centered on the estimated position of the head 3.

[0045] Next, the millimeter-wave sensor control unit 24 controls the millimeter-wave sensor 40 to irradiate millimeter waves to one irradiation position 41a of the set irradiation positions 41 (step S320). The millimeter-wave sensor 40 irradiates the irradiation position 41a with millimeter waves and receives reflected waves from the irradiation position 41a to detect displacement of the biological surface at the irradiation position 41a. The measured location estimation unit 23 determines whether the millimeter-wave sensor 40 has detected displacement at the irradiation position 41a (step S330). If the millimeter-wave sensor 40 has not detected displacement at the irradiation position 41a (No in step S330), the millimeter-wave sensor control unit 24 controls the millimeter-wave sensor 40 to irradiate millimeter waves to an unirradiated irradiation position 41b of the set irradiation positions 41 (step S340), and the measured location estimation unit 23 determines whether the millimeter-wave sensor 40 has detected displacement at the irradiation position 41b (step S330). The processes of S330 and S340 are repeated until the millimeter wave sensor 40 detects a displacement (Yes in step S350). When the millimeter wave sensor 40 detects a displacement, the measured point estimation unit 23 estimates that the irradiation position 41 at that time is the position of the measured point 2 (step S350).

[0046] The irradiation position 41 need not be limited to a position that is the first distance 1 away from the estimated head position, but may be a position that is several centimeters longer than the first distance 1 or several centimeters shorter than the first distance 1.

[0047] As described above, even when the person being watched over 1 is draped over with a futon, for example, the measured location estimation unit 23 can estimate the measured location 2 with high accuracy by setting the irradiation location 41 using information on the position of the head 3 and information on the preset first distance 1, and having the millimeter-wave sensor 40 detect displacement at the irradiation location 41. In other words, even when only information on the position of the head 3 is obtained by the optical sensor 10 and the head position estimation unit 22, the measured location 2 can be estimated by using the millimeter-wave sensor 40 to detect displacement at the irradiation location 41, which is likely to be the measured location 2.

[0048] After the position of the measured location 2 is estimated in the measured location estimation step S300, a displacement detection step S400 is performed as shown in Fig. 5 . The displacement detection step S400 is a step in which the millimeter-wave sensor 40 detects the displacement at the measured location 2 estimated in the measured location estimation step S300, and is performed by the millimeter-wave sensor 40 and the millimeter-wave sensor control unit 24 of the information processing device 201. In detail, first, the millimeter-wave sensor control unit 24 controls the millimeter-wave sensor 40 to detect the displacement at the estimated measured location 2. Next, the millimeter-wave sensor 40 controlled by the millimeter-wave sensor control unit 24 detects the displacement at the measured location 2. Then, the input device 213, which is an input interface of the information processing device 201, receives information about the displacement at the measured location 2 detected by the millimeter-wave sensor 40.

[0049] The biological information acquisition step S500 is a process of acquiring biological information of the person being watched over 1 based on the displacement detected in the displacement detection step S400, and is performed by the biological information acquisition unit 25 of the information processing device 201. The biological information acquisition unit 25 acquires biological information such as the respiratory rate, pulse rate, and blood pressure of the person being watched over 1 based on the displacement of the body surface at the measurement location 2 detected by the millimeter wave sensor 40.

[0050] According to the above process, the position of the head 3 of the person being watched over 1 is estimated using the optical sensor 10, and the position of the measured location 2 can be estimated using information on the estimated position of the head 3 and information on the first distance 1 indicating the preset distance between the head 3 and the measured location 2. Furthermore, even if only information on the position of the head 3 is obtained by the optical sensor 10 and the head position estimation unit 22, the measured location 2 can be estimated by detecting, with the millimeter wave sensor 40, displacement at the irradiation position 41 that is likely to be the measured location 2. In this way, the measured location 2 can be estimated regardless of the state of the person being watched over 1, and therefore biological information can be acquired with high accuracy.

[0051] <Operations and Effects of First Embodiment> Next, operations and effects of the watching system 101 according to the first embodiment of the present disclosure will be described.

[0052] The monitoring system 101 according to the first embodiment of the present disclosure includes a head position estimation unit 22 that estimates the head position of the person being monitored 1, a measured location estimation unit 23 that estimates the position of the measured location 2 of the person being monitored 1 based on the head position information estimated by the head position estimation unit 22, and a biometric information acquisition unit 25 that acquires biometric information of the person being monitored 1 at the measured location 2 estimated by the measured location estimation unit.

[0053] The information processing device 201 according to the first embodiment of the present disclosure includes a head position estimation unit 22 that estimates the head position of the person being watched 1, a measured location estimation unit 23 that estimates the position of the measured location 2 of the person being watched 1 based on the head position information estimated by the head position estimation unit 22, and a biometric information acquisition unit 25 that acquires the biometric information of the person being watched 1 at the measured location 2 estimated by the measured location estimation unit 23.

[0054] The program 301 according to the first embodiment of the present disclosure enables a computer to realize a head position estimation function that estimates the head position of the person being monitored 1, a measured location estimation function that estimates the position of the measured location 2 based on the information on the head position estimated by the head position estimation function, and a biometric information acquisition function that acquires the biometric information of the person being monitored 1 at the measured location 2 estimated by the measured location estimation function.

[0055] The monitoring method according to embodiment 1 of the present disclosure includes a head position estimation step S200 for estimating the head position of the person being monitored 1, a measured location estimation step S300 for estimating the position of the measured location 2 based on the head position information estimated in the head position estimation step S200, and a biometric information acquisition step S500 for acquiring biometric information of the person being monitored 1 at the measured location 2 estimated in the measured location estimation step S300.

[0056] According to the monitoring system 101, information processing device 201, program 301, and monitoring method of the first embodiment of the present disclosure, the position of the head of the person being watched 1 is estimated, and the position of the measured location 2 is estimated based on information about the estimated head position, so that the position of the measured location 2 can be estimated regardless of the state of the person being watched 1. Specifically, when the optical sensor 10 cannot directly identify the measured location 2, for example, even when the person being watched 1 is covered with a blanket, the millimeter-wave sensor 40 can detect displacement of the irradiation position 41 that is likely to be the measured location 2, thereby estimating the position of the measured location 2. Furthermore, by acquiring displacement of the biological surface at the estimated position of the measured location 2 with the millimeter-wave sensor 40, biological information can be acquired with high accuracy regardless of the state of the person being watched 1.

[0057] Variation 1. Variation 1 of Embodiment 1 will be described. In Embodiment 1, a monitoring system 101 that estimates the position of the measured location 2 using only information on the position of the head 3 and information on the first distance 1 has been described. In Variation 1, a monitoring system 101a will be described that estimates the position of the measured location 2 using information on the direction of the measured location 2 relative to the head 3 in addition to information on the position of the head 3 and information on the first distance 1.

[0058] The monitoring system 101a according to the first modification of the first embodiment differs from the monitoring system 101 according to the first embodiment in that information on the direction of the measured location 2 relative to the head 3 is used to estimate the measured location 2. The information on the direction of the measured location 2 is information that indicates in which direction the measured location 2 is located as viewed from the head 3. For example, if the measured location 2 is the chest, the information on the direction of the measured location 2 indicates the direction from the head 3 toward the torso 4.

[0059] The configuration of a monitoring system 101a according to Variation 1 of Embodiment 1 will be described with reference to FIG. 8 . FIG. 8 is a system block diagram of the monitoring system 101a. As shown in FIG. 8 , the monitoring system 101a includes an optical sensor 10 and a millimeter-wave sensor 40 similar to those of the monitoring system 101 according to Embodiment 1, and an information processing device 201a. The information processing device 201a includes a processing unit 20a and a storage unit 30a. The processing unit 20a includes a measured location estimation unit 23a that estimates the measured location 2 using information on the direction of the measured location 2 relative to the head 3, and the other configuration is similar to that of the processing unit 20 according to Embodiment 1. The storage unit 30a includes a program 301a for causing a computer to function as the information processing device 201a according to Variation 1 of Embodiment 1 and to execute the monitoring method according to Variation 1 of Embodiment 1, and a database 31 similar to that of Embodiment 1.

[0060] The measured location estimation unit 23a estimates the direction of the measured location 2 using information about the shape of the person being watched over 1 acquired by the optical sensor 10, and estimates the position of the measured location 2 using the information about the estimated direction of the measured location 2. The procedure for estimating the measured location 2 by the measured location estimation unit 23a will be described with reference to FIG. 9. FIG. 9 is a flowchart showing the processing procedure of the measured location estimation step S300 by the watching system 101a. Note that the watching method according to Variation 1 of Embodiment 1 is the same as the watching method according to Embodiment 1 shown in FIG. 5, except for the processing procedure in the measured location estimation step S300.

[0061] 9 , in the measurement location estimation step S300 by the monitoring system 101a, the measurement location estimation unit 23a first determines whether the shape information of the person being watched 1 acquired by the optical sensor 10 includes not only information about the head 3 but also information about the contour of the torso 4 or information about facial features (step S301). If it is determined that the optical sensor 10 has acquired information about the contour of the torso 4 or information about facial features (Yes in step S301), the direction of the measurement location 2 is calculated based on the information about the contours of the head 3 and torso 4 or the information about facial features acquired by the optical sensor 10 (step S302). When calculating the direction of the measurement location 2 based on the information about facial features, the measurement location estimation unit 23a calculates the direction of the measurement location 2 based on the information about the facial features acquired by the optical sensor 10, i.e., information about the positions of facial feature points. For example, the measurement location estimation unit 23a extracts the positions of the eyes, nose, and mouth, and calculates the direction of the measurement location 2 relative to the head 3 from the extracted positions of the eyes and mouth. Then, the position of the measurement location 2 is estimated using information on the position of the head 3 estimated in the head position estimation step S200 in FIG. 5 and information on the direction of the measurement location 2 calculated in step S302 (step S351). The information on the position of the measurement location 2 estimated in step S351 is transmitted to the millimeter-wave sensor control unit 24, and the millimeter-wave sensor control unit 24 detects the displacement of the estimated measurement location 2 (step S400 in FIG. 5). On the other hand, if the optical sensor 10 cannot acquire information on the outline of the torso 4 or information on facial features (No in step S301), the same process as in the first embodiment may be performed. That is, similar to step S310 in FIG. 6, a position spaced a first distance 1 from the position of the head 3 is set as the irradiation location 41, and millimeter waves are irradiated onto the irradiation location 41 to estimate the position of the measurement location 2. The processing procedure after setting the irradiation position 41 is the same as that of the monitoring system 101 according to the first embodiment (see FIG. 6).

[0062] According to the monitoring system 101a according to the first modification of the first embodiment, the direction of the measured location 2 is estimated using information about the shape of the person being watched over 1 acquired by the optical sensor 10, and thus the information about the direction of the measured location 2 can be used to estimate the measured location 2, thereby improving the accuracy of estimating the measured location 2. Specifically, as described above, using the information about the direction of the measured location 2 to estimate the measured location 2 allows the position of the measured location 2 to be identified as a single location, and thus acquiring displacement of the biological surface at the identified position of the measured location 2 with the millimeter-wave sensor 40 allows highly accurate acquisition of biological information regardless of the state of the person being watched over 1. Furthermore, using the information about the direction of the measured location 2 to estimate the measured location 2 eliminates the need to repeatedly irradiate the irradiation position 41 with millimeter waves until the millimeter-wave sensor 40 detects displacement, as shown in FIG. 6 , thereby simplifying the processing procedure of the measured location estimation step S300.

[0063] Modification 2. Modification 2 of Embodiment 1 will be described. Modification 1 described a monitoring system 101a that estimates the position of the measured location 2 using information on the direction of the measured location 2. Modification 2 described a monitoring system 101b that estimates the position of the measured location 2 using information on the first distance l corrected using information on the second distance L.

[0064] The monitoring system 101b according to the second modification of the first embodiment differs from the monitoring system 101 according to the first embodiment in that the monitoring system 101b uses information on the first distance l corrected using information on the second distance L to estimate the measured location 2. Here, the information on the second distance L is information indicating the distance between the person being watched over 1 and the optical sensor 10.

[0065] The information on the first distance l used by the monitoring system 101 of the first embodiment is the actual distance between the head 3 of the watching target 1 and the measured location 2. In contrast, the distance between the head 3 of the watching target 1 and the measured location 2 acquired by the optical sensor 10 is not the actual size, but an apparent distance that is smaller by the distance between the watching target 1 and the optical sensor 10. The apparent distance of the watching target 1 acquired by the optical sensor 10 changes depending on the second distance L that indicates the distance between the watching target 1 and the optical sensor 10. Therefore, the monitoring system 101b according to the second modification example corrects the information on the first distance l using the information on the second distance L, and estimates the position of the measured location 2 using the information on the corrected first distance l, thereby improving the estimation accuracy of the measured location 2.

[0066] The configuration of a monitoring system 101b according to Variation 2 of Embodiment 1 will be described with reference to FIG. 10 . FIG. 10 is a system block diagram of the monitoring system 101b. As shown in FIG. 10 , the monitoring system 101b includes an optical sensor 10 and a millimeter-wave sensor 40 similar to those of the monitoring system 101 according to Embodiment 1, and an information processing device 201b. The information processing device 201b includes a processing unit 20b and a storage unit 30b. The processing unit 20b includes a measured location estimation unit 23b that corrects information on the first distance l using information on the second distance L and estimates the position of the measured location 2 using the information on the corrected first distance l. The other configuration is the same as that of the processing unit 20 according to Embodiment 1. The storage unit 30b includes a program 301b for causing a computer to function as the information processing device 201b according to Variation 2 of Embodiment 1 and execute the monitoring method according to Variation 2 of Embodiment 1, and a database 31 similar to that of Embodiment 1.

[0067] The procedure for estimating the measured location 2 by the measured location estimation unit 23b will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the processing procedure of the measured location estimation step S300 by the monitoring system 101b. Note that the measured location estimation step S300 in the second modification of the first embodiment is the same as the processing procedure of the measured location estimation step S300 by the monitoring system 101 according to the first embodiment shown in Fig. 6, except for the processing of step S310.

[0068] As shown in FIG. 11 , in the measured location estimation step S300 by the monitoring system 101b, the measured location estimation unit 23b first determines whether the optical sensor 10 can acquire information on the second distance L (step S303). If a millimeter-wave sensor 40 is installed near the optical sensor 10, the millimeter-wave sensor 40 may be used for measurement. Next, if it is determined that information on the second distance L has been acquired by the optical sensor 10 or the millimeter-wave sensor 40 (Yes in step S303), the measured location estimation unit 23b corrects the first distance l using the measured second distance L (step S304). Then, a position that is the corrected first distance l away from the position of the head 3 estimated in the head position estimation step S200 in FIG. 5 is set as the irradiation position 41 (step S311). If it is determined that information on the second distance L has not been acquired by the optical sensor 10 or the millimeter-wave sensor 40 (No in step S303), a position that is the first distance l away from the position of the head 3 is set as the irradiation position 41 (step S310), as in the first embodiment. The processing procedure after setting the irradiation position 41 is the same as that of the monitoring system 101 according to the first embodiment (see FIG. 6 ).

[0069] According to the monitoring system 101b relating to variant example 2 of embodiment 1, the first distance l can be corrected using the second distance L acquired by the optical sensor 10 or the millimeter wave sensor 40, and information on the corrected first distance l can be used, thereby improving the accuracy of estimating the position of the measured location 2.

[0070] Modification 3. Modification 3 of Embodiment 1 will be described. Modification 2 described a monitoring system 101b that estimates the measured location 2 using the first distance l corrected using the second distance L in order to improve the estimation accuracy of the position of the measured location 2. Modification 3 described a monitoring system 101c that estimates the measured location 2 using the first distance l corrected using information on the position of furniture stored in a storage unit.

[0071] First, the configuration of a watching system 101c according to Variation 3 of Embodiment 1 will be described with reference to Fig. 12. Fig. 12 is a system block diagram of the watching system 101c. As shown in Fig. 12, the watching system 101c includes an optical sensor 10c capable of acquiring not only information about the shape of the watching target 1 but also information about the shape of furniture arranged in the room R of the watching target 1, a millimeter wave sensor 40 similar to that of the watching system 101 according to Embodiment 1, and an information processing device 201c. The information processing device 201c includes a processing unit 20c and a storage unit 30c.

[0072] The storage unit 30c includes a program 301c for causing a computer to function as the information processing device 201c according to the third modification of the first embodiment and to execute the monitoring method according to the third modification of the first embodiment, and a database 31c. In addition to the information stored in the database 31 of the monitoring system 101, the database 31c stores furniture shape information indicating the shapes of furniture (not shown) placed in the room R of the person being watched over 1, and furniture position information linked to the furniture shape information. Note that the type of furniture is not particularly important, but furniture with fixed installation positions, such as a bed or a desk, is preferable because it reduces the number of times the furniture position information can be updated.

[0073] The processing unit 20c includes a measured location estimation unit 23c that corrects the first distance l using information about the position of furniture stored in the storage unit 30c and estimates the measured location 2 using the corrected first distance l, and the other configurations are the same as those of the processing unit 20 according to the first embodiment. In detail, when the watching target 1 is located near furniture, the measured location estimation unit 23c compares information about the shape of the furniture acquired by the optical sensor 10c with information about the shape of the furniture stored in the storage unit 30c and acquires information about the position of the furniture linked to the information about the shape of the furniture from the storage unit 30c. The measured location estimation unit 23c then corrects the information about the first distance l using the information about the position of the furniture acquired from the storage unit 30c and estimates the position of the measured location 2 using the information about the corrected first distance l.

[0074] The procedure for estimating the measured location 2 by the measured location estimation unit 23c will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the processing procedure of the measured location estimation step S300 by the monitoring system 101c. Note that the measured location estimation step S300 in the third modification of the first embodiment is the same as the processing procedure of the measured location estimation step S300 by the monitoring system 101 according to the first embodiment shown in Fig. 6, except for the processing of step S310.

[0075] 13 , in the measured location estimation step S300 by the watching system 101c, first, the measured location estimation unit 23c determines whether or not the optical sensor 10c has acquired information about the shape of furniture near the watching target 1 (step S305). In detail, based on the information about the shape of the watching target 1 and the furniture acquired by the optical sensor 10c, the head position estimation unit 22 or the measured location estimation unit 23c estimates the position of the watching target 1 and the position of the furniture, and determines whether or not the watching target 1 is present near the position of the furniture.

[0076] If the watching target 1 is located near the furniture (Yes in step S305), the furniture shape information acquired by the optical sensor 10c is compared with the furniture shape information stored in the database 31c of the storage unit 30c to identify the furniture (step S306). Once the furniture is identified, the measured location estimation unit 23c retrieves and acquires furniture position information associated with the identified furniture shape information from the database 31c of the storage unit 30c (step S307). Here, when step S307 is executed, since the watching target 1 is determined to be located near the furniture in step S305, the furniture position acquired in step S307 can be approximated to the position of the watching target 1 located near the furniture. In other words, the furniture position information acquired in step S307 can be used as approximately the same value as the second distance L, which indicates the distance between the watching target 1 and the optical sensor 10c. Therefore, the furniture position information acquired in step S307 can be used in place of the second distance L to correct the first distance L (step S308). Then, based on the information on the position of the head 3 estimated in the head position estimation step S200 in FIG. 5, a position spaced by the first distance 1 corrected in step S308 is set as the irradiation position 41 (step S311).

[0077] If the person being watched over 1 is not near the furniture (No in step S305), a position that is a first distance 1 away from the position of the head 3 is set as the irradiation position 41 (step S310), as in the first embodiment. The processing procedure after setting the irradiation position 41 is the same as that of the watching system 101 according to the first embodiment (see FIG. 6 ).

[0078] According to the monitoring system 101c of the third modification of the first embodiment, by using the information on the position of the furniture stored in the memory unit 30c, it is possible to correct the first distance l without measuring the second distance L with the optical sensor 10c or the millimeter-wave sensor 40. In other words, according to the monitoring system 101c of the third modification of the first embodiment, even when it is not possible to acquire information on the second distance L with the optical sensor 10c or the millimeter-wave sensor 40, it is possible to correct the information on the first distance l, thereby improving the accuracy of estimating the position of the measured location 2.

[0079] Variation 4. Variation 4 of Embodiment 1 will be described. Variation 1 described a monitoring system 101a that calculates the measured location 2 using information about the direction of the measured location 2 in order to improve the accuracy of estimating the location of the measured location 2. Variation 4 described a monitoring system 101d that estimates the location of the measured location 2 using information about the direction of the measured location 2 stored in a storage unit.

[0080] First, the configuration of a watching system 101d according to Variation 4 of Embodiment 1 will be described with reference to Fig. 14. Fig. 14 is a system block diagram of the watching system 101d. As shown in Fig. 14, the watching system 101d includes an optical sensor 10c that can acquire not only information about the shape of the watching target 1 but also information about the shape of furniture arranged in the room R of the watching target 1, similar to the watching system 101c according to Variation 3, a millimeter wave sensor 40 similar to the watching system 101 according to Embodiment 1, and an information processing device 201d. The information processing device 201d includes a processing unit 20d and a storage unit 30d.

[0081] The storage unit 30d includes a program 301d for causing a computer to function as the information processing device 201d according to Variation 4 of Embodiment 1 and to execute the monitoring method according to Variation 4 of Embodiment 1, and a database 31d. In addition to the information stored in the database 31 of the monitoring system 101, the database 31d stores information on the shape of the bed placed in the room R of the person being watched over 1, and information on the direction of the measured location of the person being watched over when the person being watched over is on a bed linked to the information on the shape of the bed.

[0082] The processing unit 20d includes a measured location estimation unit 23d that estimates the measured location 2 using information on the direction of the measured location 2 stored in the storage unit 30d, and other configurations are the same as those of the processing unit 20 according to embodiment 1. In detail, when the person being watched over 1 is on a bed, the measured location estimation unit 23d compares the information on the shape of the bed acquired by the optical sensor 10c with the information on the shape of the bed stored in the storage unit 30d, and acquires information on the direction of the measured location 2 linked to the information on the shape of the bed from the storage unit 30d. Then, the measured location estimation unit 23d estimates the position of the measured location 2 using the information on the direction of the measured location 2 acquired from the storage unit 30d.

[0083] The procedure for estimating the measured location 2 by the measured location estimation unit 23d will be described with reference to Fig. 15. Fig. 15 is a flowchart showing the processing procedure of the measured location estimation step S300 by the monitoring system 101d. Note that the monitoring method according to the fourth modification of the first embodiment is the same as the monitoring method according to the first embodiment shown in Fig. 5, except for the processing procedure in the measured location estimation step S300.

[0084] 15 , in the measured location estimation step S300 by the watching system 101d, first, similar to the watching system 101c according to Modification 3, the measured location estimation unit 23d determines whether or not information on the shape of furniture near the watching target 1 is acquired by the optical sensor 10 (step S305). In detail, the head position estimation unit 22 or the measured location estimation unit 23d estimates the position of the watching target 1 and the position of the furniture based on the information on the shape of the watching target 1 and the furniture acquired by the optical sensor 10. Based on the estimated position of the watching target 1 and the position of the furniture, it is determined whether or not furniture is present near the watching target 1.

[0085] If the watching target 1 is present near the furniture (Yes in step S305), the information on the shape of the furniture acquired by the optical sensor 10 is compared with the information on the shape of the bed stored in the database 31d of the storage unit 30d to identify whether the furniture acquired by the optical sensor 10c is a bed (step S306a). If the furniture acquired by the optical sensor 10 is identified as a bed (Yes in step S306a), the measured location estimation unit 23d determines whether the watching target 1 is present on the identified bed (S306b). In detail, the measured location estimation unit 23d determines whether the watching target 1 is present on the bed based on the estimated positions of the watching target 1 and the bed, as in step S305. If the watching over target person 1 is on the bed (Yes in step S306b), the measured part estimation unit 23d reads and acquires information on the direction of the measured part 2 of the watching over target person 1 when the watching over target person 1 is on the bed associated with the identified bed shape information from the database 31d in the storage unit 30d (step S309). Then, the position of the measured part 2 is estimated using the information on the position of the head 3 estimated in the head position estimation step S200 in Figure 5 and the information on the direction of the measured part 2 calculated in step S309 (step S351).

[0086] If the person being watched over 1 is not near any furniture (No in step S305), if the furniture detected by the optical sensor 10c is identified as not being a bed (No in step S306a), or if the person being watched over 1 is not on the bed (No in step S306b), the process may be the same as in embodiment 1. That is, as in step S310 in Fig. 6 , a position that is a first distance 1 away from the position of the head 3 is set as the irradiation position 41, and millimeter waves are irradiated onto the irradiation position 41 to estimate the position of the measured location 2. The processing procedure after setting the irradiation position 41 is the same as that of the watching system 101 according to embodiment 1 (see Fig. 6 ).

[0087] According to the monitoring system 101d of the fourth modification of the first embodiment, by using the information on the direction of the measured location 2 stored in the memory unit 30d, it is possible to acquire the direction of the measured location 2 without calculating the direction of the measured location 2 from the information acquired by the optical sensor 10c. In other words, according to the monitoring system 101d of the fourth modification, even when it is not possible to acquire information on the shape of the torso or the feature amount of the face by the optical sensor 10, it is possible to acquire information on the direction of the measured location 2, thereby improving the estimation accuracy of the measured location 2.

[0088] Note that Modification 1, Modification 2, Modification 3, or Modification 4 may be implemented in appropriate combination. For example, when the monitoring system 101a of Modification 1 and the monitoring system 101d of Modification 4 are implemented in combination, two pieces of information about the direction of the measured location 2 can be obtained by two different measured location estimation steps S300. The reliability of the information about the direction of the measured location 2 can be determined based on whether the two pieces of information about the direction of the measured location 2 match.

[0089] Furthermore, by implementing a combination of the monitoring system 101a of Modification 1 and the monitoring system 101d of Modification 4, it is possible to simplify the estimation of the position of the measured location 2. Here, a case where the monitoring system 101a of Modification 1 and the monitoring system 101d of Modification 4 are implemented in combination and information on the directions of two non-coincident measured locations 2 is obtained will be described with reference to Fig. 16. Fig. 16 is a schematic diagram showing the millimeter wave irradiation position 41 set in the measured location estimation step S310 when information on the directions of two non-coincident measured locations 2 is obtained.

[0090] As shown in FIG. 16 , when information on the directions of two inconsistent measured locations 2 is acquired, two millimeter-wave irradiation positions 41 are set in step S310. Compared to when the irradiation positions 41 are set without using information on the directions of the measured locations 2 (see FIG. 7 ), the number of irradiation positions 41 is smaller when information on the directions of the measured locations 2 is used (see FIG. 16 ). That is, by combining the monitoring system 101a of Modification 1 and the monitoring system 101d of Modification 4, the number of irradiation positions 41 can be narrowed down and the number of times millimeter waves are irradiated to estimate the measured locations 2 can be reduced. Furthermore, by combining the monitoring system 101a of Modification 1 and the monitoring system 101d of Modification 4, the accuracy of estimating the measured locations 2 can be improved compared to when estimating the measured locations 2 without acquiring information on the directions of the measured locations 2.

[0091] The above description deals with a case in which the optical sensor 10 acquires shape information of the watching target 1 as information about the watching target 1. Alternatively, the optical sensor 10 may acquire skeletal information or movement information as information about the watching target 1. In this case, the optical sensor 10 acquires skeletal information or movement information about the watching target 1 in the target information acquisition step S100, and estimates the position of the head 3 of the watching target 1 based on the skeletal information or movement information about the watching target 1 in the head position estimation step S200. Estimation of the position of the head 3 using the skeletal information or movement information about the watching target 1 can be achieved in the same way as when shape information about the watching target 1 is used, or by applying other known techniques. Furthermore, even when skeletal information or movement information is used as information about the watching target 1, the same effects as when shape information about the watching target 1 is used can be obtained.

[0092] Embodiment 2. In the first embodiment of the present disclosure, a monitoring system 101 was described in which an optical sensor 10 acquires shape information of the monitored person 1 as information about the monitored person 1 in order to estimate the position of the head 3 of the monitored person 1. In the second embodiment, a monitoring system 102 is described in which an infrared sensor 11, which is a type of optical sensor 10, acquires information about the thermal energy radiated by the monitored person 1 as information about the monitored person 1. Note that in the second embodiment, the same components as those in the first embodiment of the present disclosure are designated by the same reference numerals, and descriptions of identical or corresponding parts will be omitted. The monitoring system 102 according to the second embodiment will be described below with reference to the drawings. The second embodiment of the present disclosure relates to a monitoring system 102, an information processing device 202 that executes processing related to the monitoring system 102, a program 302 that causes a computer to execute processing related to the monitoring system 102, and a monitoring method using the monitoring system 102.

[0093] <Configuration of Second Embodiment> The configuration of a monitoring system 102 according to a second embodiment of the present disclosure will be described with reference to Fig. 17. Fig. 17 is a system block diagram of the monitoring system 102.

[0094] As shown in Figure 17, the monitoring system 102 of embodiment 2 includes an infrared sensor 11, which is a type of optical sensor 10, an information processing device 202 that controls the infrared sensor 11, and a millimeter wave sensor 40 similar to that of embodiment 1.

[0095] As in the first embodiment, the infrared sensor 11 is a type of optical sensor 10 and is a sensor that acquires information about the person being watched over 1 in order to estimate the position of the head 3 of the person being watched over 1. The infrared sensor 11 is a sensor that can detect infrared rays in the mid-infrared region of 3-5 μm or the far-infrared region of 8-14 μm, and acquires information about the thermal energy radiated by the person being watched over 1 as information about the person being watched over 1. The infrared sensor 11 acquires information about the thermal energy radiated by the person being watched over 1, for example, as a thermal image. Generally, when a thermal image of a human body is acquired using the infrared sensor 11, the head 3 exhibits the maximum brightness value. Therefore, by using the infrared sensor 11, the information processing device 202 (described later) can estimate the location of the acquired image that exhibits the maximum brightness value as the head 3. Because the infrared sensor 11 can acquire the thermal energy radiated by the person being watched over 1, the position of the head 3 of the person being watched over 1 can be estimated regardless of day or night.

[0096] The infrared sensor 11 can also obtain shape information of the person being watched over 1 using information on the thermal energy radiated by the person being watched over 1. The infrared sensor 11 can obtain shape information of the person being watched over 1 because, when the infrared sensor 11 captures the person being watched over 1, which has a higher temperature than the surrounding area, the infrared sensor 11 can obtain a thermal image that shows the temperature distribution of the human body. Furthermore, by recording changes over time in the thermal image obtained by the infrared sensor 11, information on the movements of the person being watched over 1 can also be obtained. The information processing device 202, which will be described later, may estimate the position of the head 3 using information on the shape or movements of the person being watched over 1 obtained by the infrared sensor 11.

[0097] In order to improve the accuracy of estimating the position of the head 3, the monitoring system 102 may further include an optical sensor other than the infrared sensor 11 as a sensor for acquiring information about the person being watched over 1. Examples of the other optical sensor include a visible light camera, an infrared camera, a stereo camera, and LiDAR.

[0098] The information processing device 202 includes a processing unit 220 and a storage unit 230. The processing unit 220 includes an optical sensor control unit 221 that controls the infrared sensor 11, and a head position estimation unit 222 that estimates the position of the head 3 based on information on thermal energy radiated by the watching target 1 acquired by the infrared sensor 11, and other configurations are the same as those of the information processing device 201 of embodiment 1. The storage unit 230 includes a program 302 that causes a computer to function as the information processing device 202 according to embodiment 2 and execute the watching method according to embodiment 2, and a database 31 similar to that of embodiment 1.

[0099] <Monitoring method by watching system according to embodiment 2> Next, the procedure for estimating the position of the measured location 2 of the person being watched over 1 and acquiring biological information of the person being watched over 1, which is performed by the watching system 102 according to embodiment 2, will be described with reference to Fig. 18. Fig. 18 is a flowchart showing the procedure of the watching method by the watching system 102 according to embodiment 2.

[0100] 18 , the monitoring method by the monitoring system 102 includes a subject information acquisition step S100a, a head position estimation step S200a, a measured location estimation step S300, a displacement detection step S400, and a biological information acquisition step S500. The measured location estimation step S300, the displacement detection step S400, and the biological information acquisition step S500 are the same as those in the first embodiment.

[0101] The target person information acquisition step S100a is a step in which the infrared sensor 11 acquires information on the heat energy radiated by the watching target 1. In detail, first, the optical sensor control unit 221 of the information processing device 202 controls the infrared sensor 11 to acquire information on the heat energy radiated by the watching target 1. Next, the infrared sensor 11 controlled by the optical sensor control unit 221 acquires information on the heat energy radiated by the watching target 1. Then, the input device 213, which is an input interface of the information processing device 202, receives the information on the heat energy radiated by the watching target 1 acquired by the infrared sensor 11.

[0102] The head position estimation step S200a is a step of estimating the position of the head 3 of the watching target 1 based on the information on the thermal energy radiated by the watching target 1 acquired in the target information acquisition step S100a. The head position estimation step S200a is executed by the head position estimation unit 222 of the information processing device 202. The head position estimation unit 222 estimates the location showing the maximum brightness value as the position of the head 3 from the acquired thermal energy information.

[0103] In the head position estimation step S200a, a region of interest may be set to estimate the position of the head 3 of the person being watched over 1. The region of interest can be set from information about the shape or movement of the person being watched over 1. For example, information about the shape or movement of the person being watched over 1 can be obtained from a thermal image acquired by the infrared sensor 11, and the region of interest can be set. Specifically, a moving object in the thermal image acquired by the infrared sensor 11 that has higher thermal energy than the surrounding area is determined to be the person being watched over 1, and the region of interest is set. Furthermore, if the monitoring system 102 includes an optical sensor such as a visible camera in addition to the infrared sensor 11, a bed or the like may be identified using the optical sensor such as the visible camera, and the area around the bed may be set as the region of interest. By setting a region of interest and estimating the position of the head 3 in the head position estimation step S200a, the accuracy of estimating the position of the head 3 can be further improved.

[0104] The procedure of the monitoring method after the position of the head 3 is estimated in the head position estimation step S200a is the same as that in the first embodiment.

[0105] <Operations and Effects of Second Embodiment> Next, operations and effects of the monitoring system 102 according to the second embodiment of the present disclosure will be described.

[0106] The monitoring system 102 according to the second embodiment of the present disclosure includes an infrared sensor 11 that acquires information on the thermal energy radiated by the person being monitored 1, a head position estimation unit 222 that estimates the position of the head 3 of the person being monitored 1 based on the information on the thermal energy radiated by the person being monitored 1 acquired by the infrared sensor 11, a measured location estimation unit 23 that estimates the position of the measured location 2 based on the information on the position of the head 3 estimated by the head position estimation unit 222, a millimeter wave sensor 40 that detects displacement at the measured location 2 estimated by the measured location estimation unit 23, and a biometric information acquisition unit 25 that acquires biometric information of the person being monitored 1 based on the displacement detected by the millimeter wave sensor 40.

[0107] The information processing device 202 according to the second embodiment of the present disclosure includes a head position estimation unit 222 that estimates the position of the head 3 of the person being watched 1 based on information on the thermal energy radiated by the person being watched 1 acquired by the infrared sensor 11, a measured location estimation unit 23 that estimates the position of the measured location 2 based on the information on the position of the head 3 estimated by the head position estimation unit 222, and a biometric information acquisition unit 25 that, when the millimeter wave sensor 40 detects a displacement at the measured location 2 estimated by the measured location estimation unit 23, acquires biometric information of the person being watched 1 based on the detected displacement.

[0108] The program 302 according to the second embodiment of the present disclosure enables a computer to realize a head position estimation function that estimates the position of the head 3 of the person being watched 1 based on information on the thermal energy radiated by the person being watched 1 acquired by the infrared sensor 11, a measured location estimation function that estimates the position of the measured location 2 based on information on the position of the head 3 estimated by the head position estimation function, and a biometric information acquisition function that, when the millimeter wave sensor 40 detects a displacement at the measured location 2 estimated by the measured location estimation function, acquires biometric information of the person being watched 1 based on the detected displacement.

[0109] The monitoring method according to the second embodiment of the present disclosure includes a subject information acquisition step S100a in which an infrared sensor 11 acquires information on the thermal energy radiated by the person being watched 1, a head position estimation step S200a in which the position of the head 3 of the person being watched 1 is estimated based on the information on the thermal energy radiated by the person being watched 1 acquired in the subject information acquisition step S100a, a measured location estimation step S300 in which the position of the measured location 2 is estimated based on the information on the position of the head 3 estimated in the head position estimation step S200a, a displacement detection step S400 in which a millimeter wave sensor 40 detects the displacement at the measured location 2 estimated in the measured location estimation step S300, and a biometric information acquisition step S500 in which biometric information of the person being watched 1 is acquired based on the displacement detected in the displacement detection step S400.

[0110] According to the monitoring system 102, information processing device 202, program 302, and monitoring method of the second embodiment of the present disclosure, the position of the head 3 of the person being watched 1 is estimated based on information about the thermal energy radiated by the person being watched 1 obtained by the infrared sensor 11, and therefore the position of the head 3 can be estimated with high accuracy even in the absence of a light source. Therefore, according to the monitoring system 102, information processing device 202, program 302, and monitoring method of the second embodiment of the present disclosure, the position of the head 3 can be estimated regardless of day or night, and further, by using information about the estimated position of the head 3 and information about the preset first distance 1, the position of the measured location 2 can be estimated regardless of day or night.

[0111] Furthermore, according to the monitoring system 102, information processing device 202, program 302, and monitoring method of the second embodiment of the present disclosure, it is possible to estimate the location showing the maximum brightness value as the position of the head 3 from the information on thermal energy acquired by the infrared sensor 11. That is, according to the configuration of the second embodiment, it is possible to easily estimate the location showing the maximum brightness value directly as the position of the head 3, and therefore it is possible to reduce the calculation load on the information processing device 202 compared to the first embodiment in which the position of the head 3 is estimated based on information on the shape of the person being watched over 1.

[0112] Variation 1. Variation 1 of Embodiment 2 will be described. In Embodiment 2, a monitoring system 102 including an infrared sensor 11 as the optical sensor 10 that acquires information about the person being watched 1 has been described. In Variation 1, a monitoring system 102a will be described that further includes an infrared irradiation device that emits infrared light to which the infrared sensor 11 is sensitive. Note that in Variation 1 of Embodiment 2, the same components as in Embodiment 2 of the present disclosure will be designated by the same reference numerals, and descriptions of identical or corresponding parts will be omitted. Hereinafter, a monitoring system 102a according to Variation 1 of Embodiment 2 will be described with reference to the drawings. Variation 1 of Embodiment 2 of the present disclosure relates to a monitoring system 102a, an information processing device 202a that executes processing related to the monitoring system 102a, a program 302a that causes a computer to execute processing related to the monitoring system 102a, and a monitoring method using the monitoring system 102a.

[0113] The configuration of a monitoring system 102a according to a first modification of the second embodiment of the present disclosure will be described with reference to Fig. 19. Fig. 19 is a system block diagram of the monitoring system 102a.

[0114] 19 , a monitoring system 102a according to Modification 1 of Embodiment 2 further includes an infrared irradiation device 50 in addition to the configuration of Embodiment 2. The monitoring system 102a also includes an information processing device 202a including a processing unit 220a that further includes an infrared irradiation device control unit 27 in addition to the configuration of the processing unit 220 of Embodiment 2, and a storage unit 230a that stores a program 302a for executing the functions of the processing unit 220a and a database 31.

[0115] The infrared irradiation device 50 is a device that irradiates infrared rays to which the infrared sensor 11 is sensitive. As in the second embodiment, the infrared sensor 11 is a sensor that can detect infrared rays in the mid-infrared region of 3-5 μm or the far-infrared region of 8-14 μm. Therefore, the infrared irradiation device 50 is a device that irradiates infrared rays in the mid-infrared region of 3-5 μm or the far-infrared region of 8-14 μm to which the infrared sensor 11 is sensitive. The irradiation of the infrared irradiation device 50 is controlled by the infrared irradiation device control unit 27 of the information processing device 202a.

[0116] The monitoring system 102a according to Modification 1 of Embodiment 2 further includes an infrared irradiator 50 to acquire the second distance L using the infrared sensor 11. The infrared sensor 11 alone cannot acquire the distance to the target, i.e., the second distance L. The infrared sensor 11 can acquire information on the second distance L indicating the distance between the person being watched over 1 and the infrared sensor 11 by detecting infrared light irradiated from the infrared irradiator 50 and reflected by the head 3. The information on the second distance L acquired by the infrared irradiator 50 and the infrared sensor 11 is used to correct the information on the first distance l. The procedure for correcting the information on the first distance l using the information on the second distance L is as shown in the measured location estimation step S300a of Modification 1 of Embodiment 1 or the measured location estimation step S300c of Modification 3 (see FIGS. 8 and 10 ).

[0117] The operation and effect of the monitoring system 102a according to the first modification of the second embodiment of the present disclosure will be described.

[0118] According to the monitoring system 102a, information processing device 202a, program 302a, and monitoring method of Variation 1 of Embodiment 2 of the present disclosure, information on the second distance L can be acquired by the infrared irradiation device 50 and the infrared sensor 11, and therefore the first distance 1 can be corrected using the information on the second distance L. By calculating the position of the measured location 2 using the corrected first distance 1, the calculation accuracy of the position of the measured location 2 can be improved, and the estimation accuracy of the measured location 2 can be improved.

[0119] Embodiment 3. In the second embodiment of the present disclosure, the monitoring systems 102 and 102a were described in which the infrared sensor 11 acquires information on the thermal energy radiated by the person being watched 1 and estimates the position of the head 3 of the person being watched 1. In the third embodiment, a monitoring system 103 is described in which the infrared sensor 11 acquires information on the thermal energy radiated by the person being watched 1 and estimates the position of the head 3 of the person being watched 1 and acquires the temperature of the head 3. Note that in the third embodiment, the same components as those in the first or second embodiment of the present disclosure are designated by the same reference numerals, and descriptions of the same or corresponding parts are omitted. The monitoring system 103 according to the third embodiment will be described below with reference to the drawings. The third embodiment of the present disclosure relates to the monitoring system 103, an information processing device 203 that executes processing related to the monitoring system 103, a program 303 that causes a computer to execute processing related to the monitoring system 103, and a monitoring method using the monitoring system 103.

[0120] <Configuration of Third Embodiment> The configuration of a monitoring system 103 according to a third embodiment of the present disclosure will be described with reference to Fig. 20. Fig. 20 is a system block diagram of the monitoring system 103.

[0121] As shown in Figure 20, the monitoring system 103 of embodiment 3 includes an infrared sensor 11 and a millimeter wave sensor 40 similar to those of embodiment 2, and an information processing device 203 that estimates the position of the head 3 of the person being monitored 1 based on the thermal energy information acquired by the infrared sensor 11 and acquires the temperature of the head 3.

[0122] The information processing device 203 includes a processing unit 320 and a storage unit 330. The processing unit 320 includes an optical sensor control unit 321 instead of the optical sensor control unit 221 in the processing unit 220 of the second embodiment, and further includes a determination unit 26 and a head temperature acquisition unit 28.

[0123] The determination unit 26 determines whether the biometric information of the person being watched over 1 acquired by the biometric information acquisition unit 25 is normal or abnormal. The determination unit 26 determines that the acquired biometric information, such as respiratory rate, pulse rate, blood pressure, etc., is normal if it is within a predetermined range, and determines that the acquired biometric information is abnormal if it is outside the predetermined range. The predetermined range is a range that is predetermined as normal values ​​based on medical knowledge, or a range that is predetermined based on the value of the biometric information of the person being watched over 1 under normal circumstances. The predetermined range is, for example, set in advance in the database 31. Note that the determination unit 26 may determine whether the acquired biometric information is normal or abnormal based on changes in the acquired biometric information over time, without comparing the acquired biometric information with a predetermined value. In particular, the determination unit 26 may determine that the acquired biometric information of the person being watched over 1 acquired by the biometric information acquisition unit 25 is abnormal if the value shows irregular changes or if the value fluctuation is large.

[0124] In addition to the functions of the optical sensor control unit 221 of the second embodiment, the optical sensor control unit 321 switches the infrared sensor 11 to a high-precision drive method capable of acquiring thermal energy information with high precision when the determination unit 26 determines an abnormality. The high-precision drive method is, for example, a drive method that improves the S / N ratio by increasing frame integration. The high-precision drive method may also be a noise reduction drive method that suppresses output variations by calibrating pixels using a shutter whose temperature and emissivity are known immediately before temperature measurement, or a signal intensity increase drive method that improves temperature sensitivity by increasing the drive voltage. When the determination unit 26 determines an abnormality, the infrared sensor 11 is switched to the high-precision drive method under the control of the optical sensor control unit 321, and acquires thermal energy information of the head 3 with high precision.

[0125] The head temperature acquisition unit 28 acquires the temperature of the head 3 based on information about the thermal energy acquired when the infrared sensor 11 is driven by the high-precision driving method.

[0126] The storage unit 330 stores the program 303 and the database 31. The program 303 is a computer program for causing the processing unit 320 to execute the functions.

[0127] <Watching Method by Watching System According to Third Embodiment> Next, the procedure of the watching method performed by the watching system 103 according to the third embodiment will be described with reference to Fig. 21. Fig. 21 is a flowchart showing the procedure of the watching method by the watching system 103 according to the third embodiment.

[0128] As shown in FIG. 21, the monitoring method by the monitoring system 103 includes, in addition to the same steps as in the second embodiment, a determination step S600 and a head temperature acquisition step S700.

[0129] 21 , the procedure from the subject information acquisition step S100a to the biological information acquisition step S500 is the same as in the second embodiment, and first, the position of the measurement point 2 of the person being watched over 1 is estimated, and biological information of the person being watched over 1 is acquired. Then, after the biological information of the person being watched over 1 is acquired in the biological information acquisition step S500, the process proceeds to the determination step S600.

[0130] The determination step S600 is a step for determining whether the biometric information of the watching target 1 acquired in the biometric information acquisition step S500 is normal or abnormal. The determination step S600 is executed by the determination unit 26 of the information processing device 204. For example, the determination unit 26 determines that the acquired biometric information of the watching target 1 is normal if the value is within a predetermined range, and that the value is abnormal if the value is outside the predetermined range.

[0131] If the determination in step S600 is normal (Yes in S600), the processing unit 320 ends the processing of the monitoring system 103 (END). If the determination in step S600 is abnormal (No in S600), the processing unit 320 executes the head temperature acquisition step S700. The head temperature acquisition step S700 is a step in which the infrared sensor 11 acquires the temperature of the head 3 if the determination in step S600 is abnormal. The head temperature acquisition step S700 is executed by the optical sensor control unit 321, the infrared sensor 11, and the head temperature acquisition unit 28 of the information processing device 203.

[0132] In the head temperature acquisition step S700, first, the optical sensor control unit 321 switches the infrared sensor 11 to a high-precision drive mode that enables highly accurate acquisition of thermal energy information. The infrared sensor 11 is switched to the high-precision drive mode under the control of the optical sensor control unit 321, and acquires highly accurate information about the thermal energy of the head 3. Then, based on the thermal energy information acquired by the infrared sensor 11, the head temperature acquisition unit 28 acquires the temperature of the head 3.

[0133] The temperature of the head 3 acquired in the head temperature acquisition step S700 may be displayed on a display device (not shown) together with the biological information acquired in the biological information acquisition step S500. Similarly, the determination result in the determination step S600 may also be displayed on a display device (not shown).

[0134] <Operations and Effects of Third Embodiment> Operations and effects of the monitoring system 103 according to the third embodiment of the present disclosure will be described.

[0135] According to the monitoring system 103, information processing device 203, program 303, and monitoring method of the third embodiment of the present disclosure, biological information can be acquired by the millimeter wave sensor 40 regardless of the state of the person being watched 1, and further, in the event of an abnormality, the temperature of the head 3 can be acquired by the infrared sensor 11. That is, according to the monitoring system 103, information processing device 203, program 303, and monitoring method of the third embodiment of the present disclosure, in the event of an abnormality, body temperature can be acquired in addition to biological information such as respiratory rate, pulse rate, and blood pressure.

[0136] Furthermore, according to the monitoring system 103, information processing device 203, program 303, and monitoring method of the third embodiment of the present disclosure, the infrared sensor 11 is switched to the high-precision drive method only when an abnormality occurs, thereby reducing power consumption compared to when the infrared sensor 11 is always driven by the high-precision drive method. In other words, according to the monitoring system 103, information processing device 203, program 303, and monitoring method of the third embodiment of the present disclosure, various types of biometric information can be acquired while reducing power consumption.

[0137] Variation 1. Variation 1 of Embodiment 3 will be described. In Embodiment 3, a configuration was described in which the optical sensor control unit 321 changes the infrared sensor 11 to a high-precision drive method in the event of an abnormality, thereby acquiring the temperature of the head 3 of the person being watched over 1. In Variation 1, a configuration is described in which an infrared irradiation device 50 is further provided, and the infrared irradiation device 50 irradiates infrared light to which the infrared sensor 11 is sensitive in the event of an abnormality, thereby acquiring the temperature of the head 3 of the person being watched over 1. Note that in Variation 1 of Embodiment 3, the same components as in Embodiment 3 of the present disclosure are designated by the same reference numerals, and descriptions of identical or corresponding parts will be omitted. Below, a monitoring system 103a according to Variation 1 of Embodiment 3 will be described with reference to the drawings. Variation 1 of Embodiment 3 of the present disclosure relates to a monitoring system 103a, an information processing device 203a that executes processing related to the monitoring system 103a, a program 303a that causes a computer to execute processing related to the monitoring system 103a, and a monitoring method using the monitoring system 103a.

[0138] The configuration of a monitoring system 103a according to a first modification of the third embodiment of the present disclosure will be described with reference to Fig. 22. Fig. 22 is a system block diagram of the monitoring system 103a.

[0139] As shown in Fig. 22 , a monitoring system 103a according to Variation 1 of Embodiment 3 includes the same infrared sensor 11 and millimeter-wave sensor 40 as those of Embodiment 2 or 3, an information processing device 203a, and an infrared irradiation device 50. The information processing device 203a includes a processing unit 320a and a storage unit 330a. The processing unit 320a includes a head temperature acquisition unit 28a instead of the head temperature acquisition unit 28 in the processing unit 320 of Embodiment 3, and further includes an infrared irradiation device control unit 27. The storage unit 330a stores a program 303a and a database 31. The program 303a is a computer program for executing the functions of the processing unit 320a.

[0140] The infrared irradiation device 50 is a device that irradiates infrared rays to which the infrared sensor 11 is sensitive. As in the first modification of the second embodiment, the infrared irradiation device 50 is a device that irradiates infrared rays in the mid-infrared region of 3-5 μm or the far-infrared region of 8-14 μm to which the infrared sensor 11 is sensitive. The infrared irradiation device 50 is activated by the infrared irradiation device control unit 27 when the determination unit 26 determines that an abnormality has occurred.

[0141] The infrared sensor 11 detects infrared rays emitted from the infrared irradiation device 50 and reflected by the head 3. The head temperature acquisition unit 28a calculates the emissivity ε of the head 3 based on the infrared rays detected by the infrared sensor 11, corrects the thermal energy information based on the calculated emissivity ε, and acquires the temperature of the head 3.

[0142] The method of correcting the thermal energy information by the head temperature acquisition unit 28a will be described. Let us consider a case where infrared rays are irradiated by the infrared irradiation device 50 and the infrared sensor 11 detects the infrared rays reflected by the target. Let the luminance value detected by the infrared sensor 11 be P ｒ , the intensity of the light emitted by the infrared irradiation device 50 is P ０ , the reflectance of the person being watched over 1 is R, and the distance between the person being watched over 1 and the infrared sensor 11 is L, the luminance value P detected by the infrared sensor 11 is ｒ is expressed by the following formula: Here, the intensity P of the light emitted by the infrared irradiation device 50 ０ is a known value. Furthermore, the distance L between the person being watched over 1 and the infrared sensor 11, i.e., the information on the second distance L, can be measured by the infrared sensor 11 or the millimeter wave sensor 40. Therefore, by measuring the second distance L in advance, the luminance value P detected by the infrared sensor 11 can be calculated. ｒ The reflectance R can be calculated from the above equation. Furthermore, according to Kirchhoff's law, the reflectance R and the emissivity ε are expressed by the following equation: According to Kirchhoff's law, if the reflectance R is known, the emissivity ε can be calculated.

[0143] Here, the target temperature when the infrared sensor 11 calculates the temperature is T ｏｂｊ , the reference temperature such as the shutter temperature is T ｓ , the correction temperature due to temperature sensitivity, etc. is Tｃ Then, the target temperature T ｏｂｊ , corrected temperature T ｃ is given by the following formula: Reference temperature T Ｓ Since is a known value, the target temperature T ｏｂｊ That is, by using the emissivity ε, the accuracy of temperature measurement by the infrared sensor 11 can be improved.

[0144] According to the monitoring system 103a, information processing device 203a, program 303a, and monitoring method of Modification 1 of Embodiment 3 of the present disclosure, biological information can be acquired by the millimeter wave sensor 40 regardless of the state of the person being watched over 1, and further, in the event of an abnormality, the emissivity ε can be calculated by the infrared irradiation device 50 and the infrared sensor 11, and the temperature of the head 3 can be acquired with high accuracy. That is, according to the monitoring system 103a, information processing device 203a, program 303a, and monitoring method of Modification 1 of Embodiment 3 of the present disclosure, in the event of an abnormality, body temperature can also be acquired with high accuracy in addition to biological information such as respiratory rate, pulse rate, and blood pressure.

[0145] Furthermore, according to the monitoring system 103a, information processing device 203a, program 303a, and monitoring method of Modification 1 of Embodiment 3 of the present disclosure, the infrared irradiation device 50 is activated only when an abnormality occurs, thereby reducing power consumption compared to constantly activating the infrared irradiation device 50. In other words, according to the monitoring system 103a, information processing device 203a, program 303a, and monitoring method of Modification 1 of Embodiment 3 of the present disclosure, various types of biometric information can be acquired with high accuracy while reducing power consumption.

[0146] Variation 2. Next, Variation 2 of Embodiment 3 will be described. Variation 1 described a configuration further including an infrared irradiator 50, in which the infrared irradiator 50 irradiates infrared light to which the infrared sensor 11 is sensitive in the event of an abnormality, thereby acquiring the temperature of the head 3 of the person being watched over 1. Variation 2 described a configuration including a reference temperature device 60 instead of the infrared irradiator 50, in which the infrared sensor 11 detects reference light emitted by the reference temperature device 60 in the event of an abnormality, thereby acquiring the temperature of the head 3 of the person being watched over 1. Note that in Variation 2 of Embodiment 3, the same components as those in Embodiment 3 or Variation 1 of the present disclosure are designated by the same reference numerals, and descriptions of identical or corresponding parts will be omitted. Below, a monitoring system 103b according to Variation 2 of Embodiment 3 will be described with reference to the drawings. Variation 2 of Embodiment 3 of the present disclosure relates to a monitoring system 103b, an information processing device 203b that executes processing related to the monitoring system 103b, a program 303b that causes a computer to execute processing related to the monitoring system 103b, and a monitoring method using the monitoring system 103b.

[0147] The configuration of a monitoring system 103b according to a second modification of the third embodiment of the present disclosure will be described with reference to Fig. 23. Fig. 23 is a system block diagram of the monitoring system 103b.

[0148] As shown in Fig. 23 , a monitoring system 103b according to Variation 2 of Embodiment 3 includes the same infrared sensor 11 and millimeter-wave sensor 40 as those of Embodiment 2 or 3, an information processing device 203b, and a reference temperature device 60. The information processing device 203b includes a processing unit 320b and a storage unit 330b. The processing unit 320b includes a head temperature acquisition unit 28b instead of the head temperature acquisition unit 28 in the processing unit 320 of Embodiment 3, and further includes a reference temperature device control unit 29. The storage unit 330b stores a program 303b and a database 31. The program 303b is a computer program for executing the functions of the processing unit 320b.

[0149] The reference temperature device 60 is a device, such as a blackbody furnace, that generates reference light for the infrared sensor 11. The reference temperature device 60 is activated by the reference temperature device control unit 29 when the determination unit 26 determines that an abnormality has occurred.

[0150] The infrared sensor 11 detects the infrared rays emitted by the person being watched over 1 and the reference light generated by the reference temperature device 60. The head temperature acquisition unit 28b corrects the temperature sensitivity of the infrared sensor 11 based on the reference light detected by the infrared sensor 11, i.e., corrects the information on the thermal energy radiated by the person being watched over 1, thereby acquiring the temperature of the head 3.

[0151] According to the monitoring system 103b, information processing device 203b, program 303b, and monitoring method of Modification 2 of Embodiment 3 of the present disclosure, biological information can be acquired by the millimeter wave sensor 40 regardless of the state of the person being watched over 1, and further, in the event of an abnormality, the temperature sensitivity of the infrared sensor 11 can be corrected by the reference temperature device 60, and the temperature of the head 3 can be acquired with high accuracy. That is, according to the monitoring system 103b, information processing device 203b, program 303b, and monitoring method of Modification 2 of Embodiment 3 of the present disclosure, in the event of an abnormality, body temperature can also be acquired with high accuracy in addition to biological information such as respiratory rate, pulse rate, and blood pressure.

[0152] Furthermore, according to the monitoring system 103b, information processing device 203b, program 303b, and monitoring method of Modification 2 of Embodiment 3 of the present disclosure, the reference temperature device 60 is activated only when an abnormality occurs, thereby reducing power consumption compared to constantly activating the reference temperature device 60. In other words, according to the monitoring system 103b, information processing device 203b, program 303b, and monitoring method of Modification 2 of Embodiment 3 of the present disclosure, it is possible to acquire biological information with high accuracy while reducing power consumption.

[0153] Although the present disclosure has been described above based on the embodiments, the present disclosure is not limited to the embodiments. Furthermore, appropriate combinations, modifications, omissions, etc. of the embodiments are also included within the scope of the technical idea of ​​the present disclosure.

[0154] REFERENCE SIGNS LIST 1 Person being monitored 2 Location to be measured 3 Head 4 Torso 10 Optical sensor 11 Infrared sensor 20, 220, 320 Processing unit 21, 221, 321 Optical sensor control unit 22, 222 Head position estimation unit 23 Location to be measured estimation unit 24 Millimeter wave sensor control unit 25 Biometric information acquisition unit 26 Determination unit 27 Infrared irradiation device control unit 28 Head temperature acquisition unit 29 Reference temperature device control unit 30, 230, 330 Storage unit 31 Database 40 Millimeter wave sensor 50 Infrared irradiation device 60 Reference temperature device 101, 102, 103 Monitoring system 201, 202, 203 Information processing device 301, 302, 303 Program l First distance L Second distance

Claims

1. A monitoring system comprising: a head position estimator that estimates the position of the head of a person to be monitored; a measurement location estimator that estimates the position of a measurement location of the person to be monitored based on the information on the position of the head estimated by the head position estimator; and a biological information acquirer that acquires the biological information of the person to be monitored at the measurement location estimated by the measurement location estimator.

2. The monitoring system according to claim 1, wherein the measurement location estimator estimates the position of the measurement location using the information on the position of the head and the information on a first distance indicating the distance between the head and the measurement location of the person to be monitored that is preset.

3. The monitoring system according to claim 1 or 2, further comprising: an optical sensor that acquires information on the person to be monitored; and a millimeter-wave sensor that detects displacement at the measurement location estimated by the measurement location estimator, wherein the head position estimator estimates the position of the head based on the information on the person to be monitored acquired by the optical sensor, and the biological information acquirer acquires the biological information of the person to be monitored based on the detected displacement when the millimeter-wave sensor detects the displacement at the measurement location estimated by the measurement location estimator.

4. The monitoring system according to claim 3, wherein the measurement location estimator sets a position at a distance of the first distance from the position of the head estimated by the head position estimator as an irradiation position, irradiates the irradiation position with millimeter waves by the millimeter-wave sensor, and estimates the irradiation position as the measurement location when displacement is detected at the irradiation position.

5. The information on the person to be monitored is information on the shape of the person to be monitored, and the measurement location estimator estimates the direction of the measurement location with respect to the head using the information on the shape of the person to be monitored acquired by the optical sensor, and estimates the position of the measurement location using the information on the estimated direction of the measurement location. The monitoring system according to claim 3 or 4.

6. The monitoring system according to any one of claims 3 to 5, wherein the measurement location estimator corrects the information on the first distance using the information on a second distance indicating the distance between the person to be monitored and the optical sensor, and estimates the position of the measurement location using the corrected information on the first distance.

7. The monitoring system according to any one of claims 3 to 6, further comprising a storage unit that stores information on the shape of furniture arranged in the room of the person to be monitored and information on the position of the furniture associated with the information on the shape of the furniture. The optical sensor can acquire information on the shape of the furniture. When the person to be monitored is present near the furniture, the measured location estimation unit collates the information on the shape of the furniture acquired by the optical sensor with the information on the shape of the furniture stored in the storage unit, acquires the information on the position of the furniture associated with the information on the shape of the furniture from the storage unit, corrects the information on the first distance using the acquired information on the position of the furniture, and estimates the position of the measured location using the corrected information on the first distance.

8. The monitoring system according to any one of claims 3 to 7, further comprising a storage unit that stores information on the shape of a bed arranged in the room of the person to be monitored and information on the direction of the measured location with respect to the head of the person to be monitored associated with the information on the shape of the bed. The optical sensor can acquire information on the shape of the bed. When the person to be monitored is present on the bed, the measured location estimation unit collates the information on the shape of the bed acquired by the optical sensor with the information on the shape of the bed stored in the storage unit, acquires the information on the direction of the measured location associated with the information on the shape of the bed from the storage unit, and estimates the position of the measured location using the acquired information on the direction of the measured location.

9. The information processing apparatus according to any one of claims 3 to 6, wherein the optical sensor is an infrared sensor that detects infrared rays, the information on the person to be monitored includes information on the thermal energy radiated by the person to be monitored, and the head position estimation unit estimates the position of the head of the person to be monitored based on the information on the thermal energy.

10. The monitoring system according to claim 9, further comprising an infrared irradiation device that irradiates infrared rays to which the infrared sensor is sensitive, wherein the infrared sensor detects infrared rays irradiated from the infrared irradiation device and reflected by the head, obtains the second distance based on the detected infrared rays, and the measurement location estimation unit corrects the information on the first distance using the information on the second distance indicating the distance between the person under monitoring and the infrared sensor, and estimates the position of the measurement location using the corrected information on the first distance.

11. The monitoring system according to claim 9 or 10, further comprising: a determination unit that determines whether the biological information of the person under monitoring acquired by the biological information acquisition unit is normal or abnormal; an optical sensor control unit that changes the infrared sensor to a high-precision driving method capable of acquiring the information on the thermal energy with high precision when the determination unit determines that it is abnormal; and a head temperature acquisition unit that acquires the temperature of the head based on the information on the thermal energy acquired by the infrared sensor.

12. The monitoring system according to any one of claims 9 to 11, further comprising: an infrared irradiation device that irradiates infrared rays to which the infrared sensor is sensitive; a determination unit that determines whether the biological information of the person under monitoring acquired by the biological information acquisition unit is normal or abnormal; an infrared irradiation device control unit that activates the infrared irradiation device when the determination unit determines that it is abnormal; and a head temperature acquisition unit that calculates the emissivity of the head by detecting, by the infrared sensor, the infrared rays irradiated from the infrared irradiation device and reflected by the head, and acquires the temperature of the head by correcting the information on the thermal energy based on the calculated emissivity.

13. The monitoring system according to any one of claims 9 to 12, further comprising: a reference temperature device that generates reference light for the infrared sensor; a determination unit that determines whether the biological information of the person under monitoring acquired by the biological information acquisition unit is normal or abnormal; a reference temperature device control unit that activates the reference temperature device when the determination unit determines that it is abnormal; and a head temperature acquisition unit that acquires the temperature of the head based on the information on the thermal energy obtained by the infrared sensor detecting the infrared rays emitted by the person under monitoring and the reference light generated by the reference temperature device.

14. An information processing apparatus comprising: a head position estimation unit that estimates the position of the head of a person to be monitored; a measurement location estimation unit that estimates the position of the measurement location of the person to be monitored based on the information on the position of the head estimated by the head position estimation unit; and a biological information acquisition unit that acquires the biological information of the person to be monitored at the measurement location estimated by the measurement location estimation unit.

15. A program for causing a computer to realize: a head position estimation function that estimates the position of the head of a person to be monitored; a measurement location estimation function that estimates the position of the measurement location of the person to be monitored based on the information on the position of the head estimated by the head position estimation function; and a biological information acquisition function that acquires the biological information of the person to be monitored at the measurement location estimated by the measurement location estimation function.

16. A monitoring method comprising: a head position estimation step of estimating the position of the head of a person to be monitored; a measurement location estimation step of estimating the position of the measurement location of the person to be monitored based on the information on the position of the head estimated in the head position estimation step; and a biological information acquisition step of acquiring the biological information of the person to be monitored at the measurement location estimated in the measurement location estimation step.

Images