Estimation system, estimation method and program

Abstract

To estimate motion of a person even when the person is not present in a detection area of an infrared sensor.SOLUTION: An estimation system 1 has a first estimation part 232 and a second estimation part 233. The first estimation part 232 estimates an object which is a heat source in a detection area on the basis of temperature distribution information in the detection area acquired by an infrared sensor 3. The second estimation part 233 estimates motion of a person associated with the estimated result on the basis of the estimated result by the first estimation part 232.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure generally relates to an estimation system, an estimation method, and a program, and more particularly to an estimation system, an estimation method, and a program that use an infrared sensor. [Background technology]

[0002] An image processing device is disclosed in Patent Document 1. The image processing device in Patent Document 1 measures a thermal image with a sensor unit and estimates a person's posture from the thermal image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 6-117836 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the image processing device of Patent Document 1 has a problem in that it cannot detect a person when there is no person in the detection area of ​​the sensor unit.

[0005] The present disclosure has been made in consideration of the above-mentioned reasons, and aims to provide an estimation system, an estimation method, and a program capable of estimating human movements even when a human is not present in the detection area of ​​an infrared sensor. [Means for solving the problem]

[0006] An estimation system according to an embodiment of the present disclosure includes a first estimation unit and a second estimation unit. The first estimation unit estimates an object that is a heat source in a detection area based on temperature distribution information in the detection area acquired by an infrared sensor. The second estimation unit estimates a human motion related to an estimation result based on the estimation result by the first estimation unit.

[0007] An estimation method according to one aspect of the present disclosure is an estimation method executed by one or more processors. The estimation method includes a first estimation step and a second estimation step. In the first estimation step, an estimation is made regarding an object that is a heat source existing in a detection area based on temperature distribution information of the detection area acquired by an infrared sensor. In the second estimation step, a human movement related to the estimation result is estimated based on the estimation result in the first estimation step.

[0008] A program according to one embodiment of the present disclosure is a program for causing one or more processors to execute the estimation method. Effect of the Invention

[0009] According to the estimation system, estimation method, and program according to the above aspects of the present disclosure, it is possible to estimate the movement of a person even when the person is not present in the detection area of ​​the infrared sensor. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram showing a configuration of an estimation system according to the first embodiment. [Diagram 2] FIG. 2 is a schematic diagram showing a thermal image based on temperature distribution information acquired by an infrared sensor. [Diagram 3] FIG. 3 is a flowchart showing the operation of the estimation system according to the first embodiment. [Figure 4] FIG. 4 is a schematic diagram showing the configuration of an estimation system according to the second embodiment. [Diagram 5] FIG. 5 is a schematic diagram showing a configuration of an estimation system according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, each preferred embodiment of the present disclosure will be described in detail with reference to the drawings. In addition, elements common to each embodiment described below are given the same reference numerals, and duplicated explanations of the common elements may be omitted. Each embodiment below is merely one of various embodiments of the present disclosure. Each embodiment can be modified in various ways depending on the design, etc., as long as the object of the present disclosure can be achieved. In addition, each embodiment (including each modified example) may be realized in appropriate combination.

[0012] (Embodiment 1) (1) Overview First, an overview of an estimation system 1 according to a first embodiment will be described with reference to FIG.

[0013] As shown in FIG. 1, the estimation system 1 of the first embodiment is provided in a facility 100. The estimation system 1 is a system that estimates the movement of a person in the facility 100. In the first embodiment, it is assumed that the facility 100 is an office. In the present disclosure, the term "facility" includes residential facilities used for residential purposes, as well as non-residential facilities such as stores, offices, welfare facilities, educational facilities, hospitals, and factories. Non-residential facilities also include restaurants, amusement parks, hotels, inns, kindergartens, nurseries, and community centers. In other words, the facility 100 may be a residential facility such as an apartment building, or a non-residential facility such as an office. Furthermore, the facility 100 also includes a facility in which residential facilities and non-residential facilities are mixed, for example, where the lower floors are stores and the upper floors are residences.

[0014] The estimation system 1 includes a first estimating unit 232 and a second estimating unit 233.

[0015] The first estimation unit 232 estimates an object that is a heat source in the detection area based on the temperature distribution information of the detection area acquired by the infrared sensor 3. Note that the "object that is a heat source" in this disclosure includes people and objects that radiate or generate heat. Also, the "object" in this disclosure is not intended to include people.

[0016] The second estimation unit 233 estimates a motion of a person related to the estimation result by the first estimation unit 232 based on the estimation result by the first estimation unit 232.

[0017] For example, when a person who was working on a personal computer sitting on a chair leaves the seat for a reason such as going to the toilet, heat from the use of the person who left the seat remains in the display unit such as a liquid crystal display of the personal computer used by the person who left the seat, and the chair, etc. For example, when the display unit and chair used by the person who left the seat are present in the detection area of ​​the infrared sensor 3, the first estimation unit 232 estimates that the object that is a heat source present in the detection area is the display unit and the chair. Then, the second estimation unit 233 estimates the person's behavior related to the display unit and the chair based on the estimation result that the object that is a heat source present in the detection area is the display unit and the chair. For example, the second estimation unit 233 estimates the person's behavior that the person is working or the person is away from the seat based on the temperature of the display unit and the chair. In other words, according to the estimation system 1 according to the first embodiment, even if there is no person in the detection area of ​​the infrared sensor 3, it is possible to estimate the person's behavior related to the object that is a heat source present in the detection area.

[0018] (2) Details Hereinafter, a detailed configuration of the estimation system 1 according to the first embodiment will be described with reference to FIGS.

[0019] (2.1) Estimation system configuration As shown in FIG. 1, the estimation system 1 includes an estimation device 2 and an infrared sensor 3.

[0020] (2.2) Infrared sensor configuration The infrared sensor 3 is directly or indirectly installed on construction materials such as a wall or a ceiling of the facility 100. In the first embodiment, the infrared sensor 3 is an infrared camera that generates a thermal image G1 (see FIG. 2) based on temperature distribution information acquired by the infrared sensor 3.

[0021] The infrared sensor 3 includes optical components such as a lens, a thermal image pickup element (or a plurality of infrared detection units), and a thermal image generation unit. The plurality of infrared detection units are arranged, for example, in a two-dimensional array.

[0022] The infrared sensor 3 acquires temperature distribution information of the detection area of ​​the infrared sensor 3 by detecting infrared rays such as far-infrared rays emitted by an object present in the detection area of ​​the infrared sensor 3 through multiple infrared detection units via optical components. Moreover, the infrared sensor 3 of the first embodiment acquires multiple pieces of temperature distribution information along a time series. The "temperature distribution information" is information that indicates the temperature distribution of the detection area of ​​the infrared sensor 3. More specifically, the "temperature distribution information" is information that associates information on infrared rays (i.e., temperature information) detected by each of the multiple infrared detection units with two-dimensional arrangement information.

[0023] The thermal image generating unit of the infrared sensor 3 generates a two-dimensional thermal image G1 (see FIG. 2) based on the temperature distribution information of the detection area. In other words, the thermal image G1 is a thermal image based on the temperature distribution information of the detection area of ​​the infrared sensor 3. In this disclosure, the "thermal image" means thermal imaging data, and may include still images (frames), videos, and frame-by-frame thermal images. In this embodiment, it is assumed that the thermal image G1 generated by the infrared sensor 3 is a video including multiple frames in a time series. In other words, the thermal image G1 generated by the infrared sensor 3 is a video including multiple frames corresponding to multiple pieces of temperature information in a time series.

[0024] Fig. 2 shows an example of a thermal image G1 generated by the infrared sensor 3. In Fig. 2, an example of the temperature distribution in the thermal image G1 generated by the infrared sensor 3 (i.e., the temperature distribution in the detection area of ​​the infrared sensor 3) is shown by the density of dot hatching. In the example of Fig. 2, the lower the density of the dot hatching, the higher the temperature. The parts without dot hatching in Fig. 2 are the parts with the highest temperature in the thermal image G1.

[0025] The infrared sensor 3 transmits the generated thermal image G1 to the estimation device 2.

[0026] (2.3) Configuration of the estimation device The estimation device 2 includes a communication unit 21, a storage unit 22, and a processing unit 23.

[0027] The communication unit 21 includes a communication interface configured to be able to communicate with the infrared sensor 3. The communication unit 21 receives the thermal image G1 from the infrared sensor 3.

[0028] The storage unit 22 is a semiconductor memory such as a Read Only Memory (ROM), a Random Access Memory (RAM), or an Electrically Erasable Programmable Read Only Memory (EEPROM). The storage unit 22 is not limited to a semiconductor memory, and may be a hard disk drive or the like. For example, the storage unit 22 stores spatial information of the facility 100. The spatial information includes dimensional design information of the facility 100, three-dimensional model data such as BIM (Building Information Modeling) data of the facility 100, position information of the infrared sensor 3, and position information of the seats of people working in the office.

[0029] The processing unit 23 includes an acquisition unit 231 , a first estimation unit 232 , and a second estimation unit 233 .

[0030] The acquisition unit 231 acquires the thermal image G1 received by the communication unit 21 from the infrared sensor 3. In other words, the acquisition unit 231 acquires, via the communication unit 21, temperature distribution information of the detection area of ​​the infrared sensor 3 acquired by the infrared sensor 3.

[0031] The first estimation unit 232 estimates an object that is a heat source and exists in the detection area of ​​the infrared sensor 3, based on the thermal image G1 acquired by the acquisition unit 231. In other words, the first estimation unit 232 estimates an object that is a heat source and exists in the detection area of ​​the infrared sensor 3, based on the temperature distribution information of the detection area acquired by the infrared sensor 3. In the following description, the estimation performed by the first estimation unit 232 may be referred to as a first estimation process (first estimation step).

[0032] The first estimation unit 232 performs object detection processing on the thermal image G1 to detect an area R0 (see FIG. 2) in the thermal image G1 that includes an image corresponding to an object that is a heat source. FIG. 2 shows an example in which the first estimation unit 232 detects areas R1 and R2. Area R1 is an area that includes an image G11 that corresponds to a person present in the detection area of ​​the infrared sensor 3. Area R2 is an area that includes an image G12 that corresponds to a display unit, such as a liquid crystal display, present in the detection area. By performing object detection processing to detect area R0 that includes an image corresponding to an object that is a heat source, the accuracy of image analysis processing such as image segmentation and skeleton estimation processing can be improved.

[0033] The first estimation unit 232 performs a difference detection process as the object detection process. The difference detection process is, for example, a process of detecting the region R0 using a background difference, or a process of detecting the region R0 by deriving the amount of movement of the image G11 corresponding to a person using an inter-frame difference.

[0034] The first estimation unit 232 of the first embodiment performs image segmentation, skeleton estimation processing, and the like on the region R0 of the thermal image G1 to estimate an object that serves as a heat source and is included in the region R0.

[0035] Image segmentation is a process of determining whether each pixel constitutes a person, whether the pixel constitutes a background, and whether the pixel constitutes an object other than a person. The first estimation unit 232 performs image segmentation using a trained model generated by machine learning. The trained model is generated, for example, by supervised learning using multiple pieces of teacher data. The multiple pieces of teacher data are, for example, multiple thermal images generated by capturing an image of a situation in which an object that serves as a heat source exists in the detection area of ​​the infrared sensor 3. The first estimation unit 232 can obtain information on the type of object that serves as a heat source existing in the detection area as the output of the trained model by inputting the thermal image G1 generated by the infrared sensor 3 to the trained model. Note that the machine learning algorithm is, for example, a neural network.

[0036] The skeleton estimation process is a process of detecting the main parts of a person, such as the face, neck, shoulders, hands, and feet, from the region R1 of the thermal image G1, and estimating the skeleton of the person by connecting these parts. The first estimation unit 232 performs the skeleton estimation process using a trained model generated by machine learning. The trained model is generated, for example, by supervised learning using a plurality of teacher data. The plurality of teacher data is, for example, a plurality of thermal images capturing a situation in which a person is present in the detection area. The first estimation unit 232 can obtain information on the posture of a person present in the detection area, for example, as an output of the trained model, by using the thermal image G1 generated by the infrared sensor 3 as an input of the trained model. That is, the first estimation unit 232 of the first embodiment can estimate the posture of a person present in the detection area of ​​the infrared sensor 3 based on the temperature distribution information acquired by the infrared sensor 3. The machine learning algorithm is, for example, a neural network.

[0037] For example, the first estimation unit 232 of the first embodiment estimates the posture of a person present in the detection area of ​​the infrared sensor 3 in a time series based on a thermal image G1 including a plurality of frames in a time series (i.e., time series data of frames). In other words, the first estimation unit 232 estimates the posture of a person present in the detection area of ​​the infrared sensor 3 in a time series based on a plurality of pieces of temperature distribution information in a time series. The first estimation unit 232 performs, for example, a skeleton estimation process, and estimates the posture of a person present in the detection area of ​​the infrared sensor 3 in a time series.

[0038] Also, for example, the first estimation unit 232 of the first embodiment estimates the posture of a person present in the detection area of ​​the infrared sensor 3 and estimates an object that is a heat source present in the detection area based on the thermal image G1. In other words, the first estimation unit 232 estimates the posture of a person present in the detection area of ​​the infrared sensor 3 and estimates an object that is a heat source present in the detection area based on the temperature distribution information. The first estimation unit 232 estimates the posture of a person present in the detection area of ​​the infrared sensor 3 by, for example, performing a skeleton estimation process, and estimates the type of object that is a heat source present in the detection area by, for example, performing image segmentation.

[0039] Also, for example, the first estimation unit 232 flattens the histogram of the temperature distribution in the detection area based on the temperature distribution information, generates (or converts) a thermal image of the detection area, and estimates the object that is a heat source existing in the detection area. By the first estimation unit 232 estimating the object that is a heat source existing in the detection area based on the thermal image after the histogram flattening, the estimation accuracy of the estimation performed by the first estimation unit 232 can be improved.

[0040] As described above, the second estimation unit 233 estimates the movement of a person related to the estimation result of the first estimation process based on the estimation result by the first estimation unit 232. In the following description, the estimation performed by the second estimation unit 233 may be referred to as the second estimation process (second estimation step). Note that "a person related to the estimation result" includes a person present in the detection area of ​​the infrared sensor 3, a person who holds or uses an object that is a heat source present in the detection area of ​​the infrared sensor 3, and a person in the vicinity of an object that is a heat source present in the detection area of ​​the infrared sensor 3. "In the vicinity of an object that is a heat source" means that the linear distance from the object that is a heat source is within a predetermined distance. The predetermined distance is, for example, 1 m. However, the predetermined distance may be arbitrarily set, and the predetermined distance may be, for example, 2 m.

[0041] For example, the second estimation unit 233 estimates the movement of a person present in the detection area of ​​the infrared sensor 3 based on the estimation result of the person's posture along a time series by the first estimation unit 232. For example, when the first estimation unit 232 estimates that a person present in the detection area has been sitting in a chair with both hands out in front for a predetermined period of time, the second estimation unit 233 estimates that the person present in the detection area is using a personal computer. According to the estimation system 1 of the first embodiment, the movement of a person present in the detection area is estimated based on the estimation result of the person's posture along a time series, so that the estimation accuracy of the estimation performed by the second estimation unit 233 can be improved.

[0042] For example, the second estimation unit 233 estimates the movement of a person present in the detection area based on the estimation result of the person's posture by the first estimation unit 232 and the estimation result of the object that serves as a heat source. For example, when the first estimation process estimates that a person present in the detection area is sitting in a chair with both hands out in front and that the person's face is facing the display unit present in the detection area, the second estimation unit 233 estimates that the person present in the detection area is using a personal computer. Also, for example, when the first estimation process estimates that a person present in the detection area is putting both hands out in front and placing their hands on a keyboard, the second estimation unit 233 estimates that the person present in the detection area is using a personal computer. According to the estimation system 1 of the first embodiment, the movement of a person present in the detection area is estimated based on the estimation result of the person's posture by the first estimation unit 232 and the estimation result of the object that serves as a heat source, so that the estimation accuracy of the estimation performed by the second estimation unit 233 can be improved.

[0043] The second estimation unit 233 of the first embodiment performs the second estimation process using a trained model generated by machine learning. The trained model is generated by supervised learning using a plurality of teacher data, for example. The plurality of teacher data are, for example, the estimation result of the first estimation process by the first estimation unit 232. The second estimation unit 233 can obtain information on the movement of a person present in the detection area of ​​the infrared sensor 3, for example, as the output of the trained model, by using the estimation result of the first estimation process by the first estimation unit 232 as the input of the trained model. That is, the second estimation unit 233 of the first embodiment can estimate the movement of a person present in the detection area of ​​the infrared sensor 3 based on the estimation result of the first estimation process by the first estimation unit 232. The machine learning algorithm is, for example, a neural network.

[0044] (3) Operation of the estimation system Next, the operation of the estimation system 1 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the operation of the estimation system 1 of the first embodiment.

[0045] First, the estimation system 1 acquires temperature distribution information (S1). More specifically, the infrared sensor 3 acquires temperature distribution information in the detection area of ​​the infrared sensor 3.

[0046] Next, the estimation system 1 generates a thermal image G1 (S2). More specifically, the infrared sensor 3 generates the thermal image G1 based on temperature distribution information of the detection area.

[0047] Next, the estimation system 1 performs a first estimation process (first estimation step) (S3). More specifically, the first estimation unit 232 performs processes such as object detection process, image segmentation, and skeleton estimation process, and estimates an object that is present in the detection area of ​​the infrared sensor 3 and serves as a heat source.

[0048] Next, the estimation system 1 performs a second estimation process (second estimation step) (S4). More specifically, the second estimation unit 233 estimates a human motion related to the estimation result based on the estimation result by the first estimation unit 232.

[0049] The flowchart shown in FIG. 3 is merely an example, and processing may be added or deleted as appropriate.

[0050] (4) Variations Modifications of the first embodiment are listed below.

[0051] (4.1) Variation 1 The first estimation unit 232 may estimate the emissivity of an object that is a heat source and exists in the detection area of ​​the infrared sensor 3 based on the thermal image G1. The first estimation unit 232 then estimates the type of object that is a heat source and exists in the detection area based on the estimated emissivity of the object and, for example, emissivity information stored in advance in the storage unit 22. "Emissivity information" is information that associates a substance with an emissivity. The substance may be, for example, the material of the object surface. The emissivity information includes, for example, information that associates a human body with the approximate emissivity of the human body, and information that associates a liquid crystal display with the approximate emissivity of the liquid crystal display.

[0052] The second estimation unit 233 estimates a movement of a person related to the estimation result by the first estimation unit 232, based on at least the type of object that will be a heat source estimated by the first estimation unit 232. For example, the second estimation unit 233 estimates a movement of a person related to the estimation result by the first estimation unit 232, based on the type of object estimated by the first estimation unit 232 and the posture of the person estimated by the first estimation unit 232.

[0053] The first estimation unit 232 estimates the emissivity and then estimates the type of object that is present in the detection area and serves as a heat source based on the estimated emissivity, thereby making it possible to improve the accuracy of the first estimation process.

[0054] (4.2) Other Modifications The function equivalent to that of the estimation device 2 according to the first embodiment may be embodied in an estimation method, a program, or a non-transitory recording medium having a program recorded thereon. The estimation method according to one aspect is an estimation method executed by one or more processors. The estimation method has a first estimation step and a second estimation step. In the first estimation step, an estimation is made regarding an object that is a heat source existing in the detection area based on the temperature distribution information of the detection area acquired by the infrared sensor 3. In the second estimation step, a human movement related to the estimation result is estimated based on the estimation result in the first estimation step. The program according to one aspect is a program for causing one or more processors to execute the above estimation method.

[0055] The execution subject of the estimation system 1 or the estimation method in the present disclosure includes a computer system. The computer system is mainly composed of a processor and a memory as hardware. The processor executes a program recorded in the memory of the computer system to realize the function of the execution subject of the estimation system 1 or the estimation method in the present disclosure. The program may be pre-recorded in the memory of the computer system, provided through an electric communication line, or provided by recording it in a non-transitory recording medium such as a memory card, an optical disk, or a hard disk drive that can be read by the computer system. The processor of the computer system is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or a large-scale integrated circuit (LSI). The integrated circuits such as IC or LSI here are called by different names depending on the degree of integration, and include integrated circuits called system LSI, VLSI (Very Large Scale Integration), or ULSI (Ultra Large Scale Integration). Furthermore, a field-programmable gate array (FPGA) that is programmed after the manufacture of the LSI, or a logic device that can reconfigure the connection relationship inside the LSI or the circuit partition inside the LSI, can also be adopted as a processor. The electronic circuits may be integrated in one chip or distributed among multiple chips. The chips may be integrated in one device or distributed among multiple devices. The computer system referred to here includes a microcontroller having one or more processors and one or more memories. Thus, the microcontroller is also composed of one or more electronic circuits including a semiconductor integrated circuit or a large-scale integrated circuit.

[0056] Furthermore, it is not essential for the estimation system 1 that multiple functions are integrated into one housing, and the components of the estimation system 1 may be distributed across multiple housings. Furthermore, at least some of the functions of the estimation system 1, for example, some of the functions of the estimation device 2, may be realized by the cloud (cloud computing) or the like.

[0057] In the first embodiment, at least a part of the functions of the estimation system 1 that are distributed among a plurality of devices may be integrated into one housing. For example, a part of the functions of the estimation system 1 that are distributed among the estimation device 2 and the infrared sensor 3 may be integrated into one housing.

[0058] In the first embodiment, the infrared sensor 3 is an infrared camera. However, it is sufficient for the infrared sensor 3 to acquire temperature distribution information of the detection area. For example, the infrared sensor 3 may transmit the acquired temperature distribution information to the estimation device 2. Then, the estimation device 2 generates a thermal image G1 based on the received temperature distribution information, and estimates the motion of a person with respect to an object present in the detection area of ​​the infrared sensor 3 using the thermal image G1.

[0059] The infrared sensor 3 may be an infrared sensor equipped with a lens capable of capturing an image of a relatively wide range, such as a wide-angle lens, a fisheye lens, or a compound lens composed of a plurality of lenses with different optical axis directions. When the infrared sensor 3 is an infrared sensor equipped with a lens capable of capturing an image of a relatively wide range, the infrared sensor 3 may be installed, for example, on the ceiling of the facility 100, and disposed so as to have a detection area covering substantially the entire floor.

[0060] In the first embodiment, the first estimation unit 232 performs the difference detection process as the object detection process. However, the first estimation unit 232 may perform the object detection process using a trained model generated by machine learning. The trained model is generated by supervised learning using a plurality of teacher data, for example. The plurality of teacher data are, for example, a plurality of thermal images capturing a situation in which a person is present in the detection area of ​​the infrared sensor 3. The first estimation unit 232 can obtain information on whether or not a person is present in the detection area (or the thermal image G1) of the infrared sensor 3 as the output of the trained model by inputting the thermal image G1 generated by the infrared sensor 3 to the trained model. The machine learning algorithm is, for example, a neural network.

[0061] In the first embodiment, the machine learning algorithm is exemplified as a neural network. However, the machine learning algorithm is not limited to a neural network, and may be, for example, an eXtreme Gradient Boosting (XGB) regression, a Random Forest, a decision tree, a Logistic Regression, a Support Vector Machine (SVM), a Naive Bayes classifier, or a k-nearest neighbors. Furthermore, the machine learning algorithm may be, for example, a Gaussian Mixture Model (GMM), or a k-means clustering.

[0062] The trained model may be updated by performing additional training.

[0063] It is not essential that the first estimation unit 232 performs the first estimation process using a trained model generated by machine learning. For example, the first estimation unit 232 may perform the first estimation process based on a preset method (rule).

[0064] Furthermore, it is not essential that the second estimation unit 233 performs the second estimation process using a trained model generated by machine learning. For example, the second estimation unit 233 may perform the second estimation process based on a preset method (rule).

[0065] (Embodiment 2) As shown in FIG. 4, an estimation system 1A according to the second embodiment differs from the estimation system 1 according to the first embodiment in that an estimation system 1A according to the second embodiment further includes a detection unit 4 different from the infrared sensor 3.

[0066] The detection unit 4 is a human detection unit such as a seating sensor or a human presence sensor. The detection unit 4 detects a person present in the detection area of ​​the infrared sensor 3. For example, the detection unit 4, which is a seating sensor, is disposed on a chair present in the detection area of ​​the infrared sensor 3. When, for example, a person is sitting in a chair, the detection unit 4 transmits seating information indicating that the person is sitting in the chair to the estimation device 2.

[0067] The second estimation unit 233 estimates a person's movement related to the estimation result by the first estimation unit 232, based on the estimation result by the first estimation unit 232 and the detection result by the detection unit 4 which is a human detection unit that detects people. For example, when the first estimation unit 232 estimates that a display unit such as a liquid crystal display present in the detection area of ​​the infrared sensor 3 is operating, and the detection unit 4 detects that a person is seated, the second estimation unit 233 estimates that the person sitting in the chair is using a personal computer.

[0068] The detection unit 4 may be a visible light camera. The detection unit 4, which is a visible light camera, detects a person present in the detection area of ​​the infrared sensor 3.

[0069] The second estimation unit 233 estimates the movement of the person related to the estimation result by the first estimation unit 232, based on the estimation result by the first estimation unit 232 and the detection result by the detection unit 4 which is a visible light camera. For example, when the first estimation unit 232 estimates that a display unit such as a liquid crystal display present in the detection area of ​​the infrared sensor 3 is operating, and the detection unit 4 detects that a person is present in the detection area of ​​the infrared sensor 3, the second estimation unit 233 estimates that the person detected by the detection unit which is a visible light camera is using a personal computer.

[0070] Further, the second estimation unit 233 estimates a movement of the person related to the estimation result by the first estimation unit 232, based on the estimation result by the first estimation unit 232 and the position information of the person. The position information of the person is, for example, position information of the person acquired by a positioning system such as a Local Positioning System (LPS) provided in the facility 100.

[0071] For example, when the first estimation unit 232 estimates that a display unit such as an LCD display present in the detection area of ​​the infrared sensor 3 is operating, and the location information indicates that a person is present in the detection area of ​​the infrared sensor 3, the second estimation unit 233 estimates that the person present in the detection area of ​​the infrared sensor 3 is using a personal computer.

[0072] That is, the second estimation unit 233 according to the second embodiment estimates a motion of a person related to the estimation result by the first estimation unit 232, based on the estimation result by the first estimation unit 232 and at least one of the detection result of a human detection unit (detection unit 4) that detects a person, the position information of the person, and the captured image by the visible light camera (detection unit 4). By using the estimation result by the first estimation unit 232 and the detection result of the detection unit 4 other than the infrared sensor 3 or the position information of the person, it is possible to improve the estimation accuracy of the estimation by the second estimation unit 233. In addition, it is possible to reduce the processing load on the first estimation unit 232.

[0073] (Embodiment 3) As shown in FIG. 5, an estimation system 1B according to the third embodiment differs from the estimation system 1 according to the first embodiment and the estimation system 1A according to the second embodiment in that the estimation system 1B includes a first infrared sensor 3A and a second infrared sensor 3B.

[0074] The first infrared sensor 3A is, for example, the infrared sensor 3 of the first embodiment. The second infrared sensor 3B is disposed at a position different from the first infrared sensor 3A. The second infrared sensor 3B is installed so that an object present in the detection area of ​​the first infrared sensor 3A is included in the detection area of ​​the second infrared sensor 3B. In other words, the first infrared sensor 3A and the second infrared sensor 3B are installed so that an object at a predetermined position is included in the detection area from different angles at the same time. In other words, an object present in the detection area of ​​the first infrared sensor 3A is also present in the detection area of ​​the second infrared sensor 3B. Note that the second infrared sensor 3B of the third embodiment is an infrared camera, like the infrared sensor 3.

[0075] The first estimation unit 232 estimates objects that become heat sources existing in both the first detection area of ​​the first infrared sensor 3A and the second detection area of ​​the second infrared sensor 3B based on the thermal image G1 generated by the first infrared sensor 3A and the thermal image generated by the second infrared sensor 3B. In other words, the first estimation unit 232 estimates objects that become heat sources existing in both the first detection area of ​​the first infrared sensor 3A and the second detection area of ​​the second infrared sensor 3B based on the first temperature distribution information acquired by the first infrared sensor 3A and the second temperature distribution information acquired by the second infrared sensor 3B.

[0076] The second estimation unit 233 estimates a human movement related to the estimation result by the first estimation unit 232, based on the estimation result by the first estimation unit 232 regarding an object present in both the first detection area and the second detection area.

[0077] According to the estimation system 1B of the third embodiment, the first estimation unit 232 performs the first estimation process based on two thermal images generated by two infrared sensors arranged at different positions, so that the estimation accuracy of the first estimation process can be improved. Furthermore, the estimation accuracy of the second estimation process by the second estimation unit 233 can be improved by improving the estimation accuracy of the first estimation process by the first estimation unit 232.

[0078] (summary) As is clear from the above-described embodiment, the estimation system (1; 1A; 1B) according to the first aspect includes a first estimation unit (232) and a second estimation unit (233). The first estimation unit (232) estimates an object that is a heat source present in the detection area based on temperature distribution information of the detection area acquired by the infrared sensor (3). The second estimation unit (233) estimates a human movement related to the estimation result based on the estimation result by the first estimation unit (232).

[0079] According to this embodiment, even if no person is present in the detection area of ​​the infrared sensor (3), the movement of the person relative to an object that is a heat source present in the detection area can be estimated.

[0080] In the estimation system (1; 1A; 1B) according to the second aspect, in the first aspect, the first estimation unit (232) estimates the posture of a person present in the detection area in a time series based on a plurality of pieces of temperature distribution information in a time series. The second estimation unit (233) estimates the movement of the person present in the detection area based on the estimation result of the posture of the person in a time series by the first estimation unit (232).

[0081] According to this embodiment, it is possible to improve the accuracy of the estimation performed by the second estimating section (233).

[0082] In the estimation system (1; 1A; 1B) according to the third aspect, in the first or second aspect, the first estimation unit (232) estimates the posture of a person present in the detection area and estimates an object that is a heat source present in the detection area based on temperature distribution information. The second estimation unit (233) estimates the movement of the person present in the detection area based on the estimation result of the posture of the person and the estimation result of the object that is a heat source by the first estimation unit (232).

[0083] According to this embodiment, it is possible to improve the accuracy of the estimation performed by the second estimating section (233).

[0084] In the estimation system (1B) according to the fourth aspect, in any one of the first to third aspects, there is a first infrared sensor (3A) as the infrared sensor (3) and a second infrared sensor (3B) arranged at a position different from the first infrared sensor (3A). The first estimation unit (232) performs estimation regarding an object that is a heat source existing in both the first detection area of ​​the first infrared sensor (3A) and the second detection area of ​​the second infrared sensor (3B) based on the first temperature distribution information acquired by the first infrared sensor (3A) and the second temperature distribution information acquired by the second infrared sensor (3B). The second estimation unit (233) estimates a human movement related to the estimation result based on the estimation result by the first estimation unit (232) regarding the object existing in both the first detection area and the second detection area.

[0085] According to this embodiment, the estimation accuracy of the first estimating section (232) and the estimation accuracy of the second estimating section (233) can be improved.

[0086] In the estimation system (1A) according to the fifth aspect, in any one of the first to fourth aspects, the second estimation unit (233) estimates the movement of a person related to the estimation result based on the estimation result by the first estimation unit (232) and at least one of the detection result of a human detection unit (detection unit 4) that detects a person, the position information of the person, and the captured image of the visible light camera (detection unit 4).

[0087] According to this embodiment, it is possible to improve the estimation accuracy of the second estimation unit (233) and to reduce the processing load on the first estimation unit (232).

[0088] In the estimation system (1; 1A; 1B) according to the sixth aspect, in any one of the first to fifth aspects, the first estimation unit (232) estimates the emissivity of an object that is present in the detection area and serves as a heat source. The first estimation unit (232) estimates the type of the object that is present in the detection area and serves as a heat source, based on the estimated emissivity of the object and emissivity information in which the substance and the emissivity are associated with each other. The second estimation unit (233) estimates a human movement related to the estimation result, based on at least the type of the object that is the heat source estimated by the first estimation unit (232).

[0089] According to this embodiment, the estimation accuracy of the first estimator (232) can be improved.

[0090] In the estimation system (1; 1A; 1B) according to the seventh aspect, in any one of the first to sixth aspects, the first estimation unit (232) flattens a histogram of the temperature distribution in the detection area based on the temperature distribution information, generates a thermal image of the detection area, and makes an estimation regarding an object that is a heat source present in the detection area.

[0091] According to this embodiment, it is possible to improve the accuracy of the estimation performed by the first estimating section (232).

[0092] The configurations other than those of the first aspect are not essential to the estimation system and may be omitted as appropriate.

[0093] An estimation method according to an eighth aspect is an estimation method executed by one or more processors. The estimation method includes a first estimation step and a second estimation step. In the first estimation step, an estimation is made regarding an object that is a heat source existing in the detection area based on temperature distribution information of the detection area acquired by an infrared sensor (3). In the second estimation step, a human movement related to the estimation result is estimated based on the estimation result in the first estimation step.

[0094] According to this embodiment, even if no person is present in the detection area of ​​the infrared sensor (3), the movement of the person relative to an object that is a heat source present in the detection area can be estimated.

[0095] A program according to a ninth aspect is a program for causing one or more processors to execute the estimation method according to the eighth aspect.

[0096] According to this embodiment, even if no person is present in the detection area of ​​the infrared sensor (3), the movement of the person relative to an object that is a heat source present in the detection area can be estimated. [Explanation of symbols]

[0097] 1. Estimation System 1A Estimation System 1B Estimation System 232 1st estimation part 233 Second estimation part 3. Infrared Sensor 3A 1st infrared sensor (infrared sensor) 3B Second infrared sensor

Claims

1. a first estimation unit that estimates an object that is a heat source and exists in the detection area based on temperature distribution information of the detection area acquired by the infrared sensor; a second estimation unit that estimates a motion of a person associated with an estimation result based on the estimation result by the first estimation unit; Equipped with Estimation system.

2. The first estimation unit estimates a posture of a person present in the detection area in time series based on a plurality of pieces of the temperature distribution information in time series; The second estimation unit estimates a motion of the person present in the detection area based on a result of the estimation of the posture of the person along a time series by the first estimation unit. The estimation system according to claim 1 .

3. The first estimation unit estimates a posture of a person present in the detection area and an object that serves as the heat source present in the detection area based on the temperature distribution information, The second estimation unit estimates a movement of the person present in the detection area based on the estimation result of the posture of the person by the first estimation unit and the estimation result regarding the object that serves as the heat source. The estimation system according to claim 1 .

4. The infrared sensor includes a first infrared sensor and a second infrared sensor disposed at a position different from that of the first infrared sensor, the first estimation unit estimates an object that serves as a heat source and is present in both a first detection area of ​​the first infrared sensor and a second detection area of ​​the second infrared sensor, based on first temperature distribution information acquired by the first infrared sensor and second temperature distribution information acquired by the second infrared sensor; The second estimation unit estimates a motion of a person related to an estimation result based on an estimation result by the first estimation unit regarding the object present in both the first detection area and the second detection area. The estimation system according to any one of claims 1 to 3.

5. The second estimation unit estimates a movement of the person related to the estimation result based on the estimation result by the first estimation unit, a detection result of a human detection unit that detects a person, position information of the person, and at least one of an image captured by a visible light camera. The estimation system according to any one of claims 1 to 3.

6. The first estimation unit is Estimating the emissivity of the heat source object present in the detection area; estimating a type of the object that is present in the detection area and serves as the heat source based on the estimated emissivity of the object and emissivity information in which a substance and an emissivity are associated with each other; The second estimation unit estimates a motion of the person related to the estimation result based on at least the type of the object that is the heat source estimated by the first estimation unit. The estimation system according to any one of claims 1 to 3.

7. The first estimation unit flattens a histogram of the temperature distribution in the detection area based on the temperature distribution information, generates a thermal image of the detection area, and performs estimation regarding an object that is present in the detection area and serves as the heat source. The estimation system according to any one of claims 1 to 3.

8. 1. A method of estimation executed by one or more processors, comprising: a first estimation step of estimating an object that is a heat source and exists in the detection area based on temperature distribution information of the detection area acquired by the infrared sensor; a second estimation step of estimating a human motion related to the estimation result based on the estimation result in the first estimation step; having Estimation method.

9. The method of claim 8 is performed by one or more processors, program.

Images