Human surface temperature calculation system, human surface temperature calculation method, and program

Abstract

A human body surface temperature calculation system (200) comprises: an infrared sensor (10); an acquisition unit (121) for acquiring temperature distribution data showing the temperature distribution in a target area, acquired by the infrared sensor (10); a thermal image generation unit (122) for generating a thermal image of the target area on the basis of the temperature distribution data acquired by the acquisition unit (121); an extraction unit (123) for extracting a human area indicating a person, reflected in the thermal image, by using a machine learning model (132); and a calculation unit (124) which extracts a temperature value group corresponding to the human area from the temperature distribution data, and calculates a human body surface temperature on the basis of the extracted temperature value group.

Description

Technical Field

[0001] The present invention relates to a human surface temperature calculation system, a human surface temperature calculation method, and a program.

Background Art

[0002] A method is known for estimating the amount of clothing on a clothed part of a subject reflected in a thermal image showing the temperature distribution of a target space, and estimating the user's perceived temperature based on the temperature of the clothed part and clothing information regarding the covering (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present invention provides a human surface temperature calculation system, a human surface temperature calculation method, and a program that can accurately calculate the surface temperature of a person existing in a target space.

Means for Solving the Problems

[0005] A human surface temperature calculation system according to an aspect of the present invention includes an infrared sensor, an acquisition unit that acquires temperature distribution data indicating the temperature distribution of a target space acquired by the infrared sensor, a thermal image generation unit that generates a thermal image of the target space based on the temperature distribution data acquired by the acquisition unit, an extraction unit that extracts a human region indicating a person reflected in the thermal image using a machine learning model, and a calculation unit that extracts a group of temperature values corresponding to the human region from the temperature distribution data and calculates the surface temperature of the person based on the extracted group of temperature values.

[0006] A method for calculating the human surface temperature according to one aspect of the present invention includes an acquisition step of acquiring temperature distribution data indicating the temperature distribution of a target space, a thermal image generation step of generating a thermal image of the target space based on the temperature distribution data acquired in the acquisition step, an extraction step of extracting a human region indicating a human reflected in the thermal image using a machine learning model, and a calculation step of extracting a group of temperature values corresponding to the human region from the temperature distribution data and calculating the surface temperature of the human based on the extracted group of temperature values.

[0007] A program according to one aspect of the present invention is a program for causing a computer to execute the method for calculating the human surface temperature.

Advantages of the Invention

[0008] The human surface temperature calculation system, the human surface temperature calculation method, and the program of the present invention can accurately calculate the surface temperature of a human existing in a target space.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiment for Carrying Out the Invention

[0010] Hereinafter, embodiments will be described with reference to the drawings. Note that each of the embodiments described below shows comprehensive or specific examples. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, etc. shown in the following embodiments are merely examples and are not intended to limit the present invention. In addition, among the components in the following embodiments, components not described in the independent claims are described as optional components.

[0011] Note that each figure is a schematic diagram and is not necessarily drawn precisely. Also, in each figure, substantially the same configuration is denoted by the same reference numeral, and duplicate descriptions may be omitted or simplified.

[0012] (Embodiment) [Configuration] First, the configuration of the human surface temperature calculation system according to the embodiment will be described. FIG. 1 is a block diagram showing the functional configuration of the human surface temperature calculation system according to the embodiment.

[0013] The human surface temperature calculation system 200 is a system that generates a thermal image based on the temperature distribution data of the target space acquired from the infrared sensor 10, extracts a human region showing a person reflected in the thermal image using a machine learning model, and calculates the surface temperature of the person based on the temperature value group corresponding to the human region. The target space is, for example, an office space, but may also be an indoor space in other facilities such as a space in a commercial facility or a space in a house. As shown in FIG. 1, the human surface temperature calculation system 200 includes an infrared sensor 10 and a server device 100. Note that the human surface temperature calculation system 200 may include a plurality of infrared sensors 10.

[0014] [Infrared Sensor] The infrared sensor 10 is installed, for example, on the ceiling of the target space, and acquires temperature distribution data indicating the temperature distribution when the target space is viewed from above. Note that the infrared sensor 10 may generate a thermal image based on the acquired temperature distribution data. The infrared sensor 10 may be, for example, an infrared array sensor (thermal image sensor) composed of an array of M×L infrared detection elements. In other words, the temperature distribution data acquired by the infrared sensor 10 is a matrix of temperature values of M rows × L columns (M, N: integers of 2 or more), and the thermal image generated based on the temperature distribution data has M×L pixels. The thermal image shows the temperature distribution in the sensing range of the infrared sensor 10 with a resolution of M×L.

[0015] For example, the infrared sensor 10 may be detachably connected to a power supply terminal of a lighting device installed on the ceiling of the target space. In this case, the infrared sensor 10 operates by receiving power supply from the lighting device. The power supply terminal is, for example, a USB (Universal Serial Bus) terminal. Also, the infrared sensor 10 may be directly fixed to the ceiling of the target space without passing through the lighting device. Further, the infrared sensor 10 may be fixed to a wall or the like to generate a thermal image showing the temperature distribution when the target space is viewed from the side.

[0016] [Server device] The server device 100 generates a thermal image based on the temperature distribution data acquired from the infrared sensor 10, extracts a person area indicating a person shown in the thermal image using a machine learning model, extracts a group of temperature values corresponding to the person area from the temperature distribution data, and calculates the surface temperature of the person based on the group of temperature values. The server device 100 is an edge computer provided in a facility (building) constituting the target space, but may be a cloud computer provided outside the facility. The server device 100 includes, for example, a communication unit 110, an information processing unit 120, a storage unit 130, and a learning unit 140.

[0017] The communication unit 110 is a communication module (communication circuit) for the server device 100 to communicate with the infrared sensor 10. The communication unit 110 receives, for example, temperature distribution data of the target space from the infrared sensor 10. The communication performed by the communication unit 110 may be wireless communication or wired communication. The communication standard used for communication is not particularly limited either.

[0018] The information processing unit 120 acquires the temperature distribution data received by the communication unit 110, creates a thermal image based on the acquired temperature distribution data, extracts a human region from the created thermal image, and performs information processing for calculating the surface temperature of a person based on the temperature value group corresponding to the extracted human region. Specifically, the information processing unit 120 is realized by a processor or a microcomputer. Specifically, the information processing unit 120 includes an acquisition unit 121 that acquires the temperature distribution data of the target space received by the communication unit 110, a thermal image generation unit 122 that generates a thermal image of the target region based on the acquired temperature distribution data, an extraction unit 123 that extracts a human region indicating a person reflected in the thermal image using the machine learning model 132, and a calculation unit 124 that extracts a temperature value group corresponding to the human region from the temperature distribution data and calculates the surface temperature of a person based on the extracted temperature value group. The functions of the acquisition unit 121, the thermal image generation unit 122, the extraction unit 123, and the calculation unit 124 are realized by the processor or microcomputer constituting the information processing unit 120 executing a computer program stored in the storage unit 130. Details of the functions of the acquisition unit 121, the thermal image generation unit 122, the extraction unit 123, and the calculation unit 124 will be described later.

[0019] The storage unit 130 is a storage device that stores the temperature distribution data received by the communication unit 110 and the computer program executed by the information processing unit 120, etc. The storage unit 130 also stores a database 131, a machine learning model 132, etc. The storage unit 130 may store teacher data (not shown) used for training the machine learning model 132. Specifically, the storage unit 130 is realized by a semiconductor memory or an HDD (Hard Disk Drive), etc.

[0020] The database 131 stores, for example, temperature distribution data, a group of temperature values corresponding to the human region, a group of high-temperature values corresponding to the high-temperature object region, a group of low-temperature values corresponding to the low-temperature object region, and the like.

[0021] The machine learning model 132 takes a thermal image as input and outputs a human region indicating a person shown in the thermal image. The human region is, for example, a region surrounded by the contour of a person shown in the thermal image. The machine learning model 132 may have a convolutional layer, and for example, may be a convolutional neural network (CNN), but is not limited thereto. Further, the machine learning model 132 may be composed of two or more models. For example, the machine learning model 132 may include a first machine learning model that takes a thermal image as input and outputs a super-resolution image, and a second machine learning model that takes the super-resolution image output from the first machine learning model as input and outputs a human region. The first machine learning model may be, for example, SRGAN (Generative Adversarial Network for Super-Resolurion) or SRCNN (Super-Resolution Convolutional Neural Network). The second machine learning model may be, for example, R-CNN (Region-Convolutional Neural Network), YOLO (You Only Look at Once), or SSD (Single Shot Multibox Detector). Note that the above models are merely examples and are not limited to these models. Also, the first machine learning model and the second machine learning model may employ different types of machine learning models as described above, or may employ the same type of machine learning model.

[0022] The teacher data has, as input data, a thermal image showing the temperature distribution of the target space, and, as output data, a human region showing the person reflected in the thermal image. Specifically, the teacher data may be a data set including a pair of a thermal image as input data and a human region as output data, or may include a plurality of data sets (for example, first teacher data and second teacher data). For example, the first teacher data is teacher data used for training a first machine learning model, and is, for example, a data set including a pair of a thermal image as input data and a super-resolution thermal image obtained by super-resolving the thermal image as output data. Also, for example, the second teacher data is teacher data used for training a second machine learning model, and is a data set including a pair of a super-resolution thermal image as input data and a human region as output data. Note that the machine learning model 132 may be an AI (Artificial Intelligence) model.

[0023] The learning unit 140 performs machine learning using the teacher data. The learning unit 140 generates, by machine learning, a machine learning model that takes, as input, a thermal image showing the temperature distribution of the target space and outputs, as output, a human region showing the person reflected in the thermal image. The learning unit 140 updates the machine learning model 132 by storing the learned machine learning model in the storage unit 130. The learning unit 140 is realized, for example, by a processor executing a program stored in the storage unit 130.

[0024] In FIG. 1, an example has been described in which the learning unit 140 of the server device 100 generates a learned machine learning model and updates the machine learning model 132 by storing the generated learned machine learning model in the storage unit 130, but the example is not limited to this. For example, a learned machine learning model may be generated by a cloud server provided outside the building, and the cloud server may transmit the learned machine learning model to the server device 100 to update the machine learning model 132.

[0025] [Operation Example] Next, the operation of the human surface temperature calculation system 200 will be described. FIG. 2 is a flowchart showing an operation example of the human surface temperature calculation system 200. FIG. 3 is a diagram schematically showing the flow shown in FIG. 2.

[0026] The communication unit 110 of the server device 100 receives temperature distribution data from the infrared sensor 10 (not shown). At this time, the information processing unit 120 stores the received temperature distribution data in the storage unit 130 (not shown).

[0027] Next, the acquisition unit 121 acquires the temperature distribution data received by the communication unit 110 and stored in the storage unit 130 (S01), and outputs it to the thermal image generation unit 122. The thermal image generation unit 122 generates a thermal image of the target space based on the acquired temperature distribution data (S02). For example, as shown in FIG. 3(a), the temperature distribution data acquired in step S01 is a matrix of temperature values (also referred to as temperature physical quantities). Further, as shown in FIG. 3(b), the thermal image generated in step S02 is, for example, an 8-bit image obtained by converting the temperature range from 20°C to 40°C into 0 to 255 gradations.

[0028] Next, the extraction unit 123 extracts a human region indicating a person in the thermal image using the machine learning model 132 (S03). More specifically, as shown in FIGS. 3(b) and 3(c), the extraction unit 123 performs segmentation on the thermal image using the machine learning model 132 and classification processing for classifying the region indicating the person reflected in the segmented thermal image into the human class. As a result, the extraction unit 123 can detect a person reflected in the thermal image and extract the region (the region surrounded by the contour of the detected person) indicating the detected person. The segmentation may be semantic segmentation or instance segmentation. Note that the extraction unit 123 may or may not increase the resolution of the thermal image. For example, the extraction unit 123 may super-resolve the thermal image using a first machine learning model, or may increase the resolution of the thermal image by a method of inserting a new pixel having a pixel value corresponding to the average value between adjacent pixels by obtaining the average value of the pixel values of adjacent pixels. The extraction unit 123 performs human detection processing on the thermal image using the machine learning model 132, thereby detecting a person reflected in the thermal image and extracting the region (so-called human region) surrounded by the contour of the detected person.

[0029] Next, the calculation unit 124 extracts a group of temperature values corresponding to the human region from the temperature distribution data (S04), and calculates the surface temperature of the person based on the extracted group of temperature values (S05). More specifically, as shown in FIGS. 3(d) and 3(e), the calculation unit 124 may calculate the surface temperature of the person by extracting the temperature values corresponding to the pixels in the human region in the thermal image from the matrix of the temperature distribution data and calculating the average value or the median value of the extracted group of temperature values.

[0030] Next, the calculation unit 124 outputs the calculation result (not shown). Specifically, the calculation unit 124 outputs the surface temperature of the person existing in the target space as the calculation result. The calculation unit 124 may output, as the calculation result, information indicating the coordinates of the person in addition to the surface temperature of the person. Note that the calculation result is stored in the storage unit 130.

[0031] As described above, the human surface temperature calculation system 200 can detect a person existing in the target space, calculate the surface temperature of the detected person, and output the calculation result.

[0032] The output calculation result may be provided to a control device (not shown) that controls devices such as an air conditioner. Thereby, the control device can control the device based on the surface temperature of the person existing in the target space.

[0033] [Specific Example of Calculation Step] In the above operation example, in the calculation step (S05 in FIG. 2), an example was described in which a person reflected in the thermal image is detected, a person area indicating the detected person is extracted, and the surface temperature of the person is calculated based on the temperature value group corresponding to the extracted person area. Hereinafter, a method for calculating the human surface temperature when a person and a heat source overlap in the thermal image will be specifically described.

[0034] [First Example] In the first example, a processing example when a person and one heat source overlap in the thermal image will be described. FIG. 4 is a diagram showing an example of a thermal image. FIG. 5 is a flowchart showing a first example of the detailed flow of step S05 in FIG. 2.

[0035] The calculation unit 124 detects the maximum value of the temperature value group corresponding to the person area extracted in step S04 of FIG. 2 (S11), and determines whether the detected maximum value exceeds a first temperature value (for example, 50°C) (S12). Note that the temperature value group corresponding to the person area is the temperature value corresponding to each pixel value of a plurality of pixels indicating the person area in the thermal image.

[0036] When the calculation unit 124 determines that the maximum value exceeds the first temperature value (Yes in S12), it identifies, as a high-temperature temperature value group corresponding to a high-temperature object region indicating a high-temperature object different from a person, a temperature value group equal to or higher than the temperature value obtained by subtracting a first value (for example, 10°C) from the maximum value (not shown). The high-temperature object is, for example, a warm drink such as hot coffee, food, a warmer, or the like. Next, the calculation unit 124 calculates the surface temperature of the person based on the remaining temperature value group obtained by removing the high-temperature temperature value group from the temperature value group corresponding to the person region (S13). More specifically, the calculation unit 124 may calculate the surface temperature of the person by calculating the average value or the median value of the remaining temperature value group obtained by removing the high-temperature temperature value group from the temperature value group corresponding to the person region.

[0037] On the other hand, when the calculation unit 124 determines that the maximum value does not exceed the first temperature value (No in S12), it calculates the surface temperature of the person based on the temperature value group corresponding to the person region (S14). At this time, the calculated surface temperature of the person may be the average value or the median value of the temperature value group.

[0038] As described above, when a high-temperature object region exists within the person region (that is, when a person and a high-temperature object overlap in the thermal image), the human surface temperature calculation system 200 can remove the high-temperature temperature value group corresponding to the high-temperature object region from the temperature value group corresponding to the person region. As a result, in calculating the surface temperature of the person, the human surface temperature calculation system 200 is less likely to be affected by heat sources other than the person, such as high-temperature objects existing within the person region, and thus can calculate the surface temperature of the person with higher accuracy.

[0039] [Second Example] In the second example, a processing example when a person and a low-temperature heat source overlap in the thermal image will be described. FIG. 6 is a flowchart showing a second example of the detailed flow of step S05 in FIG. 2.

[0040] The calculation unit 124 detects the minimum value of the temperature value group corresponding to the person region extracted in step S04 of FIG. 2 (S21), and determines whether or not the detected minimum value is lower than a second temperature value (for example, 10°C) (S22).

[0041] When the calculation unit 124 determines that the minimum value is lower than the second temperature value (Yes in S22), it identifies, as a low-temperature temperature value group corresponding to a low-temperature object region indicating a low-temperature object different from a person, a temperature value group equal to or lower than the temperature value obtained by adding a second value (for example, 20°C) to the minimum value (not shown). The low-temperature object is, for example, a cold drink such as iced coffee, a cold food such as ice cream, a cold storage agent, or the like. Next, the calculation unit 124 calculates the surface temperature of a person based on the remaining temperature value group obtained by removing the low-temperature temperature value group from the temperature value group corresponding to the person region (S23). More specifically, the calculation unit 124 may calculate the surface temperature of a person by calculating the average value or the median value of the remaining temperature value group obtained by removing the low-temperature temperature value group from the temperature value group corresponding to the person region.

[0042] On the other hand, when the calculation unit 124 determines that the minimum value is not lower than the second temperature value (No in S22), it calculates the surface temperature of a person based on the temperature value group corresponding to the person region (S24). At this time, the calculated surface temperature of a person may be the average value or the median value of the temperature value group.

[0043] As described above, when there is a low-temperature object region within the person region (that is, when a person and a low-temperature object overlap in the thermal image), the human surface temperature calculation system 200 can remove the low-temperature temperature value group corresponding to the low-temperature object region from the temperature value group corresponding to the person region. Thereby, in calculating the surface temperature of a person, the human surface temperature calculation system 200 is less likely to be affected by heat sources other than a person, such as low-temperature objects existing within the person region, and thus can calculate the surface temperature of a person with higher accuracy.

[0044] Note that the first example and the second example of the calculation process may be executed in parallel. FIG. 7 is a diagram showing another example of a thermal image. FIG. 7 shows an example in which a person, a high-temperature heat source, and a low-temperature heat source overlap in the thermal image.

[0045] For example, the calculation unit 124 detects the maximum value and the minimum value among the temperature value group corresponding to the person region, and determines whether the maximum value exceeds the first temperature value and whether the minimum value is lower than the second temperature value.

[0046] Next, when the maximum value exceeds the first temperature value and the minimum value is lower than the second temperature value, the calculation unit 124 calculates the surface temperature of a person based on the remaining temperature value group obtained by removing the high-temperature temperature value group and the low-temperature temperature value group from the temperature value group corresponding to the person area.

[0047] Note that when the maximum value exceeds the first temperature value and the minimum value is not lower than the second temperature value, the calculation unit 124 calculates the surface temperature of a person based on the remaining temperature value group obtained by removing the high-temperature temperature value group from the temperature value group corresponding to the person area.

[0048] Note that when the maximum value does not exceed the first temperature value and the minimum value is lower than the second temperature value, the calculation unit 124 calculates the surface temperature of a person based on the remaining temperature value group obtained by removing the low-temperature temperature value group from the temperature value group corresponding to the person area.

[0049] As described above, when there are a high-temperature object area and a low-temperature object area within the person area (that is, when a person, a high-temperature object, and a low-temperature object overlap in the thermal image), the human surface temperature calculation system 200 can remove the high-temperature temperature value group and the low-temperature temperature value group from the temperature value group corresponding to the person area. Thereby, the human surface temperature calculation system 200 is less likely to be affected by heat sources other than the person, such as high-temperature objects or low-temperature objects existing within the person area, when calculating the surface temperature of the person, and thus can calculate the surface temperature of the person with higher accuracy.

[0050] [Example 3] In the first example, a processing example when a person and one high-temperature heat source overlap in the thermal image was described. In the third example, a processing example when a person and a plurality of (for example, two) high-temperature heat sources overlap in the thermal image will be described. FIG. 8 is a flowchart showing a third example of the detailed flow of step S05 in FIG. 2.

[0051] The calculation unit 124 detects the maximum value and the second largest temperature value of the temperature value group corresponding to the human region extracted in step S04 of FIG. 2 (S31). The calculation unit 124 determines whether the maximum value exceeds the first temperature value (for example, 50°C) (S32). If it is determined that the maximum value does not exceed the first temperature value (No in S32), the calculation unit 124 calculates the surface temperature of the person based on the temperature value group corresponding to the human region (S33). On the other hand, when the calculation unit 124 determines that the maximum value exceeds the first temperature value (Yes in S32), the calculation unit 124 determines whether the second largest temperature value after the maximum value exceeds the first temperature value (S34).

[0052] When the calculation unit 124 determines that the second largest temperature value after the maximum value does not exceed the first temperature value (No in S34), the calculation unit 124 identifies, as a high-temperature temperature value group (not shown), the temperature value group that is equal to or higher than the temperature value obtained by subtracting the first value (for example, 10°C) from the maximum value, and calculates the surface temperature of the person based on the remaining temperature value group obtained by removing the high-temperature temperature value group from the temperature value group (S36). On the other hand, when the calculation unit 124 determines that the second largest temperature value after the maximum value exceeds the first temperature value (Yes in S34), the calculation unit 124 determines whether the pixel indicating the maximum value and the pixel indicating the second largest temperature value in the thermal image are located at a predetermined distance apart (S35). The predetermined distance may be, for example, 10% or more of the number of horizontal pixels of the thermal image and / or 10% or more of the number of vertical pixels of the thermal image. For example, the predetermined distance will be specifically described with reference to FIG. 7. Since the above-mentioned predetermined distance changes according to the distance between the infrared sensor and the heat source, it may be a distance exceeding the size (for example, width in the width direction and height in the vertical direction) of each of the high-temperature heat sources (for example, warm drinks, etc.) and low-temperature heat sources (for example, cold canned drinks, etc.) on the thermal image.

[0053] When the calculation unit 124 determines that the pixel indicating the maximum value and the pixel indicating the temperature value next largest to the maximum value in the thermal image are not located at a predetermined distance from each other (in other words, are not separated by a predetermined distance) (No in S35), it identifies, as a high-temperature temperature value group (not shown), a group of temperature values equal to or higher than the temperature value obtained by subtracting a first value (for example, 10°C) from the maximum value, and calculates the surface temperature of a person based on the remaining temperature value group obtained by excluding the high-temperature temperature value group from the temperature value group (S36). On the other hand, when the calculation unit 124 determines that the pixel indicating the maximum value and the pixel indicating the temperature value next largest to the maximum value in the image are located at a predetermined distance from each other (in other words, are separated by a predetermined distance) (Yes in S35), it identifies, as a high-temperature temperature value group 1 (not shown), a group of temperature values equal to or higher than the temperature value obtained by subtracting a first value (for example, 10°C) from the maximum value, and further identifies, as a high-temperature temperature value group 2 (not shown), a group of temperature values equal to or higher than the temperature value obtained by subtracting the first value from the temperature value next largest to the maximum value, and calculates the surface temperature of a person based on the remaining temperature value group obtained by excluding the high-temperature temperature value group 1 and the high-temperature temperature value group 2 from the temperature value group (S37).

[0054] As described above, when there are a plurality of (here, two) high-temperature object regions within the person region (that is, when a plurality of high-temperature objects overlap in the thermal image), the human surface temperature calculation system 200 can identify the high-temperature temperature value groups of each of the plurality of high-temperature object regions and exclude these plurality of high-temperature temperature value groups from the temperature value group corresponding to the person region. Thereby, even when there are a plurality of heat sources different from a person in the thermal image, the human surface temperature calculation system 200 is less likely to be affected by those heat sources in calculating the surface temperature of the person, and thus can accurately calculate the surface temperature of the person.

[0055] [Fourth Example] In the second example, a processing example in the case where a person and one low-temperature heat source overlap in the thermal image was described. In the fourth example, a processing example in the case where a person and a plurality of (for example, two) low-temperature heat sources overlap in the thermal image will be described. FIG. 9 is a flowchart showing a fourth example of the detailed flow of step S05 in FIG. 2.

[0056] The calculation unit 124 detects the minimum value and the second smallest temperature value of the temperature value group corresponding to the human region extracted in step S04 of FIG. 2 (S41). The calculation unit 124 determines whether the minimum value is lower than the second temperature value (for example, 10°C) (S42). If it is determined that the minimum value is not lower than the second temperature value (No in S42), the calculation unit 124 calculates the surface temperature of the human based on the temperature value group corresponding to the human region (S43). On the other hand, when it is determined that the minimum value is lower than the second temperature value (Yes in S42), the calculation unit 124 determines whether the second smallest temperature value is lower than the second temperature value (S44).

[0057] When it is determined that the second smallest temperature value is not lower than the second temperature value (No in S44), the calculation unit 124 identifies, as a low-temperature temperature value group (not shown), the temperature value group that is equal to or lower than the temperature value obtained by adding a second value (for example, 20°C) to the minimum value, and calculates the surface temperature of the human based on the remaining temperature value group obtained by removing the low-temperature temperature value group from the temperature value group (S46). On the other hand, when it is determined that the second smallest temperature value is lower than the second temperature value (Yes in S44), the calculation unit 124 determines whether the pixel indicating the minimum value and the pixel indicating the second smallest temperature value in the thermal image are located at a predetermined distance apart (S45). Note that since the predetermined distance has been described above, the description here is omitted.

[0058] When the calculation unit 124 determines that the pixel indicating the minimum value and the pixel indicating the temperature value next smaller than the minimum value in the thermal image are not located at a predetermined distance from each other (in other words, are not separated by a predetermined distance) (No in S45), it identifies, as a low-temperature temperature value group (not shown), a group of temperature values equal to or lower than the temperature value obtained by adding a second value (for example, 20°C) to the minimum value, and calculates the surface temperature of a person based on the remaining temperature value group obtained by removing the low-temperature temperature value group from the temperature value group (S46). On the other hand, when the calculation unit 124 determines that the pixel indicating the minimum value and the pixel indicating the temperature value next smaller than the minimum value in the image are located at a predetermined distance from each other (in other words, are separated by a predetermined distance) (Yes in S45), it identifies, as a low-temperature temperature value group 1 (not shown), a group of temperature values equal to or lower than the temperature value obtained by adding a second value (for example, 20°C) to the minimum value, and further identifies, as a low-temperature temperature value group 2 (not shown), a group of temperature values equal to or lower than the temperature value obtained by adding the second value to the temperature value next smaller than the minimum value, and calculates the surface temperature of a person based on the remaining temperature value group obtained by removing the low-temperature temperature value group 1 and the low-temperature temperature value group 2 from the temperature value group (S47).

[0059] As described above, when there are a plurality of (here, two) low-temperature object regions within the human region (that is, when a plurality of low-temperature objects overlap in the thermal image), the human surface temperature calculation system 200 can identify the low-temperature temperature value groups of the respective plurality of low-temperature object regions and remove these plurality of low-temperature temperature value groups from the temperature value group corresponding to the human region. Thereby, even when there are a plurality of heat sources different from a person in the thermal image, the human surface temperature calculation system 200 is less likely to be affected by those heat sources in calculating the surface temperature of a person, and thus can accurately calculate the surface temperature of a person.

[0060] Note that the third and fourth examples of the calculation process may be executed in parallel.

[0061] For example, the calculation unit 124 detects the maximum value, the second largest temperature value, the minimum value, and the second smallest temperature value among the temperature value group corresponding to the human region. Next, the calculation unit 124 determines whether each of the maximum value and the second largest temperature value exceeds a first temperature value, and whether each of the minimum value and the second smallest temperature value is lower than a second temperature value.

[0062] Based on the determination result, the calculation unit 124 calculates the surface temperature of a person according to the flow shown in FIGS. 8 and 9.

[0063] As described above, even if there are one or more high-temperature object regions and one or more low-temperature object regions in the human region, the human surface temperature calculation system 200 can exclude the high-temperature temperature value group and the low-temperature temperature value group from the temperature value group corresponding to the human region. Thereby, even when there are a plurality of high-temperature and low-temperature heat sources in the thermal image, the human surface temperature calculation system 200 is less likely to be affected by heat sources other than humans, such as high-temperature objects or low-temperature objects existing in the human region, in calculating the surface temperature of a person. Therefore, the surface temperature of a person can be accurately calculated.

[0064] [Effects, etc.] As described above, the human surface temperature calculation system 200 includes an infrared sensor 10, an acquisition unit 121 that acquires temperature distribution data indicating the temperature distribution of the target space acquired by the infrared sensor 10, a thermal image generation unit 122 that generates a thermal image of the target space based on the temperature distribution data acquired by the acquisition unit 121, an extraction unit 123 that extracts a human region indicating a person shown in the thermal image using a machine learning model 132, and a calculation unit 124 that extracts a temperature value group corresponding to the human region from the temperature distribution data and calculates the surface temperature of a person based on the extracted temperature value group.

[0065] Such a human surface temperature calculation system 200 can accurately calculate the surface temperature of a person existing in the target space.

[0066] Further, for example, when the maximum value of the temperature value group exceeds a first temperature value (for example, 50°C), the calculation unit 124 sets the temperature value group equal to or higher than the temperature value obtained by subtracting a first value (for example, 10°C) from the maximum value as the high-temperature temperature value group corresponding to the high-temperature object region indicating an object different from a person, and calculates the surface temperature of a person based on the remaining temperature value group obtained by excluding the high-temperature temperature value group from the temperature value group.

[0067] Such a human surface temperature calculation system 200 can exclude a high-temperature temperature value group from a group of temperature values even when a human and a high-temperature heat source other than a human (so-called high-temperature object) overlap in a thermal image. Therefore, in calculating the surface temperature of a human, the human surface temperature calculation system 200 is less likely to be affected by heat sources other than a human, such as high-temperature objects existing within the human region, and can calculate the surface temperature of a human with higher accuracy.

[0068] Also, for example, when the maximum value and the temperature value next largest to the maximum value exceed the first temperature value, and in the thermal image, the pixel indicating the maximum value and the pixel indicating the temperature value next largest to the maximum value are located at a predetermined distance apart, the calculation unit 124 further includes in the high-temperature temperature value group a group of temperature values equal to or higher than the temperature value obtained by subtracting a first value from the temperature value next largest to the maximum value.

[0069] Such a human surface temperature calculation system 200 can exclude a high-temperature temperature value group from a group of temperature values even when a human and a plurality of (for example, two) high-temperature heat sources overlap in a thermal image. Therefore, in calculating the surface temperature of a human, the human surface temperature calculation system 200 is less likely to be affected by those heat sources, and can calculate the surface temperature of a human with high accuracy.

[0070] Also, for example, when the minimum value of the group of temperature values is lower than a second temperature value (for example, 10°C), the calculation unit 124 sets a group of temperature values equal to or lower than the temperature value obtained by adding a second value (for example, 20°C) to the minimum value as a low-temperature temperature value group corresponding to a low-temperature object region indicating a low-temperature object different from a human, and calculates the surface temperature of a human based on the remaining group of temperature values obtained by excluding the low-temperature temperature value group from the group of temperature values.

[0071] Such a human surface temperature calculation system 200 can exclude a low-temperature temperature value group from a group of temperature values even when a human and a low-temperature heat source other than a human (so-called low-temperature object) overlap in a thermal image. Therefore, in calculating the surface temperature of a human, the human surface temperature calculation system 200 is less likely to be affected by heat sources other than a human, such as low-temperature objects existing within the human region, and can calculate the surface temperature of a human with higher accuracy.

[0072] Further, for example, when the minimum value and the temperature value next smaller than the minimum value are lower than the second temperature value, and in the thermal image, the pixel indicating the minimum value and the pixel indicating the temperature value next smaller than the minimum value are located at a predetermined distance apart, the calculation unit 124 further includes in the low-temperature temperature value group a temperature value group having a temperature equal to or lower than the temperature value obtained by adding a second value (for example, 20°C) to the temperature value next smaller than the minimum value.

[0073] Such a human surface temperature calculation system 200 can exclude the low-temperature temperature value group from the temperature value group even when a human and a plurality of (for example, two) low-temperature heat sources overlap in the thermal image. Therefore, in calculating the human surface temperature, it is less likely to be affected by those heat sources, and the human surface temperature can be calculated accurately.

[0074] Further, a human surface temperature calculation method executed by a computer such as the human surface temperature calculation system 200 includes an acquisition step of acquiring temperature distribution data indicating the temperature distribution of a target space, a thermal image generation step of generating a thermal image of the target space based on the temperature distribution data acquired in the acquisition step, an extraction step of extracting a human region indicating a human reflected in the thermal image using a machine learning model, and a calculation step of extracting a temperature value group corresponding to the human region from the temperature distribution data and calculating the human surface temperature based on the extracted temperature value group.

[0075] Such a human surface temperature calculation method can accurately calculate the human surface temperature of a human existing in the target space.

[0076] (Other Embodiments) As described above, the human surface temperature calculation system and the human surface temperature calculation method according to the embodiments have been described. However, the present invention is not limited to the above embodiments.

[0077] For example, in the above embodiment, as an example where a plurality of high-temperature object regions exist within the human region, an example where two high-temperature object regions exist was shown, but it is not limited to this example. For example, the plurality of high-temperature object regions existing within the human region may be three or more. Also, in the above embodiment, as an example where a plurality of low-temperature object regions exist within the human region, an example where two low-temperature object regions exist was shown, but it is not limited to this example. For example, the plurality of low-temperature object regions existing within the human region may be three or more. In addition, when one or more high-temperature object regions and one or more low-temperature object regions exist within the human region, the surface temperature of a person may also be calculated by appropriately combining the processes exemplified in the above embodiment.

[0078] Also, in the above embodiment, the human surface temperature calculation system was realized by a plurality of devices, but it may also be realized as a single device. For example, the human surface temperature calculation system may be realized as a single device corresponding to a server device. When the human surface temperature calculation system is realized by a plurality of devices, each component included in the human surface temperature calculation system may be distributed among the plurality of devices in any manner.

[0079] Also, in the above embodiment, the process executed by a specific processing unit may be executed by another processing unit. Also, the order of a plurality of processes may be changed, or a plurality of processes may be executed in parallel.

[0080] Also, in the above embodiment, each component may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

[0081] Also, each component may be implemented by hardware. For example, each component may be a circuit (or an integrated circuit). These circuits may form one circuit as a whole or may be separate circuits respectively. Also, each of these circuits may be a general-purpose circuit or a dedicated circuit respectively.

[0082] Furthermore, the general or specific aspects of the present invention may be implemented in a system, apparatus, method, integrated circuit, computer program, or a recording medium such as a computer-readable CD-ROM. Also, it may be implemented by any combination of a system, apparatus, method, integrated circuit, computer program, and recording medium. For example, the present invention may be implemented as a human surface temperature calculation method executed by a computer such as a human surface temperature calculation system. Also, the present invention may be implemented as a program for causing a computer to execute the human surface temperature calculation method, or may be implemented as a computer-readable non-transitory recording medium storing such a program.

[0083] In addition, forms obtained by applying various modifications conceivable by those skilled in the art to each embodiment, or forms realized by arbitrarily combining the components and functions in each embodiment without departing from the spirit of the present invention are also included in the present invention.

Description of Reference Numerals

[0084] 10 Infrared sensor 121 Acquisition unit 122 Thermal image generation unit 123 Extraction unit 124 Calculation unit 132 Machine learning model 200 Human surface temperature calculation system

Claims

1. An infrared sensor, an acquisition unit that acquires temperature distribution data indicating the temperature distribution of a target space obtained by the infrared sensor, a thermal image generation unit that generates a thermal image of the target space based on the temperature distribution data acquired by the acquisition unit, an extraction unit that extracts a human region indicating a person shown in the thermal image using a machine learning model, a calculation unit that extracts a group of temperature values corresponding to the human region from the temperature distribution data, and calculates the surface temperature of the person based on the remaining group of temperature values excluding the group of temperature values outside a predetermined temperature value range among the extracted group of temperature values, comprising: the calculation unit: when the maximum value of the group of temperature values exceeds a first temperature value, a group of temperature values equal to or higher than the temperature value obtained by subtracting a first value from the maximum value is defined as a high-temperature temperature value group corresponding to a high-temperature object region indicating an object at a temperature higher than that of the person, and calculates the surface temperature of the person based on the remaining group of temperature values excluding the high-temperature temperature value group from the group of temperature values, a human surface temperature calculation system.

2. the calculation unit: when the maximum value and the second largest temperature value exceed the first temperature value, and in the thermal image, a pixel indicating the maximum value and a pixel indicating the second largest temperature value are located at a predetermined distance from each other, a group of temperature values equal to or higher than the temperature value obtained by subtracting the first value from the second largest temperature value is further included in the high-temperature temperature value group, The human surface temperature calculation system according to Claim 1.

3. An infrared sensor, an acquisition unit that acquires temperature distribution data indicating the temperature distribution of a target space obtained by the infrared sensor, a thermal image generation unit that generates a thermal image of the target space based on the temperature distribution data acquired by the acquisition unit, an extraction unit that extracts a human region indicating a person shown in the thermal image using a machine learning model, a calculation unit that extracts a group of temperature values corresponding to the human region from the temperature distribution data, and calculates the surface temperature of the person based on the remaining group of temperature values excluding the group of temperature values outside a predetermined temperature value range among the extracted group of temperature values, comprising: the calculation unit: when the minimum value of the group of temperature values is lower than a second temperature value, a group of temperature values equal to or lower than the temperature value obtained by adding a second value to the minimum value is defined as a low-temperature temperature value group corresponding to a low-temperature object region indicating an object at a temperature lower than that of the person, and calculates the surface temperature of the person based on the remaining group of temperature values excluding the low-temperature temperature value group from the group of temperature values, a human surface temperature calculation system.

4. the calculation unit: When the minimum value and the temperature value next smaller than the minimum value are lower than the second temperature value, and in the thermal image, the pixel indicating the minimum value and the pixel indicating the temperature value next smaller than the minimum value are located at a predetermined distance from each other, a temperature value group having a temperature value equal to or lower than the temperature value obtained by adding the second value to the temperature value next smaller than the minimum value is further included in the low-temperature temperature value group. The human surface temperature calculation system according to claim 3.

5. An acquisition step of acquiring temperature distribution data indicating the temperature distribution of a target space; A thermal image generation step of generating a thermal image of the target space based on the temperature distribution data acquired in the acquisition step; An extraction step of extracting a human region indicating a human reflected in the thermal image using a machine learning model; An extraction step of extracting a temperature value group corresponding to the human region from the temperature distribution data, and calculating the surface temperature of the human based on the remaining temperature value group obtained by removing the temperature value group outside a predetermined temperature value range from the extracted temperature value group; including In the calculation step, When the maximum value of the temperature value group exceeds the first temperature value, a temperature value group having a temperature value equal to or higher than the temperature value obtained by subtracting the first value from the maximum value is defined as a high-temperature temperature value group corresponding to a high-temperature object region indicating an object different from the human; Calculating the surface temperature of the human based on the remaining temperature value group obtained by removing the high-temperature temperature value group from the temperature value group. Human surface temperature calculation method.

6. An acquisition step of acquiring temperature distribution data indicating the temperature distribution of a target space; A thermal image generation step of generating a thermal image of the target space based on the temperature distribution data acquired in the acquisition step; An extraction step of extracting a human region indicating a human reflected in the thermal image using a machine learning model; An extraction step of extracting a temperature value group corresponding to the human region from the temperature distribution data, and calculating the surface temperature of the human based on the remaining temperature value group obtained by removing the temperature value group outside a predetermined temperature value range from the extracted temperature value group; including In the calculation step, When the minimum value of the temperature value group is lower than the second temperature value, a temperature value group having a temperature value equal to or lower than the temperature value obtained by adding the second value to the minimum value is defined as a low-temperature temperature value group corresponding to a low-temperature object region indicating an object different from the human; Calculating the surface temperature of the human based on the remaining temperature value group obtained by removing the low-temperature temperature value group from the temperature value group. Human surface temperature calculation method.

7. For causing a computer to execute the human surface temperature calculation method according to claim 5 or 6, Program.

Images