Thermal image processing device, program, and thermal image processing method

Abstract

This thermal image processing device (100) comprises: a subject detection unit (103) that detects an area of a subject from a thermal image in which each of a plurality of pixels expresses a pixel value corresponding to a received intensity of infrared rays; an area measurement unit (104) that identifies the size of the area of the subject in the thermal image; a prior data storage unit (105) that stores data indicating a relationship in which the smaller the area on an imaging surface where infrared rays are received from an object of detection, the lower the pixel value corresponding to the received intensity of the infrared rays from the object of detection; a correction value calculation unit (106) that references the data to identify correction information for compensating for the amount of decrease, which is the amount by which the pixel value corresponding to the received intensity decreases in relation to the size of the area of the subject; and a thermal image correction unit (107) that uses the correction information to correct a pixel value in the area of the subject so as to compensate for the amount of decrease in the pixel value due to the size of the area of the subject.

Description

Thermal image processing device, program, and thermal image processing method

[0001] The present disclosure relates to a thermal image processing device, a program, and a thermal image processing method.

[0002] Demand for infrared sensors that can display images and measure temperature without touching the object being measured is increasing, and development efforts to improve the accuracy of temperature measurement using infrared sensors are underway in various places.

[0003] When measuring the temperature of an object using an infrared sensor, the accuracy of the temperature measurement may be reduced depending on the distance between the object and the sensor or the size of the object. In such cases, the technology described in Patent Document 1 calculates the angle and distance of the infrared thermography relative to the object as measurement conditions based on the three-dimensional coordinates of the object calculated by a stereo camera, and corrects the measured temperature using a correction coefficient map created in advance for each measurement condition.

[0004] JP 2007-67792 A

[0005] However, conventional technology creates a correction coefficient map based on the angle and distance to the subject, but does not take into account differences in correction due to the size of the subject reflected on the sensor surface or the distance to the subject, which reduces the accuracy of temperature measurement.

[0006] Therefore, one or more aspects of the present disclosure aim to improve the accuracy of non-contact temperature measurement.

[0007] A thermal image processing device according to a first aspect of the present disclosure is characterized by comprising: a subject detection unit that detects a subject area from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of infrared light received; a subject size determination unit that determines the size of the subject area in the thermal image; a data storage unit that stores data indicating a relationship in which the smaller the area on the imaging surface that receives infrared light from the detection target, the lower the pixel value corresponding to the intensity of infrared light received from the detection target; a correction information determination unit that refers to the data and determines correction information to compensate for the amount of decrease, which is the amount by which the pixel value corresponding to the intensity of infrared light decreases over the size of the subject area; and a thermal image correction unit that uses the correction information to correct the pixel values ​​of the area to compensate for the amount of decrease.

[0008] A thermal image processing device according to a second aspect of the present disclosure is characterized by comprising: a subject detection unit that detects a subject area from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of received infrared light; a distance determination unit that determines the distance to the subject; a data storage unit that stores data indicating a relationship in which the pixel value corresponding to the intensity of received infrared light from the detection target decreases as the distance from an imaging surface that receives infrared light from the detection target increases; a correction information determination unit that refers to the data and determines correction information to compensate for the amount by which the pixel value corresponding to the intensity of received infrared light at the distance to the subject decreases from the pixel value corresponding to the intensity of received light at a predetermined distance; and a thermal image correction unit that uses the correction information to correct the pixel values ​​of the area to compensate for the amount of decrease.

[0009] A thermal image processing device according to a third aspect of the present disclosure includes a subject detection unit that detects a subject area from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of received infrared light; a subject size determination unit that determines the size of the area in the thermal image; a first data storage unit that stores first data indicating a relationship in which the smaller the area on the imaging surface that receives infrared light from the detection target, the lower the pixel value corresponding to the intensity of received infrared light from the detection target; a first correction information determination unit that refers to the first data and determines first correction information for compensating for a first decrease amount, which is the amount by which the pixel value corresponding to the intensity of received infrared light decreases over the size of the area of ​​the subject; a second data storage unit that stores second data indicating a relationship in which the pixel value corresponding to the received light intensity of infrared light from the detection target decreases as the distance from the imaging surface increases; a second correction information determination unit that refers to the second data and determines second correction information to compensate for a second decrease amount, which is an amount by which the pixel value corresponding to the received light intensity at the distance to the subject decreases from the pixel value corresponding to the received light intensity at a predetermined distance; and a thermal image correction unit that uses the first correction information and the second correction information to correct the pixel values ​​of the region to compensate for the first decrease amount and the second decrease amount.

[0010] A program according to a first aspect of the present disclosure causes a computer to function as: a subject detection unit that detects a subject area from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of infrared light received; a subject size determination unit that determines the size of the subject area in the thermal image; a data storage unit that stores data indicating the relationship that the smaller the area on the imaging surface that receives infrared light from the detection target, the lower the pixel value corresponding to the intensity of infrared light received from the detection target; a correction information determination unit that refers to the data and determines correction information to compensate for the amount of decrease, which is the amount by which the pixel value corresponding to the intensity of infrared light decreases over the size of the subject area; and a thermal image correction unit that uses the correction information to correct the pixel values ​​of the area to compensate for the amount of decrease.

[0011] A program according to a second aspect of the present disclosure causes a computer to function as a subject detection unit that detects a subject area from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of received infrared light; a distance determination unit that determines the distance to the subject; a data storage unit that stores data indicating a relationship in which the pixel value corresponding to the intensity of received infrared light from the detection target decreases as the distance from an imaging surface that receives infrared light from the detection target increases; a correction information determination unit that refers to the data and determines correction information to compensate for the amount by which the pixel value corresponding to the intensity of received infrared light at the distance to the subject decreases from the pixel value corresponding to the intensity of received infrared light at a predetermined distance; and a thermal image correction unit that uses the correction information to correct the pixel values ​​of the area to compensate for the amount of decrease.

[0012] A program according to a third aspect of the present disclosure includes a computer including a subject detection unit that detects a subject area from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of received infrared light, a subject size determination unit that determines the size of the area in the thermal image, a first data storage unit that stores first data indicating a relationship in which the smaller the area on the imaging surface that receives infrared light from the detection target, the lower the pixel value corresponding to the intensity of received infrared light from the detection target, a first correction information determination unit that refers to the first data and determines first correction information for compensating for a first decrease amount, which is an amount by which the pixel value corresponding to the intensity of received infrared light decreases depending on the size of the area of ​​the subject, and a thermal image processing unit that determines the size of the area to the subject. a second data storage unit that stores second data indicating a relationship in which the pixel value corresponding to the received light intensity at the distance to the subject decreases as the distance from the imaging surface to the subject increases; a second correction information determination unit that refers to the second data and determines second correction information to compensate for a second decrease amount, which is the amount by which the pixel value corresponding to the received light intensity at the distance to the subject decreases from the pixel value corresponding to the received light intensity at a predetermined distance; and a thermal image correction unit that uses the first correction information and the second correction information to correct the pixel values ​​of the region to compensate for the first decrease amount and the second decrease amount.

[0013] A thermal image processing method according to a first aspect of the present disclosure is characterized in that it detects an area of ​​a subject from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of infrared light received, identifies the size of the area of ​​the subject in the thermal image, refers to data indicating a relationship in which the smaller the area on the imaging surface that receives infrared light from the detection target, the lower the pixel value corresponding to the intensity of infrared light received from the detection target, identifies correction information to compensate for the amount of decrease in the pixel value corresponding to the intensity of infrared light received within the size of the area of ​​the subject, and uses the correction information to correct the pixel values ​​of the area to compensate for the amount of decrease.

[0014] A thermal image processing method according to a second aspect of the present disclosure is characterized in that it detects an area of ​​a subject from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of received infrared light, identifies the distance to the subject, refers to data indicating a relationship in which the pixel value corresponding to the intensity of received infrared light from the detection target decreases as the distance from an imaging surface that receives infrared light from the detection target increases, identifies correction information to compensate for the amount by which the pixel value corresponding to the intensity of received infrared light at the distance to the subject decreases from the pixel value corresponding to the intensity of received infrared light at a predetermined distance, and uses the correction information to correct the pixel values ​​of the area to compensate for the amount of decrease.

[0015] A thermal image processing method according to a third aspect of the present disclosure includes: detecting an area of ​​a subject from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of infrared light received; identifying a size of the area in the thermal image; referring to first data indicating a relationship in which the pixel value corresponding to the intensity of infrared light received from the detection target decreases as the area on the imaging surface that receives infrared light from the detection target becomes smaller; identifying first correction information for compensating for a first decrease amount, which is the amount by which the pixel value corresponding to the intensity of infrared light received from the detection target decreases in the size of the area of ​​the subject; identifying a distance to the subject; referring to second data indicating a relationship in which the pixel value corresponding to the intensity of infrared light received from the detection target decreases as the distance from the imaging surface to the detection target becomes longer; identifying second correction information for compensating for a second decrease amount, which is the amount by which the pixel value corresponding to the intensity of infrared light received at the distance to the subject decreases from the pixel value corresponding to the intensity of infrared light received at a predetermined distance; and correcting the pixel values ​​of the area using the first correction information and the second correction information to compensate for the first decrease amount and the second decrease amount.

[0016] According to one or more aspects of the present disclosure, it is possible to improve the accuracy of non-contact temperature measurement.

[0017] 1 is a block diagram illustrating a schematic configuration of a thermal image processing device according to a first embodiment. (A) and (B) are schematic diagrams illustrating differences in temperature accuracy depending on the area. (A) and (B) are schematic diagrams illustrating temperature changes depending on the pixel position of an object in a thermal image processing device according to the first embodiment. (A) and (B) are schematic diagrams illustrating differences in light intensity between the center of an imaging lens and its periphery. (B) is a graph illustrating the relationship between object size and pixel values ​​of a thermal image when the object temperature is 30°C. (B) is a graph illustrating the relationship between object size and pixel values ​​of a thermal image when the object temperature is 40°C. (B) is a graph illustrating the relationship between object size and pixel values ​​of a thermal image when the object temperature is 50°C. (B) is a graph illustrating the relationship between object temperature for each object size and pixel values ​​of a thermal image according to the first embodiment. (A) to (C) are schematic diagrams illustrating a correction value calculation method according to the first embodiment. (A) and (B) are block diagrams illustrating an example hardware configuration. (B) is a block diagram illustrating a schematic configuration of a thermal image processing device according to a second embodiment. (B) is a graph illustrating the relationship between object distance and pixel values ​​of a thermal image according to the second embodiment. Fig. 10 is a schematic diagram for explaining a correction value calculation method in embodiment 2. Fig. 11 is a block diagram schematically showing the configuration of a thermal image processing device according to embodiment 3.

[0018] 1 is a block diagram showing a schematic configuration of a thermal image processing device 100 according to embodiment 1. The thermal image processing device 100 includes a thermal image sensor 101, a thermal image acquisition unit 102, an object detection unit 103, an area measurement unit 104, a preliminary data storage unit 105, a correction value calculation unit 106, and a thermal image correction unit 107.

[0019] The thermal image sensor 101 captures a thermal image and sends a voltage signal VI indicating a voltage corresponding to the temperature for each pixel to the thermal image acquisition unit 102. For example, the thermal image sensor 101 is installed in an electrical appliance installed indoors, and captures images of the interior of the room, which is a space including a subject, at regular intervals while the electrical appliance is operating. Next, the thermal image sensor 101 receives infrared rays, which are electromagnetic waves with a wavelength of approximately 8 μm to 12 μm, emitted from objects present in the room, and outputs a voltage signal VI indicating a voltage corresponding to the intensity of the infrared rays for each pixel. In other words, the voltage signal VI is a signal indicating the intensity of the received infrared rays.

[0020] Specifically, the thermal image sensor 101 has an imaging surface composed of a plurality of infrared detection elements that detect infrared rays emitted from an object and output a voltage corresponding to the intensity of the infrared rays. The infrared detection elements are, for example, pyroelectric elements. The plurality of infrared detection elements are arranged in a matrix, and a single drive line is connected to the elements arranged in the row direction to supply power. The thermal image sensor 101 provides a voltage signal VI indicating the voltage output by the infrared detection elements to the thermal image acquisition unit 102. Here, the thermal image sensor 101 provides the voltage signal VI corresponding to the infrared rays to the thermal image acquisition unit 102 in an order corresponding to the arrangement of the infrared detection elements. Incidentally, when an electrical appliance is operating, the electrical appliance itself or an object around it is heated.

[0021] Here, we will explain the area of ​​the subject in the thermal image and the detected temperature. Figures 2(A) and (B) are schematic diagrams for explaining that temperature accuracy varies depending on the area. Figures 2(A) and (B) show an example in which the thermal image sensor 101 has a total of 5 x 5 pixels, and each partitioned area represents one pixel. In Figures 2(A) and (B), it is assumed that the subject is captured in the filled-in pixels.

[0022] 2A, when an object covers the entire surface of the thermal image sensor 101, for example, when focusing on the central pixel, the object reflected in the central pixel also emits infrared rays to the surrounding pixels, and the central pixel also absorbs infrared rays from the object reflected in the surrounding pixels.

[0023] On the other hand, as shown in FIG. 2B, if the subject is located in only one central pixel, the subject will only radiate infrared light to the surrounding pixels, and the amount of infrared light received by that pixel will be reduced.

[0024] Therefore, if the objects in FIGS. 2A and 2B are at the same temperature, the measured temperature in FIG. 2B will be lower.

[0025] 3A and 3B are schematic diagrams for explaining temperature changes depending on pixel positions of an object in the thermal image processing device 100 according to embodiment 1. In both of FIGS. 3A and 3B, an example is shown in which the total number of pixels in the thermal image sensor 101 is 5 x 5, and each partitioned area represents one pixel. Also in both of FIGS. 3A and 3B, it is assumed that the object is captured in the filled-in pixels.

[0026] As shown in FIG. 3A, when the object is contained within one pixel, the correct temperature can be measured as the temperature of that pixel.

[0027] However, as shown in Figure 3(B), if an image contains a subject in part of it, the correct temperature cannot be measured at that pixel. For example, pixels a2, b1, b3, and c2 contain the subject in half of their image, while pixels a1, a3, c1, and c3 contain the subject in one-quarter of their image. If only a small portion of the subject is captured in a pixel, the amount of infrared light received will be reduced, resulting in a lower measured temperature than a pixel that contains the subject in its entirety.

[0028] FIG. 4 is a schematic diagram illustrating the difference in light intensity between the center of the lens capturing an image and its periphery. FIG. 4 also shows an example in which the total number of pixels in the thermal image sensor 101 is 5×5, with each divided area representing one pixel. In FIG. 4, the filled-in circle represents the lens's angle of view. Because the thermal image sensor 101 is square and the lens is round, pixels on the periphery of the thermal image sensor 101 are vignetted by the lens, making it difficult to accurately measure the temperature. Furthermore, lens distortion causes blurring toward the periphery, reducing the accuracy of the temperature measurement. For this reason, in the first embodiment, when measuring the subject area, the number of pixels is counted excluding the pixels on the periphery of the thermal image sensor 101.

[0029] 1 , the thermal image acquisition unit 102 acquires a thermal image TI in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of received infrared light, and provides the thermal image TI to the subject detection unit 103. For example, the thermal image acquisition unit 102 generates the thermal image TI from a voltage signal VI from the thermal image sensor 101.

[0030] Specifically, the thermal image acquisition unit 102 generates a thermal image TI by sequentially arranging pixel values ​​corresponding to the voltage indicated by the voltage signal VI sent from the thermal image sensor 101. The thermal image TI is assumed to be an image captured in an exposed state of multiple areas in a room with different temperatures. The thermal image TI includes multiple pixels with different pixel values ​​at a specific pixel position from the first frame to the second frame. For example, the first frame and the second frame are adjacent frames. The frame number is a number used to represent, for example, if there are 10 images, as images with frame number 1 to frame number 10. Note that the pixel value is a value obtained by converting the voltage value of the voltage signal VI into a digital value.

[0031] Specifically, the pixel values ​​are determined by the following method. The thermal image acquisition unit 102 stores the voltage values ​​indicated by the voltage signal VI in a memory or the like (not shown). Next, the thermal image acquisition unit 102 reads out the voltage values ​​and sequence information, and converts the voltage values ​​into pre-associated pixel values. The thermal image acquisition unit 102 then arranges the pixel values ​​using the sequence information. The thermal image acquisition unit 102 generates one thermal image from the arranged pixel values ​​and stores the image in a memory or the like (not shown).

[0032] Here, the thermal image acquisition unit 102 acquires the thermal image TI based on the voltage signal VI from the thermal image sensor 101, but the first embodiment is not limited to this example. For example, the thermal image acquisition unit 102 may acquire the thermal image TI from a network such as a LAN (Local Area Network) or the Internet via a communication unit such as a communication I / F (Interface) (not shown). Furthermore, if the thermal image TI is already stored in a storage unit such as a memory (not shown), the thermal image acquisition unit 102 may acquire the thermal image TI from the storage unit. In these cases, the thermal image sensor 101 may be omitted.

[0033] The subject detection unit 103 detects the subject area from the thermal image TI, and provides subject information LP indicating a label indicating the subject and the coordinate position of the subject area together with the thermal image TI to the area measurement unit 104. The subject detection unit 103 also provides a labeled thermal image LTI, in which the label indicating the subject and the coordinate position of the subject area are attached to the thermal image TI, to the thermal image correction unit 107.

[0034] For example, the subject detection unit 103 detects the location of a subject in the thermal image TI. The subject is a heat source with 10 or more pixels that shows a temperature higher than the shutter surface temperature after shutter calibration. If there are multiple subjects within the field of view, the subject detection unit 103 labels each subject and records the pixel positions of the labeled pixels. The subject detection unit 103 then sends subject information LP indicating the pixel positions for each label to the area measurement unit 104. The label is subject identification information for identifying the subject.

[0035] The area measurement unit 104 is a subject size determination unit that determines the size of the subject area in the thermal image TI. Here, the area measurement unit 104 measures the area of ​​the subject from the thermal image TI and provides area data SA indicating the measured area to the correction value calculation unit 106.

[0036] For example, the area measurement unit 104 counts the number of pixels for each label sent from the subject detection unit 103. Here, if only a portion of a pixel receives light, the area measurement unit 104 does not count the pixel as a pixel containing an object. Specifically, if the luminance value is lower than a predetermined threshold, the area measurement unit 104 does not count the pixel. Then, the area measurement unit 104 provides the label and the pixel count number corresponding to the label as one piece of area data SA to the correction value calculation unit 106. When forming one piece of area data SA, the area measurement unit 104 may indicate a number indicating the label at the beginning, followed by the pixel count number and pixel position.

[0037] The preliminary data storage unit 105 is a data storage unit that stores preliminary data, which is data indicating the relationship that the smaller the area on the imaging surface that receives infrared light from a detection target, the object whose temperature is to be detected, the lower the pixel value corresponding to the intensity of infrared light received from the detection target. For example, the preliminary data indicates a function or graph that represents the relationship between the pixel value corresponding to the intensity of infrared light received from the detection target and the temperature of the detection target, for each ratio of the area that receives infrared light from the detection target to the size of the imaging surface. Here, the preliminary data indicates the relationship between the area ratio, which is the ratio of the area in which the subject is captured to the area of ​​the imaging surface, and the pixel value and temperature of the thermal image TI at that area ratio.

[0038] The preliminary data will now be described. FIG. 5 is a graph showing the relationship between the object size and the pixel value (brightness value, here) of the thermal image TI when the object temperature is 30°C. The vertical axis of the graph shown in FIG. 5 represents the brightness value of the thermal image TI, and the horizontal axis represents the area ratio between the imaging surface of the thermal image sensor 101 and the object captured on the imaging surface of the thermal image sensor 101. The area ratio is set to 100% when the object is captured on the entire surface of the thermal image sensor 101. FIG. 6 is a graph showing the relationship between the object size and the pixel value (brightness value, here) of the thermal image TI when the object temperature is 40°C. FIG. 7 is a graph showing the relationship between the object size and the pixel value (brightness value, here) of the thermal image TI when the object temperature is 50°C.

[0039] 5 to 7, the brightness values ​​on the vertical axis differ due to differences in the temperature of the subject, but the scale width is the same for all graphs. In all temperature graphs, the brightness values ​​tend to decrease as the area ratio decreases. However, the magnitude of the decrease differs for each graph.

[0040] FIG. 8 is a graph showing the relationship between the object temperature for each object size and the pixel value (here, brightness value) of the thermal image TI according to the first embodiment. The vertical axis of the graph shown in FIG. 8 represents the brightness value of the thermal image TI, and the horizontal axis represents the object temperature. In FIG. 8, a graph is shown for each area ratio. The prior data storage unit 105 stores a function or graph for each area ratio as prior data. Here, the function is a linear function, but it may also be a quadratic or cubic function. The prior data is assumed to be obtained in advance by experiment or the like.

[0041] 1 , the correction value calculation unit 106 is a correction information specification unit that refers to the prior data and specifies correction information to compensate for the amount of decrease in pixel values ​​corresponding to the received light intensity depending on the size of the area of ​​the subject. For example, the correction value calculation unit 106 specifies, as correction information, a correction value for matching pixel values ​​of a function or graph corresponding to the ratio of the size of the area of ​​the subject to the size of the thermal image TI with pixel values ​​of the corresponding temperature of the function or graph when infrared rays from the detection target are received over the entire imaging surface.

[0042] In other words, the correction value calculation unit 106 specifies, as correction information, a correction value for matching pixel values ​​at each object temperature in a function or graph corresponding to the ratio of the size of the object area to the size of the thermal image TI with pixel values ​​at each object temperature in a function or graph obtained when infrared rays from the detection target are received over the entire imaging surface. As an example, the graph corresponding to the ratio of the size of the object area is shown in FIG. 8 when the area ratio between the imaging surface of the thermal image sensor and the object captured on the imaging surface of the thermal image sensor is 5%, 25%, 50%, or 75%. Furthermore, the graph obtained when infrared rays from the detection target are received over the entire imaging surface is shown in FIG. 8 when the area ratio is 100%.

[0043] Here, the correction value calculation unit 106 refers to the preliminary data AD stored in the preliminary data storage unit 105, calculates a correction value CC for correcting the pixel value of the thermal image TI from the subject size, and provides the correction value CC to the thermal image correction unit 107 as correction information.

[0044] For example, the correction value calculation unit 106 receives the label number and the count number from the area measurement unit 104, and calculates, for each label number, the area ratio between the imaging surface of the thermal image sensor 101 and the object captured on the imaging surface of the thermal image sensor 101 as the object size from the count number. Then, the correction value calculation unit 106 calculates, for each label number, a correction value for matching the function of the calculated area ratio in the advance data AD with the function when the area ratio is 100%.

[0045] 2A and 2B, the temperature can be measured more accurately when the object is reflected on the entire surface of the thermal image sensor 101. Therefore, when the object size is small, the gain and offset are calculated as correction values ​​to bring the image closer to the graph when the area ratio is 100% shown in FIG.

[0046] 9A to 9C are schematic diagrams illustrating a correction value calculation method according to the first embodiment. Here, it is assumed that the function for each area ratio is a linear function. Therefore, as shown in FIG. 9A, the correction value calculation unit 106 calculates a slope correction value, as a gain, for correcting the slope of the linear function. In the example shown in FIG. 9A, the gain is a value obtained by dividing the slope of the graph for an area ratio of 100% by the slope of the graph for an area ratio of 5%. Also, as shown in FIG. 9B, the correction value calculation unit 106 calculates an intercept correction value, as an offset, for correcting the intercept of the linear function. In the example shown in FIG. 9B, the offset is a value obtained by subtracting the intercept of the graph for an area ratio of 5% from the intercept of the graph for an area ratio of 100%. Then, as shown in FIG. 9C, the correction value calculation unit 106 can correct the identified area ratio function to a function for an area ratio of 100% using the gain and offset. Even when the function is not a linear function but a quadratic or cubic function, similar correction is possible by adjusting the gain and offset.

[0047] The correction value calculation unit 106 provides the thermal image correction unit 107 with a correction value CC indicating the gain and offset for each label number.

[0048] The thermal image correction unit 107 corrects the pixel values ​​of the subject area so as to compensate for the decrease in pixel values ​​of the thermal image TI, using the correction information identified by the correction value calculation unit 106. Here, the thermal image correction unit 107 multiplies the pixel values ​​by a gain and adds an offset.

[0049] For example, the thermal image correction unit 107 corrects the labeled thermal image LTI sent from the object detection unit 103 using the correction value CC sent from the correction value calculation unit 106. Specifically, the thermal image correction unit 107 acquires the pixel value z of the object coordinates sent from the object detection unit 103. The thermal image correction unit 107 then checks the label of the pixel value z and calculates a corrected pixel value z# using the correction value CC corresponding to the label. The thermal image correction unit 107 replaces the pixel value z with the corrected pixel value z# to generate a corrected thermal image TO from the thermal image TI. The corrected thermal image TO is output to the output destination.

[0050] As described above, according to the first embodiment, correction is possible regardless of the cost of the sensor, and therefore, the temperature accuracy can be improved even with a low-cost sensor.

[0051] As shown in FIG. 10A , some or all of the above-described thermal image acquisition unit 102, object detection unit 103, area measurement unit 104, correction value calculation unit 106, and thermal image correction unit 107 can be configured with a memory 10 and a processor 11 such as a CPU (Central Processing Unit) that executes a program stored in the memory 10. Such a program may be provided via a network or may be provided by being recorded on a recording medium. That is, such a program may be provided as, for example, a computer program product.

[0052] 10B , the thermal image acquisition unit 102, the object detection unit 103, the area measurement unit 104, the correction value calculation unit 106, and the thermal image correction unit 107 may be partly or entirely configured as a processing circuit 12 such as a single circuit, a composite circuit, a programmable processor, a programmable parallel processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array). As described above, the thermal image acquisition unit 102, the object detection unit 103, the area measurement unit 104, the correction value calculation unit 106, and the thermal image correction unit 107 may be realized by a processing circuit network.

[0053] The advance data storage unit 105 can be realized by a storage such as a hard disk drive (HDD), a solid state drive (SSE), a volatile memory, or a non-volatile memory.

[0054] 11 is a block diagram showing a schematic configuration of a thermal image processing device 200 according to embodiment 2. The thermal image processing device 200 includes a thermal image sensor 101, a thermal image acquisition unit 102, a subject detection unit 103, a preliminary data storage unit 205, a correction value calculation unit 206, a thermal image correction unit 207, a distance sensor 210, and a distance acquisition unit 211.

[0055] The thermal image sensor 101, the thermal image acquisition unit 102, and the object detection unit 103 of the thermal image processing device 200 according to the second embodiment are the same as the thermal image sensor 101, the thermal image acquisition unit 102, and the object detection unit 103 of the thermal image processing device 100 according to the first embodiment. However, the object detection unit 103 according to the second embodiment provides object information LP, which indicates a label indicating an object detected from the thermal image TI and the coordinate position of the object, to the distance acquisition unit 211 together with the thermal image TI, and provides a labeled thermal image LTI, in which the label indicating the object and the coordinate position of the object are attached to the thermal image TI, to the thermal image correction unit 207.

[0056] The thermal image processing device 100 of the first embodiment uses a thermal image sensor 101. An infrared sensor is used as the thermal image sensor 101, but when measuring the temperature of an object using an infrared sensor, the accuracy of the measured temperature also decreases depending on the distance between the object and the sensor. In the second embodiment, to address this issue, the distance to the object is identified and corrected, thereby suppressing the decrease in temperature detection accuracy.

[0057] The distance sensor 210 measures the distance of an object included in the field of view of the thermal image TI. The distance sensor 210 may be any sensor capable of measuring distance, such as a laser distance sensor or an ultrasonic sensor.

[0058] The distance acquisition unit 211 is a distance determination unit that determines the distance to the subject. Here, the distance acquisition unit 211 measures the distance to the subject for each label indicated in the subject information LP sent from the subject detection unit 103. The distance is set to the average distance of each pixel of the subject. The distance acquisition unit 211 sends distance data DA indicating the distance for each label to the correction value calculation unit 206.

[0059] For example, the distance acquisition unit 211 can determine the distance for each subject by calibrating the thermal image sensor 101 and the distance sensor 210 in advance and measuring the distance with the distance sensor 210 to the coordinate position of the subject for each label indicated in the subject information LP.

[0060] In addition, when the distance to the subject is predetermined according to the coordinate position of the thermal image TI, for example, when the subject is required to be at a predetermined position in the field of view captured by the thermal image sensor 101, the distance acquisition unit 211 can identify the distance to the subject from the coordinate position of the image. In such a case, the distance sensor 210 is not required. Specifically, when measuring body temperature, a subject may be required to stand at a mark in front of the thermal image sensor 101, or an animal may be examined at a fixed location.

[0061] The prior data storage unit 205 is a data storage unit that stores, as prior data, data indicating a relationship in which the pixel value corresponding to the intensity of infrared light received from the detection target decreases as the distance from the imaging surface that receives infrared light from the detection target increases. For example, the prior data indicates a function or graph that represents the pixel value corresponding to the intensity of infrared light received from the detection target for each length of distance from the imaging surface to the detection target. The prior data is assumed to be obtained in advance through experiments or the like.

[0062] Fig. 12 is a graph showing the relationship between the distance to the subject and the pixel value (here, brightness value) of the thermal image in the second embodiment. The vertical axis of the graph shown in Fig. 12 represents the brightness value of the thermal image TI, and the horizontal axis represents the distance to the subject (mm). Here, the size of the subject is adjusted to be the same at each distance so as not to be affected by the size and temperature of the subject. The temperature of the subject is also the same at any distance.

[0063] 12, the brightness value of the thermal image TI decreases as the distance to the subject increases. Therefore, the preliminary data storage unit 205 stores a function or graph of brightness values ​​for each distance to the subject as preliminary data. Here, the function is a linear function, but it may also be a quadratic or cubic function.

[0064] 11 , the correction value calculation unit 206 is a correction information specification unit that refers to the preliminary data AD stored in the preliminary data storage unit 205 and specifies correction information for compensating for the decrease in pixel value corresponding to the received light intensity at a distance to the subject, which is the amount by which the pixel value corresponding to the received light intensity at a predetermined distance decreases. Here, the correction value calculation unit 206 refers to the preliminary data AD stored in the preliminary data storage unit 205 and calculates a correction value CC for correcting the pixel value of the thermal image TI based on the distance to the subject, and provides the correction value CC to the thermal image correction unit 207. For example, the correction value calculation unit 206 specifies, as correction information, a correction value for matching the pixel value corresponding to the received light intensity at the distance to the subject in the thermal image TI with the pixel value corresponding to the received light intensity when the distance from the imaging surface to the detection target is a predetermined distance.

[0065] Here, the correction value calculation unit 206 receives the label number and distance from the distance acquisition unit 211, and calculates, for each label number, a correction value that will make the subject's luminance value equal to the luminance value when measured at a predetermined distance from the distance.

[0066] Specifically, the correction value calculation unit 206 determines, as a correction value, an offset that ensures a constant distance from the subject, as shown in FIG. 13. For example, assuming that the luminance value when the subject is 100 mm away is an accurate luminance value, the correction value calculation unit 206 calculates an offset of approximately 30 as a correction value when the subject is 400 mm away, as shown in FIG. 11. This makes the luminance value of the subject at a distance of 400 mm equivalent to the luminance value measured at a distance of 100 mm. In the case of distance-based correction, only offset correction is performed. The calculated correction value for each label number is provided to the thermal image correction unit 207.

[0067] The thermal image correction unit 207 corrects the labeled thermal image LTI sent from the object detection unit 103 using the correction value CC sent from the correction value calculation unit 206. For example, the thermal image correction unit 207 corrects the pixel values ​​of the object area so as to compensate for the amount of decrease in pixel value using the correction information from the correction value calculation unit 206. Here, the thermal image correction unit 207 adds the correction value to the pixel value.

[0068] Specifically, the thermal image correction unit 207 acquires the pixel value z of the coordinates of the object sent from the object detection unit 103. Then, the thermal image correction unit 107 checks the label of the pixel value z and calculates a corrected pixel value z# using the correction value CC corresponding to the label. The thermal image correction unit 107 replaces the pixel value z with the corrected pixel value z# to generate a corrected thermal image TO from the thermal image TI. The corrected thermal image TO is output to the output destination.

[0069] The distance acquisition unit 211 described above can also be configured with a memory 10 and a processor 11, for example, as shown in Fig. 10(A). The distance acquisition unit 211 can also be configured with a processing circuit 12. As described above, the distance acquisition unit 211 can be realized by a processing circuit network.

[0070] 14 is a block diagram showing a schematic configuration of a thermal image processing device 300 according to embodiment 3. The thermal image processing device 300 includes a thermal image sensor 101, a thermal image acquisition unit 102, a subject detection unit 103, an area measurement unit 104, a first advance data storage unit 305A, a second advance data storage unit 305B, a first correction value calculation unit 306A, a second correction value calculation unit 306B, a thermal image correction unit 307, a distance sensor 210, and a distance acquisition unit 211.

[0071] The thermal image sensor 101, thermal image acquisition unit 102, object detection unit 103, and area measurement unit 104 of the thermal image processing device 300 in the third embodiment are the same as those of the thermal image processing device 100 in the first embodiment. However, the object detection unit 103 in the third embodiment provides object information LP, which indicates a label indicating an object detected from the thermal image TI and the coordinate position of the object, together with the thermal image TI to the area measurement unit 104 and the distance acquisition unit 211. The labeled thermal image LTI, which includes the label indicating the object and the coordinate position of the object attached to the thermal image TI, is provided to the thermal image correction unit 307. The area measurement unit 104 in the third embodiment provides area data SA to the first correction value calculation unit 306A. The distance acquisition unit 211 in the third embodiment provides distance data DA to the second correction value calculation unit 306B.

[0072] The first preliminary data storage unit 305A and the first correction value calculation unit 306A of the thermal image processing device 300 in the third embodiment are similar to the preliminary data storage unit 105 and the correction value calculation unit 106 of the thermal image processing device 100 in the first embodiment. However, the first correction value calculation unit 306A in the third embodiment provides the calculated correction value as the first correction value FCC to the thermal image correction unit 307.

[0073] The first advance data storage unit 305A is also referred to as a first data storage unit, and the advance data stored in the first advance data storage unit 305A is also referred to as first data. The first correction value calculation unit 306A is also referred to as a first correction information determination unit, and the correction value calculated by the first correction value calculation unit 306A is also referred to as a first correction value or first correction information.

[0074] The second preliminary data storage unit 305B and the second correction value calculation unit 306B of the thermal image processing device 300 in the third embodiment are similar to the preliminary data storage unit 205 and the correction value calculation unit 206 of the thermal image processing device 200 in the second embodiment. However, the second correction value calculation unit 306B in the third embodiment provides the calculated correction value as the second correction value SCC to the thermal image correction unit 307.

[0075] The second advance data storage unit 305B is also referred to as a second data storage unit, and the advance data stored in the second advance data storage unit 305B is also referred to as second data. The second correction value calculation unit 306B is also referred to as a second correction information determination unit, and the correction value calculated by the second correction value calculation unit 306B is also referred to as a second correction value or second correction information.

[0076] The thermal image correction unit 307 corrects pixel values ​​of the subject area using the first and second correction information to compensate for the first and second reduction amounts. Here, the first reduction amount is the amount by which pixel values ​​corresponding to the received light intensity within the size of the subject area are reduced. The second reduction amount is the amount by which pixel values ​​corresponding to the received light intensity at a distance to the subject are reduced from pixel values ​​corresponding to the received light intensity at a predetermined distance.

[0077] Here, the thermal image correction unit 307 calculates a modified correction value by weighting and combining the first correction value FCC sent from the first correction value calculation unit 306A and the second correction value SCC sent from the second correction value calculation unit 306B. Then, the thermal image correction unit 307 uses the modified correction value to correct the labeled thermal image LTI sent from the object detection unit 103.

[0078] For example, the first correction value calculation unit 306A provides a gain and an offset as the first correction value FCC, while the second correction value calculation unit 306B provides only an offset as the second correction value SCC. Therefore, the thermal image correction unit 307 uses the value included in the first correction value FCC as the gain.

[0079] The thermal image correction unit 307 also adds the offset included in the first correction value FCC and the offset included in the second correction value SCC to calculate a total value. Next, the thermal image correction unit 307 calculates the ratio of the offset included in the first correction value FCC to the total value and the ratio of the offset included in the second correction value SCC to the total value. The thermal image correction unit 307 then multiplies the calculated ratios by the offset included in the first correction value FCC and the offset included in the second correction value SCC, and then adds them together to calculate the offset correction value.

[0080] Specifically, if the offset included in the first correction value FCC is a and the offset included in the second correction value SCC is b, their total value is a+b. Therefore, the ratio of the offset included in the first correction value FCC to the total value is a÷(a+b), and the ratio of the offset included in the second correction value SCC to the total value is b÷(a+b). Therefore, the thermal image correction unit 307 calculates a ２ ÷(a + b) + b ２ The weighted correction value of the offset is calculated by dividing (a+b).

[0081] From the above, the thermal image corrector 307 sets the value included in the first correction value FCC as the gain and the weighted correction value of the offset as the modified correction value.

[0082] The thermal image processing devices 100 to 300 described above can be installed in electrical appliances to use thermal images to control the electrical appliances. The thermal image processing devices 100 to 300 can also use thermal images to check the temperature conditions in the room where the electrical appliance is installed. Furthermore, the thermal image processing devices 100 to 300 can be used not only in electrical appliances but also as security or surveillance cameras installed indoors or outdoors.

[0083] 100, 200, 300 Thermal image processing device, 101 Thermal image sensor, 102 Thermal image acquisition unit, 103 Subject detection unit, 104 Area measurement unit, 105, 205 Preliminary data storage unit, 106, 206 Correction value calculation unit, 107, 207, 307 Thermal image correction unit, 210 Distance sensor, 211 Distance acquisition unit, 305A First preliminary data storage unit, 305B Second preliminary data storage unit, 306A First correction value calculation unit, 306B Second correction value calculation unit.

Claims

1. A thermal image processing device comprising: an object detection unit that detects an object area from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of received infrared light; an object size determination unit that determines the size of the object area in the thermal image; a data storage unit that stores data indicating the relationship that the smaller the area on the imaging surface that receives infrared light from the detection object, the lower the pixel value corresponding to the intensity of received infrared light from the detection object; a correction information determination unit that references the data and determines correction information to compensate for the amount of decrease in pixel value corresponding to the intensity of received infrared light within the size of the object area; and a thermal image correction unit that uses the correction information to correct the pixel values ​​of the area to compensate for the amount of decrease.

2. The thermal image processing device of claim 1, wherein the data indicates a function or graph that represents the correspondence between pixel values ​​corresponding to the intensity of infrared light received from the detection target and the temperature of the detection target, for each ratio of the size of the area where infrared light from the detection target is received to the size of the imaging surface, and the correction information identification unit identifies as the correction information a correction value for matching the pixel values ​​of the function or graph that correspond to the ratio of the size of the area of ​​the subject to the size of the thermal image with the pixel values ​​of the corresponding temperature according to the function or graph when infrared light from the detection target is received over the entire imaging surface.

3. A thermal image processing device comprising: a subject detection unit that detects a subject area from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of received infrared light; a distance determination unit that determines the distance to the subject; a data storage unit that stores data indicating the relationship that the pixel value corresponding to the intensity of received infrared light from the detection target decreases as the distance from an imaging surface that receives infrared light from the detection target increases; a correction information determination unit that refers to the data and determines correction information to compensate for the amount by which the pixel value corresponding to the intensity of received infrared light at the distance to the subject decreases from the pixel value corresponding to the intensity of received light at a predetermined distance; and a thermal image correction unit that uses the correction information to correct the pixel values ​​of the area to compensate for the amount of decrease.

4. The thermal image processing device described in claim 3, characterized in that the data indicates a function or graph representing pixel values ​​corresponding to the intensity of infrared light received from the detection target for each length of distance from the imaging surface to the detection target, and the correction information identification unit identifies as the correction information a correction value for matching the pixel value corresponding to the intensity of received light corresponding to the distance to the subject in the thermal image with the pixel value corresponding to the intensity of received light when the distance from the imaging surface to the detection target is the predetermined distance.

5. A subject detection unit that detects a subject area from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of infrared light received; a subject size determination unit that determines the size of the area in the thermal image; a first data storage unit that stores first data indicating a relationship in which the pixel value corresponding to the intensity of infrared light received from the detection object decreases as the area on the imaging surface that receives infrared light from the detection object becomes smaller; a first correction information determination unit that refers to the first data and determines first correction information to compensate for a first decrease amount, which is the amount by which the pixel value corresponding to the intensity of infrared light received in the size of the subject area decreases; a distance determination unit that determines the distance to the subject; a second data storage unit that stores second data indicating a relationship in which the pixel value corresponding to the intensity of infrared light received from the detection object decreases as the distance from the imaging surface to the detection object becomes longer; and a second correction information determination unit that refers to the second data and determines second correction information to compensate for a second decrease amount, which is the amount by which the pixel value corresponding to the intensity of infrared light received at the distance to the subject decreases from the pixel value corresponding to the intensity of infrared light received at a predetermined distance. a thermal image correction unit that corrects pixel values ​​of the region so as to compensate for the first amount of decrease and the second amount of decrease using the first correction information and the second correction information.

6. The thermal image processing device according to any one of claims 1 to 5, further comprising: a thermal image sensor that generates a signal indicating the intensity of infrared light received in the space including the subject; and a thermal image acquisition unit that acquires the thermal image by generating the thermal image from the signal.

7. A program that causes a computer to function as: an object detection unit that detects an object area from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of received infrared light; an object size determination unit that determines the size of the object area in the thermal image; a data storage unit that stores data indicating the relationship that the smaller the area on the imaging surface that receives infrared light from the detection object, the lower the pixel value corresponding to the intensity of received infrared light from the detection object; a correction information determination unit that references the data and determines correction information to compensate for the amount of decrease in pixel value corresponding to the intensity of received infrared light within the size of the object area; and a thermal image correction unit that uses the correction information to correct the pixel values ​​of the area to compensate for the amount of decrease.

8. A program causing a computer to function as: a subject detection unit that detects a subject area from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of received infrared light; a distance determination unit that determines the distance to the subject; a data storage unit that stores data indicating the relationship that the pixel value corresponding to the intensity of received infrared light from the subject decreases as the distance from an imaging surface that receives infrared light from the subject increases; a correction information determination unit that references the data and determines correction information to compensate for the amount by which the pixel value corresponding to the intensity of received infrared light at the distance to the subject decreases from the pixel value corresponding to the intensity of received light at a predetermined distance; and a thermal image correction unit that uses the correction information to correct the pixel values ​​of the area to compensate for the amount of decrease.

9. A computer includes a subject detection unit that detects a subject area from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of infrared light received; a subject size determination unit that determines the size of the area in the thermal image; a first data storage unit that stores first data indicating a relationship in which the pixel value corresponding to the intensity of infrared light received from the detection object decreases as the area on the imaging surface that receives infrared light from the detection object becomes smaller; a first correction information determination unit that refers to the first data and determines first correction information to compensate for a first decrease amount, which is the amount by which the pixel value corresponding to the intensity of infrared light received in the size of the area of ​​the subject decreases; a distance determination unit that determines the distance to the subject; a second data storage unit that stores second data indicating a relationship in which the pixel value corresponding to the intensity of infrared light received from the detection object decreases as the distance from the imaging surface to the detection object becomes longer; a second correction information determination unit that refers to the second data and determines second correction information to compensate for a second decrease amount, which is the amount by which the pixel value corresponding to the intensity of infrared light received at the distance to the subject decreases from the pixel value corresponding to the intensity of infrared light received at a predetermined distance; a thermal image correction unit that corrects pixel values ​​of the region to compensate for the first amount of decrease and the second amount of decrease using the first correction information and the second correction information.

10. A thermal image processing method comprising: detecting an area of ​​a subject from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of infrared light received; identifying the size of the area of ​​the subject in the thermal image; referring to data indicating a relationship in which the smaller the area on the imaging surface that receives infrared light from the detection target, the lower the pixel value corresponding to the intensity of infrared light received from the detection target; identifying correction information to compensate for the amount of decrease in the pixel value corresponding to the intensity of infrared light received within the size of the area of ​​the subject; and correcting the pixel values ​​of the area using the correction information to compensate for the amount of decrease.

11. A thermal image processing method comprising: detecting an area of ​​a subject from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of received infrared light; identifying the distance to the subject; referring to data indicating a relationship in which the pixel value corresponding to the intensity of received infrared light from the subject decreases as the distance from an imaging surface that receives infrared light from the subject increases; identifying correction information for compensating for the amount by which the pixel value corresponding to the intensity of received infrared light at the distance to the subject decreases from the pixel value corresponding to the intensity of received infrared light at a predetermined distance; and using the correction information, correcting the pixel values ​​of the area to compensate for the amount of decrease.

12. A thermal image processing method comprising: detecting an area of ​​a subject from a thermal image in which each of a plurality of pixels indicates a pixel value corresponding to the intensity of received infrared light; identifying the size of the area in the thermal image; referring to first data indicating a relationship in which the pixel value corresponding to the intensity of received infrared light from the detection object decreases as the area on the imaging surface that receives infrared light from the detection object becomes smaller; identifying first correction information for compensating for a first decrease amount, which is the amount by which the pixel value corresponding to the intensity of received infrared light decreases in the size of the area of ​​the subject; identifying the distance to the subject; referring to second data indicating a relationship in which the pixel value corresponding to the intensity of received infrared light from the detection object decreases as the distance from the imaging surface to the detection object becomes longer; identifying second correction information for compensating for a second decrease amount, which is the amount by which the pixel value corresponding to the intensity of received infrared light at the distance to the subject decreases from the pixel value corresponding to the intensity of received infrared light at a predetermined distance; and correcting the pixel values ​​of the area to compensate for the first decrease amount and the second decrease amount using the first correction information and the second correction information.

Images