Thermal image processing device, thermal image processing module, thermal image processing method and program recording medium

Abstract

The thermal image processing device (100) comprises: an acquisition unit (11) for acquiring a thermal image from a thermal image sensor (101); a first processing unit (12) for performing image processing on a high-temperature pixel representing a temperature higher than a first temperature among a plurality of pixels included in the acquired thermal image so as to lower the temperature represented by the high-temperature pixel; and a second processing unit (13) for performing high-frequency emphasis processing on the thermal image after image processing, i.e., a transformed image, so as to emphasize the high-frequency components included in the transformed image.

Description

Technical Field

[0001] The present disclosure relates to a thermal image processing device, a thermal image processing module, a thermal image processing method and a program recording medium. Background Art

[0002] Conventionally, there is a known technology for controlling a device based on a thermal image detected by a thermal image sensor. Since a cost-effective thermal image sensor is used, the detected thermal image may have a low resolution. In response to this, there is a known method for obtaining a thermal image with an expanded spatial resolution by super-resolution processing, such as disclosed in Patent Document 1.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2014-022845 Summary of the invention

[0006] Problems to be solved by the invention

[0007] In addition, the super-resolution process may include a high-frequency emphasis process for emphasizing the high-frequency components contained in the thermal image. In such a high-frequency emphasis process, there are cases where the generated thermal image is inappropriate. Therefore, in the present disclosure, a thermal image processing device or the like that generates a more appropriate thermal image is provided.

[0008] Means used to solve problems

[0009] In order to solve the above-mentioned problems, a thermal image processing device according to a technical solution disclosed in the present invention comprises: an acquisition unit, which acquires a thermal image from a thermal image sensor; a first processing unit, which performs image processing on high-temperature pixels representing a temperature higher than a first temperature among a plurality of pixels included in the acquired thermal image, so as to lower the temperature represented by the high-temperature pixels; and a second processing unit, which performs high-frequency emphasis processing on the thermal image after the image processing, i.e., a transformed image, so as to emphasize the high-frequency components included in the transformed image.

[0010] Furthermore, a thermal image processing module according to one technical solution of the present disclosure includes the thermal image sensor and the thermal image processing device described above.

[0011] In addition, a thermal image processing method related to a technical solution of the present disclosure includes: an acquisition step of acquiring a thermal image from a thermal image sensor; a first processing step of performing image processing on high-temperature pixels representing a temperature higher than a first temperature among a plurality of pixels contained in the acquired thermal image to lower the temperature represented by the high-temperature pixels; and a second processing step of performing high-frequency emphasis processing on the thermal image after the image processing, i.e., a transformed image, to emphasize the high-frequency components contained in the transformed image.

[0012] Furthermore, a program recording medium according to one aspect of the present disclosure is a non-transitory computer-readable recording medium recording a program for causing a computer to execute the thermal image processing method described above.

[0013] Effects of the Invention

[0014] According to the present disclosure, a more appropriate thermal image can be generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a diagram for explaining a problem when emphasizing high-frequency components of a thermal image.

[0016] Figure 2 This is a schematic diagram showing a usage example of the thermal image processing device related to the embodiment.

[0017] Figure 3 This is an enlarged view showing the appearance of a display device according to the embodiment.

[0018] Figure 4 This is a block diagram showing the functional structure of a thermal image processing device according to an embodiment.

[0019] Figure 5 It is a schematic diagram for explaining the processing of thermal images related to the embodiment.

[0020] Figure 6 is a flowchart showing a thermal image processing method according to an embodiment.

[0021] Fig. 7A It is a diagram showing a first example of a conversion function according to the embodiment.

[0022] Figure 7B It is a diagram showing a second example of the conversion function according to the embodiment.

[0023] Figure 7C It is a diagram showing a third example of the conversion function according to the embodiment.

[0024] Fig.7D It is a diagram showing a fourth example of the conversion function according to the embodiment.

[0025] Figure 8This is the first diagram illustrating the image processing according to the embodiment.

[0026] Fig. 9 FIG. 2 is a second diagram illustrating image processing according to the embodiment.

[0027] Fig.10 It is a diagram for explaining the conversion function related to the embodiment.

[0028] Fig.11 This is a diagram showing an example of a combination of a first temperature and a second temperature according to an embodiment. DETAILED DESCRIPTION

[0029] (Background of this Disclosure)

[0030] Super-resolution processing is an image processing technology that generates a high-resolution image close to the original state by calculating and extrapolating from a low-resolution image. Usually, due to the optical degradation characteristics (blurring, downsampling, noise, etc.) of the equipment used in capturing the image, degradation can be seen in the obtained image. Super-resolution processing is known as one of the processes for restoring an image obtained in such a degraded state to its original high-resolution state.

[0031] As mentioned above, improving the resolution of an image means compensating and restoring information lost due to degradation characteristics through prediction. For example, in the process of restoring a high-resolution image from a degraded image, most of them include a process of compensating for high-frequency components that should have been observed but were lost due to degradation characteristics. In other words, in many super-resolution processes, a high-frequency emphasis process of emphasizing high-frequency components may be included.

[0032] In particular, when using cheap thermal image sensors to capture thermal images, optical systems (lenses, photosensitive elements, etc.) with large degradation characteristics are often used. In this case, a stronger emphasis on high-frequency components is required. Furthermore, in thermal images, the accuracy of the temperature of each pixel (or pixel) decreases due to degradation, so super-resolution processing including high-frequency emphasis processing is implemented to improve the accuracy of the temperature.

[0033] Here, if high-frequency emphasis processing is applied to the thermal image, the brightness value of the pixel (i.e., the temperature in the thermal image) is different from the brightness values ​​of the adjacent pixels, and undershoot and overshoot may occur at the location where a gradient is formed between the brightness values ​​of the pixels. Undershoot is a phenomenon in which a vibration including a value lower than the original value occurs in the surrounding pixels due to a sharp decrease in the input value in the high-frequency emphasis processing and a delay in the convergence of the output value. In addition, overshoot is a phenomenon in which a vibration including a value higher than the original value occurs in the surrounding pixels due to a sharp increase in the input value in the high-frequency emphasis processing and a delay in the convergence of the output value. In the high-frequency emphasis processing, such so-called oscillations in the image space occur due to undershoot and overshoot, so it is often necessary to deal with the oscillations.

[0034] When high-frequency emphasis processing is applied to a steep gradient portion where the brightness value of a pixel is greatly different from the brightness value of an adjacent pixel, the above problem has a particularly large impact because the high-frequency component is large at the steep gradient portion. In the case of an apparatus for processing thermal images, the temperature of each pixel is often more important. That is, in an apparatus for processing thermal images, the accuracy of the brightness value of each pixel is required compared to an apparatus for processing visible images, and the influence of the above-mentioned oscillation needs to be eliminated as much as possible.

[0035] If the parameters are designed so that oscillation does not occur regardless of the image used as the input image in the high-frequency emphasis processing, the emphasis effect of the high-frequency component will have to be reduced. On the other hand, in order to input all images as the input image in the high-frequency emphasis processing while maintaining the emphasis effect of the high-frequency component, it is necessary to design appropriate parameters for each image, which is unrealistic.

[0036] As described above, in the high-frequency emphasis processing of thermal images, sometimes the purpose of the device for processing thermal images is clear. For example, if it is a device that uses thermal images to observe the human body in a normal temperature environment, the temperature difference between the human body and the surrounding environment is about several degrees Celsius, and a thermal image with such a temperature gradient may be input. When there are objects such as lamps and stoves that are much hotter than the human body over a certain number of pixels in the thermal image, the above-mentioned oscillations caused by the high-frequency emphasis processing will have a greater impact. In addition, since the purpose of the device is limited, there are many cases where the upper limit of the brightness value of each pixel is set low, and the brightness value is easily saturated due to the presence of high-temperature objects. In such a case, the occurrence of oscillations caused by the high-frequency emphasis processing also becomes a problem.

[0037] In the above-mentioned example in which the human body is the observation object, the temperature range in which the temperature is to be observed accurately is usually limited. That is, the temperature in the range of about 20 to 40°C, including comfortable room temperature or human body temperature, fits into such a temperature range. In such a temperature range, although it is necessary to observe the correct temperature, outside this temperature range, sometimes the operation of the device is not affected even if the allowable range of the observed temperature in accuracy is expanded. Depending on the situation, for temperatures above a certain level, there are also cases where it is sufficient to know that it is "above a certain level." As such, when the purpose of the thermal image obtained is clear, there may be a specific range in which the accuracy of the observed temperature is required and other ranges in which accuracy is not required.

[0038] The present disclosure relates to a thermal image processing device and the like that performs high-frequency emphasis processing by different processing within a temperature range and outside a temperature range in which the accuracy of the observed temperature is ensured.

[0039] (Overview of the present disclosure)

[0040] A thermal image processing device according to a technical solution of the present disclosure for solving the above-mentioned problems comprises: an acquisition unit for acquiring a thermal image from a thermal image sensor; a first processing unit for performing image processing on high-temperature pixels representing a temperature higher than a first temperature among a plurality of pixels included in the acquired thermal image so as to lower the temperature represented by the high-temperature pixels; and a second processing unit for performing high-frequency emphasis processing on the thermal image after image processing, i.e., a transformed image, so as to emphasize the high-frequency components included in the transformed image.

[0041] Such a thermal image processing device reduces the temperature difference in the steep gradient portion where the temperature change between pixels is large, and suppresses the amplitude of the temperature that is the cause of the oscillation, before performing high-frequency emphasis processing to cause oscillations in the thermal image. Thus, when the high-frequency conversion processing is performed, the amplitude of the temperature is also suppressed, so the occurrence of oscillations is suppressed. In addition, such processing is only performed on pixels representing temperatures higher than the first temperature, and the above-mentioned processing to suppress the amplitude of the temperature is not applied to pixels representing temperatures below the first temperature. That is, for pixels representing temperatures below the first temperature, high-frequency emphasis processing is performed at the original temperature, so a thermal image is generated that represents temperatures with high accuracy for temperatures below the first temperature. Therefore, the thermal image processing device can apply processing to suppress the occurrence of oscillations with the first temperature as the boundary, and generate a more appropriate thermal image that maintains accuracy for the temperature range that is desired to be focused on (i.e., temperatures below the first temperature).

[0042] In addition, for example, the first processing unit may also increase the temperature of low-temperature pixels representing a temperature lower than a second temperature among a plurality of pixels included in the acquired thermal image during image processing, and the second temperature is lower than the first temperature.

[0043] Thus, the process of suppressing the occurrence of oscillation can be performed even in pixels indicating a temperature lower than the second temperature. Therefore, the process of suppressing the occurrence of oscillation can be applied to pixels indicating a temperature higher than the first temperature and pixels indicating a temperature lower than the second temperature, and a more appropriate thermal image can be generated that maintains accuracy within the temperature range of the first temperature to the second temperature that is of interest.

[0044] In addition, for example, the thermal image processing device may further include an input unit for setting at least one of the first temperature and the second temperature.

[0045] Thus, the first temperature and the second temperature are set based on the input from the user received in the input unit. Since the first temperature and the second temperature can be set each time, the thermal image processing device 100 can be applied to various devices for various purposes using thermal images. In other words, the application range of the thermal image processing device 100 can be expanded.

[0046] Furthermore, for example, in setting at least one of the first temperature and the second temperature, the first temperature and the second temperature may be set by one combination selected from a plurality of combinations of the first temperature and the second temperature.

[0047] Thus, a plurality of combinations of the first temperature and the second temperature that can be conceived according to the purpose are pre-registered, and the user only needs to select a combination from the plurality of combinations to set an appropriate combination of the first temperature and the second temperature. The operation for the user is simple because only a combination needs to be selected. Therefore, the ease of use of the thermal image processing device 100 can be improved.

[0048] In addition, for example, in image processing, a low-temperature differential value obtained by subtracting a second temperature from the temperature represented by a low-temperature pixel among multiple pixels included in a thermal image is calculated; and a low-temperature transformation algorithm that outputs a value higher than the input value is used to transform the temperature represented by the low-temperature pixel by the value outputted in response to the input of the low-temperature differential value.

[0049] Thus, in pixels indicating a temperature lower than the second temperature, the temperature can be converted to a higher temperature based on the low temperature conversion algorithm. That is, in pixels indicating a temperature lower than the second temperature, a process for suppressing the occurrence of oscillation can be applied according to the low temperature conversion algorithm. Therefore, a more appropriate thermal image can be generated.

[0050] In addition, for example, in image processing, a high temperature differential value obtained by subtracting a first temperature from the temperature represented by a high temperature pixel among multiple pixels included in a thermal image is calculated; and a high temperature transformation algorithm that outputs a value lower than the input value is used to transform the temperature represented by the high temperature pixel by the value outputted in response to the input of the high temperature differential value.

[0051] Thus, in pixels indicating a temperature higher than the first temperature, the temperature can be converted to a lower temperature based on the high temperature conversion algorithm. That is, in pixels indicating a temperature higher than the first temperature, a process for suppressing the occurrence of oscillation can be applied according to the high temperature conversion algorithm. Therefore, a more appropriate thermal image can be generated.

[0052] Furthermore, for example, the low-temperature conversion algorithm and the high-temperature conversion algorithm may be conversion functions that can input continuous values.

[0053] Thus, regardless of the input value, the process of suppressing the occurrence of ringing can be applied to pixels indicating a temperature higher than the first temperature and pixels indicating a temperature lower than the second temperature by calculation. Therefore, a more appropriate thermal image can be generated.

[0054] Furthermore, for example, the low-temperature conversion algorithm and the high-temperature conversion algorithm may be conversion tables that establish correspondence between input values ​​and output values.

[0055] Thus, the conversion table can be referred to according to the input value, and the process of suppressing the occurrence of ringing can be applied to pixels indicating a temperature higher than the first temperature and pixels indicating a temperature lower than the second temperature. Therefore, a more appropriate thermal image can be generated.

[0056] Furthermore, for example, the conversion function may output a value of 0 when the input value is 0, and may output a differential value of 1 when the input value is 0.

[0057] Thus, at the temperatures before and after the first temperature and the second temperature, which are the boundary temperatures for applying or not applying the process of suppressing the occurrence of oscillation, a sharp change in the value after the process of suppressing the occurrence of oscillation is performed is suppressed. In addition, at the temperatures before and after the first temperature and the second temperature, the dispersion of the value after the process of suppressing the occurrence of oscillation can be suppressed. Therefore, after the process of suppressing the occurrence of oscillation is performed, the values ​​are smoothly connected in the entire temperature range, and a more appropriate thermal image can be generated.

[0058] In addition, for example, the thermal image processing device may also include a third processing unit, which calculates a differential image by subtracting corresponding pixels of the acquired thermal image and the transformed image, and performs compensation processing by adding corresponding pixels of the transformed image after high-frequency emphasis processing and the calculated differential image.

[0059] Thus, a thermal image is generated to which a part of the temperature information missing by the image processing of temperature conversion is added. Thus, a more appropriate thermal image can be generated.

[0060] Furthermore, for example, the second processing unit may further perform super-resolution processing for increasing the spatial resolution of the transformed image.

[0061] This makes it possible to perform super-resolution processing for increasing spatial resolution together with high-frequency emphasis processing. Therefore, the thermal image processing apparatus can be applied to an apparatus that performs super-resolution processing of a thermal image.

[0062] In addition, a thermal image processing module according to one technical solution of the present disclosure includes a thermal image sensor and any one of the thermal image processing devices described above.

[0063] Therefore, by capturing a thermal image with a thermal image sensor and processing the thermal image with a thermal image processing device, a more appropriate thermal image can be generated.

[0064] Furthermore, for example, the thermal image processing module may further include a display device that displays the transformed image after the high-frequency emphasis processing as at least one of an image and temperature data.

[0065] Thereby, a more appropriate thermal image generated can be displayed in the form of at least one of an image and temperature data.

[0066] In addition, a thermal image processing method related to a technical solution of the present disclosure includes: an acquisition step of acquiring a thermal image from a thermal image sensor; a first processing step of performing image processing on high-temperature pixels representing a temperature higher than a first temperature among a plurality of pixels contained in the acquired thermal image to lower the temperature represented by the high-temperature pixels; and a second processing step of performing high-frequency emphasis processing on the thermal image after image processing, i.e., a transformed image, to emphasize the high-frequency components contained in the transformed image.

[0067] Thus, before performing high-frequency emphasis processing to generate oscillations in the thermal image, the temperature difference in the steep gradient portion where the temperature change between pixels is large is reduced, thereby suppressing the amplitude of the temperature that is the cause of the oscillations. Thus, even when the high-frequency conversion processing is performed, the amplitude of the temperature is also suppressed, thereby suppressing the occurrence of oscillations. In addition, such processing is only performed on pixels representing temperatures higher than the first temperature, and the above-mentioned processing to suppress the amplitude of the temperature is not applied to pixels representing temperatures below the first temperature. That is, for pixels representing temperatures below the first temperature, high-frequency emphasis processing is performed at the original temperature, thereby generating a thermal image that represents temperatures with high accuracy for temperatures below the first temperature. Therefore, the thermal image processing device can apply processing to suppress the occurrence of oscillations with the first temperature as the boundary, and generate a more appropriate thermal image that maintains accuracy for the temperature range that is desired to be focused on (i.e., temperatures below the first temperature).

[0068] Furthermore, a program recording medium according to one aspect of the present disclosure is a non-transitory computer-readable recording medium recording a program for causing a computer to execute the thermal image processing method described above.

[0069] Thereby, the thermal image processing method described above can be executed by a computer.

[0070] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In addition, the embodiments described below all represent inclusive or specific examples. Therefore, the numerical values, constituent elements, configuration positions and connection forms of constituent elements, and steps and the order of steps, etc., represented in the following embodiments are examples and are not intended to limit the present disclosure. Therefore, constituent elements among the constituent elements of the following embodiments that are not described in the independent claims of the present disclosure are described as arbitrary constituent elements.

[0071] In addition, each figure is a schematic diagram and does not necessarily illustrate the figure strictly. Therefore, the scales and the like in each figure are not necessarily the same. In each figure, substantially the same structure is given the same reference numeral, and repeated descriptions are omitted or simplified.

[0072] (Implementation Method)

[0073] [High frequency emphasis processing]

[0074] First, use Figure 1 The oscillation generated by the high frequency emphasis processing will be described. Figure 1 This is a diagram for explaining a problem when emphasizing high-frequency components of a thermal image. Figure 1 (a)~ Figure 1 (c) shows the change in temperature when a straight line is drawn on the two-dimensional surface of the acquired thermal image and the temperature of each pixel on the straight line (ie, the brightness value of the image) is taken on the horizontal axis. In addition, Figure 1 (a) shows the actual situation in which the thermal image is obtained, which corresponds to the ideal thermal image that should be obtained.

[0075] As in Figure 1 As shown by the dotted circle in (a), the temperature originally forms an edge that is bent and connected from the high temperature part to the low temperature part. However, as described above, due to the degradation characteristics when obtaining the thermal image, the obtained thermal image is like Figure 1 The edge shown in (b) is destroyed by blurring, etc., and the low temperature is pulled closer to the high temperature and the high temperature is pulled closer to the low temperature, so that the baseline (zero point of brightness) is unstable. In particular, there is a problem that the original brightness value cannot be maintained in areas with large temperature changes (steep gradient areas).

[0076] Therefore, high frequency emphasis is usually performed on the acquired thermal image, such as Figure 1 As shown in (c), a thermal image close to the actual situation is generated by image processing. Figure 1 In (c), a disordered oscillation as a waveform represented by a dotted ellipse occurs. As described above, the occurrence of oscillation is omitted, but such a problem occurs particularly in a portion where the temperature changes more. In addition, the influence of the oscillation spreads to the surrounding pixels, so the accuracy of the brightness value including the pixels surrounding the pixel where the oscillation occurs is reduced.

[0077] [Thermal image processing module]

[0078] use Figure 2 A description will be given of a usage example of the thermal image processing device and the thermal image processing module according to the present embodiment. Figure 2 : is an overview diagram showing a use case of the thermal image processing device of the embodiment. In addition, the use case of the thermal image processing device and the thermal image processing module described here is only one example for convenience, and is not intended to limit the use of the thermal image processing device and the thermal image processing module. The thermal image processing device and the thermal image processing module can be applied to any use that can obtain a thermal image and perform high-frequency emphasis processing on the thermal image.

[0079] like Figure 2 As shown, the thermal image processing device 100 is implemented by being built into, for example, an air conditioning device 201 in a room 200 provided in a residence. The air conditioning device 201 has a built-in control unit (not shown), and the control unit determines the airflow in the room 200 brought by the air conditioning device 201 based on the thermal image. The air conditioning device 201 operates to form the airflow determined in the control unit using a blower, a heating device, a cooling device, etc. (none of which are shown).

[0080] The thermal image processing device 100 is a device that communicates with the thermal image sensor 101 provided on the outer surface of the air conditioning device 201 to obtain a thermal image from the thermal image sensor 101, and outputs the processed thermal image, i.e., the processed image, as an output image after high-frequency emphasis processing. The control unit uses the output image, i.e., the processed thermal image, outputted by the thermal image processing device 100, to infer the air, airflow, position of a person, and temperature sensation of a person in the room 200 based on the output image, and determines the airflow in the room 200.

[0081] The thermal image sensor 101 is a device that captures and outputs thermal images, and has detection elements that detect infrared rays radiated from an object arranged in a two-dimensional surface. These detection elements correspond to pixels of the output thermal image, respectively. Therefore, in the thermal image sensor 101, the infrared rays detected in each detection element are arranged in a two-dimensional surface as brightness values ​​corresponding to the temperature to generate a thermal image. Such a thermal image is output differently from the actual situation according to the degradation characteristics as described above. The greater the degradation characteristics of the thermal image sensor 101, the greater the effect of the thermal image processing device 100 etc. disclosed in the present invention. In other words, through the thermal image processing device 100 etc., it is possible to expand the options of the thermal image sensor 101 used in devices for processing thermal images etc.

[0082] The output image generated by the thermal image processing device 100 is output to the control unit of the air conditioning device 201, and is also output to the display device 102. The display device 102 is a device that displays the processed thermal image as an output image as at least one of an image and temperature data. The display device 102 is implemented by, for example, being mounted on a remote controller of the air conditioning device 201. The display device 102 displays a display image on a display panel using a liquid crystal, an organic EL, a micro LED, etc., and displays the thermal image as at least one of an image and temperature data.

[0083] Figure 3 FIG. 1 is an enlarged view showing the appearance of a display device according to an embodiment. The display of the thermal image by the display device 102 can be performed as follows, for example. Figure 3 The thermal image is shown as being displayed only, but it may also be displayed in the form of heat flow based on the temperature of people in the room 200, the room temperature of the room 200, the temperature difference of various places, etc., but it is not particularly limited to such a display form. In addition, the thermal image sensor 101, the thermal image processing device 100 and the display device 102 mentioned above may be added together to realize a thermal image processing module.

[0084] Below, use Figure 4 The thermal image processing device 100 will be described in more detail. Figure 4 FIG. 2 is a block diagram showing the functional structure of a thermal image processing device according to an embodiment of the present invention. Figure 4 As shown, the thermal image processing device 100 and the thermal image sensor 101 are built into the air conditioning device 201, and the thermal image processing device 100 and the display device 102 mounted on the remote controller are communicably connected.

[0085] The thermal image processing device 100 includes an acquisition unit 11 , a first processing unit 12 , a second processing unit 13 , a third processing unit 14 , an input unit 15 , and an output unit 16 .

[0086] The acquisition unit 11 is a processing unit that acquires a thermal image from the thermal image sensor 101. The acquisition unit 11 acquires the thermal image output from the thermal image sensor 101 as it is or in a format that can be processed by the first processing unit described later, via a communication module that is communicatively connected to the thermal image sensor 101.

[0087] The first processing unit 12 is a processing unit that performs temperature conversion image processing on pixels that represent a temperature higher than a preset first temperature among a plurality of pixels included in the acquired thermal image, i.e., high temperature pixels. Specifically, the first temperature is subtracted from the temperature represented by the high temperature pixels to calculate a high temperature difference value as their difference value.

[0088] The first processing unit 12 converts the temperature using a high temperature conversion algorithm that outputs a value lower than the input value. The temperature represented by the high temperature pixel is replaced with the converted temperature, and a converted image is output as a thermal image after the temperature conversion. The high temperature conversion algorithm will be described in detail later, but by processing the high temperature pixels by the first processing unit 12, a converted image in which the temperature of pixels higher than the first temperature in the thermal image is reduced can be obtained.

[0089] In addition, the first processing unit 12 performs image processing for converting the temperature of the pixels, i.e., low-temperature pixels, among the plurality of pixels included in the acquired thermal image, which represent a temperature lower than the preset second temperature. Specifically, the second temperature is subtracted from the temperature represented by the low-temperature pixel to calculate a low-temperature difference value as their difference value. The low-temperature difference value is a negative value because it is obtained by subtracting the second temperature from the temperature represented by the low-temperature pixel lower than the second temperature.

[0090] The first processing unit 12 converts the temperature using a low temperature conversion algorithm that outputs a value higher than the input value. The temperature represented by the low temperature pixel is replaced with the converted temperature, and a conversion image is output as a thermal image after temperature conversion. The low temperature conversion algorithm will be described in detail later, but by processing the low temperature pixels by the first processing unit 12, a conversion image in which the temperature of pixels lower than the second temperature in the thermal image rises can be obtained.

[0091] In this way, by the image processing of the first processing unit 12, the high temperature side is lowered and the low temperature side is raised except for the temperature range between the first temperature and the second temperature, thereby reducing the amplitude of the temperature of each pixel in the thermal image.

[0092] The second processing unit 13 is a processing unit that performs high-frequency emphasis processing on the thermal image, that is, the transformed image, which has been subjected to the image processing by the first processing unit 12, so as to emphasize the high-frequency components contained in the transformed image. The high-frequency emphasis processing of the second processing unit 13 may also be applied to any existing processing. Through the high-frequency emphasis processing performed by the second processing unit 13, the high-frequency components of the thermal image are restored, and a processed image close to the actual situation is generated.

[0093] The third processing unit 14 is a processing unit that complements the processed image after the high-frequency emphasis processing of the second processing unit 13 by compensating a part of the information lost by the image processing of the first processing unit 12 (i.e., the high temperature difference value and the low temperature difference value). The third processing unit 14 calculates and generates a differential image obtained by subtracting corresponding pixels of the acquired thermal image and the transformed image after the image processing by the first processing unit 12. In addition, the third processing unit 14 performs compensation processing by adding corresponding pixels of the processed image and the differential image, and outputs the compensated image as the output image of the thermal image processing device 100.

[0094] The input unit 15 is a processing unit for accepting input to the thermal image processing device 100. Specifically, the input unit 15 accepts the first temperature and the second temperature to be used in the first processing unit 12 as input from the user. Such a first temperature and second temperature can be arbitrarily set and input by the user, or can be input at the manufacturing stage of the device using the thermal image processing device 100. In the former case, for example, the input unit 15 is communicatively connected to the operation panel of the device using the thermal image processing device 100, and accepts the input of the first temperature and the second temperature from the user via the operation panel. In the present embodiment, it is also assumed that the input unit 15 accepts the input of the first temperature and the second temperature from the user via the remote control of the air conditioning device 201 for description.

[0095] The output unit 16 is a processing unit for outputting information such as a thermal image generated by the thermal image processing device 100. Specifically, the output unit 16 generates at least one of an image and temperature data for display on the display device 102. The output unit 16 sends the generated image and temperature data to the display device 102 via the communication module.

[0096] The acquisition unit 11 , the first processing unit 12 , the second processing unit 13 , the third processing unit 14 , the input unit 15 , and the output unit 16 are all implemented by executing a processing program corresponding to each processing unit by a processor or a dedicated circuit and memory.

[0097] Here, use Figure 5 The processing flow of the present disclosure from the thermal image to the generation of the output image will be described. Figure 51 is a schematic diagram for explaining the processing of thermal images according to an embodiment of the present invention. Figure 5 In the figure, rectangles are used to represent images, and circles with symbols are used to represent the processing of the image.

[0098] like Figure 5 As shown, first, the thermal image processing device 100 obtains the thermal image I LR The image is input to the temperature conversion image processing performed by the first processing unit 12 with the symbol h, and the conversion image I is generated. LR '. That is, I LR '=h(I LR ). Then, the thermal image processing device 100 generates the transformed image I LR ' is input to the high frequency emphasis processing performed by the second processing unit 13 with the symbol g, and the processed image I is generated HR '. That is, I HR '=g(I LR '). In addition, the high frequency emphasis processing here may also be a super-resolution processing that also serves as a processing for expanding the spatial resolution. That is, the second processing unit 13 may also perform the transformation image I LR 'Super-resolution processing for spatial resolution expansion.

[0099] On the other hand, the third processing unit 14 uses the thermal image I LR and the transformed image I LR ', generate the difference image I according to their difference CLR . That is, I CLR =I LR -I LR Here, in the case where super-resolution processing is performed in the processing with the symbol g, the processed image I HR ' and the difference image I CLR The resolutions of the images are different, so direct compensation processing (addition) cannot be performed. Therefore, in this case, it is necessary to CLR The third processing unit 14 also performs well-known resolution complement processing such as bicubic, bilinear, nearest, and neighbor. CLR Input to the resolution completion process with the symbol u to generate the extended differential image I CHR . That is, I CHR =u(I CLR ). The processed image I generated in this way HR 'With the extended difference image I CHR Add together to generate the output image I HR . That is, I HR =I HR '+I CHR .

[0100] In addition, in the processing with the symbol g, when only high-frequency emphasis processing is performed, such resolution supplementation processing is not necessary. In addition, when only the temperature between the first temperature and the second temperature is required and the temperature outside the range is not required, it is no longer necessary to obtain the resolution from the difference image I CLR The series of processing from the generation of to the compensation processing and the third processing unit 14. That is, as a minimum configuration, it is sufficient to implement the processing performed by the first processing unit 12 and the second processing unit 13. Figure 6 The processing of this minimum configuration is described in more detail. Figure 6 is a flowchart showing a thermal image processing method according to an embodiment.

[0101] like Figure 6 As shown, in the thermal image processing of this embodiment, first, a thermal image acquisition step (step S101) is implemented. The curve graphs shown in each step in the figure represent the thermal image acquisition step (step S101). Figure 3 The luminance value corresponding to the pixel position of each pixel on the A-A line is shown. In addition, the preset first temperature and second temperature are shown, and the pixel x1 representing the temperature y1 within the temperature range between the first temperature and the second temperature and the pixel x2 representing the temperature y2 outside the temperature range are also shown.

[0102] Next, the first processing unit 12 implements the first processing step (step S102) of image processing the acquired thermal image to lower the temperature represented by the high-temperature pixels and to raise the temperature represented by the low-temperature pixels. The content of the processing in the first processing step is the same as the description of the first processing unit 12, and the description here is omitted. Through the processing in the first processing step, the temperature y1 of the pixel x1 within the temperature range is the original temperature y1, and the temperature y2' of the pixel x2 outside the temperature range is different from the original temperature y2. Specifically, since the pixel x2 representing the temperature y2 is a high-temperature pixel, it is converted to the temperature y2' by lowering the temperature.

[0103] Next, the second processing unit 13 performs a second processing step (step S103) of performing high-frequency emphasis processing on the transformed image after the image processing in which the temperature conversion is performed. The content of the processing in the second processing step is the same as that described in the second processing unit 13, and the description here is omitted. In this way, the occurrence of ringing in the processed image after the high-frequency emphasis processing is suppressed, and the brightness value of each pixel is converted to a value close to the actual situation.

[0104] Here, by setting the first temperature and the second temperature, high-frequency emphasis processing is performed while maintaining the original brightness value in the temperature range where accuracy is required according to the purpose of the thermal image, so the accuracy of the temperature in the temperature range between the first temperature and the second temperature in the processed image is kept constant. In addition, since the occurrence of oscillation is suppressed as described above, the change in the brightness value of the pixels around the gradient part is suppressed, so from this point of view, the accuracy of the temperature is also kept constant. Therefore, in the thermal image processing device 100 or the thermal image processing module disclosed in the present invention, an appropriate thermal image is generated in which the accuracy of the temperature is kept constant in the temperature range where accuracy is required according to the purpose of the thermal image.

[0105] The high temperature conversion algorithm and the low temperature conversion algorithm are further described below. Fig. 7A is a diagram showing a first example of a conversion function according to an embodiment of the present invention. Figure 7B is a diagram showing a second example of a conversion function according to the embodiment. Figure 7C is a diagram showing a third example of the conversion function according to the embodiment. Fig.7D It is a diagram showing a fourth example of the conversion function according to the embodiment.

[0106] In the thermal image processing device 100, the high temperature conversion algorithm and the low temperature conversion algorithm (hereinafter also referred to as the temperature conversion algorithm) are implemented as conversion functions that can input continuous values. Figure 7A to Figure 7D In the temperature conversion algorithm in , a function is represented in which the high temperature difference value and the low temperature difference value calculated in the high temperature pixel and the low temperature pixel, respectively, are input as input values. Figure 7A to Figure 7D In the figure, the relationship between the input value and the output value on the x-y two-dimensional coordinate axis is shown, where the horizontal axis (x axis) takes the input value and the vertical axis (y axis) takes the output value. In addition, the output value here is the value added to the first temperature if the input value is a high temperature differential value, and the value added to the second temperature if the input value is a low temperature differential value. That is, in the temperature conversion algorithm, the new differential value is output to replace each differential value input.

[0107] The positive side of the input value corresponds to a high temperature differential value with a positive differential value, and is described as a high temperature transformation function. In addition, the negative side of the input value corresponds to a low temperature differential value with a negative differential value, and is described as a low temperature transformation function. Hereinafter, the high temperature transformation function and the low temperature transformation function are also referred to as transformation functions. In addition, for convenience, a point of input value 0 connecting the high temperature transformation function and the low temperature transformation function is added, and these transformation functions are described as a series of functions, but the high temperature transformation function and the low temperature transformation function may also be different functions.

[0108] In the present disclosure, any function that lowers the temperature of an original high-temperature pixel and raises the temperature of an original low-temperature pixel can achieve the above-mentioned effect. That is, any function that passes through a low temperature range in the first quadrant and a high temperature range in the third quadrant compared to a linear function with a slope of 1 can be used. In addition, in the case where a negative value is output for a positive input, and in the case where a positive value is output for a negative input in the transformation function, it means that values ​​outside the temperature range enter the temperature range through image processing of temperature transformation, and they should be excluded. FIG. 7A to FIG. 7D The functions shown all meet this condition. In addition, other functions can also be used as long as they meet the above conditions.

[0109] like Fig. 7A As shown, the first example of the conversion function is expressed by the following formula 1.

[0110] [Formula 1]

[0111] y=f(x), f(x)=0…(Formula 1)

[0112] In the above-mentioned equation 1, regardless of the input, the output value is 0. Therefore, in a pixel indicating a temperature higher than the first temperature, the temperature is fixed to the first temperature. In addition, in a pixel indicating a temperature lower than the second temperature, the temperature is fixed to the second temperature.

[0113] like Figure 7B As shown, the second example of the transformation function is expressed by the following formula 2.

[0114] [Formula 2]

[0115] y=f(x), f(x)=ax, (0<a<1)...(Formula 2)

[0116] In the above formula 2, the output value changes linearly in accordance with the input. Therefore, in the pixel indicating a temperature higher than the first temperature, it is converted to a temperature corresponding to the original observed value. In addition, in the pixel indicating a temperature lower than the second temperature, it is converted to a temperature corresponding to the original observed value.

[0117] like Figure 7C As shown, the third example of the transformation function is expressed by the following formula 3.

[0118] [Formula 3]

[0119]

[0120] In the above formula 3, the output value changes in a curve shape in response to the input. Therefore, in the pixel indicating a temperature higher than the first temperature, it is converted to a temperature corresponding to the original observed value. In addition, in the pixel indicating a temperature lower than the second temperature, it is converted to a temperature corresponding to the original observed value.

[0121] like Fig.7D As shown, the fourth example of the transformation function is expressed by the following formula 4.

[0122] [Formula 4]

[0123]

[0124] In the above formula 4, the output value changes in a curve corresponding to the input. Therefore, in the pixel representing a temperature higher than the first temperature, it is converted to a temperature corresponding to the original observed value. In addition, in the pixel representing a temperature lower than the second temperature, it is converted to a temperature corresponding to the original observed value. In addition, image processing for temperature conversion can also be performed without using such a function. That is, the temperature conversion algorithm can also be a conversion table that establishes a correspondence between the input value and the output value. The first processing unit 12 can also obtain the output value by referring to such a conversion table based on the input value. This can be used, for example, in a case where the temperature range to be observed is limited and the temperature as a brightness value is obtained in discrete numerical values ​​such as 1°C scale.

[0125] Here, use Figure 8 and Fig. 9 , for the use of the above Fig. 7A and Fig.7D The image processing of the temperature range of the transformation function is explained. Figure 8 FIG. 1 is a diagram illustrating image processing according to an embodiment of the present invention. Fig. 9 FIG. 2 is a second diagram illustrating image processing according to an embodiment of the present invention. Figure 8 and Fig. 9 In each figure, (a) to (c) show the Figure 6 The graphs of the steps of the flowchart shown correspond to the graphs of the graphs.

[0126] For example, as mentioned above, Figure 8 Use shown Fig. 7A In the example of the conversion function of , in the pixel representing a temperature higher than the first temperature, the temperature is fixed to the first temperature, and in the pixel representing a temperature lower than the second temperature, the temperature is fixed to the second temperature. Therefore, the generated conversion image has a table-like plane, and the accuracy of the temperature is not maintained outside the temperature range between the first temperature and the second temperature. However, since the calculation load is small, the thermal image processing device 100 can be implemented even if a processor with relatively low processing power is used.

[0127] On the other hand, for example, as described above, Fig. 9 Use shown Fig.7D In the example of the transformation function, in the pixel representing a temperature higher than the first temperature, it is transformed into a temperature corresponding to the original observed value. In addition, in the pixel representing a temperature lower than the second temperature, it is transformed into a temperature corresponding to the original observed value. Therefore, the generated transformation image is a shape that maintains a shape close to the original thermal image and in which the amplitude of the temperature is suppressed. In addition, since the temperatures between the pixels across the first temperature are also smoothly connected after the image processing of the temperature transformation, the parts of the temperature change that are newly bent by the image processing are suppressed.

[0128] Considering the above situation, through Fig.10 The conditions for a function to be suitable as a transformation function are described. Fig.10 This is a diagram for explaining the conversion function of the related implementation method. Fig.10 , three examples showing suitable examples (left column) and unsuitable examples (center and right columns) as conversion functions are shown, and the relationship between the temperatures before and after the conversion in the image processing of the temperature conversion in each case is shown.

[0129] like Fig.10 As shown, in a function suitable as a conversion function, when the input value is 0, the output value is 0 (f(0) = 0). When the input difference value is 0, the output is 0, and after image processing, the discreteness of the value at the temperature conversion boundary between the first temperature and the second temperature can be suppressed. When this is not true (f(0) ≠ 0), since the value is discrete at the temperature conversion boundary between the first temperature and the second temperature, a new high-frequency component will be included in the conversion image.

[0130] In addition, if Fig.10 As shown in FIG. 1 , in a function suitable as a conversion function, when the input value is 0, the differential value of the output value is 1 (f'(0) = 1). That is, the slope of the tangent line of the function at the position where the input differential value is 0 is 1, and after image processing, the rapid change of the value at the temperature conversion boundary between the first temperature and the second temperature can be suppressed. When this is not true (f'(0) ≠ 1), since the value changes rapidly at the temperature conversion boundary between the first temperature and the second temperature, a new high-frequency component will be included in the conversion image.

[0131] (Other embodiments)

[0132] The thermal image processing device and the like related to the present disclosure are described above based on the above-mentioned embodiments, but the present disclosure is not limited to the above-mentioned embodiments and various modifications. For example, the above-mentioned embodiments and the like are subjected to various modifications that a person skilled in the art would like to obtain, or the embodiments and the like are realized by arbitrarily combining the constituent elements and functions of the embodiments and the like within the scope of the present disclosure.

[0133] For example, in the above-mentioned embodiment, the process executed by a specific processing unit may be executed by another processing unit. In addition, the order of a plurality of processes may be changed, or a plurality of processes may be executed in parallel.

[0134] In addition, the thermal image processing device of the present disclosure may be implemented by a plurality of devices having a part of each of the plurality of constituent elements, or by a single device having all of the plurality of constituent elements. In addition, a part of the function of a constituent element may be implemented as a function of another constituent element, and the functions may be distributed to each constituent element in any manner. As long as the form has all the components that can substantially implement the functions of the thermal image processing device of the present disclosure, it is included in the present disclosure.

[0135] In addition, in the above-mentioned embodiments, each component can also be implemented by executing a software program suitable for each component. Each component can also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory.

[0136] In addition, each component may also be implemented by hardware. For example, each component may also be a circuit (or integrated circuit). These circuits may constitute one circuit as a whole, or they may be different circuits. In addition, these circuits may be general circuits or dedicated circuits.

[0137] In addition, these global or specific technical solutions can also be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs. In addition, they can also be implemented by any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0138] Furthermore, in the above-mentioned embodiment, an example of setting the temperature range of image processing to which temperature transformation is applied using the first temperature and the second temperature is described, but the temperature range of image processing to which temperature transformation is applied may also be set using only the first temperature. For example, as the use of the thermal image obtained, it is conceivable to detect the human body in a room, etc. In such use, since it is rare to include extremely low temperature parts such as those that produce oscillations, the second temperature may not be set. On the other hand, in such use, since it is easy to conceive that extremely high temperature parts such as stoves, lamps, and heating appliances are included, the setting of the first temperature is useful. In this way, the second temperature is not necessary, and it can be appropriately set according to the situation.

[0139] The first temperature and the second temperature may be set by accepting input of separate values. On the other hand, as described above, the first temperature and the second temperature can be estimated to some extent as normal values ​​corresponding to the purpose. Therefore, from the perspective of user convenience, the thermal image processing device may set both the first temperature and the second temperature by accepting selection of a combination of the first temperature and the second temperature prepared in advance. Fig.11 An example of a combination of the first temperature and the second temperature will be described. Fig.11 This is a diagram showing an example of a combination of a first temperature and a second temperature according to an embodiment.

[0140] like Fig.11 As shown, in the case of a "room mode" in which the use of a device with a built-in thermal image processing device is to detect the spatial temperature of a room by thermal images, for example, a combination of 40°C as the first temperature and 20°C as the second temperature is used, which is a combination that can be seen in a normal room. In addition, in the case of a "vehicle mode" in which the spatial temperature of a vehicle compartment is detected by thermal images, for example, a combination of 50°C as the first temperature and 10°C as the second temperature is used, which is a combination that can be seen in a normal vehicle compartment. In addition, in the case of a "human detection mode" in which a human in a space is detected by thermal images, for example, a combination of 40°C as the first temperature and 32°C as the second temperature is used, which is a combination that can be seen at a normal human body temperature.

[0141] Such combination options may be displayed on a display device mounted on a remote controller of the air conditioning device, for example, and one combination may be selected by operating the remote controller.

[0142] In addition to the above, the "unrestricted mode" in which the settings of the first temperature and the second temperature are released may be used. In this way, in the setting of the first temperature and the second temperature, the first temperature and the second temperature may be set by selecting one combination from a plurality of combinations of the first temperature and the second temperature. In addition, in the case of using only the first temperature without setting the second temperature as described above, the first temperature may be set by the user selecting a purpose from displayed purposes as in the "room mode".

[0143] In addition, in the above-mentioned embodiment, the display device is described as being mounted on a remote controller of an air conditioning device, but it can also be implemented by a portable terminal such as a PC, a smart phone, or a tablet terminal. In addition, it can also be implemented as an application that implements the above-mentioned functions on the portable terminal. That is, the processing function of the thermal image processing device can also be executed by a processor possessed by the portable terminal.

[0144] In addition, when the thermal image is used only for internal processing of the device, a display device is not necessary, so a thermal image processing module that does not have a display device but only has a thermal image sensor and a thermal image processing device can be realized.

[0145] Industrial Applicability

[0146] The thermal image processing device and the like disclosed in the present invention can be used in a device and the like that changes its operation mode and the like using an acquired thermal image.

[0147] Description of symbols

[0148] 11 Acquisition Department

[0149] 12. Processing Unit 1

[0150] 13. Second processing unit

[0151] 14. Processing Unit 3

[0152] 15 Input section

[0153] 16 Output

[0154] 100 Thermal image processing device

[0155] 101 Thermal Image Sensor

[0156] 102 Display device

[0157] 200 rooms

[0158] 201 Air conditioning device

Claims

1. A thermal image processing device, wherein: have: An acquisition unit, which acquires a thermal image from a thermal image sensor; a first processing unit that performs image processing to lower the temperature represented by the high-temperature pixels, which represent a temperature higher than a first temperature, among a plurality of pixels included in the acquired thermal image; as well as The second processing unit performs a high-frequency emphasis process for emphasizing a high-frequency component included in the transformed image, on the thermal image after the image processing, that is, the transformed image.

2. The thermal image processing device according to claim 1, wherein: During the image processing, the first processing unit further increases the temperature of low-temperature pixels indicating a temperature lower than a second temperature among a plurality of pixels included in the acquired thermal image, wherein the second temperature is lower than the first temperature.

3. The thermal image processing device according to claim 2, wherein: The device further includes an input unit for setting at least one of the first temperature and the second temperature.

4. The thermal image processing device according to claim 3, wherein: In setting of at least one of the first temperature and the second temperature, the first temperature and the second temperature are set by one combination selected from a plurality of combinations of the first temperature and the second temperature.

5. The thermal image processing device according to claim 2, wherein: In the above image processing, calculating a low temperature difference value obtained by subtracting the second temperature from the temperature represented by the low temperature pixel among the plurality of pixels included in the thermal image, The temperature represented by the low-temperature pixel is converted using a value outputted in response to the input of the low-temperature difference value, using a low-temperature conversion algorithm that outputs a value higher than the input value.

6. The thermal image processing device according to claim 1, wherein: In the above image processing, calculating a high temperature difference value obtained by subtracting the first temperature from the temperature represented by the high temperature pixel among the plurality of pixels included in the thermal image, The temperature represented by the high temperature pixel is converted using a high temperature conversion algorithm that outputs a value lower than the input value, with respect to the input high temperature difference value.

7. The thermal image processing device according to claim 5, wherein: In the above image processing, calculating a high temperature difference value obtained by subtracting the first temperature from the temperature represented by the high temperature pixel among the plurality of pixels included in the thermal image, The temperature represented by the high temperature pixel is converted by the value outputted in response to the input of the high temperature difference value using a high temperature conversion algorithm that outputs a value lower than the input value. The low-temperature conversion algorithm and the high-temperature conversion algorithm are conversion functions that can input continuous values.

8. The thermal image processing device according to claim 5, wherein: In the above image processing, calculating a high temperature difference value obtained by subtracting the first temperature from the temperature represented by the high temperature pixel among the plurality of pixels included in the thermal image, The temperature represented by the high temperature pixel is converted by the value outputted in response to the input of the high temperature difference value using a high temperature conversion algorithm that outputs a value lower than the input value. The low-temperature conversion algorithm and the high-temperature conversion algorithm are conversion tables that establish correspondence between input values ​​and output values.

9. The thermal image processing device according to claim 7, wherein: The above conversion function outputs a value of 0 when the input value is 0, and the differential value of the output value is 1 when the input value is 0.

10. The thermal image processing device according to claim 1, wherein: It also includes a third processing unit, which calculates a differential image by subtracting corresponding pixels of the acquired thermal image and the transformed image, and performs compensation processing by adding corresponding pixels of the transformed image after the high-frequency emphasis processing and the calculated differential image.

11. The thermal image processing device according to any one of claims 1 to 10, wherein: The second processing unit further performs super-resolution processing for increasing the spatial resolution of the transformed image.

12. A thermal image processing module, wherein: have: The thermal image sensor described above; and The thermal image processing device as claimed in claim 1.

13. The thermal image processing module according to claim 12, wherein: The device further includes a display device that displays the converted image after the high-frequency emphasis processing as at least one of an image and temperature data.

14. A thermal image processing method, wherein: include: An acquisition step of acquiring a thermal image from a thermal image sensor; A first processing step of performing image processing to lower the temperature represented by the high temperature pixels, which represent a temperature higher than a first temperature, among a plurality of pixels included in the acquired thermal image; as well as The second processing step is to perform a high-frequency emphasis process on the thermal image after the image processing, that is, the transformed image, for emphasizing the high-frequency components included in the transformed image.

15. A program recording medium, wherein: A program for causing a computer to execute the thermal image processing method according to claim 14 is recorded.

Images