Infrared image processing device and infrared image processing method

Abstract

Provided is an infrared image processing device that can reduce stripe noise by calculating a correction coefficient for correcting stripe noise based on a plurality of images. The infrared image processing device includes: a thermal image sensor that receives infrared rays and outputs a signal corresponding to the infrared rays; a thermal image generating unit that generates a plurality of thermal images based on the signal; a smoothing processing unit that performs smoothing processing using pixel values ​​of surrounding pixels for each pixel of each of the plurality of thermal images, calculates a plurality of smoothed images, and calculates each smoothed pixel value of each image, i.e., a smoothed pixel value; a correction coefficient calculating unit that calculates a correction coefficient group including a first correction coefficient and a second correction coefficient based on the thermal image and the smoothed image; and a thermal image correcting unit that corrects the thermal image using the correction coefficient group.

Description

Technical Field

[0001] The present disclosure relates to an infrared image processing device and an infrared image processing method for performing noise reduction processing on a thermal image. Background Art

[0002] An infrared image processing device that generates a thermal image by receiving infrared rays has a plurality of infrared detection elements for receiving infrared rays. For example, the infrared detection elements are arranged in a matrix in a manner corresponding to the generated image. The infrared detection elements arranged in a matrix are connected to different drive lines for each row or column of the element arrangement, and power is supplied from each drive line. When there is a deviation in the characteristics of these drive lines, the relationship between the input value to the infrared detection element and the output value from the infrared detection element changes for each row or column. Due to this change, striped noise extending in the row direction or column direction is sometimes generated in the thermal image generated by the infrared image processing device.

[0003] In order to reduce such streak noise, the following method is proposed in the imaging device of Patent Document 1. The imaging device of Patent Document 1 captures images in a light-shielded state at regular intervals and measures output signals when light is shielded and when light is exposed. Then, the imaging device of Patent Document 1 generates a correction signal corresponding to the offset component and the light-amount-dependent component of the noise based on the measured value and the ideal value, thereby reducing the noise.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2009-105966 Summary of the invention

[0007] Problems to be solved by the invention

[0008] As in the imaging device of Patent Document 1 mentioned above, when an image in a light-shielded state is photographed at regular intervals and the light-shielded image is used for correction, the noise characteristics change moment by moment due to changes in the ambient temperature and the like. Therefore, in the imaging device, the effect of suppressing streak noise in the imaging device decreases due to the passage of time from the light-shielded state (the shutter time). As a result, streak noise gradually occurs in the imaging device. Generally speaking, in an imaging device, the image becomes darker at the shutter time, so it is preferred that the shutter frequency is not increased. However, if the shutter frequency is not increased, the light-shielded image cannot be frequently used for correction in the imaging device. In this case, in the imaging device, the correction accuracy of the noise over the total time is reduced.

[0009] The present disclosure has been made to solve the above-mentioned problem, and its object is to provide an infrared image processing device that can calculate a correction coefficient for correcting streak noise from a plurality of images and can reduce streak noise with higher accuracy using the calculated correction coefficient.

[0010] Means for solving problems

[0011] The infrared image processing device disclosed in the present invention is characterized in that it has: a thermal image sensor, which receives infrared rays and outputs a signal corresponding to the infrared rays; a thermal image generating unit, which generates a plurality of thermal images based on the signal; a smoothing processing unit, which performs smoothing processing on each pixel of each of the plurality of thermal images using the pixel values ​​of the surrounding pixels, calculates a plurality of smoothed images, and calculates each smoothed pixel value, i.e., the smoothed pixel value; a correction coefficient calculating unit, which calculates a correction coefficient group including a first correction coefficient and a second correction coefficient based on the thermal image and the smoothed image; and a thermal image correction unit, which uses the correction coefficient group to correct the thermal image.

[0012] Effects of the Invention

[0013] The infrared image processing device of the present disclosure calculates a correction coefficient based on the difference in pixel values ​​of pixels included in a plurality of images, and can reduce streak noise with higher accuracy using the calculated correction coefficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a block diagram showing the structure of the infrared image processing device according to the first embodiment.

[0015] Figure 2 : is a graph showing the relationship between the input value and the output value of the thermal image sensor according to the first embodiment.

[0016] Figure 3 This is a conceptual diagram for explaining the smoothing process performed by the infrared image processing device according to the first embodiment.

[0017] Figure 4 This is a conceptual diagram for explaining a thermal image and its target pixel value sequence in the infrared image processing device according to the first embodiment.

[0018] Figure 5 This is a conceptual diagram for explaining a smoothed thermal image and a target pixel value sequence thereof in the infrared image processing device according to the first embodiment.

[0019] Figure 6 This is a conceptual diagram for explaining a method of calculating a correction coefficient in the infrared image processing device according to the first embodiment.

[0020] Figure 7This is a flowchart showing the process up to outputting a smoothed image by the infrared image processing device 1 according to the first embodiment.

[0021] Figure 8 This is a flowchart showing the process up to the output correction coefficient of the infrared image processing device 1 according to the first embodiment.

[0022] Fig. 9 This is a flowchart until the infrared image processing device 1 of the first embodiment outputs a corrected thermal image.

[0023] Fig.10 This is a block diagram showing the structure of an infrared image processing device according to the second embodiment.

[0024] Fig.11 It is a schematic diagram of a thermal image sensor and a sensor moving unit of an infrared image processing device according to a second embodiment, and a schematic diagram of a thermal image.

[0025] Fig.12 This is a diagram showing an example of the hardware configuration of the respiratory information estimating device according to the second embodiment.

[0026] Fig.13 This is a diagram showing another example of the hardware configuration of the respiratory information estimating device according to the second embodiment. DETAILED DESCRIPTION

[0027] Hereinafter, the embodiments will be described with reference to the drawings. The same reference numerals in the drawings represent the same or corresponding parts.

[0028] Implementation Method 1

[0029] The configuration of the infrared image processing device 1 according to the first embodiment will be described. Figure 1 It is a block diagram showing the structure of the infrared image processing device of embodiment 1. The infrared image processing device 1 has a thermal image sensor 2 and a thermal image processing device 3. The thermal image processing device 3 has a thermal image generating unit 31, a smoothing processing unit 32, a correction coefficient calculating unit 33 and a thermal image correcting unit 34. The thermal image sensor 2 sends a voltage signal VI to the thermal image generating unit 31. The thermal image generating unit 31 sends a thermal image TI to the smoothing processing unit 32, the correction coefficient calculating unit 33 and the thermal image correcting unit 34. The smoothing processing unit 32 sends a smoothed image FI to the correction coefficient calculating unit 33. The correction coefficient calculating unit 33 sends a correction coefficient c and a correction coefficient d to the thermal image correcting unit 34 via a memory or the like. The thermal image correcting unit 34 sends the corrected thermal image TO to an output destination as an output of the thermal image processing device 3 and the infrared image processing device 1.

[0030] The thermal image sensor 2 is installed in an electrical product installed indoors, and takes pictures of the indoor space at regular intervals when the electrical product is working. Next, the thermal image sensor 2 receives infrared rays (electromagnetic waves of approximately 8μm to 12μm) radiated from objects present in the room, and outputs a voltage signal VI corresponding to the intensity of the infrared rays. The thermal image sensor 2 is composed of a plurality of infrared detection elements that detect infrared rays radiated from an object and output a voltage signal VI corresponding to the intensity of the infrared rays. The infrared detection element is, for example, a pyroelectric element. The plurality of infrared detection elements are arranged in a matrix, and a drive line is connected to the elements arranged in the row direction, and power is supplied. The thermal image sensor 2 sends the voltage signal VI output by the infrared detection element to the thermal image processing device 3.

[0031] The thermal image sensor 2 sends the voltage signal VI corresponding to the infrared ray to the thermal image processing device 3 in an order corresponding to the configuration of the infrared detection element. In addition, since the electrical product is working, the electrical product itself or the objects around it become hot. The thermal image TI is generated by the thermal image processing device 3 based on the voltage signal VI of the thermal image sensor 2, and becomes an image obtained by photographing multiple areas with different temperatures in the room under exposure. The thermal image TI contains multiple pixels with different pixel values ​​from the first frame to the second frame at a certain specific pixel position. For example, the first frame and the second frame are adjacent frames. The frame number is the number used to represent the image of frame number 1 to the image of frame number 10, for example, when there are 10 images. In addition, the pixel value is a value obtained by converting the voltage value of the voltage signal VI into a digital value. Specifically, the pixel value is obtained by the following method. The thermal image processing device 3 stores the voltage value of the voltage signal VI in Figure 1 Then, the thermal image processing device 3 reads the voltage value and the order information, and converts the voltage value into a pre-correlated pixel value. Then, the thermal image processing device 3 arranges the pixel values ​​using the order information. The thermal image processing device 3 generates a thermal image based on the arranged pixel values ​​and stores it in a memory, etc., not shown in the figure.

[0032] Figure 2 : is a graph showing the relationship between the input value and the output value of the thermal image sensor of Embodiment 1. The infrared detection element described here is affected by, for example, manufacturing errors, usage environment, etc. Therefore, in the infrared detection element, the relationship between the input value (infrared intensity) input to the infrared detection element and the output value (pixel value of the thermal image) output from the infrared detection element is different. Figure 2As shown, the relationship between the input value and the output value of each infrared detection element can be approximately expressed by a linear function y=ax+b (y: output value, x: input value, a and b: coefficients). The coefficient a and coefficient b of the linear function are different for each infrared detection element. For example, infrared detection element E can be expressed by a linear function using coefficient a1 and coefficient b1. In addition, infrared detection element F can be expressed by a linear function using coefficient a2 and coefficient b2. In addition, infrared detection element G can be expressed by a linear function using coefficient a3 and coefficient b3. The output value of the infrared detection element E in the case of an input value X1 is Y1 (=a1·X1+b1). The output value of the infrared detection element G in the case of an input value X1 is Y3 (=a3·X1+b3). As the relationship between the output value and the input value X1, the output value Y3 of the infrared detection element G is greater than the output value Y1 of the infrared detection element E.

[0033] As described above, the infrared detection elements arranged in the row direction are connected to the same drive line and receive power supply. Therefore, the coefficient a representing the sensitivity of the infrared detection elements arranged in the row direction is affected by the characteristics of the drive line. If the infrared detection elements arranged in the row direction have a larger slope than other infrared detection elements, they become highly sensitive compared to other infrared detection elements. In addition, the coefficient b as the intercept component also changes according to the infrared detection element in the same way as the coefficient a. As described above, the output values ​​of the infrared detection elements arranged in the row direction vary respectively. In particular, the different characteristics of the drive line in the upper and lower rows will affect the upper and lower output values ​​of the infrared detection elements. In this case, striped noise (hereinafter referred to as striped noise) extending along the row direction from one end to the other end of the thermal image is generated in the thermal image. In addition, in the thermal image sensor, due to the temporal change of the characteristics of the drive line, the coefficient a representing the sensitivity of the infrared detection elements arranged in the row direction or the coefficient b as the intercept component changes. Therefore, the intensity of the striped noise also changes over time.

[0034] The thermal image processing device 3 receives the voltage signal VI from the thermal image sensor 2. That is, the thermal image generation unit 31 closest to the input unit in the thermal image processing device 3 receives the voltage signal VI from the thermal image sensor 2. The thermal image generation unit 31 converts the voltage signal VI of the thermal image sensor 2 into a pixel value. Next, the thermal image generation unit 31 generates a thermal image TI including a plurality of pixels with different pixel values ​​in the direction in which striped noise is generated based on the converted pixel values. Then, the thermal image generation unit 31 sends the generated thermal image TI to the smoothing processing unit 32, the correction coefficient calculation unit 33 and the thermal image correction unit 34.

[0035] The smoothing processing unit 32 performs smoothing processing on the thermal image TI using the adjacent pixel values ​​according to each object pixel value. The smoothing processing unit 32 generates a smoothed image FI after the smoothing processing and sends it to the correction coefficient calculation unit 33. The above-mentioned object pixel is a pixel at a pixel position that becomes a reference when performing smoothing processing, correction coefficient calculation processing or correction processing. The object pixel is included in a set of pixels arranged in the direction of generating stripe noise in the thermal image TI generated by the thermal image generation unit 31, that is, a pixel column. The above-mentioned adjacent pixels are included in the adjacent pixel columns in the direction intersecting the direction of generating stripe noise. The adjacent pixels are adjacent to the object pixel. The smoothed pixel value is the average value of the object pixel value and the adjacent pixel value.

[0036] The processing of the smoothing processing unit 32 will be specifically described. Figure 3 This is a conceptual diagram for explaining the smoothing process performed by the infrared image processing device 1 according to the first embodiment. Figure 3 (a) shows a thermal image of L rows and N columns. Figure 3 (b) shows a smoothed thermal image with L rows and N columns. The row with the object pixel value in the L rows is set to the L1th row. Figure 3 When smoothing the object pixel value of (a), the smoothing processing unit 32 calculates the average value of the object pixel value PX1 and the corresponding adjacent pixel values ​​PX2 and PX3. The average value is calculated using the leftmost object pixel value and the leftmost adjacent pixel value of the pixel column of the L1th row. The object pixel values ​​and adjacent pixel values ​​from the second one from the left of the pixel column of the L1th row are processed in the same way to calculate the next average value. For example, Figure 3 The average value is calculated within the range of frame H in (a).

[0037] By performing the above-mentioned smoothing process, Figure 3 As shown in (b), a smoothed thermal image consisting of smoothed pixel values ​​with reduced stripe noise in the L1th row is obtained. The distribution of infrared rays in the L1th row is correlated with the distribution of infrared rays in the adjacent L1-1th row and L1+1th row. Based on this correlation, in the smoothing processing unit 32, the correspondence between each object pixel value contained in the Lth row and the smoothed pixel value contained in the smoothed thermal image is maintained. Thus, the smoothing processing unit 32 calculates the smoothed pixel value after reducing the stripe noise. Let the smoothed pixel value be z, and calculate the correction coefficient of the correction formula z=cm+d for each row as described below.

[0038] The correction coefficient calculation unit 33 calculates the correction coefficient c and the correction coefficient d. The correction coefficient c and the correction coefficient d are correction coefficients included in the correction formula z=cm+d. The correction coefficient c can also be called the first correction coefficient, the correction coefficient d can also be called the second correction coefficient, and the first correction coefficient and the second correction coefficient can also be called a correction coefficient group. In addition, the pixel value z is the corrected pixel value, and the pixel value m is the pixel value of the thermal image. The correction coefficient calculation unit 33 stores the calculated correction coefficient c and correction coefficient d in a memory not shown in the figure, etc. The thermal image correction unit 34 uses the correction coefficient c and correction coefficient d read from the memory to correct the image TI generated by the thermal image generation unit 31.

[0039] The specific processing of the correction coefficient calculation unit 33 will be described. Figure 4 This is a conceptual diagram for explaining a thermal image and its target pixel value sequence in the infrared image processing device according to the first embodiment. Figure 5 This is a conceptual diagram for explaining the smoothed thermal image and its object pixel value column in the infrared image processing device of the first embodiment. For example, the case where the coordinate value is set according to the object pixel value of the Lth row and Nth column and the smoothed pixel value is explained. The correction coefficient calculation unit 33 reads the object pixel value column from the pixel of the Lth row and Nth column of the first frame to the pixel of the Lth row and Nth column of the second frame of the thermal image. Figure 4 (a) In such a thermal image, L=1, N=1, that is, the target pixel setting of the first row and the first column is Figure 4 In addition, the correction coefficient calculation unit 33 reads out the smoothed pixel values ​​from the pixel of the Lth row and Nth column of the first frame to the pixel of the Lth row and Nth column of the second frame of the smoothed thermal image. Then, the correction coefficient calculation unit 33 calculates the smoothed pixel values ​​according to Figure 5 (a) In the smoothed thermal image, L=1, N=1, that is, the target pixel setting of the first row and the first column is Figure 5 Then, the correction coefficient calculation unit 33 uses the object pixel value and the corresponding smoothed pixel value to set the coordinate value (for example, (R1, R1h)) in which the object pixel value is set to the m coordinate and the smoothed pixel value is set to the z coordinate.

[0040] Figure 6This is a conceptual diagram for explaining a method of calculating a correction coefficient in an infrared image processing device according to Embodiment 1. The correction coefficient calculation unit 33 depicts a set coordinate group in the mz coordinate space, performs a straight line approximation, and obtains a straight line formula, i.e., a correction formula z=cm+d. The correction coefficient calculation unit 33 calculates a correction coefficient c and a correction coefficient d. The correction coefficient c and the correction coefficient d are different according to each pixel position. Here, a plurality of coordinate values ​​are depicted at different positions. As described above, a plurality of thermal images contain a plurality of pixels having different pixel values ​​in the direction (row direction) in which streak noise is generated, and therefore, the depicted positions are offset. The correction coefficient calculation unit 33 calculates the correction coefficient c and the correction coefficient d based on the difference in the position, i.e., the difference in the pixel value. The correction coefficient calculation unit 33 calculates the correction coefficient c and the correction coefficient d so as to minimize the distance between the set coordinate value and the straight line formula z=cm+d. The correction coefficient calculation unit 33 uses, for example, the least square method in calculating the correction coefficient c and the correction coefficient d. The correction coefficient calculation unit 33 stores the correction coefficients c and d in a memory, etc., not shown in the figure.

[0041] The thermal image correction unit 34 obtains the pixel value z in the thermal image TI generated by the thermal image generation unit 31. Here, the pixel value z is a pixel value related to the pixel column from the object pixel of the first frame to the object pixel of the second frame. Next, the thermal image correction unit 34 substitutes the pixel column from the object pixel of the first frame to the object pixel of the second frame into the correction formula using the stored correction coefficient c and correction coefficient d to obtain the pixel value z. The thermal image correction unit 34 outputs the thermal image TO obtained by outputting the pixel value with respect to the image TI to the output destination.

[0042] Next, use Figure 7 , Figure 8 and Fig. 9 The procedure of processing by the infrared image processing device 1 according to the first embodiment will be described. Figure 7 This is a flowchart showing the process up to the output of the smoothed image FI by the infrared image processing device 1 according to the first embodiment. Figure 7 The flowchart of illustrates the operation of the thermal image sensor 2 in the infrared image processing device 1 and the smoothing processing unit 32 of the thermal image processing device 3. The infrared image processing device 1 generates a thermal image TI, performs smoothing processing on the thermal image TI using adjacent pixel values ​​according to each object pixel value, and outputs a smoothed image FI.

[0043] First, in step S001 , the thermal image generation unit 31 generates a thermal image TI according to the voltage signal VI of the thermal image sensor 2 .

[0044] Next, in step S002, the smoothing processing unit 32 obtains the pixel values ​​of the pixel columns of the Lth row, the L-1th row, and the L+1th row in the plurality of thermal images generated by the thermal image generating unit 31. Here, L is a natural number. At the beginning of the processing of this flowchart, the infrared image processing device 1 is set to L=1 as an initial value.

[0045] In addition, when L=1, that is, when the pixel values ​​of the pixel columns of the uppermost section of the thermal image are read, the L-1th row in the thermal image does not exist. Therefore, for example, it is instead treated as if the same pixel column as the Lth row or the L+1th row exists in the L-1th row in the thermal image for processing. That is, the pixel column of the Lth row or the L+1th row read out separately is processed as the pixel column of the L-1th row in the thermal image. The same is true for the lower section of the thermal image, where the pixel column of the L+1th row does not exist, so the pixel column of the Lth row or the L-1th row read out separately is processed as the pixel column of the L+1th row in the thermal image. In addition, here, the pixel values ​​contained in the Lth row are the object pixel values, and the pixel values ​​contained in the L-1th row and the L+1th row are the adjacent pixel values.

[0046] Next, in step S003, the smoothing processing unit 32 smoothes each object pixel value using the adjacent pixel values ​​of the pixels in the L-1th and L+1th rows adjacent to each object pixel in the Lth row. Then, the smoothing processing unit 32 obtains the smoothed pixel values ​​of each pixel in the Lth row.

[0047] Next, in step S004, the smoothing processing unit 32 replaces L with L+1. Next, in step S005, the smoothing processing unit 32 determines whether there is a pixel column in the Lth row replaced with L+1 in the previous step. If there is a pixel column (if "yes" in step S005), the processing of steps S002 to S004 is repeated. If there is no pixel column (if "no" in step S005), the process proceeds to the next step.

[0048] Figure 8 This is a flowchart showing the processing up to the output correction coefficient c and the correction coefficient d of the infrared image processing device 1 according to the first embodiment. Figure 8 The flowchart of illustrates the operation of the correction coefficient calculation unit 33 of the thermal image processing device 3 in the infrared image processing device 1. The correction coefficient c and the correction coefficient d are correction coefficients included in the correction formula z=cm+d. The pixel value z is the pixel value after correction, and the pixel value m is the pixel value of the thermal image. The correction coefficient calculation unit 33 calculates the correction coefficient c and the correction coefficient d.

[0049] Step S006 is a step following step S005. In step S006, the correction coefficient calculation unit 33 obtains the pixel values ​​of L rows and N columns and the smoothed pixel values ​​of L rows and N columns, and sets a plurality of coordinate values ​​using the corresponding object pixel columns and smoothed pixel columns. The pixel values ​​of L rows and N columns are the pixel values ​​associated with the pixel columns from the Lth row and N columns of the first frame to the Lth row and N columns of the second frame. The smoothed pixel values ​​of L rows and N columns are the smoothed pixel values ​​associated with the pixel columns from the Lth row and N columns of the first frame to the Lth row and N columns of the second frame. L and N are natural numbers. As initial values ​​at the start of processing by the infrared image processing device 1, the infrared image processing device 1 sets L and N to L=1 and N=1.

[0050] Next, in step S007, the correction coefficient calculation unit 33 depicts the set coordinate group in the mz coordinate space, performs straight line approximation, and obtains the straight line formula z=cm+d. Here, multiple coordinate values ​​are depicted at different positions. As described above, multiple thermal images contain multiple pixels with different pixel values ​​in the direction (row direction) in which striped noise is generated, so the depicted positions are offset. The correction coefficient calculation unit 33 calculates the correction coefficients c and d based on the difference in the position, that is, the difference in the pixel value. The correction coefficient calculation unit 33 uses the least squares method to minimize the distance between the set coordinate value and the straight line formula z=cm+d.

[0051] Next, in step S008, the correction coefficient calculation unit 33 performs a process of replacing L with L+1. Next, in step S009, the correction coefficient calculation unit 33 determines whether there is a pixel column in the Lth row that was replaced with L+1 in the previous step. When there is a pixel column in the Lth row (in the case of "Yes" in step S009), the process of steps S006 to S009 is repeated. In the case that there is no pixel column in the Lth row (in the case of "No" in step S009), the correction of the pixel column of the pixels in the Nth column is completed, and the process is transferred to the next step S010.

[0052] Next, in step S010, the correction coefficient calculation unit 33 performs a process of replacing N with N+1. Next, in step S011, the correction coefficient calculation unit 33 determines whether there is a pixel column in the Nth column replaced with N+1 in the previous step. If there is a pixel column in the Nth column (if "yes" in step S011), the process of steps S006 to S011 is repeated.

[0053] When there is no pixel column in the Nth column (“No” in step S011 ), the calculation of the correction coefficients c and d is completed in all rows of the thermal image, and the process proceeds to the next step.

[0054] Fig. 9This is a flowchart of the infrared image processing device 1 according to the first embodiment until the output of the corrected thermal image TO. Fig. 9 The flowchart explains the operation until the thermal image correction unit 34 in the infrared image processing device 1 outputs the corrected thermal image TO.

[0055] Step S012 is a step following step S011. In step S012, the thermal image correction unit 34 obtains the pixel value of the pixel in the Pth row and Qth column. The pixel value of the pixel in the Pth row and Qth column is each pixel value related to the pixel column from the pixel in the Pth row and Qth column of the first frame to the pixel in the Pth row and Qth column of the second frame in the thermal image. P and Q are natural numbers. When the processing of the infrared image processing device 1 starts, P and Q are set to P=1 and Q=1 as initial values.

[0056] Next, in step S013, the thermal image correction unit 34 uses the corresponding correction formula to correct the pixel values ​​of the pixels in the P row and Q column. The pixel values ​​of the pixels in the P row and Q column are the pixel values ​​related to the pixel columns from the pixels in the P row and Q column of the first frame to the pixel columns in the P row and Q column of the second frame. Next, in step S014, the thermal image correction unit 34 performs a process of replacing P with P+1. Next, in step S015, the correction coefficient calculation unit 33 determines whether there is a pixel column in the P row that was replaced with P+1 in the previous step. When there is a pixel column in the P row (in the case of "Yes" in step S015), the processing of steps S012 to S014 is repeated. In the case that there is no pixel column in the P row (in the case of "No" in step S015), the correction of the pixel column of the pixels in the Q column is completed, and the next step is entered.

[0057] Next, in step S016, the thermal image correction unit 34 performs processing to replace Q with Q+1. Next, in step S017, it is determined whether there is a pixel column in the Qth column that was replaced with Q+1 in the previous step. When there is a pixel column in the Qth column (in the case of "Yes" in step S017), the processing of steps S012 to S016 is repeated. In the case that there is no pixel column in the Qth column (in the case of "No" in step S017), the correction of the pixel column of the pixels in the Qth column is completed, and the next step is entered.

[0058] Next, in step S018, the thermal image correction unit 34 reconstructs the image using the corrected pixel values ​​to generate a corrected image. Specifically, the thermal image correction unit 34 rearranges the pixel columns according to the information of the pixel positions based on the corrected pixel values ​​of each pixel column and the information of the pixel positions to generate a plurality of thermal images.

[0059] Finally, in step S019 , the thermal image correction unit 34 outputs the corrected thermal image TO to an output destination.

[0060] The infrared image processing device 1 of the embodiment 1 of the present disclosure is constructed as described above, and exerts the following effects. The infrared image processing device 1 obtains the correction coefficient c and the correction coefficient d, and uses the correction coefficient c and the correction coefficient d to correct the thermal image TI. In contrast, when a correction coefficient is obtained and the thermal image is corrected, the infrared image processing device 1 cannot sufficiently reduce the striped noise. For example, when the difference between the object pixel value and the smoothed pixel value is obtained (the coefficient d in z=m+d is obtained) and the thermal image is corrected using only the difference value, no matter what the input value is, only the correction by adding the difference value can be performed. However, if Figure 2 As shown, the coefficient a is different for each infrared detection element. When a=1, the input value and the output value are considered to be equal, and the output value of the infrared detection element with a coefficient larger than a=1, that is, a high sensitivity, changes more than the change in the input value when the input value changes. Even if the output value of such an element is corrected only by the differential value, the correction amount is insufficient. In addition, for the output value of the infrared detection element with a coefficient smaller than a=1, that is, a low sensitivity, the correction amount is too large. The infrared image processing device 1 calculates multiple correction coefficients (correction coefficient c and correction coefficient d) to reduce streak noise with high accuracy even if the input value changes.

[0061] Furthermore, the infrared image processing device 1 uses smoothed pixel values ​​obtained from the thermal image, and therefore can calculate the correction coefficient using the thermal image during operation where the temperature condition is not constant. Thus, the correction coefficient can be updated while the infrared image processing device 1 is processing.

[0062] In addition, the infrared image processing device 1 sets the coordinate value using the object pixel value of the thermal image and the smoothed pixel value of the smoothed thermal image, performs linear approximation on the coordinate value, and thereby calculates the correction coefficient. Thus, the correction coefficient can be calculated by only performing the setting of the coordinate value and the linear approximation process once, and the correction coefficient can be updated efficiently.

[0063] When the edge of the heat source is located in the row for calculating the correction coefficient, the edge is smoothed, and the accuracy of the smoothed pixel value of the smoothed thermal image is reduced. When the object pixel value is corrected using such a smoothed pixel value, the thermal image processing device 3 cannot obtain an appropriate thermal image. However, in the infrared image processing device 1, a plurality of coordinate values ​​are approximated by a straight line, thereby reducing the proportion of the edge in the entire pixel of the smoothed pixel value. Therefore, the infrared image processing device 1 can suppress the reduction of the correction accuracy caused by the edge.

[0064] In addition, in the above description, regarding the object pixel values ​​of the thermal image and the smoothed pixel values ​​of the smoothed thermal image, the thermal image is generated in the order in which the data is input to the thermal image sensor 2 and the correction and other processes are performed. The order is the order of processing from the pixels at one end of the 1st column and the 1st row toward the pixels at the other end, and then processing from the pixels at one end of the 1st column and the 2nd row toward the pixels at the other end. In addition, the direction from the pixels at one end toward the pixels at the other end is the row direction of the thermal image. However, for example, the thermal image processing device 3 can also sort the object pixel values ​​for the thermal image and perform linear approximation on the object pixel values ​​arranged in descending or ascending order. In this case, for example, there is a sorting unit, which sorts the pixel values ​​of multiple thermal images in descending or ascending order in the frame direction. In addition, the thermal image processing device 3 can also perform the following processing: sorting the smoothed pixel values ​​for the smoothed thermal image, and performing linear approximation and correction on the smoothed pixel values ​​arranged in descending or ascending order. In this case, for example, there is a sorting unit that sorts the pixel values ​​of the plurality of smoothed thermal images in descending order or ascending order in the frame direction. The thermal image processing device 3 may also set the direction of sorting to be from the pixels at one end toward the pixels at the other end. In addition, the thermal image processing device 3 may also set the direction of sorting to be the direction of the frame, that is, the pixels of the first column and the first row, then the pixels of the first column and the first row of the next frame, and then the pixels of the first column and the first row of the next frame.

[0065] In addition, as described above, in the thermal image processing device 3, when the direction of sorting is set to the direction from the pixel at one end toward the pixel at the other end, the pixel value at the end of the pixel column formed by arranging the sorted pixels can also be removed. Specifically, in the thermal image processing device 3, it is determined in sequence whether it is higher than the first threshold value from the maximum value of the sorted pixel column. Next, the thermal image processing device 3 deletes the pixel values ​​higher than the threshold value from the pixel column. Then, the thermal image processing device 3 determines in sequence whether it is lower than the second threshold value from the minimum value of the sorted pixel column. Then, the thermal image processing device 3 deletes the pixel values ​​lower than the second threshold value from the pixel column. In addition, the first threshold value is greater than the second threshold value. In the thermal image processing device 3, pixel values ​​lower than the specified first threshold value and pixel values ​​higher than the specified second threshold value are used. In the thermal image processing device 3, pixel values ​​lower than the specified first threshold value or pixel values ​​higher than the specified second threshold value can also be used. It is also possible not to use a threshold value, predetermine the number of pixel values ​​to be removed, and remove the corresponding number of pixel values ​​from the maximum value of the pixel column and the minimum value of the pixel column.

[0066] In addition, removing the pixel values ​​at the ends of the pixel columns formed by arranging the sorted pixels means not using the large and small values ​​in the pixel columns. For example, when there are light bulbs, gas stoves, or people in the room, their outlines are shown as edges in the thermal image. When the thermal image is smoothed, the edge portion is smoothed, thereby reducing the accuracy of the smoothed pixel value. The same is true for windows that become cold due to external air. By removing the edge portion in the thermal image, the reduction in the accuracy of the smoothed pixel value can be suppressed. In addition, regarding the method of removing the pixel values ​​at the ends of the pixel columns formed by arranging the sorted pixels, for example, a method of detecting the maximum pixel value and the minimum pixel value in the image may be used, but is not limited to this.

[0067] In the above-mentioned infrared image processing device 1, the direction in which stripe noise is generated is set to the row direction of the thermal image, but it may also be the column direction or the oblique direction. The connection of the driving line to the infrared detection element arranged in the row direction is described as the cause of the stripe noise in the row direction, but when the driving line is connected in the column direction or the oblique direction, the stripe noise is generated in the column direction or the oblique direction.

[0068] In the above-mentioned infrared image processing device 1, striped noise is generated due to the difference in the characteristics of the driving line, but the cause of the striped noise is not limited to this. In the case where the infrared detection elements arranged in the row direction are connected by the same output line (the line that sends the voltage signal), striped noise may be generated due to the deviation of the characteristics of the output line. In addition, the same is true for the case where the characteristics of the A / D converter, amplifier, etc. connected to the output line deviate. Furthermore, in the process of generating an image, in addition to the process of reducing the striped noise, noise reduction processing is sometimes performed. In the case where this process is performed for each row, the degree of noise reduction processing varies for each row, and striped noise may be generated. The above-mentioned infrared image processing device can also reduce these striped noises.

[0069] Implementation Method 2

[0070] Next, Embodiment 2 of the present disclosure will be described. Fig.10 This is a block diagram showing the structure of an infrared image processing device according to the second embodiment. Fig.11 This is a schematic diagram of a thermal image sensor and a sensor moving unit and a schematic diagram of a thermal image of an infrared image processing device according to Embodiment 2. Description of the same structures and operations as those described in Embodiment 1 is omitted, and the parts different from Embodiment 1 are described below.

[0071] In the infrared image processing device of the first embodiment, a thermal image sensor 2 in which infrared detection elements are arranged in a matrix is ​​used. In the infrared image processing device 501 of the second embodiment, a thermal image sensor 502 in which a plurality of infrared detection elements are arranged in a row in the longitudinal direction is used. The infrared image processing device moves the thermal image sensor 502 in the lateral direction, synthesizes the output of the infrared detection elements, that is, the voltage signal VI, and thereby generates a thermal image. By repeating this operation, the infrared image processing device 501 generates a plurality of images.

[0072] As described above, the infrared image processing device 501 of the second embodiment differs from the thermal image sensor 502 in structure from the thermal image sensor 2 of the first embodiment. Fig.10 As shown, a sensor moving unit 504 is added to the structure of the infrared image processing device 1 of embodiment 1. The thermal image sensor 502 is formed by arranging a plurality of infrared detection elements in a row in the longitudinal direction. The sensor moving unit 504 moves the thermal image sensor 502 in the transverse direction to change the position of the thermal image sensor 502. That is, the sensor moving unit 504 moves the thermal image sensor 502 in a direction intersecting the direction in which the thermal image sensor 502 is arranged. For example, the sensor moving unit 504 is a motor. The rotating shaft of the motor is connected to the thermal image sensor 502, whereby the rotational motion of the motor is transmitted to the thermal image sensor 502. In Fig.10 , the connection between the thermal image sensor 502 and the sensor moving unit 504 is indicated by a dotted line connecting both sides. In addition, the sensor moving unit 504 has a rotary encoder. The sensor moving unit 504 sends a signal indicating the rotation amount of the motor, that is, the movement amount of the thermal image sensor 502, to the thermal image processing device 503 (more specifically, the thermal image generating unit 531).

[0073] The thermal image generation unit 531 in the infrared image processing device 501 of the second embodiment receives from the thermal image sensor 502 a voltage signal VI corresponding to infrared rays received accompanying the rotational movement of the thermal image sensor 502 and a signal indicating the movement amount from the sensor moving unit 504. The thermal image generation unit 531 rearranges the voltage signal VI to generate a thermal image.

[0074] like Fig.11 As shown in (a), the thermal image sensor 502 rotates in a manner that draws an arc concentric with the rotation axis, and transmits a voltage signal VI corresponding to the received infrared light. Fig.11 In (a), the thermal image sensor 502 and the sensor moving part 504 are connected by a dotted line, but they are actually connected by a structure. Fig.11As shown in (b), the thermal image sensor 502 synthesizes the thermal image I1 generated based on the voltage signal VI at a certain position during the rotational movement and the thermal images I1 at multiple positions, thereby obtaining a thermal image I2.

[0075] In the infrared image processing device 1 described above, the thermal image sensor 502 is configured such that a plurality of infrared detection elements are arranged in a row in the longitudinal direction. Fig.11 As in (c), the infrared image processing device 501 may also arrange the infrared detection elements in several rows (in the Fig.11 In the infrared image processing device 1, a voltage signal VI corresponding to infrared rays received accompanying the rotational movement of the thermal image sensor 502b and a signal representing the movement amount from the sensor moving part 504 are received, thereby generating a thermal image.

[0076] The infrared image processing device 501 of the second embodiment of the present disclosure is configured as described above, and has the following effects. The infrared image processing device 501 of the second embodiment can also capture thermal images using fewer infrared detection elements. By reducing the number of infrared detection elements, the structure of the infrared image processing device 501 in the first embodiment can be implemented with a smaller-scale device.

[0077] In addition, when the infrared image processing device 501 of the second embodiment is used to capture a thermal image, stripe noise may also be generated. If the coefficient a (sensitivity) or the coefficient b as the intercept component in the formula y=ax+b representing the relationship between input and output is different for each infrared detection element arranged in the vertical direction, horizontal stripe noise may be generated when the image is synthesized. However, the infrared image processing device 501 of the second embodiment can calculate the correction coefficient in the same way as the first embodiment, and therefore can perform processing to reduce the stripe noise. In addition, the same effect as the first embodiment is exerted.

[0078] In Embodiment 2, an example is shown in which the position of the thermal image sensor 502 is moved by the sensor moving unit 504 . However, the thermal image sensor 502 may be fixed to the rotating shaft of the motor so that the direction of the thermal image sensor 502 can be changed.

[0079] In the infrared image processing device 1 and the infrared image processing device 501, a linear function is used as a linear equation and a correction equation to calculate coefficients and correction coefficients, but a polynomial function such as a cubic function may also be used. By using a high-order polynomial function, the pixel value can be approximated with high accuracy. In addition, when a high-order polynomial is used, the correction coefficient becomes two or more.

[0080] In the above-mentioned infrared image processing device 1 and infrared image processing device 501, a pyroelectric element is used as an infrared detection element. However, the infrared image processing device 501 may also use an infrared detection element of a thermopile type connected to a thermocouple that generates the Seebeck effect, a bolometer type that uses a change in resistance value based on temperature information, or the like. In addition, the infrared image processing device 501 may be of any type as long as it can detect infrared rays.

[0081] In the above-mentioned infrared image processing device 1, the thermal image generation unit 31 included in the thermal image processing device 3 receives the voltage signal VI of the thermal image sensor to generate a thermal image. Alternatively, in the infrared image processing device 501, the thermal image generation unit 531 included in the thermal image processing device 503 receives the voltage signal VI of the thermal image sensor to generate a thermal image. However, the thermal image generation unit 31 may also be provided in the thermal image sensor 2, and the thermal image generation unit 531 may also be provided in the thermal image sensor 502.

[0082] In the above-mentioned infrared image processing device 1 and infrared image processing device 501, the process of finding the average value of the object pixel value and the adjacent pixel values ​​is performed as the smoothing process. However, the infrared image processing device 1 or the infrared image processing device 501 may also weight the object pixel value and the adjacent pixel values ​​used for finding the average value, respectively, to find the average value. In addition, the infrared image processing device 1 or the infrared image processing device 501 may not use the object pixel value, but only use the adjacent pixel values ​​to find the average value. In addition, the infrared image processing device 1 and the infrared image processing device 501 use three pixel values ​​including the object pixel value and two adjacent pixel values ​​for smoothing processing. However, the infrared image processing device 1 or the infrared image processing device 501 may use two pixel values ​​further adjacent to the adjacent pixel value, and use five pixel values ​​for smoothing processing. The infrared image processing device 1 or the infrared image processing device 501 may also use adjacent pixel values, and use more than five pixel values ​​for smoothing processing.

[0083] The infrared image processing device 1 and the infrared image processing device 501 are installed in indoor electrical products. However, by installing the infrared image processing device 1 or the infrared image processing device 501 in the electrical product, it is possible to use thermal images in the control of the electrical product. In addition, the infrared image processing device 1 or the infrared image processing device 501 can use thermal images to confirm the temperature conditions in the room where the electrical product is installed. Furthermore, in the infrared image processing device 1 or the infrared image processing device 501, the infrared image processing device can be installed not only in the electrical product, but also as a camera for anti-theft or surveillance installed indoors or outdoors.

[0084] Fig.12 This is a diagram showing an example of the hardware configuration of the thermal image processing device 503 (thermal image processing device 3 ). Fig.13 It is a diagram showing another example of the hardware configuration of the thermal image processing device 3 .

[0085] The thermal image processing device 503 is composed of, for example, at least one processor 101a and a memory 101b. The processor 101a is, for example, a central processing unit (CPU) that executes a program stored in the memory 101b. In this case, the functions of the thermal image processing device 503 are implemented by software, firmware, or a combination of software and firmware. The software and firmware are stored in the memory 101b as programs. Thus, the program for implementing the functions of the thermal image processing device 503 is executed by a computer.

[0086] The memory 101 b is a computer-readable recording medium, and is, for example, a volatile memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory), a nonvolatile memory, or a combination of a volatile memory and a nonvolatile memory.

[0087] The thermal image processing device 503 may be constituted by a processing circuit 101c which is dedicated hardware such as a single circuit or a composite circuit. In this case, the function of the thermal image processing device 503 is realized by the processing circuit 101c.

[0088] Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments.

[0089] Description of symbols

[0090] 1: infrared image processing device; 2: thermal image sensor; 3: thermal image processing device; 31: thermal image generating unit; 32: smoothing processing unit; 33: correction coefficient calculating unit; 34: thermal image correcting unit; 504: sensor moving unit.

Claims

1. An infrared image processing device, characterized in that: The infrared image processing device comprises: A thermal image sensor that receives infrared rays and outputs a signal corresponding to the infrared rays; a thermal image generating unit, which generates a plurality of thermal images according to the signal; A smoothing processing unit, which performs smoothing processing on each pixel of each of the plurality of thermal images using pixel values ​​of surrounding pixels, calculates a plurality of smoothed images, and calculates each smoothed pixel value of each image, i.e., a smoothed pixel value; a correction coefficient calculation unit that calculates a correction coefficient group including a first correction coefficient and a second correction coefficient based on pixel values ​​at specific pixel positions of the plurality of thermal images and corresponding pixel values ​​at the specific pixel positions of the plurality of smoothed images; and A thermal image correction unit is configured to correct the thermal image using the correction coefficient group.

2. The infrared image processing device according to claim 1, characterized in that: The correction coefficient calculation unit calculates the correction coefficient group by approximating the following multiple coordinate values ​​using a polynomial function, wherein: The plurality of coordinate values ​​are represented by pixel values ​​from the first frame to the second frame at specific pixel positions of the plurality of thermal images and corresponding pixel values ​​from the first frame to the second frame at specific pixel positions of the plurality of smoothed thermal images.

3. The infrared image processing device according to claim 1, characterized in that: The infrared image processing device further comprises a sorting unit, which sorts the pixel values ​​of the plurality of thermal images and the pixel values ​​of the plurality of smoothed thermal images in descending order or ascending order in a frame direction. The correction coefficient calculation unit uses polynomial functions to approximate the coordinate values ​​represented by the pixel values ​​and their order of pixels arranged in the frame direction and the coordinate values ​​represented by the pixel values ​​and their order of the smoothed thermal image, and calculates the correction coefficient group using the coefficients of each polynomial function.

4. The infrared image processing device according to claim 3, characterized in that: The correction coefficient calculation unit calculates the correction coefficient group using pixel values ​​below a specified first threshold or pixel values ​​above a specified second threshold, The first threshold is greater than the second threshold.

5. The infrared image processing device according to claim 1, characterized in that: The thermal image is an image captured by the thermal image sensor in an exposed state.

6. The infrared image processing device according to claim 1, characterized in that: The infrared image processing device further comprises a sensor moving unit, which moves the thermal image sensor so that the position or direction of the thermal image sensor changes. The thermal image generation unit generates one or more thermal images based on the signal corresponding to the infrared rays received by the thermal image sensor while moving.

7. An infrared image processing method, characterized in that: receiving infrared rays and outputting a signal corresponding to the infrared rays, generating a plurality of thermal images based on the signal, According to each pixel of each of the plurality of thermal images, a smoothing process is performed using the pixel values ​​of the surrounding pixels, a plurality of smoothed images are calculated, and each smoothed pixel value of each image, i.e., a smoothed pixel value, is calculated. Calculating a correction coefficient group according to pixel values ​​at specific pixel positions of the plurality of thermal images and corresponding pixel values ​​at the specific pixel positions of the plurality of smoothed images, wherein the correction coefficient group includes a first correction coefficient and a second correction coefficient, The thermal image is corrected using the correction coefficient set.

Images