Method and apparatus for detecting defective pixels in thermal images

Abstract

A method and apparatus for detecting defective pixels in a thermal image is provided. [Solution] The method for detecting defective pixels in a thermal image according to the present invention includes the steps of: a data capture circuit capturing an original image using a thermal imaging device; a processor executing a non-uniformity correction algorithm on the original image to generate a plurality of correction coefficients for each of a plurality of pixels in the original image; a processor generating a plurality of feature coordinates in a feature space based on the pixel values ​​of each pixel in the original image and the plurality of correction coefficients for each pixel; and a processor performing outlier detection on the plurality of feature coordinates to generate at least one outlier coordinate as at least one defective pixel coordinate, wherein the at least one defective pixel coordinate indicates at least one pixel in the original image corresponding to the at least one defective pixel coordinate as an anomaly.

Description

[Technical Field]

[0001] The present invention relates to the field of thermal imaging technology, and more particularly to a method and device of detecting bad pixels for thermal imaging. [Background technology]

[0002] Thermal imaging technology is prone to process issues that can cause thermal imaging device sensors to fail or respond abnormally. These issues often result in bad pixels in the images captured by thermal imaging devices. Therefore, previous methods for detecting bad pixels compare the numerical value difference between each pixel and its surrounding neighboring pixels to determine whether it is an abnormal pixel. Summary of the Invention [Problem to be solved by the invention]

[0003] However, the prior art methods simply determine whether a pixel is defective based on its numerical value, which increases the risk of misjudgment. Furthermore, to prevent overlooking defective pixels, prior art methods must constantly and repeatedly perform defective pixel detection during real-time imaging, which increases the burden on real-time computing resources. Therefore, engineers in this field have sought a more accurate and efficient method for detecting defective pixels to solve this problem.

[0004] The present invention has been made in view of the above-mentioned conventional situation, and its purpose is to provide a method and device for detecting defective pixels in thermal images, which solves the problem that in the conventional technology, every time a thermal imaging device captures an image, defective pixel detection must be performed once for the captured image to determine which pixel the defective pixel is located in. [Means for solving the problem]

[0005] In order to achieve the above object, a method for detecting defective pixels using a thermal image according to one aspect of the present invention comprises: a step a in which a data capture circuit captures an original image using a thermal imager; a step b) of a processor executing a non-uniformity correction algorithm on the original image to generate a plurality of correction factors for each of a plurality of pixels of the original image; a step c) of the processor generating a plurality of feature coordinates in a feature space based on a pixel value of each of the pixels of the original image and a plurality of the correction coefficients for each of the pixels; and step d, in which the processor performs outlier detection on a plurality of the feature coordinates to generate at least one outlier coordinate as at least one bad pixel coordinate, the at least one bad pixel coordinate indicating as an anomaly at least one of the pixels corresponding to the at least one bad pixel coordinate in the original image.

[0006] In order to achieve the above object, another aspect of the present invention is a defective pixel detection device using a thermal image, a data capture circuit arranged to capture the original image by the thermal imager; a memory arranged to store a plurality of instructions; a data capture circuit coupled to the memory and having access to a plurality of the instructions; an operation a) of performing a non-uniformity correction algorithm on the original image to generate a plurality of correction coefficients for each of a plurality of pixels of the original image; an operation b) of generating a plurality of feature coordinates in a feature space based on a pixel value of each of the pixels of the original image and a plurality of the correction coefficients of each of the pixels; and a processor that performs an operation c of performing outlier detection on the plurality of feature coordinates to generate at least one outlier coordinate as at least one bad pixel coordinate, wherein the at least one bad pixel coordinate indicates as an anomaly at least one pixel in the original image that corresponds to the at least one bad pixel coordinate. [Effects of the Invention]

[0007] The present invention is configured as described above and therefore provides the following effects. The present invention uses a non-uniformity correction factor in addition to the pixel values ​​of the image generated by the thermal imaging device to calculate the feature point coordinates of each pixel. Next, the present invention uses an outlier algorithm to search for outlier features and identify the corresponding image pixels as abnormal pixels (i.e., defective pixels). In this way, the present invention effectively improves the accuracy of defective pixel detection and completes accurate defective pixel detection in one go during correction. There is no need to run a separate defective pixel detection algorithm multiple times during real-time imaging to compensate for missed defective pixels.

[0008] At least the following points will become clear from the description and drawings to be described later. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram illustrating an apparatus for detecting bad pixels using thermal imaging in accordance with some embodiments of the present invention. [Figure 2] 1 is a flowchart illustrating a method for detecting bad pixels in a thermal image in accordance with some embodiments of the present invention. [Figure 3] FIG. 2 is a schematic diagram illustrating an example of converting a source image into a modified image in some embodiments of the present invention. [Figure 4] FIG. 1 is a schematic diagram illustrating outlier detection in some embodiments of the present invention. [Figure 5]2 is a schematic diagram illustrating at least one pixel corresponding to at least one bad pixel coordinate in some embodiments of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0010] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0011] 1 is a block diagram illustrating an apparatus for detecting defective pixels in a thermal image according to some embodiments of the present invention. In this embodiment, the apparatus for detecting defective pixels in a thermal image 100 includes a data capture circuit 110, a storage 120, and a processor 130. The processor 130 is coupled to the data capture circuit 110 and the storage 120.

[0012] In some embodiments, the device 100 for detecting defective pixels using thermal imaging can be implemented as any data processing device (e.g., desktop computer, laptop, tablet, etc.) or server (e.g., cloud server, virtual server, rack server, etc.). In this embodiment, the data capture circuit 110 is used to capture the original image img1 using the thermal imaging device 200. In some embodiments, the original image img1 is generated by the thermal imaging device 200 capturing an image of a black body having a black body temperature in a detection area having an ambient temperature. Specifically, a user uses the thermal imaging device 200 to capture an image of a black body having a specific black body temperature (e.g., ambient temperature 25°C or a higher temperature (100°C)) in a detection area having a specific ambient temperature (e.g., ambient temperature 25°C), thereby generating the original image img1. Next, the device 100 for detecting defective pixels using thermal imaging is connected to the thermal imaging device 200 via the data capture circuit 110, and the original image img1 is captured by the thermal imaging device 200.

[0013] In some embodiments, the detection area is a stable area with stable environmental temperature and humidity and no air flow (such as a laboratory or a sealed indoor space). Furthermore, only the above-mentioned blackbody is placed in the detection area, and no other objects are placed therein. In some embodiments, the thermal imaging device 200 includes multiple sensors 211-21N, which detect thermal radiation in the detection area and generate pixel values ​​for each of the multiple pixels in the original image img1. In other words, the sensors 211-21N correspond to the multiple pixels in the original image img1 (i.e., have a one-to-one correspondence), and are used to generate pixel values ​​for the corresponding pixels. N is the total number of pixels in the original image img1. In some embodiments, the above-mentioned correspondences between the sensors 211-21N and the multiple pixels are stored in advance in the storage 120 (e.g., a correspondence table indicating the correspondences between the sensors 211-21N and the multiple pixels is stored).

[0014] In some embodiments, the thermal imaging device 200 may be implemented as any thermal imaging device (e.g., a general infrared thermography device, a quantum imaging device, or a combined optical and infrared imaging device). In some embodiments, the sensors 211-21N may be implemented as any cooled or uncooled thermal radiation sensor. In some embodiments, the original image img1 may be any type of image (e.g., a grayscale image or an RGB image) generated by the thermal imaging device 200 that has not undergone the non-uniformity correction (NUC) algorithm and the bad pixel detection process described below. In some embodiments, the data capture circuit 110 may be any wireless communication circuit (e.g., an Ethernet communication circuit, a Wi-Fi communication circuit, or a Bluetooth® communication circuit) or wired communication circuit.

[0015] It should be noted that a blackbody is an ideal object that can absorb all incident radiation (all radiation wavelengths and angles of incidence), does not reflect or transmit any light (i.e., its reflectance and transmittance are 0), and radiates thermal radiation perfectly uniformly. Because a blackbody radiates thermal radiation perfectly uniformly, theoretically, all pixels of the blackbody object in the captured original image img1 have the same pixel value, and all pixels in the background of the original image img1 also have another identical pixel value. These pixel values ​​can be considered theoretical values. In some embodiments, the predetermined ambient temperature, blackbody temperature, and the correspondence between temperature and pixel values ​​are all stored in storage 120 in advance. Therefore, the processor 130 calculates the theoretical values ​​of all pixels in the original image img1 based on the predetermined ambient temperature, blackbody temperature, and the correspondence between temperature and pixel values. These theoretical values ​​are used to calculate the correction coefficients described below.

[0016] However, sensors 211-21N may have different thermal radiation detection sensitivities due to process errors, degradation, failures, etc. Therefore, the pixel values ​​of pixels in a portion of the blackbody object in original image img1 and the pixel values ​​of pixels in a portion of the background in original image img1 may not match the theoretical values ​​described above. Therefore, the present invention requires that the non-uniformity correction algorithm and defective pixel detection described below be performed on original image img1.

[0017] In this embodiment, storage 120 is used to store instructions. In some embodiments, the instructions may be implemented as any firmware or software, and processor 130 accesses these instructions to execute the method for detecting defective pixels in a thermal image, as described below. In some embodiments, storage 120 may be implemented as a flash memory, a read-only memory (ROM), a hard disk, or any equivalent storage device. In some embodiments, processor 130 may be implemented as a central processing unit (CPU), a microcontrol unit (MCU), a programmable logic controller (PLC), a system on chip (SoC), or a field programmable gate array (FPGA).

[0018] Also referring to Figure 2, Figure 2 is a flow chart showing a method for detecting defective pixels using a thermal image in some embodiments of the present invention. This method for detecting defective pixels using a thermal image is applied to the device 100 for detecting defective pixels using a thermal image shown in Figure 1. The method for detecting defective pixels using a thermal image includes steps S210 to S240 (see Figure 2).

[0019] First, in step S210, the data capture circuit 110 captures the original image img1 using the thermal imaging device 200. In step S220, the processor 130 performs a non-uniformity correction algorithm on the original image img1 to generate a plurality of correction coefficients for each of the plurality of pixels in the original image img1.

[0020] In some embodiments, the processor 130 calculates a gain coefficient and an offset coefficient for each pixel as multiple correction coefficients for the linear relationship between the pixel value of each pixel of the original image img1 and the theoretical value of each pixel. In some embodiments, the linear relationship between the pixel value of each pixel and the theoretical value of each pixel is shown in the following equation (1):

number

[0021] In some embodiments, the processor 130 calculates a sum of squared residuals from a plurality of linear relationships corresponding to each of the plurality of pixels in the original image img1, and generates a plurality of correction coefficients for each of the plurality of pixels by minimizing the sum of squared residuals. In some embodiments, the above-mentioned sum of squared residuals is shown in the following equation (2):

number

[0022] In some embodiments, the processor 130 minimizes this residual sum of squares using an algorithm such as the least squares method, stepwise regression, or maximum likelihood estimation (MLE), and calculates the gain and offset coefficients for each pixel (i.e., the gain coefficient a for the pixel at pixel coordinate (i,j)). ij and offset coefficient b ij ) as its own correction coefficients. In some embodiments, processor 130 may further normalize the gain and offset coefficients for each pixel. Such normalization may improve the efficiency of subsequent processing and reduce the impact of coefficient anomalies. In some embodiments, the normalization may be achieved by min-max normalization, Z-score normalization, or the like.

[0023] In some embodiments, processor 130 transforms multiple pixels of original image img1 into a modified image using multiple modification coefficients for each of the multiple pixels of original image img1. In some embodiments, processor 130 generates a linear transformation equation (i.e., similar to Equation (1) above) for each pixel using the multiple modification coefficients for each pixel of original image img1, and transforms the pixel value of each pixel of original image img1 into the pixel value of the corresponding pixel of the modified image using the linear transformation equation for each pixel. Each pixel of original image img1 and the corresponding pixel of the modified image have the same pixel coordinates.

[0024] The original image img1 and the corrected image will be described below using a practical example. Please also refer to FIG. 3. FIG. 3 is a schematic diagram showing an example of converting the original image img1 into the corrected image img2 in some embodiments of the present invention. The processor 130 applies correction to all pixels of the original image img1 using a non-uniformity correction algorithm to generate all pixels of the corrected image img2 (see FIG. 3). As can be seen from the corrected image img2, the non-uniformity correction algorithm has significantly eliminated hot spots, stripes, and noise in the thermal image, so that the distribution of pixel values ​​of the pixels in the corrected image img2 is significantly smoothed and has good clarity.

[0025] Returning to FIG. 2, in step S230, the processor 130 calculates the pixel value of each pixel of the original image img1 and a plurality of correction factors for each pixel (i.e., the above-mentioned gain coefficients a ij and offset coefficient b ij ), a plurality of feature coordinates in a feature space are generated based on the pixel values ​​of a plurality of pixels in the original image img1. In some embodiments, the pixel values ​​of a plurality of pixels in the original image img1 correspond to a plurality of feature coordinates, respectively. In some embodiments, the processor 130 sets the pixel values ​​of each pixel in the original image img1 and the plurality of correction coefficients for each pixel as a plurality of elements of each feature coordinate in the feature space. In some embodiments, the feature coordinates are shown in Equation (3) below.

number

[0026] In step S240, processor 130 performs outlier detection on the plurality of feature coordinates to generate at least one outlier coordinate as at least one bad pixel coordinate. The at least one bad pixel coordinate indicates at least one pixel in original image img1 that corresponds to the at least one bad pixel coordinate as an anomaly. In other words, processor 130 selects at least one outlier coordinate as at least one bad pixel coordinate from the plurality of feature coordinates in feature space through outlier detection. Next, processor 130 selects at least one pixel from original image img1 that corresponds to the at least one bad pixel coordinate in feature space (e.g., if the bad pixel coordinate is (V xy ,a xy ,b xy ) and the pixel coordinates of the corresponding pixel in the original image img1 are (x,y)), identify this at least one pixel as anomalous (i.e., cannot be corrected by the non-uniformity correction algorithm described above).

[0027] In some embodiments, outlier detection may be achieved using a k-nearest neighbors (KNN) algorithm, a local outlier factor (LOF) algorithm, a density-based spatial clustering of applications with noise (DBSCAN) algorithm, a K-means algorithm, or a principal components analysis (PCA) algorithm, among others.

[0028] Outlier detection will be described below using a practical example. Also referring to FIG. 4, FIG. 4 is a schematic diagram illustrating outlier detection in some embodiments of the present invention. A plurality of feature coordinates exist in a feature space 400. Processor 130 performs outlier detection on all feature coordinates, and finds that most of the feature coordinates are all in cluster 410. Among the plurality of feature coordinates, three outlier coordinates are identified as defective pixel coordinates 420-440, respectively (see FIG. 4).

[0029] 5, which is a schematic diagram illustrating pixels 510-530 corresponding to the defective pixel coordinates 420-440 in some embodiments of the present invention. Processor 130 selects three pixels 510-530 from original image img1 that correspond to the three defective pixel coordinates 420-440 in feature space 400, respectively (see FIG. 5). Processor 130 then identifies these three pixels 510-530 as abnormal pixels that cannot be corrected by the non-uniformity correction algorithm described above.

[0030] In some embodiments, processor 130 selects a pixel corresponding to at least one bad pixel coordinate from the original image img1 as at least one abnormal pixel. Then, processor 130 calculates at least one substitute value as the pixel value of the at least one abnormal pixel based on pixel values ​​of a plurality of pixels surrounding the at least one abnormal pixel. In some embodiments, processor 130 randomly selects a pixel value of one pixel from the plurality of pixels surrounding the at least one abnormal pixel as the at least one substitute value. For example, processor 130 may select a 3×3 pixel region centered around the one abnormal pixel and randomly select a pixel value of one pixel other than the abnormal pixel from this pixel region as the substitute value.

[0031] In some other embodiments, the processor 130 calculates the average value of pixel values ​​of a plurality of pixels surrounding at least one abnormal pixel as the at least one substitute value. For example, the processor 130 may select a 3x3 pixel region with an abnormal pixel as its center point, and calculate the average value of pixel values ​​of all pixels other than the abnormal pixel in the pixel region as the substitute value.

[0032] In some embodiments, the processor 130 identifies at least one sensor corresponding to at least one abnormal pixel as abnormal. For example, extending the example of FIG. 5 , if the pixel values ​​of the abnormal pixels 510-530 are detected by the sensors 211-213, respectively, the processor 130 identifies the abnormal pixels 510-530 corresponding to the sensors 211-213 based on the correspondence between the sensors 211-21N in the storage 120 and all pixels in the original image img1. The processor 130 then identifies the sensors 211-213 as abnormal and stores the identification results in the storage 120. In some embodiments, the apparatus 100 for detecting defective pixels using thermal images periodically or irregularly performs the above steps to update the identification results in the storage 120. In this way, when the thermal imaging device 200 captures a new image again, the processor 130 can detect that an abnormality has occurred in the sensors 211-213 based on the identification result in the storage 120, and identify the pixels in the new image corresponding to the sensors 211-213 as defective pixels. In this way, the present invention eliminates the need to perform defective pixel detection on the captured image to determine which pixel has a defective pixel each time an image is captured thereafter.

[0033] In summary, the method and apparatus for detecting defective pixels in a thermal image according to the present invention combines a non-uniformity correction algorithm and feature space outlier calculation to identify pixels corresponding to defective pixel coordinates in an original image as abnormal defective pixel coordinates. In this way, when capturing an image, the present invention eliminates the need to perform defective pixel detection on the captured new image to determine which pixels contain defective pixels, and instead substitutes the pixel values ​​of the defective pixels with substitute values. Furthermore, the method and apparatus for detecting defective pixels in a thermal image according to the present invention pre-stores the correspondence between the sensors and pixels of the thermal imaging device and identifies the defective sensor from the abnormal pixel based on this correspondence. This allows the user to immediately know which sensor is abnormal and replace or repair the defective sensor.

[0034] Although the embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0035] 100. Device for detecting defective pixels using thermal images 110 Data capture circuit 120 storage 130 processors 200 Thermal Imaging Device 211~21N sensor img1 Original image S210~S240 Step img2 Corrected image 400 Feature Space 410 Cluster 420~440 Bad pixel coordinates 510~530 Abnormal pixels

Claims

1. a step a in which a data capture circuit captures an original image using a thermal imaging device; a step b) of a processor executing a non-uniformity correction algorithm on the original image to generate a plurality of correction factors for each of a plurality of pixels of the original image; a step c) of the processor generating a plurality of feature coordinates in a feature space based on a pixel value of each of the pixels of the original image and a plurality of the correction coefficients for each of the pixels; and (d) the processor performs outlier detection on a plurality of the feature coordinates to generate at least one outlier coordinate as at least one bad pixel coordinate, wherein the at least one bad pixel coordinate indicates as an anomaly at least one pixel in the original image that corresponds to the at least one bad pixel coordinate.

2. The original image is generated by the thermal imaging device capturing an image of a black body having a black body temperature in a detection area having an ambient temperature; The method for detecting defective pixels using a thermal image comprises: the processor selecting, from the original image, the pixel corresponding to at least one of the bad pixel coordinates as at least one abnormal pixel; 2. The method of claim 1, further comprising: calculating at least one substitute value as the pixel value of at least one of the abnormal pixels based on pixel values ​​of a plurality of the pixels surrounding the at least one abnormal pixel.

3. The step b 2. The method for detecting defective pixels in a thermal image according to claim 1, further comprising the step of: the processor calculating a gain coefficient and an offset coefficient for each of the pixels as the plurality of correction coefficients for each of the pixels for a linear relationship between the pixel value of each of the pixels in the original image and the theoretical value of each of the pixels.

4. The step c 2. The method for detecting defective pixels in a thermal image according to claim 1, further comprising a step in which the processor sets the pixel value of each pixel of the original image and the plurality of correction coefficients of each pixel as a plurality of elements of each feature coordinate in the feature space.

5. the thermal imaging device includes a plurality of sensors respectively corresponding to the plurality of pixels of the original image; The method for detecting defective pixels using a thermal image comprises: the processor selecting, from the original image, the pixel corresponding to at least one of the bad pixel coordinates as at least one abnormal pixel; 2. The method of claim 1, further comprising the step of: the processor identifying at least one of the sensors corresponding to at least one of the abnormal pixels as abnormal.

6. a data capture circuit arranged to capture the original image by the thermal imager; a memory arranged to store a plurality of instructions; a data capture circuit coupled to the memory and having access to a plurality of the instructions; an operation a) of performing a non-uniformity correction algorithm on the original image to generate a plurality of correction coefficients for each of a plurality of pixels of the original image; an operation b) of generating a plurality of feature coordinates in a feature space based on a pixel value of each of the pixels of the original image and a plurality of the correction coefficients for each of the pixels; and c) performing outlier detection on a plurality of said feature coordinates to generate at least one outlier coordinate as at least one bad pixel coordinate, wherein the at least one bad pixel coordinate indicates as an anomaly at least one pixel in the original image that corresponds to the at least one bad pixel coordinate.

7. The original image is generated by the thermal imaging device capturing an image of a black body having a black body temperature in a detection area having an ambient temperature; The processor: selecting, from the original image, the pixel corresponding to at least one of the defective pixel coordinates as at least one abnormal pixel; 7. The apparatus for detecting defective pixels in a thermal image according to claim 6, further configured to perform an operation of calculating at least one substitute value as the pixel value of at least one of the abnormal pixels based on pixel values ​​of a plurality of the pixels surrounding the at least one abnormal pixel.

8. In the operation a, the processor:

7. The device for detecting defective pixels in a thermal image according to claim 6, wherein the device is arranged to perform an operation of calculating a gain coefficient and an offset coefficient for each of the pixels as the plurality of correction coefficients for each of the pixels for a linear relationship between the pixel value of each of the pixels in the original image and the theoretical value of each of the pixels.

9. In the operation b, the processor:

7. The device for detecting defective pixels in a thermal image according to claim 6, wherein the device is configured to perform an operation in which the pixel value of each pixel of the original image and the plurality of correction coefficients of each pixel are set as a plurality of elements of each feature coordinate in the feature space, respectively.

10. the thermal imaging device includes a plurality of sensors respectively corresponding to the plurality of pixels of the original image; The processor: selecting, from the original image, the pixel corresponding to at least one of the defective pixel coordinates as at least one abnormal pixel; and identifying at least one of the sensors corresponding to at least one of the defective pixels as defective.

Images