Method for thermal infrared non-uniformity correction based on motion scene

Abstract

A kind of hot infrared non-uniform noise correction method based on motion scene, comprising: recording multiple continuous motion scene images with hot infrared camera;Motion scene image is divided into multiple non-overlapping regions;Estimate the difference of non-uniform noise corresponding to different pixels at a time;Establish noise mathematical model, solve actual non-uniform noise using least square method;Design mask, remove non-uniform noise estimation error part;After removing outliers, non-uniform noise graph is filled, original image is subtracted from noise graph, and non-noise image after correction can be obtained.The application proposes a new non-uniform noise estimation modeling idea, and designs improved median extraction algorithm, which can effectively extract the difference relationship of non-uniform noise between different pixels.All processing processes only need a few original dynamic video frames, avoid the strict requirement of traditional non-uniform correction algorithm to scene and video frame number, so that it can be widely applied to non-uniform correction in different environments.

Description

Technical Field

[0001] This invention relates to the field of infrared image processing, and more specifically to a method for correcting non-uniformity of thermal infrared images based on motion scenes. Background Technology

[0002] With the rapid development of infrared thermal imaging technology, infrared technology has been widely applied in various fields such as civilian and military applications. However, due to the limitations of its component materials and manufacturing processes, thermal infrared cameras, without cooling measures, are susceptible to temperature fluctuations in the ambient environment and the camera's own heat generation. This leads to changes in pixel properties within the camera core, such as thermistor values, temperature coefficient of resistance, heat capacity, and thermal conductivity of the supporting structure. This results in significant non-uniformity in the infrared image, severely impacting image quality. Furthermore, the non-uniformity of infrared images is greatly affected by changes in the external environment, thus requiring non-uniformity correction. Currently, correction methods for thermal infrared cameras can be divided into two categories: calibration-based non-uniformity correction methods and scene-based non-uniformity correction methods.

[0003] In publicly available literature both domestically and internationally, calibration-based non-uniformity correction methods exist. For example, patent application number 202110531184.4 proposes an intelligent calibration correction method and system for infrared imaging non-uniformity. This method first performs two-point correction on the currently acquired raw infrared image, then inputs the corrected image into a trained deep convolutional neural network for non-uniformity noise detection. If the detection result indicates the presence of non-uniform noise, the infrared detector baffle switch is activated, resetting the infrared detector calibration parameters. These calibration-based non-uniformity correction methods often require shutter calibration of the detector to achieve good results. However, shutter application increases cost and power consumption and can cause imaging interruptions, making it unsuitable for certain demanding applications.

[0004] Scene-based nonuniformity correction algorithms do not require pre-calibration of the detector before image correction; instead, they obtain nonuniformity correction parameters by estimating the image scene. Depending on their underlying principles, there are various types of these algorithms, with representative techniques including constant statistical averaging, adaptive filtering-based correction algorithms, time-domain high-pass filtering algorithms, and neural network-based correction algorithms. However, existing algorithms often suffer from excessive computational complexity and over-reliance on scene content, resulting in poor performance in engineering applications. Summary of the Invention

[0005] The present invention aims to overcome the above-mentioned shortcomings of the prior art and proposes a method for correcting the non-uniformity of thermal infrared images based on motion scenes.

[0006] This invention presents a method for non-uniformity correction of thermal infrared images based on moving scenes. It pre-captures a set of dynamic scene images for non-uniform noise estimation, partitions the moving images, estimates the difference in non-uniform noise corresponding to different pixels within the same region at a given time, establishes a mathematical model of the noise, solves for the actual non-uniform noise, designs a mask to remove erroneous portions of the non-uniform noise estimation, and fills the non-uniform noise image after outlier removal to obtain the current actual non-uniform noise. This invention eliminates the need for a shutter, achieving high-quality correction results while maintaining low computational complexity.

[0007] The specific implementation steps of the method of the present invention are as follows:

[0008] S1: Use a thermal infrared camera to record multiple consecutive images of a moving scene {f 1 ,f 2 ,…,f m ,…,f M}

[0009] S2: Divide the motion scene image into multiple non-overlapping regions.

[0010] S3: Estimate the difference in non-uniform noise corresponding to different pixels in the same region at a certain time using the following method:

[0011] S3.1 For the image {f 1 ,f 2 ,…,f m ,…,f M} Take the same region s, and subtract the pixel at position (i,j) from the pixel at position (p,q) within the region to obtain a one-dimensional difference vector in the time dimension. Calculate vectors Median

[0012] S3.2 will Subtract the median A new difference sequence is obtained. Calculate vectors Median when and When the absolute value of the difference is less than the threshold TH1, As The exact median, otherwise let Repeat step S32 until the iteration terminates or the maximum number of iterations is reached;

[0013] S3.3 According to and difference vector similarity of elements Is it an outlier?

[0014] S3.4 Take different pixels within the s region and repeat steps 3.1 to 3.4 until all pixels within the region have been traversed;

[0015] S3.5 For M images, take the other regions of the images and repeat steps 3.1 to 3.5 until all regions of the images have been traversed.

[0016] S4: Establish a mathematical model of non-uniform noise between pixels, and use the least squares method to approximate the solution of the actual non-uniform noise.

[0017] S5: Design a mask to remove non-uniform noise and estimation errors.

[0018] S6: Fill the non-uniform noise map after removing outliers to obtain a complete non-uniform noise map.

[0019] The beneficial effects of this invention are as follows: By estimating non-uniform noise based on motion scenes, it avoids the need to manipulate the radiation shield in the infrared imaging system, as required by existing calibration-based non-uniformity correction methods, thus enabling its widespread application in non-uniformity correction under various environments. Using an improved median extraction algorithm, a robust model is established, and the non-uniform noise is approximated using the least squares method, effectively ensuring the quality of the thermal infrared camera's output image and achieving better non-uniformity correction results. The research results of this invention are beneficial to promoting the development of the field of infrared image processing and have high academic and engineering application value. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method of the present invention.

[0021] Figure 2 This is a schematic diagram of an outlier mask estimated by the present invention.

[0022] Figure 3 This is a non-uniform noise map estimated by the present invention.

[0023] Figures 4a-4d This is a schematic diagram illustrating the effects of an embodiment of the present invention, wherein, Figure 4a This is the non-uniform noise map before correction. Figure 4b This is the corrected non-uniform noise map. Figure 4c This is the non-uniform noise map before correction. Figure 4d This is the corrected non-uniform noise map. Detailed Implementation

[0024] The following description, in conjunction with specific embodiments and accompanying drawings, further illustrates a method for correcting non-uniform thermal infrared noise based on motion scenes. The calculation process is as follows: Figure 1 As shown, it includes the following steps:

[0025] S1: Record multiple consecutive images of a moving scene using a thermal infrared camera, denoted as {f}. 1 ,f 2 ,…,f m ,…,f M}

[0026] Whether motion has occurred in the scene is determined by whether the sum of the absolute values ​​of the pixel differences between two frames is less than a threshold TH2. In this embodiment, TH2 is set to 5.

[0027] The actual observed thermal infrared image f m Represented as:

[0028] f m (i,j)=x m (i,j)+o(i,j),m=1,2,…,M

[0029] Where (i,j) represents the row and column coordinates of the detector array, x m Let o represent an ideal noise-free thermal infrared image, 'o' represent the non-uniformity of pixel output voltage values ​​caused by changes in pixel attributes within the camera mechanism, and M be the number of image frames. Since the time required to acquire a series of thermal infrared images is short, it is assumed that o(i,j) remains constant within a short period of time. In this embodiment, M is set to 800.

[0030] S2: Divide the image into non-overlapping regions, specifically:

[0031] thermal infrared image {f 1 ,f 2 ,…,f m ,…,f M Divide the region evenly into K non-overlapping regions In this embodiment, the image resolution output by the mechanism is 512×640, the number of pixels in each area is 20, and K is 16384.

[0032] S3: Estimate the difference in non-uniform noise between different pixels using the improved median extraction algorithm. The steps are as follows:

[0033] S3.1 For image F, taking the same region s, to avoid blockiness in the results, each region is included in the calculation of the pixel with coordinates (0,0) in the image; each pixel is subtracted from other pixels in the region in turn, and each difference is denoted as . Where s ij,pq This represents the difference between the grayscale value of pixel (i,j) at coordinate (i,j) and the value of pixel (p,q) within region s, resulting in a one-dimensional difference vector along the time dimension, denoted as:

[0034]

[0035] Calculate vectors Median

[0036] S3.2 will Subtract the median A new difference sequence is obtained:

[0037]

[0038] Find the new difference sequence median over time When the following conditions are met:

[0039]

[0040] Will As The exact median, otherwise let Repeat step 3.2 until the iteration terminates or the maximum number of iterations is reached. In this embodiment, TH1 is 0.25, and the maximum number of iterations is 20.

[0041] S3.3 will use the difference vector Subtract the estimated accurate median Calculate the number of elements in the difference vector that are less than the threshold TH3. If this number is less than 0.1 times the number of image frames M, it is considered an outlier. In this embodiment, TH3 is set to 10.

[0042] S3.4 Take different pixels within the s region and repeat steps 3.1 to 3.4 until all pixels within the region have been traversed;

[0043] S3.5 For M images, take the other regions of the images and repeat steps 3.1 to 3.5 until all regions of the images have been traversed.

[0044] S4: The median of the differences between different pixels is obtained through an improved median extraction algorithm. This median is the non-uniform noise difference between the corresponding pixels. The non-uniform noise difference between pixels is expressed as:

[0045]

[0046] Among them, o ′ (i,j) represents the non-uniform noise in the pixel estimation at coordinates (i,j), for o ′ (i,j) can be used to list multiple difference equations related to it, and then the least squares method can be used to approximate the solution to obtain the estimated non-uniform noise o′(i,j). The modeling and solution are applied to all pixels in turn to estimate the non-uniform noise o′ on all camera pixels.

[0047] S5: Combine the mask generated in step 3.3 to remove outliers in the non-uniform noise o′ obtained by solving;

[0048] S6: For the missing parts, fill them directly based on the surrounding values. In this embodiment, an outlier mask is calculated as follows: Figure 2 As shown;

[0049] Subtracting the original image from the estimated non-uniform noise removes the non-uniform noise and pot lid effect, resulting in a corrected noise-free image. In this embodiment, an estimated non-uniform noise image (see...) Figure 3 The results of non-uniform noise removal are shown in Figure 4. All images have been linearly stretched, with a grayscale range of 0–255.

[0050] This invention proposes a novel modeling approach for non-uniform noise estimation and designs an improved median extraction algorithm, which can effectively extract the difference relationship of non-uniform noise between different pixels. All processing requires only a few original dynamic video frames, avoiding the strict requirements of traditional non-uniform correction algorithms on scene and video frame count, thus enabling its wide application in non-uniform correction under different environments.

[0051] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A method for correcting thermal infrared non-uniform noise based on motion scenes, characterized in that, The process includes image partitioning, estimation of actual non-uniform noise and outliers, and the specific steps are as follows: S1: Use a thermal infrared camera to record multiple consecutive images of a moving scene {f 1 ,f 2 ,…,f m ,…,f M }; S2: Divide the motion scene image into multiple non-overlapping regions; S3: Estimate the difference in non-uniform noise corresponding to different pixels in the same region at a certain time using the following method: S31: For image {f 1 ,f 2 ,…,f m ,…,f M Take the same region s, and subtract the pixel at position (i,j) from the pixel at position (p,q) within the region to obtain a one-dimensional difference vector in the time dimension. Calculate vectors Median S32: Will Subtract the median A new difference sequence is obtained. Calculate vectors Median when and When the absolute value of the difference is less than the threshold TH1, As The exact median, otherwise let Repeat step S32 until the iteration terminates or the maximum number of iterations is reached; S33: According to and difference vector similarity of elements Is it an outlier? S34: Take different pixels in the s region and repeat steps S31 to S34 until all pixels in the region have been traversed. S35: For M images, take the other regions of the images and repeat steps S31 to S35 until all regions of the images have been traversed. S4: Establish a mathematical model of non-uniform noise between pixels, and use the least squares method to approximate the solution of the actual non-uniform noise. S5: Design a mask to remove non-uniform noise estimation errors; S6: Fill the non-uniform noise map after removing outliers to obtain a complete non-uniform noise map; 2. The method for correcting thermal infrared non-uniform noise based on a motion scene according to claim 1, characterized in that, Step S1 determines whether scene motion has occurred based on whether the sum of the absolute values ​​of the pixel differences between two frames is less than a threshold TH2, and then uses the observed thermal infrared image f. m Represented as: f m (i,j)=x m (i,j)+o(i,j),m=1,2,…,M Where (i,j) represents the row and column coordinates of the detector array, x m Let o represent an ideal noise-free thermal infrared image, o represent the non-uniformity of pixel output voltage values ​​caused by changes in pixel attributes within the camera mechanism, and M be the number of image frames. Since the time required to acquire a series of thermal infrared images is relatively short, it is assumed that o(i,j) remains constant in a short period of time.

3. The method for correcting thermal infrared non-uniform noise based on a motion scene according to claim 1, characterized in that, Step S3 estimates the difference in non-uniform noise between different pixels in the region at a certain time. In the same region of the image, the difference between different pixels will fluctuate around a benchmark in the time dimension. This fluctuation is caused by changes in the image scene. The benchmark is the difference in non-uniform noise between real pixels. Therefore, an improved median extraction algorithm is used to estimate the difference in non-uniform noise between pixels.

4. The method for correcting thermal infrared non-uniform noise based on a motion scene according to claim 1, characterized in that, Step S4 establishes a mathematical model of non-uniform noise between pixels. A certain pixel can be found by subtracting it from all other pixels in the region. Through the median extraction algorithm, multiple related difference equations can be listed for the non-uniform noise of a certain pixel. The actual non-uniform noise is approximated by the least squares method, which further reduces the error.

5. The method for correcting thermal infrared non-uniform noise based on a motion scene according to claim 1, characterized in that, Step S5 designs a mask based on the reliability of the extracted median pixel difference. When the reliability is low, the estimated non-uniform noise on the corresponding pixel is regarded as an outlier.

6. The method for correcting thermal infrared non-uniform noise based on a motion scene according to claim 1, characterized in that, Step S6 fills the outlier region based on the surrounding values.

Images