Infrared image noise reduction method and device based on local noise template

Abstract

The application discloses an infrared image denoising method and device based on a local noise template. Firstly, the application acquires temperature drift images under different temperatures, and filters the temperature drift images based on a three-stage filter sequentially constructed by a spatial mean filter, a one-dimensional longitudinal guide filter and a one-dimensional horizontal guide filter to obtain a noise template. Then, for a real-time infrared image, high-frequency image information is extracted based on the three-stage filter, real-time noise is estimated based on the noise template and the real-time high-frequency image information, and finally, the real-time noise is removed from the real-time infrared image to obtain a denoised infrared image. The application can have a good denoising effect under the condition of less resource occupation, has a strong reservation ability for details, and realizes a substantial improvement of the image stability time and the image quality.

Description

Technical Field

[0001] The present invention relates to the technical field of uncooled infrared imaging, and in particular to a method and device for reducing infrared image noise based on a local noise template. Background Art

[0002] Due to limitations in imaging principles and detector technology, uncooled infrared imaging produces images containing significant amounts of fixed noise. A variety of noise reduction algorithms are required to achieve high-quality, low-noise images. Typical noise reduction algorithms include those based on image layering, local mean, spatiotemporal, and deep learning. Currently, these and local mean are the most widely used. Typical examples include guided filtering and bilateral filtering. These algorithms offer low computational complexity and are suitable for implementation on embedded platforms such as FPGAs. However, they can easily filter out details in images with rich scenes or subtle details. Spatiotemporal noise reduction algorithms primarily remove temporal noise, but their ability to remove fixed noise is limited. They also require the storage of multiple images, resulting in high resource consumption. Deep learning algorithms offer excellent denoising results, but they consume significantly more resources than the other algorithms, making them unsuitable for FPGA implementation. Summary of the Invention

[0003] In view of this, the present invention provides an infrared image denoising method and device based on a local noise template, which can ensure good denoising effect with less resource occupation, and at the same time has a strong ability to retain details, thereby significantly improving the image stabilization time and image quality.

[0004] The infrared image denoising method based on a local noise template of the present invention comprises the following steps:

[0005] Step 1: Obtain temperature drift images TI after correction at different temperatures (assuming N images are collected) i , i = 1, 2, ..., N. The acquisition method of each temperature drift image is as follows:

[0006] Assume that the temperatures of N temperature points are T i , i = 1, 2, ..., N, then when collecting the temperature drift image of the i-th temperature point, first place the imaging component in an environment with an ambient temperature of T i The photosensitive surface of the imaging component is placed in a constant temperature box and faces the black body with a uniform temperature field. After two-point non-uniformity correction, the initial image I is obtained. i , gradually increase the temperature of the thermostat at a certain speed to reach temperature T i+1 , collect the image after temperature drift I i+1 , then the temperature drift image of the i-th temperature point is:

[0007] TI i =I i+1 -I i .

[0008] By analogy, the temperature drift images of all temperature points can be obtained.

[0009] Single-point or multi-point calibration can also be used.

[0010] Step 2: Extract the noise template from the temperature drift image through a multi-stage low-pass filter and store it in the memory, where the multi-stage low-pass filter is a spatial mean filter, a one-dimensional longitudinal guide filter, and a one-dimensional horizontal guide filter in sequence:

[0011] NoiseTemp1 i =LP1(TI i )

[0012] NoiseTemp2 i =LP2(TI i )

[0013] NoiseTemp3 i =LP3(TI i )

[0014] Among them, LP1 represents spatial mean filtering, LP2 represents one-dimensional longitudinal guided filtering, and LP3 represents one-dimensional horizontal guided filtering. i , NoiseTemp2 i , NoiseTemp3 i The three noise templates are the first, second, and third level. The third level noise template can achieve the best noise reduction effect, but if resources are limited, using only the first or second level noise template can also achieve a relatively significant noise reduction effect.

[0015] Step 3: Extract high-frequency image information ImageHF from the real-time infrared image (the image after correction). Assuming that the real-time infrared image is Image, the calculation method is:

[0016] ImageHF1=LP1(Image)

[0017] ImageHF2=LP1(Image)

[0018] ImageHF3=LP1(Image)

[0019] Among them, LP1 represents spatial mean filtering, LP2 represents one-dimensional longitudinal guided filtering, and LP3 represents one-dimensional horizontal guided filtering. ImageHF1 i, ImageHF2 i , ImageHF3 i They are the first, second and third level high frequency image information respectively. The high frequency information image here must use ImageHF3 i In order to effectively suppress over-extraction of information, the final high-frequency image information is ImageHF=ImageHF3.

[0020] Step 4: There is a strong correlation between the real-time noise and the noise template. The real-time noise can be obtained by local linear transformation of the noise template. Therefore, the real-time noise can be estimated based on the pre-stored noise template and the high-frequency image information calculated in real time. The estimation method is as follows:

[0021]

[0022]

[0023] Where a(h,w) represents the noise gain coefficient of the pixel in the hth row and the wth column, a is the gain coefficient map, and w rand is a local window centered at coordinates (h, w), and ImageHF(m, n) is the local window of ImageHF in window w. rand The pixel in row m and column n, NoiseTemp3 i (m,n) is NoiseTemp3 i In the window rand The pixel in the mth row and nth column, wd a It has a serial port centered on the coordinate (h, w). is the final noise gain coefficient after local mean filtering of local a(h,w), Noise(h,w) is the estimated noise of the pixel in the hth row and wth column, and Noise is the estimated noise map.

[0024] The step 5: obtaining a real-time image after noise reduction, the calculation method is:

[0025] Image out =Image-Noise.

[0026] Image out The final image after the image Image is denoised.

[0027] The present invention also provides a noise reduction device using the above noise reduction method, comprising an image correction module, a three-stage low-pass filter, a storage unit, a real-time noise estimation unit and a noise reduction unit;

[0028] Among them, the image correction module is used to correct the real-time infrared image;

[0029] The three-stage low-pass filter is a spatial mean filter, a one-dimensional longitudinal guide filter, and a one-dimensional horizontal guide filter, which are used to perform primary, secondary, and tertiary filtering on the image.

[0030] The storage unit is used to store the corrected temperature drift image at each temperature and the primary, secondary and tertiary noise templates at each temperature obtained by filtering the temperature drift image with a three-stage filter;

[0031] The real-time noise estimation unit estimates the real-time noise based on the high-frequency image information obtained after the real-time infrared image is filtered by the three-stage filter and the first-level, second-level or third-level noise template at the same temperature in the storage unit;

[0032] The denoising unit is used to subtract the real-time noise estimated by the real-time noise estimation unit from the real-time infrared image to obtain a denoised infrared image.

[0033] The present invention also provides a computer program product for infrared image noise reduction based on a local noise template. The computer program product includes a non-volatile readable storage medium and a computer program. The computer program is tangibly stored on the non-volatile readable storage medium. The computer program is executed by a processor in an FPGA, DSP, ARM or other computer to implement the steps of the above-mentioned noise reduction method.

[0034] Beneficial effects:

[0035] 1) Combined with the passing noise template, the noise level is greatly reduced without losing detail information, which has a significant effect on improving the quality and stabilization time of uncooled infrared imaging.

[0036] 2) The algorithm complexity of this technology is low and it is easy to implement on different platforms such as FPGA, DSP, ARM, PC, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Flow chart of the method of the present invention.

[0038] Figure 2 is the image to be denoised.

[0039] Figure 3 is the extracted noise template.

[0040] Figure 4 The image after template-based denoising. DETAILED DESCRIPTION

[0041] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0042] The present invention provides an infrared image noise reduction method based on a local noise template. The method is described using an infrared core component with an operating temperature range of -40°C to 60°C and a resolution of 640×512 as an example.

[0043] Step 1: Obtain temperature drift images TI at different temperatures (assuming N = 11) after two-point correction i , i = 1, 2, ..., 11. The acquisition method of each temperature drift image is as follows:

[0044] Assume that the temperatures of N temperature points are T i , i=1,2,…,11,the collected temperatures are evenly distributed, then T1=-40℃,T2=-30℃,…,T 11 = 60℃. When collecting the temperature drift image at the i-th temperature point, first place the imaging component in an environment with an ambient temperature of T i The photosensitive surface of the imaging component is placed in a constant temperature box and faces the black body with a uniform temperature field. After two-point non-uniformity correction, the initial image I is obtained. i , gradually increase the temperature of the thermostat at a certain speed to reach temperature T i+1 , collect the image after temperature drift I i+1 , then the temperature drift image of the i-th temperature point is:

[0045] TI i =I i+1 -I i .

[0046] By analogy, the temperature drift images of all temperature points can be obtained.

[0047] Step 2: Extract the noise template from the temperature drift image through a multi-stage low-pass filter and store it in the memory, where the multi-stage low-pass filter is a spatial mean filter, a one-dimensional longitudinal guide filter, and a one-dimensional horizontal guide filter in sequence:

[0048] NoiseTemp1 i =LP1(TI i )

[0049] NoiseTemp2 i =LP2(TI i )

[0050] NoiseTemp3 i =LP3(TI i )

[0051] Among them, LP1 represents spatial mean filtering, LP2 represents one-dimensional longitudinal guided filtering, and LP3 represents one-dimensional horizontal guided filtering. i , NoiseTemp2 i, NoiseTemp3 i They are the first, second and third level noise templates respectively. Using the third level noise template, we can get Figure 3 shown.

[0052] Step 3: Extract high-frequency image information ImageHF from the real-time infrared image (the image after two-point correction). The real-time infrared image is ImageHF. Figure 2 As shown, the calculation method is:

[0053] ImageHF1=LP1(Image)

[0054] ImageHF2=LP2(Image)

[0055] ImageHF3=LP3(Image)

[0056] Among them, LP1 represents spatial mean filtering, LP2 represents one-dimensional longitudinal guided filtering, and LP3 represents one-dimensional horizontal guided filtering. ImageHF1 i , ImageHF2 i , ImageHF3 i They are the first, second, and third level high-frequency image information respectively, so the final high-frequency image information is ImageHF=ImageHF3.

[0057] Step 4: Estimate the real-time noise based on the pre-stored noise template and the high-frequency image information calculated in real time. Assume that the window size of the noise gain coefficient is 16. The estimation method is as follows:

[0058]

[0059]

[0060]

[0061] Where a(h,w) represents the noise gain coefficient of the pixel in the hth row and the wth column, a is the gain coefficient map, and w rand is a local window centered at coordinates (h, w), and ImageHF(m, n) is the local window of ImageHF in window w. rand The pixel in row m and column n, NoiseTemp3 i (m,n) is NoiseTemp3 i In the window rand The pixel in the mth row and nth column, wd a It has a serial port centered on the coordinate (h, w). is the final noise gain coefficient after local mean filtering of local a(h,w), Noise(h,w) is the estimated noise of the pixel in the hth row and wth column, and Noise is the estimated noise map.

[0062] The step 5: obtaining a real-time image after noise reduction, the calculation method is:

[0063] Image out =Image-Noise.

[0064] Image out The final image after the image Image is denoised.

[0065] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for infrared image denoising based on local noise template, characterized in that: include: S1, obtain different temperatures The temperature drift image after correction , ; S2, constructing a three-stage low-pass filter, wherein the three-stage low-pass filters are: spatial mean filter, one-dimensional longitudinal guidance filter and one-dimensional horizontal guidance filter; The temperature drift image in step S1 is subjected to primary, secondary and tertiary filtering in sequence using the three-stage low-pass filter to obtain primary, secondary and tertiary noise templates at different temperatures; S3, correcting the real-time infrared image, and then performing three-stage filtering based on the three-stage low-pass filter to extract the high-frequency image information of the real-time infrared image ; S4, high-frequency image information based on real-time infrared images Estimating real-time noise using a primary, secondary, or tertiary noise template at the same temperature as step S2, wherein the real-time noise is obtained by a local linear transformation of the noise template; S5 , subtracting the real-time noise estimated in step S4 from the real-time infrared image to obtain a noise-reduced infrared image.

2. The method according to claim 1, wherein In step S1, the temperature Temperature drift image under The method to obtain is as follows: Place the imaging assembly in an environment with a temperature of The initial image is obtained after two-point non-uniformity correction. ; Gradually increase the temperature of the thermostat at a certain speed to reach the temperature , collect images after temperature drift , then the temperature The temperature drift image below is: 。 3. The method according to claim 1, wherein In step S4, the real-time noise is estimated in the following manner: in Indicates the Rank The noise gain coefficient of the column pixel, is the gain coefficient graph, The coordinates A local window centered on yes In the window Middle Rank Column pixels, is the noise template In the window Middle Rank Column pixels, the noise template That is the first, second or third level noise template in step S2, The coordinates A local window centered on yes The final noise gain coefficient after local mean filtering is, It is Rank The estimated noise of the column pixels, is the estimated noise map.

4. The method according to any one of claims 1 to 3, wherein: In S1 and S3, the calibration adopts single-point, two-point or multi-point calibration.

5. An infrared image denoising device based on a local noise template, characterized in that: The apparatus comprises an image correction module, a three-stage low-pass filter, a storage unit, a real-time noise estimation unit and a denoising unit, and adopts the denoising method according to any one of claims 1 to 4 for denoising; Among them, the image correction module is used to correct the real-time infrared image; The three-stage low-pass filter is a spatial mean filter, a one-dimensional longitudinal guide filter, and a one-dimensional horizontal guide filter, which are used to perform primary, secondary, and tertiary filtering on the image; The storage unit is used to store the corrected temperature drift image at each temperature and the primary, secondary and tertiary noise templates at each temperature obtained by filtering the temperature drift image with a three-stage filter; The real-time noise estimation unit estimates the real-time noise based on the high-frequency image information obtained after the real-time infrared image is filtered by the three-stage filter and the first-level, second-level or third-level noise template at the same temperature in the storage unit; The denoising unit is used to subtract the real-time noise estimated by the real-time noise estimation unit from the real-time infrared image to obtain a denoised infrared image.

6. A computer program product for infrared image denoising based on a local noise template, characterized in that: The method comprises a non-transitory readable storage medium and a computer program, wherein the computer program is tangibly stored on the non-transitory readable storage medium, and the computer program is executed by a processor in an FPGA, DSP, ARM or other computer to implement the steps of the noise reduction method according to any one of claims 1 to 4.

Images