Infrared image noise reduction method and device based on local noise template

Abstract

The invention discloses an infrared image noise reduction method and device based on a local noise template. The method comprises the following steps: firstly, acquiring temperature drift images at different temperatures, and filtering the temperature drift images based on a three-stage filter constructed by spatial domain mean filtering, one-dimensional longitudinal guided filtering and one-dimensional horizontal guided filtering in sequence to obtain a noise template; and for the real-time infrared image, firstly extracting high-frequency image information based on a three-stage filter, then estimating real-time noise based on a noise template and the high-frequency image information calculated in real time, and finally removing the real-time noise from the real-time infrared image to obtain a denoised infrared image. According to the method, a good denoising effect can be achieved under the condition that less resources are occupied, meanwhile, the capability of retaining details is high, the stabilization time of the image is greatly prolonged, and the image quality is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of uncooled infrared imaging, and in particular to a method and device for reducing denoising of an infrared image based on a local noise template. Background Art

[0002] Due to the limitations of the imaging principle and detector technology of uncooled infrared imaging, the resulting images contain a large amount of fixed noise, and a variety of noise reduction algorithms are required to obtain high-quality and low-noise images. Typical noise reduction algorithms include noise reduction algorithms based on image stratification, noise reduction algorithms based on local mean, noise reduction algorithms based on spatiotemporal domain, and noise reduction algorithms based on deep learning. Among them, noise reduction algorithms based on image stratification and noise reduction algorithms based on local mean are currently the most widely used types of noise reduction algorithms, among which typical ones are noise reduction algorithms based on guided filtering and noise reduction algorithms based on bilateral filtering. The advantages of these algorithms are low computational complexity and suitable for implementation on embedded platforms such as FPGA. The disadvantage is that for images with rich scenes or weak details, details are easily filtered out; noise reduction algorithms based on spatiotemporal domain mainly remove noise in the time domain, and have limited ability to remove fixed noise. At the same time, multiple images need to be stored, and resources are large; algorithms based on deep learning have better denoising effects, but compared with other algorithms, resource usage increases significantly, which is not conducive to FPGA implementation. Summary of the invention

[0003] In view of this, the present invention provides an infrared image denoising method and device based on a local noise template, which can ensure a good denoising effect with less resource occupation, and at the same time has a strong ability to retain details, thereby greatly improving the image stabilization time and image quality.

[0004] The infrared image denoising method based on local noise template of the present invention comprises the following steps:

[0005] Step 1: Obtain the temperature drift images TI after correction at different temperatures (assuming that N images are collected) i , i = 1, 2, ..., N. The acquisition method of each temperature drift image is as follows:

[0006] Assume that the temperatures of N temperature points are T i , i = 1, 2, ..., N, then when collecting the temperature drift image at the i-th temperature point, first place the imaging component in an environment with an ambient temperature of T i The photosensitive surface of the imaging component is placed in a constant temperature box and faces the black body with a uniform temperature field. After two-point non-uniformity correction, the initial image I is obtained. i , gradually increase the temperature of the thermostat at a certain speed to reach temperature T i+1 , collect the image after temperature drift I i+1 , then the temperature drift image of the i-th temperature point is:

[0007] TI i =I i+1 -I i .

[0008] In this way, the temperature drift images of all temperature points can be obtained.

[0009] Single-point or multi-point calibration can also be used.

[0010] Step 2: Extract the noise template from the temperature drift image through a multi-stage low-pass filter and store it in the memory, where the multi-stage low-pass filter is spatial mean filtering, one-dimensional longitudinal guided filtering, and one-dimensional horizontal guided filtering in sequence:

[0011] NoiseTemp1 i =LP1(TI i )

[0012] NoiseTemp2 i =LP2(TI i )

[0013] NoiseTemp3 i =LP3(TI i )

[0014] Among them, LP1 represents spatial mean filtering, LP2 represents one-dimensional longitudinal guided filtering, and LP3 represents one-dimensional horizontal guided filtering. i , NoiseTemp2 i , NoiseTemp3 i They are first-level, second-level, and third-level noise templates. The third-level noise template can achieve the best noise reduction effect, but when resources are limited, using only the first-level or second-level noise template can also achieve a relatively obvious noise reduction effect.

[0015] Step 3: Extract high-frequency image information ImageHF from the real-time infrared image (the image after correction). Assuming that the real-time infrared image is Image, the calculation method is:

[0016] ImageHF1=LP1(Image)

[0017] ImageHF2=LP1(Image)

[0018] ImageHF3=LP1(Image)

[0019] Among them, LP1 represents the spatial mean filter, LP2 represents the one-dimensional longitudinal guided filter, and LP3 represents the one-dimensional horizontal guided filter. ImageHF1 i, ImageHF2 i , ImageHF3 i They are the first, second and third level high frequency image information respectively. The high frequency information image here must use ImageHF3 i Only in this way can the over-extraction of information be effectively suppressed, so the final high-frequency image information is ImageHF=ImageHF3.

[0020] Step 4: There is a strong correlation between the real-time noise and the noise template. The real-time noise can be obtained by local linear transformation of the noise template. Therefore, the real-time noise can be estimated based on the noise template stored in advance and the high-frequency image information calculated in real time. The estimation method is as follows:

[0021]

[0022]

[0023] Where a(h,w) represents the noise gain coefficient of the pixel in the hth row and the wth column, a is the gain coefficient map, and w rand is a local window centered at coordinates (h,w), and ImageHF(m,n) is the local window of ImageHF in window w. rand The pixel in the mth row and nth column, NoiseTemp3 i (m,n) is NoiseTemp3 i In the window rand The pixel in the mth row and nth column, wd a It is centered at coordinates (h,w) and has a serial port. is the final noise gain coefficient after local mean filtering of a(h,w), Noise(h,w) is the estimated noise of the pixel in the hth row and wth column, and Noise is the estimated noise map.

[0024] The step 5: obtaining a real-time image after noise reduction, the calculation method is:

[0025] Image out =Image-Noise.

[0026] Image out It is the final image after the image Image is denoised.

[0027] The present invention also provides a noise reduction device using the above noise reduction method, comprising an image correction module, a three-stage low-pass filter, a storage unit, a real-time noise estimation unit and a noise reduction unit;

[0028] Among them, the image correction module is used to correct the real-time infrared image;

[0029] The three-stage low-pass filter is a spatial mean filter, a one-dimensional longitudinal guide filter, and a one-dimensional horizontal guide filter, respectively; it is used to perform primary, secondary, and tertiary filtering on the image;

[0030] The storage unit is used to store the corrected temperature drift image at each temperature and the primary, secondary and tertiary noise templates at each temperature obtained by filtering the temperature drift image with a three-stage filter;

[0031] The real-time noise estimation unit estimates the real-time noise based on the high-frequency image information obtained after the real-time infrared image is filtered by the three-stage filter and the primary, secondary or tertiary noise template at the same temperature in the storage unit;

[0032] The denoising unit is used to subtract the real-time noise estimated by the real-time noise estimation unit from the real-time infrared image to obtain a denoised infrared image.

[0033] The present invention also provides a computer program product for infrared image denoising based on a local noise template, the computer program product comprising a non-volatile readable storage medium and a computer program, the computer program being tangibly stored on the non-volatile readable storage medium, and the computer program being executed by a processor in an FPGA, DSP, ARM or other computer to implement the steps of the above-mentioned denoising method.

[0034] Beneficial effects:

[0035] 1) Combined with the passing noise template, the noise level can be greatly reduced without losing detail information, which has a significant effect on improving the quality and stabilization time of uncooled infrared imaging.

[0036] 2) The algorithm complexity of this technology is low, and it is easy to implement on different platforms such as FPGA, DSP, ARM, and PC. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The figure is a flow chart of the method of the present invention.

[0038] Figure 2 is the image to be denoised.

[0039] Figure 3 is the extracted noise template.

[0040] Figure 4 This is the image after template-based denoising. DETAILED DESCRIPTION

[0041] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0042] The present invention provides an infrared image denoising method based on a local noise template, which is described by taking an infrared core component with an operating temperature range of -40°C to 60°C and a resolution of 640×512 as an example:

[0043] Step 1: Obtain temperature drift images TI at different temperatures (assuming that N = 11 images are collected) after two-point correction i , i = 1, 2, ..., 11. The acquisition method of each temperature drift image is as follows:

[0044] Assume that the temperatures of N temperature points are T i , i = 1, 2, ..., 11, and the collected temperatures are uniformly distributed, then T1 = -40 °C, T2 = -30 °C, ..., T 11 =60°C. When collecting the temperature drift image at the i-th temperature point, first place the imaging component in an environment with a temperature of T i The photosensitive surface of the imaging component is placed in a constant temperature box and faces the black body with a uniform temperature field. After two-point non-uniformity correction, the initial image I is obtained. i , gradually increase the temperature of the thermostat at a certain speed to reach temperature T i+1 , collect the image after temperature drift I i+1 , then the temperature drift image of the i-th temperature point is:

[0045] TI i =I i+1 -I i .

[0046] In this way, the temperature drift images of all temperature points can be obtained.

[0047] Step 2: Extract the noise template from the temperature drift image through a multi-stage low-pass filter and store it in the memory, where the multi-stage low-pass filter is spatial mean filtering, one-dimensional longitudinal guided filtering, and one-dimensional horizontal guided filtering in sequence:

[0048] NoiseTemp1 i =LP1(TI i )

[0049] NoiseTemp2 i =LP2(TI i )

[0050] NoiseTemp3 i =LP3(TI i )

[0051] Among them, LP1 represents spatial mean filtering, LP2 represents one-dimensional longitudinal guided filtering, and LP3 represents one-dimensional horizontal guided filtering. i , NoiseTemp2 i, NoiseTemp3 i They are the first, second, and third level noise templates. Using the third level noise template, we can get Figure 3 shown.

[0052] Step 3: Extract high-frequency image information ImageHF from the real-time infrared image (the image after two-point correction), the real-time infrared image is Image Figure 2 As shown, the calculation method is:

[0053] ImageHF1=LP1(Image)

[0054] ImageHF2=LP2(Image)

[0055] ImageHF3=LP3(Image)

[0056] Among them, LP1 represents the spatial mean filter, LP2 represents the one-dimensional longitudinal guided filter, and LP3 represents the one-dimensional horizontal guided filter. ImageHF1 i , ImageHF2 i , ImageHF3 i They are the first, second and third level high frequency image information respectively, so the final high frequency image information is ImageHF=ImageHF3.

[0057] Step 4: Estimate the real-time noise based on the noise template stored in advance and the high-frequency image information calculated in real time. Assume that the window size of the noise gain coefficient is 16. The estimation method is as follows:

[0058]

[0059]

[0060]

[0061] Where a(h,w) represents the noise gain coefficient of the pixel in the hth row and the wth column, a is the gain coefficient map, and w rand is a local window centered at coordinates (h,w), and ImageHF(m,n) is the local window of ImageHF in window w. rand The pixel in the mth row and nth column, NoiseTemp3 i (m,n) is NoiseTemp3 i In the window rand The pixel in the mth row and nth column, wd a It is centered at coordinates (h,w) and has a serial port. is the final noise gain coefficient after local mean filtering of a(h,w), Noise(h,w) is the estimated noise of the pixel in the hth row and wth column, and Noise is the estimated noise map.

[0062] The step 5: obtaining a real-time image after noise reduction, the calculation method is:

[0063] Image out =Image-Noise.

[0064] Image out It is the final image after the image Image is denoised.

[0065] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for infrared image denoising based on local noise template, characterized in that: include: S1, obtain different temperatures T i The temperature drift image TI after correction i , i=1,2,…,N; S2, constructing a three-stage low-pass filter, wherein the three-stage low-pass filters are: spatial mean filter, one-dimensional longitudinal guide filter and one-dimensional horizontal guide filter; The temperature drift image in step S1 is filtered in the first, second and third order by using the three-order low-pass filter to obtain the first, second and third order noise templates at different temperatures; S3, correcting the real-time infrared image, and then performing three-stage filtering based on the three-stage low-pass filter to extract high-frequency image information ImageHF of the real-time infrared image; S4, estimating real-time noise based on the high-frequency image information ImageHF of the real-time infrared image and the primary, secondary or tertiary noise template at the same temperature as step S2, wherein the real-time noise is obtained by local linear transformation of the noise template; S5, subtracting the real-time noise estimated in step S4 from the real-time infrared image to obtain a de-noised infrared image.

2. The method according to claim 1, characterized in that In step S1, the temperature T i Temperature drift image TI i The method to obtain is as follows: Place the imaging component in an environment with a temperature of T i The photosensitive surface of the imaging component is placed in a constant temperature box and faces the black body with a uniform temperature field. After two-point non-uniform correction, the initial image I is obtained. i ; The temperature of the thermostat is gradually increased at a certain speed to reach temperature T i+1 , collect the image after temperature drift I i+1 , then the temperature T i The temperature drift image below is: YOU i =I i+1 -AND i 。 3. The method according to claim 1, characterized in that In step S4, the real-time noise is estimated in the following manner: Where a(h,w) represents the noise gain coefficient of the pixel in the hth row and the wth column, a is the gain coefficient map, and w rand is a local window centered at coordinates (h,w), and ImageHF(m,n) is the local window of ImageHF in window w. rand For the pixel in the mth row and nth column, NoiseTemp(m,n) is the noise template NoiseTemp in the window w. rand The pixel in the mth row and nth column in the image, the noise template NoiseTemp is the primary, secondary or tertiary noise template in step S2, wd a It is centered at coordinates (h,w) and has a serial port. is the final noise gain coefficient after local mean filtering of a(h,w), Noise(h,w) is the estimated noise of the pixel in the hth row and wth column, and Noise is the estimated noise map.

4. The method according to any one of claims 1 to 3, characterized in that: In S1 and S3, the calibration adopts single-point, two-point or multi-point calibration.

5. An infrared image denoising device based on a local noise template, characterized in that: It comprises an image correction module, a three-stage low-pass filter, a storage unit, a real-time noise estimation unit and a denoising unit, and adopts the denoising method according to any one of claims 1 to 4 to perform denoising; Among them, the image correction module is used to correct the real-time infrared image; The three-stage low-pass filter is a spatial mean filter, a one-dimensional longitudinal guide filter, and a one-dimensional horizontal guide filter, which are used to perform primary, secondary, and tertiary filtering on the image; The storage unit is used to store the corrected temperature drift image at each temperature and the primary, secondary and tertiary noise templates at each temperature obtained by filtering the temperature drift image with a three-stage filter; The real-time noise estimation unit estimates the real-time noise based on the high-frequency image information obtained after the real-time infrared image is filtered by the three-stage filter and the primary, secondary or tertiary noise template at the same temperature in the storage unit; The denoising unit is used to subtract the real-time noise estimated by the real-time noise estimation unit from the real-time infrared image to obtain a denoised infrared image.

6. A computer program product for infrared image denoising based on local noise template, characterized in that: It comprises a non-volatile readable storage medium and a computer program, wherein the computer program is tangibly stored on the non-volatile readable storage medium, and the computer program is executed by a processor in an FPGA, DSP, ARM or other computer to implement the steps of the noise reduction method as described in any one of claims 1 to 4.

Images