A multi-scale hierarchical collaborative optimization method for non-uniformity correction of infrared image

Abstract

This invention discloses a multi-scale, hierarchical, collaborative optimization method for infrared image non-uniformity correction, comprising: 1) constructing a multi-scale pyramid, dynamically adjusting the Gaussian kernel parameters based on local information entropy to generate an adaptive Gaussian pyramid, and extracting high-frequency residual layers through a Laplacian pyramid; 2) joint frequency-spatial domain fringe suppression, using 36-direction FFT polar coordinate spectral energy integration to detect the main fringe direction, and combining directional Gabor filters and rotated median filters to achieve directional fringe suppression; 3) multi-level collaborative optimization: blocking noise cross-scale propagation through a multi-level filtering mechanism of main-level enhancement and secondary-level attenuation; and 4) residual detail enhancement and dynamic reconstruction. This invention, by constructing a locally entropy-driven adaptive Gaussian pyramid, combining fringe suppression with a dynamic noise blocking mechanism, and employing improved nonlocal mean sharpening and high-level edge inverse injection to achieve detail enhancement and reconstruction, effectively eliminates spectral aliasing, avoids excessive smoothing and sharpening, and improves the quality of infrared images.

Description

Technical Field

[0001] This invention relates to a multi-scale hierarchical collaborative optimization method for infrared image non-uniformity correction, belonging to the field of infrared image processing technology. Background Technology

[0002] In infrared imaging systems, variations in detector pixel manufacturing processes and environmental temperature fluctuations can lead to directional, striped, non-uniform noise in the output image. This noise manifests as periodic bright and dark stripes in the spatial domain and as high-energy spectral components in a specific direction in the frequency domain, severely reducing image contrast and detail sharpness. Existing correction methods mainly suffer from the following technical bottlenecks: Limitations of single-scale filtering methods: Traditional spatial filtering methods (such as median filtering and Gaussian smoothing) use fixed-size filter kernels, making it difficult to distinguish high-frequency noise from real texture structures. Isotropic filtering operations leave significant noise along the stripe direction and cause excessive smoothing of details in the vertical direction. Especially for interwoven diagonal stripes in complex scenes, single-scale processing can cause noise and detail to be confused, resulting in blurred edges and loss of effective information.

[0003] Insufficient accuracy in directional noise detection: Existing frequency domain correction methods rely on filter banks with preset directions (such as horizontal / vertical directions), failing to accurately identify the main noise direction. When the fringe direction deviates from the filter design direction, spectral energy cannot be effectively suppressed, and residual noise leaks into adjacent frequency bands through frequency domain sidelobes, forming artifacts. Furthermore, the rotation correction processing of tilted fringes in traditional methods introduces interpolation errors, exacerbating image geometric distortion.

[0004] Multi-scale noise propagation and coupling effects: Traditional pyramid decomposition-based methods use fixed fusion weights during hierarchical reconstruction, failing to consider the differences in noise energy distribution across different scales. High-level (low-resolution) residual noise diffuses to lower levels during upsampling through interpolation, forming a cross-scale noise propagation chain. This coupling effect causes noise to be repeatedly amplified during reconstruction, especially in low signal-to-noise ratio regions, leading to blocky artifacts.

[0005] The contradiction between detail enhancement and noise suppression: Traditional nonlocal means algorithms preserve texture through a wide search window, but suffer from high computational complexity and significant parameter sensitivity. Fixed-intensity residual sharpening simultaneously amplifies high-frequency noise and details, leading to signal-to-noise ratio degradation in low-contrast areas. Existing methods fail to achieve dynamic decoupling between noise features and detail components, resulting in sharpened images prone to edge ringing and granular noise accumulation.

[0006] The common problems of existing methods can be summarized as follows: spectral aliasing in directional noise suppression, uncontrollability of multi-scale noise propagation, static trade-off between detail enhancement and noise reduction, and the contradiction between complex algorithms and hardware resources. These shortcomings limit the applicability of existing technologies in complex infrared scenes (such as rapid temperature changes and multi-angle stripe noise), and there is an urgent need for a new correction method that can achieve joint frequency-spatial domain analysis, dynamic hierarchical optimization, and efficient resource coordination. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings and deficiencies of existing technologies by providing a multi-scale, hierarchical, collaborative optimization method for infrared image non-uniformity correction. This method constructs a multi-scale pyramid, generates multi-resolution images using an adaptive Gaussian kernel, and dynamically adjusts the smoothing intensity based on local information entropy, avoiding detail loss or noise residue caused by fixed parameters in traditional Gaussian kernels. Interval sampling generates a hierarchical structure (resolution halved layer by layer), laying the foundation for subsequent multi-scale noise analysis. The Gaussian kernel guided by local information entropy avoids excessive blurring in flat areas and undersmoothing in textured areas. Interval sampling reduces redundant computation, and the hierarchical structure is compatible with subsequent parallel processing.

[0008] The technical solution adopted by this invention to solve its technical problem is: a multi-scale hierarchical collaborative optimization method for infrared image non-uniformity correction, which includes the following steps: Step 1: Construct a multi-scale pyramid; Generate a two-dimensional Gaussian kernel ,in The standard deviation smoothing parameter is adaptively adjusted through local information entropy. This represents the position coordinates of a pixel in the image. The smoothed image is sampled at intervals to generate a multi-resolution hierarchy {G0, G1, ..., G...}. n}, where G0 is the original image, and the resolution of each layer is half that of the previous layer; starting from the second-to-last layer of the Gaussian pyramid, it is upsampled to the current layer size through bilinear interpolation, and the high-frequency residual layer {H1, H2, ..., H} is obtained by calculating the Gaussian smoothed residual. n}, used for multi-scale analysis of fringe non-uniformity; Step 2: Joint frequency-spatial domain stripe suppression; Frequency domain Fast Fourier Transform (FFT) analysis was performed on the Laplace residual layer to detect the spectral energy integrals in 36 directions, determine the fringe direction, apply directional median filtering to the horizontal fringes, and apply a directional Gabor filter to suppress the fringes in the frequency domain. The formula is as follows: in, The center frequency of the fringe is The direction of the stripes, For the input frequency and direction, and These are the standard deviations of frequency and direction, respectively; Step 3: Multi-level collaborative optimization; The level with the highest noise intensity is selected as the main level, and a 1.3 times enhanced filter is applied to it; the secondary levels are weakly filtered in the main direction, with a filter intensity attenuation coefficient of 0.7; by passing through the main direction, it is ensured that the filtering direction of different levels is consistent, and residual noise caused by directional differences is avoided. Step 4: Residual detail enhancement and dynamic reconstruction; The process extracts and enhances details using a denoising process. First, slight denoising is performed. Then, the difference between the original image and the denoised image is calculated to create a detail image. This detail image is then weighted and added back to the original image to achieve a sharpening effect. The complete workflow formula is: Where ℎ represents the filter strength. As the sharpening factor, For the original input image, NLM is an improved nonlocal mean sharpening method. Based on median filtering residual estimation of noise standard deviation, the hierarchical fusion coefficients are adjusted by an adaptive forgetting factor, and high-level edge features are inversely injected. =0.2 weight) to optimize low-level detail recovery and output image.

[0009] Furthermore, the adaptive Gaussian kernel is generated in step 1 of this invention as follows: in That is the original standard deviation.

[0010] Furthermore, the frequency domain FFT analysis in step 3 of this invention includes: performing polar coordinate transformation on the FFT spectrum and calculating the energy integral of the annular region at 5° intervals; when the energy in three consecutive directions exceeds 2.5 times the mean, the middle direction is taken as the average. .

[0011] Furthermore, in step 3 of this invention, for horizontal stripes (approximately 0° or 180°), 1 A median filter window of 7 is used. For stripes in the tilt direction, the image is rotated, and then a 1 is applied. The median filter window is 7.

[0012] Furthermore, the primary level selection condition in step 4 of this invention is: in, To analyze the maximum fringe intensity in the frequency domain, As a hierarchy, The main fringe directions are shown in the frequency domain analysis.

[0013] Furthermore, the fast nonlocal means denoising in step 5 of this invention is as follows: The weighting function simplifies to: in, The original input image, The image after denoising. This is a search window centered on pixel (x,y). For an image patch centered at pixel (x,y), Normalization factor This is the filter strength parameter.

[0014] Furthermore, the residual detail extraction in step 5 of this invention is as follows: in, This is a high-frequency detail map, containing noise and texture information.

[0015] Furthermore, the adaptive detail enhancement in step 5 of this invention is as follows: in, Sharpening intensity control factor.

[0016] Furthermore, in step 6 of this invention, the forgetting factor adjustment during reconstruction is based on the noise intensity attenuation fusion coefficient ( )for: in, Forgetting factor, This represents the hierarchical noise intensity.

[0017] Furthermore, in step 6 of this invention, the feature map of the high-level pyramid is injected into the low-level pyramid through upsampling as follows: in, hierarchical The enhanced edge feature map has a fusion weight of . =0.2, edge(L i ) is the low-level feature map L i Edge features obtained by edge detection.

[0018] Furthermore, in step 6 of this invention, during the reconstruction process, the use of high-level edges to enhance low-level details is as follows: in, =0.1 to avoid over-sharpening. This is the reconstructed image.

[0019] Beneficial effects: 1. This invention constructs a multi-scale pyramid and generates multi-resolution images using an adaptive Gaussian kernel. The smoothing intensity is dynamically adjusted based on local information entropy, avoiding detail loss or noise residue caused by the fixed parameters of traditional Gaussian kernels. Interval sampling generates a hierarchical structure (resolution halved layer by layer), laying the foundation for subsequent multi-scale noise analysis. The Gaussian kernel guided by local information entropy avoids over-blurring in flat areas and under-smoothing in textured areas. Interval sampling reduces redundant computation, and the hierarchical structure is compatible with subsequent parallel processing.

[0020] 2. This invention utilizes frequency domain analysis and directional fringe suppression, employing a 36-direction FFT polar coordinate spectral energy integration detection technique. It accurately locates the main direction of fringe noise by using a continuous three-direction threshold (mean 2.5 times) criterion, combined with a directional Gabor filter and a rotated median filter (1... The synergistic effect of 7 windows enables directional suppression of periodic fringes. This scheme can eliminate most of the directional non-uniformity while avoiding the detail blurring problem caused by traditional omnidirectional filtering.

[0021] 3. This invention, through hierarchical collaborative optimization and noise propagation blocking, and based on the multi-level residual characteristics of the Laplace pyramid, designs a main-level enhancement (…). =1.3) and secondary level decay ( The dynamic optimization mechanism (=0.7) involves: the primary level applying enhanced filtering for maximum noise intensity, while the secondary level weakens the filtering intensity through directional consistency constraints, thus blocking cross-level noise propagation during reconstruction. This scheme reduces noise transmissibility while maintaining the detail integrity of each scale level.

[0022] 4. This invention enhances residual details and dynamically reconstructs the image. In the improved nonlocal means algorithm, it rapidly extracts residual details through block similarity weights and uses a sharpening factor to achieve adaptive edge enhancement, improving image edge contrast without ringing effects. During the reconstruction stage, a forgetting factor is introduced to dynamically adjust hierarchical weights, combined with high-level edge feature inverse injection technology. =0.2), upsample the enhanced edge information extracted from the high-level pyramid and backfeed it to the low level to compensate for the loss of detail. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method of the present invention.

[0024] Figure 2 This is a flowchart of the dynamic reconstruction process of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0026] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various elements, but unless otherwise stated, these elements are not limited by these terms. These terms are used only to distinguish one element from another.

[0027] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Example 1 like Figure 1 and Figure 2 As shown, this invention provides a multi-scale hierarchical collaborative optimization method for infrared image non-uniformity correction, which includes the following steps: Step 1: Construct a multi-scale pyramid; Generate a two-dimensional Gaussian kernel ,in The standard deviation smoothing parameter is adaptively adjusted through local information entropy. This represents the position coordinates of a pixel in the image. The smoothed image is sampled at intervals to generate a multi-resolution hierarchy {G0, G1, ..., G...}. n}, where G0 is the original image, and the resolution of each layer is half that of the previous layer; starting from the second-to-last layer of the Gaussian pyramid, it is upsampled to the current layer size through bilinear interpolation, and the high-frequency residual layer {H1, H2, ..., H} is obtained by calculating the Gaussian smoothed residual. n}, used for multi-scale analysis of fringe non-uniformity; Step 2: Joint frequency-spatial domain stripe suppression; Frequency domain Fast Fourier Transform (FFT) analysis was performed on the Laplace residual layer to detect the spectral energy integrals in 36 directions, determine the fringe direction, apply directional median filtering to the horizontal fringes, and apply a directional Gabor filter to suppress the fringes in the frequency domain. The formula is as follows: in, The center frequency of the fringe is The direction of the stripes, For the input frequency and direction, and These are the standard deviations of frequency and direction, respectively; Step 3: Multi-level collaborative optimization; The level with the highest noise intensity is selected as the main level, and a 1.3 times enhanced filter is applied to it; the secondary levels are weakly filtered in the main direction, with a filter intensity attenuation coefficient of 0.7; by passing through the main direction, it is ensured that the filtering direction of different levels is consistent, and residual noise caused by directional differences is avoided. Step 4: Residual detail enhancement and dynamic reconstruction; The process extracts and enhances details using a denoising process. First, slight denoising is performed. Then, the difference between the original image and the denoised image is calculated to create a detail image. This detail image is then weighted and added back to the original image to achieve a sharpening effect. The complete workflow formula is: Where ℎ represents the filter strength. As the sharpening factor, For the original input image, NLM is an improved nonlocal mean sharpening method. Based on median filtering residual estimation of noise standard deviation, the hierarchical fusion coefficients are adjusted by an adaptive forgetting factor, and high-level edge features are inversely injected. =0.2 weight) to optimize low-level detail recovery and output image.

[0029] The adaptive Gaussian kernel is generated in step 1 of this invention as follows: in That is the original standard deviation.

[0030] The frequency domain FFT analysis in step 3 of this invention includes: performing polar coordinate transformation on the FFT spectrum and calculating the energy integral of the annular region at 5° intervals; when the energy in three consecutive directions exceeds 2.5 times the mean, the middle direction is taken as the average. .

[0031] In step 3 of this invention, for horizontal stripes (close to 0° or 180°), 1 A median filter window of 7 is used. For stripes in the tilt direction, the image is rotated, and then a 1 is applied. The median filter window is 7.

[0032] The primary level selection criteria in step 4 of this invention are as follows: in, To analyze the maximum fringe intensity in the frequency domain, As a hierarchy, The main fringe directions are shown in the frequency domain analysis.

[0033] The fast nonlocal means denoising in step 5 of this invention is as follows: The weighting function simplifies to: in, The original input image, The image after denoising. This is a search window centered on pixel (x,y). For an image patch centered at pixel (x,y), Normalization factor This is the filter strength parameter.

[0034] The residual detail extraction in step 5 of this invention is as follows: in, This is a high-frequency detail map, containing noise and texture information.

[0035] The adaptive detail enhancement in step 5 of this invention is as follows: in, Sharpening intensity control factor.

[0036] In step 6 of this invention, the forgetting factor is adjusted during reconstruction according to the noise intensity attenuation fusion coefficient ( )for: in, Forgetting factor, This represents the hierarchical noise intensity.

[0037] In step 6 of this invention, the feature map of the high-level pyramid is injected into the low-level pyramid through upsampling: in, hierarchical The enhanced edge feature map has a fusion weight of . =0.2, edge(L i ) is the low-level feature map L i Edge features obtained by edge detection.

[0038] In step 6 of this invention, during the reconstruction process, the use of high-level edges to enhance low-level details is as follows: in, =0.1 to avoid over-sharpening. This is the reconstructed image.

[0039] Example 2 This invention uses images output by an infrared thermal imager with a resolution of 640*512 as the processing object. The images contain obvious horizontal and diagonal stripe non-uniform noise, and the target scene includes detailed features such as building edges and tree textures.

[0040] Multi-scale pyramid construction: Adaptive Gaussian kernel generation, with 7 7-window calculation of local information entropy, flat region (sky): smoothing parameter =1.06, suppressing noise while avoiding excessive blurring. Texture area (leaves): =1.24, preserving high-frequency details. Generate a 4-layer Gaussian pyramid (G0~G3) with a resolution of 640. 512, 320 256, 160 128, 80 64. Dimensionality reduction is achieved through interval sampling in each layer. For layer G3 (80... 64) Bilinear interpolation upsampling to 160 128, subtracting from the original G2 layer yields the H2 layer, and similarly, the H1 layer (320) is generated. 256), H0 layer (640) 512).

[0041] Frequency-spatial joint stripe suppression: First, perform FFT polar coordinate analysis on layer H1 (320). 256) After performing a Fast Fourier Transform and polarizing the spectrum, the ring energy integral in 36 directions (every 5° interval) was calculated. Three consecutive regions with energy exceeding 2.5 times the global mean were detected in the 45° direction (42.5°~52.5°), and were identified as the main noise direction. Then, directional filtering and spatial median filtering were performed, and the image was rotated 45° to make the stripes horizontal. 1 The median filtering window is 7, and finally, frequency domain Gabor filtering is performed.

[0042] Multi-level noise collaborative optimization: Principal level selection and enhancement: Calculate the maximum fringe intensity in the frequency domain for each level, select the principal level, and apply... =1.3x enhanced filtering (filter kernel size from 5) 5 expanded to 7 7). Secondary layer attenuation and direction constraints; the secondary layer adopts... =0.7 attenuation coefficient (filter core size remains 5) 5) and ensure that the filtering direction is consistent with the main level (45°±2°) through the direction transfer matrix to avoid residual noise.

[0043] Residual detail enhancement and dynamic reconstruction: Fast nonlocal mean sharpening for G0 layer, for G0 layer (640 512) An improved nonlocal mean algorithm is used, with a search window of 15. 15, block size 5 5. Filter parameter h=12, calculate residual detail plot. Adaptive fusion sharpening factor k=0.4, output image Dynamic hierarchical fusion and reverse injection are implemented. The noise standard deviation is estimated based on the median filter residual, and the hierarchical weights are dynamically adjusted according to the adaptive forgetting factor formula. Edge features E1 of layer H1 are extracted and upsampled to 640. After 512, it was injected into the H0 layer with a weight of γ=0.2.

[0044] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0045] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

[0046] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-scale hierarchical collaborative optimization method for infrared image non-uniformity correction, characterized in that: The method includes the following steps: Step 1: Construct a multi-scale pyramid; Generate a two-dimensional Gaussian kernel ,in The standard deviation smoothing parameter is adaptively adjusted through local information entropy. ) represents the position coordinates of a pixel in the image. The smoothed image is sampled at intervals to generate a multi-resolution layer {G0, G1, ..., G...}. n }, where G0 is the original image, and the resolution of each layer is half that of the previous layer; starting from the second-to-last layer of the Gaussian pyramid, it is upsampled to the current layer size through bilinear interpolation, and the high-frequency residual layer {H1,H2,...,H} is obtained by calculating the Gaussian smoothed residual. n }, used for multi-scale analysis of fringe non-uniformity; Step 2: Joint frequency-spatial domain stripe suppression; A Fast Fourier Transform (FFT) analysis was performed on the Laplace residual layer in the frequency domain to detect the spectral energy integrals in 36 directions, determine the fringe direction, apply directional median filtering to the horizontal fringes, and apply a directional Gabor filter to suppress the fringes in the frequency domain. The formula is as follows: in, The center frequency of the fringe is The direction of the stripes, For the input frequency and direction, and These are the standard deviations of frequency and direction, respectively; Step 3: Multi-level noise collaborative optimization; The level with the highest noise intensity is selected as the main level, and a 1.3 times enhanced filter is applied to it; the secondary levels are weakly filtered in the main direction, with a filter intensity attenuation coefficient of 0.7; by passing through the main direction, it is ensured that the filtering direction of different levels is consistent, and residual noise caused by directional differences is avoided. Step 4: Residual detail enhancement and dynamic reconstruction; The process of extracting and enhancing details involves several steps. First, slight denoising is performed. Then, the difference between the original image and the denoised image is calculated to create a detail image. This detail image is then weighted and added back to the original image to achieve a sharpening effect. The complete workflow formula is as follows: Where ℎ represents the filter strength. As the sharpening factor, For the original input image, NLM(*) is an improved nonlocal mean sharpening method; Based on median filtering residual estimation of noise standard deviation, the hierarchical fusion coefficients are adjusted by an adaptive forgetting factor, and then combined with high-level edge features for reverse injection. =0.2 weight optimization for low-level detail recovery, output image.

2. The infrared image non-uniformity correction method with multi-scale hierarchical collaborative optimization according to claim 1, characterized in that, The adaptive Gaussian kernel is generated in step 1 as follows: in , That is the original standard deviation.

3. The infrared image non-uniformity correction method with multi-scale hierarchical collaborative optimization according to claim 1, characterized in that, The frequency domain FFT analysis in step 2 includes: performing polar coordinate transformation on the FFT spectrum and calculating the energy integral of the annular region at 5° intervals; when the energy in three consecutive directions exceeds 2.5 times the mean, the middle direction is taken as the average. .

4. The infrared image non-uniformity correction method with multi-scale hierarchical collaborative optimization according to claim 1, characterized in that, In step 2, for horizontal stripes that are close to 0° or 180°, use 1 A median filter window of 7 is used. For stripes in the tilt direction, the image is rotated, and then a 1 is applied.

7. Median filter window.

5. The infrared image non-uniformity correction method with multi-scale hierarchical collaborative optimization according to claim 1, characterized in that, The primary hierarchy selection criteria in step 3 are as follows: in, To analyze the maximum fringe intensity in the frequency domain, As a hierarchy, The main fringe directions are shown in the frequency domain analysis.

6. The infrared image non-uniformity correction method with multi-scale hierarchical collaborative optimization according to claim 1, characterized in that, The fast nonlocal mean denoising in step 4 is as follows: The weighting function simplifies to: in, The original input image, The image after denoising. This is a search window centered on pixel (x,y). For an image patch centered at pixel (x,y), Normalization factor, This is the filter strength parameter.

7. The infrared image non-uniformity correction method with multi-scale hierarchical collaborative optimization according to claim 1, characterized in that, Residual detail extraction in step 4: in, This is a high-frequency detail map, containing noise and texture information.

8. The infrared image non-uniformity correction method with multi-scale hierarchical collaborative optimization according to claim 1, characterized in that, Adaptive detail enhancement in step 4: in, Sharpening intensity control factor.

9. The infrared image non-uniformity correction method with multi-scale hierarchical collaborative optimization according to claim 1, characterized in that, In step 4, the forgetting factor is adjusted during reconstruction based on the noise intensity attenuation fusion coefficient. : in, Forgetting factor, This represents the hierarchical noise intensity.

10. The infrared image non-uniformity correction method with multi-scale hierarchical collaborative optimization according to claim 1, characterized in that, In step 4, the feature map of the high-level pyramid is injected into the low-level pyramid through upsampling: in, hierarchical The enhanced edge feature map has a fusion weight of . =0.2, edge(L i ) is the low-level feature map L i Edge features obtained through edge detection; In step 4, during the reconstruction process, high-level edges are used to enhance low-level details: in, =0.1 to avoid over-sharpening. This is the reconstructed image.

Images