A method for predicting and correcting non-uniformity of infrared image based on wavelet transform

Abstract

The application discloses an infrared image non-uniformity prediction and correction method based on wavelet transform, which comprises the following steps: adopting double-density dual-tree complex wavelet transform to perform multi-scale decomposition on an input infrared image; in a high-frequency subband, detecting a blind element position based on a local variance statistical method; in a low-frequency subband, constructing an autoregressive model, predicting non-uniformity of a background region, and performing accurate processing on a prediction result; combining the prediction results of the high-frequency subband and the low-frequency subband to generate a pre-correction coefficient; and adopting the pre-correction coefficient to perform correction processing on an original infrared image to obtain a corrected image. Through the combination of an adaptive fusion strategy, space-time consistency constraint and dynamic gain and bias updating technology, the problems of non-uniformity noise, balance between details and background, device drift compensation and consistency in a dynamic scene in infrared image processing are solved, and the accuracy and stability of infrared image processing are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of infrared image processing technology, and in particular to a method for predicting and correcting non-uniformity in infrared images based on wavelet transform. Background Technology

[0002] With the widespread application of infrared imaging technology, especially in fields such as security, surveillance, remote sensing, and medicine, the quality and accuracy of infrared images are crucial for subsequent image analysis and applications. However, existing infrared images often suffer from non-uniform noise and brightness drift during acquisition due to factors such as equipment performance, temperature variations, and environmental interference. Particularly during long-term operation, the gain and bias of the infrared sensor can change, leading to uneven brightness, insufficient contrast, and loss of detail in the image, thus affecting the effectiveness of subsequent processing.

[0003] Most existing image correction methods are based on traditional image processing techniques, such as mean filtering and histogram equalization. However, these methods often fail to effectively handle subtle changes in dynamic scenes and exhibit poor adaptability to variations across different devices and environments. In particular, traditional methods struggle to achieve long-term stable and consistent correction for non-uniformity caused by device drift and temperature changes. Therefore, maintaining image quality and stability in various dynamic environments has become a significant challenge in current image processing technology. Summary of the Invention

[0004] Therefore, the embodiments of the present invention provide an infrared image non-uniformity prediction and correction method based on wavelet transform, which combines adaptive fusion strategy, spatiotemporal consistency constraint and dynamic gain and bias update technology to solve the problems of non-uniform noise, detail and background balance, device drift compensation and consistency in dynamic scenes in infrared image processing, and significantly improves the accuracy and stability of infrared image processing.

[0005] The technical solution of this invention is as follows:

[0006] This invention provides a method for predicting and correcting non-uniformity in infrared images based on wavelet transform, characterized by comprising the following steps:

[0007] Infrared image decomposition: The input infrared image is decomposed into high-frequency sub-band and low-frequency sub-band using dual-density dual-tree complex wavelet transform.

[0008] High-frequency domain blind cell prediction: In the high-frequency subband, the blind cell position is detected based on the local variance statistics method, and the direction-sensitive interpolation method is used for prediction and correction to further improve the correction accuracy.

[0009] Low-frequency domain non-uniformity prediction: In the low-frequency sub-band, an autoregressive model is constructed to predict the non-uniformity of the background region, and the prediction results are refined.

[0010] Threshold adaptive fusion strategy: Combine the prediction results of high-frequency subband and low-frequency subband to generate pre-correction coefficients;

[0011] Infrared image correction processing: The original infrared image is corrected using the pre-correction coefficients to obtain the corrected image.

[0012] In some implementations, the dual-density dual-tree complex wavelet transform has translation invariance and multi-directional selectivity, with 2-4 decomposition layers, each layer generating 16 directional subbands, and the number of subband directions in each layer can be adaptively adjusted according to the characteristics of the input image to further optimize the image processing effect.

[0013] In some implementations, the high-frequency domain blind cell prediction specifically includes:

[0014] In each high-frequency sub-band, a local window centered on the target pixel is selected, and the pixel variance within the window is calculated to obtain the local variance distribution;

[0015] Based on the statistical characteristics of local variance, a threshold is adaptively set in combination with the mean of global variance to distinguish abnormal pixels from normal pixels.

[0016] Pixels with local variance exceeding an adaptive threshold are marked as candidate blind pixels, and temporal consistency is judged for candidate pixels in multiple consecutive frames of infrared images. Pixels that are continuously marked are confirmed as real blind pixels.

[0017] For the confirmed blind pixels, based on the directional response characteristics of the sub-band in which they are located, neighboring pixels with the same direction are selected for directional-sensitive interpolation prediction, and interpolation weights are calculated to complete the blind pixel replacement.

[0018] During the interpolation process, high-frequency noise is smoothed and suppressed to maintain the continuity of edge structures and improve local prediction accuracy.

[0019] In some implementations, the direction-sensitive interpolation prediction is combined with temporal consistency constraints for blind pixel repair optimization, specifically including:

[0020] After completing the spatial interpolation based on the directional response characteristics, time series analysis is performed on the interpolation results at the same pixel position in consecutive frames to establish a temporal smoothing model for the inter-frame prediction values.

[0021] The temporal smoothing model uses a weighted average method to fuse the interpolated prediction values ​​of adjacent frames, wherein the time weight is adaptively adjusted according to the inter-frame brightness change rate to weaken the impact of short-term abnormal response.

[0022] When the inter-frame brightness change rate is detected to exceed the set threshold, the temporal consistency constraint correction mechanism is triggered to perform temporal smoothing re-estimation of abrupt pixel points in order to suppress prediction instability caused by dynamic scenes or instantaneous noise.

[0023] By introducing this spatiotemporal joint optimization process, the continuity of blind pixel restoration results in the temporal dimension and the preservation of spatial structure are achieved, thereby improving the overall prediction stability and visual consistency.

[0024] In some implementations, the low-frequency domain non-uniformity prediction is modeled using an autoregressive model, the coefficients of which are learned from the training dataset using a least squares estimation method.

[0025] The training dataset consists of multiple infrared images acquired under uniform radiation conditions, and each image is preprocessed to remove random noise and dead pixels.

[0026] During the model building process, independent autoregressive models were established for low-frequency sub-bands at different scales to fit the spatial smooth variation characteristics of background non-uniformity, thereby obtaining the non-uniformity prediction results in the low-frequency domain.

[0027] In some implementations, in the threshold adaptive fusion strategy, the fusion weight factor of the high-frequency prediction result and the low-frequency prediction result is adaptively determined based on the local contrast.

[0028] Local contrast is obtained by calculating the ratio of the standard deviation to the mean of brightness in the neighborhood of a pixel, and is used to characterize the texture intensity of the region.

[0029] The weighting factor changes with local contrast, and a smooth mapping is achieved through the sigmoid function to avoid fusion discontinuities caused by abrupt changes. Its expression form is as follows:

[0030]

[0031] in, As a weighting factor, For local contrast, The contrast threshold. The adjustment coefficient is used to control the steepness of the Sigmoid curve;

[0032] When the local contrast is high, the weights are biased towards high-frequency prediction results to enhance details; when the local contrast is low, the weights are biased towards low-frequency prediction results to enhance background smoothness.

[0033] In some implementations, the threshold adaptive fusion strategy further includes a fusion optimization mechanism with spatial consistency constraints, specifically including:

[0034] During the fusion process, a constraint term based on the spatial correlation of adjacent pixel regions is introduced. By minimizing the deviation between the fusion result and the local neighborhood mean, a smooth spatial distribution of the fusion weights is achieved.

[0035] The spatial correlation constraint term adopts the Laplacian regularization form to smooth the spatial gradient of the fusion weights, so as to prevent fusion discontinuities or pseudo-oscillations in image texture or edge regions.

[0036] By introducing this constraint mechanism, the fusion result achieves a dynamic balance between detail preservation and background smoothing, thereby improving the stability of the fusion prediction and the overall visual consistency.

[0037] In some implementations, the fusion weights are determined by minimizing an energy function that includes a data fidelity term and a spatial smoothness term, expressed as:

[0038]

[0039] Where E is the total energy function. For pixels The corresponding fusion weights, Based on local contrast The Sigmoid mapping function, Represents pixels The set of neighboring pixels, where λ is the spatial smoothness trade-off coefficient;

[0040] By iteratively solving the energy function and updating the fusion weight distribution until the energy function converges, a fusion weight distribution that balances data fidelity and spatial consistency is obtained.

[0041] In some embodiments, the infrared image correction process includes the following steps:

[0042] For each pixel in the original infrared image, gain compensation and bias correction are performed based on the pre-correction coefficients at its corresponding position. The correction model is expressed as follows:

[0043]

[0044] in, These are the pixel values ​​of the original infrared image. For the corrected pixel values, This is the gain coefficient. These are the bias coefficients, both of which are determined by the pre-correction coefficients;

[0045] The corrected image is subjected to global brightness normalization to eliminate brightness drift between different frames and maintain brightness consistency between frames.

[0046] Based on the statistical characteristics of the corrected image, the pre-correction coefficients are iteratively updated. By minimizing the global variance or local non-uniformity index of the image, the residual non-uniformity is further reduced and the overall uniformity is improved.

[0047] In some implementations, the gain coefficient With bias coefficient It can be dynamically updated through time series modeling;

[0048] In a multi-frame infrared video sequence, a time series model is constructed for each pixel location. The time-weighted mean and variance are calculated using a sliding window to achieve adaptive compensation for device drift or temperature drift, thereby maintaining the stability of infrared image brightness and contrast during long-term operation.

[0049] Technical effects of the present invention:

[0050] 1. Improve image uniformity: By decomposing the image into high-frequency and low-frequency sub-bands through multi-scale wavelet transform, and combining high-frequency blind pixel prediction with low-frequency background non-uniformity prediction, the uniformity of infrared images is effectively improved, and fixed pattern noise and random non-uniformity are removed.

[0051] 2. Adaptive adjustment of fusion weights: The fusion weights of high-frequency and low-frequency prediction results are adaptively adjusted using local contrast to ensure a balance between detail enhancement and background smoothing, thereby improving the visual quality and stability of the corrected image.

[0052] 3. Spatiotemporal consistency optimization: By combining temporal consistency constraints, the image prediction results in the time series are smoothed to reduce image instability caused by dynamic scenes or instantaneous noise, and to ensure the consistency and stability of images during long-term operation.

[0053] 4. Dynamic gain and bias correction: By dynamically updating the gain and bias coefficients through time series modeling, the non-uniformity caused by equipment drift or temperature changes can be effectively compensated, maintaining the stability and consistency of infrared images during long-term operation.

[0054] The above-mentioned technical improvements enable the present invention to provide high-quality image correction results in a variety of complex environments, thereby improving the accuracy and stability of infrared image processing. Attached Figure Description

[0055] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0056] Figure 1 This is a flowchart illustrating the method for predicting and correcting non-uniformity of infrared images based on wavelet transform, provided in certain embodiments of the present invention. Detailed Implementation

[0057] References to embodiments herein mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0058] This invention provides a method for predicting and correcting non-uniformity in infrared images based on wavelet transform.

[0059] See Figure 1 The wavelet transform-based method for predicting and correcting non-uniformity in infrared images provided by embodiments of the present invention includes:

[0060] Step 01, Infrared Image Decomposition: The input infrared image is decomposed into high-frequency subbands and low-frequency subbands using dual-density dual-tree complex wavelet transform.

[0061] Step 02, High-frequency domain blind cell prediction: In the high-frequency subband, the blind cell position is detected based on the local variance statistics method, and the direction-sensitive interpolation method is used for prediction and correction to further improve the correction accuracy;

[0062] Step 03, Low-frequency domain non-uniformity prediction: In the low-frequency sub-band, an autoregressive model is constructed to predict the non-uniformity of the background region, and the prediction results are refined.

[0063] Step 04, Threshold Adaptive Fusion Strategy: Combine the prediction results of high-frequency subband and low-frequency subband to generate pre-correction coefficients;

[0064] Step 05, Infrared image correction processing: The original infrared image is corrected using the pre-correction coefficients to obtain the corrected image.

[0065] The wavelet transform-based method for predicting and correcting non-uniformity in infrared images provided in this invention includes step 01, which involves processing the input raw infrared image. Multi-scale decomposition was performed using the Dual-Density Dual-Tree Complex Wavelet Transform (DD-DTCWT). DD-DTCWT constructs complex wavelet pairs through two sets of parallel filter banks, exhibiting approximate translation invariance and multi-directional selectivity. The number of decomposition layers was set to 2–4. Each layer generates 16 directional subbands, which can represent image features at different scales and directions. The high-frequency subbands obtained from the decomposition... Includes detailed information such as texture, edges, and noise; low-frequency subband. This primarily reflects the background and overall brightness distribution. Through this decomposition, the structural features of the infrared image are effectively separated in a multi-scale space, providing a good feature foundation for subsequent blind pixel detection and non-uniformity modeling.

[0066] Step 02, in each high-frequency sub-band In the process, an N×N local window is selected centered on each pixel, for example (N=5), and the pixel variance within the window is calculated:

[0067]

[0068] in, This represents the average value of the pixels within the window. The number of pixels within the window. Indicates the first Pixels in each high-frequency subband coefficient, Local window.

[0069] In a multi-frame infrared sequence, if the same pixel is marked as an anomaly in M ​​consecutive frames (e.g., M≥3), it is confirmed as a true blind pixel. For the confirmed blind pixel, based on the directional response characteristics of its sub-band, neighboring pixels with the same direction are selected for direction-sensitive interpolation prediction:

[0070]

[0071] Among them, weight Determined based on the directional gradient similarity and spatial distance between pixels:

[0072]

[0073] in, For spatial scale parameters, This is the directional gradient smoothing coefficient. Indicates along direction The gradient operator.

[0074] Step 03: The low-frequency domain non-uniformity prediction is modeled using an auto-regressive (AR) model to characterize the slowly changing non-uniform distribution features in the background region of the infrared image.

[0075] First, the low-frequency subband obtained by dual-density dual-tree complex wavelet transform decomposition... Modeling is performed. For each scale level... Establish an independent autoregressive model, with the following expression:

[0076]

[0077] in, To model neighborhood windows (e.g., 3) 3 or 5 (5 areas) For the first The autoregressive coefficients of the low-frequency subband of the layer. This represents the model residual term. The model captures the spatially smooth variations of the non-uniform background in low-frequency components through the linear relationship between a pixel and its neighboring pixels.

[0078] Step 04, Local Contrast It is obtained by calculating the ratio of the standard deviation of brightness in the neighborhood of a pixel to the mean. This ratio can represent the texture intensity of the region and is used to measure the level of detail in the local structure of the image.

[0079] First, calculate the target pixel. Local mean at and standard deviation And calculate the local contrast according to the following formula. :

[0080]

[0081] Then, local contrast is used to perform a smooth mapping using the Sigmoid function to obtain an adaptive fusion weight factor. Its expression is as follows:

[0082]

[0083] Where exp is an exponential function with base e (approximately 2.71828). For local contrast, The contrast threshold. This is an adjustment coefficient used to control the steepness of the Sigmoid curve. The Sigmoid function adjusts the weighting factor when the local contrast is high. The system tends to favor high-frequency predictions, thereby enhancing image details; however, when local contrast is low, the weighting factor... It then favors low-frequency prediction results to enhance the smoothness of the image background.

[0084] Step 05, process each pixel of the original image. Gain and bias corrections are performed based on their corresponding pre-correction coefficients, and the gain coefficients are... With bias coefficient The calculation is performed based on the corresponding correction model, which is expressed by the following formula:

[0085]

[0086] in, These are the pixel values ​​of the original infrared image. The corrected image pixel values, This is the gain coefficient. This is the bias coefficient.

[0087] The corrected image undergoes global brightness normalization to eliminate inter-frame brightness drift. After global brightness normalization, the gain and bias coefficients are updated using a prediction model based on the image's statistical characteristics. To further optimize image uniformity, a local non-uniformity index is calculated and optimized for the corrected image. The local non-uniformity index can be measured by calculating the degree of pixel value variation (e.g., standard deviation) within a local region of the image. During optimization, the local non-uniformity index is used as a constraint to adjust the pre-correction coefficients, thereby further improving the uniformity of the image within local regions.

[0088] In some implementations, the dual-density dual-tree complex wavelet transform has translation invariance and multi-directional selectivity, with 2-4 decomposition layers, each layer generating 16 directional subbands, and the number of subband directions in each layer can be adaptively adjusted according to the characteristics of the input image to further optimize the image processing effect.

[0089] Specifically, the dual-density dual-tree complex wavelet transform is used to perform multi-scale, multi-directional decomposition on the input infrared image to extract information of different frequencies and directions. This transform has good translation invariance and multi-directional selectivity, and can suppress artifacts and ringing effects while preserving structural edge information.

[0090] The dual-density dual-tree complex wavelet transform achieves the decomposition of the real and imaginary parts by constructing two independent filter banks, thereby obtaining an approximately continuous complex wavelet coefficient representation. In this embodiment, the number of decomposition layers of the transform is set to 2 to 4, with different layers corresponding to different spatial scales. The first layer mainly captures high-frequency detail information, such as edge and texture features; the second to fourth layers progressively extract low-frequency and structural information, realizing multi-scale fusion of global and local features.

[0091] In each decomposition layer, the dual-density dual-tree complex wavelet transform generates sub-band coefficients in 16 directions. Each sub-band corresponds to a response component in a different direction (e.g., ±15°, ±45°, ±75°, etc.) to achieve a fine characterization of the image's directional features. To further enhance the adaptive capability of image processing, the number of directional sub-bands in each layer in this embodiment can be adaptively adjusted according to the characteristics of the input image: when the texture or edge direction in the image is complex, the system automatically increases the number of directional sub-bands to enhance directional resolution; when the image structure is relatively simple, the number of directions is appropriately reduced to decrease computational complexity.

[0092] The above adaptive multi-scale decomposition can effectively suppress non-uniform noise while preserving the detailed information of infrared images, providing an accurate multi-scale feature basis for subsequent local variance calculation, fusion weight optimization and non-uniformity correction.

[0093] Building upon dual-density dual-tree complex wavelet transform, this implementation further proposes a directional subband adaptive decomposition mechanism to enhance the adaptability and directional resolution of infrared image decomposition. This mechanism automatically adjusts the number of wavelet directional subbands in each layer based on the local structural complexity of the input infrared image: when the image contains rich textures or complex edge structures, the system increases the number of directional subbands to enhance directional resolution; when the image region is relatively smooth or has a simple structure, the system appropriately reduces the number of directional subbands to reduce computational burden.

[0094] Specifically, the directional complexity of an image is evaluated by calculating the local gradient magnitude variance or directional entropy index. When it exceeds a set threshold, the number of wavelet directions at that scale is dynamically expanded; when it is below the threshold, the number of directions is maintained or reduced.

[0095] This adaptive decomposition mechanism can effectively suppress redundant directional components while preserving key structural information, thereby achieving a dynamic balance between decomposition accuracy and computational efficiency in different scenarios, and providing more targeted feature support for subsequent high-frequency blind element prediction and low-frequency non-uniformity modeling.

[0096] In some implementations, the high-frequency domain blind cell prediction specifically includes:

[0097] In each high-frequency sub-band, a local window centered on the target pixel is selected, and the pixel variance within the window is calculated to obtain the local variance distribution;

[0098] Based on the statistical characteristics of local variance, a threshold is adaptively set in combination with the mean of global variance to distinguish abnormal pixels from normal pixels.

[0099] Pixels with local variance exceeding an adaptive threshold are marked as candidate blind pixels, and temporal consistency is judged for candidate pixels in multiple consecutive frames of infrared images. Pixels that are continuously marked are confirmed as real blind pixels.

[0100] For the confirmed blind pixels, based on the directional response characteristics of the sub-band in which they are located, neighboring pixels with the same direction are selected for directional-sensitive interpolation prediction, and interpolation weights are calculated to complete the blind pixel replacement.

[0101] During the interpolation process, high-frequency noise is smoothed and suppressed to maintain the continuity of edge structures and improve local prediction accuracy.

[0102] Specifically, in this embodiment, firstly, local statistical analysis is performed on the coefficients of each high-frequency sub-band after decomposition by dual-density dual-tree complex wavelet transform. A system is then constructed centered on the target pixel. For a local window, calculate the pixel variance within the window:

[0103]

[0104] in, This represents the average value of the pixels within the window. The number of pixels within the window. Indicates the first Pixels in each high-frequency subband The coefficient.

[0105] Based on the statistical results, through local variance To calculate the threshold :

[0106]

[0107] in, It is an empirical coefficient (usually taken as 1.2 to 1.5) used to balance noise and structural response.

[0108] If the local variance satisfies > If the pixel is marked as a candidate for blind pixel, then the temporal consistency of these candidate pixels is determined in multiple consecutive frames of infrared images: if the pixel is marked consecutively in N frames (e.g., N≥3), it is confirmed as a true blind pixel.

[0109] For a given blind pixel, a directional window is constructed by selecting neighboring pixels that are consistent with that direction, based on the directional response characteristics of its sub-band. And perform direction-sensitive interpolation prediction:

[0110]

[0111] Among them, weight Determined based on the directional gradient similarity and spatial distance between pixels:

[0112]

[0113] in, For spatial scale parameters, This is the directional gradient smoothing coefficient. Indicates along direction The gradient operator.

[0114] The above-mentioned direction-sensitive interpolation prediction can effectively suppress the interference of high-frequency noise on the restoration results and maintain the continuity of edge structure and texture consistency.

[0115] In some implementations, the direction-sensitive interpolation prediction is combined with temporal consistency constraints for blind pixel repair optimization, specifically including:

[0116] After completing the spatial interpolation based on the directional response characteristics, time series analysis is performed on the interpolation results at the same pixel position in consecutive frames to establish a temporal smoothing model for the inter-frame prediction values.

[0117] The temporal smoothing model uses a weighted average method to fuse the interpolated prediction values ​​of adjacent frames, wherein the time weight is adaptively adjusted according to the inter-frame brightness change rate to weaken the impact of short-term abnormal response.

[0118] When the inter-frame brightness change rate is detected to exceed the set threshold, the temporal consistency constraint correction mechanism is triggered to perform temporal smoothing re-estimation of abrupt pixel points in order to suppress prediction instability caused by dynamic scenes or instantaneous noise.

[0119] By introducing this spatiotemporal joint optimization process, the continuity of blind pixel restoration results in the temporal dimension and the preservation of spatial structure are achieved, thereby improving the overall prediction stability and visual consistency.

[0120] Specifically, in this embodiment, to further improve the temporal stability of direction-sensitive interpolation prediction, a temporal consistency constraint is introduced for blind element repair optimization.

[0121] After completing spatial domain interpolation, the interpolation results at the same pixel position in consecutive frames are... Perform time series analysis and establish a temporal smoothing model for inter-frame predicted values:

[0122]

[0123] Among them, time weight Based on inter-frame brightness change rate Adaptive adjustment:

[0124]

[0125] in, The rate of change of brightness. This is the time smoothing coefficient.

[0126] If the detected inter-frame brightness change rate exceeds the threshold This triggers the temporal consistency constraint correction mechanism, performing a temporal smoothing re-estimation of abruptly changed pixels:

[0127]

[0128] in, This is a time constraint factor used to control the trade-off between the current frame prediction and the consistency of historical frames.

[0129] Through this spatiotemporal joint optimization process, the blind element repair results exhibit continuity and robustness in the time dimension, effectively suppressing brightness flicker and pseudo-response caused by dynamic targets or instantaneous noise, thereby achieving a prediction effect that balances spatial structure preservation and temporal smoothness.

[0130] In some implementations, the low-frequency domain non-uniformity prediction is modeled using an autoregressive model, the coefficients of which are learned from the training dataset using a least squares estimation method.

[0131] The training dataset consists of multiple infrared images acquired under uniform radiation conditions, and each image is preprocessed to remove random noise and dead pixels.

[0132] During the model building process, independent autoregressive models were established for low-frequency sub-bands at different scales to fit the spatial smooth variation characteristics of background non-uniformity, thereby obtaining the non-uniformity prediction results in the low-frequency domain.

[0133] Specifically, in this embodiment, the low-frequency domain non-uniformity prediction is modeled using an autoregressive model to characterize the slowly changing non-uniform distribution features in the background region of the infrared image.

[0134] First, the low-frequency subband obtained by dual-density dual-tree complex wavelet transform decomposition... Modeling is performed. For each scale level... Establish an independent autoregressive model, with the following expression:

[0135]

[0136] in, To model neighborhood windows (e.g., 3) 3 or 5 (5 areas) For the first The autoregressive coefficients of the low-frequency subband of the layer. This represents the model residual term. The model captures the spatially smooth variations of the non-uniform background in low-frequency components through the linear relationship between a pixel and its neighboring pixels.

[0137] To obtain the optimal autoregressive coefficient vector The parameters are solved using the least squares estimation method. Each image in the training dataset is preprocessed to obtain a sample set. Then the coefficient estimation formula is:

[0138]

[0139] Therefore, the analytical solution can be obtained:

[0140]

[0141] Where X is a sample matrix composed of neighboring pixels, and Y is the center pixel value vector.

[0142] The training dataset consists of multiple infrared images acquired under uniform radiation conditions to ensure that the model learning process is unaffected by scene targets. Each training image undergoes noise filtering and bad pixel repair preprocessing before being used for modeling to remove random noise and abnormal pixel interference.

[0143] After the model training is completed, for the infrared image to be corrected, its low-frequency subband Prediction is made using a pre-trained autoregressive model:

[0144]

[0145] The predicted results This represents the estimation result of the low-frequency domain non-uniformity component. By differencing it with the original low-frequency component, a distribution feature map of the background non-uniformity can be obtained, which can be used for subsequent threshold fusion and full-image correction processing.

[0146] This autoregressive model method can effectively fit the smooth variation trend of low-frequency background in infrared images and model and predict slow-varying non-uniformity, thereby improving the accuracy and stability of overall non-uniformity correction.

[0147] In some implementations, in the threshold adaptive fusion strategy, the fusion weight factor of the high-frequency prediction result and the low-frequency prediction result is adaptively determined based on the local contrast.

[0148] Local contrast is obtained by calculating the ratio of the standard deviation to the mean of brightness in the neighborhood of a pixel, and is used to characterize the texture intensity of the region.

[0149] The weighting factor changes with local contrast, and a smooth mapping is achieved through the sigmoid function to avoid fusion discontinuities caused by abrupt changes. Its expression form is as follows:

[0150]

[0151] Where C represents local contrast. is the contrast threshold, and k is the adjustment coefficient that controls the steepness of the curve;

[0152] When the local contrast is high, the weights are biased towards high-frequency prediction results to enhance details; when the local contrast is low, the weights are biased towards low-frequency prediction results to enhance background smoothness.

[0153] Specifically, in this embodiment, local contrast It is obtained by calculating the ratio of the standard deviation of brightness in the neighborhood of a pixel to the mean. This ratio can represent the texture intensity of the region and is used to measure the level of detail in the local structure of the image.

[0154] First, calculate the target pixel. Local mean at and standard deviation And calculate the local contrast according to the following formula. :

[0155]

[0156] Then, local contrast is used to perform a smooth mapping using the Sigmoid function to obtain an adaptive fusion weight factor. Its expression is as follows:

[0157]

[0158] in, For local contrast, The contrast threshold. This is an adjustment coefficient used to control the steepness of the Sigmoid curve. The Sigmoid function adjusts the weighting factor when the local contrast is high. The system tends to favor high-frequency predictions, thereby enhancing image details; however, when local contrast is low, the weighting factor... It then favors low-frequency prediction results to enhance the smoothness of the image background.

[0159] In this way, the fusion strategy can adaptively adjust the weights according to the local characteristics of the image, thereby improving the overall visual effect of the fusion result.

[0160] In some implementations, the threshold adaptive fusion strategy further includes a fusion optimization mechanism with spatial consistency constraints, specifically including:

[0161] During the fusion process, a constraint term based on the spatial correlation of adjacent pixel regions is introduced. By minimizing the deviation between the fusion result and the local neighborhood mean, a smooth spatial distribution of the fusion weights is achieved.

[0162] The spatial correlation constraint term adopts the Laplacian regularization form to smooth the spatial gradient of the fusion weights, so as to prevent fusion discontinuities or pseudo-oscillations in image texture or edge regions.

[0163] By introducing this constraint mechanism, the fusion result achieves a dynamic balance between detail preservation and background smoothing, thereby improving the stability of the fusion prediction and the overall visual consistency.

[0164] Specifically, in this embodiment, by introducing a constraint term based on the spatial correlation of adjacent pixel regions during the fusion process, the fusion weights are smoothly distributed in space, thereby improving the stability and consistency of the fusion result and avoiding discontinuous or pseudo-oscillating phenomena in the fusion of image textures or edge regions.

[0165] First, spatial consistency is calculated using the correlation between local neighboring pixels. Each pixel... weight The fusion result is influenced by the weights of its neighboring pixels to achieve a smooth transition. This process minimizes the deviation between the fusion result and the local neighborhood mean by introducing a spatial constraint term, expressed as:

[0166]

[0167] in, Represents pixels The set of neighboring pixels, For pixels The corresponding fusion weights. By minimizing this spatial consistency constraint, the fusion result is ensured to have a smooth spatial distribution, thereby avoiding visual distortion caused by abrupt transitions.

[0168] Furthermore, the fusion process employs Laplacian regularization to smooth the spatial gradient of the fusion weights, further ensuring the continuity of the fusion result in image texture and edge regions. This constraint term takes the form:

[0169]

[0170] By introducing this Laplacian regularization term, the discontinuity of image details is further suppressed, ensuring a smoother and more natural visual effect.

[0171] Ultimately, by incorporating the spatial consistency constraints mentioned above, the generated fusion result maintains strong edge features in the detail regions while ensuring a smooth transition in the background regions. This mechanism effectively addresses the dynamic balance between image detail and background smoothness, improving the stability and visual consistency of the fusion effect.

[0172] In some implementations, the fusion weights are determined by minimizing an energy function that includes a data fidelity term and a spatial smoothness term, expressed as:

[0173]

[0174] Where E is the total energy function. For pixels The corresponding fusion weights, Based on local contrast The Sigmoid mapping function, Represents pixels The set of neighboring pixels, where λ is the spatial smoothness tradeoff coefficient. For pixels Pixel index within the neighborhood, For pixels The corresponding fusion weights.

[0175] By iteratively solving the energy function and updating the fusion weight distribution until the energy function converges, a fusion weight distribution that balances data fidelity and spatial consistency is obtained.

[0176] Specifically, in this embodiment, the fusion weights are determined by minimizing the energy function, achieving a balance between data fidelity and spatial smoothness. Specifically, the energy function consists of two main parts: a data fidelity term and a spatial smoothness term.

[0177] Step 041: Energy Function Construction

[0178] First, a total energy function φ is constructed to describe the quality of the fusion result. This energy function consists of two main parts:

[0179] Data fidelity: This measure measures the difference between the fusion result and the high-frequency prediction result, ensuring that the fused image retains the fidelity in details.

[0180] Spatial smoothing term: This term smooths the fusion result by calculating the difference in weights between adjacent pixels to avoid discontinuities or pseudo-oscillations.

[0181] The energy function takes the following form:

[0182]

[0183] in, For pixels The fusion weight, Based on local contrast The Sigmoid mapping function, Represents pixels λ is the set of neighboring pixels, where λ is the spatial smoothness trade-off coefficient.

[0184] Step 042: Minimize the energy function

[0185] During the fusion process, the weights are updated iteratively. To minimize the above energy function, the specific steps are as follows:

[0186] Step 0421, Data Fidelity Optimization: By minimizing the first term (data fidelity term), ensure that the fusion result retains as much high-frequency detail information as possible;

[0187] Step 0422, Spatial Smoothing Optimization: By minimizing the second term (spatial smoothing term), the fusion weight distribution of adjacent pixels is ensured to be smooth, avoiding abrupt changes in image texture or edge regions.

[0188] The optimization process eventually converges to the optimal distribution of the fusion weights, thereby achieving a balance between data fidelity and spatial consistency.

[0189] Step 043: Iterative Optimization Process

[0190] The fusion weights are updated by iteratively calculating the minimization of the energy function. and This process continues until the energy function converges. This process ensures that the fusion result achieves an optimal balance between detail and background, avoiding over-smoothing or loss of detail.

[0191] Ultimately, through this iterative optimization, the resulting fusion not only retains the detailed information of high-frequency predictions but also maintains a smooth background in low-frequency regions, thereby improving visual consistency and stability.

[0192] In some embodiments, the infrared image correction process includes the following steps:

[0193] For each pixel in the original infrared image, gain compensation and bias correction are performed based on the pre-correction coefficients at its corresponding position. The correction model is expressed as follows:

[0194]

[0195] in, These are the pixel values ​​of the original infrared image. For the corrected pixel values, This is the gain coefficient. These are the bias coefficients, both of which are determined by the pre-correction coefficients;

[0196] The corrected image is subjected to global brightness normalization to eliminate brightness drift between different frames and maintain brightness consistency between frames.

[0197] Based on the statistical characteristics of the corrected image, the pre-correction coefficients are iteratively updated. By minimizing the global variance or local non-uniformity index of the image, the residual non-uniformity is further reduced and the overall uniformity is improved.

[0198] Specifically, in this embodiment, the infrared image correction processing includes the following steps:

[0199] Step 051, Image Decomposition and Preprocessing:

[0200] First, regarding the input raw infrared image Each pixel undergoes correction processing. Gain compensation and bias correction are performed based on the pre-correction coefficients of its position. Gain coefficient With bias coefficient The calculation is performed based on the corresponding correction model, which is expressed by the following formula:

[0201]

[0202] in, These are the pixel values ​​of the original infrared image. The corrected image pixel values, This is the gain coefficient. This is the bias coefficient.

[0203] Step 052, Global Brightness Normalization Process:

[0204] Next, the image after gain and bias correction. Global brightness normalization is performed. The purpose of this process is to eliminate brightness drift that may occur between different frames, ensuring brightness consistency across a multi-frame video sequence. This is achieved by calculating the global average brightness of the image and normalizing each pixel to ensure a consistent brightness range across all images, thus resolving brightness drift issues caused by device and environmental factors.

[0205] Step 053, update the pre-correction coefficients based on image statistical characteristics:

[0206] After global brightness normalization, the gain and bias coefficients are updated using a prediction model based on the image's statistical characteristics. Specifically, the pre-correction coefficients are optimized by minimizing the image's global variance or local non-uniformity index. This process is performed using least squares or other optimization methods to further improve image uniformity, making the image's gain and bias coefficients more aligned with practical application requirements. Ultimately, through iterative updates, the overall uniformity of the image can be effectively improved.

[0207] Step 054, Optimization of local non-uniformity index:

[0208] To further optimize image uniformity, a local non-uniformity index is calculated and optimized for the corrected image. The local non-uniformity index can be measured by calculating the degree of pixel value variation (e.g., standard deviation) within a local region of the image. During optimization, the local non-uniformity index is used as a constraint to adjust the pre-correction coefficients, thereby further improving the uniformity of the image within local regions.

[0209] In this way, the final corrected image can not only effectively remove fixed pattern noise from the original image, but also eliminate non-uniformity caused by device drift and temperature changes, thereby improving the overall quality of the infrared image.

[0210] In some implementations, the gain coefficient With bias coefficient It can be dynamically updated through time series modeling;

[0211] In a multi-frame infrared video sequence, a time series model is constructed for each pixel location. The time-weighted mean and variance are calculated using a sliding window to achieve adaptive compensation for device drift or temperature drift, thereby maintaining the stability of infrared image brightness and contrast during long-term operation.

[0212] Specifically, in this embodiment, the gain coefficient With bias coefficient The image non-uniformity can be dynamically updated through time-series modeling to adapt to device drift or temperature variations. Specifically, the dynamic updates of the gain and bias coefficients are based on modeling and calculation of multi-frame infrared video sequences.

[0213] Step 0511, Time Series Modeling and Dynamic Update: In a multi-frame infrared video sequence, for each pixel... Based on its changes over time, the gain coefficient is modeled using time series analysis. With bias coefficient Update accordingly. Specifically, assume the gain coefficient for each pixel. With bias coefficient As the values ​​change over time, we use a sliding window approach to calculate the average and variance over time, thereby smoothing the gain and bias coefficients.

[0214] Step 0512, Sliding window calculation: Using the sliding window calculation method, for each frame of the image... ,in Representing a time frame, the differences between this frame and the preceding and following frames are calculated. Within each frame, the gain and bias coefficients are updated using a weighted average method and local contrast calculation. The specific steps are as follows:

[0215] 1. Gain Coefficient Update: Calculate the gain difference between the current frame and the previous and next few frames, and update the gain coefficient based on the difference value. :

[0216]

[0217] Where N is the size of the sliding window (for example, a value of 3 or 5). It is the first Gain coefficients in a frame image.

[0218] 2. Bias coefficient update: Using a similar weighted averaging method, update the bias coefficient for each frame. Perform a smooth update:

[0219]

[0220] Where N is the size of the sliding window (for example, a value of 3 or 5). It is the first Gain coefficients in a frame image.

[0221] Through this process, the gain and bias coefficients transition smoothly over time, thus adapting to the effects of device drift or temperature changes on the image.

[0222] Step 0513, Device Drift and Temperature Variation Compensation: This method can compensate for non-uniformity caused by device drift or temperature variations in real time by smoothly updating the gain and bias coefficients. For example, when the device gain changes significantly, the update of the gain coefficient reflects this change, ensuring that the image maintains stable brightness and contrast over long periods of operation. By using time-series modeling, it can dynamically track device changes and make timely adjustments to maintain the stability of the infrared image.

[0223] Step 0514, Maintaining Image Stability and Consistency: Through the above dynamic update process, not only can adaptive compensation be achieved during image correction, but the overall stability and consistency of the image can also be maintained. This process can effectively avoid image quality degradation caused by device changes and ensure that the gain and bias coefficients of each frame remain consistent during long-term operation.

[0224] The present invention has been described above with reference to the accompanying drawings. Obviously, the implementation of the present invention is not limited to the above-described manner. Any improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other situations without modification, are all within the protection scope of the present invention.

Claims

1. A method for predicting and correcting non-uniformity in infrared images based on wavelet transform, characterized in that, Includes the following steps: Infrared image decomposition: The input infrared image is decomposed into high-frequency sub-band and low-frequency sub-band using dual-density dual-tree complex wavelet transform. High-frequency domain blind cell prediction: In the high-frequency subband, the blind cell position is detected based on the local variance statistics method, and the direction-sensitive interpolation method is used for prediction and correction to further improve the correction accuracy. Low-frequency domain non-uniformity prediction: In the low-frequency sub-band, an autoregressive model is constructed to predict the non-uniformity of the background region, and the prediction results are refined. Threshold adaptive fusion strategy: Combine the prediction results of high-frequency subband and low-frequency subband to generate pre-correction coefficients; Infrared image correction processing: The original infrared image is corrected using the pre-correction coefficients to obtain the corrected image.

2. The method according to claim 1, characterized in that, The dual-density dual-tree complex wavelet transform has translation invariance and multi-directional selectivity. It has 2-4 decomposition layers, each layer generates 16 directional sub-bands, and the number of sub-band directions in each layer can be adaptively adjusted according to the features of the input image to further optimize the image processing effect.

3. The method according to claim 1, characterized in that, The high-frequency domain blind cell prediction specifically includes: In each high-frequency sub-band, a local window centered on the target pixel is selected, and the pixel variance within the window is calculated to obtain the local variance distribution; Based on the statistical characteristics of local variance, a threshold is adaptively set in combination with the mean of global variance to distinguish abnormal pixels from normal pixels. Pixels with local variance exceeding an adaptive threshold are marked as candidate blind pixels, and temporal consistency is judged for candidate pixels in multiple consecutive frames of infrared images. Pixels that are continuously marked are confirmed as real blind pixels. For the confirmed blind pixels, based on the directional response characteristics of the sub-band in which they are located, neighboring pixels with the same direction are selected for directional-sensitive interpolation prediction, and interpolation weights are calculated to complete the blind pixel replacement. During the interpolation process, high-frequency noise is smoothed and suppressed to maintain the continuity of edge structures and improve local prediction accuracy.

4. The method according to claim 3, characterized in that, The direction-sensitive interpolation prediction combined with temporal consistency constraints for blind pixel repair optimization specifically includes: After completing the spatial interpolation based on the directional response characteristics, time series analysis is performed on the interpolation results at the same pixel position in consecutive frames to establish a temporal smoothing model for the inter-frame prediction values. The temporal smoothing model uses a weighted average method to fuse the interpolated prediction values ​​of adjacent frames, wherein the time weight is adaptively adjusted according to the inter-frame brightness change rate to weaken the impact of short-term abnormal response. When the inter-frame brightness change rate is detected to exceed the set threshold, the temporal consistency constraint correction mechanism is triggered to perform temporal smoothing re-estimation of abrupt pixel points in order to suppress prediction instability caused by dynamic scenes or instantaneous noise. By introducing this spatiotemporal joint optimization process, the continuity of blind pixel restoration results in the temporal dimension and the preservation of spatial structure are achieved, thereby improving the overall prediction stability and visual consistency.

5. The method according to claim 1, characterized in that, The low-frequency domain non-uniformity prediction is modeled using an autoregressive model, and the coefficients of the autoregressive model are learned from the training dataset using the least squares estimation method. The training dataset consists of multiple infrared images acquired under uniform radiation conditions, and each image is preprocessed to remove random noise and dead pixels. During the model building process, independent autoregressive models were established for low-frequency sub-bands at different scales to fit the spatial smooth variation characteristics of background non-uniformity, thereby obtaining the non-uniformity prediction results in the low-frequency domain.

6. The method according to claim 1, characterized in that, In the threshold adaptive fusion strategy, the fusion weight factor between high-frequency prediction results and low-frequency prediction results is adaptively determined based on local contrast. Local contrast is obtained by calculating the ratio of the standard deviation to the mean of brightness in the neighborhood of a pixel, and is used to characterize the texture intensity of the region. The weighting factor changes with local contrast, and a smooth mapping is achieved through the sigmoid function to avoid fusion discontinuities caused by abrupt changes. Its expression form is as follows: ； in, As a weighting factor, For local contrast, The contrast threshold. The adjustment coefficient is used to control the steepness of the Sigmoid curve; When the local contrast is high, the weights are biased towards high-frequency prediction results to enhance details; when the local contrast is low, the weights are biased towards low-frequency prediction results to enhance background smoothness.

7. The method according to claim 6, characterized in that, The threshold adaptive fusion strategy further includes a fusion optimization mechanism with spatial consistency constraints, which specifically includes: During the fusion process, a constraint term based on the spatial correlation of adjacent pixel regions is introduced. By minimizing the deviation between the fusion result and the local neighborhood mean, a smooth spatial distribution of the fusion weights is achieved. The spatial correlation constraint term adopts the Laplacian regularization form to smooth the spatial gradient of the fusion weights, so as to prevent fusion discontinuities or pseudo-oscillations in image texture or edge regions. By introducing this constraint mechanism, the fusion result achieves a dynamic balance between detail preservation and background smoothing, thereby improving the stability of the fusion prediction and the overall visual consistency.

8. The method according to claim 7, characterized in that, The fusion weights are determined by minimizing an energy function that includes a data fidelity term and a spatial smoothness term. This energy function is expressed as: ； Where E is the total energy function. For pixels The corresponding fusion weights, Based on local contrast The Sigmoid mapping function, Represents pixels The set of neighboring pixels, where λ is the spatial smoothness trade-off coefficient; By iteratively solving the energy function and updating the fusion weight distribution until the energy function converges, a fusion weight distribution that balances data fidelity and spatial consistency is obtained.

9. The method according to claim 1, characterized in that, The infrared image correction process includes the following steps: For each pixel in the original infrared image, gain compensation and bias correction are performed based on the pre-correction coefficients at its corresponding position. The correction model is expressed as follows: ； in, These are the pixel values ​​of the original infrared image. For the corrected pixel values, This is the gain coefficient. These are the bias coefficients, both of which are determined by the pre-correction coefficients; The corrected image is subjected to global brightness normalization to eliminate brightness drift between different frames and maintain brightness consistency between frames. Based on the statistical characteristics of the corrected image, the pre-correction coefficients are iteratively updated. By minimizing the global variance or local non-uniformity index of the image, the residual non-uniformity is further reduced and the overall uniformity is improved.

10. The method according to claim 9, characterized in that, The gain coefficient With bias coefficient It can be dynamically updated through time series modeling; In a multi-frame infrared video sequence, a time series model is constructed for each pixel location. The time-weighted mean and variance are calculated using a sliding window to achieve adaptive compensation for device drift or temperature drift, thereby maintaining the stability of infrared image brightness and contrast during long-term operation.

Images