An infrared image denoising method based on a conditional diffusion model

Abstract

The application relates to the technical field of image information processing, and particularly discloses an infrared image denoising method based on a conditional diffusion model, which comprises the following steps: acquiring clean image samples and corresponding noise image samples of different wavelengths; connecting the clean image samples and the corresponding noise image samples in the channel dimension based on the conditional diffusion model, and gradually adding Gaussian noise to obtain noise images; extracting predicted noises of the noise images of different wavelengths according to an improved U-net network; the predicted noises correspond to the wavelengths; acquiring an image to be denoised, and gradually denoising the image to be denoised based on the conditional diffusion model and the predicted noises corresponding to the wavelengths of the image to be denoised to obtain a denoised image; and the application can effectively remove noises and recover and reconstruct target information in the image to the maximum extent.

Description

Technical Field

[0001] This invention relates to the field of image information processing technology, and specifically to an infrared image denoising method based on a conditional diffusion model. Background Technology

[0002] Infrared imaging systems capture infrared radiation emitted by objects to form images. They have the advantage of operating in complex environments such as darkness or fog, and are therefore widely used in many critical fields, including industry, medicine, and the military. However, due to the inherent electrical characteristics of infrared devices and external interference during the acquisition and processing, infrared images are susceptible to noise contamination. This contamination results in erroneous or redundant pixel values ​​in the image, making it visually blurry and obscuring the true information of the target, thus affecting the accuracy of subsequent target detection tasks. In the application of infrared images, denoising is usually the first and most crucial step.

[0003] Traditional infrared image denoising methods rely on the lack of correlation between real signal pixels and noise pixels in an image, employing filtering and domain transformation to remove noise. With decades of updates and advancements in digital signal processing technology, traditional infrared image denoising methods have matured and spawned numerous derivative algorithms. These can be categorized based on the processing domain: spatial domain-based, transform domain-based, and dual-domain combined denoising methods. Traditional methods, with their early development, provided a variety of general-purpose algorithms for infrared image denoising for a long time. Furthermore, various mature filter designs and time-frequency transformation methods provided excellent theoretical support, making them still practically valuable in applications with simple noise types, limited computational resources, and high real-time requirements. However, with the further development of infrared imaging technology, the information contained in infrared images has become increasingly rich, and the requirements for the quality of the denoised image have significantly increased. The limited denoising effect of traditional methods, as well as the resulting over-smoothing and detail blurring, have gradually rendered them unable to meet the needs of current infrared image denoising tasks.

[0004] In recent years, with the significant improvement in computing power and the development of deep learning in the field of image processing, many excellent deep learning methods have been applied to the task of denoising infrared images. Conditional diffusion models possess higher training stability and better convergence, thus providing a new approach for the development of infrared image denoising methods. Summary of the Invention

[0005] To address the aforementioned problems, the purpose of this invention is to provide an infrared image denoising method based on a conditional diffusion model, which can effectively remove noise and restore and reconstruct target information in the image to the greatest extent possible.

[0006] This invention provides an infrared image denoising method based on a conditional diffusion model, comprising: Obtain clean image samples and corresponding noisy image samples at different wavelengths; Based on the conditional diffusion model, the clean image sample and the corresponding noisy image sample are concatenated along the channel dimension, and Gaussian noise is gradually added to obtain the noisy image; Predicted noise for the noise image at different wavelengths is extracted using an improved U-net network; the predicted noise corresponds to the wavelength. The image to be denoised is acquired, and denoising is performed stepwise based on the conditional diffusion model and the predicted noise corresponding to the wavelength of the image to be denoised, to obtain the denoised image; The improved U-Net network includes a shallow feature extraction module and a deep feature extraction module. The shallow feature extraction module includes: a timestep feature TimestepEmbedding module and a first residual ResBlock module; The deep feature extraction module includes: a wavelet convolution MFEW module, a second residual ResBlock module, and a dot-product-free attention AFT module.

[0007] In one possible implementation, the conditional diffusion model-based concatenation of the clean image sample and the corresponding noisy image sample along the channel dimension, followed by the gradual addition of Gaussian noise to obtain the noisy image, includes: Forward diffusion is performed according to the following formula: In the formula, Indicates the forward diffusion process. express Data distribution at different times express Data distribution at different times Indicates from Spread to The Gaussian noise to be added This indicates the parameter that controls the amount of Gaussian noise added. Represents the identity matrix, Indicates the time.

[0008] In one possible implementation, the stepwise denoising based on the conditional diffusion model and the predicted noise corresponding to the wavelength of the image to be denoised to obtain the denoised image includes: The reverse generation is performed according to the following formula: In the formula, express Data distribution at different times express Data distribution at different times Represents the data distribution of the image to be denoised. Indicates the reverse generation process, The mean of the predicted noise is represented by... The variance of the prediction noise is represented by... Represents the identity matrix, Indicates time, This represents an identifier that indicates predicted noise.

[0009] In one possible implementation, the stepwise denoising based on the conditional diffusion model and the predicted noise corresponding to the wavelength of the image to be denoised to obtain the denoised image includes: The TimestepEmbedding module encodes discrete time points and converts them into continuous values ​​containing location information. The first residual ResBlock module is used to convolve the noisy image and the continuous value to obtain a primary feature map. The primary feature map is downsampled multiple times using the wavelet convolution MFEW module and the second residual ResBlock module to obtain the first feature. When the number of samplings reaches a preset number, the first feature is further extracted by the dot-product-free attention AFT module to obtain the second feature; The second feature is upsampled multiple times by the second residual ResBlock module and the dot-product-free attention AFT module to obtain the prediction noise.

[0010] In one possible implementation, the AFT module includes an attention mechanism and a feedforward network.

[0011] In one possible implementation, the feedforward network includes multiple cascaded convolutional layers for expanding and reducing the dimensionality of the input feature map.

[0012] In one possible implementation, the attention mechanism is a multi-head attention mechanism.

[0013] In one possible implementation, the MFEW module includes: a layer set, a Concat module, a WTConv module, and a ResNet Block module; The layer set includes: convolutional layers, normalized layers, and Leaky ReLU activation function.

[0014] In one possible implementation, multiple WTConv modules are arranged in a stepped hierarchy, with the output of each WTConv module being a feature image of the same size and depth as the original input.

[0015] In one possible implementation, the loss function of the conditional diffusion model as follows: In the formula, This represents the raw data input into the network after the noisy image samples have been sampled. This represents the raw data input into the network after clean image samples have been sampled. Represents real noise, To predict noise, express Data distribution at different times Indicates time, This represents the average of all batch data output values.

[0016] The infrared image denoising method based on the conditional diffusion model provided by this invention uses noisy images as the basic condition, adopts an improved U-Net network as the main architecture for model training, and realizes the conversion between noisy images and denoised images through forward diffusion and backward generation processes. It solves the problems of poor performance of current infrared image denoising methods and easy loss of image detail information, and provides a brand-new solution for noise removal in infrared images. Attached Figure Description

[0017] Figure 1 A schematic flowchart of an infrared image denoising method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall frame provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the TimestepEmbedding module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the ResBlock module provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of the improved U-Net network provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the wavelet convolution process provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of the MFEW module provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of the AFT module provided in an embodiment of the present invention; Figure 9 This is a schematic diagram illustrating the scene in the training dataset provided in an embodiment of the present invention; Figure 10The present invention provides an embodiment with a noise content of A schematic diagram of the first noise reduction effect on the test group; Figure 11 The present invention provides an embodiment with a noise content of A schematic diagram of the second noise reduction effect on the test group; Figure 12 The present invention provides an embodiment with a noise content of A schematic diagram of the third noise reduction effect on the test group; Figure 13 Provided for embodiments of the present invention A schematic diagram of the first denoising effect on a window scene in the dataset; Figure 14 Provided for embodiments of the present invention A schematic diagram of the second denoising effect on the vehicle scene in the dataset; Figure 15 Provided for embodiments of the present invention A schematic diagram of the third denoising effect on the house scene in the dataset. Detailed Implementation

[0018] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. The following detailed description of the embodiments and the accompanying drawings are used to illustrate the principles of the present invention by way of example, but should not be used to limit the scope of the present invention. That is, the present invention is not limited to the described preferred embodiments, and the scope of the present invention is defined by the claims.

[0019] In the description of this invention, it should be noted that, unless otherwise stated, "a plurality of" means two or more; the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance; those skilled in the art can understand the specific meaning of the above terms in this invention as appropriate.

[0020] Figure 1 This is a schematic flowchart of an infrared image denoising method provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall frame provided in an embodiment of the present invention, as shown below. Figure 1 and Figure 2 As shown, this invention provides an infrared image denoising method based on a conditional diffusion model, comprising: Step S1: Obtain clean image samples and corresponding noisy image samples at different wavelengths; Step S2: Based on the conditional diffusion model, clean image samples and corresponding noisy image samples are concatenated in the channel dimension, and Gaussian noise is gradually added to obtain a noisy image. In one possible implementation, forward diffusion is performed according to the following formula: In the formula, Indicates the forward diffusion process. express Data distribution at different times express Data distribution at different times Indicates from Spread to The Gaussian noise to be added This indicates the parameter that controls the amount of Gaussian noise added. Represents the identity matrix, Indicates the time.

[0021] Step S3: Extract predicted noise from noise images of different wavelengths using the improved U-net network; The prediction noise corresponds to the wavelength. Since the feature representation of infrared images is relatively weak, this invention proposes an improved U-Net network, the specific structure of which is as follows: Figure 5 As shown, this enables it to be used to extract and learn relevant information from infrared images, thereby more effectively guiding the model's denoising process.

[0022] The improved U-Net network includes a shallow feature extraction module and a deep feature extraction module. The shallow feature extraction module includes a timestep embedding module and a first residual ResBlock module. The deep feature extraction module includes a wavelet convolution MFEW module, a second residual ResBlock module, and an attention-free Transformer (AFT) module.

[0023] The MFEW module comprises: a layer set, a Concat module, a WTConv module, and a ResNet Block module. The layer set includes convolutional layers, normalized layers, and the Leaky ReLU activation function. Multiple WTConv modules are arranged in a stepped hierarchy, with each WTConv module outputting a feature image of the same size and depth as the original input.

[0024] In one possible implementation, the discrete time points are encoded and converted into continuous values ​​containing location information by the TimestepEmbedding module; the noisy image and the continuous values ​​are convolved by the ResBlock module to obtain a primary feature map; the primary feature map is downsampled multiple times by the MFEW module and the ResBlock module to obtain a first feature; when the number of downsampling reaches a preset number, the first feature is further extracted by the AFT module to obtain a second feature; the second feature is upsampled multiple times by the ResBlock module and the AFT module to obtain the prediction noise.

[0025] In one example, the input to the U-Net network Compared with the time points processed by the TimestepEmbedding module Together, they are fed into the convolutional module for initial feature extraction. The specific structure of TimestepEmbedding is as follows: Figure 3 As shown, the TimestepEmbedding module will display discrete time points. After encoding, the data is converted into continuous values ​​containing positional information, and then input into a layer set consisting of linear layers and SiLU activation functions to transform it into a two-dimensional tensor. The primary feature map output by convolution is fed into a module set consisting of MFEW and ResBlock modules for deep feature extraction. The specific structure of ResBlock is as follows... Figure 4 As shown, in the two leftmost branches, and After undergoing linear transformation and convolution operations, the images are aligned and stacked along the channel dimension. The rightmost branch is left unprocessed to form a standard residual structure. The output of the ResBlock is directly fed into the downsampling module to extract higher-dimensional image information. To enable the model to learn data features at different scales, the proposed denoising network performs four downsampling operations. When downsampling reaches a certain depth, a dot-product attention Transformer is used instead of the ResBlock to further extract image features. The final downsampling output, after passing through a single ResBlock, undergoes four upsampling operations and corresponding functional modules to obtain the predicted noise. .

[0026] The following provides a detailed explanation of each module in the conditional diffusion model.

[0027] ResBlock is used to fuse time and location information with the input data.

[0028] MFEW is a novel feature extraction structure based on wavelet convolution (WTConv). Its specific structure is as follows: Figure 7 As shown.

[0029] Wavelet transform is a mathematical tool widely used in digital signal and image processing. It decomposes a given input into different frequency components using pre-selected wavelet basis functions, thereby aiding in the extraction and analysis of relevant image features. Wavelet convolution (WTConv) is a novel convolution module based on wavelet transform. Compared to traditional convolution operations, it can generate effective multi-frequency responses, thus improving the network's ability to extract different frequency features from images. Its specific convolution process is as follows: Figure 6 As shown.

[0030] For an input feature image, WTConv first uses wavelet transform to decompose it into a combination containing high-frequency and low-frequency components. For each frequency map in this combination, WTConv performs convolution calculations on it, and finally reconstructs the convolution result using inverse wavelet transform. This process can be represented as: in, This represents the original feature image as input. This indicates the number of channels in the convolution. To achieve convolution for each frequency component after the X-wavelet transform, The size should be Four times the number of input channels.

[0031] For WTConv containing multiple cascaded wavelet transforms, its convolution process is extended according to the following formula: in, Indicates the first step in the cascading process Level input, They represent The horizontal frequency components, vertical frequency components, and diagonal frequency components formed after wavelet transform. , , and These are the results obtained after convolving the above components.

[0032] Utilizing the linearity of the inverse wavelet transform, the summed output is formed by convolving and combining the various frequency components. It can be represented as: Given WTConv's excellent frequency division convolution capabilities, a novel Multilevel Feature Extraction Module Based on Wavelet Convolution (MFEW) is proposed to enhance the network's learning performance for different frequency features in infrared images.

[0033] For an input size of The feature map, after passing through a series of layers consisting of convolutional layers, normalization layers, and Leaky ReLU activation functions, is evenly divided along the image depth direction by MFEW. One portion of the evenly divided feature image is directly output to the Concat module for final concatenation, while the other portion is input to multiple branches containing WTConv for feature extraction. To increase the network's non-linear expressive power and create richer gradient flow information for downstream processing, multiple WTConvs are arranged in a stepped hierarchy within this branch, and the convolutional results of each layer are summarized to output a feature image of the same size and depth as the original input. Finally, the features from the two branches are concatenated along the channel dimension and passed to subsequent modules.

[0034] AFT is a novel variant of Transformer in computer vision, achieving efficient capture of global dependencies in images while maintaining low computational cost. The main structure of AFT is as follows: Figure 8 As shown, the AFT module includes an attention mechanism and a feedforward network; the attention mechanism is a multi-head attention (MHA) mechanism. The feedforward network consists of multiple cascaded convolutional layers used to expand and reduce the dimensionality of the input feature map. The Dot-Product-Free Transformer (AFT) is a novel Vit architecture that replaces the traditional dot product operation with element-wise combination, achieving a reduction in computational parameters while maintaining the Vit feature extraction effect.

[0035] For a dimension Input The multi-head attention mechanism first performs a linear transformation to generate... , and With three vectors, this process can be represented as: in, , and These are the weight matrices required for the corresponding linear transformation. Then, for each head... Perform the following dot product operation: in, It is a non-linear activation function. It is the vector dimension processed by each head. The number of heads is denoted by . MHA concatenates the results of each head along the channel dimension and then inputs them into the feedforward network. The feedforward network is used to expand and reduce the dimensionality of the input feature map, thereby increasing the network's non-linear fitting capability and ultimately producing an output with the same dimensionality as the input.

[0036] Vit achieves efficient capture of global dependency information in images through parallel computation via a multi-head attention mechanism, thereby enhancing the network's focus on important features and improving the model's expressive power. However, the matrix dot product operation included in MHA has a high number of parameters, so using the traditional Vit as the feature extraction module of an infrared image denoising network will result in higher computational costs and training time.

[0037] AFT first transforms the input feature map using the same linear transformation as traditional Vit. Transform into , and Then and It is combined with a set of learned positional biases. For each position... This process can be represented as: in, The positional bias is obtained through learning. Represents the length of the input sequence. This involves element-wise matrix multiplication. After the above processing, its output is again element-wise multiplied with Q to obtain the final output. This process can be specifically represented as: Because only one factor was considered during the weight calculation process. And w, which allows AFT to achieve [the following] without needing to compute and store expensive dot product matrices. and This enables global interaction between networks, thereby reducing the computational complexity of the feature extraction process while ensuring the effectiveness of network training.

[0038] Step S4: Obtain the image to be denoised, and perform stepwise denoising based on the conditional diffusion model and the predicted noise corresponding to the wavelength of the image to be denoised, to obtain the denoised image; In one possible implementation, reverse generation is performed according to the following formula: In the formula, express Data distribution at different times express Data distribution at time, Represents the data distribution of the image to be denoised. Indicates the reverse generation process, The mean of the predicted noise is represented by... The variance of the prediction noise is represented by... Represents the identity matrix, Indicates time, This represents an identifier that indicates predicted noise.

[0039] In one possible implementation, the ultimate goal of training the network proposed in this invention is to obtain the prediction noise in the reverse generation process. Furthermore, from noisy images The generated samples are gradually recovered. This objective is achieved by minimizing the true noise. and prediction noise The loss function of the conditional diffusion model is achieved by the mean squared error between the two sides. as follows: In the formula, This represents the raw data input into the network after the noisy image samples have been sampled. This represents the raw data input into the network after clean image samples have been sampled. Represents real noise, To predict noise, express Data distribution at different times Indicates time, This represents the average of all batch data output values.

[0040] The following description, in conjunction with specific examples, provides further details.

[0041] The training data used in this invention mainly consists of two parts: 1400 high-resolution grayscale images obtained after processing the DIV2K dataset, and infrared images captured by a long-wave infrared prototype independently developed by a certain project. The hardware specifications of this prototype lens (imaging wavelength range of 8-12 micrometers, imaging resolution of no less than 1280×1020) have reached world-class levels, possessing high ground resolution and low infrared noise equivalent temperature difference. This invention extracts and crops frames from multiple infrared images recorded using this prototype, and uniformly performs bad pixel removal, defective pixel correction, and motion blur repair on all the obtained raw image data to obtain near-noise-free ground truth samples while preserving the imaging characteristics of the infrared system. Finally, after cropping and filtering, a total of 1400 high-quality infrared images with a pixel size of 512×512 are obtained, serving as a source of clean samples for network training. These images contain various typical ground targets with different infrared radiation characteristics, such as houses, vehicles, lakes, vegetation, and building complexes, and also include various imaging scenes such as daytime, nighttime, and dusk. Some of these images are shown below. Figure 9 As shown, Figure 9 a, b, c, d, e, f, g, and h correspond to different scenarios such as houses, high-rise buildings, parks, building complexes, factories, forests, roads and vehicles, and schools, respectively.

[0042] In addition, 50 infrared images of the same size were reserved as benchmark data for testing, to evaluate the denoising performance of the trained model.

[0043] All experiments related to this invention are implemented using the PyTorch deep learning framework. The specific experimental environment configuration is shown in Table 1. Table 1 In the preprocessing stage, a mixture of Poisson noise and Gaussian noise is added to all clean image samples to form simulated noise data for model training. The standard deviation of the Gaussian noise is specified in the original text. The value range is [0, 50]. For all benchmark images in the test set, mixed noise with noise levels of NOL, NOM, and NOH is added to verify the denoising effect of the model later. Before training, the network's diffusion step T is set to 2000, the batch size is set to 4, the total number of training iterations is set to 30000, the Haar wavelet transform is selected as the wavelet basis function of WTConv in the MFEW module, and the number of cascades is set to two. The Sigmoid function is selected as the nonlinear activation function in AFT. During training, the network randomly prunes all input data to 256×256 four times to increase the number of samples available for learning. At the same time, the AdamW optimizer is used to optimize the overall training process. In the first 1000 iterations, the learning rate is linearly increased from 0 to 1e-4, and then cosine decays to 0 in the remaining iterations.

[0044] To evaluate the denoising performance of this invention under the current state of denoising methods, five classic or advanced denoising algorithms in the field of image processing—WNNM, DnCNN, RIDNet, ADNet, and RDDCNN—were selected for comparative experiments. Among them, WNNM is a traditional image denoising algorithm based on dual-domain combination; DnCNN is an image denoising algorithm based on deep convolutional neural network architecture, which mainly uses residual and batch normalization operations to learn the difference between noisy and clean images, and performs well in scenes with unknown noise distribution; RIDNet is an image denoising network built with dense residual modules, which can remove noise that satisfies Gaussian and Poisson distributions at the same time when the intensity is unknown through multi-scale feature extraction and information fusion; ADNet is a deep learning algorithm with blind image denoising capability, which can accurately reconstruct damaged pixel information from noise through the synergistic effect of sparse blocks, feature enhancement blocks, attention blocks and reconstruction blocks; RDDCNN is a robust deformable image denoising network proposed in 2023, which extracts noise features through deformable learning blocks, thereby helping to build and restore clear images, and performs well in scenes with noise of different intensities.

[0045] All comparison algorithms were derived from the relevant code provided in the original authors' papers. WNNM was tested for denoising in a MATLAB environment, while the other methods were trained using the same training set and equivalent experimental parameter settings. After training, PSNR, SSIM, and LPIPS were used as evaluation metrics to assess the denoising performance of each model on a test set containing mixed noise of three different intensities: NOL, NOM, and NOH. The experimental comparison results are shown in Table 2. The optimal values ​​in each group are indicated by bold, and the second-best values ​​are indicated by underlined values. Table 2 The data in the table above shows that the proposed network achieved the best overall denoising performance on the test set with a noise level of NOL, with all three evaluation metrics exceeding those of other comparative methods. This demonstrates that the method of this invention can effectively preserve the detailed information in the image while better removing low-intensity mixed noise. In the test set with a noise level of NOM, the proposed network achieved the best SSIM and LPIPS metrics and the second-best PSNR metric, proving that for moderate-intensity mixed noise, the denoising performance of the method of this invention reaches the state-of-the-art level in the current field of image denoising, while the perceptual quality and overall structure of the denoised image are better improved. For the test set with a mixed noise level of NOH, the proposed network achieved the best LPIPS result, and also achieved the second-best performance in both PSNR and SSIM metrics, proving that when the image is severely damaged by noise, compared with most current algorithms, the method of this invention can effectively remove noise and recover and reconstruct the target information in the image to the greatest extent.

[0046] Figures 10 to 12 The denoising effects of different methods on test groups containing mixed noise of three different intensities: NOL, NOM, and NOH are shown. Figure 10 The noise reduction effects of each method in the construction site scenario are compared. Figure 10 In the image, I, II, III, IV, V, VI, VII, and VIII represent the noisy image, the baseline image, and the denoised image obtained using WNNM, DnCNN, RIDNet, ADNet, RDDCNN, and the method of this invention, respectively. For easier comparison, specific regions in the images are magnified. At this point, the Poisson characteristic of the mixed noise in the image is obvious, and visually, the noise distribution in the bright and dark areas is not uniform. However, because the overall noise intensity is low, all algorithms effectively remove the noise from the image. Figure 10 As can be seen from the magnified image of the crane structure, the steel frame structure of the crane becomes blurred after denoising by the WNNM algorithm, resulting in the loss of a lot of image details. DnCNN and ADNet preserve the non-noise features of the image to some extent, but the background behind the crane becomes significantly rough after denoising. The target features in the image are relatively clear after processing by RIDNet and RDDCNN, but the effect of preserving the details of the truss part under the crane is not good. In contrast, the method of this invention achieves better restoration results in both the overall structure of the image and the texture features of the target, and is superior to other methods in terms of intuitive visual perception.

[0047] Figure 11 The noise reduction effects of various comparison methods on a house scene are demonstrated. Figure 11In the image, I, II, III, IV, V, VI, VII, and VIII represent the noisy image, the baseline image, and the denoised image obtained using WNNM, DnCNN, RIDNet, ADNet, RDDCNN, and the method of this invention, respectively. Comparison of the different sub-images reveals that WNNM denoising results in blurred boundaries between the roof edge and the background; while DnCNN recovers the relevant texture structure of the roof from the noise, the shape of the railing under the eaves is noticeably distorted; RIDNet performs poorly in reconstructing certain areas of the image, leading to fragmented eaves edges after denoising; ADNet and RDDCNN do not fully recover relevant features of the roof, and the denoised image still shows a significant difference in appearance compared to the baseline image; in summary, the method of this invention effectively removes mixed noise from the image while preserving the detailed features of the left-side pillars and railings of the house, and the denoised image shows the highest similarity to the baseline image in terms of overall texture and brightness changes.

[0048] Figure 12 The noise reduction effects of various comparison methods on the attic scene are demonstrated. Figure 12 In the image, I, II, III, IV, V, VI, VII, and VIII represent the noisy image, the baseline image, and the denoised image obtained using WNNM, DnCNN, RIDNet, ADNet, RDDCNN, and the method of this invention, respectively. By comparing magnified images of the window frame structure, it can be seen that WNNM's denoising process is significantly overdone, causing severe blurring in texture and lighting. While DnCNN, ADNet, and RDDCNN effectively preserve most of the texture information after denoising, a small amount of noise remains on the window outline and glass in their respective images. RIDNet effectively recovers the attic structure covered by noise, but there is obvious damage to the central eaves of the denoised image. In contrast, the window frame structure in the denoised image obtained by the method of this invention has sharper lines, a more complete overall attic structure, and is visually clearer.

[0049] To test the generalization ability of the proposed method on real infrared images, 15 images from the TNO dataset were selected for denoising performance testing. TNO is a multispectral image dataset released by the National Institute of Optics, Canada, which contains infrared images with significant noise characteristics. Furthermore, to further demonstrate the effectiveness and advancement of the proposed method, the five methods mentioned above were again selected as comparison algorithms in this section. The relevant results are shown in Figures 13 to 15. Figure 13 In the diagram, I, II, III, IV, V, VI, and VII represent the original image and the denoised image obtained using WNNM, DnCNN, RIDNet, ADNet, RDDCNN, and the method of this invention, respectively. Figure 13As can be seen, in scenarios with relatively simple target features, except for a small amount of residual noise in the image processed by DnCNN, the other methods can effectively remove noise and improve image clarity. This demonstrates that the feature preservation effect of the method of this invention in simple scenarios is comparable to that of most current methods. Figure 14 In the diagram, I, II, III, IV, V, VI, and VII represent the original image and the denoised image obtained using WNNM, DnCNN, RIDNet, ADNet, RDDCNN, and the method of this invention, respectively. Figure 14 A comparison of different images reveals that WNNM and DnCNN have poor denoising performance, leaving noticeable noise residue on the car and background after processing. RIDNet can remove most of the noise in the image, but it causes deformation at the wheel structure. ADNet and RDDCNN effectively denoise while preserving the relevant texture features of the vehicle, but their denoising is slightly overdone, causing the black brake pads of the vehicle to become blurred. Figure 15 In the diagram, I, II, III, IV, V, VI, and VII represent the original image and the denoised image obtained using WNNM, DnCNN, RIDNet, ADNet, RDDCNN, and the method of this invention, respectively. Figure 15 Comparisons between different images reveal that the method of this invention outperforms other algorithms in both denoising effectiveness and detail preservation, resulting in a more visually pleasing denoised image. In scenes with low brightness and high structural complexity, DnCNN denoising produces a coarser background texture and is insufficient for the white railings. WNNM and RIDNet denoising blurs the boundary between the railing edges and the background. ADNet denoising results in the loss of detail in the background. RDDCNN processing still leaves noise residue on the black pillars in the image. In contrast, the method of this invention not only fully restores the relevant information obscured by noise but also further improves the perceptual quality of the denoised image.

[0050] The infrared image denoising method based on the conditional diffusion model provided by this invention, compared with traditional methods, can more effectively extract data features and noise patterns contained in infrared images, thereby achieving better denoising performance and generalization effect. Furthermore, the training process of the conditional diffusion model does not depend on labeled sample data, thus enabling the learning of noise distribution in infrared images in a semi-supervised or unsupervised manner, thereby establishing an effective mapping between noisy images and denoised images.

[0051] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An infrared image denoising method based on a conditional diffusion model, characterized in that, include: Obtain clean image samples and corresponding noisy image samples at different wavelengths; Based on the conditional diffusion model, the clean image sample and the corresponding noisy image sample are concatenated along the channel dimension, and Gaussian noise is gradually added to obtain the noisy image; Predicted noise for the noise image at different wavelengths is extracted using an improved U-net network; the predicted noise corresponds to the wavelength. The image to be denoised is acquired, and denoising is performed stepwise based on the conditional diffusion model and the predicted noise corresponding to the wavelength of the image to be denoised, to obtain the denoised image; The improved U-Net network includes a shallow feature extraction module and a deep feature extraction module. The shallow feature extraction module includes: a timestep feature TimestepEmbedding module and a first residual ResBlock module; The deep feature extraction module includes: a wavelet convolution MFEW module, a second residual ResBlock module, and a dot-product-free attention AFT module.

2. The infrared image denoising method according to claim 1, characterized in that, The conditional diffusion model concatenates the clean image sample and the corresponding noisy image sample along the channel dimension, and gradually adds Gaussian noise to obtain the noisy image, including: Forward diffusion is performed according to the following formula: In the formula, Indicates the forward diffusion process. express Data distribution at different times express Data distribution at different times Indicates from Spread to The Gaussian noise to be added This indicates the parameter that controls the amount of Gaussian noise added. Represents the identity matrix, Indicates the time.

3. The infrared image denoising method according to claim 1, characterized in that, The stepwise denoising based on the conditional diffusion model and the predicted noise corresponding to the wavelength of the image to be denoised, to obtain the denoised image, includes: The reverse generation is performed according to the following formula: In the formula, express Data distribution at different times express Data distribution at different times Represents the data distribution of the image to be denoised. Indicates the reverse generation process, The mean of the predicted noise is represented by... The variance of the prediction noise is represented by... Represents the identity matrix, Indicates time, This represents an identifier that indicates predicted noise.

4. The infrared image denoising method according to claim 1, characterized in that, The stepwise denoising based on the conditional diffusion model and the predicted noise corresponding to the wavelength of the image to be denoised, to obtain the denoised image, includes: The TimestepEmbedding module encodes discrete time points and converts them into continuous values ​​containing location information. The first residual ResBlock module is used to convolve the noisy image and the continuous value to obtain a primary feature map. The primary feature map is downsampled multiple times using the wavelet convolution MFEW module and the second residual ResBlock module to obtain the first feature. When the number of samplings reaches a preset number, the first feature is further extracted by the dot-product-free attention AFT module to obtain the second feature; The second feature is upsampled multiple times by the second residual ResBlock module and the dot-product-free attention AFT module to obtain the prediction noise.

5. The infrared image denoising method according to claim 1, characterized in that, The dot-product-free attention AFT module includes an attention mechanism and a feedforward network.

6. The infrared image denoising method according to claim 5, characterized in that, The feedforward network includes multiple cascaded convolutional layers used to expand and reduce the dimensionality of the input feature map.

7. The infrared image denoising method according to claim 5, characterized in that, The attention mechanism described is a multi-head attention mechanism.

8. The infrared image denoising method according to claim 1, characterized in that, The MFEW module includes: a layer set, a Concat module, a WTConv module, and a ResNet Block module; The layer set includes: convolutional layers, normalized layers, and Leaky ReLU activation function.

9. The infrared image denoising method according to claim 8, characterized in that, The multiple WTConv modules are arranged in a stepped hierarchy, and the output of each WTConv module is a feature image of the same size and depth as the original input.

10. The infrared image denoising method according to claim 1, characterized in that, The loss function of the conditional diffusion model as follows: In the formula, This represents the raw data input into the network after the noisy image samples have been sampled. This represents the raw data input into the network after clean image samples have been sampled. Represents real noise, To predict noise, express Data distribution at different times Indicates time, This represents the average of all batch data output values.

Images