Diffusion model image denoising method based on non-subsampled shear wave transformation

Abstract

The invention discloses a diffusion model image denoising method based on non-subsampled shear wave transformation, and relates to the technical field of image denoising. The invention aims to solve the problems of low de-noising efficiency and image distortion after de-noising in the existing image de-noising method. The method comprises the following steps: performing non-subsampled shearlet transform on a noise image to obtain a high-frequency sub-band feature and a low-frequency sub-band feature, and combining the high-frequency sub-band feature and the low-frequency sub-band feature to obtain a fusion feature; performing iterative noise addition on the fusion features to obtain fusion features after each time of iterative noise addition; generating a de-noised feature sample according to the fusion feature after iterative noise addition, obtaining an image similarity classification value by using the de-noised feature sample, and obtaining a de-noised fusion feature based on the image similarity classification value; and performing NSST inverse transformation on the denoised fusion features to obtain a denoised image. The method is used for image denoising processing.

Description

Technical Field

[0001] This invention relates to the field of image denoising technology, and in particular to an image denoising method based on a diffusion model using non-subsampled shear wave transform. Background Technology

[0002] In computer image processing tasks, noise can interfere with the extraction of image features and reduce the performance of computer vision tasks. Denoising methods can effectively improve image quality, thereby improving the accuracy and robustness of image processing. Therefore, research on image denoising is a key focus in this field.

[0003] Existing image denoising methods can be categorized into two types: traditional denoising methods and deep learning-based denoising methods. Traditional image denoising methods focus on denoising through sparsity, low rank, self-similarity, and waveform transformation, but they are ineffective at denoising complex and nonlinear noise. With the rise of technologies such as deep learning, researchers have proposed deep learning-based image denoising methods. These methods train neural networks to learn features from noisy images and generate clear images, thereby improving image quality. However, this denoising method requires multiple iterations to achieve optimal results, taking several seconds to denoise a single image in real-world scenarios, resulting in low efficiency. Furthermore, this method uses mean squared error or other pixel-level loss functions in loss calculations, failing to effectively account for subtle differences in image structure and perceptual quality. This can lead to artifacts or inconsistencies with the real image during denoising, resulting in image distortion. Subsequently, researchers used image generation-based denoising methods, which typically rely on generative adversarial networks (GANs). However, GAN-based methods often introduce artificial artifacts and alter image details, ultimately leading to severe distortion of the denoised image. Summary of the Invention

[0004] This invention addresses the problems of low denoising efficiency and image distortion after denoising in existing image denoising methods by proposing a diffusion model image denoising method based on non-subsampled shear wave transform.

[0005] Image denoising methods based on diffusion models using non-subsampled shear wave transform include:

[0006] Obtain the noisy image to be denoised, input the noisy image to be denoised into the trained denoising model, and obtain the denoised noisy image;

[0007] The denoising model includes: an NSST module, a diffusion model module, a discriminator module, and an NSST inverse transform module;

[0008] The NSST module is used for processing noisy images. Non-subsampled shear wave transform is performed to obtain high-frequency and low-frequency subband features, and the high-frequency and low-frequency subband features are combined to obtain fused features. ;

[0009] in, It is the preset number of sub-bands;

[0010] The diffusion model module includes: an iterative noise addition unit and a generator unit;

[0011] The iterative noise-adding unit is used to process the fused features. Each iteration adds noise, obtaining the fused features after each iteration of noise addition. Meanwhile, the iteratively noise-added fused features are sent to the generator unit;

[0012] The generator unit is used to determine, according to Generate denoised feature samples and the denoised feature samples , and Send to the discriminator module;

[0013] The discriminator module is used to determine the feature samples. , and Obtain image similarity classification values, and then obtain the denoised fusion features based on these classification values. ;

[0014] The NSST inverse transform module is used to process the denoised fusion features. Perform inverse NSST transform to obtain the denoised image. .

[0015] Furthermore, the trained denoising model is obtained by: forming a training set with the noisy image and the denoised image, training the denoising model using the training set, and obtaining the trained denoising model.

[0016] Furthermore, the NSST module is used for processing noisy images. The high-frequency and low-frequency subband features are obtained by performing a non-subsampled shear wave transform, specifically as follows:

[0017]

[0018]

[0019]

[0020] in, It is the first high-frequency sub-band feature. It is the first High-frequency sub-band characteristics, It is a low-frequency sub-band characteristic. It is the first One directional filter bank, It is the first directional filter bank. , It is a high-pass filter. , , , , It is a low-pass filter. It is a continuous shear wave transform function. This indicates the composition of operators. , It's a label. It's convolution.

[0021] Furthermore, the conditional distribution for each iteration in the iterative noise-adding unit is as follows:

[0022]

[0023] in, It is the first Fusion features after secondary noise addition The conditional probability, , It is variance scheduling. It is an identity matrix.

[0024] Furthermore, the generator unit includes: an initialization subunit and a denoising subunit;

[0025] The initialization subunit is used to initialize the iteration label. ;

[0026] The noise reduction subunit is used to... Generate corresponding denoised feature samples and the denoised feature samples , and Send to the discriminator module;

[0027] according to Generate corresponding denoised feature samples Specifically:

[0028]

[0029] in, It is the first Fusion features after adding noise and These are the mean and variance of the conditional distribution. , It is standard Gaussian noise. It is a generator unit.

[0030] Furthermore, the discriminator module is used to determine the features based on the sample. , and Obtain image similarity classification values, and then obtain the denoised fusion features based on these classification values. Specifically:

[0031] First, obtain and similarity probability , and similarity probability ;

[0032] Then, using and similarity probability and and similarity probability The output image similarity classification values ​​are as follows:

[0033] like and If the difference is within a preset range, the output image similarity classification value is 1; otherwise, the output image similarity classification value is 0.

[0034] Finally, if the image similarity classification value is 0, the generator unit returns to its initialization subunit to regenerate. If the image similarity classification value is 1, then determine... Is it 0? If Then the output is the denoised fused feature. Otherwise, let And return the denoising subunit in the generator unit.

[0035] Furthermore, the aforementioned and similarity probability , and similarity probability Specifically:

[0036]

[0037]

[0038] in, It is a multilayer perceptron.

[0039] Furthermore, the NSST inverse transform module is used to process the denoised fused features. Perform inverse NSST transform to obtain the denoised image. Specifically:

[0040]

[0041]

[0042] in, It is the inverse NSST transform function. , … These are preset weighting coefficients. , yes The total number of sub-band features in the middle yes Sub-band feature labels in yes The first in Individual band characteristics, yes The results of the inverse NSST transform.

[0043] Furthermore, the loss function used for training the generator unit in the diffusion model module is as follows:

[0044]

[0045]

[0046]

[0047] in, It is the probability distribution of the denoised feature samples generated by the generator. It is the KL divergence. It is the output of the diffusion model. These are preset weighted hyperparameters. It is the loss function of the generator unit.

[0048] Furthermore, the discriminator module is trained using the following loss function:

[0049]

[0050] in, It is the loss function of the discriminator module. Therefore Logarithmic operations with base 0.

[0051] The beneficial effects of this invention are as follows:

[0052] This invention constructs a collaborative "decomposition-generation-discrimination" closed-loop denoising model. It employs Non-Subsampled Shear Wave Transform (NSST) as a feature scheduler, efficiently decoupling the input image into sub-bands of different scales and directions. This achieves precise separation and purification of noise and essential image features (such as edges and textures), avoiding the iterative burden of the model and improving the speed and efficiency of image denoising. Furthermore, by introducing a discriminator, this invention forces the generator unit's output to approximate the denoised image as closely as possible to the real, clean image in each denoising iteration, preventing artifacts and distortions caused by accumulated errors in the denoised image. Attached Figure Description

[0053] Figure 1 This is a diagram of the architecture of the present invention;

[0054] Figure 2 This is a schematic diagram of a non-subsampled shear wave transform method;

[0055] Figure 3 NSST flowchart;

[0056] Figure 4 This is a flowchart of the present invention;

[0057] Figure 5 The results of the PSNR parameter analysis are shown in the figure.

[0058] Figure 6 The results of the LPIPS parameter analysis are shown in the figure.

[0059] Figure 7 The result of FID parameter analysis is shown in the figure;

[0060] Figure 8 The graph shows the generation loss function curves of DDPM and diffusion-based image generation methods.

[0061] Figure 9 The results of the PSNR parameter analysis;

[0062] Figure 10 The results of LPIPS parameter analysis;

[0063] Figure 11 The result of FID parameter analysis is shown in the figure;

[0064] Figure 12 Loss function curves are generated for DDPM and image denoising methods based on non-subsampled shear wave transform.

[0065] Figure 13 This is a comparison chart of the image denoising effects of the present invention and existing methods on a synthetic dataset;

[0066] Figure 14This is a comparison chart of the image denoising effects of the present invention and existing methods on real datasets;

[0067] Figure 15 This invention provides a comparison of denoising effects on UAV ship image datasets.

[0068] Figure 16 This is a comparison chart of the denoising effects of the present invention and existing methods on a lung dataset. Detailed Implementation

[0069] Specific implementation method one: as follows Figure 1 and Figure 4 As shown, the specific process of the diffusion model image denoising method based on non-subsampled shear wave transform in this embodiment is as follows:

[0070] Obtain the noisy image to be denoised, input the noisy image to be denoised into the trained denoising model, and obtain the denoised noisy image.

[0071] The trained denoising model is obtained through the following method:

[0072] The noisy image and the denoised image are combined into a training set. The denoising model is trained using the training set to obtain a well-trained denoising model.

[0073] The denoising model includes: an NSST module, a diffusion model module, a discriminator module, and an NSST inverse transform module;

[0074] The NSST module is used for processing noisy images. Non-subsampled shear wave transform (NSST) is performed to obtain high-frequency and low-frequency subband features, and the high-frequency and low-frequency subband features are combined to obtain fused features. ;

[0075] in, It is the preset number of sub-bands.

[0076] The diffusion model module includes: an iterative noise addition unit and a generator unit;

[0077] The iterative noise-adding unit is used to process the fused features. Each iteration adds noise, obtaining the fused features after each iteration of noise addition. Meanwhile, the iteratively noise-added fused features are sent to the generator unit;

[0078] in, , It is the iteration label. It is the preset total number of iterations;

[0079] The generator unit is used to determine, according to Generate denoised feature samples and the denoised feature samples , and Send to the discriminator module;

[0080] The generator unit includes: an initialization subunit and a noise reduction subunit;

[0081] The initialization subunit is used to initialize the iteration label. ;

[0082] The noise reduction subunit is used to... Generate corresponding denoised feature samples and the denoised feature samples , and Send to the discriminator module;

[0083] According to Generate corresponding denoised feature samples Specifically:

[0084]

[0085] in, It is the first Fusion features after adding noise and These are the mean and variance of the conditional distribution. , It is standard Gaussian noise. It is a generator unit;

[0086] and This is obtained by: the generator unit in each back-diffusion step... and time step coding As input, the forward prediction obtains and These represent the mean and standard deviation of the conditional distribution, respectively.

[0087] The discriminator module is used to determine the feature samples. , and Obtain image similarity classification values, and then obtain the denoised fusion features based on these classification values. Specifically:

[0088] First, obtain and similarity probability , and similarity probability Specifically:

[0089]

[0090]

[0091] in, It is a multilayer perceptron, and its output is a scalar probability value;

[0092] Then, using and similarity probability and and similarity probability Output image similarity classification values, specifically: If and If the difference is within a preset range, the output image similarity classification value is 1; otherwise, the output image similarity classification value is 0.

[0093] Finally, if the image similarity classification value is 0, the generator unit returns to its initialization subunit to regenerate. If the image similarity classification value is 1, then determine... Is it 0? If Then the output is the denoised fused feature. Otherwise, let And return the denoising subunit in the generator unit;

[0094] The NSST inverse transform module is used to process the denoised fusion features. Perform inverse NSST transform to obtain the denoised image. .

[0095] Specific implementation method two: such as Figure 2-3 As shown, this embodiment differs from specific embodiment one in that the NSST module is used for processing noisy images. Non-subsampled shear wave transform (NSST) is performed to obtain high-frequency and low-frequency subband features, and the high-frequency and low-frequency subband features are combined to obtain fused features. Specifically:

[0096] A1. Constructing the continuous shear wave transform function:

[0097] NSST is an extension of wavelets in multidimensional space. This transform combines a non-subsampled pyramid (NSP) filter with the shearlet transform (ST) to provide a multi-scale image decomposition method. The shearlet transform approximates the optimal sparse representation, and its affine system representation is shown below:

[0098]

[0099] in For shear wave function, For the mother wavelet function, and All A matrix of size ,parameter Indicates the number of scale layers, For the integer index of the decomposition direction, at the th There are a total of 100 floors In one direction, , respectively corresponding to the first One direction (shear angle); This is the spatial displacement index. Let... For the mother wavelet function, Representing image features, if the family of functions The tight frame condition must be met:

[0100]

[0101] in, It is defined in two-dimensional real space The space of square-integrable functions on;

[0102] but This is called a composite wavelet, and its expression is:

[0103] Let the mother wavelet function be... Its Fourier transform is .in It is a transformation corresponding to the frequency domain scale. It is a transformation corresponding to the frequency domain direction. For any frequency domain coordinates Let represent the frequency components in the horizontal and vertical directions in a two-dimensional frequency space. Then, the frequency domain form of the composite wavelet is defined as:

[0104]

[0105] Based on composite wavelet Obtaining image features The continuous shear wave transform function is as follows:

[0106]

[0107] in, It represents the scale of the input noisy image. Decomposition direction and displacement The continuous shear wave transform coefficient at the location, It is the scale-direction-displacement. Shear wave basis function The Fourier transform, in the medical image denoising scenario of this invention, For scale indexing, For decomposition direction index, For spatial displacement index;

[0108] A2. Obtain the high-frequency sub-band characteristics and low-frequency sub-band characteristics using the continuous shear wave transform function, specifically:

[0109]

[0110]

[0111]

[0112] in, It is the first high-frequency sub-band feature. It is the first High-frequency sub-band characteristics, It is a low-frequency sub-band characteristic. It is the first One directional filter bank, , It is a high-pass filter. , , , , It is a low-pass filter. It is a continuous shear wave transform function. This indicates the composition of operators. , It's a label. It is the first directional filter bank. It's convolution;

[0113] A3. Combine high-frequency subband features and low-frequency subband features to obtain fused features. ;

[0114] This embodiment addresses noisy images. Non-Subsampled Shearlet Transform (NSST) is performed to decompose image structural information at multiple scales and directions, extracting high- and low-frequency features to obtain fused features. NSST possesses translation-invariant characteristics, effectively reducing registration errors. The NSST employs a three-level pyramid decomposition framework to decouple and represent multi-scale geometric features of the image. Figure 3 As shown, given the number of subbands In this case, first input an image. After processing by the first-level multi-scale decomposition module, the filter passes through a dual-channel filter bank (low-pass filter). With high-pass filter This achieves initial frequency band separation. High-frequency components undergo geometric direction correction via shearing filtering and are then passed through a directional filter. The first high-frequency subband carrying pixel-level texture details is generated, while the low-frequency components enter the second-layer decomposition module and undergo secondary filtering. The intermediate-scale edge feature components are separated, and after adjusting the direction sensitivity of these components through shearing filtering, they are then used based on the input direction filter bank. The second high-frequency subband is generated, and the third-layer decomposition module performs final scale separation on the first two low-frequency signals, which are then filtered by a directional filter bank. The constrained output includes the third high-frequency subband and the global energy distribution subband.

[0115] The other steps and parameters are the same as in Specific Implementation Method 1.

[0116] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that: the iterative noise-adding unit is used to process the fused features. Each iteration adds noise, obtaining the fused features after each iteration of noise addition. , No. The conditional distribution of the additional noise is as follows:

[0117]

[0118] in, , It is variance scheduling. It is the identity matrix. It is the first Fusion features after secondary noise addition The conditional probability, It is a Gaussian distribution.

[0119] The other steps and parameters are the same as in Specific Implementation Method 1 or Specific Implementation Method 2.

[0120] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that: the NSST inverse transform module is used to perform an NSST inverse transform on the denoised fused features to obtain the denoised image. Specifically:

[0121] First, construct the inverse transform function of NSST:

[0122]

[0123] in, It is a two-dimensional real number space. This refers to the scale parameter and the discrete scale index. One-to-one correspondence; , , These are the scale, decomposition direction, and spatial displacement index, respectively. Features to be inversely transformed The inner product coefficient between the shear wave basis function and the shear wave.

[0124] symbol Indicated in scale ,direction Displacement The shear wave basis function itself Representing a noisy image In scale ,direction Displacement Basis function values;

[0125] Then, using the inverse transform function of NSST to... Perform an inverse transform to obtain the denoised image. :

[0126]

[0127]

[0128] in, It is the inverse NSST transform function. , … These are preset weighting coefficients. , yes The total number of sub-band features in the middle yes Sub-band feature labels in yes The first in Individual band characteristics, yes High-frequency subband characteristics in yes Low-frequency subband characteristics in yes The results of the inverse NSST transform.

[0129] The other steps and parameters are the same as in any of the specific implementation methods one to three.

[0130] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that the loss function used for training the generator unit in the diffusion model module is obtained in the following way:

[0131] First, obtain the KL loss. Specifically:

[0132]

[0133] in, Represents the distribution of real data. It is the probability distribution of the denoised feature samples generated by the generator. It is the KL divergence;

[0134] Then, to prevent the loss of frequency information, a reconstruction term loss was used in the denoising process. The corresponding formula is as follows:

[0135]

[0136] in, It is the output of the diffusion model;

[0137] Finally, using the reconstruction term loss and KL losses Obtain the total loss of the generator unit:

[0138]

[0139] in, These are preset weighted hyperparameters. It is the loss function of the generator unit.

[0140] The other steps and parameters are the same as in any of the specific implementation methods one to four.

[0141] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that the discriminator module is trained using the following loss function:

[0142]

[0143] in, It is the loss function of the discriminator module. Therefore Logarithmic operations with base 0.

[0144] This invention improves the performance of generative adversarial networks by minimizing losses; it stops iterative output when the generator loss and discriminator loss reach their minimum values. .

[0145] The other steps and parameters are the same as in any one of the specific implementation methods one to five.

[0146] The present invention provides an image denoising method based on a diffusion model using non-subsampled shear wave transform, as shown in Table 1.

[0147] Table 1

[0148]

[0149] Example: To verify the beneficial effects of the present invention, the following experiments were conducted in this example:

[0150] I. Experimental Dataset and Evaluation Metrics:

[0151] The experimental dataset includes real denoised images and synthetic denoised images. Synthetic denoised images are created by introducing noise into the original image and then applying denoising methods to reduce noise. The datasets used for synthetic denoised images include CBD68, Kodak24, and McMaster. CBD68 contains 68 synthetic denoised images of size 321×481; Kodak24 contains 24 synthetic denoised images; and McMaster contains 18 cropped images of size 500×500. During the experiment, the images in these three datasets were cropped to 256×256, and the three datasets were combined. Random sampling was used to divide the dataset into training and test sets. Realistic denoised images refer to images captured from actual scenes. These images contain various real-world noises and interferences. The selected datasets are Nam and PolyU. The Nam dataset contains 500 JPEG images of 11 scenes, with an image size of 512×512. PolyU consists of real-world scene images released by Hong Kong Polytechnic University, including urban landscapes and traffic scenes, with an image size of 512×512. The two datasets are cropped to 256×256 and combined. Using random sampling, images are selected from the datasets to divide them into training and test sets.

[0152] The evaluation metrics used for the diffusion model image denoising method based on non-subsampled shear wave transform are: Peak Signal to Noise Ratio (PSNR), Learned Perceptual Image Patch Similarity (LPIPS), and Frechet inception distance (FID).

[0153] (1) Peak signal-to-noise ratio

[0154] PSNR is a metric used to measure image or video quality, and its calculation formula is as follows:

[0155]

[0156] Where MAX represents the maximum pixel value of the image, and MSE is the mean squared error, which is the cumulative error between the processed image and the original image. The lower the MSE, the better the quality of the seepage prevention image. The calculation formula is:

[0157]

[0158] in, and Represents the width and height of the image. Indicates the original clean image in Pixel value at that location, This indicates the final denoised image. The pixel value at that location. The higher the PSNR value, the more noise is removed, and the better the noise reduction effect.

[0159] (2) Learning to perceive image patch similarity

[0160] Learning to perceive image patch similarity is a method for measuring the similarity between two image patches. It utilizes the AlexNet or VGG model to map the input image into a high-dimensional space. Using pre-similarity of image features as the similarity metric can better capture the semantic information of the image and improve the accuracy of similarity measurement. The corresponding calculation formula is as follows:

[0161]

[0162] in, , These represent the original image and the denoised image in the VGG or AlexNet model network, respectively. Features obtained from the layer and These represent the width and height of the image, respectively, and the feature ratio between layers is represented by... The lower the LPIPS value, the smaller the difference in the obtained image, and the more realistic the image.

[0163] (3) Frechet starting distance

[0164] The FID metric measures the authenticity of generated samples by calculating the distance between generated samples and real samples in the feature space. The calculation process is shown in the following formula:

[0165]

[0166] in, and Let represent the mean and covariance of the true sample, respectively. and Let represent the mean and covariance of the generated samples, respectively. Represents the trace of the matrix. This represents the Euclidean distance norm. The smaller the FID value, the higher the similarity between the generated sample and the real sample.

[0167] II. Parameter Analysis:

[0168] To address the slow training and inference speeds and image distortion issues of traditional denoising methods, this invention proposes a denoising diffusion model based on non-subsampled shear wave transform. This diffusion model incorporates non-subsampled shear wave transform into its structure to enhance multi-scale and multi-directional feature representation. Gaussian noise is then added to the synthetic dataset. The key parameters of the model are analyzed.

[0169] For parameter analysis, the CPU used was an 11th Gen Intel(R) Core(TM) i7-11800H @2.30GHz, the GPU was an NVIDIA A100 NVLink 40G and an NVIDIA GeForce RTX 3090 Laptop GPU with 24GB of RAM, and the memory was 32GB. The software environment was Ubuntu 20.04; CUDA version was 11.3, Python version was 3.8, and the deep learning framework was PyTorch 1.10.3. The method of this invention was applied to the DDPM method to explore the optimal results.

[0170] (1) Parameter analysis of image generation method based on diffusion model

[0171] To optimize the real-time performance and accuracy of this invention, the time step T and noise adjustment coefficient in the diffusion model need to be adjusted during image diffusion. To achieve the best results, a combination of settings was performed, and the corresponding experimental results are as follows: Figures 5-7 As shown.

[0172] The parameter analysis results show that as the time step increases and As the value increases, the PSNR value first increases and then decreases. When the time step is 4, When the value is 0.2, the PSNR value is the highest, at 28.29%; under the same conditions, the values ​​of LPIPS and FID both decrease first and then increase, with the value increasing at a time step of 2. When the value is 0.2, the LPIPS value is the smallest, at 0.112. When the time step is 4, When the time step is 0.2, the FID value is the smallest, at 36.45. This shows that the experimental results are as expected, meaning that as the time step increases, the denoising process in the diffusion model can generate accurate images after multiple iterations. However, when the time step is too large and... As the size increases, it will focus too much on local information, causing the generated image to differ from the actual image.

[0173] To further verify the accuracy of the experimental results, a time step T of 4 was plotted. The generated loss curve with a value of 0.2 and the loss curve generated by DDPM are shown in the following results. Figure 8 As shown in the figure. The experimental results show that the image generation method based on the diffusion model can stabilize faster and with smaller errors when calculating the loss, proving that generative adversarial networks can be used to generate more accurate images, which is in line with the experimental expectations.

[0174] (2) Parameter analysis of image denoising method based on non-subsampled shear wave transform

[0175] A non-subsampled shear wave transform (NSST)-based image denoising method is introduced. DDPM directly denoises the image, which is time-consuming. Therefore, the NSST transform is used to decompose the image, reducing its dimensionality and fully utilizing information from different frequencies to improve denoising accuracy and real-time performance. The number of subbands in the NSST decomposition is also considered. Parameter analysis reveals that too many subbands lead to significant information redundancy, excessively long training times, and overfitting. Too few subbands prevent the full utilization of low-to-mid-frequency and high-frequency information in image generation. Therefore, a suitable number of subbands needs to be found in the NSST decomposition to balance the utilization of various frequency information in the image. Parameter selection... An optimal image generator based on non-subsampled shear wave transform was constructed, and the experimental results are as follows. Figures 9-11 As shown.

[0176] The parameter analysis results show that when the original image is decomposed into different numbers of sub-bands, the PSNR value first increases and then decreases as the number of sub-bands increases. The PSNR value is the highest at n=4, reaching 28.75%. Under the same conditions, the values ​​of LPIPS and FID both first decrease and then increase. The LPIPS value is the lowest at n=8, reaching 0.164, and the FID value is the lowest at n=4, reaching 56.71. Therefore, the experimental results are as expected, indicating that multiple sub-band information can improve image denoising. However, too much sub-band information results in a large amount of redundant information, leading to poor training performance.

[0177] To further verify the accuracy of the experimental results, this embodiment provides the number of NSST decomposition subbands. Loss curves for DDPM at time 4 and for image denoising methods based on non-subsampled shear wave transform are generated, such as... Figure 12 As shown in the figure, the loss curves demonstrate that the image denoising method based on non-subsampled shear wave transform can make fuller use of the information in different subbands of the image, thus achieving rapid convergence and stabilization, further illustrating the accuracy of the experimental results.

[0178] III. Ablation Experiment

[0179] To verify the effectiveness of the present invention, the image generation method based on the diffusion model and the image denoising method based on the non-subsampled shear wave transform were applied to the DDPM method, and ablation experiments were conducted on a synthetic dataset. The results are shown in Table 2.

[0180] Table 2 Results of ablation experiments performed on the synthetic dataset

[0181]

[0182] Note: Method 1 represents the image generation method based on the diffusion model; Method 2 represents the image denoising method based on non-subsampled shear wave transform embedding; Experiment 1 is the experimental result of DDPM; Experiment 2 incorporates the image generation method based on the diffusion model; Experiment 3 incorporates the image denoising method based on non-subsampled shear wave transform; Experiment 4 incorporates the image denoising method based on non-subsampled shear wave transform and the image generation method based on the diffusion model.

[0183] As shown in Table 2, compared with Experiment 1, Experiment 2 showed a 1.55% improvement in PSNR, a 0.068 decrease in LPIPS, and a 0.16 increase in FID. This is because a Generative Adversarial Network (GAN) mechanism was introduced on top of DDPM. The discriminator, by comparing the real image with the generated image, forced the generator to learn high-frequency textures that better conform to the real distribution, thereby effectively suppressing artifacts and improving perceptual quality. However, due to the randomness introduced by GAN, FID fluctuated slightly. Compared with Experiment 1, Experiment 3 showed a 1.09% improvement in PSNR, a 0.138 decrease in LPIPS, and a 20.1 decrease in FID. This significant improvement in FID stemmed from the introduction of non-subsampled shear wave transform (NSST) at the front end of the diffusion model. NSST decomposes the image into low-frequency and high-frequency subbands, making the generated denoising results closer to the real image in structure and texture, while reducing the computational burden of the diffusion process. Compared with Experiment 1, Experiment 4 showed a 3.82% improvement in PSNR, a 0.145 decrease in LPIPS, and a 21.18 decrease in FID. The best performance across all experiments was achieved, indicating that the two methods complement each other in the diffusion model. Their synergistic effect significantly outperforms the DDPM model in both subjective visual effects and objective metrics. Regarding params, since NSST decomposes the image into subbands during preprocessing, the diffusion and generation processes handle low-dimensional subband data rather than the complete image, significantly reducing the time complexity of feature extraction. Experiment 4's params are reduced by 5.25M compared to Experiment 1. In terms of computational complexity (FLOPs), the denoising diffusion model based on non-subsampled shear wave transform can better utilize information from different subbands to accelerate computation during diffusion, thus reducing computational complexity. Compared to Experiment 1, the method in this embodiment reduces FLOPS by 41.98G in Experiment 4. From a memory perspective, the method in Experiment 4 of this embodiment uses subband information during training and testing, reducing the image dimension compared to the original and thus reducing memory usage. Compared to Experiment 1, the memory usage of Experiment 4 in this embodiment is reduced by 2.14G. These three parameters all reflect that the method of this invention has reduced both time and space complexity compared to DDPM, which is in line with experimental expectations. Regarding the time required to generate denoised images, the method of this invention can accelerate image generation using subband information. Compared to DDPM, the method of this invention can generate denoised images in only 0.7s, shortening the time by 68%, ensuring real-time denoising and meeting experimental expectations.

[0184] IV. Comparison of Noise Reduction Methods

[0185] To verify the effectiveness of the diffusion model image denoising method based on non-subsampled shear wave transform, PSNR, LPIPS, and FID were used as evaluation metrics for the model. Params, FLOPs, and MEM were used to measure the computational complexity of the model. The time required to generate one denoised image was also given. All experimental groups used the Adam optimizer with a batch size of 32. The learning rate of the generator in the GAN-based method was set to 0.000125, the learning rate of the discriminator was set to 0.00016, and the learning rate of the other methods was 0.0002.

[0186] First, the proposed method is compared with DnCNN, FFDNet, DRUnet, Restormer, and DDPM on synthetic datasets. The synthetic dataset is constructed using CBD68, Kodak24, and McMaster datasets, and different sizes of Gaussian noise are added to it. , , , , To explore optimal performance, the experimental results of the method of this invention and existing methods on synthetic datasets are shown in Table 3.

[0187] Table 3

[0188]

[0189] As shown in Table 3, when adding different levels of Gaussian noise, the method of this invention achieves the best results in PSNR, LPIPS, and FID compared to the state-of-the-art method. Considering the parameters required for model execution (Params), computational complexity, and memory (MEM), the method of this invention also achieves optimal performance, but the value of Params is significantly greater than that of DnCNN. This is because DnCNN uses a simpler convolutional neural network, thus reducing the number of parameters, but requiring multiple iterations and extensive computation to complete the image denoising task. Therefore, the method of this invention is more suitable for practical applications. Regarding the time to generate denoised images, because this invention employs a non-subsampled shear wave transform method, it can fully utilize information from different frequencies in the image. Therefore, the method of this invention generates denoised images faster, 0.4 seconds faster than the current fastest method, DnCNN. In summary, the method of this invention shows varying degrees of improvement compared to the state-of-the-art method, and performs better in practical applications, meeting experimental expectations.

[0190] Secondly, random sampling was performed on the Nam and PolyU datasets to create training and testing sets, forming a real-world dataset. A real-world dataset refers to data subjected to noise in real-world scenarios, rather than artificially added noise. This type of dataset better reflects the performance of the denoising algorithm in practical applications, thus allowing for a more accurate assessment of the algorithm's accuracy. The method of this invention was compared with DaNet, MIRNet, DamNet, NAFNet, and Uformer methods. The results using the real-world dataset are shown in Table 4.

[0191] Table 4

[0192]

[0193] Comparative experimental results show that the method of this invention significantly improves upon state-of-the-art methods in generating denoised images. Specifically, PSNR is improved by 0.48% compared to the state-of-the-art method, LPIPS is reduced by 0.008, and FID is reduced by 1.2. Regarding parameter count, DaNet and MIRNet have relatively few parameters because they use simple convolutional neural networks without setting too many parameters. However, this also leads to the need for multiple iterations to generate accurate results, and their denoising effect on untrained images is poor, making them unsuitable for practical applications. In terms of memory, the method of this invention fully utilizes information from different subbands of the image, improving speed while reducing memory usage, as expected. Regarding time, the method of this invention can denoise images at the millisecond level, possessing the strongest real-time performance, as expected.

[0194] V. Analysis of Image Denoising Results

[0195] To further verify the accuracy of the method for image denoising in this invention, the results are visualized and compared. Different image denoising results are shown below. Figure 13 , Figure 14 , Figure 15 , Figure 16 As shown. Figure 13 The images used are from the CBD68 dataset. Other methods smooth edge information when denoising images, which leads to image distortion. For example, the mouth, beard, and hair of the person in the image will not match reality. However, this invention uses an image generator based on non-subsampled shear wave transform to generate images. When generating images, it uses information from different subbands to retain the details in the image, which can solve the image distortion problem. Figure 13 In the image, (a) shows the addition of Gaussian noise to the original image. (a) is the noisy image, (b) is the denoising result of DnCNN, (c) is the denoising result of FFDNet, (d) is the denoising result of DRUnet, (e) is the denoising result of Restormer, (f) is the denoising result of DDPM, (g) is the denoising result of the method in this chapter, and (h) is the original image.

[0196] like Figure 14 As shown, the images are from the Kodak24 dataset. When denoising real images, other methods only perform well on Gaussian noise, showing poor denoising effects for other types of noise, such as salt-and-pepper noise. Furthermore, during model testing, elements not originally present in the pixels appear, resulting in image distortion. This invention uses a diffusion model based on non-subsampled shear wave transform. During image generation, multiple comparisons ensure accuracy. Simultaneously, non-subsampled shear wave transform decomposes the image, fully utilizing features of different frequencies. This improves both the quality of the denoised image generation and the real-time performance of denoising, ensuring its usability in real-world scenarios. Figure 14 In the image, (a) is a noisy image, (b) is the DeamNet denoising result, (c) is the NAFNet denoising result, (d) is the Restormer denoising result, (e) is the Uformer denoising result, (f) is the DDPM denoising result, (g) is the denoising result of the method in this chapter, and (h) is the original image.

[0197] like Figure 15 As shown, when training with non-UAV ship images, the denoising accuracy of image denoising methods based on convolutional neural networks and Transformers, such as Uformer, will decrease or even fail to denoise. However, the image denoising method of this invention, which uses non-subsampled shear wave transform and diffusion model embedding, achieves good denoising results in UAV ship images even when trained with non-UAV data. Compared with current methods, this invention achieves the best results, meeting experimental expectations. Figure 15 In the diagram, (a) is the original image, (b) is the denoising result of DnCNN, (c) is the denoising result of FFDNet, (d) is the denoising result of NAFNet, (e) is the denoising result of Uformer, (f) is the denoising result of Restormer, (g) is the denoising result of DDPM, and (h) is the denoising result of the present invention.

[0198] like Figure 16As shown, the images used are samples from the LIDC-IDRI dataset. When trained and tested using a lung CT image dataset, the denoising method of this invention is more effective than other denoising methods in removing noise caused by equipment or low-dose scanning on lung CT images, making the lung parenchyma clearer. This will help reduce interference with the detection of lesion areas and improve the accuracy of lesion area detection. Figure 16 In the diagram, (a) is the original image, (b) is the denoising result of FFDNet, (c) is the denoising result of DnCNN, (d) is the denoising result of NAFNet, (e) is the denoising result of Uformer, (f) is the denoising result of Restormer, (g) is the denoising result of DDPM, and (h) is the denoising result of the present invention.

[0199] In summary, the experimental results show that the present invention exhibits excellent performance on synthetic, real, UAV / ship, and lung CT datasets. It outperforms existing mainstream methods in terms of evaluation metrics such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), and has a significant advantage in inference speed.

Claims

1. A diffusion model image denoising method based on a non-subsampled shearlet transform, characterized in that The method specifically comprises the following steps: Obtaining a noise image to be denoised, inputting the noise image to be denoised into a trained denoising model, and obtaining a denoised noise image; The denoising model comprises an NSST module, a diffusion model module, a discriminator module and an NSST inverse transformation module; The NSST module is used for performing non-subsampled shearlet transform on a noisy image to obtain high-frequency subband features and low-frequency subband features, and combining the high-frequency subband features and the low-frequency subband features to obtain fused features ; wherein, is a preset subband number; The diffusion model module comprises an iterative noise adding unit and a generator unit; The iterative noise adding unit is configured to add noise to the fusion feature obtain the fusion feature after each iteration of noise adding and send the fusion feature after the iteration of noise adding to the generator unit. The generator unit is configured to generate the de-noised feature samples according to the de-noised feature samples , and send the de-noised feature samples , and to the discriminator module. The discriminator module is configured to obtain an image similarity classification value according to the feature sample , and obtain the fused feature after denoising based on the image similarity classification value . The NSST inverse transform module is configured to perform NSST inverse transform on the de-noised fused features perform NSST inverse transform to obtain a de-noised image .

2. The non-subsampled shearlet transform based diffusion model image denoising method of claim 1, wherein: The trained denoising model is obtained by the following method: a training set is formed by a noise image and a denoised image, the denoising model is trained by using the training set, and the trained denoising model is obtained.

3. The non-subsampled shearlet transform based diffusion model image denoising method of claim 2, wherein: The NSST module is used for performing non-subsampled shearlet transform on a noisy image obtaining high-frequency sub-band features and low-frequency sub-band features, specifically: wherein, is a 1st high-frequency sub-band feature, is a 2nd high-frequency sub-band feature, is a 3rd high-frequency sub-band feature, is a low-frequency sub-band feature, is a 1st directional filter bank, is a 2nd directional filter bank, is a 3rd directional filter bank, , is a high-pass filter, , , , , is a low-pass filter, is a continuous shearlet transform function, denotes an operator composition, , is an index, is a convolution.

4. The non-subsampled shearlet transform based diffusion model image denoising method of claim 3, wherein: The conditional distribution of each iteration in the iterative noise adding unit is as follows: wherein, is the first noise-added fused feature conditional probability, , is the variance schedule, is the identity matrix.

5. The non-subsampled shearlet transform based diffusion model image denoising method of claim 4, wherein: The generator unit comprises an initialization subunit and a denoising subunit; The initialization sub-unit is configured to initialize an iteration label ; The denoising sub-unit is configured to generate corresponding denoised feature samples according to the denoised feature samples , 、 and send the denoised feature samples to the discriminator module. According to Generating corresponding denoised feature samples Specifically: where, is the th fused feature after and are the mean and variance of the conditional distribution, , is a standard Gaussian noise, is a generator unit.

6. The non-subsampled shearlet transform based diffusion model image denoising method of claim 5, wherein: The discriminator module is configured to obtain an image similarity classification value according to the feature sample , and obtain the fused feature after denoising based on the image similarity classification value , specifically: First, obtain the similarity probability of , the similarity probability of ; Then, the similarity probability of the output image and the image in the image database is calculated. The similarity probability of the output image and the image in the image database is calculated. The similarity probability of the output image and the image in the image database is calculated. The similarity probability of the output image and the image in the image database is calculated. The similarity probability of the output image and the image in the image database is calculated. The similarity probability of the output image and the image in the image database is calculated. ​ If With the difference is within a preset range, the image similarity classification value is 1; Otherwise, the image similarity classification value is 0; Finally, if the image similarity classification value is 0, the initialization subunit of the generator unit is returned to regenerate ; if the image similarity classification value is 1, it is determined whether it is 0, if , the de-noised fusion feature is output , otherwise, let , and return to the de-noising subunit in the generator unit.

7. The non-subsampled shearlet transform based diffusion model image denoising method of claim 6, wherein: The Similarity probability of , Similarity probability of , Similarity probability of , specifically: wherein, is a multi-layer perceptron.

8. The non-subsampled shearlet transform based diffusion model image denoising method of claim 7, wherein: The NSST inverse transform module is configured to perform NSST inverse transform on the denoised fusion feature perform NSST inverse transform to obtain a denoised image Specifically, wherein, is an NSST inverse transform function, , , is a preset weight coefficient, , is a total number of subband features in is a subband feature index in is a thsubband feature in is an NSST inverse transform result of .

9. The non-subsampled shearlet transform based diffusion model image denoising method of claim 8, wherein: The loss function used in the training of the generator unit in the diffusion model module is as follows: wherein, is the probability distribution of the denoised feature samples generated by the generator, is the KL divergence, is the output of the diffusion model, is the preset weighting hyperparameter, is the loss function of the generator unit.

10. The non-subsampled shearlet transform based diffusion model image denoising method of claim 9, wherein: The discriminator module is trained by using the following loss function: wherein, is a loss function of the discriminator module, is a logarithm operation with base .

Images