Image denoising method based on conditional diffusion model

Abstract

The invention discloses an image denoising method based on a conditional diffusion model, which comprises the following steps of: downloading a noise-containing image, extracting edge information in the noise-containing image through an image edge extractor, and guiding the training of the conditional diffusion model by using the edge information until the conditional diffusion model converges; in the denoising process, the trained conditional diffusion model iteratively refines a noise image from a rough state to a fine state by taking edge information as a diffusion guide condition, and finally an accurate and noiseless high-quality image is obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and more specifically, relates to an OFDMA-based fusion access network system. Background Art

[0002] During image acquisition and transmission, various factors often reduce image quality and introduce image noise, such as equipment noise and environmental interference. This noise not only weakens the accuracy of information extraction, but also leads to the loss of valuable image data. Therefore, image denoising has become an important research hotspot in the field of image processing.

[0003] Image denoising is an important part of image processing, which aims to restore clear and real image content from images disturbed by noise. Existing denoising methods include traditional methods and deep learning-based methods. Traditional methods include filtering-based, wavelet transform-based, sparse representation-based, and deep learning-based methods. Although these methods are effective to a certain extent, they may often introduce problems such as blurring and distortion during the denoising process. Among the deep learning-based methods, generative methods perform best. These methods use generative models to restore images corrupted by noise. Generative models learn the high-dimensional distribution of image data and can generate new samples similar to the training data. In image denoising, these models are trained to produce noise-free images, facilitating the recovery of clean images from noisy inputs.

[0004] Among these methods, the diffusion model-based methods have achieved state-of-the-art performance due to their unique sampling mechanism. The diffusion model simulates a thermodynamic process, gradually adding noise in the forward process and gradually removing the noise to reconstruct noise-free data in the reverse process. During training, the diffusion model learns to predict the backward transition probability and optimizes the model parameters by minimizing the difference between the predicted backward probability and the true data distribution. The ability of diffusion models to gradually remove noise during the sampling process highlights their important research value and potential applications in image denoising.

[0005] Despite the progress made in diffusion models, there is still a gap between their sampling process and their practical application in image denoising. First, the diffusion model sampling starts with pure Gaussian noise, while the image denoising task starts with a noisy image, causing a mismatch between the two processes. Second, adding random noise to the diffusion model sampling process leads to an uncontrollable process, which causes semantic inconsistency in the results. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an image denoising method based on a conditional diffusion model, which uses the edge information of the noisy image as a guiding condition and obtains a high-quality image through multiple denoising iterations.

[0007] To achieve the above object of the invention, the present invention provides an image denoising method based on a conditional diffusion model, characterized in that it comprises the following steps:

[0008] (1) Download the noisy image x from the database;

[0009] (2) Extract edge information of the noisy image x;

[0010] Input the noise image x to the noise image edge extraction module, so as to extract the edge information edge of the noise image x;

[0011] (3) Calculate the number of steps of the noise image x in the diffusion process;

[0012] First, the variance v of the noise image x is estimated using the wavelet transform method; then the number of steps t of the noise image x in the diffusion process is calculated using the following formula;

[0013] t=100*v / 255

[0014] (4) De-noising the noisy image x according to the trained conditional diffusion noise predictor D;

[0015] Set the diffusion step number t of the conditional diffusion noise predictor D, and then input the noise image x into the trained conditional diffusion noise predictor D to obtain the output D(x, t, edge) of the conditional diffusion noise predictor D;

[0016] Get the denoised image x 0 :

[0017]

[0018] in, is the preset parameter, α i is the preset parameter in the i-th step diffusion;

[0019] (5) Multiple rounds of iterative denoising;

[0020] (5.1) Set the number of iterative denoising steps N;

[0021] (5.2) Perform weighted fusion of image x0 and noise image x to obtain a new noise image x 1 ;

[0022] (5.3) Using the noisy image x 1 Replace the noise image x, and then return to step (2);

[0023] (5.4) Repeat the iterative denoising step N times to obtain a clean image after multiple rounds of denoising.

[0024] The object of the invention of the present invention is achieved in this way:

[0025] The invention discloses an image denoising method based on a conditional diffusion model. The method comprises the following steps: firstly downloading a noisy image, then extracting edge information in the noisy image through an image edge extractor, and using the edge information to guide the training of the conditional diffusion model until the conditional diffusion model converges. In the denoising process, the trained conditional diffusion model uses the edge information as a diffusion-guiding condition to iteratively refine the noisy image from a coarse state to a fine state, and finally obtaining an accurate and noise-free high-quality image.

[0026] At the same time, the image denoising method based on the conditional diffusion model of the present invention also has the following beneficial effects:

[0027] (1) Unlike the traditional diffusion model that starts sampling from pure Gaussian noise, the present invention uses a noisy image as the initial input of the diffusion process. Therefore, the diffusion model proposed in the present invention significantly reduces the number of required training steps;

[0028] (2) To ensure semantic consistency while removing noise, the present invention integrates edge information obtained from the image as a guiding condition throughout the training and sampling stages. This integration helps preserve the structural details of the image.

[0029] (3) In order to further reduce the number of denoising steps and improve the perceived quality, the present invention performs multiple denoising iterations and uses the results of each iteration to provide edge and texture information for subsequent iterations. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of an image denoising method based on a conditional diffusion model of the present invention;

[0031] Figure 2 is the structure diagram of the conditional diffusion noise predictor;

[0032] Figure 3 It is a display diagram of the edge information of the noise image extracted during each round of iteration;

[0033] Figure 4 This is the effect picture after multiple rounds of denoising; DETAILED DESCRIPTION

[0034] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0035] Example

[0036] Figure 1 The present invention is a flow chart of an image denoising method based on a conditional diffusion model.

[0037] In this embodiment, if Figure 1 As shown, the present invention provides an image denoising method based on a conditional diffusion model, comprising the following steps:

[0038] (1) Download the noisy image x from the database. We can first divide all pixel values ​​of the noisy image x by 255 so that all pixel values ​​of the noisy image x are scaled to between 0 and 1.

[0039] (2) Extract edge information of the noisy image x;

[0040] Input the noise image x to the noise image edge extraction module, so as to extract the edge information edge of the noise image x;

[0041] (3) Calculate the number of steps of the noise image x in the diffusion process;

[0042] First, the variance v of the noise image x is estimated by wavelet transform method; then the number of steps t of the noise image x in the diffusion process is calculated by the following formula;

[0043] t=100*v / 255

[0044] (4) De-noising the noisy image x according to the trained conditional diffusion noise predictor D;

[0045] Set the diffusion step number t of the conditional diffusion noise predictor D, and then input the noise image x into the trained conditional diffusion noise predictor D to obtain the output D(x, t, edge) of the conditional diffusion noise predictor D;

[0046] Get the denoised image x 0 :

[0047]

[0048] in, is the preset parameter, α i is the preset parameter for the i-th step diffusion, where α i It is a set of 100 fixed arithmetic progressions, from 0.002 to 0.01;

[0049] In this embodiment, a conditional diffusion noise predictor D is used when image denoising is performed based on the conditional diffusion model, and its structure is shown in Figure 2. Therefore, we need to first use a large number of images in the image library to train the conditional diffusion noise predictor D. The specific training process is as follows:

[0050] 1) Download N clean images and use the noise module to denoise the jth clean image Continuously add T times of random noise to obtain a series of noise samples The specific operations are as follows:

[0051]

[0052] Where i = 1, 2, ..., N, t = 1, 2, ..., T, and the parameters α k is the preset parameter for the kth iteration, ε t is the tth time Gaussian noise is added, and it follows a distribution with a mean of 0 and a variance of 1;

[0053] 3) The noise samples Input to the conditional diffusion noise predictor D, predict each time added to the i-th sample Z i The amount of noise

[0054] 4) Calculate the loss function value after this round of iteration;

[0055]

[0056] 5) Update the parameters in the conditional diffuse noise predictor D through back propagation of the loss value, and then conduct the next round of training until the conditional diffuse noise predictor D converges.

[0057] (5) Multiple rounds of iterative denoising;

[0058] (5.1) Set the number of iterative denoising steps N = 5;

[0059] (5.2) Perform weighted fusion of image x0 and noise image x to obtain a new noise image x 1 , that is, x 1 =0.5*x0+0.5*x, the weight value is 0.5 in weighted fusion;

[0060] (5.3) Using the noisy image x 1 Replace the noise image x, and then return to step (2);

[0061] (5.4) Repeat the iterative denoising step N times to obtain a clean image after multiple rounds of denoising.

[0062] Figure 3 It is a display diagram of the edge information of the noise image extracted during each round of iteration;

[0063] In this embodiment, if Figure 3As shown in the figure, iteration 0 to iteration 4 represent the edge extraction effect diagram of the edge extraction module from the 0th iteration to the 4th iteration. The Canny edge column represents the image edge information extracted from the clean image by the canny algorithm. By comparison, it can be seen that the edge extraction module can accurately extract the edge information of the noisy image.

[0064] Figure 4 This is the effect picture after multiple rounds of denoising;

[0065] In this embodiment, if Figure 4 As shown in the figure, the OURs column is the denoising result of the present invention; the Noised column represents the noisy image that needs to be denoised, and the Clean column represents the clean image corresponding to the noisy image. By comparing the three columns of images, it can be found that the present invention can remove noise well and obtain a clean image.

[0066] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. An image denoising method based on a conditional diffusion model, characterized in that: The following steps are involved: (1) Download the noisy image x from the database; (2) Extract edge information of the noisy image x; Input the noise image x to the noise image edge extraction module, so as to extract the edge information edge of the noise image x; (3) Calculate the number of steps of the noise image x in the diffusion process; First, use the wavelet transform method to estimate the variance v of the noise image x; Then the number of steps t of the noise image x in the diffusion process is calculated by the following formula; t=100*v / 255 (4) De-noising the noisy image x according to the trained conditional diffusion noise predictor D; Set the diffusion step number t of the conditional diffusion noise predictor D, and then input the noise image x into the trained conditional diffusion noise predictor D to obtain the output D(x, t, edge) of the conditional diffusion noise predictor D; Get the denoised image x0: in, is the preset parameter, α i is the preset parameter in the i-th step diffusion; (5) multiple rounds of iterative denoising; (5.1) Set the number of iterative denoising steps N; (5.2) Perform weighted fusion of image x0 and noise image x to obtain a new noise image x 1 ; (5.3) Using the noisy image x 1 Replace the noise image x, and then return to step (2); (5.4) Repeat the iterative denoising step N times to obtain a clean image after multiple rounds of denoising.

2. The image denoising method based on the conditional diffusion model according to claim 1, characterized in that: The pre-training process of the conditional diffuse noise predictor D is: (2.1) Download N clean images and use the noise module to denoise the jth clean image. Continuously add T times of random noise to obtain a series of noise samples The specific operations are as follows: Where i = 1, 2, ..., N, t = 1, 2, ..., T, and the parameters α k is the preset parameter for the kth iteration, ε t is the tth time Gaussian noise is added, and it follows a distribution with a mean of 0 and a variance of 1; (2.3), the noise sample Input to the conditional diffusion noise predictor D, predict each time added to the i-th sample Z i The amount of noise (2.4) Calculate the loss function value after this round of iteration; (2.5) The loss value is used to update the parameters in the conditional diffuse noise predictor D through back propagation, and then the next round of training is carried out until the conditional diffuse noise predictor D converges.

Images