An Image Denoising Method Based on Conditional Diffusion Model

Abstract

The present invention discloses an image denoising method based on a conditional diffusion model. First, a noisy image is downloaded, and then an image edge extractor is used to extract the edge information in the noisy image, and the edge information is used to guide the training of the conditional diffusion model until the conditional diffusion model converges; in the denoising process, the trained conditional diffusion model uses the edge information as the condition for diffusion guidance, iteratively refining the noisy image from a rough state to a fine state, and finally obtaining a high-quality image that is accurate and noise-free.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and more specifically, relates to a converged access network system based on OFDMA. Background Art

[0002] During the image acquisition and transmission process, various factors often reduce the image quality and introduce image noise, such as device noise and environmental interference. This kind of noise not only weakens the accuracy of information extraction, but also causes the loss of valuable image data. Therefore, image denoising has become an important research hotspot in the field of image processing.

[0003] Image denoising is an important link in image processing, aiming to restore clear and real image content from a noise-disturbed image. Existing denoising methods include traditional methods and deep learning-based methods. Traditional methods include those based on filtering, wavelet transform, sparse representation, and deep learning. Although these methods are effective to a certain extent, they may often introduce problems such as blurring and distortion during the denoising process. Among the deep learning-based methods, generative methods perform the best. These methods use generative models to restore images damaged by noise. The generative models learn the high-dimensional distribution of image data and can generate new samples similar to the training data. In image denoising, these models are trained to produce noise-free images, facilitating the restoration of clean images from noisy inputs.

[0004] Among these methods, the method based on diffusion models has achieved state-of-the-art performance due to its unique sampling mechanism. The diffusion models simulate a thermodynamic process, gradually adding noise in the forward process and gradually removing noise, and reconstructing noise-free data in the reverse process. During the training process, the diffusion models learn to predict the reverse transition probability and optimize the model parameters by minimizing the difference between the predicted reverse probability and the true data distribution. The ability of diffusion models to gradually remove noise during the sampling process highlights their important research value and potential applications in image denoising.

[0005] Despite the progress of diffusion models, there is still a gap between their sampling process and their practical applications in image denoising. First, the diffusion model sampling starts from pure Gaussian noise, while the image denoising task starts from a noisy image, resulting in a mismatch between the two processes. Second, adding random noise during the diffusion model sampling process leads to an uncontrollable process, resulting in semantic inconsistencies in the results. Summary of the Invention

[0006] The purpose of the present invention is to overcome the deficiencies of the prior art and provide an image denoising method based on a conditional diffusion model, using the edge information of a noisy image as a guiding condition, and obtaining a high-quality image through multiple denoising iterations.

[0007] To achieve the above-mentioned invention objective, an image denoising method based on a conditional diffusion model according to the present invention is characterized by comprising the following steps:

[0008] (1) Download a noisy image x from a database;

[0009] (2) Extract the edge information of the noisy image x;

[0010] Input the noisy image x into a noisy image edge extraction module to extract the edge information edge of the noisy image x;

[0011] (3) Calculate the number of steps in the diffusion process of the noisy image x;

[0012] First, use the wavelet transform method to estimate the variance v of the noisy image x; then calculate the number of steps t in the diffusion process of the noisy image x through the following formula:

[0013] t = 100 * v / 255

[0014] (4) Denoise the noisy image x according to the trained conditional diffusion noise predictor D;

[0015] Set the diffusion step t of the conditional diffusion noise predictor D, then input the noisy image x into the trained conditional diffusion noise predictor D to obtain the output D(x, t, edge) of the conditional diffusion noise predictor D;

[0016] Obtain the denoised image x 0 :

[0017]

[0018] Wherein, is a preset parameter, α i is the parameter preset at the i-th step of diffusion;

[0019] (5) Denoise through multiple rounds of iteration;

[0020] (5.1) Set the number of iteration denoising steps N;

[0021] (5.2) Perform weighted fusion on the image x 0 and the noisy image x to obtain a new noisy image x 1 ;

[0022] (5.3) Replace the noisy image x with the noisy image x 1 , and then return to step (2);

[0023] (5.4) Repeat the iteration denoising step N times to obtain a clean image after multiple rounds of denoising.

[0024] The object of the present invention is achieved as follows:

[0025] A method for image denoising based on a conditional diffusion model according to the present invention first downloads a noisy image, then extracts the edge information in the noisy image through an image edge extractor, and uses the edge information to guide the training of the conditional diffusion model until the conditional diffusion model converges; during the denoising process, the trained conditional diffusion model uses the edge information as the condition for diffusion guidance, iteratively refines the noisy image from a rough state to a fine state, and finally obtains a high-quality image that is accurate and noise-free.

[0026] Meanwhile, a method for image denoising based on a conditional diffusion model according to the present invention also has the following beneficial effects:

[0027] (1) Different from the method of starting sampling from pure Gaussian noise in traditional diffusion models, the present invention uses a noisy image as the initial input for the diffusion process. Therefore, the diffusion model proposed by the present invention significantly reduces the number of training steps required;

[0028] (2) In order to ensure semantic consistency while removing noise, the present invention integrates the edge information obtained from the image as a guiding condition throughout the training and sampling phases. This integration helps to preserve the structural details of the image;

[0029] (3) In order to further reduce the denoising steps and improve the perceptual quality, the present invention performs multiple denoising iterations, and uses the results of each iteration to provide edge and texture information for subsequent iterations. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flowchart of a method for image denoising based on a conditional diffusion model according to the present invention;

[0031] Figure 2 is a structural diagram of a conditional diffusion noise predictor;

[0032] Figure 3 is a display diagram of extracting the edge information of a noisy image during each iteration process;

[0033] Figure 4 is an effect diagram after multiple rounds of denoising; DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] The following describes the specific embodiments of the present invention with reference to the accompanying drawings, so that those skilled in the art can better understand the present invention. It should be particularly noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0035] Embodiment

[0036] Figure 1 It is a flowchart of an image denoising method based on a conditional diffusion model of the present invention.

[0037] In this embodiment, as Figure 1 shown, an image denoising method based on a conditional diffusion model of the present invention includes the following steps:

[0038] (1). Download the noisy image x from the database. We can first divide all pixel values of the noisy image x by 255 so that all pixel values of the noisy image x are scaled between 0 and 1.

[0039] (2). Extract the edge information of the noisy image x.

[0040] Input the noisy image x into the noisy image edge extraction module to extract the edge information edge of the noisy image x.

[0041] (3). Calculate the number of steps in the diffusion process of the noisy image x.

[0042] First, use the wavelet transform method to estimate the variance v of the noisy image x; then calculate the number of steps t in the diffusion process of the noisy image x through the following formula:

[0043] t = 100 * v / 255

[0044] (4). Denoise the noisy image x according to the trained conditional diffusion noise predictor D.

[0045] Set the diffusion step t of the conditional diffusion noise predictor D, then input the noisy image x into the trained conditional diffusion noise predictor D to obtain the output D(x, t, edge) of the conditional diffusion noise predictor D.

[0046] Obtain the denoised image x 0 :

[0047]

[0048] Among them, is a preset parameter, α i is the parameter preset at the i-th step of diffusion, where α i is a fixed arithmetic sequence of 100 numbers in total, ranging from 0.002 to 0.01;

[0049] In this embodiment, when performing image denoising based on the conditional diffusion model, the conditional diffusion noise predictor D will be used, and its structure is as shown in Figure 2; therefore, we need to first train the conditional diffusion noise predictor D using a large number of pictures in the picture library, and the specific training process is as follows:

[0050] 1), Download N clean images, and use the noise addition module to add random noise to the j-th clean image continuously for T times to obtain a series of noise samples The specific operations are as follows:

[0051]

[0052] where i = 1, 2, …, N, t = 1, 2, …, T, and the parameter α k is the parameter preset in the k-th iteration, and ε t is the Gaussian noise added in the t-th time and follows a distribution with a mean of 0 and a variance of 1;

[0053] 3), Input the noise samples into the conditional diffusion noise predictor D to predict the amount of noise i added to the i-th sample Z each time

[0054] 4), Calculate the value of the loss function after this round of iteration;

[0055]

[0056] 5), Update the parameters in the conditional diffusion noise predictor D by backpropagation of the loss value, and then perform the next round of training until the conditional diffusion noise predictor D converges.

[0057] (5), Denoise through multiple rounds of iteration;

[0058] (5.1), Set the number of denoising steps for iteration N = 5;

[0059] (5.2), Perform weighted fusion on the image x 0 and the noisy image x to obtain a new noisy image x 1 , that is, x 1 = 0.5 * x 0 + 0.5 * x, and the weight takes the value of 0.5 during weighted fusion;

[0060] (5.3), Replace the noisy image x with the noisy image x 1 , and then return to step (2);

[0061] (5.4), Repeat the denoising steps for iteration N times to obtain a clean image after multiple rounds of denoising.

[0062] Figure 3 is a display diagram of extracting the edge information of the noisy image during each round of iteration;

[0063] In this embodiment, as Figure 3As shown, iterations 0 to 4 represent the edge extraction effect diagrams obtained by the edge extraction module during the process from the 0th iteration to the 4th iteration. The Canny edge column represents the edge information of the clean image extracted by the Canny algorithm. By comparison, it can be seen that the edge extraction module can accurately extract the edge information of the noisy image.

[0064] Figure 4 are the effect diagrams after multiple rounds of denoising;

[0065] In this embodiment, as Figure 4 shown, in the figure, the OURs column is the result diagram after denoising by the present invention; the Noised column represents the noisy image to be denoised, and the Clean column represents the clean image corresponding to the noisy image. By comparing the three columns of images, it can be found that the present invention can well remove noise and obtain a clean image.

[0066] Although the above description of the illustrative specific embodiments of the present invention is provided for the convenience of those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concept of the present invention are within the scope of protection.

Claims

1. An image denoising method based on a conditional diffusion model, characterized in that: The following steps are involved: (1) Download the noisy image x from the database; (2) Extract edge information of the noisy image x; Input the noise image x to the noise image edge extraction module, so as to extract the edge information edge of the noise image x; (3) Calculate the number of steps of the noise image x in the diffusion process; First, use the wavelet transform method to estimate the variance v of the noise image x; Then the number of steps t of the noise image x in the diffusion process is calculated by the following formula; t=100*v / 255 (4) De-noising the noisy image x according to the trained conditional diffusion noise predictor D; Set the diffusion step number t of the conditional diffusion noise predictor D, and then input the noise image x into the trained conditional diffusion noise predictor D to obtain the output D(x, t, edge) of the conditional diffusion noise predictor D; Get the denoised image x0: in, is the preset parameter, α i is the preset parameter in the i-th step diffusion; (5) multiple rounds of iterative denoising; (5.1) Set the number of iterative denoising steps N; (5.2) Perform weighted fusion of image x0 and noise image x to obtain a new noise image x 1 ; (5.3) Using the noisy image x 1 Replace the noise image x, and then return to step (2); (5.4) Repeat the iterative denoising step N times to obtain a clean image after multiple rounds of denoising.

2. The image denoising method based on the conditional diffusion model according to claim 1, characterized in that: The pre-training process of the conditional diffuse noise predictor D is: (2.1) Download N clean images and use the noise module to denoise the jth clean image. Continuously add T times of random noise to obtain a series of noise samples The specific operations are as follows: Where i = 1, 2, ..., N, t = 1, 2, ..., T, and the parameters α k is the preset parameter for the kth iteration, ε t is the tth time Gaussian noise is added, and it follows a distribution with a mean of 0 and a variance of 1; (2.3), the noise sample Input to the conditional diffusion noise predictor D, predict each time added to the i-th sample Z i The amount of noise (2.4) Calculate the loss function value after this round of iteration; (2.5) The loss value is used to update the parameters in the conditional diffuse noise predictor D through back propagation, and then the next round of training is carried out until the conditional diffuse noise predictor D converges.

Images