Image noise removal method based on generalized diffusion model normal form

Abstract

The invention discloses an image noise removal method based on a generalized diffusion model normal form, and belongs to the technical field of image noise removal. The problems that an existing diffusion model cannot effectively deal with various types of noise and the efficiency of a step-by-step iterative sampling process is low are solved. The method comprises the following steps: processing an obtained original image data set without noise to obtain a training set image without noise; determining a primary function set required during diffusion model training according to the training set image; generating noise according to the primary function set, defining noise disturbance operation, and training a diffusion model according to the defined noise disturbance operation, the generated noise and the training set image; and inputting a to-be-denoised image into the trained diffusion model, and removing noise in the to-be-denoised image in a multi-step sampling mode to obtain a denoised image. The method can be used for removing the noise in the image.

Description

Technical Field

[0001] The present invention belongs to the technical field of image analysis, and particularly relates to an image noise removal method based on the paradigm of a generalized diffusion model. Background Art

[0002] The problem of image noise exists universally in various imaging systems. For example, in medical imaging, limitations of imaging devices can introduce noises such as metal artifacts and bias fields; in natural scenes, insufficient lighting or poor sensor performance may also lead to a decline in image quality. The noise problem in medical images may lead to a decrease in the accuracy of doctors' diagnoses, while the noise in natural scene images will affect the detection and recognition performance of the visual system.

[0003] Traditional image denoising methods are mainly based on filtering (such as mean filtering, median filtering) or model assumptions (such as Gaussian distribution or Poisson distribution). However, these methods often perform poorly when dealing with complex noise environments. In recent years, deep learning technology has provided new ideas for the problem of image denoising. Among them, the diffusion model has become an important method due to its generative modeling ability. However, existing diffusion models usually assume a specific noise distribution and cannot effectively handle various types of noises. At the same time, its step-by-step iterative sampling process leads to high computational overhead and low efficiency, which limits its application in actual scenarios. In summary, it is very necessary to propose a fast and accurate image denoising method for multiple noise types. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems that existing diffusion models cannot effectively handle various types of noises and the low efficiency of the step-by-step iterative sampling process, and to propose an image noise removal method based on the paradigm of a generalized diffusion model.

[0005] The technical solution adopted by the present invention to solve the above technical problems is:

[0006] An image noise removal method based on the paradigm of a generalized diffusion model, the method specifically includes the following steps:

[0007] Step S1, obtain an original image dataset without noise, and after processing the original image dataset, obtain a training set image without noise;

[0008] Step S2, determine the set of basis functions required for training the diffusion model according to the training set image;

[0009] Step S3, generate noise according to the set of basis functions, define a noise perturbation operation, and train the diffusion model according to the defined noise perturbation operation, the generated noise, and the training set image;

[0010] Step S4: Obtain the image to be denoised, input the image to be denoised into the trained diffusion model, and remove the noise in the image to be denoised through multi-step sampling to obtain the denoised image.

[0011] Further, in step S1, after processing the original image dataset, a training set image without noise is obtained; specifically:

[0012] For any image in the original image dataset, adjust the image to an image centered on the target area with a size of a×a; then map the pixel values in the adjusted image to the standardized range through linear transformation to obtain the processed image;

[0013] Similarly, process each image in the original image dataset separately, and use all the obtained processed images to form the training set image.

[0014] Further, the specific process of step S2 is as follows:

[0015] (1) When the noise type in the noise image corresponding to the training set image is smooth noise, for any image in the training set, the basis function set G generated by the image is:

[0016]

[0017] Where Generated by trigonometric functions, Generated by two-dimensional Legendre polynomials;

[0018]

[0019] S ω(x,y) ={cos(ωf(x,y,θ,φ)),sin(ωf(x,y,θ,φ))}

[0020] f(x,y,θ,φ)=(xcos(θ)+ysin(θ))sin(φ)

[0021] The angular variable θ < 180 and mod(θ,30)=0, the angular variable φ < 180 and mod(φ,30)=0, the units of θ and φ are both degrees, mod(θ,30) represents the remainder of θ divided by 30, mod(φ,30) represents the remainder of φ divided by 30, (x,y) represents the coordinates of the pixel in the image, ω is an integer, ω = 2,3,…,N 2 ;

[0022] Traverse all combinations of the values of θ, φ and ω, and generate for each combination. Denote all the generated Then are respectively mapped into a set range, and the corresponding mapping result is g i , where i = 1, 2,..., c;

[0023]

[0024] Among them, represents the minimum value in represents the maximum value in;

[0025]

[0026] P m,n (x, y) = P m (x)P n (y)

[0027] Among them, P m (x) represents the m-th Legendre polynomial, P n (x) represents the n-th Legendre polynomial, and both m and n are integers;

[0028] All the generated are respectively briefly recorded as g c+1 , g c+2 ,..., g C ;

[0029] Using g 1 , g 2 ,..., g c and g c+1 , g c+2 ,..., g c to form a basis function set G = {g 1 , g 2 ,..., g C}, and C is the number of elements in the basis function set G;

[0030] (2) When the noise type of the noise image corresponding to the training set image is other than smooth noise, for any image in the training set, the basis function set g corresponding to this image is the linear difference between this image and the noise image corresponding to this image. Similarly, the basis function sets of each image in the training set are obtained respectively.

[0031] Furthermore, the N 1 = 3, N 2 = 5.

[0032] Furthermore, the diffusion model is a 2D Unet network.

[0033] Further, generating noise according to the basis function set specifically includes:

[0034] (1) When the noise type in the noise image corresponding to the training set image is smooth noise, the generated noise is:

[0035]

[0036] where ∈ i is the Gaussian weight, and η is a hyperparameter that controls the randomness of the diffusion noise;

[0037] Use the generated noise as the noise corresponding to each image in the training set.

[0038] (2) When the noise type of the noise image corresponding to the training set image is other noise except smooth noise, for any image in the training set, the noise generated for this image using the basis function set of this image is:

[0039]

[0040] where ∈ is the Gaussian weight;

[0041] Similarly, generate noise for each image in the training set respectively.

[0042] Further, in step S3, define the noise perturbation operation as:

[0043]

[0044] where x 1 and x 2 represent the independent variables of the invertible function, and φ is the invertible function;

[0045]

[0046] where x represents the independent variable of the invertible function, and b is a scalar.

[0047] Further, training the diffusion model according to the defined noise perturbation operation, the generated noise, and the training set image specifically includes:

[0048] Denote any image in the training set as Let be φ(N), and define the generalized forward process of the diffusion model according to the defined noise perturbation operation and the generated noise as:

[0049]

[0050] where, β jis the j-th element in the noise schedule sequence β of length T, where t = 1, 2, …, T, denotes the image after noise addition at the t-th step;

[0051] Take and the time step t as the input of the diffusion model, and calculate the loss function L according to the predicted noise output by the diffusion model:

[0052]

[0053] where, denotes the noise predicted by the diffusion model after taking and the time step t as the input of the diffusion model; ‖·‖ 2 denotes calculating the 2-norm;

[0054] Fine-tune the parameters of the diffusion model according to the calculated loss function value, and finally obtain the trained diffusion model.

[0055] Furthermore, the specific process of step S4 is as follows:

[0056] Step S41: After adjusting the size of the image to be denoised, map the pixel values in the resized image to be denoised to the standardized range through a linear transformation to obtain the processed image to be denoised

[0057] Step S42: Initialize t = T;

[0058] Step S43: Take and t as the input of the trained diffusion model, then the noise prediction result output by the diffusion model is Take through the invertible function φ to obtain

[0059] Step S44: According to obtain the image after denoising at the t-th step

[0060]

[0061] Step S45: Judge whether t = 1 is satisfied;

[0062] If t = 1 is satisfied, then use the denoised image to continue to execute step S46;

[0063] If t = 1 is not satisfied, then set t = t - 1 and return to execute step S43;

[0064] Step S46: For the denoised image Inverse mapping is performed on each pixel value in it, and the inverse mapping result is used as the denoised image corresponding to the image to be denoised.

[0065] The beneficial effects of the present invention are as follows:

[0066] The image noise removal method based on the generalized diffusion model paradigm proposed by the present invention can explicitly remove the image noise caused by the limitations of imaging devices, illumination changes, etc. in the image, avoid the problem of implicit non-explainability in the process, directly complete various noise image denoising tasks, avoid the disadvantages brought by using Gaussian noise irrelevant to the task, and solve the theoretical limitation that only Gaussian noise can be used to gradually contaminate the image in the diffusion framework. In addition, by replacing the diffusion noise from Gaussian noise irrelevant to the task with the image noise to be removed in the task, the diffusion process changes from the diffusion process of other diffusion models (from the noisy image contaminated with Gaussian noise to the clean image) to the diffusion process of the present invention (from the image with image noise to the clean image), the image transformation distance in the whole diffusion process becomes smaller, the difficulty of image restoration is reduced, and the sampling efficiency is improved. The method of the present invention is verified in three tasks: MRI bias field correction, metal artifact removal in CT, and shadow removal in natural images, verifying that the method of the present invention can significantly improve the sampling efficiency and performance of the diffusion model in image restoration. Description of the Drawings

[0067] Figure 1 is a schematic diagram of the noise removal process of the method of the present invention;

[0068] Figure 2 is a flowchart of an image noise removal method based on the generalized diffusion model paradigm of the present invention;

[0069] Figure 3 is a schematic diagram of the application scope of the model of the present invention and the original diffusion model;

[0070] Figure 4 is a comparison diagram of the noise removal effects of the method of the present invention and other methods in the bias field correction task;

[0071] Figure 5 is a comparison diagram of the noise removal effects of the method of the present invention and other methods in the metal artifact removal task;

[0072] Figure 6 is a comparison diagram of the noise removal effects of the method of the present invention and other methods in the shadow removal task. Detailed Embodiments

[0073] Detailed Embodiment 1: Combine Figure 1 and Figure 2Describe this embodiment. An image noise removal method based on the generalized diffusion model paradigm described in this embodiment specifically includes the following steps:

[0074] Step S1: Obtain the original image dataset without noise (obtained from different imaging devices or public datasets). After processing the original image dataset, obtain the training set images without noise;

[0075] Step S2: Determine the set of basis functions required for training the diffusion model according to the training set images (a distribution with Gaussian characteristics and flexible noise characteristics can be generated according to the set of basis functions, which is used to simulate different types of image noise);

[0076] Step S3: Generate noise according to the set of basis functions, define the noise perturbation operation, and train the diffusion model according to the defined noise perturbation operation, the generated noise, and the training set images;

[0077] Step S4: Obtain the image to be denoised (i.e., an image with the same noise type as the noise image corresponding to the training set images), input the image to be denoised into the trained diffusion model, and remove the noise in the image to be denoised through multi-step sampling to obtain the image after removing the noise.

[0078] According to the actual denoising task, the present invention can obtain the training set images with the same noise type as in the denoising task. After training the model using the obtained training set, the trained model can be used for the actual denoising task. For example, the dataset uses a dataset with metal artifacts for the metal artifact removal task; the dataset can be sourced from CT scan images or other image data containing metal artifacts to simulate the image degradation problem in the actual scenario. According to the needs of the actual denoising task, the dataset can be replaced with other datasets related to image denoising tasks. From Figure 3 It can be seen that by setting hyperparameters, the original diffusion model using Gaussian noise is only a special case of the model of the present invention, and the applicable range of the model of the present invention is wider. From Figure 4 、 Figure 5 and Figure 6 It can be seen that the method of the present invention can remove the non-uniformity of the image gray distribution caused by limitations of imaging devices, light changes, etc., improve the picture clarity and the contrast between tissues in the picture. It shows that the method of the present invention can adapt to various noise distributions, can quickly and accurately remove various image noises, and solves the deficiencies of existing image denoising techniques in terms of sampling efficiency and image detail retention.

[0079] Specific Embodiment 2: The difference between this embodiment and Specific Embodiment 1 is that in step S1, after processing the original image dataset, the training set images without noise are obtained; specifically:

[0080] For any image in the original image dataset, adjust the image to an image centered on the target area with a size of a×a; then map the pixel values in the adjusted image to the standardized range (ensuring that the pixel values in the image are all within the range of [0,1]) through linear transformation to obtain the processed image;

[0081] Similarly, process each image in the original image dataset separately, and use all the obtained processed images to form the training set images.

[0082] Other steps and parameters are the same as those in the first specific implementation manner.

[0083] In this implementation manner, the value of a can be 256, but it is not limited to 256. Cropping according to the position information of the target area in the original image can avoid loss of key details. If the image resolution is too high or too low, bilinear interpolation or other methods can be used to scale the image to ensure that the cropped image can fully express the target features and meet the requirements of subsequent model training.

[0084] Specific implementation manner three: The difference between this implementation manner and the first or second specific implementation manner is that the specific process of step S2 is as follows:

[0085] (1) When the noise type in the noise image corresponding to the training set image (i.e., the noise type usually included when obtaining the image of the target in the training set) is smooth noise, for any image in the training set, the basis function set G generated by using this image is:

[0086]

[0087] Among them, Generated by trigonometric functions, Generated by two-dimensional Legendre polynomials;

[0088]

[0089] S ω(x,y) ={cos(ωf(x,y,θ,φ)),sin(ωf(x,y,θ,φ))}

[0090] f(x,y,θ,φ)=(xcos(θ)+ysin(θ))sin(φ)

[0091] The angular variable θ < 180 and mod(θ,30) = 0, the angular variable φ < 180 and mod(φ,30) = 0, the units of θ and φ are both degrees, mod(θ,30) represents the remainder of θ divided by 30, mod(φ,30) represents the remainder of φ divided by 30, (x,y) represents the coordinates of the pixel in the image, ω is an integer, ω = 2,3,…,N2 ;

[0092] Traverse all combinations of the values of θ, φ, and ω, and generate respectively according to each combination All the generated Are respectively abbreviated as Then Are respectively mapped to the set range, The corresponding mapping result is g i , i = 1, 2,..., c;

[0093]

[0094] Among them, Represents The minimum value in Represents The maximum value in;

[0095]

[0096] P m,n (x, y) = P m (x)P n (y)

[0097] Among them, P m (x) represents the m-th Legendre polynomial, P n (x) represents the n-th Legendre polynomial, and both m and n are integers;

[0098] All the generated Are respectively abbreviated as g c+1 , g c+2 ,..., g C ;

[0099] Use g 1 , g 2 ,..., g c And g c+1 , g c+2 ,..., g C To form a basis function set G = {g 1 , g 2 ,..., g C}, where C is the number of elements in the basis function set G; it should be noted that each image in the training set corresponds to the same basis function set G;

[0100] (2) When the noise type of the noise image corresponding to the training set image is other than smooth noise, for any image in the training set, the basis function set g corresponding to this image is the linear difference between this image and the noise image corresponding to this image (when the noise type is other than non-smooth noise, when obtaining the training set, the noise image corresponding to each image in the training set can be obtained simultaneously for generating the basis function set). Similarly, the basis function sets of each image in the training set are obtained respectively.

[0101] Other steps and parameters are the same as those in the first or second specific implementation manner.

[0102] That is, when the image noise is not smooth noise and the noise of all images in the training set cannot be represented by the same basis function set G, the model will handle it by setting a unique element g for each image pair (clean and noisy images) in the basis function set G. The element g is the linear difference between the images, that is, the pixels at the corresponding positions in the two images are subtracted. Therefore, the training data corresponds to multiple sets of basis function sets. One basis function set corresponds to one diffusion process, so it is equivalent to training multiple diffusion processes during the training process.

[0103] Specific implementation manner four: The difference between this implementation manner and one of the first to third specific implementation manners is that the N 1 = 3, N 2 = 5.

[0104] Other steps and parameters are the same as those in one of the first to third specific implementation manners.

[0105] Specific implementation manner five: The difference between this implementation manner and one of the first to fourth specific implementation manners is that the diffusion model is a 2D Unet network.

[0106] Other steps and parameters are the same as those in one of the first to fourth specific implementation manners.

[0107] Specific implementation manner six: The difference between this implementation manner and one of the first to fifth specific implementation manners is that generating noise according to the basis function set is specifically:

[0108] (1) When the noise type in the noise image corresponding to the training set image is smooth noise (smooth noise refers to a noise image with a slow intensity change in space and has a smooth change characteristic, such as a bias field), the generated noise is:

[0109]

[0110] where ∈ i is the Gaussian weight, η is a hyperparameter controlling the randomness of the diffusion noise, represents a Gaussian distribution with a mean of 0 and a variance of 1;

[0111] Use the generated noise as the noise corresponding to each image in the training set;

[0112] Specifically, when the hyperparameter η is large enough, the diffusion noise N is approximately deterministic (cold diffusion), otherwise the diffusion noise is random (hot diffusion), and the model establishes the connection between hot diffusion and cold diffusion. The original diffusion noise is extended from following the unit standard normal distribution to a multivariate Gaussian distribution, and the covariance matrix of the noise distribution is changed by adjusting the set of basis functions, so as to flexibly control the noise characteristics, enabling it to simulate different types of image noise and laying a foundation for subsequent diffusion modeling.

[0113] (2) When the noise type of the noise image corresponding to the training set image is other than smooth noise, for any image in the training set, the noise generated for this image using the basis function set of this image is:

[0114]

[0115] where ∈ is the Gaussian weight;

[0116] Similarly, noise is generated for each image in the training set (the generated noise and the image are used for subsequent training together, that is, for any image, the generalized forward process is carried out using this image and the noise corresponding to this image).

[0117] Other steps and parameters are the same as those in any one of the first to fifth specific embodiments.

[0118] When the noise type in the image is the bias field, the value of η is 0, and when the noise type in the image is other types, the value of η is 10.

[0119] Specific embodiment seven: The difference between this embodiment and any one of the first to sixth specific embodiments is that in step S3, the noise perturbation operation is defined as:

[0120]

[0121] where x 1 and x 2 represent the independent variables of the invertible function, and φ is the invertible function;

[0122]

[0123] where x represents the independent variable of the invertible function, and b is a scalar.

[0124] Other steps and parameters are the same as those in any one of the first to sixth specific embodiments.

[0125] For additive noise, the invertible function is the identity function; for multiplicative noise, the invertible function is the exponential function. By defining a new way of noise influence through the invertible function, complex noise distributions in real scenarios can be simulated, and the invertible function is used to ensure that the noise process can be efficiently restored during the reverse sampling process.

[0126] Specific Embodiment VIII: The difference between this embodiment and any one of Embodiments I to VII is that training the diffusion model using the defined noise perturbation operation, the generated noise, and the training set images is specifically as follows:

[0127] Denote any image in the training set as Let be φ(x t ), be φ(N), and define the generalized forward process of the diffusion model according to the defined noise perturbation operation and the generated noise as:

[0128]

[0129] where, β j is the j-th element in the noise schedule sequence β of length T, t = 1, 2,..., T, represents the image after being noise-added in the t-th step;

[0130] The elements in the linear sequence β are successively the left endpoint of the interval [0.0001, 0.03], T - 2 equally divided points within the interval [0.0001, 0.03], and the right endpoint of the interval [0.0001, 0.03]. The T - 2 equally divided points within the interval divide the entire interval into T - 1 equal parts, and the value of each equally divided point is accurate to ten decimal places. In the present invention, the total number of diffusion steps T is set to 100, and the total number of diffusion steps and the noise schedule can be adjusted according to actual needs and the characteristics of the data set.

[0131] The generalized forward process adopts a step-by-step degradation strategy to generate images with different degradation degrees by multi-step diffusion of high-quality images. Among them, t is the number of steps, used to control the degradation degree of each step.

[0132] Take and the time step t as the input of the diffusion model, and calculate the loss function L according to the predicted noise output by the diffusion model:

[0133]

[0134] where, represents the noise predicted by the diffusion model after inputting and the time step t into the diffusion model; ‖·‖ 2Indicates the calculation of the 2-norm;

[0135] Fine-tune the parameters of the diffusion model according to the calculated loss function value (in the present invention, the optimizer used to optimize the network parameters is the Nadam optimizer, and other optimizers can also be used instead), and finally obtain a trained diffusion model.

[0136] Other steps and parameters are the same as those in any one of the specific embodiments 1 to 7.

[0137] In the present invention, the establishment process of the loss function is as follows:

[0138] After inputting the training set images into the diffusion network, simulate the gradual degradation process of the input images under different noise types by gradually adding noise to obtain contaminated images.

[0139] Ordinary diffusion models can only be applied to noises that follow a unit Gaussian distribution and are not applicable to actual various types of diffusion noises. The present invention proposes to use variational inference of a multivariate Gaussian distribution to optimize the restoration model. Variational inference maximizes the variational lower bound:

[0140]

[0141] Among them, C is a constant term. Define J σ as the negative value of the variational inference target, that is Use '≈' because only most numerical terms are retained in the formula derivation process, and several special terms are omitted for subsequent calculation convenience. It is known that follows a multivariate Gaussian distribution where μ is the mean, is the covariance, where K is the dimension of x t The model distribution p θ (x t-1 |x t )(obtained from ) fits the data distribution q σ (x t-1 |x t ,x 0 ) by optimizing the variational lower bound. Among them N θ is the predicted diffusion noise. Then, after calculating the KL divergence of the multivariate Gaussian distribution in the above formula, the following formula is obtained:

[0142]

[0143] To simplify the covariance complexity, set to 1, set Σ -1 to 1, and further simplify the training objective to:

[0144]

[0145] After obtaining the predicted N θ it is equivalent to obtaining that is, φ(N θ ), because φ is known, the loss function is defined according to the above formula as:

[0146]

[0147] The diffusion noise model and noise perturbation method of the present invention enable the forward process to diffuse any noise, and the reverse stage can remove any noise through the sampling process and variational inference optimization.

[0148] According to actual needs, the loss function can be optimized for the characteristics of different data sets to optimize the denoising effect. For example, in order to balance the artifact intensity between the metal and non-metal domains, weighted MSE can be used to optimize the loss function, and the optimized loss function is:

[0149]

[0150] where m is the metal mask,

[0151] Specific Embodiment Nine: The difference between this embodiment and any one of Embodiments One to Eight is that the specific process of step S4 is as follows:

[0152] Step S41: After adjusting the size of the image to be denoised, map each pixel value in the resized image to be denoised to the standardized range through linear transformation to obtain the processed image to be denoised

[0153] Step S42: Initialize t = T;

[0154] Step S43: Take and t as the input of the trained diffusion model, then the noise prediction result output by the diffusion model is Take through the invertible function φ to obtain

[0155] Step S44: According to obtain the image after denoising at the t-th step

[0156]

[0157] Step S45: Determine whether t = 1 is satisfied;

[0158] If t = 1 is satisfied, use the denoised image to continue to execute step S46;

[0159] If \(t = 1\) is not satisfied, then let \(t=t - 1\), and return to execute step S43;

[0160] Step S46, for the denoised image Perform inverse mapping on each pixel value in it, and use the inverse mapping result as the denoised image corresponding to the image to be denoised.

[0161] Other steps and parameters are the same as those in any one of the first to eighth specific embodiments.

[0162] According to the generalized forward process of the diffusion model, the reverse process can be deduced:

[0163]

[0164] From To The sampling process is a one-step inverse process from To where \(t = 1,2,\cdots,T\). Referring to the denoising diffusion implicit model DDIM, set \(\sigma\) t To 0, and the reverse stage becomes deterministic.

[0165] According to the above formula, we get:

[0166]

[0167] This embodiment adopts a denoising strategy based on variational inference, and gradually removes the diffusion noise in the image to be denoised through multi-step sampling to obtain the denoised image.

[0168] Example

[0169] Step S1, obtain the original magnetic resonance image with bias field from the Human Connectome Project (HCP), crop or scale the original picture into an image with a unified size of \(256\times256\), perform normalization processing on the picture, and normalize the pixel values to \([0,1]\) to obtain the original bias field data set. After preprocessing, randomly select 2206 slices for training and validation, and use 1000 slices for testing;

[0170] Step S2, select an image from the training set in step S1 as the basis function set required for network training. Set the basis function set \(G\) to include low-order Legendre polynomials and slowly varying trigonometric functions to satisfy the smooth property of the diffusion noise. Set the intermediate variable \(\eta\) to zero.

[0171] Generate a distribution with Gaussian characteristics and flexible noise characteristics to simulate different types of image noise:

[0172]

[0173] Let \(P_i(x)\) i represent the \(i\)-th Legendre polynomial. The two-dimensional Legendre polynomial is \(P_{i,j}(x,y)=P_i(x)P_j(y)\). Using the two-dimensional Legendre polynomial as the basis function, m,n (x,y) = P m (x)P n (y). Among them, \(N\) is a hyperparameter. Functions with \(n\lt2\) are excluded because they change too slowly. The set of basis functions is: 1 where the ranges of both the \(x\) and \(y\) coordinates are \([-1,1]\).

[0174]

[0175] Each basis function in is linearly mapped to the range centered at \([0.9,1.1]\) to achieve uniform fairness. Set \(N_i = 3\), \(N_j = 5\); 1 = 3, N 2 = 5;

[0176] Step S3: Determine the influence mode of the noise, expand diverse noise perturbation methods, and define a new noise perturbation operation using a reversible function. Let \(\varphi\) be an exponential function in the real number field such that the noise perturbation operation is a multiplication operation;

[0177] Define the generalized forward process of the diffusion model. The total number of diffusion steps \(T\) is set to 100, and the noise schedule \(\beta\) t increases linearly from 0.0001 to 0.03. Input the original image into the network, and by gradually adding noise, simulate the gradual degradation process of the input image under different noise types to obtain a contaminated image;

[0178] Adopt a denoising strategy based on variational inference, and gradually remove the diffusion noise from the obtained damaged image through multi-step sampling to obtain the predicted clean image. During the training process, calculate the loss:

[0179]

[0180] Use \(L(\theta)\) to supervise the model training until the set maximum number of iterations is reached. The number of training epochs is set to 800, and the number of training iterations can be adjusted according to the actual application scenario and computing resources. When there is sufficient training time, the maximum number of iterations can be appropriately increased to further improve the performance, while in real-time processing tasks, the number of iterations can be appropriately reduced to speed up the model inference speed. The Adam optimizer with an adaptive learning rate optimization method is used, and the momentum parameters are set to \((0.9,0.999)\). The initial learning rate is set to \(2\times10\) N (-4). -5, the decay factor is 0.6. If the performance on the validation set does not improve within 25 consecutive epochs, the learning rate will decay. Exponential Moving Average (EMA) is used with a decay rate of 0.9999.

[0181] Step S4: Obtain the image with the bias field to be removed, input the image with the bias field to be removed into the trained diffusion model, and remove the bias field in the image through multi-step sampling to obtain the image after removing the bias field.

[0182] The above examples of the present invention are only for illustrating in detail the calculation model and calculation process of the present invention, rather than limiting the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is impossible to enumerate all the implementation manners here. Any obvious changes or modifications derived from the technical solutions of the present invention still fall within the protection scope of the present invention.

Claims

1. A method for removing image noise based on a generalized diffusion model paradigm, characterized in that: The method specifically comprises the following steps: Step S1, obtaining an original image data set without noise, and processing the original image data set to obtain a training set image without noise; Step S2, determining a basis function set required for diffusion model training according to the training set images; Step S3, generating noise according to the basis function set, and defining a noise disturbance operation, and training the diffusion model according to the defined noise disturbance operation, the generated noise, and the training set image; Step S4, obtaining an image to be denoised, inputting the image to be denoised into a trained diffusion model, removing noise from the image to be denoised by multi-step sampling, and obtaining an image after denoising.

2. The image noise removal method based on the generalized diffusion model paradigm according to claim 1, characterized in that: In step S1, after processing the original image data set, a training set image without noise is obtained; specifically: For any image in the original image data set, adjust the image into an image with a size of a×a centered on the target area; then map each pixel value in the adjusted image to a standardized range through linear transformation to obtain a processed image; Similarly, each image in the original image data set is processed separately, and all the processed images obtained are used to form a training set image.

3. The image noise removal method based on the generalized diffusion model paradigm according to claim 1, characterized in that: The specific process of step S2 is: (1) When the noise type in the noise image corresponding to the training set image is smooth noise, for any image in the training set, the basis function set G generated by using the image is: in, Generated by trigonometric functions, Generated by two-dimensional Legendre polynomials; S ω(x,y) ={cos(ωf(x,y,θ,φ)),sin(ωf(x,y,θ,φ))} f(x,y,θ,φ)=(x cos(θ)+y sin(θ))sin(φ) The angle variable θ is less than 180 and mod(θ,30) is equal to 0, the angle variable φ is less than 180 and mod(φ,30) is equal to 0, the units of θ and φ are both degrees, mod(θ,30) represents the remainder of θ divided by 30, mod(φ,30) represents the remainder of φ divided by 30, (x,y) represents the coordinates of the pixel in the image, ω is an integer, ω=2,3,…,N2; Traverse all combinations of θ, φ and ω values, and generate All generated They are respectively Then Mapped to the set range respectively, The corresponding mapping result is g i , i=1,2,...,c; in, express The minimum value in express The maximum value in ; P m,n (x,y)=P m (x)P n (y) Among them, P m (x) represents the mth Legendre polynomial, P n (x) represents the nth Legendre polynomial, m and n are both integers; All generated They are abbreviated as g c+1 ,g c+2 ,...,g C ; Using g1,f2,...,g c and c+1 ,g c+2 ,...,g C The basis function set G = {g1, g2, ..., g C }, C is the number of elements in the basis function set G; (2) When the noise type of the noise image corresponding to the training set image is other than smooth noise, for any image in the training set, the basis function set g corresponding to the image is the linear difference between the image and the noise image corresponding to the image. Similarly, the basis function set of each image in the training set is obtained.

4. The image noise removal method based on the generalized diffusion model paradigm according to claim 3 is characterized in that: Said N1=3, N2=5.

5. The image noise removal method based on the generalized diffusion model paradigm according to claim 1, characterized in that: The diffusion model is a 2D Unet network.

6. The image noise removal method based on the generalized diffusion model paradigm according to claim 3, characterized in that: The noise is generated according to the basis function set, specifically: (1) When the noise type in the noise image corresponding to the training set image is smooth noise, the generated noise is: Among them, ∈ i is the Gaussian weight, η is a hyperparameter that controls the randomness of the diffusion noise; The generated noise is used as the noise corresponding to each image in the training set; (2) When the noise type of the noise image corresponding to the training set image is other noise except smooth noise, for any image in the training set, the noise generated for the image using the basis function set of the image is: Among them, ∈ is the Gaussian weight; Similarly, noise is generated for each image in the training set.

7. The image noise removal method based on the generalized diffusion model paradigm according to claim 6, characterized in that: In step S3, the noise disturbance operation is defined as: Among them, x1 and x2 represent the independent variables of the reversible function, and φ is a reversible function; Among them, x represents the independent variable of the reversible function, and b is a scalar.

8. The method for removing image noise based on the generalized diffusion model paradigm according to claim 7, characterized in that: The diffusion model is trained according to the defined noise perturbation operation, the generated noise and the training set image, specifically: Any image in the training set is denoted as make For φ(N), the generalized forward process of the diffusion model defined by the noise perturbation operation and the generated noise is: in, β j is the jth element in the noise scheduling sequence β of length T, t = 1, 2, ..., T, express The image after the t-th step of noise addition; Will and time step t as the input of the diffusion model, and the loss function L is calculated based on the predicted noise output by the diffusion model: in, Indicates that The noise predicted by the diffusion model after the time step t is input into the diffusion model; ‖·‖2 means calculating the 2-norm; The parameters of the diffusion model are fine-tuned according to the calculated loss function value, and finally a trained diffusion model is obtained.

9. The image noise removal method based on the generalized diffusion model paradigm according to claim 8, characterized in that: The specific process of step S4 is as follows: Step S41: after resizing the image to be denoised, map each pixel value in the resized image to be denoised to a standardized range through linear transformation to obtain a processed image to be denoised. Step S42, initializing t=T; Step S43: and t as the input of the trained diffusion model, the noise prediction result output by the diffusion model is Will After the reversible function φ, we get Step S44: Get the image after denoising in step t Step S45, determine whether t=1 is satisfied; If t=1, the denoised image is used. Continue to execute step S46; If t=1 is not satisfied, set t=t-1 and return to step S43; Step S46: De-noising the image The pixel values ​​in are inversely mapped, and the inverse mapping result is used as the denoised image corresponding to the image to be denoised.

Images