A diffusion image restoration method based on robust fidelity steering

Abstract

This invention relates to the field of image processing and computational imaging technology, and discloses a diffusion image restoration method based on robust fidelity guidance. The invention acquires an initial image and its corresponding degradation operator, simultaneously initializes a diffusion model, performs image estimation and time step prediction on the current initial image based on the diffusion model, obtains an estimated image and a predicted image of the current initial image, calculates the residual between the two images, constructs a robust influence function, calculates a robust gradient term based on this influence function, and then calculates the back-projection guidance gradient. The robust gradient term and the back-projection guidance gradient are weighted and fused to obtain a hybrid guidance gradient. The estimated image of the initial image is updated based on the hybrid guidance gradient, and the image for the next time step is predicted. This process constitutes one iteration, iterating until the final restored image meets the standard.

Description

Technical Field

[0001] This invention relates to the field of image processing and computational imaging technology, specifically to a robust fidelity-guided diffusion image restoration method. Background Technology

[0002] Image restoration is a core problem in computer vision and computational imaging, aiming to reconstruct high-quality images from degraded observations. The degradation process can typically be modeled as follows:

[0003]

[0004] in, These are observed degraded images. This represents a degenerate operator (such as a fuzzy kernel, a downsampling matrix, etc.). It is a clear image to be recovered. It is additive noise. Image restoration tasks include, but are not limited to, deblurring, super-resolution, denoising, and deraining.

[0005] In recent years, methods based on denoising diffusion probability models (DDPMs) have achieved significant success in image generation and restoration tasks. These methods reconstruct sharp images from random noise through a stepwise denoising process, and are particularly effective in restoring high-perceptual-quality images.

[0006] To introduce observation data constraints into the diffusion sampling process, existing methods typically incorporate data consistency guidance mechanisms during backdiffusion to ensure that the generated results match the observed images. Currently, mainstream diffusion guidance frameworks generally employ a least-squares data fidelity term based on the L2 norm as the consistency constraint, assuming that the observation noise follows a Gaussian distribution. Under this assumption, data consistency control is achieved by calculating the residual gradient and performing backprojection or least-squares updates.

[0007] However, the aforementioned L2 fidelity terms have significant limitations: L2 loss is highly sensitive to outliers, and in the presence of non-Gaussian degradation such as salt-and-pepper noise, impulse noise, or mixed noise, it will impose excessive gradient penalties on outlier pixels; outlier residuals will cause the gradient magnitude to amplify rapidly, thereby causing oscillations or even numerical instability in the diffusion sampling process; under high proportion of non-Gaussian noise conditions, L2 guidance is prone to over-smoothing or structural distortion; existing methods usually do not effectively control the gradient magnitude and lack theoretical guarantees of gradient boundedness.

[0008] In real-world applications, imaging noise is often not strictly Gaussian. For example, impulse noise caused by sensor defects, mixed noise introduced by compressed transmission, and non-uniform interference in complex environments all fall under the category of non-Gaussian degradation. Therefore, diffusion-guided mechanisms that rely solely on the L2 fidelity term show a significant decrease in recovery performance under real-world complex degradation environments.

[0009] Furthermore, with the widespread application of diffusion models in high-resolution image restoration, the requirements for guidance stability and robustness have become increasingly stringent. If the gradient update process becomes unstable due to outliers, it not only affects the restoration quality but may also lead to slow convergence or even divergence in the sampling process. Therefore, how to construct a robust data consistency guidance mechanism in diffusion image restoration that can suppress outlier residuals, has bounded gradient magnitudes, and maintains computational efficiency has become an urgent technical problem to be solved. Summary of the Invention

[0010] To address the shortcomings of existing technologies, this invention provides a robust fidelity-guided diffusion image restoration method, which has the advantages of high image restoration efficiency and high robustness, thus solving the aforementioned technical problems.

[0011] To achieve the above objectives, the present invention provides the following technical solution: a robust fidelity-guided diffusion image restoration method, comprising the following steps:

[0012] S1: Obtain the initial image to be processed. and its corresponding degenerate operators And initialize the diffusion model parameters, including the total number of diffusion steps. Noise scheduling parameters Pre-trained denoising network and regularization parameters ;

[0013] S2: Based on the diffusion model, perform image estimation and time step prediction on the current initialized image to obtain the estimated image of the current initialized image. and predicted images And calculate the predicted image With initial image residual ;

[0014] S3: Constructing a robust influence function based on residuals The robust gradient term is calculated based on the robust influence function. ;

[0015] S4: Based on robust gradient terms The back-projected guided gradient was calculated. ;

[0016] S5: Time-Step Scheduling Weights The robust gradient term obtained from S3 Guided gradient obtained by back projection from S4 Weighted fusion is performed to obtain a hybrid guided gradient. ;

[0017] S6: Based on Hybrid Guided Gradient Update the estimated image of the currently initialized image and predict the image of the next time step based on the diffusion model;

[0018] S7: Iterate through S2 and S6, then output the final restored image. .

[0019] As a preferred embodiment of the present invention, the calculation formula for the S2 image estimation is as follows:

[0020]

[0021] in, An estimated image representing the state at the current time step; For degenerate operators; Indicates transpose; For noise scheduling parameters; Standard Gaussian noise; Initialize the image for the current task;

[0022] The expression for the S2 time step prediction is as follows:

[0023] in, Indicates at time step Under the given conditions, the target sharpness image estimation result is obtained by the pre-trained denoising network, where the subscript 0 indicates the time step corresponding to the target sharpness image, and t indicates that the prediction result is based on the current time step. status get; Let represent the denoising prediction function in the diffusion model, used to determine the state at the current time step. Predict the corresponding sharp image estimate; This indicates that the diffusion model is at time step [missing information]. The current estimated image at that time; Denotes the time step in the backsampling process of the diffusion model. in, This represents the total number of diffusion steps.

[0024] S2 calculates the predicted image. With initial image residual The expression is as follows:

[0025] in, The residual between the predicted image and the estimated image is used to initialize the image.

[0026] As a preferred technical solution of the present invention, the S3 robust influence function The expression is as follows:

[0027]

[0028] in, This represents the residual between the initialized image, the predicted image, and the estimated image. This represents the residual gradient after robust constraints.

[0029] The S3 robust gradient term The expression is as follows:

[0030]

[0031] in, For degenerate operators; This indicates transpose.

[0032] As a preferred embodiment of the present invention, the S4 back projection guides the gradient. The expression is as follows:

[0033]

[0034] in, For degenerate operators; Indicates transpose; For regularization parameters; Represents the identity matrix; For time step Under the given conditions, the result of estimating a sharp image of the target obtained by the pre-trained denoising network; Initialize the current image.

[0035] As a preferred technical solution of the present invention, the S5 hybrid guided gradient The expression is as follows:

[0036]

[0037] in, For time step scheduling weights, ; Guide the gradient for back projection; This is the robust gradient term.

[0038] As a preferred embodiment of the present invention, S6 is based on a hybrid guided gradient. The expression for updating the estimated image of the currently initialized image is as follows:

[0039]

[0040] in, Indicates at time step Below is a clear image after robust guided gradient correction; Indicates time step Under the given conditions, the result of estimating a sharp image of the target obtained by the pre-trained denoising network; This is the step size parameter; For hybrid guided gradients;

[0041] The expression for S6, which predicts the image at the next time step based on the diffusion model, is as follows:

[0042]

[0043] in, This represents the state image at the next time step during the back diffusion process; Indicates time step The corresponding cumulative noise scheduling coefficient; Indicates at time step Below is a clear image after robust guided gradient correction; Indicates time step Gaussian random noise obtained by downsampling.

[0044] As a preferred embodiment of the present invention, S7 ultimately restores the image. Specifically: when time step The image corresponding to the time step, the expression for the time step is: , This represents the total number of diffusion steps.

[0045] Compared with existing technologies, this invention provides a robust fidelity-guided diffusion image restoration method, which has the following advantages:

[0046] 1. This invention introduces a robust influence function to replace the traditional L2 norm data fidelity term, nonlinearly truncating or compressing the residuals, effectively suppressing the excessive influence of abnormal residuals on gradient updates. Compared with existing L2-based guided methods, this invention avoids gradient explosion, making the backdiffusion sampling process more stable and less prone to oscillations or numerical divergence.

[0047] 2. This invention, through the weighted fusion of robust gradient terms and back-projection guided gradients, can adaptively suppress the interference of outliers on data consistency, recover clearer textures and edges, and avoid over-smoothing. Simultaneously, under Gaussian noise conditions, the robust influence function of this invention can degenerate into an approximately linear response, thus maintaining performance comparable to existing L2 guided methods and possessing good adaptability across all scenarios.

[0048] 3. This invention, by modifying the guided gradient, does not require changing the main structure of the pre-trained diffusion model. It only requires replacing the original L2-fidelity gradient with a hybrid form of robust gradient and backprojection gradient during the sampling process. Therefore, this invention can be easily integrated into existing diffusion image restoration frameworks as a plug-and-play enhancement module, without requiring retraining or fine-tuning of the model.

[0049] 4. The robust influence function used in this invention involves only basic arithmetic operations and simple conditional judgments, without introducing additional iterative optimization or large-scale matrix inversion operations. Compared with traditional L2 guidance, the computational overhead of this invention is almost negligible, yet it brings significant improvements in robustness and restoration quality, making it particularly suitable for real-time restoration of high-resolution images and possessing extremely high practical value. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Please see Figure 1 A robust fidelity-guided diffusion image restoration method includes the following steps:

[0053] S1: Obtain the initial image to be processed. and its corresponding degenerate operators And initialize the diffusion model parameters, including the total number of diffusion steps. Noise scheduling parameters Pre-trained denoising network and regularization parameters ;

[0054] S2: Based on the diffusion model, perform image estimation and time step prediction on the current initialized image to obtain the initial image of the current initialized image. and predicted images And calculate the predicted image With estimated image residual ;

[0055] The formula for image estimation is as follows:

[0056]

[0057] in, An estimated image representing the state at the current time step; For degenerate operators; Indicates transpose; For noise scheduling parameters; Standard Gaussian noise; Initialize the image for the current task;

[0058] The expression for time-step prediction is as follows:

[0059] in, Indicates at time step Under the given conditions, the target sharpness image estimation result is obtained by the pre-trained denoising network, where the subscript 0 indicates the time step corresponding to the target sharpness image; t indicates that the prediction result is based on the current time step. status get; Let represent the denoising prediction function in the diffusion model, used to determine the state at the current time step. Predict the corresponding sharp image estimate; This indicates that the diffusion model is at time step [missing information]. The current estimated image at that time is an intermediate state image containing noise; Let denote the time step in the backsampling process of the diffusion model, satisfying: in, Total diffusion steps; time steps The larger the value, the stronger the noise component in the image; as the backdiffusion process proceeds... As the image gradually decreases, it gradually returns to a clear state.

[0060] Calculate the predicted image With initial image residual The expression is as follows:

[0061] in, The residual between the predicted image and the estimated image is used to initialize the image;

[0062] S3: Constructing a robust influence function based on residuals The robust gradient term is calculated based on the robust influence function. ;

[0063] Robust influence function The expression is as follows:

[0064]

[0065] in, This represents the residual between the initialized image, the predicted image, and the estimated image. This represents the residual gradient after robust constraints;

[0066] Robust gradient term The expression is as follows:

[0067]

[0068] in, For degenerate operators; Indicates transpose;

[0069] S4: Based on robust gradient terms The back-projected guided gradient was calculated. Its expression is as follows:

[0070]

[0071] in, For degenerate operators; Indicates transpose; For regularization parameters; This represents the identity matrix, used to ensure the numerical stability of the matrix inversion process; For time step Under the given conditions, the result of estimating a sharp image of the target obtained by the pre-trained denoising network; Initialize the image for the current task;

[0072] S5: Time-Step Scheduling Weights The robust gradient term obtained from S3 Guided gradient obtained by back projection from S4 Weighted fusion is performed to obtain a hybrid guided gradient. Its expression is as follows:

[0073]

[0074] in, For time step scheduling weights, It can change with time step; Guide the gradient for back projection; For robust gradient terms;

[0075] S6: Based on Hybrid Guided Gradient Update the estimated image of the currently initialized image and predict the image of the next time step based on the diffusion model;

[0076] Based on hybrid guided gradient The expression for updating the estimated image of the currently initialized image is as follows:

[0077]

[0078] in, Indicates at time step Below is a clear image after robust guided gradient correction; Indicates time step Under the given conditions, the result of estimating a sharp image of the target obtained by the pre-trained denoising network; This is the step size parameter; For hybrid guided gradients;

[0079] The expression for predicting the image at the next time step based on the diffusion model is as follows:

[0080]

[0081] in, This represents the state image at the next time step during the back diffusion process; Indicates time step The corresponding cumulative noise scheduling coefficient; Indicates at time step Below is a clear image after robust guided gradient correction; Indicates time step Gaussian random noise obtained by downsampling;

[0082] S7: Iterate through S2 and S6, then output the final restored image. The image specifically shows: when time step The image corresponding to the time step, the expression for the time step is: , This represents the total number of diffusion steps.

[0083] The specific implementation method is as follows:

[0084] To verify the effectiveness of the image restoration method in this embodiment, experiments were conducted on the commonly used CelebA-HQ dataset, which contains 1000 high-resolution 1024×1024 face images. In the experiment, the images were first cropped to 256×256 patches and normalized. A pre-trained diffusion model was used as the backbone. Image restoration was performed through 100 iterations from steps S2 to S6. Finally, the restored result was converted back to the [0,255] range, and an evaluation metric was calculated to obtain the restored image.

[0085] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A robust fidelity-guided diffusion image restoration method, characterized in that: Includes the following steps: S1: Obtain the initial image to be processed. and its corresponding degenerate operators And initialize the diffusion model parameters, including the total number of diffusion steps. Noise scheduling parameters Pre-trained denoising network and regularization parameters ; S2: Based on the diffusion model, perform image estimation and time step prediction on the current initialized image to obtain the estimated image of the current initialized image. and predicted images And calculate the predicted image With initial image residual ; S3: Constructing a robust influence function based on residuals The robust gradient term is calculated based on the robust influence function. ; S4: Based on robust gradient terms The back-projected guided gradient was calculated. ; S5: Time-Step Scheduling Weights The robust gradient term obtained from S3 Guided gradient obtained by back projection from S4 Weighted fusion is performed to obtain a hybrid guided gradient. ; S6: Based on Hybrid Guided Gradient Update the estimated image of the currently initialized image and predict the image of the next time step based on the diffusion model; S7: Iterate through S2 and S6, then output the final restored image. .

2. The diffusion image restoration method based on robust fidelity guidance according to claim 1, characterized in that: The calculation formula for the S2 image estimation is as follows: in, An estimated image representing the state at the current time step; For degenerate operators; Indicates transpose; For noise scheduling parameters; Standard Gaussian noise; Initialize the image for the current task; The expression for the S2 time step prediction is as follows: in, Indicates at time step Under the given conditions, the target sharpness image estimation result is obtained by the pre-trained denoising network, where the subscript 0 indicates the time step corresponding to the target sharpness image, and t indicates that the prediction result is based on the current time step. status get; Let represent the denoising prediction function in the diffusion model, used to determine the state at the current time step. Predict the corresponding sharp image estimate; This indicates that the diffusion model is at time step [missing information]. The current estimated image at that time; Denotes the time step in the backsampling process of the diffusion model. in, This represents the total number of diffusion steps. S2 calculates the predicted image. Initialize image residual The expression is as follows: in, The residual between the predicted image and the estimated image is used to initialize the image.

3. The diffusion image restoration method based on robust fidelity guidance according to claim 2, characterized in that: The S3 robust influence function The expression is as follows: in, This represents the residual between the initialized image, the predicted image, and the estimated image. This represents the residual gradient after robust constraints. The S3 robust gradient term The expression is as follows: in, For degenerate operators; This indicates transpose.

4. The diffusion image restoration method based on robust fidelity guidance according to claim 3, characterized in that: The S4 back projection guides the gradient. The expression is as follows: in, For degenerate operators; Indicates transpose; For regularization parameters; Represents the identity matrix; For time step Under the given conditions, the result of estimating a sharp image of the target obtained by the pre-trained denoising network; Initialize the current image.

5. The diffusion image restoration method based on robust fidelity guidance according to claim 4, characterized in that: The S5 hybrid guided gradient The expression is as follows: in, For time step scheduling weights, ; Guide the gradient for back projection; This is the robust gradient term.

6. The diffusion image restoration method based on robust fidelity guidance according to claim 5, characterized in that: The S6 is based on a hybrid guided gradient. The expression for updating the estimated image of the currently initialized image is as follows: in, Indicates at time step Below is a clear image after robust guided gradient correction; Indicates time step Under the given conditions, the result of estimating a sharp image of the target obtained by the pre-trained denoising network; This is the step size parameter; For hybrid guided gradients; The expression for S6, which predicts the image at the next time step based on the diffusion model, is as follows: in, This represents the state image at the next time step during the back diffusion process; Indicates time step The corresponding cumulative noise scheduling coefficient; Indicates at time step Below is a clear image after robust guided gradient correction; Indicates time step Gaussian random noise obtained by downsampling.

7. The diffusion image restoration method based on robust fidelity guidance according to claim 6, characterized in that: The S7 finally restores the image. Specifically: when time step The image corresponding to the time step, the expression for the time step is: , This represents the total number of diffusion steps.

Images