Image deblurring method, system and device based on robust diffusion posterior sampling
By constructing a robust diffusion posterior sampling image deblurring method, and utilizing improvements to the noise perturbation conditional distribution and likelihood score function, the problem of non-robust noise perturbation estimation in existing technologies is solved, achieving higher quality image deblurring results and improving the accuracy and efficiency of image restoration.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NAT UNIV OF DEFENSE TECH
- Filing Date
- 2023-12-25
- Publication Date
- 2026-07-14
AI Technical Summary
Existing image deblurring methods are not robust when estimating the conditional distribution of noise perturbations, resulting in low computational efficiency and biased generation results. Furthermore, the approximation method of the likelihood score function is inaccurate, affecting the image deblurring effect.
A robust diffusion posterior sampling-based image deblurring method is constructed. By acquiring the observed image with additive noise, the transfer matrix, the posterior expectation, and the posterior covariance, the noise perturbation conditional distribution function and the likelihood score function are constructed. Then, the variance sequence of Gaussian noise is estimated using a neural network, and iterative denoising is performed using the diffusion denoising probability model to improve image quality.
It improves the quality of deblurred images by enhancing computational efficiency and image restoration performance through an improved likelihood score function and noise perturbation conditional distribution estimation. This is manifested in higher peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM), as well as lower Frechet initiation distance (FID) and learned perceptual patch similarity distance (LPIPS).
Smart Images
Figure CN117611493B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to an image deblurring method, system and device based on robust diffusion posterior sampling. Background Technology
[0002] Image deblurring can be formulated as a linear inverse problem, and its forward model can be denoted as: y = Ax + n. It is a known transfer matrix. It is additive noise, and the goal of solving the linear inverse problem is to extract information from noisy observations. Estimating unknown signals The difficulty in solving the linear inverse problem lies in its ill-posedness, thus requiring the addition of reliable prior information to constrain the solution space. Inspired by the powerful generative capabilities of diffusion-based denoising probabilistic models, a deep generative prior is incorporated when solving the linear inverse problem. This can help overcome ill-posed problems. It combines known forward models. Sampling can then be performed from the posterior distribution p(x|y). The sampling at time step t in the diffusion process is called the diffusion posterior sample p(x|y). t |y). According to Bayes' theorem, the key to diffusion posterior sampling is estimating the conditional distribution p(y|x) of the noise perturbation. t Since y,x t Regarding the conditional independence of the original signal x0, this term can be written as p(y|x t )=∫p(y|x0)p(x0|x t )dx0, p(y|x0) can be derived from the forward model, p(x0|x t The estimation of ) is the key to estimating the distribution of noise disturbance conditions.
[0003] Existing estimation methods, such as DPS, will equate p(x0|x) to p(x0|x) t The distribution is simplified to a single point, discarding the randomness of x0. DMPS directly uses the uninformative prior of x0 to approximate the estimate of p(x0|x). t This is a simple but biased estimate. It depends on the condition x. t of The approximation method used in calculating the likelihood score function affects both computational efficiency and the final result.
[0004] Therefore, for p(x0|x) t Robust improvements in estimation and the associated likelihood score function Approximate improvements are necessary. Summary of the Invention
[0005] The purpose of this invention is to provide an image deblurring method, system, and device based on robust diffusion posterior sampling, which improves the quality of the deblurred image.
[0006] To achieve the above objectives, the present invention provides the following solution:
[0007] An image deblurring method based on robust diffusion posterior sampling includes:
[0008] Obtain the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, and the value of the posterior covariance;
[0009] Construct the noise perturbation conditional distribution function and the likelihood score function in the diffusion denoising probability model; the noise perturbation conditional distribution function is a function of the observed image with additive noise, the deblurred observed image, the transfer matrix, the posterior expectation, and the posterior covariance, and the likelihood score function is a function of the transfer matrix and the posterior expectation;
[0010] Based on the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, the value of the posterior covariance, the conditional distribution function of the noise disturbance, and the likelihood score function, an approximate value of the likelihood score function is determined.
[0011] Substituting the approximate value of the likelihood score function into the diffusion denoising probability model, the observed image with additive noise is denoised for a preset number of iterations in the diffusion denoising probability model denoising process to obtain the corresponding deblurred observed image.
[0012] Optionally, the process of obtaining the posterior expected value includes:
[0013] A neural network for obtaining the variance sequence of Gaussian noise and estimating the score function during the noise addition process of a diffusion denoising probability model;
[0014] The posterior expectation is determined by a neural network based on the variance sequence of Gaussian noise during the noise addition process and the estimated score function in a diffusion denoising probability model.
[0015] Optionally, the process of obtaining the value of the posterior covariance includes:
[0016] A neural network for obtaining the variance sequence of Gaussian noise and estimating the score function during the noise addition process of a diffusion denoising probability model;
[0017] The value of the posterior covariance is determined by a neural network based on the variance sequence of Gaussian noise during the noise addition process and the estimated score function in a diffusion denoising probability model.
[0018] Optionally, the process of constructing the noise disturbance conditional distribution function includes:
[0019] Define the expression for p(y|x0):
[0020] Where p(y|x0) is the distribution of the observed image y given the deblurred image x0; For the observed image with additive noise, C is the number of channels, m1 is the width of the observed image, m2 is the height of the observed image; A is the transfer matrix; The image is the deblurred observed image; σ is the variance of the additive noise following a normal distribution; I is the identity matrix; It follows a normal distribution;
[0021] Given p(x0|x t The expression for the estimate of ) is:
[0022] Where p(x0|x t Let x be the image after deblurring at step t in a given diffusion denoising probability model denoising process. t The distribution of the final blurred image x0 obtained under the given conditions; e is the posterior expectation; C is the posterior covariance; x t In the diffusion denoising probability model, the noisy image at step t is diffused starting from x0;
[0023] The expression based on p(y|x0) and p(x0|x t The expression for the estimated noise disturbance conditional distribution function is used to determine the noise disturbance conditional distribution function.
[0024] Optionally, the noise disturbance conditional distribution function is:
[0025]
[0026] Where p(y|x) t Let x be the image after deblurring at step t in a given diffusion denoising probability model denoising process. t The distribution of the observed image y under the given conditions; A T This is the transpose of the transfer matrix.
[0027] Optionally, the likelihood score function is:
[0028]
[0029] Where L = logp(y|x) t ); α t =1-β t ,β t The variance sequence of Gaussian noise during the noise addition process for the diffusion denoising probability model The variance of the Gaussian noise at step t; U and V are both orthogonal matrices, VT Let V denote the transpose of V, and Λ be a diagonal matrix.
[0030] An image deblurring system based on robust diffusion posterior sampling includes:
[0031] The parameter acquisition module is used to acquire the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, and the value of the posterior covariance.
[0032] The function construction module is used to construct the noise perturbation conditional distribution function and the likelihood score function in the diffusion denoising probability model; the noise perturbation conditional distribution function is a function of the observed image with additive noise, the deblurred observed image, the transfer matrix, the posterior expectation, and the posterior covariance; the likelihood score function is a function of the transfer matrix and the posterior expectation.
[0033] The likelihood score function approximation determination module is used to determine the approximate value of the likelihood score function based on the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, the value of the posterior covariance, the conditional distribution function of the noise disturbance, and the likelihood score function.
[0034] The denoising module is used to substitute the approximate value of the likelihood score function into the diffusion denoising probability model, and to perform denoising on the observed image with additive noise for a preset number of iterations of the diffusion denoising probability model denoising process to obtain the corresponding deblurred observed image.
[0035] An apparatus includes a memory and a processor, the memory storing a computer program, the processor running the computer program to cause the apparatus to perform the image deblurring method based on robust diffusion posterior sampling as described above.
[0036] Optionally, the memory is a readable storage medium.
[0037] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:
[0038] This invention discloses an image deblurring method, system, and device based on robust diffusion posterior sampling. First, it acquires the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, and the value of the posterior covariance. Second, it constructs the noise perturbation conditional distribution function and the likelihood score function in the diffusion denoising probability model. The noise perturbation conditional distribution function is a function of the observed image with additive noise, the deblurred observed image, the transfer matrix, the posterior expectation, and the posterior covariance; the likelihood score function is a function of the transfer matrix and the posterior expectation. Third, based on the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, the value of the posterior covariance, the noise perturbation conditional distribution function, and the likelihood score function, it determines an approximate value of the likelihood score function. Finally, it substitutes the approximate value of the likelihood score function into the diffusion denoising probability model and performs denoising on the observed image with additive noise for a preset number of iterations of the diffusion denoising probability model denoising process, obtaining the corresponding deblurred observed image, thus improving the quality of the deblurred image. Attached Figure Description
[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0040] Figure 1 This is a schematic diagram of the image deblurring method based on robust diffusion posterior sampling provided in Embodiment 1 of the present invention;
[0041] Figure 2 Here is a flowchart illustrating the specific process of an image deblurring method based on robust diffusion posterior sampling;
[0042] Figure 3 This is the original ground truth image of the roof;
[0043] Figure 4 This is the original truth image of the toy car;
[0044] Figure 5 A blurred image of the roof;
[0045] Figure 6 A blurry image of a toy car;
[0046] Figure 7 To obtain a deblurred roof image using the method described in this invention;
[0047] Figure 8 To obtain a deblurred toy car image using the method of this invention;
[0048] Figure 9The image of the rooftop after deblurring using DPS;
[0049] Figure 10 The image of the toy car after deblurring using DPS;
[0050] Figure 11 The image shows the rooftop after deblurring using DMPS.
[0051] Figure 12 The image shows a toy car image after deblurring using DMPS. Detailed Implementation
[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0053] The purpose of this invention is to provide an image deblurring method, system, and device based on robust diffusion posterior sampling, which aims to improve the quality of the deblurred image.
[0054] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0055] Example 1
[0056] Figure 1 This is a schematic diagram of the image deblurring method based on robust diffusion posterior sampling provided in Embodiment 1 of the present invention. Figure 1 As shown, the image deblurring method based on robust diffusion posterior sampling in this embodiment includes:
[0057] Step 101: Obtain the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, and the value of the posterior covariance.
[0058] As an optional implementation, the process of obtaining the posterior expected value includes:
[0059] A neural network is used to obtain the variance sequence of Gaussian noise and estimate the score function during the noise addition process of a diffusion denoising probability model.
[0060] The posterior expectation is determined by a neural network based on the variance sequence of Gaussian noise during the noise addition process and the estimated score function in a diffusion denoising probability model.
[0061] As an optional implementation, the process of obtaining the value of the posterior covariance includes:
[0062] A neural network is used to obtain the variance sequence of Gaussian noise and estimate the score function during the noise addition process of a diffusion denoising probability model.
[0063] The value of the posterior covariance is determined by a neural network based on the variance sequence of Gaussian noise during the noise addition process and the estimated score function in a diffusion denoising probability model.
[0064] Step 102: Construct the noise perturbation conditional distribution function and likelihood score function in the diffusion denoising probability model; the noise perturbation conditional distribution function is a function of the observed image with additive noise, the deblurred observed image, the transfer matrix, the posterior expectation, and the posterior covariance, and the likelihood score function is a function of the transfer matrix and the posterior expectation.
[0065] As an optional implementation, the process of constructing the noise perturbation conditional distribution function includes:
[0066] Define the expression for p(y|x0):
[0067] Where p(y|x0) is the distribution of the observed image y given the deblurred image x0; For the observed image with additive noise, C is the number of channels, m1 is the width of the observed image, m2 is the height of the observed image; A is the transfer matrix; The image is the deblurred observed image; σ is the variance of the additive noise following a normal distribution; I is the identity matrix; It follows a normal distribution.
[0068] Given p(x0|x t The expression for the estimate of ) is:
[0069] Where p(x0|x t Let x be the image after deblurring at step t in a given diffusion denoising probability model denoising process. t The distribution of the final blurred image x0 obtained under the given conditions; e is the posterior expectation; C is the posterior covariance; x t This is used to diffuse the noisy image at step t, starting from x0, in the diffusion denoising probability model.
[0070] The expression based on p(y|x0) and p(x0|x t The estimated expression of ) is used to determine the conditional distribution function of noise disturbance.
[0071] As an optional implementation method, the noise disturbance conditional distribution function is:
[0072]
[0073] Where p(y|x)t Let x be the image after deblurring at step t in a given diffusion denoising probability model denoising process. t The distribution of the observed image y under the given conditions; A T This is the transpose of the transfer matrix.
[0074] As an optional implementation, the likelihood score function is:
[0075]
[0076] Where L = logp(y|x) t ); α t =1-β t ,β t The variance sequence of Gaussian noise during the noise addition process for the diffusion denoising probability model The variance of the Gaussian noise at step t; U and V are both orthogonal matrices, V T Let V denote the transpose of V, and Λ be a diagonal matrix.
[0077] Step 103: Based on the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, the value of the posterior covariance, the conditional distribution function of the noise disturbance, and the likelihood score function, determine the approximate value of the likelihood score function.
[0078] Step 104: Substitute the approximate value of the likelihood score function into the diffusion denoising probability model, and perform denoising on the observed image with additive noise for a preset number of iterations of the diffusion denoising probability model denoising process to obtain the corresponding deblurred observed image.
[0079] Specifically, such as Figure 2 The image deblurring method is shown below, and includes the following steps:
[0080] S1: Conditional distribution estimation of noise disturbance.
[0081] S1.1: Give the expression for p(y|x0).
[0082] For the linear inverse problem of image deblurring, y and x0 come from the forward model y = Ax0 + n. For the observed image with additive noise, C is the number of channels, m1 is the width of the observed image, m2 is the height of the observed image; A is the transfer matrix; Let x0 be the original signal corresponding to the deblurred observed image (x0 is the estimated original signal; the blurred image obtained by rearranging the x0 vector into a matrix is the deblurred observed image), and n be the additive noise in the forward model, where it is assumed that n follows a normal distribution. σ is the variance of the normal distribution, and I is the identity matrix. Therefore, the expression for p(y|x0) is:
[0083]
[0084] S1.2: Give p(x0|x t (New estimate).
[0085] According to Tweedie's formula, p(x0|x t The posterior expectation e and posterior covariance C of ) can be expressed as:
[0086]
[0087]
[0088] Where, x t In the diffusion denoising probability model, the noisy image at step t is diffused starting from x0; s θ (x t (t) is the estimated score function Neural networks; The variance sequence of Gaussian noise during the noise addition process of the diffusion denoising probability model; T is the total number of diffusion steps in the diffusion denoising probability model; It is a Jacobian matrix.
[0089] Given the distribution p(x0|x) of the posterior expectation e and the posterior covariance C. t The optimal estimate of ) follows a normal distribution in the sense of the maximum entropy principle, and has the following:
[0090]
[0091] in, Let g be a normal distribution set; g is a normal distribution within the normal distribution set. Let p(x0|x) be the KL divergence. Therefore, we can obtain p(x0|x) t Approximate estimate of ):
[0092] S1.3: Provide an estimate of the conditional distribution of noise disturbance.
[0093] according to and Combine p(y|x) t )=∫p(y|x0)p(x0|x t )dx0, for the conditional distribution of noise disturbance p(y|x t The estimate is:
[0094] S2: Provides an approximation of the likelihood score function.
[0095] S1 gives the conditional distribution of noise disturbance p(y|x) tThe estimate of ) is nonlinear and depends on condition x. t This provides a basis for calculating the likelihood score function. This presents difficulties, so in S2, an approximation of the likelihood score function is given.
[0096] Let L = logp(y|x) t If ), then the likelihood score function It can be written as:
[0097]
[0098] Where μ=Ae,∑ i,j Let Σ be the element in the i-th row and j-th column of matrix ∑, where Σ = ACA T +σ 2 I, for For each term i,j = 1, 2, ..., n, taking the derivative with respect to each component, and letting k = 1, 2, ..., n, we have:
[0099]
[0100] Among them, a i and a j These are the i-th and j-th rows of A, respectively; Let's consider the Jacobian matrix of the variables within the parentheses. (Considering computational complexity,) Involving s θ (x t The derivative term of (t) is difficult to implement using neural networks, therefore let Right now To further simplify, in C Since the term is approximately 0, C can be approximated as:
[0101]
[0102] Then, A is decomposed using SVD as follows: A = UΛV T U and V are both orthogonal matrices, V T Let V be the transpose of matrix V, and Λ be a diagonal matrix. It can be calculated using the following formula:
[0103]
[0104] Among them, Λ 2 It is the square of the element in Λ.
[0105] S3: Solve the linear inverse problem of image deblurring.
[0106] The likelihood score function is obtained through S2. After approximating the expression, the linear inverse problem of image deblurring is solved iteratively using a diffusion denoising probability model. The specific steps are as follows:
[0107] S3.1: Given the relevant parameters for solving the linear inverse problem based on the diffusion denoising probability model.
[0108] When using the diffusion denoising probabilistic model to solve the linear inverse problem of image deblurring, the quantities required are: the noisy observed image y, the transfer matrix A (in the image deblurring problem, the transfer matrix is obtained by expanding the blur kernel), and the score function pre-trained by the unconditional diffusion denoising probabilistic model (DDPM). The variance sequence of Gaussian noise during the noise addition process in the diffusion denoising probabilistic model The variances σ and α of additive noise n t =1-β t , and the variance sequence of the noise in the reverse diffusion process From the formula Provided.
[0109] S3.2: Iteratively solve to generate the deblurred image.
[0110] Based on the steps of the diffusion denoising probability model, the initialization of t=T iterative process is as follows:
[0111] 1.
[0112] 2. Update x t-1 ,
[0113] 3. If t = t - 1, continue with step 1 if t ≠ 0; if t = 0, obtain x0.
[0114] in, This is a preliminary estimate of the deblurred image after step t-1; x t-1 η represents the final estimate of the deblurred image after step t-1; t , where is the Gaussian noise term at step t. γ is the weighting coefficient, representing the importance of the posterior likelihood term; The result is obtained by calculating using the formula derived from step S2.
[0115] Finally, x0 is obtained as the original signal to be estimated. Rearranging the x0 vector into a matrix gives the deblurred image, i.e., the deblurred observation image. Specific Implementation
[0117] To verify the advantages of the present invention, specific embodiments were further studied.
[0118] In this specific embodiment, let the original truth image be as follows: Figure 3 and Figure 4 As shown, the image includes a rooftop image and a toy car image; the blurred image is as follows. Figure 5 and Figure 6 As shown, the pixel values of the 256×256 RGB image are normalized to the range [0, 1]. Two blur kernels are set: a Gaussian kernel of size 61×61 with a standard deviation of 3.0 and a uniform kernel of size 9×9. The transfer matrix A in the forward process is obtained by unfolding the operation of the blur kernel with the image into matrix multiplication, where a Gaussian kernel is added to the roof image and a uniform kernel is added to the toy car image. The noise n is set to additive white Gaussian noise with σ = 0.05.
[0119] The specific steps for image deblurring using the robust posterior diffusion sampling-based image deblurring method of this invention are as follows:
[0120] S1: Conditional distribution estimation of noise disturbance.
[0121]
[0122] Where e is p(x0|x t The posterior expectation of ) C is p(x0|x t The posterior covariance of ) s θ (x t (t) is the estimated score function Neural networks. α t =1-β t , Let T be the variance sequence of Gaussian noise during the noise addition process of the diffusion denoising probability model, and let T be the total number of diffusion steps of the diffusion denoising probability model. Let z be a normal distribution with mean a and variance b.
[0123] S2: Provides an approximation of the likelihood score function.
[0124]
[0125] Among them, A is obtained by SVD decomposition as A = UΛV T U and V are orthogonal matrices, V T Let V be the transpose of matrix V, and Λ be a diagonal matrix. 2 It is the square of the elements in Λ.
[0126] S3: Solve the linear inverse problem of image deblurring.
[0127] S3.1: Given the relevant parameters for solving the linear inverse problem based on the diffusion denoising probability model.
[0128] y is the vectorized form of the noisy image, and A is obtained by expanding the blur kernel into matrix multiplication. It is a pre-trained score function of the unconditional diffusion denoising probability model, T=1000, and the variance sequence of Gaussian noise during the noise addition process of the diffusion denoising probability model. Defined as a 1000-step numerical sequence with equal intervals from 0.0001 to 0.02, let α t =1-β t , From the formula Given that γ = 0.5.
[0129] S3. Iteratively solve to generate the deblurred image.
[0130] Iteration from t=T to t=0:
[0131] 1.
[0132] 2. Update x t-1 ,
[0133] The final result, x0, is the original signal to be estimated. Rearranging x0 into an image yields the desired deblurred image.
[0134] The above steps are used to deblur the blurred image, and the deblurring result is as follows: Figure 7 and Figure 8 As shown, the comparison methods are DPS and DMPS methods, and the deblurring results of roof images with added Gaussian blur kernels are compared. Figure 9 and Figure 11 As shown, the deblurring result of a toy car image with a uniform blur kernel is compared to, for example... Figure 10 and Figure 12 As shown in Tables 1 and 2, to quantitatively evaluate the performance of the above method, standard metrics for image restoration and perceptual scores of the generated images are combined. The standard metrics include peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM), while the perceptual scores mainly refer to Frechet initiation distance (FID) and learned perceptual patch similarity distance (LPIPS). The results are shown in Tables 1 and 2. The results show that the method proposed in this invention (i.e., RDPS) is superior to the comparison methods in terms of both image quality improvement and image fidelity.
[0135] Table 1. Quantitative Results of Gaussian Kernel Deblurring
[0136] method PSNR (dB) SSIM FID LPIPS DPS 22.16 0.63 61.06 0.34 DMPS 22.24 0.71 55.28 0.33 RDPS 22.62 0.74 52.73 0.30
[0137] Table 2 Quantitative Results of Uniform Kernel Defuzzification
[0138] method PSNR (dB) SSIM FID LPIPS DPS 23.51 0.63 62.57 0.36 DMPS 24.14 0.76 58.04 0.35 RDPS 24.66 0.75 55.62 0.33
[0139] Example 2
[0140] The robust diffusion posterior sampling-based image deblurring system in this embodiment includes:
[0141] The parameter acquisition module is used to acquire the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, and the value of the posterior covariance.
[0142] The function building module is used to construct the noise perturbation conditional distribution function and the likelihood score function in the diffusion denoising probability model. The noise perturbation conditional distribution function is a function of the observed image with additive noise, the deblurred observed image, the transfer matrix, the posterior expectation, and the posterior covariance. The likelihood score function is a function of the transfer matrix and the posterior expectation.
[0143] The likelihood score function approximation determination module is used to determine the approximate value of the likelihood score function based on the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, the value of the posterior covariance, the noise perturbation conditional distribution function, and the likelihood score function.
[0144] The denoising module is used to substitute the approximate value of the likelihood score function into the diffusion denoising probability model, and to perform denoising on the observed image with additive noise for a preset number of iterations of the diffusion denoising probability model denoising process, so as to obtain the corresponding deblurred observed image.
[0145] Example 3
[0146] An apparatus includes a memory and a processor, the memory storing a computer program and the processor running the computer program to cause the apparatus to perform the robust diffusion posterior sampling-based image deblurring method of Embodiment 1.
[0147] As an optional implementation, the memory is a readable storage medium.
[0148] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.
[0149] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. An image deblurring method based on robust diffusion posterior sampling, characterized in that, The method includes: Obtain the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, and the value of the posterior covariance; Construct the noise perturbation conditional distribution function and the likelihood score function in the diffusion denoising probability model; the noise perturbation conditional distribution function is a function of the observed image with additive noise, the deblurred observed image, the transfer matrix, the posterior expectation, and the posterior covariance, and the likelihood score function is a function of the transfer matrix and the posterior expectation; Based on the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, the value of the posterior covariance, the conditional distribution function of the noise disturbance, and the likelihood score function, an approximate value of the likelihood score function is determined. Substitute the approximate value of the likelihood score function into the diffusion denoising probability model, and perform denoising on the observed image with additive noise for a preset number of iterations of the diffusion denoising probability model denoising process to obtain the corresponding deblurred observed image. The noise disturbance conditional distribution function is: ； in, Given the image after deblurring at step t in the diffusion denoising probability model denoising process. Observational images under certain conditions Distribution; It follows a normal distribution; For the transfer matrix; For posterior expectation; For posterior covariance; This is the transpose of the transfer matrix; The variance of the additive noise following a normal distribution; It is the identity matrix; The likelihood score function is: ； in, ; , The variance sequence of Gaussian noise during the noise addition process for the diffusion denoising probability model The variance of the Gaussian noise at step t in the expression; and All are orthogonal matrices. express transpose, It is a diagonal matrix.
2. The image deblurring method based on robust diffusion posterior sampling according to claim 1, characterized in that, The process of obtaining the posterior expected value includes: A neural network for obtaining the variance sequence of Gaussian noise and estimating the score function during the noise addition process of a diffusion denoising probability model; The posterior expectation is determined by a neural network based on the variance sequence of Gaussian noise during the noise addition process and the estimated score function in a diffusion denoising probability model.
3. The image deblurring method based on robust diffusion posterior sampling according to claim 1, characterized in that, The process of obtaining the value of the posterior covariance includes: A neural network for obtaining the variance sequence of Gaussian noise and estimating the score function during the noise addition process of a diffusion denoising probability model; The value of the posterior covariance is determined by a neural network based on the variance sequence of Gaussian noise during the noise addition process and the estimated score function in a diffusion denoising probability model.
4. The image deblurring method based on robust diffusion posterior sampling according to claim 1, characterized in that, The process of constructing the noise disturbance conditional distribution function includes: set up The expression: ; in, To perform a deblurring on a given image Observational images under certain conditions Distribution; For observation images with additive noise, For the number of channels, To observe the width of the image, To observe the height of the image; The image is the deblurred observation image; Give The expression for the estimate is: ; in, Given the image after deblurring at step t in the diffusion denoising probability model denoising process. The final blurred image obtained under the conditions Distribution; In the diffusion denoising probability model, Starting point for diffusion A noisy image of the step; based on expressions and The estimated expression is used to determine the conditional distribution function of the noise disturbance.
5. An image deblurring system based on robust diffusion posterior sampling, characterized in that, The system includes: The parameter acquisition module is used to acquire the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, and the value of the posterior covariance. The function construction module is used to construct the noise perturbation conditional distribution function and the likelihood score function in the diffusion denoising probability model; the noise perturbation conditional distribution function is a function of the observed image with additive noise, the deblurred observed image, the transfer matrix, the posterior expectation, and the posterior covariance; the likelihood score function is a function of the transfer matrix and the posterior expectation. The likelihood score function approximation determination module is used to determine the approximate value of the likelihood score function based on the observed image with additive noise, the value of the transfer matrix, the value of the posterior expectation, the value of the posterior covariance, the conditional distribution function of the noise disturbance, and the likelihood score function. The denoising module is used to substitute the approximate value of the likelihood score function into the diffusion denoising probability model, and to denoise the observed image with additive noise by performing a preset number of iterations of the diffusion denoising probability model denoising process to obtain the corresponding deblurred observed image. The noise disturbance conditional distribution function is: ； in, Given the image after deblurring at step t in the diffusion denoising probability model denoising process. Observational images under certain conditions Distribution; It follows a normal distribution; For the transfer matrix; For posterior expectation; For posterior covariance; This is the transpose of the transfer matrix; The variance of the additive noise following a normal distribution; It is the identity matrix; The likelihood score function is: ； in, ; , The variance sequence of Gaussian noise during the noise addition process for the diffusion denoising probability model The variance of the Gaussian noise at step t in the expression; and All are orthogonal matrices. express transpose, It is a diagonal matrix.
6. A device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the device to perform the image deblurring method based on robust diffusion posterior sampling as described in any one of claims 1 to 4.
7. The device according to claim 6, characterized in that, The memory is a readable storage medium.