Gaussian relative operator dual discriminant image blind deblurring method

Abstract

The present invention belongs to the field of artificial intelligence and discloses a Gaussian relative operator dual-discriminative image blind deblurring method, comprising S1. designing an image prior based on the dual-discriminative principle; S2. substituting the image prior into an optimization model for blind image deblurring, and using an alternating iterative algorithm to solve the clear image and blur kernel respectively, transforming the complex intermediate image estimation into a simple and intuitive image filtering problem; S3. employing non-blind restoration to obtain the final clear image; and S4. conducting a large number of numerical experiments on simulated benchmark data and real blurred images of multiple scenes, including natural, artificial, text, nighttime, and portrait scenes, and adjusting intermediate parameters, ultimately obtaining an effective and robust image blind deblurring method for natural images. The present invention can effectively improve the restoration effect of blurred images and is more theoretically feasible than traditional gradient-based image prior deblurring.

Description

Technical Field

[0001] The invention belongs to the field of artificial intelligence, and in particular relates to a Gaussian relative operator dual discriminant image blind deblurring method. Background Art

[0002] As a crucial medium for acquiring, storing, and transmitting information, images are now ubiquitous in intelligent driving, security monitoring, joystick detection, astronomical exploration, smart healthcare, and other fields, playing an irreplaceable role in every area of ​​modern society. In the information age, all human activities are closely related to images.

[0003] However, images can be affected by various factors during their generation, acquisition, transmission, and storage, resulting in the loss of partial image information and degradation of image quality, directly impacting the accuracy of information read by humans or machines. In medicine, medical professionals cannot accurately diagnose conditions based on blurred images; in traffic, traffic police cannot accurately obtain relevant vehicle information from blurred images captured by electronic cameras. Therefore, how to restore high-quality images from blurred and degraded images to ensure accurate information acquisition has become a crucial issue. This problem has therefore become a fundamental issue in the field of computer vision and a cutting-edge issue in computational imaging.

[0004] Image deblurring is a classic, highly ill-posed problem that presents significant challenges. Broadly speaking, image deblurring can be categorized into blind deblurring and non-blind deblurring. The former involves a known blur kernel when restoring the image, while the latter does not. In real life, due to the complexity of the relative motion between the imaging device and the object, the image's blur function (i.e., point spread function) cannot be easily obtained. Therefore, blind deblurring is more practical and universally applicable.

[0005] Specifically, current representative image deblurring problems are mostly based on probabilistic statistics methods, mainly including variational Bayesian (VB) methods, maximum a posterior (MAP) methods, and edge prediction-based methods.

[0006] With the rapid development of deep learning, data-driven methods for blind deblurring have become the latest approach in computer vision. However, the generalization capability of existing CNN-based blind deblurring methods is far from sufficient to handle blur in different imaging scenarios. Summary of the Invention

[0007] To further improve the theoretical and practical feasibility of the current blind image deblurring problem, this paper proposes a blind image deblurring method based on the Gaussian relative operator. By introducing an operator used in image filtering into the traditional blind image deblurring problem, it serves as an image prior operator, thus providing dual discriminative properties. This method can effectively improve the restoration of blurred images and is more theoretically feasible than traditional gradient-based prior image deblurring methods.

[0008] In order to achieve the above object, the present invention is achieved through the following technical solutions:

[0009] The present invention is a Gaussian relative operator dual discriminant image blind deblurring method, comprising the following steps:

[0010] S1. Design image priors based on the principle of dual discriminability;

[0011] Will As an image prior for the model of this application.

[0012] Among them, for the original stσ1<σ2 can be used to distinguish strong edges from weak textures. Because weak textures have more diverse directions and are easier to cancel out, while strong edges have more gradients with similar directions, when gradients are superimposed, the values ​​obtained for the edge gradients of large-scale features are often larger than those obtained for weak textures. Small-scale features contain almost all gradients. Clearly, for this prior, weak textures are given larger values ​​than strong edges. During the minimization iteration process, the final image converges towards strong edges, erasing the less important weak textures.

[0013] For the updated exist Based on the introduction of The purpose of the weight term is to make the image prior satisfy the ability to identify strong edges and weak textures while also having the ability to identify clear images and blurred images. Its ability to distinguish between clear and blurred images can be demonstrated by changes. For a clear image, this means that the ratio of the result of convolving the blur kernel with the gradient of the clear image to the gradient of the clear image has a much smaller numerator than the denominator, and the result is obviously less than 1.

[0014] If the blurred image is regarded as the convolution of the clear image and a blur kernel, then It can be expressed as:

[0015]

[0016] For blurred images, since the similarity of their gradients is lower, it is assumed that the gradient of an image and the gradient of the blurred image after convolution with the same blur kernel are not much different. Then the value of formula (1) always tends to 1. Therefore, we have:

[0017]

[0018] That is, in this prior In the , the value of the clear image is often smaller than that of the blurred image. In the process of minimizing the model iteration, the final image will approach the direction of the clear image.

[0019] above, The image prior designed by the present invention based on the dual discriminability principle needs to be introduced into the image blind deblurring model to perform image deblurring in the next step.

[0020] S2. Substitute the image prior into the optimization model to blur the blind area of ​​the image, and use the alternating iterative algorithm to solve the clear image and blur kernel respectively, transforming the complex intermediate image estimation into a simple and intuitive image filtering problem.

[0021] (i) Bringing image priors into the model

[0022] For the image blind area blur problem, it is assumed that the blur is spatially invariant. Due to the high degree of ill-posedness of the image deblurring problem, traditional algorithms often use the following mathematical model to model the image degradation process, namely:

[0023] b=u*k+n (3)

[0024] Where b represents the blurred image; u represents the clear image to be solved; k is the blur kernel, also known as the point spread function; n is the additive noise, usually Gaussian random noise; * represents the convolution operator. From the model point of view, the image deblurring problem is a deconvolution problem.

[0025] The image prior of this application needs to be combined with the maximum a posteriori MAP method under the traditional method, and the solution formula is:

[0026]

[0027] Among them, α and β are non-negative weights, To encode the degree of deviation of the image s relative to the original image b, that is, the restored intermediate image s and the blurred image b should be consistent in the sense of convolution, and the prior constraint of the blur kernel k adopts the simplest norm, a model can be constructed to solve the problem.

[0028] (ii) Decomposition of the problem to be solved

[0029] The problem to be solved contains two variables to be solved, s and k. In order to simplify the problem and solve it quickly, this application adopts an alternating iteration method to decompose the complex problem containing two variables to be solved in formula (4) into sub-problems about s and sub-problems about k, and iterate downward step by step, that is: through s (i) =J(s, k (i-1) ) and k (i) =J(s (i) , k) get s (i) and k (i) The alternating estimation results.

[0030] According to this logic, for the s sub-problem, if the fuzzy kernel k is fixed, the solution of the s sub-problem can be obtained by Similarly, for the k-subproblem, if the intermediate latent image s is fixed, the solution of the k-subproblem can be obtained by Obtain.

[0031] (iii) Estimate image s using blur kernel k

[0032] Since the subproblem about s The inclusion term is relatively complex. The problem can be considered non-convex. Therefore, the minimum term of this formula is difficult to solve. Therefore, the present invention uses an effective optimization strategy, the semi-quadratic splitting method, to solve this problem. It is necessary to introduce an auxiliary variable z corresponding to s, that is, s = z, and then a new objective function is obtained:

[0033]

[0034] When ρ approaches infinity, solving the above equation to minimize the result is equivalent to J(s, k (i-1) ). Obviously, for each minimization solution on s and z, s can be efficiently calculated by using the Fast Fourier Transform (FFT) as a closed-form solution. This is expressed as follows:

[0035]

[0036] Where F(·) represents Fourier transform, F -1 (·) represents the inverse Fourier transform, represents the complex conjugate of the Fourier transform.

[0037] Furthermore, z is simply initialized to a zero image. Given s, z is then numerically computed by minimizing the following:

[0038]

[0039] At this point, the solution to z in equation (7) is actually equivalent to an image filtering implementation derived from the proposed prior. In fact, in practice, the present invention transforms the estimation of the auxiliary intermediate image z into a simple and intuitive image filtering problem, greatly simplifying the difficulty of the problem and essentially realizing the plug-and-play application of dual discriminant filtering in the blind deblurring problem.

[0040] Regarding the solution of (7). Due to the double discriminant prior of Gaussian relative operator Include norm, so it is difficult to get its solution by simple calculation. To this end, an iterative reweighted least squares method is introduced to solve this type of non-convex regularization problem. of Weighted least squares transformation in the x / y direction.

[0041] Specifically, of In the x / y direction, it can be approximately converted into

[0042]

[0043] of Do the same approximate transformation in the x / y direction. Then, introduce the nonlinear weight w x / y 、υ x / y :

[0044]

[0045]

[0046] final, can be rewritten as:

[0047]

[0048] Among them, t x =ω x +v x , t y =ω y +v y To facilitate mathematical expression and solution, it is further expressed in the following matrix-vector form:

[0049]

[0050] Where S and Z are the vectorized representations of s and z respectively, and Expressed as:

[0051]

[0052] Among them, C X / Y (Z) is the x / y direction of the x / y The constructed diagonal weighted matrix, and D X / Y is the block circulant matrix corresponding to the circulant blocks constructed by the derivative operator in the x / y direction, and T represents the transpose operation.

[0053] At this point, the minimization of Equation (12) can be reduced to iteratively solving the following linear system:

[0054] (ρI+αL)Z=ρS (14)

[0055] Where I is a diagonal identity matrix and L is expressed as follows:

[0056]

[0057] In specific implementation, through the forward difference approximate derivative operator, L will become a positive definite sparse five-point Laplace matrix, so that the preconditioned conjugate gradient method can be used to effectively solve Equation (15).

[0058] (iv) Estimate k using s

[0059] In (iii) it has been estimated that the (i-1) Find s (i) , in s (i) When it has been estimated, it can be quickly solved by fast Fourier transform In the specific implementation, the gradient domain solution strategy is applied to obtain a more accurate blur kernel estimation result:

[0060]

[0061] Similarly, for the above equation, FFT can be used to efficiently obtain the analytical result in a closed domain.

[0062] In addition, after obtaining k, some normalization processing is required, including: setting negative values ​​to 0 and normalizing the elements so that the sum is 1.

[0063] In summary, (i)(ii)(iii)(iv) have introduced in detail the specific use process of the innovative dual-discriminative image prior in the image deblurring model in S1 of this application. It can be seen that the present invention has transformed a complex and large image blind deblurring problem into a non-blind problem, greatly reducing the difficulty of the problem.

[0064] S3. Use non-blind restoration to obtain the final clear image;

[0065] The present invention can further restore images using a method similar to ringing suppression methods to eliminate fine details in natural images. First, a latent image is estimated using a Laplacian operator. Second, the gradient information from this stage is used to estimate the latent image. Then, a difference map is calculated between these two estimated images, and bilateral filtering is used to remove artifacts. Finally, the filtered difference map is subtracted from the estimated latent image. This method is applicable to natural images and performs well compared to ringing suppression methods.

[0066] S4. A large number of numerical experiments were conducted on real blurred images of simulated benchmark data in multiple scenes, such as natural, artificial, text, night, and portrait. By adjusting the intermediate parameters, we finally obtained an effective and robust image blind deblurring method model for natural images.

[0067] The beneficial effects of the present invention are:

[0068] 1. Starting from image gradients, this paper proposes the guiding concept of "double discriminability," which provides valuable insights for subsequent research on blind deblurring using image priors. Furthermore, based on this concept, the core image prior of this paper is designed. This concept performs well in the model and yields accurate experimental results on a large number of real images. This further demonstrates the necessity of a blind fuzzy problem-solving model guided by this concept.

[0069] 2. The present invention is designed Image priors, which are both theoretically and practically applicable, are conducive to the subsequent in-depth research on image priors and image blind area blurring algorithms.

[0070] 3. For the reconstructed Image priors may seem to complicate the problem, but in fact, this change in image priors does not increase the difficulty of the problem or make it more difficult to solve. It only slightly increases the computational complexity of the problem, but achieves clearer image deblurring results and improves model accuracy.

[0071] 4. The semi-quadratic splitting method and alternating iteration method used in the present invention are used to solve the core formulas to be solved in the blind spot fuzzy model, which can transform difficult problems into equivalent easy-to-solve problems, shorten the solution time, solve them quickly, and reduce model consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a synthetic data set blur kernel used in a Gaussian relative operator dual discriminant image blind deblurring method provided in an embodiment of the present application.

[0073] Figure 2This is an application of a Gaussian relative operator dual discriminative image blind deblurring method provided in an embodiment of the present application to synthesized "portrait" data.

[0074] Figure 3 This is an application of a Gaussian relative operator dual discriminant image blind deblurring method provided in an embodiment of the present application to synthesized "natural" data.

[0075] Figure 4 This is an application of a Gaussian relative operator dual discriminant image blind deblurring method provided in an embodiment of the present application to synthesized "artificial" data.

[0076] Figure 5 This is an application of a Gaussian relative operator dual discriminant image blind deblurring method provided in an embodiment of the present application on real "saturated" data.

[0077] Figure 6 This is an application of a Gaussian relative operator dual discriminant image blind deblurring method provided in an embodiment of the present application on real "saturated" data.

[0078] Figure 7 This is an application of a Gaussian relative operator dual discriminative image blind deblurring method provided in an embodiment of the present application to real "portrait" data.

[0079] Figure 8 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The temporary and first in the present invention are for illustrating different stages in the algorithm training and have no limiting meaning. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0081] like Figure 8 As shown, the present invention is a Gaussian relative operator dual discriminant image blind deblurring method, comprising the following steps:

[0082] S1. Design image prior based on the principle of dual discriminability.

[0083] Inspired by the cross-scale convolution of RoG filter, we give The expression is as follows:

[0084]

[0085] stσ1<σ2, σ4<σ3, 0<t<1

[0086] Among them, G σ Defined as a local Gaussian kernel for scale selection:

[0087]

[0088] σ is the scale parameter and (x0, y0) is the center of the kernel.

[0089] In this part, inspired by the RoG image filter, we consider Image blind deblurring is introduced as image prior, and in the experimental process, it is found that replace As an image prior, it has a more convincing theoretical explanation and more excellent experimental results. Therefore, it is finally determined that As an image prior for the model of this application.

[0090] Among them, for the original stσ1<σ2 can be used to distinguish strong edges from weak textures. Because weak textures have more diverse directions and are easier to cancel out, while strong edges have more gradients with similar directions, when gradients are superimposed, the values ​​obtained for the edge gradients of large-scale features are often larger than those obtained for weak textures. Small-scale features contain almost all gradients. Clearly, for this prior, weak textures are given larger values ​​than strong edges. During the minimization iteration process, the final image converges towards strong edges, erasing the less important weak textures.

[0091] For the updated exist Based on the introduction of The purpose of the weight term is to make the image prior satisfy the ability to identify strong edges and weak textures while also having the ability to identify clear images and blurred images. Obviously, its ability to distinguish between clear and blurred images can be demonstrated by changes. For a clear image, this means that the ratio of the result of convolving the blur kernel with the gradient of the clear image to the gradient of the clear image has a much smaller numerator than the denominator, and the result is obviously less than 1.

[0092] If the blurred image is regarded as the convolution of the clear image and a blur kernel, then It can be expressed as:

[0093]

[0094] For blurred images, since the similarity of their gradients is lower, it is assumed that the gradient of an image and the gradient of the blurred image after convolution with the same blur kernel are not much different. Then the value of formula (1) always tends to 1. Therefore, we have:

[0095]

[0096] That is, in this prior In the , the value of the clear image is often smaller than that of the blurred image. In the process of minimizing the model iteration, the final image will approach the direction of the clear image.

[0097] above, This is the image prior designed by this application based on the principle of double discrimination. The next step is to bring it into the image blind deblurring model to perform image deblurring. Among them, stσ1<σ2,σ4<σ3,0<t<1,G σ Defined as a local Gaussian kernel for scale selection:

[0098]

[0099] σ is the scale parameter and (x0, y0) is the center of the kernel.

[0100] S2. The image prior is substituted into the optimization model for blind image deblurring. The alternating iterative algorithm is used to solve the clear image and blur kernel respectively, transforming the complex intermediate image estimation into a simple and intuitive image filtering problem.

[0101] The process of solving the intermediate latent image utilizes the semi-quadratic splitting method, which requires the introduction of an auxiliary variable corresponding to the intermediate latent image, resulting in a new objective function. The intermediate latent image can be efficiently calculated using the fast Fourier transform as a closed-form solution. The auxiliary variable is simply initialized to a zero image and updated based on the initial input image minimization solution. The solution is actually equivalent to an image filtering implementation derived from the proposed prior.

[0102] The blur kernel is quickly solved using the Fast Fourier Transform (FFT). In its implementation, the gradient domain solution strategy is used to obtain a more accurate blur kernel estimate. Within a closed domain, the Fourier transform can be used to efficiently obtain analytical results. Furthermore, after obtaining the blur kernel, some normalization is performed, including: setting negative values ​​to 0 and normalizing the elements so that the sum is 1.

[0103] (i) Bringing image priors into the model

[0104] For the image blind area blur problem, it is assumed that the blur is spatially invariant. Due to the high degree of ill-posedness of the image deblurring problem, traditional algorithms often use the following mathematical model to model the image degradation process, namely:

[0105] b=u*k+n (5)

[0106] Here, b represents the blurred image; u represents the desired clear image; k is the blur kernel, also known as the point spread function; n is additive noise, typically Gaussian random noise; and * represents the convolution operator. From a model perspective, image deblurring is a deconvolution problem.

[0107] The innovative image prior of this application needs to be combined with the maximum a posteriori MAP method under the traditional method, and its solution formula is:

[0108]

[0109] Among them, α and β are non-negative weights, To encode the degree of deviation of the image s relative to the original image b, that is, the restored intermediate image s and the blurred image b should be consistent in the sense of convolution, and the prior constraint of the blur kernel k adopts the simplest norm, a model can be constructed to solve the problem.

[0110] In the objective function (4), there are seven adjustable parameters: α, β, σ1, σ2, σ3, σ4, and t. Experiments show that when α = 0.01, β = 6, σ1 = 1, σ2 = 3, σ3 = 1, σ4 = 0.1, and t = 0.2, the results perform better under this parameter setting.

[0111] (ii) Decomposition of the problem to be solved

[0112] The problem to be solved contains two variables to be solved, s and k. In order to simplify the problem and solve it quickly, this application adopts an alternating iteration method to decompose the complex problem containing two variables to be solved in (4) into sub-problems about s and sub-problems about k, and iterate downward step by step, that is: through s (i) =J(s, k (i-1) ) and k (i) =J(s (i) , k) get s (i) and k (i) The alternating estimation results.

[0113] According to this logic, for the s sub-problem, if the fuzzy kernel k is fixed, the solution of the s sub-problem can be obtained by Similarly, for the k-subproblem, if the intermediate latent image s is fixed, the solution of the k-subproblem can be obtained by Obtain.

[0114] (iii) Estimate image s using blur kernel k

[0115] Since the subproblem about s The inclusion term is relatively complex. The problem can be considered non-convex. Therefore, the minimum term of this equation is difficult to solve. Therefore, an effective optimization strategy, the semi-quadratic splitting method, is used here to solve this problem. It is necessary to introduce an auxiliary variable z corresponding to s, that is, s = z, and then a new objective function is obtained:

[0116]

[0117] When ρ approaches infinity, solving the above equation to minimize the result is equivalent to J(s, k (i-1) ). Obviously, for each minimization solution on s and z, s can be efficiently calculated by using the Fast Fourier Transform (FFT) as a closed-form solution. This is expressed as follows:

[0118]

[0119] Where F(·) represents Fourier transform, F -1 (·) represents the inverse Fourier transform, represents the complex conjugate of the Fourier transform.

[0120] Furthermore, z is simply initialized to a zero image. Given s, z is then numerically computed by minimizing the following:

[0121]

[0122] At this point, the solution for z (7) is actually equivalent to an image filtering implementation derived from the proposed prior. In fact, in the specific operation, the present invention transforms the estimation of the auxiliary intermediate image z into a simple and intuitive image filtering problem, greatly simplifying the difficulty of the problem and essentially realizing the plug-and-play application of dual discriminant filtering in the blind deblurring problem.

[0123] The following discusses the solution of (7). Due to the double discriminant prior of Gaussian relative operator Include norm, so it is difficult to get its solution by simple calculation. To this end, an iterative reweighted least squares method is introduced to solve this type of non-convex regularization problem. of Weighted least squares transformation in the x / y direction.

[0124] Specifically, of In the x / y direction, it can be approximately converted into

[0125]

[0126] of Do the same approximate transformation in the x / y direction. Then, introduce the nonlinear weight w x / y 、υ x / y :

[0127]

[0128]

[0129] final, can be rewritten as:

[0130]

[0131] Among them, t x =ω x +v x , t y =ω y +v y To facilitate mathematical expression and solution, it is further expressed in the following matrix-vector form:

[0132]

[0133] Where S and Z are the vectorized representations of s and z respectively, and Expressed as:

[0134]

[0135] Among them, C X / Y (Z) is the x / y direction of the x / y The constructed diagonal weighted matrix, and D X / Y is the block circulant matrix corresponding to the circulant blocks constructed by the derivative operator in the x / y direction, and T represents the transpose operation.

[0136] At this point, the minimization of Equation (12) can be reduced to iteratively solving the following linear system:

[0137] (ρI+αL)Z=ρS (16) where I is a diagonal identity matrix and L is expressed as follows:

[0138]

[0139] In specific implementation, through the forward difference approximate derivative operator, L will become a positive definite sparse five-point Laplace matrix, so that the preconditioned conjugate gradient method can be used to effectively solve Equation (15).

[0140] (iv) Estimate k using s

[0141] In (iii) it has been estimated that the (i-1) Find s (i) , in s (i) When it has been estimated, it can be quickly solved by fast Fourier transform In the specific implementation, the present invention continues to use the gradient domain solution strategy to obtain a more accurate blur kernel estimation result:

[0142]

[0143] Similarly, the above equation can be efficiently solved using FFT in a closed domain. Furthermore, after obtaining k, some normalization is required, including setting negative values ​​to 0 and normalizing the sum of its elements to 1.

[0144] In summary, (i), (ii), (iii), and (iv) have thoroughly described the specific use of the innovative dual-discriminative image prior in the image deblurring model in S2 of this application. Thus, it can be seen that the present invention has transformed a complex and large blind image deblurring problem into a non-blind one, greatly reducing the difficulty of the problem.

[0145] S3. Use non-blind restoration to obtain the final clear image.

[0146] Although the latent image and blur kernel have been estimated, the final clear image has been obtained. However, for scenes with complex backgrounds or fine texture details, the restoration effect relying on traditional formulas is poor. While the non-blind blur method using Laplace prior has been shown to preserve fine details, the present invention can further restore the image by a method similar to the ringing suppression method using the following method to eliminate natural images with fine details. First, the latent image is estimated by using a method using the Laplace operator. Secondly, the latent image is estimated using the gradient information of this stage. Then, the difference map between the two estimated images is calculated, and bilateral filtering is used to remove artifacts. Finally, the filtered difference map is subtracted from the estimated latent image. This method is applicable to natural images and performs well compared to the ringing suppression method.

[0147] S4. A large number of numerical experiments were conducted on simulated benchmark data and real blurred images in multiple scenes such as natural, artificial, text, night, and portrait. By adjusting the intermediate parameters, we finally obtained an effective and robust image blind deblurring method model for natural images.

[0148] To illustrate the Gaussian relative operator dual discriminant image blind deblurring algorithm provided by the present invention, the present invention takes the Lai dataset as an example. The Lai dataset contains 100 images, covering five categories, namely natural, manmade, saturated, text, and people. Each category has 5 images, and each image is compared with 4 blur kernels, such as Figure 1 As shown in the figure, the convolutions are sized 31×31, 51×51, 55×55, and 75×75 from left to right, resulting in 100 synthetic blurred images. The Lai dataset corresponds to a larger blur kernel size, resulting in a greater degree of blur, making restoration difficult. However, the good results achieved by this method on this dataset further demonstrate the effectiveness and feasibility of the proposed prior.

[0149] The present invention uses PSNR (dB) and SSIM as indicators to quantitatively compare the experimental results of the present invention with those of other methods.

[0150] PSNR (Peak Signal to Noise Ratio), peak signal-to-noise ratio, is a full-reference image quality evaluation indicator.

[0151]

[0152]

[0153] MSE represents the mean square error (MSE) between the current image X and the reference image Y, H and W are the image height and width, respectively. n is the number of bits per pixel, typically 8, meaning the number of pixel grayscale levels is 256. PSNR is measured in dB; larger values ​​indicate less distortion.

[0154] PSNR is the most common and widely used objective image evaluation indicator.

[0155] SSIM (structural similarity) is also a full-reference image quality evaluation indicator, which measures image similarity from three aspects: brightness, contrast, and structure.

[0156]

[0157] where μ X 、μ Y Represents the mean of images X and Y, σ X , σ Y Represents the variance of images X and Y, σ XY Represents the covariance of images X and Y, that is

[0158]

[0159] C1, C2, and C3 are constants. To avoid the denominator being 0, C1 is usually taken as (K1·L). 2 , C2=(K2·L) 2 , G3=C2 / 2, generally K1=0.01, K2=0.03, L=255, then

[0160] SSIM(X,Y)=l(X,Y)·c(X,Y)·s(X,Y) (23)

[0161] The SSIM value range is [0, 1], and the larger the value, the smaller the image distortion.

[0162] Tables 1-4 are the existing representative methods [1] Pan J et al. "Deblurring images via dark channel prior" (J. Pan, D. Sun, H. Pfister, M. H. Yang. Deblurring images via dark channel prior. IEEE Trans. Pattern Analysis and Machine Intelligence, 2018), [2] F. Wen et al. "A simple local minimak intensity prior and an improved algorithm for blind image deblurring" (F. Wen, R. Ying, Y. Liu, P. Liu and T.-K. Truong. A simple local minimak intensity prior and an improved algorithm for blind image deblurring. IEEE Trans. Circuits and Systems for Video Technology, 2020), [3] Y. Bai et al. "Graph-based blind image deblurring from a single photograph" (Y. Bai, G. Cheung, X. Liu, W. Gao, "Graph-based blind image deblurringfrom asingle photograph. IEEE Trans. Image Processing, 2019), [4] W.Shao et al. published the article “Gradient-based discriminative modeling for blind image deblurring” (W.Shao, Y.Lin, Y.Liu, L.Wang, Q.Ge, B.Bao, H.Li.Gradient-based discriminative modeling for blind image deblurring. Neurocomputing, 2020) and the average PSNR and SSIM values ​​of the method of the present invention on the Lai dataset.

[0163] Table 1 Average statistics of PSNR (dB) and SSIM after restoration of 25 blurred images generated by kernel01

[0164]

[0165] Table 2 Average statistics of PSNR (dB) and SSIM after restoration of 25 blurred images generated by kernel02

[0166]

[0167] Table 3 Average statistics of PSNR (dB) and SSIM after restoration of 25 blurred images generated by kernel03

[0168]

[0169] Table 4 Average statistics of PSNR (dB) and SSIM after restoration of 25 blurred images generated by kernel04

[0170]

[0171] In the table, the bold method ranks first, the underlined method ranks second, and the italicized method ranks third.

[0172] It can be seen that the method of the present invention ranks first in kernel01, kernel02, and kernel04, and ranks second in kernel03. It can be seen that the Gaussian relative operator dual discriminant image blind deblurring algorithm does have a strong deblurring performance. In addition, the blur kernel estimated by the present invention contains less noise, has clear boundaries and sharp corners, and is closer to the real blur kernel. The restored image has almost no ringing effect, clear details, and a comfortable viewing experience. In order to more intuitively show the superiority of the present invention, the methods that have been tested with good results on the Lai dataset are compared, and more representative images of various types are selected. The image restoration and blur kernel estimation results are shown in the figure below. Figure 2-4 shown.

[0173] Thus, we obtain an effective and robust blind deblurring method model for natural images, which can be used to restore real images. For real images, there is no quantitative indicator to measure the restoration performance of the method. Therefore, the evaluation of the restoration ability of real images can only be based on visual perception and comparison with other methods to reflect the advantages and disadvantages of the method. The restoration and comparison results are shown in Figure 2. Figure 5-7 The image restored by the present invention has clear details and distinct blur kernels, which shows that the present invention can effectively improve the restoration effect of the model on natural images.

[0174] The deblurring method of the present invention utilizes a computer-readable storage medium storing a processor-executable program. When executed by the processor, the processor-executable program implements the Gaussian relative operator dual-discriminative image blind deblurring method of the first aspect. The aforementioned computer device includes a mobile phone, tablet computer, personal digital assistant, wearable device, or server, and the present embodiment does not impose any specific limitations on the computer.

[0175] The above embodiments merely illustrate several implementations of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A Gaussian relative operator dual discriminant image blind deblurring method, characterized by: The method comprises the following steps: Step 1. Design an image prior based on the principle of dual discriminability. This dual discriminability requires two aspects: first, the image prior must be able to distinguish between sharp and blurred images to effectively alleviate the de-degradation problem of blur kernel estimation; second, the image prior must be able to distinguish between strong edges and weak texture to significantly reduce the interference that texture may have on blur kernel estimation. Step 2: Substitute the image prior into the optimization model for blind image deblurring. Use the alternating iterative algorithm to solve the intermediate latent image and blur kernel respectively, and transform the complex intermediate image estimation into a simple and intuitive image filtering problem. Step 3. Use non-blind restoration to obtain the final clear image; Step 4. Conduct numerical experiments on multi-scene simulation benchmark data and real blurred images, adjust intermediate parameters, and finally obtain the image blind deblurring method, where: The final expression of the image prior of the dual discriminability principle is: in, is a small-scale feature, For large-scale features, Used to identify strong edges and weak textures, Used to distinguish clear images from fuzzy images, G σ Defined as a local Gaussian kernel for scale selection: σ is the scale parameter, (x0, y0) is the center of the kernel, The value of σ4 is close to 0. When σ4 is small enough, For clear images, is the blur kernel G σ3 The ratio of the result after convolution with the clear image gradient to the clear image gradient has a numerator much smaller than the denominator, and the result is less than 1; If the blurred image is regarded as the convolution of the clear image and a blur kernel k, then (3) can be expressed as: For blurred images, since the similarity of their gradients is lower, the gradient of an image and the gradient of the blurred image are similar to the values ​​obtained after convolution with the same blur kernel k. Therefore, the value of formula (4) always tends to 1, so: That is In the process of minimizing iterations, the final image will approach the clear image; In step 2, the image prior is substituted into the optimization model for blind image deblurring, which means that the image prior To substitute into Among them, α and β are non-negative weights, To encode the degree of deviation of the image s relative to the original image b, that is, the restored intermediate image s is consistent with the blurred image b in the sense of convolution, the prior constraint of the blur kernel k adopts the simplest norm; The intermediate latent image is solved by using a semi-quadratic splitting method, introducing an auxiliary variable corresponding to the intermediate latent image, and obtaining a new objective function. The intermediate latent image is calculated by using a fast Fourier transform as a closed-form solution. The auxiliary variable is initialized to a zero image and updated by minimizing the initial input image. The solution is equivalent to an image filtering implementation derived by the proposed prior. The blur kernel estimated in step 2 and the intermediate latent image are non-blindly restored as follows: first, the latent image is estimated by using the Laplacian operator method. Second, the latent image is estimated using the gradient information in step 2. Then, the difference map between the two estimated images is calculated and bilateral filtering is used to remove artifacts. Finally, the filtered difference map is subtracted from the estimated latent image.

2. The Gaussian relative operator dual discriminant image blind deblurring method according to claim 1, characterized in that: Step 2 specifically includes the following steps: Step 2-1-1: When solving the intermediate latent image, the optimization strategy, namely the semi-quadratic splitting method, is used to solve it. An auxiliary variable z corresponding to the intermediate image s is introduced, that is, s = z, and a new objective function is obtained: Among them: parameter ρ represents the penalty factor, When ρ approaches infinity, the solution to equation (7) is close to the minimum value of J(s, k (i-1) ), at each minimization solution over s and z, s is computed as a closed-form solution using the Fast Fourier Transform (FFT): Where F(·) represents Fourier transform, F -1 (·) represents the inverse Fourier transform, represents the complex conjugate of the Fourier transform; Step 2-1-2: z is initialized to a zero image. Given s, z is numerically calculated by minimizing equation (9): of In the x / y direction, it is approximately converted to of Do the same approximate transformation in the x / y direction, and then introduce the nonlinear weight ω x / y 、v x / y : final, is rewritten as: among them,t x =ω x +v x ,t y =ω y +v y , It can be further expressed in the following matrix-vector form: Where S and Z are the vectorized representations of s and z respectively, and Expressed as: Among them, C X / Y (Z) is the x / y direction of the x / y The constructed diagonal weighted matrix, D X / Y is the block circulant matrix with circulant blocks (BCCB) constructed by the derivative operator in the x / y direction, and T represents the transpose operation; At this time, the minimization of Equation (13) is reduced to iteratively solving the following linear system: (ρI+αL)Z+×ρS (15) Where I is a diagonal identity matrix and L is expressed as follows: Through the forward difference approximation derivative operator, L becomes a positive definite sparse five-point Laplace matrix, and finally the preconditioned conjugate gradient (PCG) method is used to effectively solve equation (16).

3. The Gaussian relative operator dual discriminant image blind deblurring method according to claim 1, characterized in that: The process of solving the blur kernel in step 2 is as follows: Continuing the gradient domain solution strategy, the blur kernel estimation result is obtained:

4. The Gaussian relative operator dual discriminative image blind deblurring method according to claim 3, characterized in that: After obtaining the blur kernel in step 2, some normalization processing is performed, including: setting negative values ​​to 0 and normalizing the elements so that the sum is 1.

Images