Motion image deblurring method based on variational blur kernel estimation

Abstract

The present invention relates to a motion image deblurring method based on variational blur kernel estimation. First, the implicit physical structure distribution of the blur kernel is fitted through a variational inference algorithm, which is expressed as a latent variable, such as direction and range, so as to obtain non-parametric blur kernel characteristics. Secondly, a blur generation model is adopted to approximate the expected statistical distribution of the blur kernel in a data-driven manner. A dual-head decoder structure is adopted in the blur generation stage. The blur kernel estimation method of the present invention overcomes the limitations of existing non-uniform motion blur estimation methods and can generate a large number of extremely accurate motion blur kernels. By training the model, it is not only possible to deblur real motion blurred images, but also to generate a large number of clear-blur data pairs, thereby effectively enriching and expanding the existing benchmark data set. Comprehensive experiments have shown that the method of the present invention has better deblurring effect than the existing end-to-end deep learning-based methods.

Description

Technical Field

[0001] The present invention relates to the field of image deblurring, and in particular to a motion image deblurring method based on variational blur kernel estimation. Background Art

[0002] Image deblurring aims to restore a blurred image to a clear one. Blur types can be broadly categorized as motion blur, Gaussian blur, and out-of-focus blur. This article investigates and discusses image motion blur. Generally, the distribution of motion blur in an image is a spatial variation of the image. Approaches to image deblurring can be categorized into two main categories: those based on image priors and those based on deep neural networks. Image prior-based methods target degraded linear convolution models and categorize deblurring algorithms into non-blind deconvolution and blind deconvolution, depending on whether blur kernel information is known. Traditional non-blind deconvolution deblurring algorithms, such as Wiener filtering and non-blind deconvolution, produce restored images with significant "ringing" artifacts, impacting image quality. Furthermore, since the blur kernel k is often unknown in real-world scenarios, mainstream motion deblurring algorithms today primarily target blind deconvolution. These methods leverage prior information about natural images, such as their sparsity, to overcome the poor image quality often associated with non-blind deblurring. Blind image deconvolution is a typical ill-posed problem, whose solution is primarily based on the Maximization of Posterior Probability (MAP) model. For MAP, many theoretical prior models for clean images and blur kernels have been proposed, such as gradient dilution priors, spectral priors, and low-rank priors. However, the blur kernels estimated by these MAP-based methods often fail, and the resulting restored images are far from ideal. VB's posterior mean estimation is inherently robust, reducing the risk of falling into local minima during the iterative estimation of edges and blur kernels.

[0003] For deep deblurring models combined with blur models, in order to train them to achieve good results, a large number of high-quality clear-blur image pair datasets are required. However, in reality, it is difficult to directly capture paired clear-blur image pairs.

[0004] In 2015, researchers proposed a method for estimating non-uniform blur based on deep convolutional neural networks. First, a deep convolutional neural network is used to estimate the blur probability distribution for each image region. Then, a Markov random field model is used to predict dense non-uniform blur. Finally, a non-blind image deblurring method is used to estimate the clear image. This method can effectively estimate the blur kernels in different image regions from the blurred image, achieving good results in deblurring images in dynamic scenes. To address the shortcomings of explicit clear edge selection methods, some researchers first estimate depth features from the blurred image using a deep neural network. These depth features are transformed into the frequency domain to estimate the motion blur kernel. Based on the motion blur kernel, a latent clear image is estimated. Finally, the estimated latent clear image and the original blurred image are passed through the same network to further update the depth features, thereby better estimating the motion blur kernel. Some non-blind image deblurring methods estimate the clear image, but they assume that blur is locally linear, which can fully utilize more useful image information. Some researchers use the clear image after L0 smoothing filtering as a constraint in the network. To construct a training network with clear edge information, an L0 smoothing filter is applied to the clear image, and the gradient of the filter result is used as the clear edge, maximizing the deblurring effect. Accurate kernel estimation in the frequency domain is used to restore a clear image degraded by an arbitrary blur kernel. Some researchers have combined conditional GANs with a deblurring-oriented optimization objective to learn an end-to-end conversion from the spectrum of a degraded image to an unknown kernel, achieving higher performance than state-of-the-art blind deblurring methods.

[0005] Deficiencies in existing technical solutions:

[0006] 1. Relying on a large number of manual designs of motion blur kernel priors is time-consuming, labor-intensive and inaccurate.

[0007] Some traditional deblurring methods and some deep learning methods combined with blur physics models build models to estimate the blur kernel by assuming motion fields, camera motion parameters, or directly manually parameterizing the kernel length and direction. The resulting blur kernel is single and inaccurate, resulting in poor deblurring effect.

[0008] 2. Skipping the kernel estimation step, the model lacks support for physical properties, and the deblurring effect is limited by the dataset used

[0009] Many end-to-end deep deblurring methods skip the kernel estimation step and directly implement the mapping from blurred to clear images through the network. These methods rely on a large amount of synthetic training datasets and lack the support of physical structure, resulting in a significant decrease in the generalization performance across datasets. The network may also capture a large amount of camera-specific mappings, resulting in a decrease in its deblurring effect.

[0010] 3. Poor deblurring generalization ability on real complex motion blurred scene images

[0011] For real motion blurred images, due to their complex blur types and uneven blur structures, the previous traditional and deep image deblurring effects are limited by synthetic datasets and have poor generalization capabilities. Summary of the Invention

[0012] In view of the shortcomings of the existing technology, the present invention proposes a motion image deblurring method based on variational blur kernel estimation, which is characterized in that the method uses a variational Bayesian inference algorithm to extract the image from the encoder.

[0013] The method uses a variational Bayesian inference algorithm to reparameterize the image distribution features extracted by the encoder to obtain a potential fuzzy implicit distribution and explore the intrinsic generation mechanism of the blur kernel. Then, through a fuzzy group generation module, combined with a decoder structure, the variables containing implicit distribution information are reconstructed into a non-parametric fuzzy group. In a data-driven manner, fuzzy features with physical properties are obtained and a true fuzzy image is estimated, thereby improving the model's ability to characterize the blur kernel. Finally, the trained model is input into a deblurring module to obtain a clear deblurred image. The deblurring method specifically includes:

[0014] Step 1: Prepare a deblurring training set, which includes image pairs consisting of blurred images and corresponding clear images;

[0015] Step 2: Input the blurred image into the variational kernel generative network for processing, and output the blurred linear group. Combined with the clear image, the estimated blurred image is obtained through the local convolutional network, and the trained variational blur generative model is obtained at the same time. Specifically, it includes:

[0016] Step 21: Extract the feature vector of the blurred image, and input the blurred image into the encoder to extract the image feature vector;

[0017] Step 22: Decode to obtain a mixing coefficient, input the image feature vector into the decoder 1, and the image feature vector is decoded into a mixing coefficient;

[0018] Step 23: Use the variational Bayesian inference algorithm to explore the implicit distribution of the fuzzy kernel and obtain the fuzzy system with fuzzy physical properties through the generator, which includes two stages:

[0019] Step 231: In the first stage, the image feature vector is subjected to a variational Bayesian inference algorithm to explore the implicit distribution of the blur kernel, and the implicit distribution information of the blur kernel is constructed by reparameterizing the latent variables to obtain a reparameterized distribution;

[0020] Step 232: In the second stage, the reparameterized distribution is input into the generator including decoder 2, and a fuzzy system is generated through decoding and linearization. The fuzzy kernel is modeled as a fuzzy system with a dimension of M. The entire fuzzy system is modeled as a deep generative model conditioned on the latent variables.

[0021] Step 24: Perform a weighted combination of the mixing coefficients obtained in steps 22 and 23 and the fuzzy group system to obtain a fuzzy linear group, which is then input into a local convolutional network with the clear image. The local convolutional network convolves the clear image corresponding to the blurred image with the fuzzy linear group to obtain an estimated blurred image.

[0022] Step 3: Based on the variational blur generation model trained in step 2, an optimized distribution deblurring model is constructed to process the blurred image to obtain the final clear deblurred image. Specifically, the following steps are performed:

[0023] Step 31: Input a set of blurred images into the variational blur generation model trained in step 2 to obtain a fuzzy linear group with physical properties corresponding to the blurred images;

[0024] Step 32: Construct a deep image prior module to reparameterize the clear image prior distribution and obtain the clear image prior. Specifically:

[0025] First, a standard normal random vector is used to represent the clear image distribution. It is not optimized here, but is used as the random output of the neural network.

[0026] Then, based on the deep image prior module, the regularized Richardson-Lucy natural image prior algorithm is used to regularize the gradient of the clear image distribution.

[0027] Finally, a variational regularization term is used to further constrain the clear image to avoid trivial solutions and obtain a random clear image prior.

[0028] Step 33: Perform local convolution on the clear image prior and the fuzzy linear group obtained in step 31 to obtain an estimated blurred image, calculate the image reblurring loss of the estimated blurred image and the input blurred image, and reversely control the deep image prior module of step 32 through the fuzzy loss to obtain the optimized clear image prior. The entire optimization distribution model is constrained by minimizing the loss and finally outputs a clear deblurred image.

[0029] The beneficial effects of the present invention are:

[0030] 1. The present invention adopts the variational Bayesian inference algorithm to more effectively capture the physical structure of motion blur of different directions, lengths, and densities, and can fully extract the statistical distribution of motion blur in images in a data-driven manner based on the training set.

[0031] 2. The present invention generates an accurate spatial variation kernel through the variational kernel generation network in the proposed variational blur generation model, which simplifies the blurred image imaging problem and can produce high-quality motion deblurred images. The method of the present invention effectively overcomes the limitations of model-based technologies in generating artifacts in deblurred images due to inaccurate motion blur representation.

[0032] 3. The variational blur generation model trained by the present invention can also automatically and fully generate diverse and high-quality clear-blur image training pairs when only clear images are input to enhance the existing benchmark dataset. The variational blur generation model can also apply forward model consistency to some end-to-end methods to improve their deblurring performance and contribute an enhanced dataset. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a structural diagram of the variational fuzzy generation model of the present invention;

[0034] Figure 2 It is a structural diagram of the optimized distribution defuzzification model of the present invention;

[0035] Figure 3 It is a schematic diagram of fuzzy group system generation;

[0036] Figure 4 This is the visualization result of the fuzzy transfer generation experiment;

[0037] Figure 5 It is the visualization result of the fuzzy group system generation experiment;

[0038] Figure 6 These are the deblurring results of different methods on the REDS dataset;

[0039] Figure 7 There are different methods in Deblurring results on the dataset. DETAILED DESCRIPTION

[0040] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present invention.

[0041] The following is a detailed description with reference to the accompanying drawings.

[0042] In the present invention Figure 1 The latent variable ω containing fuzzy physical characteristics in the heavy parameterization module is represented by potential fuzzy distribution; the mixing coefficient is represented by X, and the fuzzy group is represented by K.

[0043] To address the shortcomings of the existing technology, the present invention proposes a motion image deblurring method based on variational blur kernel estimation. The method uses a variational Bayesian inference algorithm to reparameterize the image distribution features extracted by the encoder to obtain a potential fuzzy implicit distribution, explore the inherent generation mechanism of the blur kernel, and then use a fuzzy group generation module to combine the decoder structure to reconstruct the variables containing implicit distribution information into a non-parametric fuzzy group. In a data-driven manner, fuzzy features with physical properties are obtained and a real blurred image is estimated, thereby improving the model's ability to characterize the blur kernel. Finally, the trained model is input into the deblurring module to obtain a clear deblurred image. The deblurring method specifically includes:

[0044] Step 1: Prepare a deblurring training set, which includes image pairs consisting of blurred images and corresponding clear images.

[0045] Step 2: Input the blurred image into the variational kernel generation network for processing, and output the blurred linear group. Combined with the clear image, the estimated blurred image is obtained through the local convolutional network, and the trained variational blur generation model is obtained at the same time. Figure 1 This is the structural diagram of the variational fuzzy generation model of the present invention. Specifically, it includes:

[0046] Step 21: Extract the feature vector of the blurred image and input the blurred image into the encoder to extract the image feature vector.

[0047] The encoder consists of five downsampling blocks, each of which consists of two convolutional layers and a max pooling layer. After each downsampling block, the spatial size is divided by 2, and the final output is an image feature vector.

[0048] Step 22: Decode to obtain a mixing coefficient, input the image feature vector into the decoder 1, and the image feature vector is decoded into a mixing coefficient.

[0049] Decoder 1 has a U-Net structure with skip connection layers.

[0050] Step 23: Use the variational Bayesian inference algorithm to explore the implicit distribution of the fuzzy kernel and obtain the fuzzy system with fuzzy physical properties through the generator, which includes two stages:

[0051] Step 231: In the first stage, the image feature vector is used to explore the implicit distribution of the blur kernel through the variational Bayesian inference algorithm, and the implicit distribution information of the blur kernel is constructed by reparameterizing the latent variables to obtain the reparameterized distribution. Specifically, the image feature vector of step 1 is first resized to output n-dimensional latent variables containing fuzzy physical characteristics. The fuzzy physical characteristics include direction, density, and length. The latent variables are pre-modeled as Gaussian prior distributions with isotropy. The variational Bayesian inference algorithm is used to maximize the KL hash function, and the mean and variance of the implicit distribution of the latent variables are reparameterized to obtain the fuzzy physical feature distribution that fits the real fuzzy space, that is, the reparameterized distribution.

[0052] Step 232: In the second stage, the reparameterized distribution is input into the generator including decoder 2, and a fuzzy system is generated by decoding and linearization. The fuzzy kernel is modeled as a fuzzy system with a dimension of M. The entire fuzzy system is modeled as a deep generative model conditioned on the latent variables, as shown in the following example: Figure 3 As shown, Figure 3 This is a schematic diagram of fuzzy group generation.

[0053] Specifically, the reparameterized distribution is first input to Decoder 2, which consists of five upsampling blocks and five downsampling blocks. Like Decoder 1, Decoder 2 uses a U-Net structure with skip connection layers. The reparameterized latent variables are sequentially input to the five upsampling blocks. Each upsampling step consists of bilinear upsampling and three convolutional layers. In addition, a skip connection is used between the second convolution in the downsampling block in Decoder 2 and the second convolution in the upsampling block. After the last convolution, a linear operation is performed, and then a maximum normalized activation function (softmax) is applied to each dimensional channel along the batch size to output the fuzzy group K.

[0054] Normalization is performed to ensure that the sum of the blending coefficients for each pixel associated with the blur kernel is 1; the convolution kernel size is 3 and the padding is 1, except for the first 64-channel convolution, which has a kernel size of 2.

[0055] In step 2, a specific variational Bayesian inference algorithm is used in combination with the fuzzy generation model process. The fuzzy system is obtained by adding the fuzzy kernel to the low-rank structure and decomposing the dense non-uniform fuzzy kernel in the fuzzy image into a linear combination of smaller basic units. The variational idea is applied to the fuzzy distribution exploration process, and a fuzzy system that can reflect the physical characteristics of the real fuzzy space is generated through a data-driven method.

[0056] The specific implementation process is:

[0057] First, express the complete distribution as a general statistical distribution Where P(ω) is the prior distribution of the latent variable ω of the blur kernel group, and then a nonparametric variational Bayesian framework is used to learn the logarithmic estimate of P(b). It is expressed as the variational posterior T and the KL divergence. In the case of minimizing the KL divergence, the variational posterior is directly used as the logarithmic estimate of the blurred image distribution:

[0058] logP(b)~T(ω;b)=E Q(ω|b) [logP θ (b|ω)P(ω)-logQ(ω|b)] (1)

[0059] In mathematical expression (1), P(ω|b) is the probability density function. Q(ω|b) is the variational approximation of the true posterior P(ω|b) with respect to the latent variable ω. Maximizing T(ω; b) approximates the distribution of b. According to variational theory, a Gaussian distribution is used to approximate the prior P(ω). The variational posterior is expressed using the KL divergence as follows:

[0060]

[0061] Among them, μ i f(b) and σ i f(b) is the inference function of the posterior distribution parameters of ω. It is used as VBLoss to control the generation of the entire fuzzy system. The variational inference part can infer the fuzzy system in the real space. Apart from the generator parameters, no other setting parameters are introduced.

[0062] Step 24: The mixing coefficients obtained in step 22 and step 23 are first weightedly combined with the fuzzy group to obtain a fuzzy linear group, which is then input into the local convolutional network with the clear image. The local convolutional network convolves the clear image corresponding to the blurred image b with the fuzzy linear group to obtain an estimated blurred image b. f .

[0063] The mathematical expression of the image reblurring loss function is as follows:

[0064]

[0065] The above expression represents the estimated blurred image b f The distribution distance between the blurred image b. i It is the inverse of the weight between the fuzzy group kernels of the same image.

[0066] Step 3: Based on the variational blur generation model trained in step 2, an optimized distribution deblurring model is constructed to process the blurred image and finally obtain a deblurred image. Figure 2 It is a structural diagram of the optimized distribution defuzzification model of the present invention; Figure 2 The training model in is the variational fuzzy generation model obtained by training in step 1. Specifically, it includes:

[0067] Step 31: Input a set of blurred images into the variational fuzzy generation model trained in step 2 to obtain a fuzzy system with physical properties corresponding to the blurred images.

[0068] Step 32: Construct a deep image prior module to reparameterize the clear image prior distribution. Specifically:

[0069] First, a standard normal random vector is used to represent the clear image distribution. Instead of optimizing it, it is used as the random output of the neural network. Then, based on the deep image prior module, the regularized Richardson-Lucy natural image prior algorithm is used to regularize the gradient of the clear image. This ensures the gradient sparsity of natural images and reduces image noise. Furthermore, a variational regularization term is used to further constrain the image, avoiding trivial solutions and obtaining a random clear image prior.

[0070] Step 33: Perform local convolution on the clear image prior and the fuzzy linear group obtained in step 31 to obtain an estimated blurred image. Calculate the image reblurring loss of the estimated blurred image and the input blurred image. Use this loss to reversely control the deep image prior module in step 32 to obtain the optimized clear image prior. The entire optimization distribution model minimizes the loss constraint and finally outputs a clear deblurred image.

[0071] Figure 4 Generate experimental visualization results for blur migration of clear images, migrating from the dataset REDS to the COCO dataset, Figure 4 (a) is the clear image before blur migration. Figure 4 (b) Figure 4 (c) and Figure 4 (d) is the blurred image obtained according to the scale factor, the scale factors are 60%, 80%, 100%, Figure 4 (e) is the original blurred image. Through the blur migration experiment, an enhanced blur-clear dataset can be constructed.

[0072] Figure 5 To generate experimental visualization results for the fuzzy group system, two blurred images are input into the trained variational fuzzy generation model, and the fuzzy kernel group corresponding to the blurred image can be directly obtained, which intuitively reflects the physical situation of its fuzzy distribution. Figure 5 (a) is a blurred image, Figure 5 (b) is the generated fuzzy system.

[0073] To further illustrate the beneficial effects of the present invention, the comparison methods selected include end-to-end methods: Method 1: Deblurring Generative Adversarial Networks, Method 2: Scaled Recursive Network Deblurring Method; and model-based deblurring methods: Method 3: Self-deblurring Method, Method 4: Non-uniform Blur Kernel Estimation Method, and Method 5: Blur Kernel Space Method. The present invention uses three evaluation metrics to evaluate the deblurring effect of the present invention: Image Perceptual Similarity (LPIPS), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity (SSIM). The results show that the deblurring results of the present invention are superior to existing deblurring methods.

[0074] Figure 6 Comparison of deblurring results of different methods on REDS dataset. Figure 6 (a) is a blurred image (the structural similarity evaluation indicators of the pictures from top to bottom are: 0.682, 0.641, 0.430, 0.603). Figure 6 (b) Deblurring results of method 1 (the structural similarity evaluation indicators of the images from top to bottom are: 0.543, 0.543, 0.423, 0.467). Figure 6 (c) is the deblurring result of method 2 (the structural similarity evaluation indicators of the pictures from top to bottom are: 0.761, 0.753, 0.573, and 0.687 respectively). Figure 6 (d) is the deblurring result of method 3 (the structural similarity evaluation indicators of the images from top to bottom are: 0.897, 0.962, 0.964, and 0.883 respectively). Figure 6 (e) is the deblurring result of method 4 (the structural similarity evaluation indicators of the pictures from top to bottom are: 0.562, 0.472, 0.389, 0.492). Figure 6 (f) is the deblurring result of the method of the present invention (the structural similarity evaluation indexes of the pictures from top to bottom are: 0.504, 0.522, 0.376, 0.455 respectively). Figure 6 (g) is a clear image.

[0075] Figure 7 For different methods Comparison of deblurring results on the dataset, Figure 7 (a) is a blurred image (the peak signal-to-noise ratio evaluation indicators of the pictures from top to bottom are: 24.69dB, 25.73dB), Figure 7 (b) is the deblurring result of method 1 (the peak signal-to-noise ratio evaluation indicators of the pictures from top to bottom are: 27.82dB and 28.96dB respectively). Figure 7 (c) Deblurring results of method 2 (peak signal-to-noise ratio evaluation indicators of the images from top to bottom are: 28.69dB and 30.48dB respectively). Figure 7(d) Deblurring results of method 3 (peak signal-to-noise ratio evaluation indicators of the images from top to bottom are: 29.37dB, 29.86dB respectively). Figure 7 (e) Deblurring results of method 4 (peak signal-to-noise ratio evaluation indicators of the images from top to bottom are: 30.07dB, 30.16dB respectively). Figure 7 (f) Deblurring results of method 5 (peak signal-to-noise ratio evaluation indicators of the images from top to bottom are: 27.93dB, 29.76dB), Figure 7 (g) is the deblurring result of the proposed method (the peak signal-to-noise ratio evaluation indicators of the pictures from top to bottom are: 32.93dB and 33.14dB respectively). Figure 7 (h) Clear image. Figure 6 and Figure 7 It can be seen intuitively from the visualization results that the deblurring method of the present invention is closer to the real clear image than the deblurring effect of other methods, and the restored details are more complete and accurate.

[0076] A fuzzy generation and transfer experiment is designed to evaluate the quality of fuzzy kernel generation. Finally, an enhanced deblurring experiment is designed to evaluate the generated enhanced fuzzy dataset and explore the improvement effect of the enhanced dataset of the present invention on the classic deblurring deep learning method.

[0077] Deblurring comparison experiment: In order to verify the deblurring ability of the model, the proposed model is compared with multiple current deep learning deblurring models. All experiments are conducted across datasets to verify the generalization performance and robustness of the model in practical applications. The present invention compares the deblurring results on multiple standard datasets including real image blur datasets and synthetic datasets, including the REDS dataset. The evaluation indicators LPIPS, PSNR, and SSIM were used to evaluate and compare the proposed method. The results show that the deblurring results of the proposed method are superior to those of the existing deblurring methods.

[0078] Table 1 shows the comparison results of image perception similarity LPIPS of various comparison methods on the REDS dataset, the smaller the better. Table 2 shows the comparison results of various methods on the REDS dataset, the smaller the better. The comparison results of objective evaluation indicators on the data set are as large as possible.

[0079] Table 1 Comparison of objective evaluation indicators on the REDS dataset

[0080] Adoption Method Method 1 Method 2 Method 3 Method 4 Method 5 Method of the present invention Image Perceptual Similarity 0.467 0.531 0.768 0.477 0.896 0.441

[0081] Table 2 Comparison of objective evaluation indicators on the dataset

[0082] Adoption Method Method 1 Method 2 Method 3 Method 4 Method 5 Method of the present invention Peak signal-to-noise ratio 27.18 27.57 25.24 28.23 28.01 28.46 Structural similarity 0.79 0.81 0.76 0.81 0.80 0.83

[0083] Enhanced deblurring experiment: The present invention first uses the fuzzy groups generated by the model to fine-tune the Deblur GAN model of method 1 and performs enhanced deblurring. Secondly, the fuzzy groups extracted by the model using the blurred images of the REDS test set on the GOPRO training set are collected. Then, by changing the K value, different blur effects and blur degrees are transferred to the COCO dataset. The Deblur GAN network is then retrained on the entire enhanced dataset. The results are shown in Table 3. It can be seen that the new enhanced blurred dataset generated by the present invention has a certain improvement on the Deblur GAN.

[0084] Table 3 Comparison of objective evaluation indicators on the GOPRO dataset

[0085] Adoption Method Method 1 Method 1 + Method 4 Method 1+method of the present invention Method 1 + Enhanced COCO dataset Peak signal-to-noise ratio 27.14 28.08 28.11 29.23 Structural similarity 0.79 0.85 0.87 0.91

[0086] It should be noted that the above-described specific embodiments are illustrative only. Those skilled in the art may devise various solutions based on the disclosure of the present invention, and such solutions fall within the scope of the disclosure and the scope of protection of the present invention. Those skilled in the art should understand that the present description and the accompanying drawings are illustrative only and do not constitute limitations of the claims. The scope of protection of the present invention is defined by the claims and their equivalents.

Claims

1. A motion image deblurring method based on variational blur kernel estimation, characterized in that: The method uses a variational Bayesian inference algorithm to reparameterize the image distribution features extracted by the encoder to obtain a potential fuzzy implicit distribution and explore the intrinsic generation mechanism of the blur kernel. Then, through a fuzzy group generation module, combined with a decoder structure, the variables containing implicit distribution information are reconstructed into a non-parametric fuzzy group. In a data-driven manner, fuzzy features with physical properties are obtained and a true fuzzy image is estimated, thereby improving the model's ability to characterize the blur kernel. Finally, the trained model is input into a deblurring module to obtain a clear deblurred image. The deblurring method specifically includes: Step 1: Prepare a deblurring training set, which includes image pairs consisting of blurred images and corresponding clear images; Step 2: Input the blurred image into the variational kernel generative network for processing, and output the blurred linear group. Combined with the clear image, the estimated blurred image is obtained through the local convolutional network, and the trained variational blur generative model is obtained at the same time. Specifically, it includes: Step 21: Extract the feature vector of the blurred image, and input the blurred image into the encoder to extract the image feature vector; Step 22: Decode to obtain a mixing coefficient, input the image feature vector into the decoder 1, and the image feature vector is decoded into a mixing coefficient; Step 23: Use the variational Bayesian inference algorithm to explore the implicit distribution of the fuzzy kernel and obtain the fuzzy system with fuzzy physical properties through the generator, which includes two stages: Step 231: In the first stage, the image feature vector is subjected to a variational Bayesian inference algorithm to explore the implicit distribution of the blur kernel, and the implicit distribution information of the blur kernel is constructed by reparameterizing the latent variables to obtain a reparameterized distribution; Step 232: In the second stage, the reparameterized distribution is input into the generator including decoder 2, and a fuzzy system is generated through decoding and linearization. The fuzzy kernel is modeled as a fuzzy system with a dimension of M. The entire fuzzy system is modeled as a deep generative model conditioned on the latent variables. Step 24: Perform a weighted combination of the mixing coefficients obtained in steps 22 and 23 and the fuzzy group system to obtain a fuzzy linear group, which is then input into a local convolutional network with the clear image. The local convolutional network convolves the clear image corresponding to the blurred image with the fuzzy linear group to obtain an estimated blurred image. Step 3: Based on the variational blur generation model trained in step 2, an optimized distribution deblurring model is constructed to process the blurred image to obtain the final clear deblurred image. Specifically, the following steps are performed: Step 31: Input a set of blurred images into the variational blur generation model trained in step 2 to obtain a fuzzy linear group with physical properties corresponding to the blurred images; Step 32: Construct a deep image prior module to reparameterize the clear image prior distribution and obtain the clear image prior. Specifically: First, a standard normal random vector is used to represent the clear image distribution. It is not optimized here, but is used as the random output of the neural network. Then, based on the deep image prior module, the regularized Richardson-Lucy natural image prior algorithm is used to regularize the gradient of the clear image distribution. Finally, a variational regularization term is used to further constrain the clear image to avoid trivial solutions and obtain a random clear image prior. Step 33: Perform local convolution on the clear image prior and the fuzzy linear group obtained in step 31 to obtain an estimated blurred image, calculate the image reblurring loss of the estimated blurred image and the input blurred image, and reversely control the deep image prior module of step 32 through the blur loss to obtain the optimized clear image prior. The entire optimization distribution model is constrained by minimizing the loss and finally outputs a clear deblurred image.

Images