An Image Blind Deblurring Network and Method Based on Fuzzy Kernel Priori Learning

Abstract

The present invention relates to a network and method for image deblurring, and particularly to an image blind deblurring network and method based on fuzzy kernel prior learning. It solves the technical problem that the prior art regards image blind deblurring as an image regression task, without considering the differences in different data sets and the changes in local image fuzzy kernels, resulting in poor generalization performance. The network of the present invention includes a fuzzy kernel estimation network and a deblurring network. The fuzzy kernel estimation network is used to estimate the fuzzy kernel image of the blurred image, and it includes a feature extraction network, an uncertainty learning module, and a normalizing flow model. The deblurring network is used to obtain a clear image through the blurred image and the estimated fuzzy kernel image, and it includes a downsampling unit, an intermediate unit, and an upsampling unit connected in sequence. The downsampling unit includes a plurality of downsampling layers connected in sequence; the intermediate unit includes a plurality of basic modules connected in sequence; the upsampling unit includes a plurality of upsampling layers connected in sequence.

Description

Technical Field

[0001] The present invention relates to a network and method for image deblurring, in particular to an image blind deblurring network and method based on fuzzy kernel prior learning, which can be applied to occasions such as image classification, target recognition, and target tracking that require obtaining clear images. Background Art

[0002] During the image shooting process, image blurring may occur due to the defocus of the camera lens or the relative movement between the scene target and the camera. Especially in low-light scenarios, due to the long exposure time required by the camera, the camera shake or target movement makes the image blurring problem more serious. Image blurring not only affects the visual effect but also seriously reduces the performance of image analysis algorithms in many visual tasks, such as image target detection and recognition tasks.

[0003] Image deblurring, as an important task in the field of computer low-level vision, aims to reconstruct a clear image from the corresponding blurred image and enhance the details therein, and has important applications in national defense security, video surveillance, mobile phone cameras, etc. Image deblurring can be divided into uniform image deblurring and non-uniform image deblurring according to the blurred area. Uniform blurring is mainly caused by the shake of the shooting device and usually forms a global blur of the background. The movement of an object usually causes non-uniform local blurring of the image. It can also be divided into image non-blind deblurring and image blind deblurring according to whether the blur kernel is known or unknown. In reality, the blur type of an image is usually non-uniform and the information of the blur kernel is unknown. Therefore, how to solve non-uniform blind image deblurring has become a hot issue with great research value at present.

[0004] Traditional blind image deblurring methods first estimate the blur kernel from a blurred image and then recover the clear image through an iterative optimization method (see Amit Goldstein and Raanan Fattal. Blur-kernel estimation from spectral irregularities. In European Conference on Computer Vision, pages 622–635. Springer, 2012. and Jinshan Pan, Zhe Hu, Zhixun Su, and Ming-Hsuan Yang. Deblurring text images via l0-regularized intensity and gradient prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2901–2908, 2014.). However, traditional methods have the following two problems: one is that they cannot accurately estimate the blur kernel of the real image, and the other is that they usually ignore the motion prior information. Since the blur kernel in the real scene is very complex and the formation of blur degradation is usually related to the motion trajectories of objects and cameras, this information is crucial for removing motion blur in images.

[0005] In recent years, with the rapid development of deep learning technology, image deblurring methods based on deep learning have achieved excellent performance (see Orest Kupyn, Volodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Jiˇr′1 Matas. Deblurgan: Blind motion deblurring using conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8183–8192, 2018. and Orest Kupyn, Tetiana Martyniuk, Junru Wu, and Zhangyang Wang. Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. In Proceedings of the IEEE / CVF International Conference on Computer Vision, pages 8878–8887, 2019. and Sung-Jin Cho, Seo-Won Ji, Jun-Pyo Hong, Seung-Won Jung, and Sung-Jea Ko. Rethinking coarse-to-fine approach in single image deblurring. In Proceedings of the IEEE / CVF international conference on computer vision, pages 4641–4650, 2021. and Dasong Li, Yi Zhang, Ka Chun Cheung, Xiaogang Wang, Hongwei Qin, and Hongsheng Li. Learning degradation representations for image deblurring. In European Conference on Computer Vision, pages 736–753. Springer, 2022.), and the powerful representation ability of the feature extraction network has enabled deep learning-based image deblurring algorithms to far exceed traditional image deblurring algorithms.

[0006] Existing deep learning-based blind image deblurring algorithms can be divided into two categories. The first category of methods uses a convolutional neural network to directly estimate the non-uniform blur kernel from the blurred image. The work closest to the present invention in this category is the method proposed by Jian Sun et al. in 2015 (see Jian Sun, Wenfei Cao, Zongben Xu, and Jean Ponce. Learning a convolutional neural network for non-uniform motion blur removal. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 769–777, 2015.), which uses a convolutional neural network to predict the probability distribution of the motion blur kernel of image patches, and then uses a Markov random field model to infer the dense non-uniform motion blur field, and removes the motion blur by using a non-uniform deblurring model based on image priors. The other category of methods does not estimate the blur kernel and directly recovers the original clear image from the blurred image end-to-end. The work similar to the present invention in this category is the DeepDeblur method (see Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3883–3891, 2017.), which designs a multi-scale convolutional neural network to simulate traditional coarse-to-fine optimization and directly recovers the clear image without assuming any constrained blur kernel model.

[0007] Since the formation of image blur is related to the relative motion between the object and the camera and is random, the true blur kernel is very complex, and it is very difficult to accurately estimate a non-uniform (i.e., spatially varying) blur kernel. Therefore, it is still a very big challenge to simultaneously estimate a clear image and an image blur kernel from a single image using a deep learning-based image blind deblurring method. Although existing deep learning-based image blind deblurring methods have obtained good results on some public datasets, especially simulation datasets, these methods perform poorly on many real blur images. The main reason is that existing deep learning methods regard image blind deblurring as an image regression task, do not consider the differences in different datasets and the changes in local image blur kernels, and ignore the blur kernel prior, resulting in poor generalization performance, that is, poor performance on real blur images. Summary of the Invention

[0008] The object of the present invention is to solve the technical problem that existing deep learning-based image blind deblurring methods regard image blind deblurring as an image regression task, do not consider the differences in different datasets and the changes in local image blur kernels, resulting in poor generalization performance, and to provide an image blind deblurring network and method based on blur kernel prior learning.

[0009] The image blind deblurring network and method based on blur kernel prior learning proposed by the present invention has the following concept: using a normalizing flow model and an uncertainty learning module to map the complex motion blur kernel space into a simple Gaussian distribution space, adopting a method of estimating the blur kernel in the Gaussian distribution space, and using the prior information of the motion blur kernel, thereby greatly improving the accuracy of blur kernel estimation, and using the estimated blur kernel to improve the effect of image deblurring, and at the same time restoring a higher image visual quality, which is applicable to scenarios where clear images need to be generated.

[0010] The technical solution of the present invention is as follows:

[0011] An image blind deblurring network based on blur kernel prior learning, which is characterized in that it includes a blur kernel estimation network and a deblurring network;

[0012] The blur kernel estimation network is used to estimate the blur kernel image of the blurred image, and it includes a feature extraction network, an uncertainty learning module and a normalizing flow model connected in sequence;

[0013] The deblurring network is used to input a blurred image to be processed and obtain a clear image by using the estimated blurred kernel image. It includes a downsampling unit, an intermediate unit, and an upsampling unit connected in sequence. The downsampling unit includes a plurality of downsampling layers connected in sequence, numbered as the first downsampling layer, the second downsampling layer... the m-th downsampling layer in the signal output direction. The intermediate unit includes a plurality of basic modules connected in sequence. The upsampling unit includes a plurality of upsampling layers connected in sequence and having the same number as the downsampling layers, numbered as the first upsampling layer, the second upsampling layer... the m-th upsampling layer in the reverse direction of the signal output.

[0014] The downsampling layer includes a blurred kernel attention module, a basic module, and a downsampling module connected in sequence.

[0015] The upsampling layer includes a basic module and an upsampling module connected in sequence.

[0016] The input end of the blurred kernel attention module of each downsampling layer is connected to the output end of the normalizing flow model, and the other input end of the blurred kernel attention module of the first downsampling layer is used to input the blurred image. Between adjacent numbered downsampling layers, the output end of the previous numbered downsampling module is connected to the other input end of the blurred kernel attention module of the next numbered layer. The output end of the downsampling module of the m-th downsampling layer is connected to the input end of the basic module at the front end of the intermediate unit. Between the upsampling layer and the downsampling layer with the same number, the output end of the downsampling module of the upsampling layer is connected to the input end of the basic module of the downsampling layer.

[0017] Between adjacent numbered upsampling layers, the input end of the previous numbered basic module is connected to the output end of the next numbered upsampling module. The input end of the basic module of the m-th upsampling layer is connected to the output end of the basic module at the end of the intermediate unit. The output end of the upsampling module of the first upsampling layer is used to output the clear image.

[0018] Further, the blurred kernel attention module includes an image feature convolution layer, a blurred kernel feature convolution layer, a fused feature convolution layer, a multiplication layer, and an addition layer. The input end of the blurred kernel feature convolution layer is connected to the output end of the normalizing flow model. The output ends of the image feature convolution layer and the blurred kernel feature convolution layer are respectively connected to the input end of the fused feature convolution layer. The two input ends of the multiplication layer are respectively connected to the output end of the fused feature convolution layer and the output end of the image feature convolution, and are used to perform a multiplication operation on the output result of the fused feature convolution layer and the output result of the image feature convolution. The two input ends of the addition layer are respectively connected to the output end of the multiplication layer and the input end of the image feature convolution, and are used to perform an addition operation on the output result of the multiplication layer and the blurred image to obtain the output result of the blurred kernel attention module.

[0019] The input end of the image feature convolution layer of the blur kernel attention module of the first downsampling layer is used to input a blurred image, and the input ends of the image feature convolution layers of the blur kernel attention modules of other sampling layers are respectively connected to the output ends of the downsampling modules of the previous numbered sampling layer;

[0020] The output end of the addition layer is connected to the input end of the basic module of the same blur kernel attention module.

[0021] Further, the basic module includes a first LayerNorm layer, a first convolution layer, a second convolution layer, a first SimpleGate layer, a Simplified Channel Attention layer, a third convolution layer, a second LayerNorm layer, a fourth convolution layer, a second SimpleGate layer, and a fifth convolution layer connected in sequence; the Simplified Channel Attention layer includes an AveragePooling layer and a sixth convolution layer connected; the first SimpleGate layer is used to divide the image feature map into two parts on average by channel, and multiply these two parts by pixel position to obtain an output result; the first SimpleGate layer and the second SimpleGate layer have the same structure.

[0022] Further, the feature extraction network includes an encoder residual module, a downsampling layer, an intermediate convolution layer, an upsampling layer, and a decoder residual module connected in sequence; wherein, both the encoder residual module and the decoder residual module include a seventh convolution layer, a ReLIU activation function layer, an eighth convolution layer, a ReLIU activation function layer, and a ninth convolution layer connected in sequence; the normalizing flow model includes twenty flowblocks connected in sequence; the flowblock includes a batchnormalization layer, a permutationlayer, and an affine transformation layer connected in sequence; the number of downsampling layers is three, numbered as the first downsampling layer, the second downsampling layer, and the third downsampling layer in sequence along the signal output direction; the number of upsampling layers is three, numbered as the first upsampling layer, the second upsampling layer, and the third upsampling layer in sequence along the reverse direction of the signal output; the intermediate unit includes twenty-eight basic modules.

[0023] Meanwhile, the present invention also provides an image blind deblurring method based on blur kernel prior learning, which is characterized in that it includes the following steps:

[0024] 1) Obtain a blurred image set Y and a corresponding clear image set X;

[0025] 2) Use the method of random trajectory generation to obtain a blur kernel image set;

[0026] 3) Construct the image blind deblurring network based on fuzzy kernel prior learning as described above;

[0027] 4) Randomly initialize the weights and biases of the normalizing flow model, the blur kernel estimation network, and the deblurring network;

[0028] 5) Conduct training

[0029] 5.1) Training of the normalizing flow model: Randomly select a blur kernel image from the blur kernel image set K as the input training sample, and train the normalizing flow model with the negative log-likelihood loss function;

[0030] 5.2) Training of the blur kernel estimation network: Randomly select a blurred image from the blurred image set Y as the input sample, and at the same time select the corresponding clear image from the clear image set X as the reference sample, and train the blur kernel estimation network with the self-supervised loss function;

[0031] 5.3) Training of the deblurring network: Randomly select a blurred image from the blurred image set Y as the input sample, and at the same time select the corresponding clear image from the clear image set X as the reference sample. First, input the blurred image into the blur kernel estimation network to output the blur kernel image, and then train the deblurring network with the reconstruction loss function;

[0032] 6) Image deblurring

[0033] 6.1) Input the blurred image to be processed into the blur kernel estimation network to obtain the estimated blur kernel image;

[0034] 6.2) Input the blurred image to be processed and the corresponding blur kernel image into the deblurring network at the same time, and output the clear image.

[0035] Further, in step 1), the blurred image set Y = {Y 1 , Y 2 , …, Y i , …, Y N}, the clear image set X = {X 1 , X 2 , …, X i , …, X N}, Y i and X i are the i-th blurred image and the corresponding i-th clear image respectively, 1 ≤ i ≤ N, and N is the number of pairs of blurred images and clear images.

[0036] Further, in 5.1), the negative log-likelihood loss function L(k; θ) is:

[0037]

[0038] f θ (k) represents a normalizing flow model with parameter θ, and its input is a blurred kernel image randomly selected from the blurred kernel image set K;

[0039] represents calculating the Jacobian matrix of the normalizing flow model;

[0040] p Z (f θ (k)) represents calculating the density function for f θ (k).

[0041] Furthermore, in 5.2), the self-supervised loss function L KE is:

[0042]

[0043] x i and y i represent the i-th pair of clear and blurred image pairs;

[0044] G(y i ) represents the final latent code estimated from y i ;

[0045] f θ [G(y i )] is the blurred kernel image decoded by the normalizing flow model.

[0046] Furthermore, in step 5.3), the reconstruction loss function L recon is the weighted sum of the re-blurring loss function L reblur and the PSNR loss function L PSNR : L recon = L PSNR + λL reblur , where λ is the weight of the re-blurring loss function, and the value of λ is 0.01;

[0047] The re-blurring loss function L reblur and the PSNR loss function L PSNR are respectively:

[0048]

[0049]

[0050] F(y i ) is the reconstructed clear image;

[0051] K(y i ) represents the blurred kernel image estimated from the blurred image using the blurred kernel estimation network;

[0052] PSNR(F(y i ), x i ) represents calculating the peak signal-to-noise ratio for F(y i ) and x i .

[0053] Furthermore, step 6.1) is specifically as follows:

[0054] 6.1.1) Input the blurred image into the feature extraction network, and the feature extraction network calculates the latent code z of each pixel in the blurred image i and predicts the standard deviation σ of the latent code i , where z i ~N(0, I) follows the standard normal distribution;

[0055] 6.1.2) The feature extraction network sends the latent code z i and the standard deviation σ i into the uncertainty learning module for Gaussian resampling to obtain the uncertainty component n of each latent code z i : i

[0056]

[0057] where n i is the same size as z i , and z i and n i are defined to be independent of each other, and the final latent code is obtained by addition and follows the standard normal distribution:

[0058]

[0059] where represents the main body component of the latent code z i .

[0060] 6.1.3) The uncertainty learning module sends the final latent code into the normalizing flow model and outputs the blurred kernel image.

[0061] The beneficial effects of the present invention are as follows:

[0062] 1. The network of the present invention uses the normalizing flow model and the uncertainty learning module to map the complex motion blur kernel space into a simple Gaussian distribution space. By using the method of estimating the blur kernel within the Gaussian distribution space, it can utilize the prior information of the motion blur kernel, thereby greatly improving the accuracy of blur kernel estimation. The estimated blur kernel is used to improve the effect of image deblurring, and at the same time, a higher image visual quality is restored.

[0063] ​2. The network and method of the present invention can predict relatively simple latent encodings by estimating non-uniform motion blur kernels in the latent space, avoiding directly estimating complex motion blur kernels and improving the accuracy of blur kernel estimation. By introducing uncertainty learning, the robustness of blur kernel estimation can be further improved. Finally, the deblurring network can utilize the learned blur kernel prior information to obtain clearer images. The present invention proves for the first time that this blur kernel estimation based on latent space prior has better effects. Experimental results on common blur datasets show that when using the uncertainty deblurring network based on normalizing flow proposed by this method, compared with other deblurring methods, better PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) index results and better visual effects are achieved.

[0064] 3. The network of the present invention uses a normalizing flow model to project the blur kernel into a standard Gaussian space, and then estimates the non-uniform blur kernel in this latent space, that is, uses a feature extraction network to estimate the encoding coefficients of the blur kernel in the latent space, thus significantly improving the accuracy of blur kernel estimation.

[0065] 4. The network of the present invention adopts a multi-scale blur kernel and attention module to better combine image features with the estimated blur kernel. Experimental results show that the image blind deblurring network proposed by the present invention has significantly better performance than existing deep networks.

[0066] 5. The method of the present invention projects a very complex spatially non-uniform motion blur kernel into a standard Gaussian distribution space through a normalizing flow model. Compared with estimating the blur kernel in the spatial domain, estimating the non-uniform blur kernel in this pre-trained standard Gaussian distribution space greatly reduces the difficulty of blur kernel estimation; the estimated non-uniform blur kernel is input into the deblurring network to guide the deblurring network to remove spatially varying blur. Compared with existing deep network image blind deblurring methods, the method proposed by the present invention greatly improves the image deblurring performance, especially on real motion blurred images, showing excellent deblurring performance.

[0067] 6. The method of the present invention uses a self-supervised learning function to train the blur kernel estimation network to estimate non-uniform blur kernels, solving the problem that real blur kernels are difficult to obtain.

[0068] 7. The method of the present invention proposes a blur kernel estimation method based on uncertainty learning, simultaneously estimating the latent encoding and standard deviation, obtaining the uncertainty component through a Gaussian resampling process with the standard deviation as a parameter, and adding the fixed component of the latent encoding and the uncertainty component to obtain the final latent encoding, further improving the accuracy and robustness of non-uniform blur kernel estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1The motion blur kernel samples obtained by an embodiment of an image blind deblurring method based on fuzzy kernel prior learning of the present invention;

[0070] Figure 2 The display diagram of the mapping formed by the normalizing flow model of an image blind deblurring network based on fuzzy kernel prior learning of the present invention;

[0071] Figure 3 The schematic framework diagram of an image blind deblurring network based on fuzzy kernel prior learning of the present invention;

[0072] Figure 4 The schematic framework diagram of the fuzzy kernel estimation network of an image blind deblurring network based on fuzzy kernel prior learning of the present invention;

[0073] Figure 5 The schematic framework diagram of the deblurring network of an image blind deblurring network based on fuzzy kernel prior learning of the present invention;

[0074] Figure 6 The schematic framework diagram of the fuzzy kernel attention module of an image blind deblurring network based on fuzzy kernel prior learning of the present invention;

[0075] Figure 7 The flow chart of an embodiment of an image blind deblurring method based on fuzzy kernel prior learning of the present invention;

[0076] Figure 8 The visual comparison diagram of an embodiment of an image blind deblurring method based on fuzzy kernel prior learning of the present invention and different existing methods on the GoPro dataset (No. 1 represents the input blurred image; No. 2 represents the locally enlarged image cropped from the blurred image; No. 3 represents the real image of the locally enlarged image; Nos. 4-8 represent the deblurred images obtained by using the HINet, DeepRFT, Stripformer, MSDI-Net, and NAFNet methods in sequence; No. 9 represents the deblurred image obtained by using the method of the present invention);

[0077] Figure 9 The visual comparison diagram of an embodiment of an image blind deblurring method based on fuzzy kernel prior learning of the present invention and different existing methods on the HIDE dataset (No. 1 represents the input blurred image; No. 2 represents the locally enlarged image cropped from the blurred image; No. 3 represents the real image of the locally enlarged image; Nos. 4-8 represent the deblurred images obtained by using the HINet, DeepRFT, Stripformer, MSDI-Net, and NAFNet methods in sequence; No. 9 represents the deblurred image obtained by using the method of the present invention);

[0078] Figure 10Visual comparison chart of an embodiment of the image blind deblurring method based on fuzzy kernel prior learning of the present invention and different existing methods on the RealBlur dataset (number 1 represents the input blurred image; number 2 represents a locally enlarged view intercepted from the blurred image; number 3 represents the real image of the locally enlarged view; numbers 4-8 represent the deblurred images obtained by using the HINet, DeepRFT, Stripformer, MSDI-Net, and NAFNet methods in sequence; number 9 represents the deblurred image obtained by using the method of the present invention);

[0079] Figure 11 Visual comparison chart of an embodiment of the image blind deblurring method based on fuzzy kernel prior learning of the present invention and different existing methods on the RWBI dataset (number 1 represents the input blurred image; number 2 represents a locally enlarged view intercepted from the blurred image; number 3 represents the real image of the locally enlarged view; numbers 4-8 represent the deblurred images obtained by using the HINet, DeepRFT, Stripformer, MSDI-Net, and NAFNet methods in sequence; number 9 represents the deblurred image obtained by using the method of the present invention). Detailed implementation manners

[0080] The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0081] An image blind deblurring network based on fuzzy kernel prior learning of the present invention, as Figure 3 shown, includes a fuzzy kernel estimation network and a deblurring network.

[0082] The fuzzy kernel estimation network is used to calculate the fuzzy kernel image of the blurred image, as Figure 4 shown, which includes a feature extraction network, an uncertainty learning module, and a normalizing flow model connected in sequence.

[0083] As Figure 5 shown, the deblurring network is used to input the blurred image to be processed and obtain a clear image by using the estimated fuzzy kernel image. It includes a downsampling unit, an intermediate unit, and an upsampling unit connected in sequence; the downsampling unit includes a plurality of downsampling layers connected in sequence, numbered as the first downsampling layer, the second downsampling layer... the m-th downsampling layer in the signal output direction. In this embodiment, m is three; the intermediate unit includes twenty-eight basic modules connected in sequence; the upsampling unit includes a plurality of upsampling layers connected in sequence and having the same number as the downsampling layers, numbered as the first upsampling layer, the second upsampling layer... the m-th upsampling layer in the reverse signal output direction. In this embodiment, m is three.

[0084] The downsampling layer includes a blur kernel attention module, a basic module, and a downsampling module connected in sequence; the upsampling layer includes a basic module and an upsampling module connected in sequence. The input end of the blur kernel attention module of each downsampling layer is connected to the output end of the normalizing flow model, and the other input end of the blur kernel attention module of the first downsampling layer is used to input a blurred image; between adjacent numbered downsampling layers, the output end of the downsampling module of the previous numbered downsampling layer is connected to the other input end of the blur kernel attention module of the next numbered downsampling layer; the output end of the downsampling module of the third downsampling layer is connected to the input end of the basic module at the front end of the intermediate unit; between the upsampling layer and the downsampling layer with the same number, the output end of the downsampling module of the upsampling layer is connected to the input end of the basic module of the downsampling layer. Between adjacent numbered upsampling layers, the input end of the basic module of the previous numbered upsampling layer is connected to the output end of the upsampling module of the next numbered upsampling layer; the input end of the basic module of the third upsampling layer is connected to the output end of the basic module at the end of the intermediate unit; the output end of the upsampling module of the first upsampling layer is used to output a clear image.

[0085] As Figure 6 shown, the blur kernel attention module includes an image feature convolutional layer, a blur kernel feature convolutional layer, a fused feature convolutional layer, a multiplication layer, and an addition layer. The input ends of the blur kernel feature convolutional layers of the three downsampling layers are respectively connected to the output end of the normalizing flow model, and are used to embed the blur kernel images estimated by the blur kernel estimation network into each blur kernel attention module respectively. The output ends of the image feature convolutional layer and the blur kernel feature convolutional layer are respectively connected to the input end of the fused feature convolutional layer; the two input ends of the multiplication layer are respectively connected to the output end of the fused feature convolutional layer and the output end of the image feature convolution, and are used to perform a multiplication operation on the output result of the fused feature convolutional layer and the output result of the image feature convolution; the two input ends of the addition layer are respectively connected to the output end of the multiplication layer and the input end of the image feature convolution, and are used to perform an addition operation on the output result of the multiplication layer and the blurred image to obtain the output result of the blur kernel attention module. The input end of the image feature convolutional layer of the blur kernel attention module of the first downsampling layer is used to input a blurred image, and the input ends of the image feature convolutional layers of the blur kernel attention modules of other sampling layers are respectively connected to the output end of the downsampling module of the previous numbered sampling layer; the output end of the addition layer of each blur kernel attention module is connected to the input end of the basic module of the same blur kernel attention module.

[0086] The basic module includes a first LayerNorm layer, a first convolutional layer, a second convolutional layer, a first SimpleGate layer, a Simplified Channel Attention layer, a third convolutional layer, a second LayerNorm layer, a fourth convolutional layer, a second SimpleGate layer, and a fifth convolutional layer connected in sequence; the Simplified Channel Attention layer includes an AveragePooling layer and a sixth convolutional layer connected; the first SimpleGate layer is used to divide the image feature map into two parts on average by channel, and multiply these two parts by pixel position to obtain an output result; the first SimpleGate layer and the second SimpleGate layer have the same structure.

[0087] In the blur kernel estimation network, the feature extraction network includes an encoder residual module, a downsampling layer, an intermediate convolutional layer, an upsampling layer, and a decoder residual module connected in sequence. Among them, both the encoder residual module and the decoder residual module include a seventh convolutional layer, a ReLIU activation function layer, an eighth convolutional layer, a ReLIU activation function layer, and a ninth convolutional layer connected in sequence. The normalizing flow model includes twenty flowblocks connected in sequence; the flowblock includes a batchnormalization layer, a permutationlayer, and an affine transformation layer connected in sequence.

[0088] The present invention also provides an image blind deblurring method based on blur kernel prior learning for implementing the above-mentioned image blind deblurring network based on blur kernel prior learning, as Figure 7 shown, including the following steps:

[0089] 1) Obtain a blurred image set Y and a corresponding clear image set X;

[0090] Obtain the image domain R from the existing image database L , R L represents an image domain containing L pixel points, and the image domain R L contains the blurred image set Y and the corresponding clear image set X, and is defined as follows: the blurred image set Y = {Y 1 , Y 2 , …, Y i , …, Y N}, the clear image set X = {X 1 , X 2 , …, X i , …, X N}, where Yi and Xi are the i-th blurred image and the corresponding i-th clear image respectively, 1 ≤ i ≤ N, and N is the number of blurred image and clear image pairs. In this embodiment, 2103 blurred images and 2103 clear images are selected, and the blurred images and clear images are paired. R L where L = 256×256 in R

[0091] 2) Use the method of generating random trajectories to obtain a set of blur kernel images K containing M blur kernel images. The set of blur kernel images contains different blur kernel images k, and M ≥ 10000. In this embodiment, M is 50000 images. The blur kernel image samples are shown as Figure 1 shown. The number of pixels of the blur kernel image k is 19×19.

[0092] 3) Construct the above-mentioned image blind deblurring network based on blur kernel prior learning;

[0093] Construct a normalizing flow model: First, determine the number of input channels and output channels in the normalizing flow model. The number of input channels and output channels in the normalizing flow model is equal to the dimensionality of the blur kernel image. In this embodiment, the dimensionality of the blur kernel image is 1, so the number of input channels is 1 and the number of output channels is 1. The normalizing flow model is composed of a total of twenty identical flowblocks stacked together. Each flowblock consists of a batch normalization layer, a permutation layer, and an affine transformation layer connected in sequence.

[0094] Construct a blur kernel estimation network: First, determine the number of input channels and output channels of the blur kernel estimation network. The number of input channels of the blur kernel estimation network should be equal to the dimensionality of the blurred image, and the number of output channels should be equal to the dimensionality of the blur kernel. In this embodiment, the dimensionality of the blurred image is 3 and the dimensionality of the blur kernel image is 1, so the number of input channels is 3 and the number of output channels is 1. The blur kernel estimation network consists of a feature extraction network, an uncertainty learning module, and a normalizing flow model.

[0095] The feature extraction network includes an encoder residual module, a downsampling layer, an intermediate convolutional layer, an upsampling layer, and a decoder residual module connected in sequence. Among them, both the encoder residual module and the decoder residual module include a seventh convolutional layer, a ReLU activation function layer, an eighth convolutional layer, a ReLU activation function layer, and a ninth convolutional layer connected in sequence. The feature extraction network estimates the latent encoding and variance of the blur kernel in the latent space. The uncertainty learning module performs Gaussian resampling on the latent encoding and variance to obtain the final latent encoding.

[0096] Constructing a Deblurring Network: First, determine the number of downsampling layers, intermediate layers, and upsampling layers of the deblurring network: In this embodiment, the number of downsampling layers of the deblurring network is 3, the number of intermediate layers (the number of basic modules of the intermediate unit) is 28, and the number of upsampling layers is 3. The deblurring network consists of a blur kernel attention module, a basic module, a downsampling module, and an upsampling module. There are a total of three blur kernel attention modules, namely the first blur kernel attention module, the second blur kernel attention module, and the third blur kernel attention module. The blur kernel attention module sequentially includes an image feature convolution layer Conv1, a blur kernel feature convolution layer Conv2, a fused feature convolution layer Conv3, a multiplication layer, and an addition layer. The network contains a total of thirty-four basic modules. The basic module includes a first LayerNorm layer, a first convolution layer, a second convolution layer, a first SimpleGate layer, a Simplified Channel Attention layer, a third convolution layer, a second LayerNorm layer, a fourth convolution layer, a second SimpleGate layer, and a fifth convolution layer connected in sequence; the Simplified Channel Attention layer includes an AveragePooling layer and a sixth convolution layer; the first SimpleGate layer is used to divide the image feature map into two parts on average by channel, and multiply these two parts by pixel position to obtain an output result; the first SimpleGate layer and the second SimpleGate layer have the same structure.

[0097] 4) Randomly initialize the weights and biases of the normalizing flow model, the blur kernel estimation network, and the deblurring network;

[0098] Respectively determine the learning rate η, the activation function f(z), and initialize the weights W (t) and biases b (t) , where t represents the layer index of the neural network, and t = 1, 2, …, n, and n represents the total number of layers of the neural network. In this embodiment, the learning rate η = 0.001 is set, and the activation function f(z) is the ReLU function.

[0099] 5) Conduct training

[0100] 5.1) Training of the normalizing flow model: Randomly select a blur kernel image k from the blur kernel image set K as an input training sample, and train the normalizing flow model with the negative log-likelihood loss function;

[0101] The negative log-likelihood loss function L(k; θ) is:

[0102]

[0103] f θ(k) represents a normalizing flow model with parameter θ, and its input is a blurred kernel image randomly selected from the set K of blurred kernel images;

[0104] represents calculating the Jacobian matrix of the normalizing flow model;

[0105] p Z (f θ (k)) represents calculating the density function for f θ (k).

[0106] After training is completed, the normalizing flow model can establish a two-way mapping between the blurred kernel image and the corresponding latent variable z, as Figure 2 shown.

[0107] 5.2) Training of the blurred kernel estimation network model: Randomly select a blurred image y i from the set Y of blurred images as the input sample, and at the same time select the corresponding clear image x i from the set X of clear images as the reference sample. To overcome the problem that the blurred kernel has no true label, a self-supervised method is proposed to estimate the blurred kernel, and the L1 loss between the blurred image and the re-blurred image is adopted. The blurred kernel estimation network is trained using the self-supervised loss function;

[0108] The self-supervised loss function L KE is:

[0109]

[0110] x i and y i represent the i-th pair of clear image and blurred image respectively;

[0111] N represents the total number of training samples;

[0112] G(y i ) represents the final latent code estimated from y i , that is, the final latent code z output by the uncertainty module; i ;

[0113] f θ [G(y i )] is the blurred kernel decoded by the normalizing flow model.

[0114] 5.3) Deblurring network training: Randomly select a blurred image y i from the set Y of blurred images as the input sample, and at the same time select the corresponding clear image x i from the set X of clear images as the reference sample. First, the blurred image y iInput it into the blur kernel estimation network to obtain the blurred kernel image k, and then train the deblurring network with the reconstruction loss function; the reconstruction loss function L recon is the re-blurring loss function L reblur and the weighted sum of the PSNR loss function L PSNR :

[0115] L rec o n = L PSNR + λL reblur

[0116] where λ is the weight of the re-blurring loss function, and λ is 0.01;

[0117] The re-blurring loss function L reblur and the PSNR loss function L PSNR are respectively:

[0118]

[0119]

[0120] F(y i ) is the reconstructed clear image;

[0121] K(y i ) represents the blurred kernel estimated from the blurred image using the blur kernel estimation network;

[0122] PSNR(F(y i ), x i ) represents calculating the peak signal-to-noise ratio for F(y i ) and x i .

[0123] 6) Image deblurring

[0124] 6.1) Input the blurred image to be processed into the trained blur kernel estimation network to obtain the estimated blurred kernel image;

[0125] 6.1.1) Input the blurred image into the feature extraction network, and the feature extraction network calculates the hidden encoding z i of each pixel in the blurred image and predicts the standard deviation σ i of the hidden encoding, where z i ~N(0, I) follows the standard normal distribution;

[0126] 6.1.2) Because the uncertainty learning module is introduced in the blur kernel estimation network, the feature extraction network sends the hidden encoding z i and the standard deviation σ i into the uncertainty learning module for Gaussian resampling to obtain each hidden encoding z iUncertainty component:

[0127]

[0128] where n i is the same size as z i and then transforms the standard deviation of z through i Assuming that z i and n i are independent of each other, the final latent code is obtained by summation The final latent code satisfies the standard normal distribution:

[0129]

[0130] where represents the main body component of the latent code z i and n i is the uncertainty component;

[0131] 6.1.3) The uncertainty learning module sends the final latent code into the normalizing flow model and outputs the blurred kernel image.

[0132] 6.2) The blurred image to be processed and the corresponding blurred kernel image k are simultaneously input into the deblurring network, and the clear image is output.

[0133] The technical effects of the present invention are specifically described below through simulation experiments:

[0134] 1. Simulation conditions:

[0135] 1) The paired blurred images and clear images in the simulation experiment are obtained from the public datasets GoPro, RealBlur, and HIDE;

[0136] 2) The programming platform used in the simulation experiment is Python;

[0137] 3) The structure of the image blind deblurring network based on blurred kernel prior learning constructed in the simulation experiment is as Figure 3 shown;

[0138] 4) In the simulation experiment, the peak signal-to-noise ratio PSNR index and the structural similarity SSIM index are used to evaluate the denoising result

[0139] The definition of the peak signal-to-noise ratio PSNR is:

[0140]

[0141] where MSE represents the mean square error of the image after deblurring.

[0142] The structural similarity SSIM index is defined as:

[0143] SSIM(X,Y) = L(X,Y) * C(X,Y) * S(X,Y),

[0144]

[0145]

[0146]

[0147] where, μ X , μ Y represent the means of images X and Y respectively, σ X , σ Y represent the standard deviations of images X and Y respectively, represent the variances of images X and Y respectively. σ XY represents the covariance of images X and Y. C 1 , C 2 and C 3 are constants, which are used to avoid the denominator being zero and maintain stability.

[0148] (1) To prove the effectiveness of the motion prior based on normalizing flow in the blur kernel estimation, the normalizing flow model of the blur kernel estimation network was removed. This simplified network (baseline) directly estimates the blur kernel instead of estimating the latent code. To measure the accuracy of the estimated blur kernel, the PSNR and SSIM results between the original blurred image and the blurred image reconstructed using these blur kernel estimation methods were first compared. As shown in Table 1, a tick indicates the network used. Using the simplified network (baseline) of the present invention for blur kernel estimation has higher results than the traditional method Whyte et al., and adding the normalizing flow to the network can further improve the estimation accuracy. Adding the normalizing flow and uncertainty learning to the simplified network can also improve the PSNR and SSIM results.

[0149] (1) To prove the effectiveness of the motion prior based on normalizing flow in the blur kernel estimation, the normalizing flow model of the blur kernel estimation network was removed. This simplified network (baseline) directly estimates the blur kernel instead of estimating the latent code. To measure the accuracy of the estimated blur kernel, the PSNR and SSIM results between the original blurred image and the blurred image reconstructed using these blur kernel estimation methods were first compared. As shown in Table 1, a tick indicates the network used. Using the simplified network (baseline) of the present invention for blur kernel estimation has higher results than the traditional method Whyte et al., and adding the normalizing flow to the network can further improve the estimation accuracy. Adding the normalizing flow and uncertainty learning to the simplified network can also improve the PSNR and SSIM results.

[0150] Table 1. PSNR and SSIM results between the original blurred image and the reconstructed blurred image

[0151]

[0152] (2) To prove the effectiveness of uncertainty learning (UL), the blur kernel estimation network was improved into a deterministic model by removing the variance branch and random noise components of the network. Similarly, the blur results in Table 1 were compared, and when uncertainty learning was introduced, the results of blur kernel estimation could be significantly improved. Then, the estimated blur kernel image was fused into the deblurring network, and the deblurring results are shown in Table 2 (image datasets were obtained from GoPro, HIDE, RealBlur-R, and RealBlur-J respectively). The introduction of uncertainty learning can not only improve the accuracy of blur kernel estimation but also improve the deblurring performance, indicating that the closer the estimated blur kernel is to the real situation, the better the deblurring result.

[0153] Table 2. Deblurring Results Using Different Blur Kernel Estimation Methods

[0154]

[0155] (3) To prove the effectiveness of the proposed blur kernel estimation module (kernel estimation, KE) based on normalizing flow in improving image deblurring performance, some other image deblurring networks were upgraded because the blur kernel estimation network based on normalizing flow and the multi-scale blur kernel attention module can be easily embedded into the encoder-decoder structure. As shown in Table 3 (image datasets were obtained from GoPro and HIDE respectively), after estimating the blur kernel using the method of the present invention and embedding it into the deblurring network, the deblurring results were significantly improved.

[0156] Table 3. Deblurring Results of the Blur Kernel Estimation Module on Different Methods

[0157]

[0158] (2) To prove the effectiveness of the blind image deblurring method based on latent space prior non-uniform blur kernel estimation proposed by the present invention, it was compared with existing blind deblurring methods.

[0159] The network proposed by the present invention was trained on the GoPro dataset, which consists of 2103 pairs of clear and blurred images as the training set. For evaluation, the method of the present invention was tested on the GoPro, HIDE, RealBlur-R, and RealBlur-J test sets. It was also trained and tested on the RealBlur-R and RealBlur-J datasets.

[0160] The present invention compares the proposed method with several of the current best blind image deblurring methods. The PSNR and SSIM results of the image deblurring methods are shown in Table 4. The proposed method outperforms other existing methods on each test set. The method proposed by the present invention improves by 0.37 dB in terms of PSNR compared with the existing best-performing NAFNet method on the GoPro dataset. At the same time, the present invention is also tested on the HIDE dataset. To demonstrate the generalization characteristics and effectiveness of the motion prior based on normalizing flow, the method of the present invention is further evaluated on the RealBlur-R and RealBlur-J datasets. As shown in Table 4, compared with other methods, the processing results of the method of the present invention for real blurred images have been significantly improved. All the above-mentioned models are trained on the GoPro training set, demonstrating the excellent generalization performance of the method of the present invention from GoPro to other real blurred datasets. It is also trained and tested on the RealBlur dataset, and the results are shown in Table 5.

[0161] Table 4. Comparison results on the benchmark test datasets

[0162]

[0163] Table 5. Comparison results on the RealBlur test dataset

[0164]

[0165] In Figure 8 、 9 、10, the visualized results of image deblurring produced by different methods are compared. As can be seen from Figure 8 , on the GoPro dataset, the method proposed by the present invention can recover more texture information and clearer edges than other methods. As Figure 9 shows, the method of the present invention can recover more natural body features on the HIDE dataset. As Figure 10 shows, the method of the present invention has achieved good results in removing motion blur in real-scene images. Figure 11 shows the visualized results of various methods on the RWBI dataset, which only contains real blurred images without clear labeled images. It can be observed from the figure that compared with other methods, the method of the present invention can achieve higher reconstruction quality and recover more texture and edge details.

Claims

1. An image blind deblurring network based on fuzzy kernel prior learning, characterized in that: It includes a fuzzy kernel estimation network and a deblurring network; The fuzzy kernel estimation network is used to estimate the fuzzy kernel image of the blurred image, and it includes a feature extraction network, an uncertainty learning module, and a normalizing flow model connected in sequence; The deblurring network is used to input the blurred image to be processed and obtain a clear image by using the estimated fuzzy kernel image. It includes a downsampling unit, an intermediate unit, and an upsampling unit connected in sequence; the downsampling unit includes a plurality of downsampling layers connected in sequence, numbered as the first downsampling layer, the second downsampling layer... the m-th downsampling layer in sequence along the signal output direction; the intermediate unit includes a plurality of basic modules connected in sequence; the upsampling unit includes a plurality of upsampling layers connected in sequence and having the same number as the downsampling layers, numbered as the first upsampling layer, the second upsampling layer... the m-th upsampling layer in sequence along the reverse direction of the signal output; The downsampling layer includes a fuzzy kernel attention module, a basic module, and a downsampling module connected in sequence; The upsampling layer includes a basic module and an upsampling module connected in sequence; The input end of the fuzzy kernel attention module of each downsampling layer is connected to the output end of the normalizing flow model, and the other input end of the fuzzy kernel attention module of the first downsampling layer is used to input the blurred image; between adjacent numbered downsampling layers, the output end of the previous numbered downsampling module is connected to the other input end of the fuzzy kernel attention module of the next numbered layer; the output end of the downsampling module of the m-th downsampling layer is connected to the input end of the basic module at the front end of the intermediate unit; between the upsampling layer and the downsampling layer with the same number, the output end of the downsampling module of the upsampling layer is connected to the input end of the basic module of the downsampling layer; Between adjacent numbered upsampling layers, the input end of the previous numbered basic module is connected to the output end of the next numbered upsampling module; the input end of the basic module of the m-th upsampling layer is connected to the output end of the basic module at the end of the intermediate unit; the output end of the upsampling module of the first upsampling layer is used to output the clear image.

2. The image blind deblurring network based on fuzzy kernel prior learning according to claim 1, characterized in that: The fuzzy kernel attention module includes an image feature convolution layer, a fuzzy kernel feature convolution layer, a fusion feature convolution layer, a multiplication layer, and an addition layer; the input end of the fuzzy kernel feature convolution layer is connected to the output end of the normalizing flow model; the output ends of the image feature convolution layer and the fuzzy kernel feature convolution layer are respectively connected to the input end of the fusion feature convolution layer; the two input ends of the multiplication layer are respectively connected to the output end of the fusion feature convolution layer and the output end of the image feature convolution, and are used to perform a multiplication operation on the output result of the fusion feature convolution layer and the output result of the image feature convolution; the two input ends of the addition layer are respectively connected to the output end of the multiplication layer and the input end of the image feature convolution, and are used to perform an addition operation on the output result of the multiplication layer and the blurred image to obtain the output result of the fuzzy kernel attention module; The input end of the image feature convolution layer of the blur kernel attention module in the first downsampling layer is used to input the blurred image, and the input ends of the image feature convolution layers of the blur kernel attention modules in other sampling layers are respectively connected to the output ends of the downsampling modules in the previous numbered sampling layer; The output end of the addition layer is connected to the input end of the basic module in the same blur kernel attention module.

3. An image blind deblurring network based on blur kernel prior learning according to claim 1 or 2, characterized in that: The basic module includes a first LayerNorm layer, a first convolutional layer, a second convolutional layer, a first SimpleGate layer, a Simplified Channel Attention layer, a third convolutional layer, a second LayerNorm layer, a fourth convolutional layer, a second SimpleGate layer, and a fifth convolutional layer connected in sequence; The Simplified Channel Attention layer includes an AveragePooling layer and a sixth convolutional layer connected; The first SimpleGate layer is used to divide the image feature map into two parts on average by channel, and multiply these two parts according to the pixel positions to obtain the output result; The first SimpleGate layer and the second SimpleGate layer have the same structure.

4. An image blind deblurring network based on blur kernel prior learning according to claim 3, characterized in that: The feature extraction network includes an encoder residual module, a downsampling layer, an intermediate convolutional layer, an upsampling layer, and a decoder residual module connected in sequence; Among them, both the encoder residual module and the decoder residual module include a seventh convolutional layer, a ReLIU activation function layer, an eighth convolutional layer, a ReLIU activation function layer, and a ninth convolutional layer connected in sequence; The normalizing flow model includes twenty flowblocks connected in sequence; The flowblock includes a batch normalization layer, a permutation layer, and an affine transformation layer connected in sequence; The number of downsampling layers is three, numbered as the first downsampling layer, the second downsampling layer, and the third downsampling layer in sequence along the signal output direction; The number of upsampling layers is three, numbered as the first upsampling layer, the second upsampling layer, and the third upsampling layer in sequence along the reverse direction of the signal output; The intermediate unit includes twenty-eight basic modules connected in sequence.

5. An image blind deblurring method based on blur kernel prior learning, characterized by comprising the following steps: 1) Obtain a blurred image set Y and a corresponding clear image set X; 2) Use the method of random trajectory generation to obtain a blur kernel image set; 3) Construct an image blind deblurring network based on blur kernel prior learning according to any one of claims 1-4; 4) Randomly initialize the weights and biases of the normalizing flow model, the blur kernel estimation network, and the deblurring network; 5) Conduct training 5.1) Standardized flow model training: Randomly select a blurred kernel image from the blurred kernel image set K as the input training sample, and use the negative log-likelihood loss function to train the standardized flow model; 5.2) Blurred kernel estimation network training: Randomly select a blurred image from the blurred image set Y as the input sample, and at the same time select the corresponding clear image from the clear image set X as the reference sample, and use the self-supervised loss function to train the blurred kernel estimation network; 5.3) Deblurring network training: Randomly select a blurred image from the blurred image set Y as the input sample, and at the same time select the corresponding clear image from the clear image set X as the reference sample. First, input the blurred image into the blurred kernel estimation network to output the blurred kernel image, and then use the reconstruction loss function to train the deblurring network; 6) Image deblurring 6.1) Input the blurred image to be processed into the blurred kernel estimation network to obtain the estimated blurred kernel image; 6.2) Input the blurred image to be processed and the corresponding blurred kernel image into the deblurring network at the same time, and output the clear image.

6. The image blind deblurring method based on blurred kernel prior learning according to claim 5, wherein: In step 1), the set of blurred images Y = {Y 1 , Y 2 , …, Y i ,..., Y N}, and the set of clear images X = {X 1 , X 2 , …, X i , …, X N}. Y i and X i are the i-th blurred image and the corresponding i-th clear image respectively, where 1 ≤ i ≤ N, and N is the number of blurred image and clear image pairs.

7. The image blind deblurring method based on blurred kernel prior learning according to claim 6, wherein: In 5.1), the negative log-likelihood loss function L(k; θ) is: f θ (k) represents a normalizing flow model with parameter θ, whose input is a blurred kernel image randomly selected from the set K of blurred kernel images; Denotes the Jacobian matrix of the normalizing flow model; p Z (f θ (k)) represents the calculation of the density function for f θ (k).

8. The image blind deblurring method based on blurred kernel prior learning according to claim 7, wherein: In 5.2), the self-supervised loss function L KE is as follows: x i and y i represent the i-th pair of clear and blurred image pairs; G(y i ) represents the final latent code estimated from y i ; f θ [G(y i )] is the blurred kernel image decoded by the standardized flow model.

9. The image blind deblurring method based on blurred kernel prior learning according to claim 8, wherein: In step 5.3), the reconstruction loss function L recon is the weighted sum of the deblurring loss function L reblur and the PSNR loss function L PSNR : L recon = L PSNR + λL reblur , where λ is the weight of the deblurring loss function, and the value of λ is 0.01; Heavy blur loss function L reblur and PSNR loss function L PSNR are respectively as follows: F(y i ) is the reconstructed clear image; K(y i ) represents the blurred kernel image estimated from the blurred image using the blurred kernel estimation network; PSNR(F(y i ), x i ) represents calculating the peak signal-to-noise ratio for F(y i ) and x i .

10. The image blind deblurring method based on blurred kernel prior learning according to claim 9, wherein, Step 6.1) is specifically: 6.1.1) Input the blurred image into the feature extraction network, and the feature extraction network calculates the latent code z of each pixel in the blurred image i and predicts the standard deviation σ of the latent code i , where z i ~N(0, I) follows the standard normal distribution; 6.1.2) The feature extraction network sends the latent code z i and the standard deviation σ i into the uncertainty learning module for Gaussian resampling to obtain the uncertainty component n of each latent code z i : i : where n i is the same size as z i and z i and n i are defined to be independent of each other, and the final latent code is obtained by summation which satisfies the standard normal distribution: wherein represents the body component of the latent code z i and 6.1.3) The uncertainty learning module feeds the final latent encoding into the normalizing flow model to output a blurred kernel image.

Images