Depth non-blind image deblurring method combining blurring and noise auto-encoder

Abstract

The invention discloses a depth non-blind image deblurring method of a combined blurring and noise auto-encoder, aims to restore a blurred image by combining a deblurring task and a denoising task, and mainly comprises three processing processes, namely depth feature extraction of the blurred image and noise thereof, depth feature deblurring and depth deblurring feature reconstruction. Firstly, in a pre-training stage, a blurred image auto-encoder and a noise auto-encoder are trained to accurately initialize depth features of a blurred image and noise; then, in the overall training stage, depth feature deblurring and feature reconstruction in an image feature domain are achieved, noise in blurred image features is effectively processed, and structural information such as textures and edges of images can be well recovered; experimental results show that the image quality can be greatly improved, the high efficiency of the algorithm is kept, and the method is obviously superior to an existing mainstream image deblurring method.

Description

Technical Field

[0001] The present invention relates to a deep non-blind image deblurring method, in particular to a deep non-blind image deblurring method of a joint blur and noise autoencoder. Background Art

[0002] Non-blind image deblurring is an important topic in the field of image processing, which aims to restore blurred images with noise through known blur kernels. Traditional non-blind image deblurring methods usually rely on the distribution characteristics of structured information such as image edges and textures to design corresponding priors, and usually optimize the deblurring effect by manually setting parameters. However, the manual parameter adjustment process increases the complexity of the algorithm in practical applications. In addition, traditional methods often have poor adaptability when dealing with different types of blur and noise, and it is difficult to meet diverse application requirements.

[0003] In recent years, with the improvement of hardware computing power and the development of parallel computing technology, deep learning has been widely used in non-blind image deblurring algorithms. It no longer relies on manually designed priors, but trains the network through large-scale data to automatically learn the deblurring rules. Representative deep learning deblurring algorithms include deblurring algorithms based on multi-scale recurrent networks and deblurring methods based on deep denoiser priors. Although these methods show better performance and faster processing speed with the help of GPU acceleration capabilities, they still face some challenges. For example, the deblurring algorithm based on multi-scale recurrent networks mainly focuses on the overall blur information of the image, often ignoring information such as texture and edges in the image, and fails to consider the problem of noise signal amplification during the deblurring process, which further reduces the image quality. The method based on deep denoiser priors usually assumes that the standard deviation of the noise in the blurred image is known, and requires training independent denoising models for different noise variances, which not only increases the complexity of the training model, but also reduces its adaptability to deal with unknown noise during the deblurring process. Summary of the invention

[0004] The present invention aims to solve the above-mentioned technical problems existing in the prior art and provides a deep non-blind image deblurring method of a joint blur and noise autoencoder.

[0005] The technical solution of the present invention is: a deep non-blind image deblurring method of a joint blur and noise autoencoder, which is performed according to the following steps:

[0006] Step 1. Generate blur kernels and preprocess the images of the training set by adding Gaussian noise. The training set is recorded as: where x i For a clear image, y i is a noisy blurred image, is a noise-free blurred image, ki is the blur kernel, n i The noise contained in the blurred image, N train is the number of training sets;

[0007] Step 2. Define the overall network structure consisting of a fuzzy autoencoder, a noise autoencoder, a noise estimator, a deep feature deblurring and reconstruction network. The specific steps are as follows:

[0008] Step 2.1 Define the fuzzy autoencoder structure

[0009] The fuzzy autoencoder is composed of a fuzzy image encoder and a fuzzy image decoder. The fuzzy image encoder is constructed by a convolutional layer with a filter size of 3×3, a step size of 1, an input channel of 3, and an output channel of 16, and three residual blocks with a filter size of 3×3, and both the input channel and the output channel of 16. The residual block is composed of two convolutional layers and a ReLU activation function; the fuzzy image decoder is a symmetrical structure of the fuzzy image encoder; the fuzzy image encoder is to convert the noise blurred image y i Encoded into the corresponding deep features The blur image decoder is to transform the depth features Decoded as reconstructed blurred image Let the blurred image encoder be f y (.), the learnable parameter is W y , the blurred image decoder is The learnable parameters are The self-encoding process of the fuzzy autoencoder is abstracted as:

[0010] Step 2.2 Define the noise autoencoder structure

[0011] The noise autoencoder includes a noise encoder and a noise decoder, and its structure is consistent with that of the fuzzy autoencoder. The noise encoder converts the noise n in the training set into i Encoded as its corresponding deep feature The noise decoder is a deep feature Decoded as reconstructed noise Let the noise encoder be f n (.), the learnable parameter is W n , the noise decoder is The learnable parameters are The encoding and decoding process of the noise autoencoder is abstracted as follows:

[0012] Step 2.3 Define the noise estimator structure

[0013] The noise estimator includes three median filters of size 3×3, 5×5, and 7×7, a convolutional layer with a filter size of 3×3, a step size of 1, and an output channel of 16, and three residual blocks, which are composed of two convolutional layers and a ReLU activation function; the noise estimator is to accurately extract the blurred image y i Noise depth features in First, the noise blurred image y i Use three median filters to filter the blurred image respectively, then subtract the noisy blurred image from the filtered image, then fuse the three subtracted images in the channel dimension, and then send them to the convolution layer to obtain the shallow features of the noise, and finally send the shallow features of the noise to the three residual blocks to obtain the final deep features of the noise; let the noise estimator be f z (.), the learnable parameter is W z , then the execution process of the noise estimator can be abstracted as:

[0014] Step 2.4 Define the deep feature deblurring structure

[0015] The deep feature deblurring consists of alternating 7 convolutional layers with a filter size of 5×5 and 64 output channels and 7 ReLU activation functions, and the last convolutional layer with a filter size of 5×5 and 16 output channels is used for output; the deep feature deblurring receives the features output by the blur encoder and the characteristics of the noise estimator output As input, the output is the deblurred deep feature Solve the objective function using the ADMM algorithm: in, is the depth feature of the denoised blurred image, λ is the regularization term coefficient, and R(.) is the regularization term;

[0016] Step 2.5 Define the reconstructed network structure

[0017] The reconstruction network consists of a UNet network including 3 encoders and decoders; each group of encoders consists of a convolutional layer with a filter size of 5×5 and three residual blocks, the output channels are 32, 64, and 128, and the stride of the convolutional layer is 1, 2, and 2 respectively; the first two groups of decoders are composed of 3 residual blocks and a transposed convolutional layer with a filter size of 4×4 and a stride of 2, and the last group is composed of 3 residual blocks and a convolutional layer with a filter size of 5×5 and a stride of 1; at the same time, each encoder sends its output features to its corresponding decoder through a jump connection; the reconstruction network receives the deblurred features As input, reconstruct the deblurred image Let the reconstructed network be The learnable parameters are The feature reconstruction process can be expressed as:

[0018] Step 3. In the pre-training stage, the fuzzy autoencoder and the noise autoencoder are trained. The specific steps are as follows:

[0019] Step 3.1 Randomly initialize network parameters, start training the fuzzy autoencoder, and update parameters; the agreed variable epoch records the number of current network training iterations, and epoch is initially 0;

[0020] Step 3.1.1 Calculate the l1 loss function Perform back propagation and re-update network parameters through the Adam optimization algorithm;

[0021] Step 3.1.2 This round of iteration ends, set epoch = epoch + 1; when epoch reaches the set round, the network stops training and saves the fuzzy autoencoder model and parameters; otherwise, go to step 3.1 again and repeat the training of the network;

[0022] Step 3.2 Randomly initialize the network parameters, start training the noise autoencoder, and update the parameters; the agreed variable epoch records the number of current network training iterations, and epoch is initially 0;

[0023] Step 3.2.1 Calculate the l1 loss function Perform back propagation and re-update network parameters through the Adam optimization algorithm;

[0024] Step 3.2.2 This round of iteration ends, set epoch = epoch + 1; when epoch reaches the set round, the network stops training and saves the noise autoencoder model and parameters; otherwise, go to step 3.2 again and repeat the network training cycle;

[0025] Step 4. Freeze the parameters of the fuzzy autoencoder and the noise autoencoder and train the overall network. The specific steps are as follows:

[0026] Step 4.1 Randomly initialize network parameters;

[0027] Step 4.2 starts training the entire network and updates the parameters; the agreed variable epoch records the number of iterations of the current network training, and epoch is initially 0;

[0028] Step 4.2.1 calculates five l1 loss functions, which are: (1) constraining the blurred image autoencoder: (2) Constrain the noise depth features estimated by the noise estimator: (3) Constrain the noise of the noise estimator: (4) Constrain the deblurred image and clean image output by the reconstruction module: (5) Constrain the denoised features and the features of the noise-free blurred image: in, For noise-free blurred images Features output by the fuzzy image encoder After summing, the gradient is calculated for back propagation, and the overall network parameters are updated through the Adam optimization algorithm;

[0029] Step 4.2.2: This round of iteration ends, and epoch is set to epoch+1. When epoch reaches the agreed round, the network stops training and the overall network model and parameters are saved. Otherwise, enter step 4.2 again and repeat the training of the overall network.

[0030] In order to solve the problem that non-blind deblurring methods amplify noise signals during the deblurring process, the present invention proposes a deep non-blind deblurring method that combines blur and noise autoencoders, aiming to restore blurred images by combining deblurring and denoising tasks. The method mainly includes three processing steps, namely, deep feature extraction of blurred images and their noise, deep feature deblurring, and deep deblurred feature reconstruction. First, in the pre-training stage, the blurred image autoencoder and the noise autoencoder are trained to accurately initialize the deep features of the blurred image and noise; then, in the overall training stage, deep feature deblurring and feature reconstruction are implemented in the image feature domain, effectively processing the noise in the blurred image features and better restoring the image's texture, edge and other structural information. Experimental results show that the present invention can significantly improve the image quality while maintaining the high efficiency of the algorithm, which is significantly better than the existing mainstream image deblurring methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is an overall network structure diagram of an embodiment of the present invention.

[0032] Figure 2 It is a structural diagram of a fuzzy / noise autoencoder according to an embodiment of the present invention.

[0033] Figure 3 It is a structural diagram of a noise estimator according to an embodiment of the present invention.

[0034] Figure 4 It is a structural diagram of a deep feature deblurring network according to an embodiment of the present invention.

[0035] Figure 5 This is a diagram of a reconstructed network structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] A deep non-blind image deblurring method of the present invention using a combined blur and noise autoencoder is performed according to the following steps:

[0037] Step 1. Use 5144 color images as training sets, including 400 images from the BSDS500 dataset and all images from the Waterloo Exploration data. Perform preprocessing operations such as blur kernel generation and adding Gaussian noise on the above two training sets before entering the network. The specific steps are as follows:

[0038] Step 1.1 randomly cut the training image into image blocks of 256×256 pixels, and use the three-dimensional random trajectory interpolation method to generate a blur kernel with a size between 13×13 and 35×35 pixels; then, use the cut clear image blocks and the blur kernel to synthesize the blurred image, and add Gaussian noise to the blurred image, with the standard deviation of the noise ranging from 0 to 12.75;

[0039] Step 1.2 For the convenience of formal description, the training set is recorded as: where x i For a clear image, y i is a noisy blurred image, is a noise-free blurred image, k i is the blur kernel, n i The noise contained in the blurred image, N train is the number of training sets;

[0040] Step 2. Define the overall network structure as Figure 1 As shown in the figure, it consists of a fuzzy autoencoder, a noise autoencoder, a noise estimator, a deep feature deblurring and reconstruction network. The specific steps are as follows:

[0041] Step 2.1 Define the fuzzy autoencoder structure

[0042] Fuzzy autoencoder Figure 2 As shown in FIG. 1 , the fuzzy image encoder and the fuzzy image decoder are included. The fuzzy image encoder is constructed by using a convolutional layer (Conv) with a filter size of 3×3, a step size of 1, 3 input channels, and 16 output channels, and three residual blocks (ResBlock) with a filter size of 3×3, and both input channels and output channels are 16. The residual block is composed of two convolutional layers and a ReLU activation function. The structure of the fuzzy image decoder is symmetrical with that of the fuzzy image encoder. The fuzzy image encoder is designed to convert the noise blurred image y i Encoded as its corresponding deep feature The blurred image decoder aims to convert deep features Decoded as reconstructed blurred image Let the blurred image encoder be f y (.), whose learnable parameter is W y , the blurred image decoder is Its learnable parameters are The self-encoding process of the fuzzy autoencoder can be abstracted as:

[0043] Step 2.2 Define the noise autoencoder structure

[0044] The noise autoencoder includes a noise encoder and a noise decoder, and the structure is similar to Figure 2 The fuzzy autoencoder shown in the figure has the same structure; the noise encoder is to convert the noise n in the training set into i Encoded as its corresponding deep feature The noise decoder is a deep feature Decoded as reconstructed noise Let the noise encoder be f n (.), the learnable parameter is W n , the noise decoder is The learnable parameters are The encoding and decoding process of the noise autoencoder is abstracted as follows:

[0045] Step 2.3 Define the noise estimator structure

[0046] The noise estimator is Figure 3 As shown in FIG. 1 , it includes three median filtering networks with sizes of 3×3, 5×5, and 7×7, a convolutional layer (Conv) with a filter size of 3×3, a step size of 1, and an output channel of 16, and three residual blocks (ResBlock), wherein the residual block is composed of two convolutional layers and a ReLU activation function; the noise estimator is used to accurately extract the blurred image y i Noise depth features in First, the noise blurred image y i Use three median filters to filter the blurred image respectively, then subtract the noisy blurred image from the filtered image, then fuse the three subtracted images in the channel dimension, and then send them to the convolution layer to obtain the shallow features of the noise, and finally send the shallow features of the noise to the three residual blocks to obtain the final deep features of the noise; let the noise estimator be f z (.), the learnable parameter is W z , then the execution process of the noise estimator can be abstracted as:

[0047] Step 2.4 Define the deep feature deblurring structure

[0048] Deblurring of deep features Figure 4As shown, it consists of 7 alternating convolutional layers (Conv) with a filter size of 5×5 and an output channel of 64 and 7 ReLU activation functions, and the last convolutional layer with a filter size of 5×5 and an output channel of 16 is used for output; the deep feature deblurring receives the features output by the fuzzy encoder and the characteristics of the noise estimator output As input, the output is the deblurred deep feature Solve the objective function using the ADMM algorithm: in, is the depth feature of the denoised blurred image, λ is the regularization term coefficient, and R(.) is the regularization term;

[0049] Step 2.5 Define the reconstructed network structure

[0050] Reconstruct the network as Figure 5 As shown in the figure, it consists of a UNet network including 3 codecs; each group of encoders consists of a convolutional layer with a filter size of 5×5 and a UNet network including 3 codecs; each group of encoders consists of a convolutional layer (Conv) with a filter size of 5×5 and three residual blocks (ResBlock), the output channels are 32, 64, and 128, and the steps of the convolutional layers are 1, 2, and 2 respectively; the first two groups of decoders are composed of 3 residual blocks and a transposed convolutional layer (DeConv) with a filter size of 4×4 and a step size of 2, and the last group is composed of 3 residual blocks and a convolutional layer with a filter size of 5×5 and a step size of 1; at the same time, each encoder sends its output features to its corresponding decoder through a jump connection; the reconstruction network receives the deblurred features As input, reconstruct the deblurred image Let the reconstructed network be Its learnable parameters are The feature reconstruction process can be expressed as:

[0051] Step 3. In the pre-training stage, the fuzzy autoencoder and the noise autoencoder are trained. The specific steps are as follows:

[0052] Step 3.1 Randomly initialize network parameters, start training the fuzzy autoencoder, and update parameters; the agreed variable epoch records the number of current network training iterations, and epoch is initially 0;

[0053] Step 3.1.1 Calculate the l1 loss function Back propagation is performed and the network parameters are updated again using the Adam optimization algorithm with a batch size of 16 and a learning rate of 10 -4 , β=(0.9,0.999), ε=10 -8 ;

[0054] Step 3.1.2 This round of iteration ends, set epoch = epoch + 1; when epoch reaches 50 rounds, the network training stops and the fuzzy autoencoder model and parameters are saved; otherwise, go to step 3.1 again and repeat the network training cycle;

[0055] Step 3.2 Randomly initialize the network parameters, start training the noise autoencoder, and update the parameters; the agreed variable epoch records the number of current network training iterations, and epoch is initially 0;

[0056] Step 3.2.1 Calculate the l1 loss function Back propagation is performed and the network parameters are updated again using the Adam optimization algorithm with a batch size of 16 and a learning rate of 10 -4 , β=(0.9,0.999), ε=10 -8 ;

[0057] Step 3.2.2 This round of iteration ends, set epoch = epoch + 1; when epoch reaches 50 rounds, the network training stops and the noise autoencoder model and parameters are saved; otherwise, go to step 3.2 again and repeat the network training cycle;

[0058] Step 4. Freeze the parameters of the fuzzy autoencoder and the noise autoencoder and train the overall network. The specific steps are as follows:

[0059] Step 4.1 Randomly initialize network parameters;

[0060] Step 4.2 starts training the entire network and updates the parameters; the agreed variable epoch records the number of iterations of the current network training, and epoch is initially 0;

[0061] Step 4.2.1 calculates five l1 loss functions, which are: (1) constraining the blurred image autoencoder: (2) Constrain the noise depth features estimated by the noise estimator: (3) Constrain the noise of the noise estimator: (4) Constrain the deblurred image and clean image output by the reconstruction module: (5) Constrain the denoised features and the features of the noise-free blurred image: in, For noise-free blurred images Features output by the fuzzy image encoder After summing, we calculate the gradient for back propagation and re-update the overall network parameters using the Adam optimization algorithm. The batch size is 8, the learning rate is 0.0001, and the epoch rate is reduced by half every 200 rounds. β = (0.9, 0.999), ε = 10 -8;

[0062] Step 4.2.2: This round of iteration ends, and epoch is set to epoch+1. When the epoch reaches the agreed 400 rounds, the network training stops and the overall network model and parameters are saved. Otherwise, enter step 4.2 again and repeat the training of the overall network.

[0063] Step 5. Test the model. Use the CBSD68 and Sun80 test sets and Lai (4 blur kernels) and Levin (8 blur kernels) blur kernels for preprocessing to obtain the corresponding noise blurred image. Input it into the trained model to obtain the deblurring result. The specific steps are as follows:

[0064] Step 5.1 First, the Levin blur kernel is used to synthesize 544 blurred images for the CBSD68 dataset, which is named the CBSD68-Levin dataset; 640 blurred images are synthesized for the Sun80 dataset, which is named the Sun80-Levin dataset; then, Gaussian noise with standard deviations of 2.55, 7.65, 12.75, and 17.85 are added to both CBSD68-Levin and Sun80-Levin for testing; then, the Lai blur kernel is used to synthesize 272 blurred images for the CBSD68 dataset, which is named the CBSD68-Lai dataset; 320 blurred images are synthesized for the Sun80 dataset, which is named the Sun80-Lai dataset, and Gaussian noise with a standard deviation of 2.55 is added to both CBSD68-Lai and Sun80-Lai for testing;

[0065] Step 5.2 imports the trained model and parameter files into the test code;

[0066] Step 5.3 First, the deep features of the noisy blurred image are estimated through the encoder of the blurred image autoencoder; secondly, the deep features corresponding to the noise of the noisy blurred image are estimated; then, the blur kernel is used to deblur the deep features of the blurred image and the deep features of its noise under the deep features; finally, the deblurred deep features are sent to the reconstruction network to obtain the final deblurred image;

[0067] Step C5.4: The model uses the CBSD68 and Sun80 test sets to compare with the current mainstream non-blind image deblurring methods, and calculates the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). The results on the CBSD68-Levin, Sun80-Levin, CBSD68-Lai and Sun80-Lai test sets are shown in Tables 1, 2, 3 and 4.

[0068] Table 1 Comparison of PSNR and SSIM of the proposed method and other methods on the CBSD68-Levin dataset

[0069]

[0070] Table 2 Comparison of PSNR and SSIM of the proposed method and other methods on the Sun80-Levin dataset

[0071]

[0072]

[0073] Table 3 Comparison of PSNR and SSIM of the proposed method and other methods on the CBSD68-Lai dataset

[0074]

[0075] Table 4 Comparison of PSNR and SSIM of the proposed method and other methods on the Sun80-Lai dataset

[0076]

[0077] Experimental results show that the present invention can significantly improve image quality while maintaining the efficiency of the algorithm, and is significantly superior to existing mainstream image deblurring methods.

Claims

1. A deep non-blind image deblurring method based on a joint blur and noise autoencoder, characterized in that Follow these steps: Step 1. Generate blur kernels and preprocess the images of the training set by adding Gaussian noise. The training set is recorded as: where x i For a clear image, y i is a noisy blurred image, is a noise-free blurred image, k i is the blur kernel, n i The noise contained in the blurred image, N train is the number of training sets; Step 2. Define the overall network structure consisting of a fuzzy autoencoder, a noise autoencoder, a noise estimator, a deep feature deblurring and reconstruction network. The specific steps are as follows: Step 2.1 Define the fuzzy autoencoder structure The fuzzy autoencoder is composed of a fuzzy image encoder and a fuzzy image decoder. The fuzzy image encoder is constructed by a convolutional layer with a filter size of 3×3, a step size of 1, an input channel of 3, and an output channel of 16, and three residual blocks with a filter size of 3×3, and both the input channel and the output channel of 16. The residual block is composed of two convolutional layers and a ReLU activation function; the fuzzy image decoder is a symmetrical structure of the fuzzy image encoder; the fuzzy image encoder is to convert the noise blurred image y i Encoded into the corresponding deep features The blur image decoder is to transform the depth features Decoded as reconstructed blurred image Let the blurred image encoder be f y (.), the learnable parameter is W y , the blurred image decoder is The learnable parameters are The self-encoding process of the fuzzy autoencoder is abstracted as: Step 2.2 Define the noise autoencoder structure The noise autoencoder includes a noise encoder and a noise decoder, and its structure is consistent with that of the fuzzy autoencoder. The noise encoder converts the noise n in the training set into i Encoded as its corresponding deep feature The noise decoder is a deep feature Decoded as reconstructed noise Let the noise encoder be f n (.), the learnable parameter is W n , the noise decoder is The learnable parameters are The encoding and decoding process of the noise autoencoder is abstracted as follows: Step 2.3 Define the noise estimator structure The noise estimator includes three median filters of size 3×3, 5×5, and 7×7, a convolutional layer with a filter size of 3×3, a step size of 1, and an output channel of 16, and three residual blocks, which are composed of two convolutional layers and a ReLU activation function; the noise estimator is to accurately extract the blurred image y i Noise depth features in First, the noise blurred image y i Use three median filters to filter the blurred image respectively, then subtract the noisy blurred image from the filtered image, then fuse the three subtracted images in the channel dimension, and then send them to the convolution layer to obtain the shallow features of the noise, and finally send the shallow features of the noise to three residual blocks to obtain the final deep features of the noise; Let the noise estimator be f z (.), the learnable parameter is W z , then the execution process of the noise estimator can be abstracted as: Step 2.4 Define the deep feature deblurring structure The deep feature deblurring consists of alternating 7 convolutional layers with a filter size of 5×5 and 64 output channels and 7 ReLU activation functions, and the last convolutional layer with a filter size of 5×5 and 16 output channels is used for output; the deep feature deblurring receives the features output by the blur encoder and the characteristics of the noise estimator output As input, the output is the deblurred deep feature Solve the objective function using the ADMM algorithm: in, is the depth feature of the denoised blurred image, λ is the regularization term coefficient, and R(.) is the regularization term; Step 2.5 Define the reconstructed network structure The reconstruction network consists of a UNet network including 3 encoders and decoders; each group of encoders consists of a convolutional layer with a filter size of 5×5 and three residual blocks, the output channels are 32, 64, and 128, and the stride of the convolutional layer is 1, 2, and 2 respectively; the first two groups of decoders are composed of 3 residual blocks and a transposed convolutional layer with a filter size of 4×4 and a stride of 2, and the last group is composed of 3 residual blocks and a convolutional layer with a filter size of 5×5 and a stride of 1; at the same time, each encoder sends its output features to its corresponding decoder through a jump connection; the reconstruction network receives the deblurred features As input, reconstruct the deblurred image Let the reconstructed network be The learnable parameters are The feature reconstruction process can be expressed as: Step 3. In the pre-training stage, the fuzzy autoencoder and the noise autoencoder are trained. The specific steps are as follows: Step 3.1 Randomly initialize network parameters, start training the fuzzy autoencoder, and update parameters; the agreed variable epoch records the number of current network training iterations, and epoch is initially 0; Step 3.1.1 Calculate the l1 loss function Perform back propagation and re-update network parameters through the Adam optimization algorithm; Step 3.1.2 This round of iteration ends, set epoch = epoch + 1; when epoch reaches the set round, the network stops training and saves the fuzzy autoencoder model and parameters; otherwise, go to step 3.1 again and repeat the training of the network; Step 3.2 Randomly initialize the network parameters, start training the noise autoencoder, and update the parameters; the agreed variable epoch records the number of current network training iterations, and epoch is initially 0; Step 3.2.1 Calculate the l1 loss function Perform back propagation and re-update network parameters through the Adam optimization algorithm; Step 3.2.2 This round of iteration ends, set epoch = epoch + 1; when epoch reaches the set round, the network stops training and saves the noise autoencoder model and parameters; otherwise, go to step 3.2 again and repeat the network training cycle; Step 4. Freeze the parameters of the fuzzy autoencoder and the noise autoencoder and train the overall network. The specific steps are as follows: Step 4.1 Randomly initialize network parameters; Step 4.2 starts training the entire network and updates the parameters; the agreed variable epoch records the number of iterations of the current network training, and epoch is initially 0; Step 4.2.1 calculates five l1 loss functions, which are: (1) constraining the blurred image autoencoder: (2) Constrain the noise depth features estimated by the noise estimator: Constrain the noise of the noise estimator: Constrain the deblurred image and clean image output by the reconstruction module: Constrain the denoised features and the features of the noise-free blurred image: in, For noise-free blurred images Features output by the fuzzy image encoder After summing, the gradient is calculated for back propagation, and the overall network parameters are updated through the Adam optimization algorithm; Step 4.2.2: This round of iteration ends, and epoch is set to epoch+1. When epoch reaches the agreed round, the network stops training and the overall network model and parameters are saved. Otherwise, enter step 4.2 again and repeat the training of the overall network.

Images