Degenerated image restoration method based on perception guidance

Abstract

The invention discloses a degraded image recovery method based on perceptual guidance, which comprises the following steps of: firstly, constructing a double-branch neural network architecture comprising a perception branch and a recovery branch, and constructing a classification task data set and a recovery task data set which are respectively used for training a perception network and a recovery network; and finally, inputting a real degraded image into the obtained trained double-branch neural network architecture to obtain a final output recovered image. According to the method, the complexity and diversity of image degradation in a real scene are analyzed, the limitation of only aiming at a single degradation mode is overcome, the evaluation precision and generalization ability of the perception network on the image degradation state are effectively enhanced by introducing constraint conditions of a plurality of subtasks, and the image degradation state can be evaluated by aiming at different image features. The computing resources are distributed and optimized in a self-adaptive mode, dual improvement of computing efficiency and recovery quality is achieved, consumption of a large number of computing resources is avoided while high performance is ensured, and the deployment process is more convenient and flexible.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and image processing, and particularly relates to a degraded image restoration method based on perception guidance. Background Art

[0002] Image restoration technology can not only achieve satisfactory visual effects, but also facilitate the subsequent high-level visual tasks. In recent years, with the continuous improvement of computer vision technology, researchers have achieved good results in image restoration tasks. However, most of these studies are limited to single degradation conditions, such as haze, low light, noise, etc. The degradation in real scenes is often complex and changeable. Previous studies have great limitations in complex and harsh environments. Therefore, it is very important to design a method that can cope with a variety of harsh environments.

[0003] Currently, researchers have adopted two main strategies for image restoration tasks under harsh environmental conditions within the deep learning framework. The first strategy is to adopt a highly complex approach, namely, to train an independent neural network model for each specific type of image degradation (such as haze, noise pollution, low light, etc.). In practical applications, these specially trained models can be selectively called according to the specific type of degraded image to achieve targeted processing and restoration. Although this processing mode can theoretically cope with a variety of image degradation situations, its drawback is that it requires the construction and maintenance of a large model set, which will undoubtedly greatly increase the consumption of computing resources and the overall cost. At the same time, the deployment and management of the model also become more complicated.

[0004] The second strategy is a more general solution, which is to design and train a unified network model that can cover multiple degradation types. The core advantage of this strategy lies in its high efficiency and resource conservation. By sharing network structure and parameters, it significantly reduces the computational burden during model training and inference, and improves the computational speed of image restoration. The unified model achieves enhanced applicability to degraded images by learning the common characteristics of multiple degradation phenomena (such as global brightness improvement, contrast enhancement, and effective noise suppression). However, this processing approach inevitably ignores the uniqueness and differences between different degradation types, which to some extent limits the model's ability to perform fine restoration in specific or extremely degraded environments, and may lead to poor restoration effects in some complex or specific scenarios. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a degraded image restoration method based on perception guidance to better utilize the universality and uniqueness between different degradations.

[0006] The technical solution adopted by the present invention is: a degraded image restoration method based on perception guidance, the specific steps are as follows:

[0007] S1. Build a dual-branch neural network architecture consisting of a “perception branch” and a “recovery branch”;

[0008] The dual-branch neural network includes: a perception network and a recovery network; the perception branch is the perception network, and the recovery branch is the recovery network.

[0009] Among them, the perception network is a feature extraction network including 18 convolutional layers, and the recovery network is a neural network including 24 parallel convolution modules, 5 convolution modulation modules and 10 parallel convolution attention modules.

[0010] The perception network is divided into four blocks, each consisting of two 3×3 convolutional layers, a BatchNorm layer, and a ReLU activation function. The first block outputs a feature map with 64 channels and a size that is half the size of the input image; the second block outputs a feature map with 128 channels and a size that is one-quarter the size of the input image; the third block outputs a feature map with 256 channels and a size that is one-eighth the size of the input image; and the fourth block outputs a feature map with 512 channels and a size that is one-sixteenth the size of the input image.

[0011] The parallel convolution module consists of k+1 convolution layers connected in parallel and then connected in series with one convolution layer. The convolution modulation module consists of two fully connected layers and one softmax activation layer. The parallel convolution attention module consists of one parallel convolution module plus one perception-guided attention module.

[0012] Among them, the perception-guided attention module includes: 1 spatial attention module, 1 channel attention module, 1 channel shuffle layer, a GConv, and a sigmoid activation function.

[0013] S2. Construct a classification task dataset and a restoration task dataset to train the perception network and the restoration network respectively;

[0014] Collect 9N pairs of images from each of the k categories, including degraded images and their corresponding reference images. Select N of these pairs and label them as the classification task dataset for training the perception network; the remaining 8N pairs of images do not need to be labeled and are used for training the restoration network.

[0015] Where k represents the number of degenerate categories.

[0016] S3. Using the classification task dataset in step S2 to train the perception network, and then feeding the degraded image in the restoration task dataset in step S2 into the trained perception network to obtain a feature map, feature vector, and classification result corresponding to the input degraded image;

[0017] S4, using the restoration task dataset and the image classification results output by the perception network in step S3 to guide the restoration network to perform the first training and optimize the specific convolution kernel parameters in the restoration network;

[0018] S5, using the restoration task dataset and the feature vector and feature map obtained in step S3 to guide the restoration network to perform a second training;

[0019] S6. Input the real degraded image into the trained perception network to obtain the feature map and feature vector corresponding to the real degraded image. Then, input the real degraded image and the corresponding feature map and feature vector into the restoration network after completing two trainings to obtain the final output restoration image.

[0020] Furthermore, the step S3 is specifically as follows:

[0021] First, the classification task data is used as the input of the perception network. The cross entropy loss between the classification results predicted by the perception network and the actual classification results is calculated. The calculated loss result is used as the loss function to guide the training of the perception network. The cross entropy loss expression is as follows:

[0022]

[0023] Among them, i indicates that the image is the i-th type of degraded image. When the input image is the i-th type of degraded image, y i is 1, otherwise it is 0, p i Represents the probability that the output is the i-th type of degraded image.

[0024] Then the degraded image in the restoration task dataset is sent as input to the trained perception network. After the input degraded image passes through the second block of the perception network, the output channel number is 128 and the size is 1 / 4 of the input feature map F corresponding to the degraded image. p , the feature map F p Perform the flatten operation to flatten the feature vector V p The degraded image is output from the fourth block and then globally average pooled to obtain a feature map with 512 channels and a size of 1×1. The 1×1 feature map is fed into the fully connected layer and the classification result i is output.

[0025] Furthermore, the step S4 is specifically as follows:

[0026] The parallel convolution module includes: k convolution kernels W1, W2 for extracting unique features, ``` W k And a convolution kernel W0 to extract universal features.

[0027] The degraded image in the restoration task dataset is input into the restoration network, and the reconstruction loss Loss between the output of the restoration network and the reference image corresponding to the degraded image is calculated. recon1 . The reconstruction loss Loss recon1 As the loss function, and according to the classification result i in step S3, guide the i-th convolution kernel W in the parallel convolution module i And the parameters of the convolution kernel W0 are updated and optimized.

[0028] Among them, the reconstruction loss Loss recon1 The expression is as follows:

[0029] Loss recon1 =‖yy pred1 ‖

[0030] Among them, y represents the reference image corresponding to the degraded image, y pred1 represents the output of the degraded image after the restoration network. The parameter update optimization process is expressed as follows:

[0031] w=argmin w (‖f(w,x)-y‖)

[0032] Among them, w represents the i-th convolution kernel W i And the parameters of the convolution kernel W0, f(w,x) represents the output of the degraded image after the restoration network, x represents the input degraded image, and y represents the reference image corresponding to the degraded image.

[0033] Furthermore, the step S5 is specifically as follows:

[0034] The degraded image in the restoration task dataset, the feature map and feature vector obtained in step S3 are simultaneously fed as input to the restoration network after the first training, and the reconstruction loss Loss between the output of the restoration network and the reference image corresponding to the degraded image is calculated. recon2 . Using reconstruction loss Loss recon2 The constrained recovery network is trained for the second time to further optimize the recovery network performance.

[0035] Reconstruction loss Loss recon2 The expression is as follows:

[0036] Loss recon2 =‖yy pred2 ‖

[0037] Among them, y represents the reference image corresponding to the degraded image, y pred2 Represents the output of the degraded image after the restoration network completes the first training.

[0038] Then the feature map F in step S3 p Input the parallel convolution attention module in the recovery network after the first training. Then the feature map F output by the parallel convolution attention module PCB With F p Add point by point to get the same value as F PCB The fused feature map with the same size and number of channels is then passed through the spatial attention module to obtain a fused feature map with the same size and number of channels as F. PCB The same spatial attention map; the number of channels and F are obtained after the channel attention module PCB The same channel attention map with a size of 1×1 is used. The spatial attention and channel attention maps are added using the broadcast mechanism to obtain the preliminary pixel attention map PA. The number of channels and the size are the same as F. PCB The fusion feature map and PA are sequentially subjected to concat, Channel Shuffle, GConv and Sigmoid activation operations to obtain the perception-guided attention map PGA, with the same number of channels and size as F PCB Keep it consistent. PCB Perform point-by-point multiplication and output the multiplication result to the next parallel convolution module. Feature vector V p It is sent to the convolutional modulation module of the recovery network and outputs k+1 scaling factors α1, α2. ``` α k+1 , use the scaling factor to perform weighted summation on multiple parallel convolution kernels to obtain a new dynamic convolution kernel W f , the calculation formula of the dynamic convolution kernel is as follows:

[0039] W f =α1·W1+α2·W2+···+α k+1 W k+1

[0040] Finally, dynamic convolution kernel is used to extract image features and complete the second training.

[0041] Beneficial effects of the present invention: The method of the present invention first constructs a two-branch neural network architecture including a "perception branch" and a "restoration branch", and constructs a classification task data set and a restoration task data set, which are respectively used to train the perception network and the restoration network. Finally, the trained two-branch neural network architecture is input into a real degraded image to obtain the final output restoration image. Compared with the existing deep learning image restoration technology, the method of the present invention has certain advantages. By analyzing the complexity and diversity of image degradation in real scenes, it overcomes the limitation of only targeting a single degradation mode. By introducing the constraints of multiple subtasks, it effectively enhances the perception network's assessment accuracy and generalization ability of image degradation status. This innovative strategy enables the present invention to adaptively allocate and optimize computing resources for different image features, achieving a dual improvement in computing efficiency and restoration quality. While ensuring high performance, it will not lead to the consumption of a large amount of computing resources. The deployment process is more convenient and flexible, and can better adapt to and be competent for complex and changeable real-world application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flowchart of a degraded image restoration method based on perception guidance of the present invention.

[0043] Figure 2 Schematic diagram of a dual-branch neural network architecture in an embodiment of the present invention.

[0044] Figure 3 Schematic diagram of the structure of the perception-guided attention module in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The method of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0046] like Figure 1 As shown in FIG, a flow chart of a degraded image restoration method based on perception guidance of the present invention is shown, and the specific steps are as follows:

[0047] S1. Build a dual-branch neural network architecture consisting of a “perception branch” and a “recovery branch”;

[0048] In this embodiment, a dual-branch network is constructed using the pytorch framework. Figure 2 As shown, the dual-branch neural network includes: a perception network and a recovery network; the perception branch is the perception network, and the recovery branch is the recovery network.

[0049] Among them, the perception network is a feature extraction network including 18 convolutional layers, and the recovery network is a neural network including 24 parallel convolution modules (Parallel Conv Block), 5 convolution modulation modules (Modulation Block) and 10 parallel convolution attention modules (Parallel Conv Attention Block).

[0050] The perception network is divided into four blocks, each consisting of two 3×3 convolutional layers, a BatchNorm layer, and a ReLU activation function. The first block outputs a feature map with 64 channels and a size that is half the size of the input image; the second block outputs a feature map with 128 channels and a size that is one-quarter the size of the input image; the third block outputs a feature map with 256 channels and a size that is one-eighth the size of the input image; and the fourth block outputs a feature map with 512 channels and a size that is one-sixteenth the size of the input image.

[0051] The parallel convolution module consists of k+1 convolutional layers connected in parallel and then connected in series with one convolutional layer. The convolution modulation module consists of two fully connected layers and one softmax activation layer. The parallel convolution attention module consists of one parallel convolution module plus one perception-guided attention module.

[0052] Among them, such as Figure 3 As shown in the figure, the perception-guided attention module includes: 1 spatial attention module, 1 channel attention module, 1 channel shuffle layer, a GConv, and a sigmoid activation function.

[0053] S2. Construct a classification task dataset and a restoration task dataset to train the perception network and the restoration network respectively;

[0054] Collect 9N pairs of images from each of the k categories, including degraded images and their corresponding reference images. Select N of these pairs and label them as the classification task dataset for training the perception network; the remaining 8N pairs of images do not need to be labeled and are used for training the restoration network.

[0055] Among them, k represents the number of degenerate categories and is set according to actual conditions.

[0056] S3. Using the classification task dataset in step S2 to train the perception network, and then feeding the degraded image in the restoration task dataset in step S2 into the trained perception network to obtain a feature map, feature vector, and classification result corresponding to the input degraded image;

[0057] S4, using the restoration task dataset and the image classification results output by the perception network in step S3 to guide the restoration network to perform the first training and optimize the specific convolution kernel parameters in the restoration network;

[0058] S5, using the restoration task dataset and the feature vector and feature map obtained in step S3 to guide the restoration network to perform a second training;

[0059] S6. Input the real degraded image into the trained perception network to obtain the feature map and feature vector corresponding to the real degraded image. Then, input the real degraded image and the corresponding feature map and feature vector into the restoration network after completing two trainings to obtain the final output restoration image.

[0060] In this embodiment, step S3 is specifically as follows:

[0061] First, the classification task data is used as the input of the perception network. The cross entropy loss between the classification results predicted by the perception network and the actual classification results is calculated. The calculated loss result is used as the loss function to guide the training of the perception network. The cross entropy loss expression is as follows:

[0062]

[0063] Among them, i indicates that the image is the i-th type of degraded image. When the input image is the i-th type of degraded image, y i is 1, otherwise it is 0, p i Represents the probability that the output is the i-th type of degraded image.

[0064] Then the degraded image in the restoration task dataset is sent as input to the trained perception network. After the input degraded image passes through the second block of the perception network, the output channel number is 128 and the size is 1 / 4 of the input feature map F corresponding to the degraded image. p , the feature map F p Perform the flatten operation to flatten the feature vector V p The degraded image is output from the fourth block and then globally average pooled to obtain a feature map with 512 channels and a size of 1×1. The 1×1 feature map is fed into the fully connected layer and the classification result i is output.

[0065] In this embodiment, step S4 is specifically as follows:

[0066] The parallel convolution module includes: k convolution kernels W1, W2 for extracting unique features, ``` W k And a convolution kernel W0 to extract universal features.

[0067] The degraded image in the restoration task dataset is input into the restoration network, and the reconstruction loss Loss between the output of the restoration network and the reference image corresponding to the degraded image is calculated. recon1 . The reconstruction loss Loss recon1 As the loss function, and according to the classification result i in step S3, guide the i-th convolution kernel W in the parallel convolution module i And the parameters of the convolution kernel W0 are updated and optimized.

[0068] Among them, the reconstruction loss Loss recon1 The expression is as follows:

[0069] Loss recon1 =‖yy pred1 ‖

[0070] Among them, y represents the reference image corresponding to the degraded image, y pred1 represents the output of the degraded image after the restoration network. The parameter update optimization process is expressed as follows:

[0071] w=argmin w (‖f(w,x)-y‖)

[0072] Among them, w represents the i-th convolution kernel W i And the parameters of the convolution kernel W0, f(w,x) represents the output of the degraded image after the restoration network, x represents the input degraded image, and y represents the reference image corresponding to the degraded image.

[0073] In this embodiment, step S5 is specifically as follows:

[0074] The degraded image in the restoration task dataset, the feature map and feature vector obtained in step S3 are simultaneously fed as input to the restoration network after the first training, and the reconstruction loss Loss between the output of the restoration network and the reference image corresponding to the degraded image is calculated. recon2 . Using reconstruction loss Loss recon2 The constrained recovery network is trained for the second time to further optimize the recovery network performance.

[0075] Reconstruction loss Loss recon2 The expression is as follows:

[0076] Loss recon2 =‖yy pred2 ‖

[0077] Among them, y represents the reference image corresponding to the degraded image, y pred2 Represents the output of the degraded image after the restoration network completes the first training.

[0078] Then the feature map F in step S3p Input the parallel convolution attention module in the recovery network after the first training. Then the feature map F output by the parallel convolution attention module PCB With F p Add point by point to get the same value as F PCB The fusion feature map of the same size and number of channels is then passed through the spatial attention module ( Figure 3 The blue module in the middle) has a channel number of 1 and a size similar to F PCB The same spatial attention map; after the channel attention module ( Figure 3 The yellow module in the middle) gets the number of channels and F PCB The same channel attention map with a size of 1×1 is used. The spatial attention and channel attention maps are added using the broadcast mechanism to obtain the preliminary pixel attention map PA (pixel attention). The number of channels and the size are the same as F PCB The fusion feature map and PA are sequentially concat, Channel Shuffle, GConv and Sigmoid activation operations to obtain the perception-guided attention map PGA (Perception Guided Attention), the number of channels, and the size are the same as F PCB Keep it consistent. PCB Perform point-by-point multiplication and output the multiplication result to the next parallel convolution module. Feature vector V p It is sent to the convolutional modulation module of the recovery network and outputs k+1 scaling factors α1, α2. ``` α k+1 , use the scaling factor to perform weighted summation on multiple parallel convolution kernels to obtain a new dynamic convolution kernel W f , the calculation formula of the dynamic convolution kernel is as follows:

[0079] W f =α1·W1+α2·W2+···+α k+1 W k+1

[0080] Finally, dynamic convolution kernel is used to extract image features and complete the second training.

[0081] In summary, the method of the present invention fully considers the impact of different degradation factors on images. During the training process, the perception network's evaluation of degraded images can be used to guide and constrain the restoration network's restoration of images in harsh environments. During the inference process, there is no need to manually add additional prior information, and the network itself completes the evaluation and restoration of degraded images. Compared with existing deep learning-based image restoration methods in harsh environments, this method fully considers a variety of complex degradation factors and does not consume a large amount of computing resources. It can flexibly and effectively restore degraded images in real scenes, providing more reliable data support for subsequent analysis and application.

[0082] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. A degraded image restoration method based on perception guidance, the specific steps are as follows: S1. Build a dual-branch neural network architecture consisting of a "perception branch" and a "recovery branch"; The dual-branch neural network comprises: A sensing network and a recovery network; the sensing branch is the sensing network, and the recovery branch is the recovery network; The perception network is a feature extraction network with 18 convolutional layers, and the recovery network is a neural network with 24 parallel convolutional modules, 5 convolutional modulation modules, and 10 parallel convolutional attention modules. The perception network is divided into four blocks, each of which includes: 2 3×3 convolutional layers, 1 BatchNorm layer and 1 ReLU activation function; the first block outputs a feature map with 64 channels and a size of 1 / 2 of the input image; the second block outputs a feature map with 128 channels and a size of 1 / 4 of the input image; the third block outputs a feature map with 256 channels and a size of 1 / 8 of the input image; the fourth block outputs a feature map with 512 channels and a size of 1 / 16 of the input image; The parallel convolution module is composed of k+1 convolution layers connected in parallel and then connected in series with one convolution layer. The convolution modulation module is composed of two fully connected layers and one Softmax activation layer. The parallel convolution attention module is composed of one parallel convolution module plus one perception-guided attention module. Among them, the perception-guided attention module includes: 1 spatial attention module, 1 channel attention module, 1 channel shuffle layer, a GConv, and a sigmoid activation function; S2, construct a classification task dataset and a restoration task dataset, which are used to train the perception network and the restoration network respectively; Collect 9N pairs of k images, including degraded images and their corresponding reference images; select N pairs of images for category annotation as the classification task dataset for the training of the perception network; the remaining 8N pairs of images do not need to be annotated and are used for the training of the recovery network; Where k represents the number of degraded categories; S3, using the classification task data set in step S2 to train the perception network, and then sending the degraded image in the restoration task data set in step S2 to the trained perception network to obtain a feature map, feature vector and classification result corresponding to the input degraded image; S4, using the restoration task dataset and the image classification result output by the perception network in step S3, guiding the restoration network to perform the first training, and optimizing the specific convolution kernel parameters in the restoration network; S5, using the restored task data set and the feature vector and feature map obtained in step S3 to guide the restoration network to perform a second training; S6. Input the real degraded image into the trained perception network to obtain the feature map and feature vector corresponding to the real degraded image. Then, input the real degraded image and the corresponding feature map and feature vector into the restoration network after two trainings to obtain the final output restoration image.

2. The method for restoring degraded images based on perception guidance according to claim 1, characterized in that: The step S3 is specifically as follows: First, the classification task data is used as the input of the perception network, and the cross entropy loss between the classification result predicted by the perception network and the actual classification result is calculated. The calculated loss result is used as the loss function to guide the training of the perception network; the cross entropy loss expression is as follows: Where i indicates that the image is the i-th type of degraded image. When the input image is the i-th type of degraded image, y i is 1, otherwise it is 0, p i Represents the probability that the output is the i-th type of degraded image; Then the degraded image in the restoration task dataset is sent as input to the trained perception network. After the input degraded image passes through the second block of the perception network, the output channel number is 128 and the size is 1 / 4 of the input feature map F corresponding to the degraded image. p , the feature map F p Perform a flatten operation to flatten the feature vector V p ; The degraded image is output from the fourth block and then globally average pooled to obtain a feature map with 512 channels and a size of 1×1. The 1×1 feature map is sent to the fully connected layer and the classification result i is output.

3. The method for restoring a degraded image based on perception guidance according to claim 1, characterized in that: The step S4 is specifically as follows: The parallel convolution module includes: k convolution kernels W1, W2 for extracting unique features, ``` W k And a convolution kernel W0 to extract universal features; Input the degraded image in the restoration task dataset into the restoration network and calculate the reconstruction loss Loss between the output of the restoration network and the reference image corresponding to the degraded image recon1 ; Reconstruct the loss Loss recon1 As the loss function, and according to the classification result i in step S3, guide the i-th convolution kernel W in the parallel convolution module i And the parameters of the convolution kernel W0 are updated and optimized; Among them, the reconstruction loss Loss recon1 The expression is as follows: Loss recon1 =‖y-y pred1 ‖ Among them, y represents the reference image corresponding to the degraded image, y pred1 Represents the output of the degraded image after the restoration network; The parameter update optimization process expression is as follows: w=argmin w (‖f(w,x)-y‖) Among them, w represents the i-th convolution kernel W i And the parameters of the convolution kernel W0, f(w,x) represents the output of the degraded image after the restoration network, x represents the input degraded image, and y represents the reference image corresponding to the degraded image.

4. The method for restoring degraded images based on perception guidance according to claim 1, characterized in that: The step S5 is specifically as follows: The degraded image in the restoration task dataset, the feature map and feature vector obtained in step S3 are simultaneously input to the restoration network after the first training, and the reconstruction loss Loss between the output of the restoration network and the reference image corresponding to the degraded image is calculated. recon2 ; Using reconstruction loss Loss recon2 The constrained recovery network is trained for the second time to further optimize the performance of the recovery network; Reconstruction loss Loss recon2 The expression is as follows: Loss recon2 =‖y-y pred2 ‖ Among them, y represents the reference image corresponding to the degraded image, y pred2 Represents the output of the degraded image after the restoration network completes the first training; Then the feature map F in step S3 is p Input the parallel convolutional attention module in the recovery network after the first training; then output the feature map F of the parallel convolutional attention module PCB With F p Add point by point to get the same value as F PCB The fused feature map with the same size and number of channels is then passed through the spatial attention module to obtain a fused feature map with the same size and number of channels as F. PCB The same spatial attention map; the number of channels and F are obtained after the channel attention module PCB The same channel attention map with a size of 1×1 is used. The spatial attention and channel attention maps are added using the broadcast mechanism to obtain the preliminary pixel attention map PA. The number of channels and size are the same as F PCB Keep it consistent; perform concat, Channel Shuffle, GConv and Sigmoid activation operations on the fused feature map and PA in sequence to obtain the perception-guided attention map PGA, the number of channels, and the size of F PCB Keep it consistent; combine PGA with F PCB Perform point-by-point multiplication and output the multiplication result to the next parallel convolution module; the feature vector V p It is sent to the convolutional modulation module of the recovery network and outputs k+1 scaling factors α1, α2. ``` α k+1 , use the scaling factor to perform weighted summation on multiple parallel convolution kernels to obtain a new dynamic convolution kernel W f , the calculation formula of the dynamic convolution kernel is as follows: W f =α1·W1+α2·W2+···+α k+1 ·W k+1 Finally, the dynamic convolution kernel is used to extract image features and complete the second training.

Images