An image deblurring method based on local features and global representation

Abstract

The present invention discloses an image deblurring method based on local features and global representation, belonging to the field of image processing. The method first uses a dynamic convolution module in the encoder to extract adaptive local features, using smaller convolution kernels and a network with fewer layers to extract detailed information. The features are then input into a hybrid module and a locally enhanced Transformer module, which can capture global information and high-quality, globally relevant local details. The decoder then performs upsampling and uses a supervised attention module to enhance features. A cross-stage feature fusion module then fuses the encoder and decoder features and guides the encoder in the second stage, thereby adaptively extracting effective features from dynamically blurred scenes. Finally, the method passes through the same network as the first stage to obtain a high-quality deblurred image.

Description

[0001] Field:

[0002] The present invention relates to the field of image processing, and in particular to an image deblurring method. Background technology:

[0003] The goal of image deblurring is to restore a potential clear image from a blurred image. Existing methods use deep CNN to estimate the potential image from a blurred image, and use a new scale recursive network with cross-scale shared weights to reduce the difficulty of training and introduce stability advantages. However, this type of CNN-based method has fixed weights and can only capture partial information, and the resulting deblurred image quality is not high. The document "Image Deblurring Method Based on Enhanced Multi-Scale Feature Network, Laser and Optoelectronics Progress, 2022, Vol59(22), p264-271" discloses an image deblurring method based on an enhanced multi-scale feature network. This method uses a multi-scale feature extraction module and a cross-stage attention mechanism to fully extract and utilize features of different scales of the image, reduce information loss, and improve the image spatial feature representation capability. In addition, a multi-stage network structure training method is adopted to improve the effect of the multi-stage network structure. The method described in the literature obtains features at different levels through convolution. In order to increase the receptive field, it is necessary to increase the number of convolutional layers and the kernel size, or use images of different scales to learn global information. This will result in a large number of network layers and make it difficult to improve the quality of image deblurring. In addition, the filter of this method is fixed, making it difficult to adaptively extract effective features from dynamically blurred scenes. Summary of the invention:

[0004] The purpose of the present invention is to provide an image deblurring method based on local features and global representation in order to overcome the shortcomings of the existing image deblurring method based on enhanced multi-scale feature networks.

[0005] The technical solution of the present invention is: an image deblurring method based on local features and global representation, comprising the following steps:

[0006] Step 1: Perform the first stage feature extraction operation, pass the blurred image A through the convolution layer to obtain the feature F0.

[0007] Step 2: Feature F0 is input into the encoder to obtain feature Z4, which includes two sub-steps:

[0008] Sub-step 1: Feature F0 is converted into features F1, F2, and F3 through three dynamic convolution modules in the encoder. Feature F3 is then downsampled to obtain feature Z3.

[0009] The encoder consists of 4 components. The first three components have the same structure, including dynamic convolution module and downsampling, and the fourth component includes mixing module and downsampling. In the first component, the feature Fo After the dynamic convolution module, feature F1 is obtained, and then feature Z1 is obtained after downsampling. As shown below:

[0010] F1=DCB(F0) (1)

[0011] Z1=Downsample(F1) (2)

[0012] DCB(·) represents the dynamic convolution block, and Downsample(·) represents the downsampling process. The second component has the same structure as the first. Feature Z1 is used as the input of the second component, and it is used to obtain feature F2 and downsampled feature Z2. Similarly, feature F3 and downsampled feature Z3 are obtained in the third component.

[0013] The dynamic convolution module does not use the fixed parameters of CNN, but instead improves the model's capabilities by adaptively aggregating four convolution kernels through attention. First, the attention module obtains the coefficients of the four convolution kernels, then multiplies the obtained coefficients with the input convolution, and finally adds them to the input convolution. With F0 as the input of the dynamic convolution module and F1 as the output, the attention module calculates as follows during the operation of the dynamic convolution module:

[0014] F T =ReLU(FC1(AvgPool(F0))) (3)

[0015]

[0016] Where AvgPool(·) is the average pooling operation, FC1 and FC2 represent the fully connected network, and the four coefficients are obtained through the attention module The output F1 of the final dynamic convolution module is calculated by the following formula:

[0017]

[0018] Where Conv(·) represents a 1×1 convolutional layer.

[0019] Sub-step 2: Feature Z3 is input into the mixing module to obtain feature F4, and then downsampled to obtain feature Z4.

[0020] The hybrid block acts as a bridge between the dynamic convolution module and the locally enhanced Transformer module. The hybrid block consists of a local branch and a global branch, which capture both local and global features. The following describes the hybrid block in detail, taking feature Z3 as input:

[0021] (I) Feature Z3 inputs the local branch to obtain the feature

[0022] In order to adaptively obtain advantageous features from various blurs, the local branch integrates dynamic convolution into the residual block. Given the input feature Z3, the local branch uses dynamic convolution to extract the residual feature map Final output features of local branches is the input feature Z3 and the residual feature The sum of , the formula is as follows:

[0023]

[0024] (II) Input feature Z3 obtains features through global branch

[0025] The input feature Z3 in the global branch will be one-dimensionalized and embedded in the positional encoding to obtain a new feature This feature will be used as input to the locally enhanced Transformer module, outputting After upsampling, the output of the global branch is obtained

[0026] Below is As input, As an output, we introduce the local enhancement Transformer module. The local enhancement Transformer module uses some local information to enhance the fine details of the feature global representation. This module consists of a multi-head attention (MHA), a feedforward branch (FFB), and a local branch (LB). In the module, a normalization layer (Norm) is used before the multi-head attention and feedforward branches, and a residual connection is applied between the multi-head attention and the final output. Therefore, the output of this module is It can be expressed by the following formula:

[0027]

[0028]

[0029] (III) The local branch output in the hybrid module is Global branch output is the output of the locally enhanced Transformer module The upsampling is obtained, so the mixing module output F4 is as follows:

[0030]

[0031] Step 3: Feature Z4 is input into the hybrid module and the locally enhanced Transformer module to obtain the new feature Z5.

[0032] The feature Z4 obtained by the encoder is input into the hybrid module and the locally enhanced Transformer module to obtain the new feature Z5. The hybrid module and the locally enhanced Transformer module here have the same structure as the hybrid module and the locally enhanced Transformer module in the encoder.

[0033] Step 4: Feature Z5 is decoded to obtain feature Z6.

[0034] Feature Z5 is input to the decoder. The decoder is composed of 4 upsampling and dynamic convolution modules. The dynamic convolution module has the same structure as the encoder's dynamic convolution module. Similar to the encoder, the dynamic convolution module outputs 4 different stage features. Where L = 4. The decoder finally obtains the upsampled feature Z6, which is the output F′4 of the dynamic convolution module in the fourth component.

[0035] Step 5: The decoder output feature Z6 is input into the supervised attention module to obtain the salient feature Z7 and the deblurred image B.

[0036] The Supervised Attention Module (SAM) is used in a multi-stage network to provide a supervisory signal useful for progressive image restoration at each stage. It also generates an attention map to suppress features with less information in the current stage, allowing only useful features to propagate to the next stage. The upsampled features Z6 output by the decoder are processed by the Supervised Attention Module to generate salient features Z7, which serve as supervisory signals to guide image deblurring in the second stage, while outputting the deblurred image B.

[0037] The following introduces the supervised attention module using Z6 as input:

[0038] The input feature of the supervised attention module is Z6∈R C×H×W , first generate the residual image through a simple 1×1 convolution Where H×W represents the spatial dimension and C is the number of channels. Add it to the degraded input image I to get the restored image X s ∈R C×H×3 . Predict image X based on explicit supervision of real image s , using 1×1 convolution and sigmoid activation function from image X s Generate an attention mask M∈R for each pixel C×H×W These masks are then used to recalibrate the local features Z6 transformed by the 1×1 convolution, resulting in attention-guided features that are added to the identity mapping path. Finally, the attention-enhanced feature representation Z7 generated by the supervised attention module is passed to the next stage for further processing, while outputting the deblurred image B.

[0039] Step 6: Perform the second stage feature extraction operation, and the blurred image A obtains the features through this operation

[0040] The blurred image A is convolved and fused with the enhanced feature Z7 output by the first-stage supervised attention module, and then convolved to obtain the feature The calculation process is as follows:

[0041]

[0042] Step 7: First stage encoder features and decoder characteristics The fusion feature is obtained through the cross-stage feature fusion module

[0043] In order to effectively handle the information loss caused by downsampling and upsampling, the present invention adopts a cross-stage feature fusion (CSFF) module between the first stage encoder-decoder. In the cross-stage feature fusion module, the features from the encoder and decoder are convolved and added with the encoder features of the next stage to obtain the fused features. The calculation formula is as follows:

[0044]

[0045] Where, and are the encoder and decoder features of the Nth stage, is the encoder feature of the N+1 stage. Encoder features Features of the decoder The fusion feature is obtained through the cross-stage feature fusion module The formula is as follows:

[0046]

[0047] Step 8: Features and fusion features Input the second stage encoder to get the features

[0048] The encoder structure of the second stage is the same as that of the first stage, and consists of four components. The first three components have the same structure, including the dynamic convolution module and downsampling, and the fourth component includes the mixing module and downsampling. The input of each component of the second stage encoder is the output of the previous component plus the corresponding fusion feature of the first stage, that is:

[0049]

[0050] Where, is the input of the current component, F″l is the fusion feature of the first stage, is the output of the previous component, where L=4.

[0051] After passing through the four components of the encoder, the feature

[0052] Step 9: Features Input hybrid module and local enhanced Transformer module to obtain features

[0053] The hybrid module and the locally enhanced Transformer module in the second stage have the same structure as the first stage. After passing through this component, the feature

[0054] Step 10: Features Input decoder to get features

[0055] The decoder of the second stage has the same structure as the first stage. After the decoder, the upsampled features are obtained.

[0056] Step 11: Upsampling features After convolution, the final deblurred image C is obtained.

[0057] Upsampled features of decoder output After convolution, the final deblurred image C is obtained. The process is as follows:

[0058]

[0059] The beneficial effects of the present invention are as follows: the present invention first adopts a dynamic convolution module in the encoder to extract adaptive local features, and uses a smaller convolution kernel and a network with fewer layers to extract detailed information; then the features are input into a hybrid module and a locally enhanced Transformer module, and these two modules can capture global information and high-quality local details related to the global state; then upsampling is performed through the decoder, and the features are enhanced using a supervised attention module; then the features of the encoder and decoder are fused through a cross-stage feature fusion module and the encoder of the second stage is guided, so that effective features can be adaptively extracted from the dynamically blurred scene, and finally a high-quality deblurred image is obtained through the same network as the first stage.

[0060] Attached drawings:

[0061] Figure 1 This is a flowchart of the image deblurring method of the present invention.

[0062] Figure 2 is the blurred image A.

[0063] Figure 3 Deblurred image B in the first stage.

[0064] Figure 4 is the final deblurred image C. Specific implementation method:

[0065] The following describes a specific implementation method by taking an input image A of size 720*1280 as an example. Image A is as follows: Figure 2 The image deblurring method based on local features and global representation in this embodiment includes the following steps:

[0066] Step 1: Perform the first stage feature extraction operation, pass the blurred image A through the convolution layer, and obtain feature F0. Step 2: Feature F0 is input into the encoder to obtain feature Z4. It specifically includes two sub-steps:

[0067] Sub-step 1: Feature F0 is converted into features F1, F2, and F3 through three dynamic convolution modules in the encoder. Feature F3 is then downsampled to obtain feature Z3.

[0068] The encoder consists of four components. The first three components have the same structure, including a dynamic convolution module and downsampling. The fourth component includes a mixing module and downsampling. In the first component, feature F0 is converted to feature F1 through the dynamic convolution module, and then downsampled to feature Z1. This is shown below:

[0069] F1=DCB(F0) (1)

[0070] Z1=Downsample(F1) (2)

[0071] DCB(·) represents the dynamic convolution block, and Downsample(·) represents the downsampling process. The second component has the same structure as the first. Feature Z1 is used as the input of the second component, and it is used to obtain feature F2 and downsampled feature Z2. Similarly, feature F3 and downsampled feature Z3 are obtained in the third component.

[0072] Next, we'll introduce the dynamic convolution module. Rather than using the fixed parameters of CNN, the dynamic convolution module adaptively aggregates four convolution kernels through attention to improve model capabilities. First, the attention module obtains the coefficients of the four convolution kernels, then multiplies the obtained coefficients with the input convolution, and finally adds them to the input convolution. The following describes the operation of the dynamic convolution module, using F0 as the input and F1 as the output. The calculations for the attention module are as follows:

[0073] F T =ReLU(FC1(AvgPool(F0))) (3)

[0074]

[0075] Where AvgPool(·) is the average pooling operation, FC1 and FC2 represent the fully connected network, and the four coefficients are obtained through the attention module The output F1 of the final dynamic convolution module is calculated by the following formula:

[0076]

[0077] Where Conv(·) represents a 1×1 convolutional layer.

[0078] Sub-step 2: Feature Z3 is input into the mixing module to obtain feature F4, and then downsampled to obtain feature Z4.

[0079] The hybrid block acts as a bridge between the dynamic convolution module and the locally enhanced Transformer module. The hybrid block consists of a local branch and a global branch, which capture both local and global features. The following describes the hybrid block in detail, taking feature Z3 as input:

[0080] (I) Feature Z3 inputs the local branch to obtain the feature

[0081] In order to adaptively obtain advantageous features from various blurs, the local branch integrates dynamic convolution into the residual block. Given the input feature Z3, the local branch uses dynamic convolution to extract the residual feature map Final output features of local branches is the input feature Z3 and the residual feature The sum of , the formula is as follows:

[0082]

[0083] (II) Input feature Z3 obtains features through global branch

[0084] The input feature Z3 in the global branch will be one-dimensionalized and embedded in the positional encoding to obtain a new feature This feature will be used as input to the locally enhanced Transformer module, outputting After upsampling, the output of the global branch is obtained

[0085] Below is As input, As an output, we introduce the local enhancement Transformer module. The local enhancement Transformer module uses some local information to enhance the fine details of the feature global representation. This module consists of a multi-head attention (MHA), a feedforward branch (FFB), and a local branch (LB). In the module, a normalization layer (Norm) is used before the multi-head attention and feedforward branches, and a residual connection is applied between the multi-head attention and the final output. Therefore, the output of this module is It can be expressed by the following formula:

[0086]

[0087]

[0088] The specific introductions of the multi-head attention (MHA), feedforward branch (FFB) and local branch (LB) are as follows:

[0089] a. Multi-head attention (MHA) receives a query Q, a key K, and a value V as input and outputs a new feature Multi-head attention can be expressed as follows:

[0090] MHA(Q,K,V)=Concat(head0,...,head8)W o (9)

[0091] head j =Attention(QW j Q , KW j K , VW j V ) (10)

[0092] Where W j Q , W j K , W j V and W O are all parameters of linear projection, Concat(·) is the concatenation operation. Attention(·) is the attention operation, which is calculated as follows:

[0093]

[0094] Where d is the vector dimension and Softmax(·) is used to normalize the weights.

[0095] b. The feed-forward branch (FFB) consists of a normalization layer and two fully connected layers that transform features along the embedding dimension. These two fully connected (FC) layers tend to learn strong global feature representations and therefore often expand the hidden dimension between the two fully connected layers. The specific calculation is as follows:

[0096]

[0097] In the formula GReLU(·) represents an activation function with a default dimension expansion rate of γ = 4.

[0098] c. Since the results of the multi-head attention and feedforward branches only capture global features and cannot capture the interactions between adjacent features, local branches are used to enhance the representation. First, the local branch uses the Seq2Img(·) operation to convert the multi-head attention output features into Convert to 2D features, the operation is as follows:

[0099]

[0100] Then, depthwise convolution is adopted to effectively capture local information as follows:

[0101]

[0102] Where W d represents a depthwise convolution with a kernel size of 3×3, and is a 1×1 convolutional layer. In order to capture rich local features, the present invention expands the hidden dimension between the two convolutional layers and sets γ = 4. Finally, the 2D features are converted to Rearrange one-dimensional features

[0103] The final locally enhanced Transformer module output is as follows:

[0104]

[0105] (III) The local branch output in the hybrid module is Global branch output is the locally enhanced Transformer module output The upsampling is obtained, so the mixing module output F4 is as follows:

[0106]

[0107] Step 3: Feature Z4 is input into the hybrid module and the locally enhanced Transformer module to obtain the new feature Z5.

[0108] The feature Z4 obtained by the encoder is input into the hybrid module and the locally enhanced Transformer module to obtain the new feature Z5. The hybrid module and the locally enhanced Transformer module here have the same structure as the hybrid module and the locally enhanced Transformer module in the encoder.

[0109] Step 4: Feature Z5 is decoded to obtain feature Z6.

[0110] Feature Z5 is input to the decoder. The decoder is composed of 4 upsampling and dynamic convolution modules. The dynamic convolution module has the same structure as the encoder's dynamic convolution module. Similar to the encoder, the dynamic convolution module outputs 4 different stage features. The decoder finally obtains the upsampled feature Z6, which is the output F′4 of the dynamic convolution module in the fourth component.

[0111] Step 5: The decoder output feature Z6 is input into the supervised attention module to obtain the salient feature Z7 and the deblurred image B, as shown in Figure 3 shown.

[0112] The Supervised Attention Module (SAM) is used in a multi-stage network to provide a supervisory signal useful for progressive image restoration at each stage. It also generates an attention map to suppress features with less information in the current stage, allowing only useful features to propagate to the next stage. The upsampled features Z6 output by the decoder are processed by the Supervised Attention Module to generate salient features Z7, which serve as supervisory signals to guide image deblurring in the second stage, while outputting the deblurred image B.

[0113] The following introduces the supervised attention module using Z6 as input:

[0114] The input feature of the supervised attention module is Z6∈R C×H×W , first generate the residual image through a simple 1×1 convolution Where H×W represents the spatial dimension and C is the number of channels. Add it to the degraded input image I to get the restored image X s ∈R C×H×3 . Predict image X based on explicit supervision of real image s , using 1×1 convolution and sigmoid activation function from image X s Generate an attention mask M∈R for each pixel C×H×W These masks are then used to recalibrate the local features Z6 transformed by the 1×1 convolution, resulting in attention-guided features that are added to the identity mapping path. Finally, the attention-enhanced feature representation Z7 generated by the supervised attention module is passed to the next stage for further processing, while outputting the deblurred image B.

[0115] Step 6: Perform the second stage feature extraction operation, and the blurred image A obtains the features through this operation

[0116] The blurred image A is convolved and fused with the enhanced feature Z7 output by the first-stage supervised attention module, and then convolved to obtain the feature The calculation process is as follows:

[0117]

[0118] Step 7: First stage encoder features and decoder characteristics The fusion feature is obtained through the cross-stage feature fusion module

[0119] In order to effectively handle the information loss caused by downsampling and upsampling, the present invention adopts a cross-stage feature fusion (CSFF) module between the first stage encoder-decoder. In the cross-stage feature fusion module, the features from the encoder and decoder are convolved and added with the encoder features of the next stage to obtain the fused features. The calculation formula is as follows:

[0120]

[0121] Where, and are the encoder and decoder features of the Nth stage, is the encoder feature of the N+1 stage. Encoder features Features of the decoder The fusion feature is obtained through the cross-stage feature fusion module The formula is as follows:

[0122]

[0123] Step 8: Features and fusion features Input the second stage encoder to get the features

[0124] The encoder structure of the second stage is the same as that of the first stage, and consists of four components. The first three components have the same structure, including the dynamic convolution module and downsampling, and the fourth component includes the mixing module and downsampling. The input of each component of the second stage encoder is the output of the previous component plus the corresponding fusion feature of the first stage, that is:

[0125]

[0126] Where, is the input of the current component, F″ l is the fusion feature of the first stage, is the output of the previous component, where L=4.

[0127] After passing through the four components of the encoder, the feature

[0128] Step 9: Features Input hybrid module and local enhanced Transformer module to obtain features

[0129] The hybrid module and the locally enhanced Transformer module in the second stage have the same structure as the first stage. After passing through this component, the feature

[0130] Step 10: Features Input decoder to get features

[0131] The decoder of the second stage has the same structure as the first stage. After the decoder, the upsampled features are obtained.

[0132] Step 11: Upsampling features After convolution, the final deblurred image C is obtained, as shown in Figure 4 shown.

[0133] Upsampled features of decoder output After convolution, the final deblurred image C is obtained. The process is as follows:

[0134]

Claims

1. An image deblurring method based on local features and global representation, characterized in that: The steps include: Step 1: Perform the first stage feature extraction operation, pass the blurred image A through the convolution layer to obtain the feature F0; Step 2: Feature F0 is input into the encoder to obtain feature Z4, which includes two sub-steps: Sub-step 1: Feature F0 is passed through three dynamic convolution modules in the encoder to obtain features F1, F2, and F3. Feature F3 is downsampled to obtain feature Z3. The encoder consists of four components. The first three components have the same structure, including a dynamic convolution module and downsampling. The fourth component includes a mixing module and downsampling. In the first component, feature F0 is converted to feature F1 through the dynamic convolution module, and then to feature Z1 through downsampling. The process is as follows: F1=DCB(F0) (1) Z1=Downsample(F1) (2) DCB(·) represents the dynamic convolution module, and Downsample(·) represents the downsampling process. The structure of the second component is the same as the first component. Feature Z1 will be used as the input of the second component, and feature F2 and downsampled feature Z2 will be obtained through this component. Similarly, feature F3 and downsampled feature Z3 will be obtained in the third component. The dynamic convolution module does not use the fixed parameters of CNN, but instead improves the model capability by adaptively aggregating four convolution kernels through attention. First, the coefficients of the four convolution kernels are obtained through the attention module, and then the obtained coefficients are multiplied by the input convolution, and finally added to the input convolution. With F0 as the input of the dynamic convolution module and F1 as the output, during the operation of the dynamic convolution module, the attention module calculates as follows: F T =ReLU(FC1(AvgPool(F0))) (3) Where AvgPool(·) is the average pooling operation, FC1 and FC2 represent the fully connected network, and the four coefficients are obtained through the attention module The output F1 of the final dynamic convolution module is calculated by the following formula: Where Conv(·) represents a 1×1 convolutional layer; Sub-step 2: Feature Z3 is input into the mixing module to obtain feature F4, and then downsampled to obtain feature Z4; The hybrid block acts as a bridge between the dynamic convolution module and the locally enhanced Transformer module. The hybrid block consists of a local branch and a global branch, which capture both local and global features. With feature Z3 as input, the hybrid module process is as follows: (I) Feature Z3 inputs the local branch to obtain the feature In order to adaptively obtain advantageous features from various blurs, the local branch integrates dynamic convolution into the residual block. Given the input feature Z3, the local branch uses dynamic convolution to extract the residual feature map Final output features of local branches is the input feature Z3 and the residual feature The sum of , the formula is as follows: (II) Input feature Z3 obtains features through global branch The input feature Z3 in the global branch will be one-dimensionalized and embedded in the positional encoding to obtain a new feature This feature will be used as input to the locally enhanced Transformer module, outputting After upsampling, the output of the global branch is obtained by As input, As the output, the local enhanced Transformer module works as follows: The local enhanced Transformer module uses some local information to enhance the fine details of the feature global representation; the module consists of a multi-head attention (MHA), a feedforward branch (FFB) and a local branch (LB). A normalization layer (Norm) is used before the multi-head attention and feedforward branches in the module, and a residual connection is applied between the multi-head attention and the final output part; therefore, the output of this module It can be expressed by the following formula: (III) The local branch output in the hybrid module is Global branch output is the output of the locally enhanced Transformer module The upsampling is obtained, so the mixing module output F4 is as follows: Step 3 : Feature Z4 is input into the hybrid module and the locally enhanced Transformer module to obtain the new feature Z5; The feature Z4 obtained by the encoder is input into the hybrid module and the locally enhanced Transformer module to obtain the new feature Z5. The hybrid module and the locally enhanced Transformer module here have the same structure as the hybrid module and the locally enhanced Transformer module in the encoder; Step 4: Feature Z5 is decoded to obtain feature Z6; Feature Z5 is input to the decoder. The decoder consists of four upsampling and dynamic convolution modules. The dynamic convolution module has the same structure as the encoder’s dynamic convolution module. Similar to the encoder, the dynamic convolution module outputs four different stage features. Where L = 4; The decoder finally obtains the upsampled feature Z6, which is the output F of the dynamic convolution module in the fourth component. ′ 4; Step 5: The decoder output feature Z6 is input into the supervised attention module to obtain the salient feature Z7 and the deblurred image B; The supervised attention module (SAM) is used in a multi-stage network to provide a supervisory signal useful for progressive image restoration at each stage and generate an attention map to suppress features with less information in the current stage, allowing only useful features to propagate to the next stage. The upsampled feature Z6 output by the decoder is passed through the supervised attention module to obtain the salient feature Z7, which will be used as a supervisory signal to guide the second stage of image deblurring and output the deblurred image B at the same time; The following introduces the supervised attention module using Z6 as input: The input feature of the supervised attention module is Z6∈R C×H×W , first generate the residual image through a simple 1×1 convolution Where H×W represents the spatial dimension and C is the number of channels; the residual image Add it to the degraded input image I to get the restored image X s ∈R C×H×3 ; Predict image X based on explicit supervision of real image s , using 1×1 convolution and sigmoid activation function from image X s Generate an attention mask M∈R for each pixel C×H×W ; These masks are then used to recalibrate the local features Z6 transformed by 1×1 convolution, resulting in attention-guided features added to the identity mapping path; Finally, the attention-enhanced feature representation Z7 generated by the supervised attention module is passed to the next stage for further processing, and the deblurred image B is output; Step 6: Perform the second stage feature extraction operation, and the blurred image A obtains the features through this operation The blurred image A is convolved and fused with the enhanced feature Z7 output by the first-stage supervised attention module, and then convolved to obtain the feature The calculation process is as follows: Step 7: First stage encoder features and decoder characteristics The fusion feature is obtained through the cross-stage feature fusion module In order to effectively handle the information loss caused by downsampling and upsampling, the present invention adopts a cross-stage feature fusion (CSFF) module between the first stage encoder-decoder; In the cross-stage feature fusion module, the features from the encoder and decoder are convolved and added to the encoder features of the next stage to obtain the fused features. The calculation formula is as follows: Where, and are the encoder and decoder features of the Nth stage, is the encoder feature of the N+1 stage; the encoder feature Features of the decoder The fusion feature is obtained through the cross-stage feature fusion module The formula is as follows: Step 8: Features and fusion features Input the second stage encoder to get the features The encoder structure of the second stage is the same as that of the first stage, consisting of four components. The first three components have the same structure, including a dynamic convolution module and downsampling, and the fourth component includes a mixing module and downsampling. The input of each component of the second stage encoder is the output of the previous component plus the corresponding fusion feature of the first stage, that is: Where, is the input of the current component, F″ l is the fusion feature of the first stage, is the output of the previous component, where L = 4; After passing through the four components of the encoder, the feature Step 9: Features Input hybrid module and local enhanced Transformer module to obtain features The hybrid module and the locally enhanced Transformer module in the second stage have the same structure as the first stage. After passing through this component, the feature Step 10 :feature Input decoder to get features The decoder of the second stage has the same structure as the first stage. After the decoder, the upsampled features are obtained. Step 11: Upsampling features After convolution, the final deblurred image C is obtained; Upsampled features of decoder output After convolution, the final deblurred image C is obtained. The process is as follows: 。

Images