An efficient image restoration method based on global dependence modeling

Abstract

The application discloses a kind of efficient image restoration methods based on global dependence modeling, comprising: S1, obtaining damaged image;S2, input damaged image into multi-scale hierarchical neural network model, shallow feature extraction, deep feature extraction and feature fusion are carried out in turn, and fusion feature is obtained;S3, the residual map obtained by processing fusion feature is added to damaged image, and restoration image is obtained.The dependence relationship between global pixels can be captured by superpixel dependence calculation and transfer, and image restoration is carried out by the dependence relationship;The image restoration method provided by the application greatly reduces the model calculation cost, while ensuring the effectiveness of the method, the floating point operation number (FLOPs) is lower compared with the existing method.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to an efficient image restoration method based on global dependency modeling. BACKGROUND

[0002] Image restoration is one of the important techniques in the field of computer vision, aiming to recover a clear image from a corrupted one. The corruption includes various forms such as noise, blur, JPEG compression, etc. In recent years, some image restoration methods based on Transformer have been proposed, which have made significant progress in various applications such as image denoising, image deblurring, and JPEG compression artifact removal. Generally speaking, existing image restoration methods based on Transformer model the dependencies between pixels through attention mechanism, i.e., dependency modeling, and perform image restoration based on these dependencies.

[0003] Most existing image restoration methods based on Transformer model dependencies by dividing windows on the image and modeling dependencies between pixels within the windows, in order to reduce the computational cost of attention mechanism and improve efficiency. For example, a local enhanced window module is designed to divide the image into non-overlapping windows and model dependencies within the windows; an image restoration method based on window offset attention is designed, which uses two consecutive basic blocks to constitute the model, and uses different window divisions when calculating attention in the two blocks, where the window of the second block is obtained by offsetting the window of the first block, so as to expand the range of dependency modeling.

[0004] Although these methods have achieved gratifying results, their success is based on windowed attention calculation, i.e., dividing windows on the input image and modeling dependencies within the windows. On the one hand, these methods limit the range of dependency modeling within the divided windows, ignoring the modeling of dependencies in a larger range. However, the advantage of the Transformer model itself lies in long-distance dependency modeling, and this window division approach limits the range of dependency modeling, i.e., limits the performance of the Transformer model. On the other hand, although these methods reduce the computational cost of attention mechanism by dividing windows, the computational resources they require are still very large. Taking the commonly used metric for measuring the complexity of neural network models: floating-point operations (FLOPs) as an example, the FLOPs of the model in the first method described above is 373.02G, and the computational cost is still very high. Therefore, expanding the range of dependency modeling and reducing the computational cost of the model are urgent problems in the field of image restoration based on Transformer. SUMMARY

[0005] In view of the above problems in the prior art, the present application provides an efficient image restoration method based on global dependency modeling, which solves the problems of small modeling range and high model calculation cost in the prior art.

[0006] To achieve the above object, the present application adopts the technical scheme of an efficient image restoration method based on global dependency modeling, comprising the following steps:

[0007] S1, obtaining a damaged image;

[0008] S2, inputting the damaged image into a multi-scale hierarchical neural network model, sequentially performing shallow feature extraction, deep feature extraction and feature fusion to obtain fused features;

[0009] S3, adding the residual image obtained by processing the fused features to the damaged image to obtain a restored image.

[0010] Further, in the step S2, the method for shallow feature extraction is:

[0011] For a damaged image I with height H and width W and 3 channels, a convolution layer with size 3, step 1 and padding 0 is used to extract the shallow features F0 of the damaged image I; wherein the height and width of the shallow features F0 are H and W respectively, and the channel number is C, that is

[0012] Further, in the step S2, the input shallow features are extracted by four layers of encoder-decoder with different channel numbers to obtain deep features of different scales.

[0013] Wherein, each encoder-decoder structure includes a plurality of Transformer blocks, and each Transformer block includes an aggregation attention module and a double-path adaptive module connected in sequence; the aggregation attention module performs global dependency modeling according to the input feature map, and the double-path adaptive module is used to further process the pixel-level feature map using the global dependency modeling information to obtain deep features.

[0014] Further, the feature resolution of the encoder of the second to fourth layers is halved in turn, and the feature channel number is doubled in turn.

[0015] The feature resolution of the decoder of the third to first layers is doubled in turn, and the feature channel number is halved in turn.

[0016] Wherein, the processing process of the encoder for the input features is:

[0017]

[0018] In the formula, F i-1For the input feature of the i-th level of the encoder, T i For the i-th level of the encoder, ↓ is a feature resolution halving and channel doubling operation;

[0019] The processing procedure of the input feature by the decoder is as follows:

[0020]

[0021] In the formula, F′ i ↑+F i For the input feature of the i-th level of the decoder, T′ i For the i-th level of the decoder, ↑ is a feature resolution doubling and channel halving operation.

[0022] Further, in the step S2, the feature fusion is performed by a plurality of Transformer blocks to obtain the fused feature F r The expression is as follows:

[0023] F r =T″ r (F′0+F0)

[0024] In the formula, F0 is a shallow layer feature, F′0 is a deep layer feature, T″ r is a Transformer block for performing feature fusion.

[0025] Further, the attention aggregation module comprises, which are connected in sequence, a batch normalization layer, a first convolutional layer, a first group convolutional layer, a second group convolutional layer, a first Softmax layer, a third group convolutional layer, a fourth group convolutional layer, a second Softmax layer, a fifth group convolutional layer, a sixth group convolutional layer, a PixelShuffle layer and a second convolutional layer.

[0026] The dual-path adaptive module comprises, which are connected in sequence, a third convolutional layer, a seventh group convolutional layer, a GELU activation layer, a Sigmoid activation layer and a fourth convolutional layer.

[0027] Further, the process of the attention aggregation module for modeling global dependency of the input feature map is as follows:

[0028] SA1, performing feature map aggregation in the channel domain to obtain a feature map

[0029] Wherein, H and W are the height and width of the input feature map respectively, C is the number of channels, and C′=C / r c ,r c is a channel aggregation coefficient.

[0030] SA2, performing channel feature map aggregating to obtain a feature map

[0031] wherein H'=H / S, W'=W / S, C"=C'r s , S is a spatial aggregation coefficient, r s is a channel expansion coefficient;

[0032] SA3, performing attention calculation on the feature map Sequentially performing attention calculation in the channel domain and the spatial domain, and then performing global dependency modeling.

[0033] Further, in the step SA3, the method of performing attention calculation is:

[0034] SA3-1, mapping the spatial feature map to and performing channel attention calculation according to F=Softmax(QK T / a)V, and then obtaining a channel attention calculation result wherein Softmax is an activation function, a is a learnable scaling coefficient, Q, K, and V are respectively a query matrix, a key matrix, and a value matrix;

[0035] SA3-2, mapping the to and performing spatial attention calculation according to F=Softmax(QK T / a)V, and then obtaining a spatial attention calculation result to realize attention calculation.

[0036] Further, the method of mapping or to is:

[0037] mapping each channel feature of or to 3 channel features, and then transforming the channel position, so that the channel features are divided into n groups, each group contains Q, K, and V consistent with the original channel number, and then realizing feature mapping, n is a channel expansion coefficient in attention calculation.

[0038] Further, the method of using global dependency modeling information to process the pixel-level feature map by the dual-path adaptive module is:

[0039] SB1, performing feature mixing on the superpixel-level feature map by point-by-point convolution;

[0040] SB2, based on the global dependency modeling information, capturing the dependency of each pixel in the pixel-level feature map on the short-distance superpixel through the dual-path structure of the dual-path adaptive module, and extracting pixel-level features from the local region of the superpixel-level feature map.

[0041] SB3, the dependence of the pixel pair short distance superpixel is acted on the feature of the extracted pixel in a dynamic weighted manner, and further realizes the pixel level feature map processing, and its expression is:

[0042]

[0043]

[0044] In the formula, i is the index of the channel, and Phi i (·) is a grouped convolution with a convolution kernel of 7, a step of 1 and padding of 0, and sigma(·) and phi(·) are Sigmoid and GELU activation functions respectively.

[0045] The beneficial effects of the present application are:

[0046] (1) The present application can capture the dependence relationship between global pixels through superpixel dependence calculation and transfer, and restore the image through the dependence relationship, and the existing method can only capture the dependence relationship between local pixels.

[0047] (2) The image restoration method provided by the present application greatly reduces the model calculation cost, while ensuring the effectiveness of the method, and the present method is lower than the existing method in floating point operation number (FLOPs). BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 The flow chart of the efficient image restoration method based on global dependence modeling provided by the present application. DETAILED DESCRIPTION

[0049] The specific embodiments of the present application are described below to facilitate those skilled in the art to understand the present application, but it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the present application defined and determined by the appended claims, and all the inventions utilizing the concept of the present application are within the scope of protection.

[0050] Example 1:

[0051] The present application provides an efficient image restoration method based on global dependence modeling, as shown in Figure 1 The method comprises the following steps:

[0052] S1, obtaining a damaged image;

[0053] S2, inputting the damaged image into a multi-scale hierarchical neural network model, sequentially performing shallow feature extraction, deep feature extraction and feature fusion to obtain fused features;

[0054] S3. Add the residual map obtained by processing the fusion features to the damaged image to obtain the restored image.

[0055] In step S2 of this embodiment of the invention, the method for shallow feature extraction is as follows:

[0056] For a damaged image I with height and width H and width W, and 3 channels, a shallow feature F0 is extracted from the damaged image I using a convolutional layer with size 3, stride 1, and padding 0; where the height and width of the shallow feature F0 are H and W, and the number of channels is C, i.e.

[0057] In step S2 of this embodiment of the invention, deep features are extracted from the input shallow features by using four encoder-decoder layers with different numbers of channels to obtain deep features at different scales.

[0058] Each encoder-decoder structure includes several Transformer blocks, and each Transformer block includes a clustering attention module and a dual-path adaptive module connected in sequence. The clustering attention module performs global dependency modeling based on the input feature map, and the dual-path adaptive module is used to further process the pixel-level feature map using the global dependency modeling information to obtain deep features.

[0059] Specifically, the feature resolution of the encoders in the second to fourth layers is halved sequentially, while the number of feature channels is doubled sequentially; the feature resolution of the decoders in the third to first layers is doubled sequentially, while the number of feature channels is halved sequentially.

[0060] The encoder's processing of input features is as follows:

[0061]

[0062] In the formula, F i-1 Let T be the input feature of the i-th level of the encoder. i For the i-th level of the encoder, ↓ represents the operation of halving the feature resolution and doubling the number of channels;

[0063] The decoder processes the input features as follows:

[0064]

[0065] In the formula, F′ i ↑+F i Let T′ be the input feature of the i-th level of the decoder. i Let be the i-th level of the decoder, and ↑ represents the operation of doubling the feature resolution and halving the number of channels.

[0066] Further, the deep feature extraction in the embodiment includes a residual connection, i.e., the output of the encoder is connected to the input of the corresponding level decoding, so that the encoder and the decoder can better extract deep features.

[0067] In the embodiment of the application, for the feature fusion process, after the deep feature extraction, the extracted deep features and shallow features are processed by a plurality of Transformer blocks for feature fusion and refinement. The specific process is as follows:

[0068] F r =T″ r (F′0+F0),

[0069] wherein F0 is a shallow feature, F′0 is a deep feature, T″ r is a plurality of Transformer blocks for feature fusion.

[0070] In step S3 of the embodiment of the application, for the obtained fusion feature F r , a convolution layer with a size of 3, a step of 1 and padding of 0 is finally used to process the fusion feature F r , and the processing result is added to the input image I as a residual map to obtain a restored image.

[0071] Embodiment 2:

[0072] The embodiment is a further extension based on embodiment 1;

[0073] The attention aggregation module in the embodiment includes, in sequence, a batch normalization layer, a first convolution layer, a first grouped convolution layer, a second grouped convolution layer, a first Softmax layer, a third grouped convolution layer, a fourth grouped convolution layer, a second Softmax layer, a fifth grouped convolution layer, a sixth grouped convolution layer, a PixelShuffle layer and a second convolution layer. Specifically, the parameters of each layer are as follows:

[0074] Layer 1: batch normalization layer;

[0075] Layer 2: first convolution layer, input channel dimension C, output channel dimension C / r c , convolution kernel size 1, step 1, padding 0;

[0076] Layer 3: first grouped convolution layer, input channel dimension C / r c , output channel dimension Cr s / c , group number C / r c , convolution kernel size , step , padding 0;

[0077] Layer 4: a second grouped convolution layer, with an input channel dimension of Cr s / c , an output channel dimension of 3r s / c , a group number of Cr s / c , a convolution kernel size of 1, a stride of 1, and padding of 0;

[0078] Layer 5: a first Softmax layer;

[0079] Layer 6: a third grouped convolution layer, with an input channel dimension of nCr s / c , an output channel dimension of Cr s / c , a group number of Cr s / c , a convolution kernel size of 1, a stride of 1, and padding of 0;

[0080] Layer 7: a fourth grouped convolution layer, with an input channel dimension of Cr s / c , an output channel dimension of 3r s / c , a group number of Cr s / c , a convolution kernel size of 1, a stride of 1, and padding of 0;

[0081] Layer 8: a second Softmax layer;

[0082] Layer 9: a fifth grouped convolution layer, with an input channel dimension of nCr s / c , an output channel dimension of Cr s / c , a group number of Cr s / c , a convolution kernel size of 1, a stride of 1, and padding of 0;

[0083] Layer 10: a sixth grouped convolution layer, with an input channel dimension of Cr s / c , an output channel dimension of CS 2 / c , a group number of C / r c , a convolution kernel size of 1, a stride of 1, and padding of 0;

[0084] Layer 11: a PixelShuffle layer, with an up-sampling factor of S;

[0085] Layer 12: a second convolution layer, with an input channel dimension of C / r c, the output channel dimension is C, the convolution kernel size is 1, the step is 1, and the padding is 0.

[0086] The two-way adaptive module in this embodiment includes a third convolutional layer, a seventh grouped convolutional layer, a GELU activation layer, a Sigmoid activation layer, and a fourth convolutional layer connected in sequence. Specifically, the parameters of each layer are as follows:

[0087] The first layer: the third convolutional layer, the input channel dimension is C, the output channel dimension is C, the convolution kernel size is 1, the step is 1, and the padding is 0.

[0088] The second layer: the seventh grouped convolutional layer, the input channel is C, the output channel dimension is 2, the group number is C, the convolution kernel size is 7, the step is 1, and the padding is 3.

[0089] The third layer: the GELU activation layer.

[0090] The fourth layer: the Sigmoid activation layer.

[0091] The fifth layer: the fourth convolutional layer, the input channel is C, the output channel is C, the convolution kernel size is 1, the step is 1, and the padding is 0.

[0092] Based on the above aggregation attention module structure, the process of the aggregation attention module in this embodiment for global dependency modeling of the input feature map is as follows:

[0093] SA1, the feature map is aggregated in the channel domain , to obtain a channel feature map

[0094] Where H and W are the height and width of the input feature map respectively, C is the number of channels, and C' = r c ,r c is the channel aggregation coefficient;

[0095] SA2, the channel feature map is aggregated in the spatial domain , to obtain a spatial feature map

[0096] Where H' = / S, W' = / S, and C'' = r s , S is the spatial aggregation coefficient, and r s is the channel expansion coefficient;

[0097] SA3, the feature map is aggregated in the channel domain , and the attention calculation is performed in the channel domain and the spatial domain in sequence, and then the global dependency modeling is performed.

[0098] In this embodiment, the feature map is aggregated in the channel domain, reducing the channel information redundancy; the feature map is aggregated in the spatial domain, reducing the spatial information redundancy.

[0099] In step SA3 of this embodiment, the method for attention calculation is:

[0100] SA3-1, mapping the spatial feature map to and performing channel attention calculation according to F = Softmax(QK T / ), thereby obtaining the channel attention calculation result wherein Softmax is an activation function, a is a learnable scaling coefficient, Q, are query matrix, key matrix, and value matrix respectively; wherein Q and K calculate attention weights, the attention weights are multiplied by V, V is weighted, and finally the channel attention calculation result is obtained

[0101] SA3-2, mapping to and performing spatial attention calculation according to F = Softmax(QK T / ), thereby obtaining the spatial attention result to realize attention calculation.

[0102] wherein the shape of QKV in spatial attention calculation is a matrix of [HWxC], and the shape of QKV in channel attention calculation is [CxHW].

[0103] Specifically, mapping or to The method is:

[0104] mapping each channel feature of or to 3 channel features, then transforming the channel position, so that the channel features are divided into n groups, each group contains Q, K, and V consistent with the original channel number, thereby realizing feature mapping, and n is a channel expansion coefficient in attention calculation.

[0105] The mapping method provided in this embodiment improves the efficiency of multi-head attention calculation through the mechanism of channel slicing and merging.

[0106] Based on the above-mentioned dual-path adaptive module structure, the method for processing pixel-level feature maps using global dependency modeling information in the dual-path adaptive module in this embodiment is:

[0107] SB1, performing feature mixing on the superpixel-level feature map through point-by-point convolution;

[0108] SB2, based on the global dependency modeling information, capturing the dependency of each pixel in the pixel-level feature map on the short-distance superpixel through the dual-path structure of the dual-path adaptive module, and extracting pixel-level features from the local region of the superpixel-level feature map.

[0109] SB3, the dependence of the pixel pair short distance superpixel is applied to the feature extraction of the pixel in a dynamic weighted manner, and the pixel level feature map processing is realized, and the expression is:

[0110]

[0111]

[0112] In the formula, i is the index of the channel, Φ i (·) is a grouped convolution with a convolution kernel of 7, a step of 1 and padding of 0, and sigma (·) and phi (·) are Sigmoid and GELU activation functions respectively.

[0113] Embodiment 3:

[0114] The embodiment of the application provides a test example of the image restoration method in the embodiment:

[0115] Network parameters: four-layer encoder-decoder in the deep feature extraction stage, the number of channels from the first level to the fourth level is [64, 128, 256, 512], and the number of Transformer blocks in each layer is [4, 6, 6, 2]. The parameter r of the aggregated attention module in the four levels c All are 4, from the first level to the fourth level, r s is [32, 16, 8, 4], S is [16, 8, 4, 2], and n in attention calculation is 4.

[0116] Training details: the training uses an Adam optimizer, and the parameters β1=0.9 and β2=0.999. The learning rate is initialized to 2 -4 and gradually decreases to 1 -6 by a cosine annealing strategy; the number of samples used by each batch during training is 16, the samples are pictures cropped to 128*128, and 1 million iterations are trained, and at the same time, horizontal and vertical flipping, 90°, 180° and 270° rotation are used to augment the input image during training.

[0117] In order to verify the effectiveness of the method, experiments are carried out on image denoising, image deblurring and JPEG compression artifact removal three tasks, in order to reflect the superiority of the method, a plurality of advanced image restoration methods are compared in the embodiment, and PSNR is used as a measurement index; PSNR is one of the commonly used indexes in image restoration, and the larger the value is, the better the restoration effect is.

[0118] Test 1:

[0119] The method is verified on the tasks of grayscale image denoising and color image denoising, which requires removing the added additive Gaussian white noise with levels of 15, 25 and 50 respectively on the picture. In this task, we use a mixed data set to train the model, and the training data set includes DIV2K, Flickr2k, BSD400 and WED, which contain 800, 2650, 400 and 4774 pictures respectively. For the color image denoising task, verification is carried out on the CBSD68, Kodak24 and McMaster data sets. For the grayscale image denoising task, we use the grayscale version of the mixed data set mentioned above for training, and verification is carried out on the Set12 and BSD68 data sets. The experimental results of grayscale image denoising are shown in Table 1.

[0120] Table 1: Experimental results of grayscale image denoising

[0121]

[0122] The row added by the method in Table 1 represents the use of a self-integrated strategy to improve certain FLOPs to test the effectiveness of the method. From Table 1, it can be seen that the method has smaller FLOPs, that is, faster speed, and at the same time has better or close recovery effect than other methods under different noise levels, which means that in practical application, the method is more efficient and has better effect.

[0123] The experimental results of color image denoising are shown in Table 2:

[0124] Table 2: Experimental results of color image denoising

[0125]

[0126]

[0127] From Table 2, it can be seen that the method is more efficient while obtaining similar or better performance compared to other methods. For example, on the Kodak24 test set, the PSNR of the self-integrated strategy is 0.01dB-0.07dB higher than that of SwinIR, and the FLOPs is only half of that of SwinIR.

[0128] Experiment 2:

[0129] The experiment verifies the method on the image deblurring task, which requires removing blur in images. The training set of GoPro dataset is used for training, which contains 2103 pairs of clean images and blurred images. The test set of GoPro and HIDE dataset are used for testing, the test set of GoPro contains 1111 pairs of clean images and blurred images, and the HIDE dataset contains 2025 pairs of clean images and blurred images, and the size of the images is 1280x720. The experimental results are shown in Table 3:

[0130] Table 3: Experimental results of image deblurring task

[0131] Method Parameter quantity FLOPs GoPro HIDE DGAN - 16.96G 28.70 24.51 DeepDeblur 303.60M 44.00G 29.08 25.73 RNNDeblur - - 29.19 - DGANv2 7.83M 10.28G 29.55 26.61 SRN 3.76M 35.87G 30.26 28.36 HAMD - - - 28.89 DSD 2.84M - 30.90 29.11 DBGAN 11.59M 379.92G 31.10 28.94 MT-RNN 2.64M 13.72G 31.15 29.15 DMPHN 86.80M - 31.20 29.09 EBMD - - 31.79 - SAPHNet 23.00M - 31.85 29.98 The method 12.18M 22.52G 31.94 29.67

[0132] Experiment 3:

[0133] We verify the method on the JPEG compression artifact removal task, which requires removing artifacts caused by JPEG compression in images. The method uses a mixed dataset to train the model, and the training dataset includes DIV2K, Flickr2k, BSD400 and WED, which contain 800, 2650, 400 and 4774 images respectively. The Classic5 and LIVE1 datasets are used for testing. Classic5 contains five grayscale images, and LIVE1 contains 29 natural color images. The experimental results are shown in Table 4:

[0134] Table 4: Experimental results of JPEG compression artifact removal task

[0135]

[0136] From Table 4, it can be seen that the method of the present application obtains suboptimal results in most cases, and achieves the best trade-off between efficiency and effectiveness. For example, compared with DRUNet, our method has only about 37% of its parameter amount and about 31% of its FLOPs, and on the Classic5 dataset, the PSNR of the method is at most 0.02 dB higher than that of DRUNet. Although the results of the method are inferior to SwinIR, the FLOPs of the method are much smaller than that of SwinIR. This means that in practical applications, the method can be processed faster, verifying the efficiency and effectiveness of the method.

Claims

1. An efficient image restoration method based on global dependency modeling, characterized in that, Includes the following steps: S1. Obtain the damaged image; S2. Input the damaged image into a multi-scale hierarchical neural network model, and perform shallow feature extraction, deep feature extraction and feature fusion in sequence to obtain fused features; In step S2, deep features are extracted from the input shallow features by using four encoder-decoder layers with different numbers of channels to obtain deep features at different scales. Each encoder-decoder structure includes several Transformer blocks, and each Transformer block includes a convergent attention module and a dual-path adaptive module connected in sequence. The convergent attention module performs global dependency modeling based on the input feature map, and the dual-path adaptive module is used to further process the pixel-level feature map using the global dependency modeling information to obtain deep features. The attention-gathering module includes a batch normalization layer, a first convolutional layer, a first grouped convolutional layer, a second grouped convolutional layer, a first softmax layer, a third grouped convolutional layer, a fourth grouped convolutional layer, a second softmax layer, a fifth grouped convolutional layer, a sixth grouped convolutional layer, a pixelShuffle layer, and a second convolutional layer, all connected in sequence. The dual-path adaptive module includes a third convolutional layer, a seventh grouped convolutional layer, a GELU activation layer, a Sigmoid activation layer, and a fourth convolutional layer connected in sequence. The process by which the attention-focusing module performs global dependency modeling on the input feature map is as follows: SA1, Perform feature mapping in the channel domain Aggregation yields feature maps. ; in, H and W These represent the height and width of the input feature map, respectively. C For the number of channels, , This is the channel aggregation coefficient; SA2, Perform channel feature mapping in the spatial domain Aggregation yields feature maps. ; in, , The spatial aggregation coefficient, This is the channel expansion coefficient; SA3, Feature Map Attention calculations are performed sequentially in the channel domain and spatial domain, followed by global dependency modeling. The method by which the dual-path adaptive module processes pixel-level feature maps using global dependency modeling information is as follows: SB1: Perform feature fusion on superpixel-level feature maps through pointwise convolution; SB2. Based on global dependency modeling information, the dual-path structure of the dual-path adaptive module captures the dependency of each pixel in the pixel-level feature map on short-distance superpixels, and extracts pixel-level features from the local region of the superpixel-level feature map. SB3 applies a dynamic weighted approach to the feature extraction of pixels, considering the pixel's dependence on short-range superpixels, thereby achieving pixel-level feature map processing. Its expression is: In the formula, i For the channel index, This is a grouped convolution with a kernel of 7, a stride of 1, and padding of 0. and These are the Sigmoid and GELU activation functions, respectively. S3. Add the residual map obtained by processing the fusion features to the damaged image to obtain the restored image.

2. The efficient image restoration method based on global dependency modeling according to claim 1, characterized in that, In step S2, the method for shallow feature extraction is as follows: For height and width respectively H and W Damaged image with 3 channels I Damaged images are extracted using a convolutional layer with size 3, stride 1, and padding 0. I shallow features F 0; in, Shallow features F The height and width of 0 are respectively H and W The number of channels is C ,Right now .

3. The efficient image restoration method based on global dependency modeling according to claim 1, characterized in that, The feature resolution of the encoders in the second to fourth layers is halved in turn, and the number of feature channels is doubled in turn. The feature resolution of the decoder from the third layer to the first layer is doubled in turn, and the number of feature channels is halved in turn. The encoder's processing of input features is as follows: In the formula, For encoder number i Input features at each level, For the encoder's first i Hierarchy This is an operation that halves the feature resolution and doubles the number of channels. F 0 represents a shallow feature; The decoder processes the input features as follows: In the formula, For the decoder i Input features at each level, For the decoder's first Each level This involves doubling the feature resolution and halving the number of channels.

4. The efficient image restoration method based on global dependency modeling according to claim 3, characterized in that, In step S2, feature fusion is performed using several Transformer blocks to obtain fused features. Its expression is: In the formula, For deep features, Transformer block for feature fusion.

5. The efficient image restoration method based on global dependency modeling according to claim 1, characterized in that, In step SA3, the method for calculating attention is as follows: SA3-1, Spatial Feature Map Mapped to and in accordance with Perform channel attention calculation to obtain the channel attention calculation result. ;in, For activation function, For learnable scaling factors, These are the query matrix, key matrix, and value matrix, respectively. SA3-2, will Mapped to and in accordance with Perform spatial attention calculations to obtain the spatial attention calculation results. This enables attention computation.

6. The efficient image restoration method based on global dependency modeling according to claim 5, characterized in that, Will or Mapped to The method is as follows: Will or Each channel feature is mapped as Each channel feature is then divided into several channels, and the channel positions are changed to make the channel features categorized into... n Groups, each group contains the same number of channels as the original. This leads to feature mapping. n This is the channel expansion coefficient used in attention calculations.

Images