An image restoration method integrating linear attention mechanism

Abstract

The present invention discloses an image restoration method that integrates a linear attention mechanism, and relates to the field of image processing technology. The method comprises: obtaining an image sample set, combining the image to be restored and the corresponding target image into a sample pair input image restoration model; performing convolution preprocessing on the image to be restored to extract a first feature map of size W×H×C; establishing window and channel dimension attention based on the linear attention mechanism to obtain fused attention output features, repeating K times to output a final feature map; performing convolution postprocessing on the final feature map and concatenating it with the residual of the image to be restored to generate a restored image; constructing a loss function based on the target image, and optimizing the image restoration model parameters using a gradient descent method. This method achieves the technical effects of reducing computational complexity, improving model efficiency, enhancing model applicability and performance, and maintaining or improving image restoration quality.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image restoration method integrating a linear attention mechanism. Background Art

[0002] Image restoration technology currently plays a key role in numerous fields, including medical imaging, satellite image processing, and digital image restoration. With the rise of deep learning, a variety of advanced image restoration methods have been proposed. Among them, methods based on visual Transformers, such as Restormer and CAT, have gradually become mainstream. These methods, by incorporating attention mechanisms, effectively capture long-range pixel dependencies in images, achieving significant performance improvements in image restoration tasks.

[0003] However, existing visual Transformer-based image restoration methods face a major challenge in practical applications: the attention mechanism they employ leads to a significant increase in computational complexity. This increase in computational complexity directly reduces model efficiency, particularly when processing large-scale image data, making real-time processing and large-scale data training difficult. Furthermore, these methods lack targeted optimization for feature maps of varying sizes and channel counts, resulting in inefficient feature processing at both the window and channel levels. This further increases the model's computational burden and limits its potential for application in resource-constrained environments. Summary of the Invention

[0004] The present invention provides an image restoration method integrating a linear attention mechanism to solve the technical problems of high computational complexity and low efficiency in the prior art, thereby achieving the technical effects of reducing computational complexity, improving model efficiency, enhancing model applicability and performance, while maintaining or improving image restoration quality.

[0005] The present invention provides an image restoration method integrating a linear attention mechanism, comprising:

[0006] An image sample set is obtained, and a sample pair is formed by combining the image to be restored and the corresponding target image, which is input into an image restoration model, wherein the output of the image restoration model is a restored image.

[0007] The image to be restored is preprocessed, and a convolution operation is used to extract a first feature map, where the size of the first feature map is W×H×C, W and H are the image width and height respectively, C is the number of convolution channels, and W, H, and C are all positive integers greater than 0.

[0008] Based on the linear attention mechanism, window dimension attention and channel dimension attention are established on the first feature map, and the fused attention output feature is obtained.

[0009] Repeat K times to obtain K fused attention output features, which are output as the final feature map.

[0010] A convolution-based post-processing operation is performed on the final feature map, and a residual connection is performed between the convolution result and the image to be restored to obtain the restored image.

[0011] A loss function based on the target image is constructed, and the parameters of the image restoration model are optimized and updated using a gradient descent method.

[0012] In a feasible implementation, window-dimensional attention and channel-dimensional attention are established on the first feature map based on a linear attention mechanism, and a fused attention output feature is obtained, including:

[0013] The first feature map is divided into windows to obtain a set of sub-feature maps, and a sub-window attention feature is calculated for each sub-feature map, and the window attention feature is obtained by integration.

[0014] The first feature map is divided into a set of channel sub-blocks in the channel dimension, and the sub-channel attention features are calculated for each channel sub-block, and the channel attention features are obtained by integration.

[0015] The fused attention feature map is calculated based on the first feature map, the window attention feature and the channel attention feature.

[0016] In a feasible implementation, the window attention feature is obtained, including:

[0017] Based on the sliding window method, the first feature map is divided into windows of size M×M to form the sub-feature map set.

[0018] Perform a linear transformation on each sub-feature map to generate a first intermediate feature map, where the size of the intermediate feature map is W×H×3C.

[0019] The first intermediate feature map is divided into three equal parts in the channel dimension. Based on the linear attention mechanism, the first sub-linear attention feature is calculated in combination with the three-division result, and the window attention feature is further calculated.

[0020] In a feasible implementation, the channel attention features are obtained, including:

[0021] The first feature map is divided into L sub-blocks in the channel dimension, each sub-block includes C / L groups of features, and the L sub-blocks generate a channel sub-block set.

[0022] Perform a linear transformation on each channel sub-block to generate the second intermediate feature map.

[0023] The second intermediate feature map is divided into three equal parts in the channel dimension. Based on the linear attention mechanism, the second sub-linear attention feature is calculated in combination with the three-division result, and the channel attention feature is further calculated.

[0024] In a feasible implementation, each sub-feature map is linearly transformed, including:

[0025] For each of the sub-feature maps, perform The linear transformation of is the weight matrix, is the bias vector, is the first intermediate feature map, and .

[0026] In one possible implementation, a linear transformation is performed on each channel sub-block, including:

[0027] For each of the sub-feature maps, perform The linear transformation of is the weight matrix, is the bias vector, is the second intermediate feature map, and .

[0028] In one possible implementation, the linear attention function in the linear attention mechanism is defined as: ;

[0029] in, Characterize the linear attention feature of the n-th set element; , , Express Take d power bit by bit; 、 、 The result is divided into three equal parts.

[0030] In a feasible implementation, the first sub-linear attention feature is calculated as follows: ;

[0031] in, Represents the i-th first sub-linear attention feature. 、 、 is the weight matrix. 、 、 is the bias vector, Compute the result for the i-th linear attention mechanism.

[0032] In one feasible implementation, calculating the channel attention feature includes:

[0033] The second sub-linear attention features output by each of the channel sub-blocks are spliced ​​to obtain the channel attention features with the number of channels restored to C.

[0034] In a feasible implementation, the loss function is defined as: ;

[0035] in, represents the restored image, is the target image, and N is the number of training samples.

[0036] The present invention discloses an image restoration method integrating a linear attention mechanism, comprising: obtaining an image sample set, forming a sample pair of an image to be restored and a corresponding target image, and inputting the sample pair into an image restoration model, wherein the output of the image restoration model is a restored image; preprocessing the image to be restored, and extracting a first feature map by using a convolution operation, wherein the size of the first feature map is W×H×C, where W and H are respectively the image width and height, C is the number of convolution channels, and W, H, and C are all positive integers greater than 0; establishing window dimension attention and channel dimension attention on the first feature map based on the linear attention mechanism, and obtaining a fused attention map. Attention output features; repeat K times to obtain K fused attention output features, and output them as the final feature map; perform convolution-based post-processing operations on the final feature map, and perform residual connection between the convolution result and the image to be restored to obtain the restored image; construct a loss function based on the target image, and use the gradient descent method to optimize and update the parameters of the image restoration model. The image restoration method that integrates the linear attention mechanism disclosed in the present invention solves the technical problems of high computational complexity and low efficiency, and achieves the technical effect of reducing computational complexity, improving model efficiency, enhancing model applicability and performance, while maintaining or improving image restoration quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of an image restoration method integrating a linear attention mechanism according to the present invention.

[0038] Figure 2 This is a flowchart of an image restoration method that integrates a linear attention mechanism in the present invention. DETAILED DESCRIPTION

[0039] The above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods of the specification to better understand the above technical solution. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention. It should be understood that the present invention is not limited to the example embodiments used only to explain the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, it should be noted that, for the convenience of description, only the parts related to the present invention, rather than all, are shown in the drawings.

[0040] Example, Figure 1 This is a flow chart of an image restoration method integrating a linear attention mechanism according to the present invention, wherein the image restoration method integrating a linear attention mechanism includes:

[0041] S100: Acquire an image sample set, form a sample pair of an image to be restored and a corresponding target image, and input the sample pair into an image restoration model, wherein the output of the image restoration model is a restored image.

[0042] Specifically, an image sample set is a dataset containing a large number of image pairs, each consisting of an image to be restored and a target image. The image to be restored is an image affected by degradation, such as noise, blur, or damage, while the target image is the corresponding complete, clear, and unaffected image, which serves as a reference standard.

[0043] The image restoration model is based on deep learning. Its core function is to transform the image to be restored into a restored image by learning the image's features and patterns. The restored image is the output image that, after being processed by the model, is as close as possible to the target image.

[0044] Specifically, we first collect images to be restored, encompassing a variety of degradation scenarios, and their corresponding clear target images. These images can come from diverse application scenarios, such as cloud-obscured and clear, unobstructed satellite images. We then pair each image to be restored with its corresponding target image to form a sample pair. For example, for a set of images corrupted by Gaussian noise, we pair them with the original, noise-free images, forming a one-to-one pair. These sample pairs are then fed into the image restoration model, which learns the mapping between these sample pairs and gradually adjusts its parameters to effectively restore the image to be restored.

[0045] By obtaining a set of image samples and forming sample pairs to train the input model, two main technical effects can be achieved: first, the model can learn the image restoration rules under different degradation types and degrees, thereby improving the versatility and adaptability of the model; second, through training on a large number of sample pairs, the accuracy and quality of the model's restored images can be improved, making them closer to the real target image, thereby improving the overall performance of image restoration.

[0046] S200: Preprocess the image to be restored and extract a first feature map using a convolution operation, wherein the size of the first feature map is W×H×C, W and H are the image width and height respectively, C is the number of convolution channels, and W, H, and C are all positive integers greater than 0.

[0047] Specifically, preprocessing is a series of preliminary operations performed on an image to improve image quality, extract useful information, or reduce data complexity. By sliding a convolution kernel over the preprocessed image and performing element-wise multiplication and summation operations, local image features such as edges and textures can be extracted.

[0048] Specifically, a feature map is a two-dimensional or three-dimensional data structure representing image features obtained after a convolution operation. It contains various local feature information of the image, such as edge contours and texture details. It provides the basic feature representation for subsequent linear attention mechanism processing. The value at each position represents the strength of a certain feature in the corresponding image area. The image width (W) and height (H) are the number of pixels in the image in the horizontal and vertical directions, while the number of convolution channels (C) usually corresponds to the number of convolution kernels and represents the dimensionality of different features.

[0049] Specifically, taking a grayscale image to be restored with a size of 256×256 pixels as an example, a convolution operation is performed using a 3×3 convolution kernel: First, the convolution kernel is placed at the upper left corner of the image and element-wise multiplied with the 3×3 pixel values ​​at the corresponding position in the image. All products are then added together to obtain the first output value. Next, the convolution kernel is shifted one pixel to the right, and the above operation is repeated at the new position until the entire image width is covered. Similarly, the convolution kernel is shifted one pixel downward, and the convolution operation continues until the entire image height is covered. In this process, each convolution kernel corresponds to one channel. Assuming 16 different convolution kernels are used, the size of the first feature map obtained is 254×254×16 (because the convolution operation reduces the width and height of the image, it is assumed here that no padding operation is performed).

[0050] By extracting the first feature map through the convolution operation in the preprocessing, the local features of the image can be effectively extracted, the key structure and texture information in the image can be highlighted, and an important feature basis can be provided for subsequent image restoration tasks. At the same time, the data size can be reduced, the complexity of model processing can be reduced, and the key feature information can be retained at the same time, which helps to improve the operation efficiency and recovery performance of the model.

[0051] S300: Establish window dimension attention and channel dimension attention for the first feature map based on the linear attention mechanism, and obtain fused attention output features.

[0052] Specifically, the linear attention mechanism efficiently calculates attention weights through linear transformation, which can reduce computational complexity compared to the traditional dot-product-based attention mechanism.

[0053] Window-dimensional attention builds attention relationships across the spatial dimensions of feature maps (i.e., local regions of the image, measured in windows), highlighting the importance of local features in restoration. Channel-dimensional attention focuses on the channel direction of feature maps, determining the contribution of different channel features to the image restoration task and enhancing the expression of key channel features.

[0054] Specifically, the obtained fused attention output feature refers to the integration of the attention calculation results of the window dimension and the channel dimension to form a feature representation that integrates spatial and channel information for subsequent image restoration processing.

[0055] In some embodiments, as Figure 2 As shown, based on the linear attention mechanism, the first feature map is subjected to window dimension attention establishment and channel dimension attention establishment, and the fused attention output feature is obtained, including:

[0056] The first feature map is divided into windows to obtain a set of sub-feature maps, and a sub-window attention feature is calculated for each sub-feature map, and the window attention feature is obtained by integration.

[0057] The first feature map is divided into a set of channel sub-blocks in the channel dimension, and the sub-channel attention features are calculated for each channel sub-block, and the channel attention features are obtained by integration.

[0058] The fused attention feature map is calculated based on the first feature map, the window attention feature and the channel attention feature.

[0059] Specifically, the first feature map is first windowed to obtain a set of sub-feature maps. Assume that the size of the first feature map is 64×64×32. Dividing the image into 4×4 windows results in each sub-feature map being 4×4×32, and the entire set of sub-feature maps consists of (64 / 4)×(64 / 4)=16×16 sub-feature maps. Next, a sub-window attention feature is calculated for each sub-feature map. This can be achieved through a linear transformation to obtain the query, key, and value vectors. The attention weights are then calculated using the linear attention formula. Finally, the value vectors are combined with the weights to obtain the attention feature for each sub-window. These sub-window features are integrated to obtain the window attention feature, which is 64×64×1 (W×H). Furthermore, in the channel dimension, the 32 channels of the first feature map are divided into four channel sub-blocks, each containing 8 channels, forming a set of channel sub-blocks. The global features of each channel sub-block are calculated, such as average pooling or maximum pooling, to obtain the representative features of the channel sub-block. Then, the sub-channel attention features are obtained through operations such as linear transformation and activation function. Finally, these sub-channel features are integrated to obtain the channel attention features, whose size is 64×64×32.

[0060] Furthermore, a fused attention feature map is calculated based on the first feature map, the window attention feature, and the channel attention feature. This can be achieved by concatenating the three features, for example, into a 64×64×(32+32+1)=64×64×65 feature map in the channel dimension. This fusion method allows the feature map to simultaneously contain the original features, spatial attention information, and channel attention information, enhancing the expressive power of the features.

[0061] Through the above-mentioned step-by-step window and channel dimension attention calculation and fusion process, the following technical effects can be achieved: First, window division and sub-window attention calculation can capture the pixel relationship in the local area of ​​the image, highlight local features, and improve the model's ability to process image details; second, channel division and sub-channel attention calculation can effectively judge the importance of different channel features, enhance the feature expression of key channels, and reduce the interference of redundant information; third, the fusion of the attention features of the two dimensions of window and channel with the original features can form a feature representation that integrates spatial and channel information, making the feature map more discriminative and targeted, thereby improving the performance of the image restoration model and the quality of the restored image. At the same time, the application of the linear attention mechanism also reduces the computational complexity and improves the operation efficiency of the model.

[0062] In some embodiments, obtaining a window attention feature includes:

[0063] Based on the sliding window method, the first feature map is divided into windows of size M×M to form the sub-feature map set.

[0064] Perform a linear transformation on each sub-feature map to generate a first intermediate feature map, where the size of the intermediate feature map is W×H×3C.

[0065] The first intermediate feature map is divided into three equal parts in the channel dimension. Based on the linear attention mechanism, the first sub-linear attention feature is calculated in combination with the three-division result, and the window attention feature is further calculated.

[0066] In some implementations, a linear transformation is performed on each sub-feature map, including:

[0067] For each of the sub-feature maps, perform The linear transformation of is the weight matrix, is the bias vector, is the first intermediate feature map, and .

[0068] Specifically, first, based on the sliding window strategy, the first feature map of the input Divide into several sizes M × M The local window of ,in N is the number of windows. Then, for each sub-feature map Perform linear transformation to obtain the first intermediate feature map , which is calculated as:

[0069] ;

[0070] in, is the weight matrix, is the bias vector.

[0071] Specifically, then, the first intermediate feature map Divide the channel into three equal parts and use them as the Query, Key, and Value inputs in the attention mechanism to calculate linear attention.

[0072] Specifically, further, the linear attention result and the three-division result are fused to calculate multiple first sub-linear attention features corresponding to multiple sub-feature maps in the sub-feature map set.

[0073] In some implementations, the first sub-linear attention feature is calculated using the following equation:

[0074] ;

[0075] in, Represents the i-th first sub-linear attention feature. 、 、 is the weight matrix. 、 、 is the bias vector, Compute the result for the i-th linear attention mechanism.

[0076] Specifically, finally, the multiple first sub-linear attention features are spliced ​​to obtain the window attention feature, where the window attention feature is size.

[0077] In some embodiments, obtaining a channel attention feature includes:

[0078] The first feature map is divided into L sub-blocks in the channel dimension, each sub-block includes C / L groups of features, and the L sub-blocks generate a channel sub-block set.

[0079] Perform a linear transformation on each channel sub-block to generate the second intermediate feature map.

[0080] The second intermediate feature map is divided into three equal parts in the channel dimension. Based on the linear attention mechanism, the second sub-linear attention feature is calculated in combination with the three-division result, and the channel attention feature is further calculated.

[0081] Specifically, first, the first feature map Divided into channel dimensions L channel sub-blocks, where each sub-block contains C / L channels, forming a channel sub-block set; illustratively, the channel sub-block set is recorded as ; Then, for each channel sub-block Perform linear transformation to obtain the second intermediate feature map .

[0082] In some implementations, performing a linear transformation on each channel sub-block includes:

[0083] For each of the sub-feature maps, perform The linear transformation of is the weight matrix, is the bias vector, is the second intermediate feature map, and .

[0084] Specifically, further, the second intermediate feature map , divided into three equal parts in the channel dimension, as Query, Key, and Value respectively, and using the same method and principle as the above calculation of the first sub-linear attention feature, multiple second sub-linear attention features are calculated based on the linear attention mechanism.

[0085] In some implementations, calculating the channel attention feature includes:

[0086] The second sub-linear attention features output by each of the channel sub-blocks are spliced ​​to obtain the channel attention features with the number of channels restored to C.

[0087] Specifically, finally, the multiple second sub-linear attention features output by multiple channel sub-blocks are spliced ​​to restore the channel dimension to the original number of channels C and obtain the channel attention feature map.

[0088] Optionally, the original first feature map, the window attention feature map and the channel attention feature map are fused to obtain a fused attention feature map, wherein the fusion method includes but is not limited to weighted summation, convolution after splicing, attention weighted fusion, etc.

[0089] Through the dual linear attention modeling of the above-mentioned window dimension and channel dimension, the local spatial relationship and inter-channel dependency of image features can be more effectively captured; the feature expression ability is improved, and the model's ability to focus on the target area is enhanced; at the same time, compared with the traditional attention mechanism, the linear attention mechanism has higher computational efficiency and scalability; it is suitable for a variety of computer vision task scenarios such as image recognition, target detection, and image segmentation.

[0090] S400: Repeat K times to obtain K fused attention output features, and output them as the final feature map.

[0091] Specifically, the K-fold repetition here refers to looping the linear attention-based processing module (including the window- and channel-dimensional attention establishment and fusion processes) K times in the feature map processing flow. Each loop generates a new fused attention output feature, which then propagates and accumulates information.

[0092] For example, execution K The above fusion attention calculation process is used to obtain K The fused attention output feature map can then be selected to K The feature maps are cascaded, weighted summed, or otherwise integrated to generate a final feature map. The final feature map integrates the results of multiple attention calculations and contains richer and deeper feature information for subsequent image restoration processing.

[0093] S500: performing a convolution-based post-processing operation on the final feature map, and performing a residual connection between the convolution result and the image to be restored to obtain the restored image.

[0094] Specifically, convolution-based post-processing involves applying convolutional layers to the final feature map to further extract and transform features, generating a representation closer to the target image. Residual connections, on the other hand, directly add previous features to subsequent features, preserving the original information and mitigating the vanishing gradient problem. The restored image, the final output image after convolutional post-processing and residual connections, aims to be as close to the target image as possible, restoring the quality of the original image.

[0095] For example, first, a convolution operation is performed on the final feature map (assuming a size of 64×64×32). For example, using a 3×3 convolution kernel, the size of the feature map after convolution may become 62×62×16 (assuming the number of output channels is 16). Then, a residual connection is performed on this convolution result with the original image to be restored (assuming a size of 64×64×1 if it is a grayscale image). Because the size may not be consistent, the convolution result may need to be interpolated or otherwise adjusted to align it with the image to be restored. For example, the convolution result can be upsampled to 64×64×1 and then added to the image to be restored to obtain the restored image.

[0096] The above residual connection method allows the model to learn the residual information between the restored image and the image to be restored, that is, the part that needs to be corrected, rather than directly learning the complete image, making the learning process more efficient and easier to converge.

[0097] S600: Constructing a loss function based on the target image, and optimizing and updating the parameters of the image restoration model using a gradient descent method.

[0098] If feasible, the loss function may include but is not limited to one or more of the following: pixel-level loss (such as L1 and L2 losses); perceptual loss, which is based on feature differences extracted by pre-trained networks (such as VGG); adversarial loss (if a GAN structure is used) to improve the realism of the image; structural similarity loss (SSIM) to maintain image structural consistency.

[0099] In some embodiments, the loss function is defined as:

[0100] ;

[0101] in, represents the restored image, is the target image, and N is the number of training samples.

[0102] In some embodiments, the linear attention function in the linear attention mechanism is defined as:

[0103] ;

[0104] in, Characterize the linear attention feature of the n-th set element; , , Express Take d power bit by bit; 、 、 The result is divided into three equal parts.

[0105] Specifically, 、 、 is the result of the three equal divisions, corresponding to the above Query, Key, and Value respectively; t is the original input vector (for example 、 a vector element in ); vector The input vector t go through ReLU The intermediate vector result obtained after the activation function; Represents a vector The Euclidean norm of , used for scaling; Represents a vector Each dimension element in is taken bit by bit d power; d is a positive integer hyperparameter used to control the degree of nonlinearity of the feature map.

[0106] The above-mentioned linear attention function enhances the nonlinear expression ability of feature mapping by introducing bit-by-bit power transformation and normalization operations. Compared with the traditional linear attention mechanism, this method improves the model's ability to model high-order features while maintaining low computational complexity.

[0107] For example, tests were conducted on the image deraining datasets Rain200L and Rain200H, the image deblurring dataset HIDE, the image defogging dataset OTS, and the image desnowing dataset CSD. The results are shown in the following table. PSNR and SSIM are used as evaluation indicators. Higher values ​​indicate better image restoration effects. The number of parameters and Flops reflect the size and computational complexity of the model. Smaller values ​​are better.

[0108] Table 1. Example performance results on different datasets

[0109]

[0110] From the results, we can see that although the PSNR and SSIM of this method are slightly reduced, the number of parameters and Flops are much smaller than those of other methods, which proves that this method is more suitable for running on embedded devices.

[0111] In summary, the image restoration method integrating the linear attention mechanism provided by the present invention has the following technical effects:

[0112] By obtaining an image sample set, the image to be restored and the corresponding target image are formed into sample pairs and input into the image restoration model, where the output of the image restoration model is the restored image. The image to be restored is preprocessed and a convolution operation is used to extract a first feature map, where the size of the first feature map is W×H×C, where W and H are the image width and height respectively, and C is the number of convolution channels, and W, H, and C are all positive integers greater than 0. Window-dimensional attention and channel-dimensional attention are established on the first feature map based on a linear attention mechanism, and fused attention output features are obtained. This is repeated K times to obtain K fused attention output features, which are output as the final feature map. The final feature map is post-processed based on convolution, and the convolution result is residually connected with the image to be restored to obtain the restored image. A loss function based on the target image is constructed, and the parameters of the image restoration model are optimized and updated using the gradient descent method. This achieves the technical effect of reducing computational complexity, improving model efficiency, enhancing model applicability and performance, and maintaining or improving image restoration quality.

[0113] It should be understood that the embodiments disclosed in the present invention and the above description can enable those skilled in the art to use the present invention to implement the present invention. At the same time, the present invention is not limited to the embodiments mentioned above. It should be understood that those skilled in the art can still modify the technical solutions described in the above embodiments or replace some of the technical features therein with equivalents; and such modifications or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention and are all included in the scope of protection of the present invention.

Claims

1. An image restoration method integrating linear attention mechanism, characterized in that: include: Obtain an image sample set, form a sample pair of the image to be restored and the corresponding target image, and input the sample pair into an image restoration model, wherein the output of the image restoration model is a restored image; Preprocessing the image to be restored, extracting a first feature map using a convolution operation, wherein the size of the first feature map is W×H×C, where W and H are the image width and height, respectively, and C is the number of convolution channels, and W, H, and C are all positive integers greater than 0; Performing window-dimensional attention and channel-dimensional attention on the first feature map based on a linear attention mechanism, and obtaining fused attention output features; Repeat K times to obtain K fused attention output features, which are output as the final feature map; Performing a convolution-based post-processing operation on the final feature map, and performing a residual connection between the convolution result and the image to be restored to obtain the restored image; Constructing a loss function based on the target image and optimizing and updating the parameters of the image restoration model using a gradient descent method; Among them, based on the linear attention mechanism, window dimension attention and channel dimension attention are established on the first feature map, and the fused attention output feature is obtained, including: Performing window division on the first feature map to obtain a set of sub-feature maps, and calculating a sub-window attention feature for each sub-feature map, and integrating them to obtain a window attention feature; Dividing the first feature map into a set of channel sub-blocks in the channel dimension, and calculating a sub-channel attention feature for each channel sub-block, and integrating them to obtain a channel attention feature; The fused attention feature map is calculated based on the first feature map, the window attention feature and the channel attention feature.

2. The image restoration method integrating the linear attention mechanism according to claim 1, characterized in that: Get window attention features, including: Based on the sliding window method, the first feature map is divided into windows of size M×M to form the sub-feature map set; Perform a linear transformation on each sub-feature map to generate a first intermediate feature map, where the size of the intermediate feature map is W×H×3C; The first intermediate feature map is divided into three equal parts in the channel dimension. Based on the linear attention mechanism, the first sub-linear attention feature is calculated in combination with the three-division result, and the window attention feature is further calculated.

3. The image restoration method integrating the linear attention mechanism according to claim 1, wherein: Get channel attention features, including: Divide the first feature map into L sub-blocks in the channel dimension, each sub-block includes C / L groups of features, and the L sub-blocks form a channel sub-block set; Perform a linear transformation on each channel sub-block to generate a second intermediate feature map; The second intermediate feature map is divided into three equal parts in the channel dimension. Based on the linear attention mechanism, the second sub-linear attention feature is calculated in combination with the three-division result, and the channel attention feature is further calculated.

4. The image restoration method integrating the linear attention mechanism according to claim 2, wherein: Perform linear transformation on each sub-feature map, including: For each of the sub-feature maps, perform The linear transformation of is the weight matrix, is the bias vector, is the first intermediate feature map, and .

5. The image restoration method integrating the linear attention mechanism according to claim 3, characterized in that: Perform linear transformation on each channel sub-block, including: For each of the sub-feature maps, perform The linear transformation of is the weight matrix, is the bias vector, is the second intermediate feature map, and .

6. The image restoration method integrating the linear attention mechanism according to claim 1, wherein: The linear attention function in the linear attention mechanism is defined as: ； in, Characterize the linear attention feature of the n-th set element; , , Express Take d power bit by bit; 、 、 The result is divided into three equal parts.

7. The image restoration method integrating the linear attention mechanism according to claim 2, wherein: Calculate the first sub-linear attention feature, the operation formula is: + + ； in, Represent the i-th first sub-linear attention feature; 、 、 is the weight matrix; 、 、 is the bias vector, Compute the result for the i-th linear attention mechanism.

8. The image restoration method integrating the linear attention mechanism according to claim 3, wherein: Calculating the channel attention feature includes: The second sub-linear attention features output by each of the channel sub-blocks are spliced ​​to obtain the channel attention features with the number of channels restored to C.

9. The image restoration method integrating the linear attention mechanism according to claim 1, wherein: The loss function is defined as: ； in, represents the restored image, is the target image, and N is the number of training samples.

Images