Image super-resolution reconstruction method based on channel-aware aggregation transformer

Abstract

The application belongs to the technical field of image super-resolution, and particularly relates to an image super-resolution reconstruction method based on a channel perception aggregation Transformer. The method comprises the following steps: shallow feature extraction is performed on a low-resolution image to obtain a shallow feature map; the shallow feature map is input into a deep feature extraction module to perform deep feature extraction, and a deep feature map is obtained, wherein the deep feature extraction module comprises a plurality of residual groups, and each residual group comprises at least one double-aggregation interactive attention and layered slice attention; the deep feature map is combined with sub-pixel reorganization through convolution operation to realize preliminary recovery of a high-resolution image; and the preliminary recovered high-resolution image is superimposed and fused with global residual information extracted after global up-sampling of an original low-resolution image to generate a high-resolution reconstructed image. Through multi-scale and multi-path feature interaction, the image detail restoration capability of the model is enhanced.

Description

Technical Field

[0001] This invention belongs to the field of image super-resolution technology, specifically relating to an image super-resolution reconstruction method based on channel-aware aggregation Transformer. Background Technology

[0002] Image super-resolution techniques aim to recover a high-quality, high-resolution image from a given low-resolution image. Early methods were mostly based on interpolation algorithms or sparse representation reconstruction strategies, but these methods often struggled to recover the detailed textures in the image.

[0003] In recent years, convolutional neural networks and Transformers have been widely used in image reconstruction tasks, achieving significant progress in reconstruction quality. However, existing Transformer models have computational complexity that increases quadratically with the input resolution and consume a lot of resources, making them difficult to deploy on resource-constrained devices.

[0004] Before the widespread application of deep learning, image super-resolution mainly relied on interpolation methods and prior-based image modeling methods. For example, classic algorithms such as bilinear interpolation and bicubic interpolation achieve pixel-to-pixel interpolation through simple mathematical rules, which are fast but have limited recovery accuracy and often cause image blurring. Methods such as sparse representation and dictionary learning restore images by learning the mapping relationship between low-resolution (LR) and high-resolution (HR) images, but the overall effect is still insufficient in terms of detail restoration, especially in recovering high-frequency texture information.

[0005] With the development of deep learning technology, convolutional neural networks (CNNs) have been widely applied in super-resolution tasks, significantly improving image restoration quality. Early representative methods, such as SRCNN (Super-Resolution Convolutional Neural Network), were the first to use a three-layer CNN to directly learn the mapping relationship between LR and HR images. Subsequently, VDSR (Very Deep Super-Resolution) introduced a residual learning structure, further deepening the network layers to enhance modeling capabilities. RCAN (Residual Channel Attention Network) introduced a channel attention mechanism, effectively improving the network's ability to perceive key information. Although CNN methods have achieved significant results in SR tasks, their receptive field is limited by the size of the convolutional kernel, making it difficult to capture global dependencies between distant pixels.

[0006] In recent years, inspired by the Transformer architecture in Natural Language Processing (NLP), the Visual Transformer has been widely applied to image tasks. Its multi-head self-attention mechanism possesses powerful global modeling capabilities, effectively capturing long-distance image dependencies and making it suitable for SR tasks with complex textures and rich high-frequency details, achieving leading performance on multiple benchmark datasets. However, because the computational complexity of the Transformer increases quadratically with the input image size, it results in slow inference speed, high resource consumption, weak local detail modeling capabilities, and over-reliance on global information, potentially leading to the loss of local image texture. Summary of the Invention

[0007] To overcome the problems in the prior art, this invention proposes an image super-resolution reconstruction method based on channel-aware aggregation Transformer.

[0008] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0009] This invention provides an image super-resolution reconstruction method based on channel-aware aggregation Transformer, comprising:

[0010] Step 100: Perform shallow feature extraction on the low-resolution image to obtain a shallow feature map;

[0011] Step 200: Input the shallow feature map into the deep feature extraction module for deep feature extraction to obtain a deep feature map. The deep feature extraction module includes multiple residual groups, and each residual group includes at least one dual-convergence interactive attention and a hierarchical slice attention.

[0012] Step 300: The deep feature map is combined with convolution operation and subpixel reconstruction to achieve preliminary restoration of the high-resolution image; the preliminary restored high-resolution image is superimposed and fused with the global residual information extracted after global upsampling of the original low-resolution image to generate a high-resolution reconstructed image.

[0013] Further, step 100 specifically includes: using convolution operations to map the original low-resolution image from the original pixel space to a high-dimensional feature space, thereby obtaining a shallow feature map.

[0014] Further, step 200 specifically includes:

[0015] The shallow feature map is passed through a convolutional layer to extract the initial features of the image, and the initial features are subjected to dual-aggregation interaction to output dual-aggregation interaction image features; the dual-aggregation interaction image features are subjected to hierarchical slicing self-attention processing to obtain hierarchical slicing output features.

[0016] The output features of the layered slices are residually concatenated with the initial features to obtain the output features of the residual group, i.e., the deep feature map.

[0017] Further, the initial features are subjected to a dual-convergence interaction to output dual-convergence interactive image features, including:

[0018] The initial features are normalized by layer, and the dual-stream feature interaction output features are obtained through dual-stream feature interaction. The dual-stream feature interaction output features are divided into a first feature stream and a second feature stream by channel segmentation.

[0019] The first feature stream is subjected to multi-scale spatial compression attention processing, and the second feature stream is subjected to depthwise separable convolution processing. The processed first feature stream and the processed second feature stream are fused to output dual-aggregated interactive image features.

[0020] Furthermore, through two-stream feature interaction, the output features of the two-stream feature interaction are obtained, including: performing two-stream feature interaction on the features after layer normalization operation. The two-stream feature interaction adopts a parallel structure, combining convolutional branches and linear branches, jointly modeling the spatial structure information and channel dependency of the features, and multiplying the output features of the two branches element by element to obtain the output features of the two-stream feature interaction.

[0021] Furthermore, the first feature stream undergoes multi-scale spatial compression attention processing, the second feature stream undergoes depthwise separable convolution processing, and the processed first and second feature streams are fused to output dual-aggregated interactive image features, including:

[0022] The first feature stream is fed into a multi-scale spatial compression attention module for global dependency modeling to obtain the processed first feature stream; at the same time, the second feature stream is fed into a depthwise separable convolution module for local context modeling to obtain the processed second feature stream.

[0023] The processed first feature stream, the processed second feature stream, and the initial features are added element-wise to obtain the first fused feature; the first fused feature is fed into a feedforward network for nonlinear enhancement, and the enhanced first fused feature is residually connected with the first fused feature to obtain the dual-aggregated interactive image feature.

[0024] Furthermore, the first feature stream is fed into a multi-scale spatial compression attention module for global dependency modeling, including:

[0025] Apply a linear transformation to the first feature stream to generate a query vector and compress its feature dimensions;

[0026] A depthwise separable convolution is performed on the first input feature stream to obtain a spatially compressed feature map. A depthwise separable convolution and a 1×1 convolution are then applied to the spatially compressed feature map to perform further feature transformation, resulting in a transformed feature map. Different linear transformations are then applied to the transformed feature map to generate key vectors and value vectors, respectively.

[0027] The attention score is calculated by performing a dot product between the key vector and the query vector, multiplied by a scaling factor, and then normalized using the Softmax function to calculate a weighted sum, resulting in the output of the multi-scale spatially compressed attention module.

[0028] Further, the dual-aggregate interactive image features are subjected to hierarchical slicing self-attention processing to obtain hierarchical slicing output features, including:

[0029] The dual-aggregate interactive image features are then subjected to layer normalization again; the layer-normalized dual-aggregate interactive image features are then weighted using a piecewise attention mechanism to obtain the features processed by the piecewise attention mechanism.

[0030] The features processed by the slice attention mechanism are residually connected with the image features obtained by the double aggregation interaction to obtain the second fused feature. The second fused feature is then fed into a feedforward network for nonlinear enhancement. The enhanced second fused feature is residually connected with the second fused feature and passed through a convolutional layer to obtain the slice output feature.

[0031] Furthermore, the double-aggregated interactive image features after layer normalization are weighted using a piecewise attention mechanism to obtain the features processed by the piecewise attention mechanism, including:

[0032] The double-aggregated interactive image features after layer normalization are divided into several local regions. In each local region, the attention mechanism is calculated independently. The attention mechanism calculation includes mapping the features in the local region into query vector, key vector and value vector respectively.

[0033] An attention mechanism is executed based on the query vector, key vector, and value vector within each local region to obtain local attention results. All local region attention results are then restored according to their spatial location and stitched together to form a complete feature map, which is the feature map processed by the piecewise attention mechanism.

[0034] Furthermore, in step 300, the preliminary restoration of the high-resolution image by combining convolution operations and sub-pixel reconstruction of the deep feature map includes:

[0035] The deep feature map is convolved, and then upsampled to a high-resolution space through subpixel recombination to obtain a preliminary high-resolution image.

[0036] Compared with the prior art, the present invention has the following technical effects:

[0037] (1) This invention proposes to effectively capture the global dependencies and local details of an image through dual-convergent interactive attention and hierarchical slice attention. The multi-scale spatial compression attention processing and depthwise separable convolution processing in dual-convergent interactive attention model the features from different angles, enhancing the network's ability to perceive key information in the image; hierarchical slice attention reduces computational complexity through slice processing, while improving local representation capabilities. These mechanisms work together to enable the model to more accurately recover the detailed textures in the image, significantly improving the quality of the reconstructed image.

[0038] (2) The hierarchical slicing attention mechanism of the present invention divides the feature map into local regions and performs attention calculations independently within each region, which greatly reduces the amount of computation and lowers the computational complexity. This allows the model to run more efficiently while maintaining good reconstruction performance and reducing its dependence on computing resources.

[0039] (3) The method of this invention employs multiple residual group structures in the deep feature extraction module and fuses features from different stages through residual connections and other methods. This design not only helps to effectively propagate gradients and avoid gradient vanishing or exploding problems, making the model easier to train, but also enhances the model's adaptability to different types of images. The model can stably learn effective feature representations, thereby generating high-quality, high-resolution reconstructed images, improving the reliability and stability of the model in practical applications. Attached Figure Description

[0040] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating the image super-resolution reconstruction method based on channel-aware aggregation Transformer of the present invention.

[0042] Figure 2 This is a structural diagram of the image super-resolution reconstruction method based on channel-aware aggregation Transformer of the present invention;

[0043] Figure 3 This is a structural diagram of the dual-stream feature interaction module of the present invention;

[0044] Figure 4 This is a diagram of the multi-scale spatial compressed attention structure of the present invention;

[0045] Figure 5 The first original image selected;

[0046] Figure 6 (a) is Figure 5 The low-resolution image corresponding to the detailed area extracted in the image, (b) is Figure 5 (c) is the original high-resolution image corresponding to the detailed region extracted in the image, and (c) is the super-resolution image obtained after the low-resolution image is super-resolution reconstructed.

[0047] Figure 7 The selected second original image;

[0048] Figure 8 (a) is Figure 7 The low-resolution image corresponding to the detailed area extracted in the image, (b) is Figure 7 (c) is the original high-resolution image corresponding to the detailed region extracted in the image, and (c) is the super-resolution image obtained after the low-resolution image is super-resolution reconstructed.

[0049] Figure 9 The selected third original image;

[0050] Figure 10 (a) is Figure 9 The low-resolution image corresponding to the detailed area extracted in the image, (b) is Figure 9 (c) is the original high-resolution image corresponding to the detailed region extracted in the image, and (c) is the super-resolution image obtained after the low-resolution image is super-resolution reconstructed.

[0051] Figure 11 This is a diagram of the sliced ​​self-attention structure of the present invention;

[0052] Figure 12 This is a design diagram for a layered window in a sliced ​​self-attention model. Detailed Implementation

[0053] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solutions proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. Specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0054] In this embodiment, refer to Figures 1-12 This invention provides an image super-resolution reconstruction method based on channel-aware aggregation Transformer, comprising:

[0055] Step 100: Perform shallow feature extraction on the low-resolution image to obtain a shallow feature map;

[0056] Step 200: Input the shallow feature map into the deep feature extraction module for deep feature extraction to obtain a deep feature map. The deep feature extraction module includes multiple residual groups, and each residual group includes at least one dual-convergence interactive attention and a hierarchical slice attention.

[0057] Step 300: The deep feature map is combined with convolution operation and subpixel reconstruction to achieve preliminary restoration of the high-resolution image; the preliminary restored high-resolution image is superimposed and fused with the global residual information extracted after global upsampling of the original low-resolution image to generate a high-resolution reconstructed image.

[0058] The following is a detailed explanation of each of the above steps:

[0059] Step 100: Perform shallow feature extraction on the low-resolution image to obtain a shallow feature map.

[0060] The publicly available dataset DIV2K was selected as the training dataset. The input low-resolution image was cropped to the correct size. High-resolution image cropping to ,in r This is the scaling factor.

[0061] For the input low-resolution image Convolution operations are used to map the original low-resolution image from the original pixel space to a high-dimensional feature space, thereby obtaining a shallow feature map with C channels. ,in, C represents the spatial dimension, and C represents the number of channels.

[0062] ;

[0063] In the above formula, It is a 3×3 convolution operation.

[0064] Step 200: Input the shallow feature map into the deep feature extraction module for deep feature extraction to obtain a deep feature map; wherein, the deep feature extraction module includes multiple residual groups, and each residual group includes at least one dual-aggregation interactive attention and a hierarchical slice attention.

[0065] The shallow feature map The input is processed in a deep feature extraction module consisting of N residual groups (RGs) cascaded together. Convolution is used after the Nth residual group to refine the features and improve the reconstruction quality.

[0066] No. i Residual group For the first i- The output of one residual group is processed and then passed through a 3×3 convolution. Further adjust the features, and finally compare the processed results with the first... i- Output of a residual set Add them together, and output the N residual sets. F That is, deep feature map.

[0067] ;

[0068] In the above formula, Indicates the first i One set of residuals, Indicates the first i The output of each residual set; Indicates the first i- The output of one residual group. Residual connections ensure that gradients can be backpropagated effectively, avoiding the vanishing or exploding gradient problem, making the network easier to train, and better preserving and utilizing the feature information from the previous stage.

[0069] In this implementation, the residual set is mainly divided into three parts: Dual Convergence Interaction Transformer (DAIT), Hierarchical Slicing Transformer (HST), and convolutional layers. The convolutional layers are used to optimize and enhance the feature representation within the current module.

[0070] Step 210: Shallow Feature Map After passing through the first convolutional layer (Conv), the initial features of the image are extracted.

[0071] The first convolutional layer, Conv, extracts the initial features of the image by capturing local feature information through the sliding operation of the convolutional kernel across the image. Specifically, the first convolutional layer, Conv, uses a single 3x3 convolution.

[0072] Step 220: Perform dual-aggregation interaction on the initial features and output dual-aggregation interaction image features.

[0073] As an example, step 220 specifically includes the following sub-steps:

[0074] Step 2201: Perform LayerNorm operation on the initial features.

[0075] The LayerNorm operation normalizes the initial feature maps output by convolutional layers, which helps to accelerate the training convergence speed of the model and improve the stability of the model.

[0076] Step 2202: After performing layer normalization on the initial features, the dual-stream feature interaction (DSFI) output features are obtained through dual-stream feature interaction.

[0077] The features after LayerNorm operation are subjected to dual-stream feature interaction (DSFI). The dual-stream feature interaction (DSFI) adopts a parallel structure, combining convolutional branches and linear branches to jointly model the spatial structure information and channel dependencies of the features. The output features of the two branches are multiplied element-wise to obtain the output features of dual-stream feature interaction (DSFI).

[0078] Among them, reference Figure 3 Convolutional branches Employing a Bottleneck-like design, it includes sequentially concatenated 1×1 convolutions (Conv), ReLU activation, 3×3 convolutions (Conv), ReLU activation, and then another 1×1 convolution (Conv), demonstrating strong local feature extraction capabilities; linear branches The input features are directly mapped to the target dimension using a single-layer linear transformation (Linear) to capture global dependencies at the channel level. The outputs of the two branches are fused through element-wise multiplication, thereby introducing a channel-selective feature enhancement mechanism while maintaining spatial structure awareness. Specifically, let the input feature, i.e., the feature after layer normalization, be X, then the fused output is:

[0079] ;

[0080] in, This represents the output features of the dual-stream feature interaction DSFI. This indicates an element-wise multiplication operation. This fusion strategy effectively integrates multi-scale information at the spatial and channel levels, providing rich semantic support for subsequent high-quality reconstruction.

[0081] Step 2203: Divide the dual-stream feature interaction DSFI output features into a first feature stream and a second feature stream through channel splitting operation.

[0082] The DSFI output features of the two-stream feature interaction are subjected to dilated convolution (i.e., 1×1 convolution) to double the number of channels to twice the original dimension, and then... split The operation uniformly divides the expanded two-stream feature interaction DSFI output features into a first feature stream along the channel dimension. Second characteristic flow ,Right now:

[0083] ;

[0084] In the above formula, Split This represents a channel splitting operation.

[0085] Step 2204: Perform multi-scale spatial compression attention processing on the first feature stream, perform depthwise separable convolution processing on the second feature stream, fuse the processed first feature stream and the processed second feature stream, and output dual-aggregated interactive image features.

[0086] The first feature stream is fed into the Multi-Scale Spatial Compressed Attention (MSCA) module for global dependency modeling, resulting in the processed first feature stream; simultaneously, the second feature stream is fed into the Depthwise Separable Convolution (DWConv) module for local context modeling, resulting in the processed second feature stream.

[0087] Among them, the multi-scale spatial compression attention module effectively reduces the computational cost of traditional self-attention mechanisms under high-resolution input by introducing an iterative depthwise convolutional downsampling and channel dimension compression mechanism. Its architecture is as follows: Figure 4 As shown. For the first characteristic flow Where B represents the batch size. Let C be the spatial dimension and C be the number of channels. The query vector Q generation includes: applying channel compression to the first feature stream, followed by a linear transformation to generate the query vector Q, and compressing its feature dimension to 1.

[0088] Key and value vector generation includes: performing depthwise separable convolution on the first feature stream, followed by spatial downsampling through four depthwise separable convolutions with a stride of 2. Each operation halves the spatial resolution of the feature map, thereby reducing the original spatial dimension. Reduce the spatial dimension to a smaller size to obtain a spatially compressed feature map; apply a depthwise separable convolution and a 1×1 convolution to the spatially compressed feature map to further transform the features and obtain a transformed feature map; then apply a linear transformation to the transformed feature map to generate a key vector K and a value vector V, where the feature dimension of K is compressed to 1, while the feature dimension of V remains unchanged.

[0089] During attention computation, when constructing the key and query vectors, channel compression reduces the channel dimension from C to 1, effectively reducing the number of parameters and computational overhead in the dot product operation. The channel-compressed query vector is then multiplied by the transpose of the key vector, and divided by a scaling factor (the square root of the key vector dimension) to perform a scaling operation, stabilizing the numerical distribution and preventing the Softmax gradient from becoming too small. A weighting operation is then performed, followed by normalization using the Softmax function to obtain the weight coefficients. Finally, these weight coefficients are multiplied by the value vector V to obtain the attention output A.

[0090] ;

[0091] In the above formula, This represents the dimension of the key vector.

[0092] Ultimately, attention is output in the original spatial dimension. Projection is performed on the surface to ensure effective utilization and preservation of global context information. The multi-scale spatial compression attention module further reduces the computational burden through channel dimension compression, achieving the goal of significantly improving computational efficiency while maintaining the ability to model global dependencies.

[0093] The processed first feature stream (output of the multi-scale spatial compression attention module), the processed second feature stream (output of the depthwise separable convolution module), and the dual-aggregate interactive input image feature X0 (initial feature) are added element-wise to obtain the first fused feature. This fused feature is then fed into a feedforward network (FFN) for nonlinear enhancement and residually connected to the first fused feature to obtain the dual-aggregate interactive image feature.

[0094] ;

[0095] in, Representing the first fusion characteristic, Represents depthwise convolution. It represents dual-aggregate interactive image features.

[0096] Step 230: Perform hierarchical slicing self-attention processing on the dual-aggregate interactive image features to obtain hierarchical slicing output features.

[0097] As an example, step 230 specifically includes the following sub-steps:

[0098] Step 2301: Perform LayerNorm operation again on the dual-aggregated interactive image features.

[0099] Step 2302: Weight the double-aggregated interactive image features after layer normalization using a slice attention mechanism to obtain the features processed by the slice attention mechanism.

[0100] By introducing a sliced ​​self-attention module (HSA), computational complexity is reduced and local representation capabilities are improved. Its architecture is as follows: Figure 11 As shown, the method includes: normalizing the dual-converged interactive image features of the layers. Image segmentation is performed, dividing the image into several local regions based on a set size. The method for dividing local regions is as follows: Figure 12 As shown, each region has a size of P×P, and is divided into a total of [number] regions. Each local region is divided into three parts. Within each local region (the size of which is determined by the network depth), the attention mechanism is calculated independently, including mapping the features within the region to queries (Q), keys (K), and values ​​(V). The attention mechanism is then applied based on the Q, K, and V values ​​within each local region to obtain the local attention results. All local region attention results are then restored according to their spatial locations and stitched together to form a complete feature map, which is the feature map processed by the piecewise attention mechanism.

[0101] Step 2303: Perform residual connection between the features processed by the slice attention mechanism and the image features obtained by the double aggregation interaction to obtain the second fused feature; feed the second fused feature into a feedforward network (FFN) for nonlinear enhancement to obtain the enhanced second fused feature; perform residual connection between the enhanced second fused feature and the second fused feature, and pass it through two 3×3 convolutional layers to obtain the slice output feature.

[0102] Step 240: Perform residual concatenation between the layered slice output features and the initial features to obtain the output feature map of the residual group, i.e., the deep feature map.

[0103] Step 300: The deep feature map is combined with convolution operation and subpixel reconstruction to achieve preliminary restoration of the high-resolution image; the preliminary restored high-resolution image is superimposed and fused with the global residual information extracted after global upsampling of the original low-resolution image to generate a high-resolution reconstructed image.

[0104] Output characteristics of the residual set F Perform 3×3 convolution, then reassemble via subpixel. Pixel The operation upsamples it to a high-resolution space, obtaining a preliminary reconstructed high-resolution image. Simultaneously, the original low-resolution image is... Perform upsampling operation Up Finally, the initially restored high-resolution image is superimposed and fused with the global residual information extracted from the original low-resolution image after global upsampling. This fully utilizes the information from the original low-resolution image and the high-frequency detail information learned from deep features to generate a high-quality high-resolution reconstructed image. :

[0105] ;

[0106] in, This represents the reconstructed high-resolution image. Pixel Indicates subpixel recombination, Up This indicates an upsampling operation, which can be a bilinear interpolation operation. F This represents the output of N residual groups.

[0107] The present invention also includes: step 400: using the L1 loss function to perform supervised learning training on the reconstructed image and the real image, and using the Adam optimizer for iterative updates.

[0108] ;

[0109] in, n Total number of pixels For predicted values, This is the actual value.

[0110] The training iterations were set to 500,000 iterations, trained on standard SISR datasets (such as DIV2K), and evaluated on benchmark datasets such as Set5, Set14, BSD100, Urban100, and Manga109. The model was optimized using the Adam optimizer with a learning rate set to [value missing]. The loss function is L1 loss, designed to minimize the pixel difference between the reconstructed image and the original HR image. During the testing phase, a self-integrated inference strategy is employed, which enhances the input image and averages the output to further improve reconstruction quality.

[0111] During the testing phase, this invention evaluates super-resolution images using two metrics based on the Y channel of the YCbCr color space. Specifically, PSNR (Peak Signal-to-Noise Ratio) is used to measure image reconstruction error, and SSIM (Structural Similarity Score) is used to measure image structural fidelity.

[0112] The experimental results of image reconstruction using the proposed method on the Set14 and Urban100 benchmark datasets show a reconstruction scaling factor of 4. Figure 5 The original image is presented, and to explore the detailed characteristics of the image in depth, from... Figure 5 The image shows a portion of representative details (highlighted in red). Figure 6 (a) shows the low-resolution image corresponding to the cropped detail area, where the details are blurred; Figure 6 (b) shows the original high-resolution image corresponding to the cropped detail area. Figure 6 Figure (c) shows the image result obtained after reconstructing a low-resolution image using the method of the present invention. Figure 7 Another original image was presented, from Figure 7 The image precisely captures some representative details (red box). Figure 8 (a) shows the low-resolution image corresponding to the cropped detail area. Figure 8 (b) shows the original high-resolution image corresponding to the cropped detail area. Figure 8 Figure (c) shows the image result obtained after reconstructing a low-resolution image using the method of the present invention. Figure 9Another original image was presented, from Figure 9 The image shows a portion of representative details (highlighted in red). Figure 10 (a) shows the low-resolution image corresponding to the cropped detail area. Figure 10 (b) shows the original high-resolution image corresponding to the cropped detail area. Figure 10 Figure (c) shows the image result obtained after reconstructing a low-resolution image using the method of the present invention.

[0113] The image comparison results clearly demonstrate that the algorithm proposed in this invention has significant advantages in image edge sharpness, texture restoration capability, and preservation of structural details. The reconstructed image visually closely resembles the original high-resolution image, accurately restoring complex texture regions and fine-grained structures, exhibiting excellent image perceptual quality. These results fully demonstrate that the super-resolution reconstruction method proposed in this invention can significantly improve the subjective visual quality of images, and it has broad application prospects and practical value in areas such as image magnification and enhancement.

[0114] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for image super-resolution reconstruction based on channel-aware aggregation Transformer, characterized in that, The method comprises the following steps: Step 100: shallow feature extraction is performed on the original low-resolution image to obtain a shallow feature map; Step 200: the shallow feature map is input into a deep feature extraction module to perform deep feature extraction, and a deep feature map is obtained; the deep feature extraction module comprises a plurality of residual groups, and each residual group comprises at least one double-aggregation interactive attention and hierarchical slice attention; Step 300: the deep feature map is subjected to combined convolution operation and sub-pixel reorganization to realize preliminary recovery of a high-resolution image; the preliminary recovered high-resolution image is superimposed and fused with global residual information extracted from the original low-resolution image after global up-sampling to generate a high-resolution reconstructed image; The step 200 specifically comprises: The shallow feature map is subjected to a convolution layer to extract initial features of the image and perform double-aggregation interaction on the initial features to output double-aggregation interactive image features, including: performing layer normalization operation on the initial features, performing double-flow feature interaction to obtain double-flow feature interaction output features; the double-flow feature interaction output features are divided into a first feature flow and a second feature flow through a channel segmentation operation; the first feature flow is subjected to multi-scale spatial compression attention processing, and the second feature flow is subjected to depth separable convolution processing; the processed first feature flow and the processed second feature flow are fused to output the double-aggregation interactive image features; The double-aggregation interactive image features are subjected to hierarchical slice self-attention processing to obtain hierarchical slice output features, including: the double-aggregation interactive image features are subjected to layer normalization operation again; the double-aggregation interactive image features subjected to layer normalization operation are weighted through a slice attention mechanism to obtain features subjected to slice attention mechanism processing; the features subjected to slice attention mechanism processing are connected in residual to the image features obtained through double-aggregation interaction to obtain second fusion features; the second fusion features are input into a feedforward network for nonlinear enhancement, the enhanced second fusion features are connected in residual to the second fusion features, and the layer slice output features are obtained through a convolution layer; The hierarchical slice output features are connected in residual to the initial features to obtain output features of the residual group, that is, the deep feature map.

2. The channel-aware aggregation Transformer-based image super-resolution reconstruction method according to claim 1, characterized in that, The step 100 specifically comprises: using convolution operation to map the original low-resolution image from the original pixel space to a high-dimensional feature space to obtain the shallow feature map.

3. The channel-aware aggregation Transformer-based image super-resolution reconstruction method according to claim 1, characterized in that, The double-flow feature interaction output features are obtained through double-flow feature interaction, including: the features subjected to layer normalization operation are subjected to double-flow feature interaction, the double-flow feature interaction adopts a parallel structure, combines a convolution branch and a linear branch, jointly models spatial structure information and channel dependency of the features, and element-wise multiplies output features of the two branches to obtain the double-flow feature interaction output features.

4. The channel-aware aggregation Transformer-based image super-resolution reconstruction method according to claim 3, characterized in that, The first feature flow is subjected to multi-scale spatial compression attention processing, the second feature flow is subjected to depth separable convolution processing, the processed first feature flow and the processed second feature flow are fused to output the double-aggregation interactive image features, including: The first feature stream is sent into a multi-scale spatial compression attention module for global dependency modeling, and a processed first feature stream is obtained; meanwhile, the second feature stream is sent into a depth separable convolution module for local context modeling, and a processed second feature stream is obtained; The processed first feature stream, the processed second feature stream and the initial feature are added element by element to obtain a first fusion feature; the first fusion feature is sent into a feedforward network for nonlinear enhancement, and the enhanced first fusion feature is connected in residual with the first fusion feature to obtain a double-aggregated interactive image feature.

5. The channel-aware aggregation Transformer-based image super-resolution reconstruction method according to claim 4, characterized in that, The first feature stream is sent into a multi-scale spatial compression attention module for global dependency modeling, including: A linear transformation is applied to the first feature stream to generate a query vector, and the feature dimension thereof is compressed; A depth separable convolution is performed on the first input feature stream to obtain a spatially compressed feature map; a one-time depth separable convolution and a one-time 1×1 convolution are applied to the spatially compressed feature map in sequence for further feature transformation to obtain a feature-transformed feature map; then different linear transformations are applied to the feature-transformed feature map to respectively generate a key vector and a value vector; An attention score is calculated by dot product of the key vector and the query vector, multiplied by a scaling factor, and then normalized by a Softmax function to calculate a weighted sum, to obtain an output of the multi-scale spatial compression attention module.

6. The channel-aware aggregation Transformer-based image super-resolution reconstruction method according to claim 1, characterized in that, The layer-normalized double-aggregated interactive image feature is weighted processed by a patch attention mechanism to obtain a feature processed by the patch attention mechanism, including: The layer-normalized double-aggregated interactive image feature is divided into a plurality of local regions, and in each local region, attention mechanism calculation is independently performed, including respectively mapping the features in the local region into a query vector, a key vector and a value vector; Based on the query vector, the key vector and the value vector in each local region, the attention mechanism is executed to obtain a local attention result; all local region attention results are restored and spliced into a complete feature map according to the spatial position, that is, the feature processed by the patch attention mechanism.

7. The channel-aware aggregation Transformer-based image super-resolution reconstruction method according to claim 1, characterized in that, In step 300, the deep feature map is combined with convolution operation and sub-pixel reorganization to realize preliminary recovery of the high-resolution image, including: The deep feature map is subjected to convolution operation, and is up-sampled to a high-resolution space by sub-pixel reorganization operation to obtain a preliminary recovered high-resolution image.

Images