An image super-resolution reconstruction method and system based on a multi-scale strip window Transformer

Abstract

The application discloses an image super-resolution reconstruction method and system based on a multi-scale strip window Transformer, proposes a multi-scale strip window self-attention mechanism, divides input features into multiple groups along a channel dimension and respectively feeds into different size horizontal and vertical strip windows for parallel self-attention calculation, and simultaneously models long-distance anisotropic structures and multi-scale dependent relationships of an image; meanwhile, a residual distillation Transformer block is used to extract deep features through an incremental feature distillation strategy, and a gated deep convolution feedforward network is combined to enhance effective features and suppress redundant information. Finally, the multi-stage output features are fused, and a high-resolution image is reconstructed through sub-pixel convolution layer upsampling. The application significantly improves the super-resolution reconstruction quality under the premise of keeping lightweight parameter quantity and calculation quantity, and is particularly suitable for complex scene image reconstruction with rich texture.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image super-resolution reconstruction method and system based on a multi-scale strip window Transformer. Background Technology

[0002] Image super-resolution aims to reconstruct high-quality, high-resolution images from degraded, low-resolution images. This technology has broad application prospects in fields such as medical imaging, autonomous driving, industrial inspection, and satellite observation. With the increasing demand for the deployment of deep learning models in mobile devices and edge computing scenarios, lightweight image super-resolution has gradually become an important research direction, with the goal of achieving high-quality image reconstruction with limited computing resources.

[0003] In recent years, deep learning-based algorithms have made significant progress in this field. While Convolutional Neural Networks (CNNs) have significantly improved image reconstruction quality, their inherent limitations in capturing long-range global contextual information stem from the fact that convolutional operations are content-independent and constrained by local receptive fields, making it difficult to effectively recover complex textures and fine structures. Many CNN-based methods use spatially invariant kernels to extract local features, failing to flexibly and adaptively model the relationships between pixels. To overcome this deficiency, the Transformer architecture, based on a self-attention mechanism, improves performance through its global modeling capabilities. The main advantage of the Transformer in image super-resolution tasks lies in its powerful long-range dependency modeling ability.

[0004] However, the computational complexity of self-attention mechanisms increases quadratically with increasing spatial resolution. To reduce computational complexity, existing methods often divide the image into non-overlapping square windows, compute self-attention within each window, and implement information exchange between windows through a sliding window mechanism. SwinIR provides a lightweight version that utilizes window self-attention and a shifting window mechanism to reduce the computational complexity of the Transformer. ELAN computes self-attention on features of different window sizes by grouping multi-scale self-attention modules to explore long-range dependencies. ESWT introduces a striped window mechanism to more efficiently model long-range dependencies.

[0005] However, existing models typically employ fixed-size window partitioning within the network. This simplistic square window partitioning ignores the anisotropic structural features of natural images, making it impossible for the model to simultaneously capture long-range dependencies and fine-grained features. Although some research has attempted to use striped window self-attention mechanisms, these still rely on a single fixed-scale striped window, failing to model multi-scale texture patterns simultaneously. They can only adapt to features of specific sizes, performing poorly in complex scenes containing both fine details and large-scale structures. Furthermore, existing lightweight Transformer architectures often employ simple module stacking designs, failing to incorporate feature distillation strategies to further reduce computational overhead, making it difficult to achieve effective improvements in reconstruction accuracy with limited parameters and computational resources. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide an image super-resolution reconstruction method and system based on multi-scale strip window Transformer, which significantly improves the quality of super-resolution reconstruction while maintaining a lightweight parameter and computational load, and is particularly suitable for image reconstruction of complex scenes with rich textures.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: an image super-resolution reconstruction method based on multi-scale strip window Transformer, comprising the following steps:

[0008] S1: Input low-resolution image I LR shallow features F0 are extracted using convolutional layers;

[0009] S2: The shallow feature F0 is fed into a deep feature extraction module consisting of K residual distillation Transformer blocks connected in series to obtain multi-stage output features F1, F2, ..., F K Each residual distillation Transformer block employs a progressive residual feature distillation strategy, dividing the input features into a feature extraction branch and a feature compression branch for parallel processing. The feature extraction branch uses stacked multi-scale strip window Transformer modules to capture long-range dependencies, while the feature compression branch achieves channel dimensionality reduction through 1×1 convolution.

[0010] S3: Concatenate the output features from each stage along the channel dimension, then perform feature fusion and channel calibration by sequentially passing them through a 1×1 convolutional layer and a 3×3 convolutional layer. Finally, add the deep feature F to the shallow feature F0 through a residual connection to obtain the deep feature F. deep ;

[0011] S4: Transfer the deep feature F deep The image is fed into a subpixel convolutional layer for upsampling to generate a super-resolution reconstructed image I. SR .

[0012] In a preferred embodiment, the multi-scale strip window Transformer module in step S2 includes a multi-scale strip window self-attention mechanism and a gated deep convolutional feedforward network;

[0013] The multi-scale strip window self-attention mechanism is used to divide the input features into multiple groups of features along the channel dimension, and feed each group of features into strip window self-attention units with different window sizes to realize multi-scale self-attention feature modeling. The outputs of each group are concatenated along the channel dimension and then fused by 1×1 convolution.

[0014] The gated depthwise convolutional feedforward network is used to process features through a dual-branch gated structure. One branch is multiplied element-wise with the features of the other branch after being subjected to depthwise separable convolution and GELU activation, so as to amplify effective features and suppress redundant features.

[0015] In a preferred embodiment, the multi-scale strip window self-attention mechanism is specifically implemented as follows:

[0016] Input features F in Divide the features into three groups equally along the channel dimension: The three sets of features, corresponding to large, medium, and small window sizes respectively, are fed into the corresponding strip window self-attention units for processing. The outputs of the three strip window self-attention units are concatenated along the channel dimension and then fused by a 1×1 convolution to obtain the output feature F. out .

[0017] In a preferred embodiment, the strip window self-attention unit is specifically implemented as follows:

[0018] Input features Equally split into horizontal branch features along the channel dimension With vertical branch features Where, e∈{s,m,l};

[0019] For the horizontal branch feature After performing the horizontal stripe window segmentation operation, self-attention calculation is performed, and then the feature size is restored through the horizontal stripe window inverse segmentation operation to obtain... ;

[0020] For the vertical branch feature After performing the vertical stripe window segmentation operation, self-attention calculation is performed, and then the feature size is restored through the inverse vertical stripe window segmentation operation to obtain... ;

[0021] Will and By concatenating along the channel dimension, the output features of the striped window self-attention unit are obtained. .

[0022] In a preferred embodiment, in the self-attention calculation, a query matrix Q is obtained from the input feature X. The query matrix Q also serves as the key matrix K, resulting in a numerical matrix V. The self-attention calculation formula is as follows:

[0023]

[0024] Where d is the channel dimension scaling factor.

[0025] In a preferred embodiment, the gated depthwise convolutional feedforward network is specifically implemented as follows:

[0026] Input features Simultaneously, two parallel branches are fed in, and each branch performs feature upscaling through 1×1 convolution, and then extracts high-level features through 3×3 depthwise separable convolution.

[0027] Perform GELU activation on the features of one branch, and then multiply them element-wise with the features of the other branch;

[0028] Finally, a 1×1 convolution is used to restore the feature dimensions to the initial input dimensions, resulting in the output feature F. out .

[0029] In a preferred embodiment, the residual distillation Transformer block in step S2 progressively optimizes the residual features through three distillation steps. In each step, the features are divided into two parallel branches: the feature extraction branch is processed using two stacked multi-scale strip window Transformer modules, and the feature compression branch achieves channel dimensionality reduction through 1×1 convolution. The distilled features obtained from each distillation step are concatenated in the channel dimension and then fused sequentially through 1×1 convolution and 3×3 convolution. The fused features are then added to the input features of the residual distillation Transformer block through residual concatenation to obtain the output features.

[0030] This invention also provides an image super-resolution reconstruction system based on a multi-scale strip window Transformer, which runs the aforementioned image super-resolution reconstruction method based on a multi-scale strip window Transformer, including:

[0031] The shallow feature extraction module is used to extract shallow features from the input low-resolution image through a convolutional layer;

[0032] The deep feature extraction module consists of K residual distillation Transformer blocks connected in series. It is used to extract deep features from the shallow features and output multi-stage features. The residual distillation Transformer block adopts a progressive residual feature distillation strategy, including a feature extraction branch and a feature compression branch. The feature extraction branch adopts stacked multi-scale strip window Transformer modules, and the feature compression branch adopts 1×1 convolution.

[0033] The feature fusion module is used to concatenate the output features of each stage in the channel dimension, perform feature fusion and channel calibration through a 1×1 convolutional layer and a 3×3 convolutional layer in sequence, and add them to the shallow features through residual connection;

[0034] The upsampling module uses subpixel convolutional layers to upsample the fused deep features and generate a super-resolution reconstructed image.

[0035] In a preferred embodiment, the multi-scale strip window Transformer module includes:

[0036] The multi-scale strip window self-attention unit is used to divide the input features into multiple groups of features along the channel dimension and feed them into strip window self-attention modules with different window sizes. The outputs of each group are concatenated along the channel dimension and then fused by 1×1 convolution.

[0037] The gated depthwise convolutional feedforward network unit is used to process features through a two-branch gated structure, where one branch is multiplied element-wise with the features of the other branch after being subjected to depthwise separable convolution and GELU activation.

[0038] The multi-scale strip window self-attention unit is configured with a batch normalization layer at its front end, and the gated depthwise convolutional feedforward network unit is configured with a layer normalization layer at its front end. Both units are configured with residual connections.

[0039] The present invention also provides a computer-readable storage medium, comprising:

[0040] The computer-readable storage medium contains a computer program;

[0041] The computer program is executed by the processor as described in the image super-resolution reconstruction method based on a multi-scale strip window Transformer.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] This invention divides input features into multiple groups along the channel dimension and feeds them into horizontal and vertical strip windows of different sizes for parallel self-attention computation. This enables the model to simultaneously model the anisotropic features and multi-scale dependencies of images under lightweight constraints. The use of anisotropic strip windows overcomes the symmetry limitations of traditional square windows on geometric boundaries, significantly expanding the long-range receptive field in both horizontal and vertical directions. Simultaneously, the parallel design of multi-scale windows allows the network to adaptively match fine-grained lines and large-scale structures of different sizes within a single layer. While significantly reducing the computational cost of the lightweight model, it addresses the pain point of insufficient reconstruction of anisotropic heterogeneous textures by traditional fixed windows, effectively improving the adaptability to complex scenes.

[0044] This invention employs a progressive residual feature distillation strategy, dividing features into compression and extraction branches. The extraction branch captures long-range dependencies through stacked multi-scale strip window Transformer modules, while the compression branch controls computational load through 1×1 1×1 convolutional dimensionality reduction, thereby ensuring network lightweighting while progressively refining features.

[0045] This invention amplifies effective features and suppresses redundant information through a dual-branch gating structure, and further reduces computational overhead by combining deep convolution, thereby improving the feature representation efficiency of the network.

[0046] Quantitative and qualitative comparisons with various state-of-the-art methods on five publicly available benchmark datasets (Set5, Set14, B100, Urban100, and Manga109) demonstrate that the proposed method achieves superior super-resolution reconstruction results while maintaining a lightweight approach in terms of both parameter and computational complexity. This is particularly evident in textured urban scenes and anime images, proving the effectiveness and superiority of the proposed method. Attached Figure Description

[0047] Figure 1 This is an overall structural diagram of a preferred embodiment of the present invention;

[0048] Figure 2 The diagram shows the attention mechanism structure of the algorithm in a preferred embodiment of the present invention, wherein (a) is a schematic diagram of the Multi-Scale Striped Window Transformer (MST) module structure, (b) is a schematic diagram of the Striped Window Self-Attention (SSA) unit structure, and (c) is a schematic diagram of the Multi-Scale Striped Window Self-Attention (MSSA) module structure.

[0049] Figure 3 Visualization results of various image super-resolution reconstruction methods according to preferred embodiments of the present invention. Detailed Implementation

[0050] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0051] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0052] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0053] The overall structure of the method proposed in this invention is as follows: Figure 1-3 As shown, a lightweight image super-resolution framework, MSTSR, is proposed. This framework consists of three stages: given an LR image... First, a convolutional layer extracts shallow features. Then, several residual distillation Transformer blocks (RDTB) extract deep features from the shallow features. Finally, the deep features are fused with the shallow features through skip connections, and the final reconstruction result is obtained through a sub-pixel (PixelShuffle) layer. .

[0054] Specifically, the first input is a low-resolution image. After a 3 Three convolutional layers (Conv3) are used to obtain shallow features. :

[0055]

[0056] Subsequently It is fed into a deep feature extraction module composed of multiple RDTBs connected in series, and then... After processing by RDTB, multi-stage output features are obtained. ;No. The output of each RDTB It can be represented as:

[0057]

[0058] After concatenating the output features of each stage along the channel dimension, they are sequentially processed through 1... 1 convolutional layer (Conv1) and 3 Three convolutional layers (Conv3) are used for feature fusion and channel calibration to obtain deep features. :

[0059]

[0060] Finally Compared with initial shallow features The residuals are added together and then fed into the upsampling module (PixelShuffle) to obtain the final super-resolution reconstructed image. :

[0061]

[0062] The optimization objective of this model is to optimize the reconstructed super-resolution image. As close as possible to the true high-resolution image Based on existing research, this invention employs an image-by-image approach. The loss function completes model training. Specifically, given... Zhang low-resolution image and its one-to-one corresponding high-resolution ground truth image The model parameters are optimized using the following loss function:

[0063]

[0064] in Represents the total number of image sample pairs. To make the first Zhang low-resolution image The super-resolution reconstructed image output after inputting this model.

[0065] Residual distillation Transformer block (RDTB)

[0066] The residual distillation Transformer block (RDTB) proposed in this invention employs a residual characteristic distillation strategy. For example... Figure 1 As shown, RDTB learns residual information based on input features and progressively optimizes the residual features through a triple distillation step. In each step, the features are processed in two parallel branches: a feature extraction branch and a feature compression branch. The feature extraction branch uses two stacked multi-scale strip window Transformer (MST) modules to capture long-range dependencies in the image; the feature compression branch uses 1... 1. Convolution is used to reduce the dimensionality of the channels, thus ensuring the overall lightweight nature of the network.

[0067] For input features Distillation characteristics ( ) and intermediate features ( The method to obtain ) is as follows:

[0068]

[0069]

[0070]

[0071]

[0072] in This represents the computational operation of the Multi-Scale Striped Window Transformer (MST); after concatenating and fusing all distilled features, the output features are obtained. :

[0073]

[0074] Multi-scale strip window Transformer (MST)

[0075] A typical Transformer module consists of two parts: a self-attention module and a feedforward network. For example... Figure 2 As shown in (a), this invention proposes a novel linear self-attention mechanism in the MST unit—Multi-Scale Strip Window Self-Attention (MSSA). This mechanism uses strip windows of various sizes to efficiently extract long-distance dependencies. Furthermore, drawing on the existing Restormer technology, this invention introduces a gated deep convolutional feedforward network (GDFN), which extracts effective features while suppressing ineffective redundant features. Batch normalization (BN) is configured at the front end of the MSSA module, and layer normalization (LN) is configured at the front end of the GDFN module. Residual connections are added to both modules to optimize the feature transfer process and improve model training stability. The overall MST operation process is shown in equations (11) and (12):

[0076]

[0077]

[0078] in, , These are the input and output features of the MST module, respectively. These are intermediate features processed by the MSSA module; , These represent the operation functions of the MSSA and GDFN modules, respectively. , These correspond to batch normalization and layer normalization operations in sequence.

[0079] Multiscale strip window self-attention (MSSA)

[0080] This invention proposes a multi-scale strip window self-attention mechanism, which fully captures the rich long-distance dependencies of images while ensuring the model's lightweight nature. The MSSA module consists of three core components: multi-scale feature extraction, strip window partitioning, and self-attention computation, each of which will be described in detail in the following sections.

[0081] (1) Multi-scale feature extraction

[0082] like Figure 2 As shown in (c), the input features of MSSA Divided into three groups of features: The three feature sets have an equal number of channels. Then, each of the three feature sets is fed into a strip window self-attention (SSA) unit corresponding to a different window size (large, medium, small) to achieve multi-scale self-attention feature modeling. Finally, the outputs of the three SSA units are concatenated along the channel dimension and processed through a 1 1. Convolution completes feature fusion, yielding the final output features of the module. The calculation formula for this process is as follows:

[0083]

[0084]

[0085]

[0086] In the formula, This represents the self-attention operation function for strip windows.

[0087] (2) Strip window division

[0088] Striped Window Self-Attention (SSA) uses a striped window instead of a traditional square window to perform feature segmentation. First, the input features are... (in It is equally divided into two parts along the channel dimension: horizontal branch features With vertical branch features ;like Figure 2 As shown in (b), the features of the first half Perform horizontal strip window segmentation operation For the features of the second half Perform vertical strip window segmentation operation Self-attention calculation is performed on the two sets of segmented features respectively, and then the feature size is restored through window inverse segmentation operation to obtain... and Finally, the two sets of features are concatenated along the channel dimension to obtain the output features of the SSA module. The complete process is represented by the following formula:

[0089]

[0090]

[0091]

[0092] In the formula: Represents self-attention computational operations; , Horizontal strip window segmentation operation Vertical strip window segmentation operation The corresponding inverse operation.

[0093] (3) Self-attention calculation

[0094] The complete calculation process of self-attention is as follows: Figure 2 (b) As shown in the shaded area in the middle. First, from the input features... Get the query matrix (This matrix also serves as a bond matrix) ) and numerical matrix Traditional self-attention mechanisms generate... Three sets of matrices, this scheme will and Reuse is used to reduce computational overhead. The subsequent calculation process for self-attention is as follows (19):

[0095]

[0096] in, To prevent the channel dimension scaling factor from being too large, the number of channels is generally selected as the default value.

[0097] (4) Gated depthwise feedforward network (GDFN)

[0098] The GDFN operation flow is as follows: First, the input features are simultaneously fed into two parallel branches, and each branch passes through 1 One convolution is used to perform feature upscaling; then three convolutions are employed. 3. Deeply Separable Convolution (DWConv) extracts high-level features; GELU activation is performed on the features of one branch, and then multiplied pixel-by-pixel with the features of the other branch. This gating mechanism can amplify effective features and suppress irrelevant redundant features, thereby improving the network's ability to represent key image information; finally, through 1 1. Convolution restores the feature dimension to the initial input dimension. The complete operation process is represented by equations (20), (21), and (22):

[0099]

[0100]

[0101] (twenty two)

[0102] In the formula: , These are the input and output features of the GDFN module, respectively. This represents a convolution kernel size of 3. 3-depth convolution; Use the GELU activation function; This represents element-wise multiplication.

[0103] In existing lightweight image super-resolution methods, Transformer-based reconstruction schemes often employ fixed-size square windows for self-attention computation, ignoring the anisotropic structural features of natural images and making it difficult to simultaneously achieve long-distance dependency modeling and fine-grained texture feature capture. Furthermore, existing striped window self-attention mechanisms only use a single fixed-scale window partition, failing to simultaneously adapt to multi-scale texture patterns in images, thus limiting reconstruction performance in complex scenes containing rich details and large-scale structures. Simultaneously, existing lightweight Transformer architectures often employ simple module stacking designs, failing to incorporate feature distillation strategies to further reduce computational overhead, making it difficult to achieve effective improvements in reconstruction accuracy with limited parameters and computational resources.

[0104] To improve the reconstruction quality and computational efficiency of lightweight super-resolution image reconstruction, this invention proposes an innovative lightweight image super-resolution reconstruction method based on multi-scale strip window Transformers. The aim is to simultaneously model the anisotropic structure and multi-scale dependencies of the image through a multi-scale strip window self-attention mechanism, and to achieve a balance between reconstruction performance and model lightweighting by combining a residual feature distillation architecture. This invention first extracts shallow features from the input low-resolution image through convolutional layers. Then, it uses the proposed residual distillation Transformer block for deep feature extraction. This module employs a progressive residual feature distillation strategy, dividing the features into compression and extraction branches. The extraction branch captures long-distance dependencies through stacked multi-scale strip window Transformer modules, while the compression branch controls computational load through 1×1 convolutional dimensionality reduction, ensuring network lightweighting while progressively refining features. Specifically, this invention introduces a parallel multi-scale strip window self-attention mechanism, feeding the feature groups after channel segmentation into horizontal and vertical strip windows with different aspect ratios in parallel. The technical effect achieved lies in overcoming the symmetry limitations of traditional square windows on geometric boundaries by utilizing anisotropic strip windows, greatly expanding the long-range receptive field in both horizontal and vertical directions. Simultaneously, the parallel design of multi-scale windows enables the network to adaptively match fine-grained lines and large-scale structures of different sizes within a single layer, significantly reducing the computational cost of lightweight models while addressing the pain point of insufficient reconstruction of anisotropic heterogeneous textures by traditional fixed windows, effectively improving adaptability to complex scenes. Furthermore, this invention introduces a gated deep convolutional feedforward network, amplifying effective features and suppressing redundant information through a dual-branch gated structure, further reducing computational cost and improving the network's feature representation efficiency. Finally, the outputs of all residual distillation Transformer blocks are concatenated and fused, and after combining global residual connections with shallow feature fusion, upsampling is performed through sub-pixel convolutional layers to generate high-resolution reconstruction results, completing the entire process from feature extraction to reconstruction output. Quantitative and qualitative comparisons with various state-of-the-art methods on five publicly available benchmark datasets demonstrate that the proposed method achieves superior super-resolution reconstruction results while maintaining a lightweight approach in terms of both parameter and computational complexity. This is particularly evident in textured urban scenes and anime images, proving the effectiveness and superiority of the proposed method.

[0105] To ensure fairness and reproducibility of results compared with all baseline methods, this invention uses the DIV2K dataset as the training set and selects five publicly available benchmark datasets (Set5, Set14, B100, Urban100, and Manga109) as the test set. All comparison models are evaluated under completely consistent experimental protocols: a unified data preprocessing workflow is used, RGB color space to YCbCr color space conversion is performed uniformly, and evaluation metrics are calculated only on the luminance Y channel; fixed boundary clipping rules are followed, with 4, 6, and 8 boundary pixels clipped for 2x, 3x, and 4x super-resolution tasks, respectively; all models are run on identical hardware. All baseline methods are implemented using their publicly available official code to ensure reproducibility of experimental results.

[0106] This invention uses Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) as quantitative evaluation metrics for image reconstruction quality. During the training phase, the batch size is set to 64, and the input low-resolution image patch is cropped to 48×48 pixels. Data augmentation is performed using random horizontal flipping and 90° rotation. The Adam optimizer is used to optimize the network parameters, with an initial learning rate set to 5. 10⁻ 4 The model was trained for a total of 500,000 iterations, with the learning rate halved at the 250,000, 400,000, 450,000, and 475,000 iterations, respectively.

[0107] Regarding hyperparameter configuration, the number K of Residual Distillation Transformer Blocks (RDTB) was set to 4, and the feature channel dimension d was set to 48; the window sizes in the Multi-Scale Striped Window Self-Attention (MSSA) were configured as 32×8, 8×32, 16×4, 4×16, 8×2, and 2×8, respectively. The MSTSR+ variant of this invention employs a self-integrated test-time enhancement (TTA) strategy, averaging the reconstruction results of the flipped and rotated input images. All experiments were conducted on an NVIDIA GeForce RTX 3090 GPU hardware environment.

[0108] The core network topology of MTSR+ in this invention is completely consistent with that of MTSR. It only enables the self-ensemble test enhancement strategy during the test inference phase, and outputs the average value by rotating and flipping the input image from multiple angles.

[0109] Qualitative comparison

[0110] To qualitatively compare the reconstruction effects of the methods, the reconstruction results were visualized. Figure 3The reconstruction results on the Urabn109 and Manga109 datasets are shown. Compared to other models, MSTSR is able to reconstruct texture details with higher clarity.

[0111] Table 1. Quantitative comparison of the present invention with state-of-the-art algorithms on public datasets.

[0112]

[0113]

[0114] Table 1 presents the quantitative test results of the proposed method on five benchmark datasets. The table also lists the performance data of the base model MTSR and its enhanced variant MTSR+ (using a self-ensemble testing enhancement strategy). The proposed MTSR method achieves competitive reconstruction performance on all five benchmark datasets; particularly on the Urban100 and Manga109 datasets, MTSR shows significant advantages in Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) metrics compared to other comparative methods. For example, on the Manga109 dataset, MTSR's reconstruction effect is significantly better than similar lightweight super-resolution methods. The MTSR+ variant requires no additional training overhead and can achieve stable further improvements in reconstruction performance under all test configurations. Table 1 also provides a comparison of the model parameter count and computational cost of each method: in terms of both model size and computational burden, the proposed MTSR is more lightweight than existing state-of-the-art methods such as SwinIR, SRFormer, and ELAN. The above comparison results fully verify the reconstruction effectiveness and computational efficiency of the proposed MTSR model.

Claims

1. An image super-resolution reconstruction method based on a multi-scale strip window Transformer, characterized in that, Includes the following steps: S1: Input low-resolution image I LR Extract shallow features F0 using convolutional layers; S2: The shallow feature F0 is fed into a deep feature extraction module consisting of K residual distillation Transformer blocks connected in series to obtain multi-stage output features F1, F2, ..., F K Each residual distillation Transformer block employs a progressive residual feature distillation strategy, dividing the input features into a feature extraction branch and a feature compression branch for parallel processing. The feature extraction branch uses stacked multi-scale strip window Transformer modules to capture long-range dependencies, while the feature compression branch achieves channel dimensionality reduction through 1×1 convolution. S3: Concatenate the output features from each stage along the channel dimension, then perform feature fusion and channel calibration by sequentially passing them through a 1×1 convolutional layer and a 3×3 convolutional layer. Finally, add the deep feature F to the shallow feature F0 through a residual connection to obtain the deep feature F. deep ; S4: Transfer the deep feature F deep The image is fed into a subpixel convolutional layer for upsampling to generate a super-resolution reconstructed image I. SR .

2. The image super-resolution reconstruction method based on multi-scale strip window Transformer according to claim 1, characterized in that, The multi-scale strip window Transformer module mentioned in step S2 includes a multi-scale strip window self-attention mechanism and a gated deep convolutional feedforward network; The multi-scale strip window self-attention mechanism is used to divide the input features into multiple groups of features along the channel dimension, and feed each group of features into strip window self-attention units with different window sizes to realize multi-scale self-attention feature modeling. The outputs of each group are concatenated along the channel dimension and then fused by 1×1 convolution. The gated depthwise convolutional feedforward network is used to process features through a dual-branch gated structure. One branch is multiplied element-wise with the features of the other branch after being subjected to depthwise separable convolution and GELU activation, so as to amplify effective features and suppress redundant features.

3. The image super-resolution reconstruction method based on multi-scale strip window Transformer according to claim 2, characterized in that, The specific implementation of the multi-scale strip window self-attention mechanism is as follows: Input features F in Divide the features into three groups equally along the channel dimension: The three sets of features, corresponding to large, medium, and small window sizes respectively, are fed into the corresponding strip window self-attention units for processing. The outputs of the three strip window self-attention units are concatenated along the channel dimension and then fused by a 1×1 convolution to obtain the output feature F. out .

4. The image super-resolution reconstruction method based on multi-scale strip window Transformer according to claim 3, characterized in that, The specific implementation of the strip window self-attention unit is as follows: Input features Equally split into horizontal branch features along the channel dimension With vertical branch features Where, e∈{s,m,l}; For the horizontal branch feature After performing the horizontal stripe window segmentation operation, self-attention calculation is performed, and then the feature size is restored through the horizontal stripe window inverse segmentation operation to obtain... ; For the vertical branch feature After performing the vertical stripe window segmentation operation, self-attention calculation is performed, and then the feature size is restored through the inverse vertical stripe window segmentation operation to obtain... ; Will and By concatenating along the channel dimension, the output features of the striped window self-attention unit are obtained. .

5. The image super-resolution reconstruction method based on multi-scale strip window Transformer according to claim 4, characterized in that, In the self-attention calculation, the query matrix Q is obtained from the input feature X. The query matrix Q also serves as the key matrix K, resulting in a numerical matrix V. The self-attention calculation formula is as follows: Where d is the channel dimension scaling factor.

6. The image super-resolution reconstruction method based on multi-scale strip window Transformer according to claim 2, characterized in that, The specific implementation of the gated depthwise convolutional feedforward network is as follows: Input features Simultaneously, two parallel branches are fed in, and each branch performs feature upscaling through 1×1 convolution, and then extracts high-level features through 3×3 depthwise separable convolution. Perform GELU activation on the features of one branch, and then multiply them element-wise with the features of the other branch; Finally, a 1×1 convolution is used to restore the feature dimensions to the initial input dimensions, resulting in the output feature F. out .

7. The image super-resolution reconstruction method based on multi-scale strip window Transformer according to claim 1, characterized in that, In step S2, the residual distillation Transformer block progressively optimizes the residual features through three distillation steps. In each step, the features are divided into two parallel branches: the feature extraction branch is processed by two stacked multi-scale strip window Transformer modules, and the feature compression branch achieves channel dimensionality reduction through 1×1 convolution. The distilled features obtained from each distillation step are concatenated in the channel dimension and then fused by 1×1 convolution and 3×3 convolution in sequence. The fused features are then added to the input features of the residual distillation Transformer block through residual concatenation to obtain the output features.

8. An image super-resolution reconstruction system based on a multi-scale strip window Transformer, characterized in that, The image super-resolution reconstruction method based on a multi-scale strip window Transformer, as described in any one of claims 1-7, includes: The shallow feature extraction module is used to extract shallow features from the input low-resolution image through a convolutional layer; The deep feature extraction module consists of K residual distillation Transformer blocks connected in series. It is used to extract deep features from the shallow features and output multi-stage features. The residual distillation Transformer block adopts a progressive residual feature distillation strategy, including a feature extraction branch and a feature compression branch. The feature extraction branch adopts stacked multi-scale strip window Transformer modules, and the feature compression branch adopts 1×1 convolution. The feature fusion module is used to concatenate the output features of each stage in the channel dimension, perform feature fusion and channel calibration through a 1×1 convolutional layer and a 3×3 convolutional layer in sequence, and add them to the shallow features through residual connection; The upsampling module uses subpixel convolutional layers to upsample the fused deep features and generate a super-resolution reconstructed image.

9. The image super-resolution reconstruction system based on a multi-scale strip window Transformer according to claim 8, characterized in that, The multi-scale strip window Transformer module includes: The multi-scale strip window self-attention unit is used to divide the input features into multiple groups of features along the channel dimension and feed them into strip window self-attention modules with different window sizes. The outputs of each group are concatenated along the channel dimension and then fused by 1×1 convolution. The gated depthwise convolutional feedforward network unit is used to process features through a two-branch gated structure, where one branch is multiplied element-wise with the features of the other branch after being subjected to depthwise separable convolution and GELU activation. The multi-scale strip window self-attention unit is configured with a batch normalization layer at its front end, and the gated depthwise convolutional feedforward network unit is configured with a layer normalization layer at its front end. Both units are configured with residual connections.

10. A computer-readable storage medium, characterized in that, include: The computer-readable storage medium contains a computer program; The computer program is executed by the processor to perform an image super-resolution reconstruction method based on a multi-scale strip window Transformer as described in any one of claims 1 to 7.

Images