A method for super-resolution reconstruction of remote sensing images based on adaptive gated Transformer

Abstract

This invention belongs to the field of remote sensing super-resolution image technology, and particularly relates to a method for super-resolution reconstruction of remote sensing images based on adaptive gated Transformer. The method includes: S1: constructing a training set; S2: constructing an adaptive gated Transformer super-resolution reconstruction network, and training the adaptive gated Transformer super-resolution reconstruction network using the training set to obtain an adaptive gated Transformer super-resolution reconstruction model; S3: inputting the low-resolution remote sensing image to be reconstructed into the adaptive gated Transformer super-resolution reconstruction model to obtain the super-resolution reconstructed image. This invention aims to overcome the limitations of existing methods in terms of efficiency, detail recovery, and multi-scale fusion, providing a super-resolution solution for remote sensing images that balances high fidelity and high efficiency, meeting the urgent need for high-quality imagery in refined remote sensing applications.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing super-resolution image technology, and particularly relates to a method for super-resolution reconstruction of remote sensing images based on adaptive gated Transformer. Background Technology

[0002] Remote sensing image super-resolution technology aims to recover high-resolution images from low-resolution images, thereby improving the spatial detail and texture representation of images. It is an important foundation for applications such as fine-grained identification of ground features, environmental monitoring, and urban planning. However, due to the physical limitations of satellite and aerial imaging equipment, as well as the unavoidable atmospheric disturbances and noise interference during the imaging process, remote sensing images often suffer from insufficient resolution, resulting in blurred ground feature boundaries and loss of detailed textures, thus affecting subsequent high-precision analysis. Traditional interpolation methods, such as bicubic interpolation, are simple but lack the ability to recover true details. Reconstruction methods based on sparse representations or non-local self-similarity can improve texture representation to some extent, but they rely on complex prior modeling, resulting in high computational costs and limited adaptability. With the development of deep learning, convolutional neural networks (CNNs) have been widely used in super-resolution tasks, such as SRCNN, EDSR, and RCAN. These methods can learn the relationship between LR and HR images through end-to-end mapping, achieving significant breakthroughs in remote sensing image reconstruction. However, since convolution inherently relies on local receptive fields, it is difficult to effectively capture global structural relationships when facing large-area scenes, often resulting in problems such as overly smoothed textures and loss of local details.

[0003] To overcome the limitations of CNNs, the Transformer architecture has been increasingly introduced into super-resolution reconstruction tasks. The Transformer's self-attention mechanism can model long-range dependencies over a larger scope and integrate global contextual information, demonstrating excellent reconstruction results in remote sensing image super-resolution. Typical methods such as SwinIR and HAT-L have achieved excellent performance on multiple public datasets, but still face two main challenges: First, remote sensing images contain a large number of low-information-density background regions, such as the sky, water bodies, and deserts. Traditional dense self-attention is computed uniformly across all regions, leading to wasted computational resources and increased noise interference. Second, ground features in remote sensing images exhibit significant multi-scale characteristics; buildings, roads, farmland, etc., differ significantly in spatial size. However, existing Transformers often employ single-scale modeling strategies, making it difficult for the model to simultaneously maintain global consistency and local texture details in complex scenes.

[0004] To address the aforementioned issues, some studies have attempted to introduce feature selection or multi-scale fusion mechanisms to improve the quality of super-resolution reconstruction. For example, the Top-K Token Selective Transformer reduces redundant computation to some extent, but it remains insufficiently accurate in capturing key texture regions and has limited ability to recover high-frequency details. While multi-branch convolution or feature fusion structures enhance the model's ability to perceive multi-scale features, their collaborative optimization between global and local information is insufficient, and problems such as blurred details and missing textures still exist. This invention aims to overcome the limitations of existing methods in terms of efficiency, detail recovery, and multi-scale fusion, providing a super-resolution solution for remote sensing images that balances high fidelity and high efficiency, meeting the urgent need for high-quality imagery in refined remote sensing applications. Summary of the Invention

[0005] In view of this, the present invention aims to provide a remote sensing image super-resolution reconstruction method based on adaptive gated Transformer to solve the problems of insufficient collaborative optimization between global and local information in the existing technology, resulting in blurred details and missing textures.

[0006] To achieve the above objectives, the technical solution created by this invention is implemented as follows:

[0007] A method for super-resolution reconstruction of remote sensing images based on adaptive gated Transformer, specifically including the following steps:

[0008] S1: Construct the training set;

[0009] S2: Construct an adaptive gated Transformer super-resolution reconstruction network, and train the adaptive gated Transformer super-resolution reconstruction network using the training set to obtain the adaptive gated Transformer super-resolution reconstruction model.

[0010] The adaptive gated Transformer super-resolution reconstruction network includes a shallow feature extraction module, an n-level deep feature modeling and multi-scale fusion module, and a high-resolution reconstruction module.

[0011] The low-resolution image input to the adaptive gated Transformer super-resolution reconstruction network is processed by the shallow feature extraction module to obtain shallow features. Shallow features After processing by n cascaded deep feature modeling and multi-scale fusion modules, feature M is obtained; after processing by the high-resolution reconstruction module, feature M is used to obtain a super-resolution reconstructed image corresponding to the input image.

[0012] S3: Input the low-resolution remote sensing image to be reconstructed into the adaptive gated Transformer super-resolution reconstruction model to obtain the super-resolution reconstructed image.

[0013] Furthermore, in step S2, the shallow feature extraction module includes a 1×1 convolutional layer. After processing the low-resolution image through the 1×1 convolutional layer, shallow features are obtained. .

[0014] Furthermore, the deep feature modeling and multi-scale fusion module includes an adaptive gated Top-K attention mechanism, a frequency domain-aware feature learning module, and an adaptive global context module.

[0015] Based on the adaptive gating Top-K attention mechanism for shallow features The data is processed, and the results are input into the adaptive global context module for further processing to obtain features. ; Shallow features The input is processed by the frequency domain-aware feature learning module to obtain features. ; Shallow features ,feature and characteristics After adding them together, we obtain feature M1.

[0016] Furthermore, in step S2, shallow features are processed based on an adaptive gated Top-K attention mechanism. The specific process for processing is as follows:

[0017] After copying the input feature X into four copies, four independent attention branches are formed to focus on the input feature X. k Perform linear projection to obtain the input feature X k The corresponding query matrix Q, key matrix K, and value matrix V, k=1, 2, 3, or 4;

[0018] Based on input features X k The following formula is used to express the relationship between the input feature X and the input feature X. k The corresponding generation gating weight G for each token k :

[0019] ;

[0020] in, and All are learning parameters. For the Sigmoid function;

[0021] Gating weight G k Perform the same operation as with the value matrix V, and calculate the weighted value matrix using the following formula. :

[0022] ;

[0023] in, For element-wise multiplication;

[0024] For input features X k The same operation is performed on the corresponding query matrix Q, key matrix K, and value matrix V, and weights are assigned to each attention branch using the following formula. :

[0025] ;

[0026] in, Let the attention weights be those of the k-th attention branch. For the Top-K mask function corresponding to the k-th attention branch, Let d be the attention score matrix, and d be the feature dimension. It is the softmax activation function;

[0027] Based on the weight allocation results and gating weighting factors of each attention branch, the attention branches are weighted and fused using the following formula to obtain the features. :

[0028] ;

[0029] in, For weighted summation, Concat is for feature concatenation, and Conv is for convolution. is the gating weighting factor for the k-th attention branch.

[0030] Furthermore, the deep feature modeling and multi-scale fusion module also includes window-shifted attention and SwiGLU gating units, for features After processing by window-shift attention and SwiGLU gating units, the results are input into the adaptive global context module for further processing to obtain features. .

[0031] Furthermore, the input to the adaptive global context module features... The feature vector is generated by the following formula. :

[0032] ;

[0033] in, For the nth depthwise convolution operation, GAP is the global average pooling operation;

[0034] The eigenvector is expressed by the following formula. Process to obtain features :

[0035] ;

[0036] in, For Sigmoid activation function, GELU is the GELU activation function. For element-wise multiplication, The first linear transform layer arranged in the order of processing. This is the second linear transformation layer arranged in the processing order.

[0037] Furthermore, the processing procedure of the frequency domain-aware feature learning module is as follows:

[0038] shallow features The data is converted to a frequency domain feature map using Fourier transform, and the amplitude feature map is extracted from the frequency domain feature map. and phase feature map :

[0039] ;

[0040] ;

[0041] in, Let be the real part of the frequency domain feature map. This represents the imaginary part of the frequency domain feature map;

[0042] The amplitude characteristic map A and the phase characteristic map are analyzed using the following formula. Perform nonlinear mapping to obtain the enhanced amplitude feature map. and enhanced phase feature map :

[0043] ;

[0044] ;

[0045] Where Conv is depthwise separable convolution and LeakyReLU is a ReLU activation function with a leakage coefficient;

[0046] The enhanced amplitude feature map is analyzed using inverse Fourier transform. and enhanced phase feature map Process to obtain features :

[0047] ;

[0048] in, It is a complex exponential form composed of enhanced phase feature maps. This is the inverse Fourier transform.

[0049] Furthermore, in step S2, the number n of deep feature modeling and multi-scale fusion modules is greater than or equal to 1.

[0050] Furthermore, in step S1, the specific process of constructing the training set is as follows: 100 images are randomly selected from each of the 30 scene categories in the AID dataset, resulting in 3000 images as the training set.

[0051] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0052] (1) The adaptive gated Transformer-based remote sensing image super-resolution reconstruction method described in this invention accurately solves the core problems existing in remote sensing image super-resolution reconstruction. It has been tested on the AID dataset with several advanced models. The test results show that existing Transformer-based models, when processing high-resolution images, suffer from significant computational redundancy and inefficiency due to treating all tokens equally. The adaptive gated Top-K attention mechanism used in this invention can intelligently filter and focus on the most informative tokens, significantly reducing computational complexity while concentrating computational resources in key areas, greatly improving processing efficiency. On the AID dataset, AGTSR achieves a PSNR 0.07 dB higher than the current state-of-the-art HAT-L, while using only 86.64% of its computational resources.

[0053] (2) The remote sensing image super-resolution reconstruction method based on adaptive gated Transformer described in this invention addresses the complex and diverse scale of ground features in remote sensing images. The adaptive global context module proposed in this invention employs multi-branch deep convolution kernels and a squeezing excitation mechanism, enabling the model to flexibly capture and fuse feature information at different scales. This overcomes the limitation of existing methods in insufficient global context information modeling when processing multi-scale targets. Based on the adaptive global context module, this invention can better understand and reconstruct ground features at various scales, from small vehicles to large areas of farmland, thus improving reconstruction accuracy.

[0054] (3) The remote sensing image super-resolution reconstruction method based on adaptive gated Transformer described in this invention addresses the problem of loss of high-frequency details (such as texture and edges) during the degradation process. This invention adopts a frequency domain-aware feature learning module to directly learn and enhance amplitude and phase information independently in the frequency domain. This is a mechanism that complements the spatial domain method. This unique spatial-frequency collaborative learning strategy enables the model to more accurately recover those tiny details that are crucial to visual quality and ground object recognition, making up for the shortcomings of the spatial domain method in detail recovery. Attached Figure Description

[0055] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0056] Figure 1 A schematic flowchart of the remote sensing image super-resolution reconstruction method based on adaptive gated Transformer as described in the embodiments of the present invention;

[0057] Figure 2 A schematic diagram of the network structure of the adaptive gated Transformer super-resolution reconstruction model described in the embodiments of the present invention;

[0058] Figure 3 A schematic diagram of the network structure of the adaptive global context module described in the embodiments of the present invention;

[0059] Figure 4 A schematic diagram of the network structure of the frequency domain sensing feature learning module described in the embodiment of the present invention;

[0060] Figure 5 The comparison chart of the present invention with other mainstream models on the AID dataset is shown in the embodiments of the present invention.

[0061] Figure 6 A comparison image of 4x super-resolution reconstruction of different models on the AID dataset, as described in the embodiments of the present invention;

[0062] Figure 7 This invention provides an embodiment of the quality assessment map for 4x super-resolution reconstruction of different models on the AID dataset. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0064] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0065] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0066] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0067] While existing deep learning-based super-resolution methods for remote sensing images have made significant progress in performance, several key issues remain unresolved: First, the Transformer architecture generally suffers from redundant computational overhead when processing remote sensing images, treating large areas of low-information background regions equally, leading to decreased computational efficiency and the introduction of noise. Second, existing methods lack the ability to model multi-scale features in remote sensing images, and there is a lack of effective fusion between features at different scales, making it difficult to balance global structural consistency with local fine textures in the reconstruction results. Third, their ability to recover high-frequency information such as edges and textures is limited, easily resulting in missing or blurred details, which fails to meet the demand for refined reconstruction in remote sensing applications.

[0068] To address the aforementioned shortcomings, this invention aims to propose a multi-scale feature fusion super-resolution method for remote sensing images based on adaptive gated Transformer, achieving efficient and refined image reconstruction. This invention introduces adaptive gating and Top-K selection attention mechanisms to dynamically filter and enhance high-information-density regions, effectively suppressing the interference of redundant background on modeling, thereby improving computational efficiency and information capture capabilities. Furthermore, by designing an adaptive global context module, this invention achieves deep feature fusion across different receptive fields, enabling the model to simultaneously possess global structure modeling and local detail restoration capabilities. A SwiGLU-based multi-scale feature transformation unit is introduced into the feedforward network to enhance nonlinear expression capabilities and improve the flexibility of modeling complex textures. Moreover, this invention utilizes a frequency-domain perceptual feature learning module to adaptively enhance high-frequency features at both amplitude and phase levels, fundamentally improving the restoration of details and edges. Through these technical means, this invention aims to overcome the limitations of existing methods in terms of efficiency, detail restoration, and multi-scale fusion, providing a super-resolution solution for remote sensing images that balances high fidelity and high efficiency, meeting the urgent need for high-quality imagery in refined remote sensing applications.

[0069] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0070] like Figure 1 As shown, a method for super-resolution reconstruction of remote sensing images based on adaptive gated Transformer specifically includes the following steps:

[0071] S1: Construct a training set; S2: Construct an adaptive gated Transformer super-resolution reconstruction network and train it using the training set to obtain the adaptive gated Transformer super-resolution reconstruction model; the adaptive gated Transformer super-resolution reconstruction network includes a shallow feature extraction module, n deep feature modeling and multi-scale fusion modules, and a high-resolution reconstruction module; the low-resolution image input to the adaptive gated Transformer super-resolution reconstruction network is processed by the shallow feature extraction module to obtain shallow features. Shallow features After processing by n cascaded deep feature modeling and multi-scale fusion modules, feature M is obtained; after feature M is processed by the high-resolution reconstruction module (convolution and pixel rearrangement), the super-resolution reconstructed image corresponding to the input image is obtained; S3: The remote sensing low-resolution image to be reconstructed is input into the adaptive gated Transformer super-resolution reconstruction model to obtain the super-resolution reconstructed image.

[0072] It should be noted that the proposed multi-scale feature fusion super-resolution method for remote sensing images based on adaptive gating Transformer organically combines the local feature extraction capability of convolution with the global dependency modeling advantage of Transformer. Through an adaptive gating mechanism, it dynamically filters high-information-density regions, significantly reducing redundant computation and enhancing focus on key texture regions. Simultaneously, the adaptive global context module designed in this invention can extract and fuse multi-scale features under different receptive fields, enabling the model to balance global consistency with refined local detail representation in complex remote sensing scenarios. Furthermore, a SwiGLU-based multi-scale feature transformation unit is introduced into the network's feedforward structure to enhance the nonlinearity of feature transformation, improve the diversity and flexibility of information representation, and build upon this, a frequency-domain perceptual feature learning module is constructed. Through adaptive adjustment of frequency domain amplitude and phase information, it strengthens the recovery capability of high-frequency components such as edges and textures. These improvements enable this invention to not only outperform existing state-of-the-art models in quantitative metrics but also more realistically reproduce the fine textures in remote sensing images in subjective visual effects, demonstrating greater practical value and potential for wider application.

[0073] In some embodiments, in step S2, the shallow feature extraction module includes a 1×1 convolutional layer. After processing the low-resolution image through the 1×1 convolutional layer, shallow features are obtained. .

[0074] It should be noted that the shallow feature extraction module also includes a normalization layer. After the low-resolution image is processed by the 1×1 convolutional layer, the data is normalized by the normalization layer to stabilize the training process, accelerate convergence, and improve the model's generalization ability.

[0075] Furthermore, shallow feature extraction is performed on the input low-resolution remote sensing image to be reconstructed. Specifically, the low-resolution image is first input into a convolutional neural network, and several convolutional kernels are used to perform sliding operations on the local receptive field of the input image to obtain an initial feature representation containing basic information such as edges and textures. This process not only effectively preserves the low-level structural information of the input image, but also suppresses noise and redundant background information, thus laying the foundation for subsequent deep feature modeling and multi-scale fusion. The calculation formula is as follows:

[0076] ;

[0077] in, This represents the input low-resolution remote sensing image. This represents a shallow feature extraction function consisting of convolutional layers. This represents the shallow feature map output. Through this step, the model can obtain relatively stable and representative initial features while maintaining computational efficiency, enabling subsequent adaptive gating attention mechanisms and multi-scale context modeling to more accurately focus on key regions and perform detail restoration.

[0078] In some embodiments, the deep feature modeling and multi-scale fusion module includes an adaptive gated Top-K attention mechanism, a frequency domain-aware feature learning module, and an adaptive global context module, wherein...

[0079] Based on the adaptive gating Top-K attention mechanism for shallow features The data is processed, and the results are input into the adaptive global context module for further processing to obtain features. ; Shallow features The input is processed by the frequency domain-aware feature learning module to obtain features. ; Shallow features ,feature and characteristics After adding them together, we obtain feature M1.

[0080] Furthermore, the deep feature modeling and multi-scale fusion module also includes window-shifted attention and SwiGLU gating units, for features After processing by window-shift attention and SwiGLU gating units, the results are input into the adaptive global context module for further processing to obtain features. .

[0081] It should be noted that in the deep modeling and multi-scale fusion stages, this invention introduces an adaptive gated Top-K attention mechanism to dynamically filter and enhance high-information regions, effectively reducing redundant computation. Subsequently, an adaptive global context module is used to fuse features at different scales, taking into account both global structure and local texture. The feedforward network combines multi-scale transformation with SwiGLU gated units to improve nonlinear representation and detail recovery capabilities. Simultaneously, a frequency-domain perceptual feature learning module is introduced to enhance high-frequency texture and edge reconstruction using amplitude and phase information.

[0082] In some embodiments, in step S2, shallow features are processed based on an adaptive gated Top-K attention mechanism. The specific process for processing is as follows:

[0083] After copying the input feature X into four copies, four independent attention branches are formed to focus on the input feature X. k Perform linear projection to obtain the input feature X k The corresponding query matrix Q, key matrix K, and value matrix V, k=1, 2, 3, or 4;

[0084] Based on input features X k The following formula is used to express the relationship between the input feature X and the input feature X. k The corresponding generation gating weight G for each token k :

[0085] ;

[0086] in, and All are learning parameters. For the Sigmoid function;

[0087] Gating weight G k Perform the same operation as with the value matrix V, and calculate the weighted value matrix using the following formula. :

[0088] ;

[0089] in, For element-wise multiplication;

[0090] For input features X k The same operation is performed on the corresponding query matrix Q, key matrix K, and value matrix V, and weights are assigned to each attention branch using the following formula. :

[0091] ;

[0092] in, Let the attention weights be those of the k-th attention branch. For the Top-K mask function corresponding to the k-th attention branch, Let d be the attention score matrix, and d be the feature dimension. It is the softmax activation function;

[0093] Based on the weight allocation results and gating weighting factors of each attention branch, the attention branches are weighted and fused using the following formula to obtain the features. :

[0094] ;

[0095] in, For weighted summation, Concat is for feature concatenation, and Conv is for 3×3 convolution. is the gating weighting factor for the k-th attention branch.

[0096] It should be noted that, as Figure 2As shown, in the deep feature modeling stage, this invention introduces an adaptive gated Top-K attention mechanism. This mechanism can dynamically filter and enhance regions with high information content, effectively reducing redundant computation caused by complex backgrounds in remote sensing images by selectively focusing on the most critical feature tokens. Subsequently, an adaptive global context module is used to achieve effective fusion of multi-scale features. This module can take into account both global structure and local texture, ensuring that the model can recover rich details and accurate spatial structure when processing ground features at different scales.

[0097] Furthermore, the main purpose of the adaptive gated Top-K attention mechanism is to address the computational redundancy and information dispersion problems faced by traditional Transformer-based super-resolution models when processing high-resolution remote sensing images. The core design of this mechanism combines an adaptive Top-K scaling predictor with a multi-branch attention fusion strategy, enabling it to dynamically select the most critical tokens for computation based on feature content and efficiently fuse multi-branch information through a gating mechanism.

[0098] The workflow of the adaptive gated Top-K attention mechanism mainly includes four steps: feature projection, adaptive gating, multi-branch sparse attention computation, and gated weighted fusion output. The latter two steps together constitute a process of parallel processing and fusion of four attention branches:

[0099] First, the input feature X undergoes linear projection to obtain the query matrix Q, key matrix K, and value matrix V required by the attention mechanism:

[0100] , , ;

[0101] in, , , These are the learnable weight matrices.

[0102] Subsequently, the adaptive gating mechanism generates a gating weight for each token through a lightweight network consisting of fully connected layers and a sigmoid activation function. To assess its importance and filter redundant information:

[0103] ;

[0104] in, and All are learning parameters. The weight is a Sigmoid function. It was then applied to value matrices. Above, generate the weighted value matrix. This is used to enhance the influence of key tokens and suppress redundant tokens. In the multi-branch sparse attention computation stage, the adaptive gated Top-K attention mechanism designs four independent attention branches (k=1, 2, 3, 4). In each attention branch, the attention score matrix... Sparsity processing is performed using a Top-K masking function, which dynamically retains only the k most relevant attention values ​​between each query matrix and key matrix based on the output of an adaptive Top-K proportional predictor. This ensures that each attention branch efficiently focuses on key dependencies of varying degrees, resulting in four different scales of sparse attention results. Finally, in the gated weighted fusion and output stage, the sparse attention results from the four attention branches are processed by dynamically learned gating weights. Weighted fusion and concatenation are performed. Finally, an output feature is generated through a convolutional layer. This allows for the capture of information under different Top-K ratios, thereby improving information expression capabilities.

[0105] ;

[0106] ;

[0107] In some embodiments, the input to the adaptive global context module features The feature vector is generated by the following formula. :

[0108] ;

[0109] in, For the nth depthwise convolution operation, GAP is the global average pooling operation;

[0110] The eigenvector is expressed by the following formula. Process to obtain features :

[0111] ;

[0112] in, For Sigmoid activation function, GELU is the GELU activation function. For element-wise multiplication, The first linear transform layer arranged in the order of processing. This is the second linear transformation layer arranged in the processing order.

[0113] It should be noted that the adaptive global context module proposed in this invention aims to address the problem of insufficient global context information modeling capability in existing methods when dealing with multi-scale targets and complex remote sensing scenes. This invention effectively integrates feature information from different levels through an innovative deep convolutional kernel and squeezing excitation mechanism, thereby enhancing the model's ability to identify and reconstruct multi-scale targets.

[0114] The core idea of ​​the adaptive global context module is that it can dynamically aggregate global and local features to adapt to changes in object scale across different scenarios. For example... Figure 3 As shown, this module first branches the input feature map. One branch remains unchanged and serves as the input for the residual connection, while the other branch uses a channel attention mechanism to acquire global contextual information. This channel attention branch first extracts deep features from the original feature map using three deep convolutional kernel branches (3×3, 5×5, and 7×7 kernels respectively), capturing cross-channel and cross-spatial information from different receptive fields. Then, using a squeezed activation structure, the importance of each channel is learned, and these weights are used to weight the original features.

[0115] The computation process of this module can be divided into two main steps. First, through three parallel depthwise convolutional branches and subsequent global average pooling, a feature vector for generating channel attention is obtained. The calculation process is expressed as follows:

[0116] ;

[0117] in, For the nth depthwise convolution operation, GAP is the global average pooling operation.

[0118] Next, using the squeeze excitation structure, the weight W of each channel is calculated and used to weight the input features. Then, through residual connections, the weighted features are added to the original input features, thereby enhancing the features. This process can be represented as:

[0119] ;

[0120] in, For Sigmoid activation function, GELU is the GELU activation function. For element-wise multiplication, The first linear transform layer arranged in the order of processing. This is the second linear transformation layer arranged in the processing order.

[0121] like Figure 3 As shown, the processing flow of the adaptive global context module is as follows: Input features After parallel processing using 3×3, 5×5, and 7×7 depthwise separable convolutions, the results are then subjected to a 1×1 convolution operation to obtain feature A1. Feature A1 is then subjected to compression and activation operations to obtain feature A2. Feature A1 and feature A2 are multiplied to obtain feature A3. Feature A3 is then processed sequentially through convolution (1×1 kernel), ReLU activation function, convolution (1×1 kernel), and Sigmoid activation function to obtain feature A4. Feature A4 is then multiplied by the input feature... Add them together to obtain the output features. .

[0122] The squeezing operation includes adaptive average pooling; the activation operation includes concatenated convolution (1×1 kernel), ReLU activation function, convolution (1×1 kernel), and Sigmoid activation function.

[0123] In some embodiments, the processing procedure of the frequency domain-aware feature learning module is as follows:

[0124] shallow features The data is converted to a frequency domain feature map using Fourier transform, and the amplitude feature map is extracted from the frequency domain feature map. and phase feature map :

[0125] ;

[0126] ;

[0127] in, Let be the real part of the frequency domain feature map. This represents the imaginary part of the frequency domain feature map;

[0128] The amplitude characteristic map A and the phase characteristic map are analyzed using the following formula. Perform nonlinear mapping to obtain the enhanced amplitude feature map. and enhanced phase feature map :

[0129] ;

[0130] ;

[0131] Where Conv is a depthwise separable convolution, and LeakyReLU is a ReLU activation function with a leakage coefficient (learnable parameter);

[0132] The enhanced amplitude feature map is analyzed using inverse Fourier transform. and enhanced phase feature map Process to obtain features :

[0133] ;

[0134] in, It is a complex exponential form composed of enhanced phase feature maps. This is the inverse Fourier transform.

[0135] It should be noted that frequency domain enhancement and high-resolution reconstruction restore high-frequency structural details in the image, thereby further improving the quality of the reconstructed image. This step mainly consists of a frequency domain-aware feature learning module and an upsampling module for high-resolution reconstruction. The frequency domain-aware feature learning module is a key innovation of this invention; it can directly learn and recover image detail information in the frequency domain rather than the spatial domain, thus processing high-frequency components more efficiently. Unlike methods that only model features in the spatial domain, the frequency domain-aware feature learning module provides a complementary mechanism, directly enhancing feature representation at the frequency domain level. In the frequency domain, image features can be decomposed into two key components: amplitude and phase. The amplitude map characterizes the energy distribution of each frequency component in the image and is closely related to visual attributes such as brightness and contrast; while the phase map encodes the spatial location, structure, and edge information of the image, which is crucial for accurate geometric reconstruction of the image.

[0136] The frequency domain-aware feature learning module first converts the input spatial domain feature map into a frequency domain representation using Fourier transform, and then separates the amplitude and phase information from it. The amplitude and phase can be represented as follows:

[0137] ;

[0138] ;

[0139] in, and These are the real and imaginary parts of the complex features in the frequency domain, respectively.

[0140] The core of the frequency domain-aware feature learning module is the design of two independent learning branches, used to enhance amplitude and phase information respectively. Through learnable deep separable convolution operations, nonlinear mapping is performed on the amplitude and phase feature maps, thereby achieving adaptive frequency component adjustment. This process can be represented as:

[0141] ;

[0142] ;

[0143] Where Conv is a depthwise separable convolution (with a 3×3 kernel), and LeakyReLU is a ReLU activation function with a leakage coefficient. This is the enhanced amplitude feature map. This is the enhanced phase feature map.

[0144] This design ensures that amplitude and phase information can be learned and corrected independently, effectively avoiding potential mutual interference during processing. Subsequently, the enhanced amplitude and phase are used to reconstruct the frequency domain features back to the spatial domain through inverse Fourier transform, and finally fused with the original input features.

[0145] ;

[0146] in, It is a complex exponential form composed of enhanced phase feature maps. This is the inverse Fourier transform.

[0147] The frequency-domain sensing feature learning module, as a complementary mechanism, enhances the perception and recovery of high-frequency image components by operating directly in the frequency domain. Specifically, the module's amplitude enhancement helps correct the energy distribution of the image, making the brightness and contrast of the reconstructed image closer to the true high-resolution image. Furthermore, its ability to learn and correct phase ensures the accurate recovery of key structural information and edge details, which is crucial for the accurate reconstruction of ground feature boundaries in remote sensing images.

[0148] Therefore, the frequency-domain aware feature learning module and the spatial-domain Transformer architecture of the backbone network form a complementary and synergistic relationship. The Transformer is responsible for capturing the global context and semantics of the remote sensing scene, while the frequency-domain aware feature learning module focuses on extracting high-frequency details. This spatial-frequency collaborative learning strategy enables the model to achieve both excellent global consistency and accurate recovery of local details when dealing with complex remote sensing super-resolution tasks.

[0149] like Figure 4 As shown, the operation flow of the frequency domain-aware feature learning module is as follows: The input signal is transformed from the time domain to the frequency domain using a Fast Fourier Transform (FFT), resulting in a complex frequency domain representation. The real and imaginary parts of the frequency domain signal are separated and extracted through slicing. The separated real and imaginary channels are concatenated to obtain the fused feature e. A 1×1 convolution operation is performed on the concatenated feature e to achieve channel dimension transformation or feature fusion. The convolution output is nonlinearly transformed using the GELU activation function to enhance the model's expressive power. The nonlinearly transformed feature is then re-sliced ​​into real and imaginary parts through slicing again. The separated real and imaginary channels are then synthesized into a complex frequency domain feature. Finally, an inverse Fast Fourier Transform is performed on the recombined complex frequency domain feature to transform it back from the frequency domain to the time domain, yielding the final output.

[0150] Finally, in the frequency domain enhancement and high-resolution reconstruction stage, the high-resolution reconstruction module is responsible for restoring the feature maps after model depth processing into clear, detailed high-resolution images. This process is mainly achieved through sub-pixel convolution and subsequent convolutional refinement. Sub-pixel convolution is an efficient upsampling technique that converts the channel information of the feature map into spatial resolution, thus avoiding the blurring and artifacts caused by traditional interpolation methods when enlarging the image. Subsequently, a final convolutional layer refines the enlarged image to ensure its visual quality and realism. In other words, the high-resolution reconstruction module generates high-resolution images through residual connections and upsampling. This invention significantly improves the detail restoration and reconstruction quality of remote sensing images while ensuring efficiency.

[0151] Furthermore, in step S2, the number n of deep feature modeling and multi-scale fusion modules is greater than or equal to 1.

[0152] The value of n is set according to actual needs. Figure 2 The “×n” in the deep feature modeling and multi-scale fusion module means that the adaptive gated Transformer super-resolution reconstruction model has n cascaded deep feature modeling and multi-scale fusion modules.

[0153] This invention uses peak signal-to-noise ratio (PSNR) as an evaluation function to train an adaptive gated Transformer super-resolution reconstruction network.

[0154] like Figure 5 As shown in the figure, at a super-resolution scale of ×4, the performance and model complexity evaluation results of the present invention and existing methods on the AID dataset are as follows: the methods located near the top left corner achieve a better balance between reconstruction accuracy (higher PSNR) and model complexity (fewer parameters). The AGTSR of the present invention is located in the top left corner of the coordinate system, providing superior PSNR with a relatively compact model size.

[0155] like Figure 6 As shown, on the AID dataset, this invention provides a quantitative comparison of 4x super-resolution methods, covering the performance of state-of-the-art CNN, Transformer, and Mamba models across all 30 scene categories. SSIM represents structural similarity, with red indicating the best result and blue indicating the second-best result.

[0156] like Figure 7As shown, on the AID dataset, the AGTSR method proposed in this invention was compared with CNN, Transformer and Mamba methods at a scale of ×4 from a visual perspective: In the comparison of the same local region, the high-quality reconstruction result should present clearer, continuous and geometrically consistent details on small objects and regular structures, with natural textures and no obvious artifacts. At the same time, the overall subjective visual effect is closest to the real high-resolution image (GT). In the comparison results of the reconstructed images in various scenes, this invention has the best reconstruction effect.

[0157] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0158] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A remote sensing image super-resolution reconstruction method based on adaptive gating Transformer, characterized by: Specifically comprising the following steps: S1: constructing a training set; S2: constructing an adaptive gated Transformer super-resolution reconstruction network, and training the adaptive gated Transformer super-resolution reconstruction network using the training set to obtain an adaptive gated Transformer super-resolution reconstruction model; The adaptive gated Transformer super-resolution reconstruction network comprises a shallow feature extraction module, n deep feature modeling and multi-scale fusion modules, and a high-resolution reconstruction module; The low-resolution image input to the adaptive gated Transformer super-resolution reconstruction network is processed by the shallow feature extraction module to obtain shallow features ; shallow features After processing by the modeling and multi-scale fusion module via n deep features in series, the feature M is obtained. The feature M is processed by the high-resolution reconstruction module to obtain a super-resolution reconstruction image corresponding to the input image; The deep feature modeling and multi-scale fusion module comprises an adaptive gated Top-K attention mechanism, a frequency domain perception feature learning module, and an adaptive global context module, wherein, Processing the shallow feature based on the adaptive gating top-k attention mechanism , and inputting the processing result into an adaptive global context module for processing to obtain a feature ; inputting the shallow feature into a frequency domain perception feature learning module for processing to obtain a feature ; adding the shallow feature , the feature , and the feature to obtain a feature M1; In step S2, the adaptive gating Top-K attention mechanism is used to process the shallow features The specific process of processing is as follows: After replicating the input feature X into four copies, four independent attention branches are formed to project the input feature X k k corresponding query matrix Q, key matrix K and value matrix V, k = 1, 2, 3 or 4;​ Based on the input features X k , the gating weights G k corresponding to each token are generated by the following formula: k : ； wherein, and are learning parameters, is a sigmoid function; Gating weight G k The same operation is performed on the value matrix V, and the weighted value matrix is calculated by : ； wherein is an element-wise multiplication; For the input feature X k The corresponding query matrix Q, key matrix K and value matrix V are operated in the same way, and weight distribution is made for each attention branch by the following formula : ； wherein, is the attention weight of the k-th attention branch, is the Top-K mask function corresponding to the k-th attention branch, is the attention score matrix, d is the feature dimension, is the softmax activation function; Based on the weight distribution results and the gating weighting factors of each attention branch, each attention branch is weighted and fused by the following formula to obtain the feature : ； wherein, is a weighted sum operation, Concat is a feature concatenation operation, Conv is a convolution operation, is the gating weight factor of the kth attention branch. The deep feature modeling and multi-scale fusion module further comprises window displacement attention and a SwiGLU gating unit, and the features After processing by the window displacement attention and the SwiGLU gating unit, the processing result is input into an adaptive global context module for processing, to obtain features ; S3: inputting a remote sensing low-resolution image to be reconstructed into the adaptive gated Transformer super-resolution reconstruction model to obtain a super-resolution reconstruction image.

2. The remote sensing image super-resolution reconstruction method based on adaptive gating Transformer according to claim 1, characterized in that: In step S2, the shallow feature extraction module includes a 1x1 convolutional layer, and after the low-resolution image is processed through the 1x1 convolutional layer, a shallow feature is obtained .

3. The remote sensing image super-resolution reconstruction method based on adaptive gating Transformer according to claim 1, characterized in that: generating a feature vector for input to an adaptive global context module , by the following equation : ； wherein, is the nth depth convolution operation, and GAP is a global average pooling operation. The feature vector is processed by the following formula to obtain the feature : ； wherein, is a Sigmoid activation function, and GELU is a GELU activation function, is an element-wise multiplication, is a first linear transformation layer arranged in processing order, is a second linear transformation layer arranged in processing order.

4. The remote sensing image super-resolution reconstruction method based on adaptive gating Transformer according to claim 1, characterized in that: The processing process of the frequency domain perception feature learning module is as follows: Convert the shallow features to frequency domain feature maps by Fourier transform, and extract amplitude feature maps and phase feature maps from the frequency domain feature maps ； ； wherein, is a real part of the frequency domain feature map, is an imaginary part of the frequency domain feature map; The amplitude feature map A and the phase feature map are non-linearly mapped by the following formula to obtain an enhanced amplitude feature map and an enhanced phase feature map : ； ； Wherein, Conv is a depth separable convolution, and LeakyReLU is a ReLU activation function with a leakage coefficient; by inverse Fourier transform on the enhanced amplitude feature map and the enhanced phase feature map to obtain features : ； wherein, is a complex exponential form composed of the enhanced phase feature map, is an inverse Fourier transform.

5. The remote sensing image super-resolution reconstruction method based on adaptive gating Transformer according to claim 1, characterized in that: In step S2, the number n of deep feature modeling and multi-scale fusion modules is greater than or equal to 1.

6. The remote sensing image super-resolution reconstruction method based on adaptive gating Transformer according to claim 1, characterized in that: In step S1, the specific process of constructing the training set is as follows: from the 30 scene categories of the AID data set, 100 images are randomly selected from each scene category, and 3000 images obtained are used as the training set.

Images