Remote sensing image super-resolution reconstruction method based on adaptive gating Transform

Abstract

The invention belongs to the technical field of remote sensing super-resolution images, and particularly relates to a remote sensing image super-resolution reconstruction method based on an adaptive gating Transform. The method comprises the following steps: S1, constructing a training set; s2, constructing a self-adaptive gated Transform super-resolution reconstruction network, and training the self-adaptive gated Transform super-resolution reconstruction network by using the training set, so as to obtain a self-adaptive gated Transform super-resolution reconstruction model; and S3, inputting a remote sensing low-resolution image to be reconstructed into the adaptive gating Transform super-resolution reconstruction model to obtain a super-resolution reconstruction image. The invention aims to break through the limitation of an existing method in the aspects of efficiency, detail recovery and multi-scale fusion, provides a super-resolution solution with high fidelity and high efficiency for remote sensing images, and meets the urgent demand of refined remote sensing application for high-quality images.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of remote sensing super-resolution images, and particularly relates to a remote sensing image super-resolution reconstruction method based on adaptive gated Transformer. BACKGROUND

[0002] Remote sensing image super-resolution technology aims to recover high-resolution images from low-resolution images to improve the spatial details and texture expression capability of images, and is an important basis for fine feature recognition, environmental monitoring, urban planning and other applications. However, due to the physical limitations of satellite and aerial imaging devices and the inevitable atmospheric disturbances and noise interference in the imaging process, remote sensing images often have insufficient resolution, resulting in blurred feature boundaries and missing details, which affects subsequent high-precision analysis. Traditional interpolation methods such as bicubic interpolation are simple but lack the ability to recover real details, and reconstruction methods based on sparse representation or non-local self-similarity can improve texture performance to some extent, but they rely on complex prior modeling, have large computational overhead and limited adaptability. With the development of deep learning, convolutional neural networks (CNN) have been widely used in super-resolution tasks, such as SRCNN, EDSR, RCAN and other methods, which can learn the relationship between LR images and HR images through end-to-end mapping and have made great breakthroughs in remote sensing image reconstruction. However, due to the fact that convolution inherently relies on local receptive fields, it is difficult to effectively capture global structural relationships when dealing with large-scale scenes, often resulting in excessive smoothing of textures and loss of local details.

[0003] To overcome the limitations of CNN, the Transformer architecture has been gradually introduced into the super-resolution reconstruction task. The self-attention mechanism of Transformer can model long-range dependencies and integrate global context information in a larger range, and has shown good reconstruction results in remote sensing image super-resolution. Typical methods such as SwinIR and HAT-L have achieved excellent performance on multiple public datasets, but still face two main challenges: first, remote sensing images contain a large number of low-information-density background regions such as sky, water, desert, etc., and traditional dense self-attention uniformly calculates all regions, resulting in waste of computational resources and increased noise interference; second, features in remote sensing images exhibit significant multi-scale characteristics, with buildings, roads, farmland and other features differing significantly in size, and existing Transformers mostly use single-scale modeling strategies, making it difficult for the model to balance global consistency and local texture details in complex scenes.

[0004] To solve the above problems, some researches have tried to introduce feature selection or multi-scale fusion mechanism to improve the super-resolution reconstruction quality. For example, Top-K Token Selective Transformer reduces redundant calculation to some extent, but still not accurate enough in capturing key texture areas, and has limited ability to restore high-frequency details. Although the multi-branch convolution or feature fusion structure enhances the model's perception of multi-scale features, the collaborative optimization between global and local information is still insufficient, and there are still problems of blurred details and missing textures. The present application aims to break through the limitations of existing methods in efficiency, detail restoration and multi-scale fusion, and provide a super-resolution solution for remote sensing images that balances high fidelity and high efficiency, to meet the urgent need for high-quality images in fine remote sensing applications. SUMMARY

[0005] Therefore, the present application aims to provide a remote sensing image super-resolution reconstruction method based on adaptive gated Transformer to solve the problem of insufficient collaborative optimization between global and local information in the prior art, and missing details and textures.

[0006] To achieve the above-mentioned purpose, the technical solution of the present application is as follows: A remote sensing image super-resolution reconstruction method based on adaptive gated Transformer, comprising the following steps: S1: Constructing a training set; S2: Constructing an adaptive gated Transformer super-resolution reconstruction network, and training the adaptive gated Transformer super-resolution reconstruction network using the training set to obtain an adaptive gated Transformer super-resolution reconstruction model; The adaptive gated Transformer super-resolution reconstruction network includes a shallow feature extraction module, n deep feature modeling and multi-scale fusion modules, and a high-resolution reconstruction module; After the low-resolution image input into the adaptive gated Transformer super-resolution reconstruction network is processed by the shallow feature extraction module, the shallow feature is obtained. After being processed by the n deep feature modeling and multi-scale fusion modules in series, the feature M is obtained. After being processed by the high-resolution reconstruction module, the super-resolution reconstruction image corresponding to the input image is obtained. S3: Inputting the remote sensing low-resolution image to be reconstructed into the adaptive gated Transformer super-resolution reconstruction model to obtain a super-resolution reconstruction image.

[0007] Furthermore, in step S2, the shallow feature extraction module includes a 1×1 convolutional layer. After processing the low-resolution image through the 1×1 convolutional layer, shallow features are obtained. .

[0008] Furthermore, the deep feature modeling and multi-scale fusion module includes an adaptive gated Top-K attention mechanism, a frequency domain-aware feature learning module, and an adaptive global context module. Based on the adaptive gating Top-K attention mechanism for shallow features The data is processed, and the results are input into the adaptive global context module for further processing to obtain features. ; Shallow features The input is processed by the frequency domain-aware feature learning module to obtain features. ; Shallow features ,feature and characteristics After adding them together, we obtain feature M1.

[0009] Furthermore, in step S2, shallow features are processed based on an adaptive gated Top-K attention mechanism. The specific process for processing is as follows: After copying the input feature X into four copies, four independent attention branches are formed to focus on the input feature X. k Perform linear projection to obtain the input feature X k The corresponding query matrix Q, key matrix K, and value matrix V, k=1, 2, 3, or 4; Based on input features X k The following formula is used to express the relationship between the input feature X and the input feature X. k The corresponding generation gating weight G for each token k : ; in, and All are learning parameters. For the Sigmoid function; Gating weight G k Perform the same operation as with the value matrix V, and calculate the weighted value matrix using the following formula. : ; in, For element-wise multiplication; For input features X k The same operation is performed on the corresponding query matrix Q, key matrix K, and value matrix V, and weights are assigned to each attention branch using the following formula. : ; wherein, is the attention weight of the kth attention branch, is the Top-K mask function corresponding to the kth attention branch, is the attention score matrix, d is the feature dimension, is the softmax activation function; Based on the weight distribution results of each attention branch and the gating weighting factor, each attention branch is weighted and fused by the following formula to obtain the feature : ; wherein, is the weighted sum operation, Concat is the feature concatenation operation, Conv is the convolution operation, is the gating weighting factor of the kth attention branch.

[0010] Further, the deep feature modeling and multi-scale fusion module further includes window displacement attention and SwiGLU gating unit, and the feature After processing by the window displacement attention and the SwiGLU gating unit, the processing result is input to the adaptive global context module for processing to obtain the feature .

[0011] Further, for the feature input to the adaptive global context module, a feature vector is generated by the following formula: ; wherein, is the nth deep convolution operation, GAP is the global average pooling operation; The feature vector is processed by the following formula to obtain the feature : ; wherein, is the Sigmoid activation function, GELU is the GELU activation function, is the element-wise multiplication, is the first linear transformation layer arranged in the processing order, is the second linear transformation layer arranged in the processing order.

[0012] Further, the processing process of the frequency domain perception feature learning module is as follows: The shallow feature is converted into a frequency domain feature map by Fourier transform, and an amplitude feature map and a phase feature map : ; ; wherein, is a real part of the frequency domain feature map, is an imaginary part of the frequency domain feature map; The amplitude feature map A and the phase feature map are nonlinearly mapped by the following formula to obtain an enhanced amplitude feature map and an enhanced phase feature map : ; ; wherein, Conv is a depth separable convolution, and LeakyReLU is a ReLU activation function with a leakage coefficient; The enhanced amplitude feature map and the enhanced phase feature map are processed by inverse Fourier transform to obtain a feature : ; wherein, is a complex exponential form composed of the enhanced phase feature map, is inverse Fourier transform.

[0013] Further, in step S2, the number n of deep feature modeling and multi-scale fusion modules is greater than or equal to 1.

[0014] Further, in step S1, the specific process of constructing the training set is as follows: 100 images are randomly selected from each of the 30 scene categories of the AID dataset, and 3000 images are obtained as the training set.

[0015] Compared with the prior art, the application can achieve the following beneficial effects: (1) The adaptive gating Transformer-based remote sensing image super-resolution reconstruction method provided by the present application accurately solves the core problem existing in remote sensing image super-resolution reconstruction, and is tested on an AID dataset with multiple advanced models. From the test results, it can be seen that the existing Transformer-based model has huge computational redundancy and low efficiency when processing high-resolution images, as it processes all tokens equally. The adaptive gating Top-K attention mechanism used in the present application can intelligently filter and focus on the most information-valuable tokens, significantly reducing the computational complexity while concentrating computing resources in key areas, greatly improving the processing efficiency. On the AID dataset, AGTSR is 0.07dB higher than the current most advanced HAT-L in PSNR, while only using 86.64% of the computing resources of HAT-L.

[0016] (2) The adaptive gating Transformer-based remote sensing image super-resolution reconstruction method provided by the present application solves the problem of complex and diverse ground feature scales in remote sensing images. The adaptive global context module proposed by the present application uses multi-branch deep convolution kernels and squeeze-and-excitation mechanisms, enabling the model to flexibly capture and fuse feature information of different scales, thereby solving the limitation of existing methods in modeling global context information when processing multi-scale targets. Based on the adaptive global context module, the present application can better understand and reconstruct various scale ground features from small vehicles to large areas of farmland, improving the reconstruction accuracy.

[0017] (3) The adaptive gating Transformer-based remote sensing image super-resolution reconstruction method provided by the present application solves the problem of loss of high-frequency details (such as texture and edges) during the degradation process. The present application uses a frequency domain perception feature learning module to independently learn and enhance the amplitude and phase information in the frequency domain, which is a complementary mechanism to the spatial domain method. This unique spatial-frequency collaborative learning strategy enables the model to more accurately restore the tiny details that are crucial for visual quality and ground feature recognition, making up for the shortcomings of spatial domain methods in detail restoration. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of the present application, and their description, are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings: Figure 1 The flowchart of the adaptive gating Transformer-based remote sensing image super-resolution reconstruction method described in the embodiments of the present application; Figure 2A network structure schematic diagram of the adaptive gating Transformer super-resolution reconstruction model described in the embodiment of the present invention; Figure 3 A network structure schematic diagram of the adaptive global context module described in the embodiment of the present invention; Figure 4 A network structure schematic diagram of the frequency domain perception feature learning module described in the embodiment of the present invention; Figure 5 A comparison diagram of the present invention and other mainstream models on the AID dataset described in the embodiment of the present invention; Figure 6 A 4-fold super-resolution reconstruction comparison diagram of different models on the AID dataset described in the embodiment of the present invention; Figure 7 A 4-fold super-resolution reconstruction quality evaluation diagram of different models on the AID dataset described in the embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and do not constitute a limitation on the present invention.

[0020] It should be noted that the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.

[0021] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second" and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features limited by "first", "second" and the like can explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0022] In the description of the present application, it should be noted that unless specifically defined and limited, the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through intermediate medium, or the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0023] Although the existing deep learning-based remote sensing image super-resolution method has made significant progress in performance, there are still some key problems that have not been effectively solved: first, the Transformer architecture generally has redundant calculation overhead when processing remote sensing images, that is, the low information background area in the image is treated equally, which leads to a decrease in calculation efficiency and easy introduction of noise; second, the existing method has insufficient modeling ability for multi-scale features in remote sensing images, and there is a lack of effective fusion between different scale features, so the reconstruction result is often difficult to balance the global structural consistency and local fine texture; third, the recovery ability of high-frequency information such as edges and textures is limited, which is prone to missing details or blurring, and it is difficult to meet the demand for fine reconstruction in remote sensing applications.

[0024] In view of the above defects, the purpose of the present application is to provide a remote sensing image multi-scale feature fusion super-resolution method based on adaptive gated Transformer, so as to realize efficient and fine image reconstruction. The present application introduces adaptive gating and Top-K selection attention mechanism to dynamically filter and enhance high information density areas, effectively suppress the interference of redundant background on modeling, and improve the calculation efficiency and information capturing ability; the present application designs an adaptive global context module to realize feature depth fusion under different receptive fields, so that the model has both global structure modeling and local detail recovery ability; the multi-scale feature transformation unit based on SwiGLU is introduced in the feedforward network to enhance the non-linear expression ability and improve the modeling flexibility of complex textures; and the frequency domain perception feature learning module is used to adaptively enhance the high-frequency features on two levels of amplitude and phase, which fundamentally improves the recovery effect of details and edges. Through the above technical means, the present application aims to break through the limitations of existing methods in efficiency, detail recovery and multi-scale fusion, and provide a super-resolution solution for remote sensing images that takes into account high fidelity and high efficiency, to meet the urgent demand for high-quality images in fine remote sensing applications.

[0025] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0026] As Figure 1As shown, a remote sensing image super-resolution reconstruction method based on adaptive gated Transformer, specifically comprising the following steps: S1: constructing a training set; S2: constructing an adaptive gated Transformer super-resolution reconstruction network, and training the adaptive gated Transformer super-resolution reconstruction network using the training set to obtain an adaptive gated Transformer super-resolution reconstruction model; the adaptive gated Transformer super-resolution reconstruction network includes a shallow feature extraction module, n deep feature modeling and multi-scale fusion modules, and a high-resolution reconstruction module; after a low-resolution image input into the adaptive gated Transformer super-resolution reconstruction network is processed by the shallow feature extraction module, shallow features are obtained After being processed by the n deep feature modeling and multi-scale fusion modules in series, feature M is obtained; after feature M is processed (convolution and pixel rearrangement) by the high-resolution reconstruction module, a super-resolution reconstruction image corresponding to the input image is obtained; S3: inputting a remote sensing low-resolution image to be reconstructed into the adaptive gated Transformer super-resolution reconstruction model to obtain a super-resolution reconstruction image.

[0027] It should be noted that the remote sensing image multi-scale feature fusion super-resolution method based on adaptive gated Transformer proposed in the present application organically combines the local feature extraction capability of convolution and the global dependency modeling advantage of Transformer, and realizes dynamic screening of high information density areas through an adaptive gating mechanism, significantly reduces redundant calculation, and improves attention to key texture areas. At the same time, the adaptive global context module designed in the present application can extract and fuse multi-scale features under different receptive fields, so that the model can balance global consistency and fine expression of local details in complex remote sensing scenes. In addition, a multi-scale feature transformation unit based on SwiGLU is introduced in the feedforward structure of the network to enhance the non-linear capability of feature conversion, improve the diversity and flexibility of information expression, and on this basis, a frequency domain perception feature learning module is constructed, which strengthens the recovery capability of high-frequency components such as edges and textures through adaptive adjustment of frequency domain amplitude and phase information. This series of improvements not only makes the present application superior to the existing most advanced models in quantitative indicators, but also can more realistically restore fine textures in remote sensing images in subjective visual effects, has stronger practical value and popularization potential.

[0028] In some embodiments, in step S2, the shallow feature extraction module includes a 1x1 convolution layer, and after the low-resolution image is processed by the 1x1 convolution layer, shallow features are obtained.

[0029] It should be noted that the shallow feature extraction module further includes a normalization layer. After the low-resolution image is processed by the 1x1 convolution layer, the data is standardized by the normalization layer to stabilize the training process, accelerate convergence and improve the model generalization ability.

[0030] Further, the input remote sensing low-resolution image to be reconstructed is subjected to shallow feature extraction. Specifically, the low-resolution image is first input into a convolutional neural network, and a sliding operation of local receptive field is performed on the input image by a plurality of convolution kernels to obtain an initial feature representation containing basic information such as edges and textures. This process not only effectively preserves the low-level structure information of the input image, but also suppresses noise and redundant background information, thereby laying a foundation for subsequent deep feature modeling and multi-scale fusion. The calculation formula is as follows: ; wherein, represents the input low-resolution remote sensing image, represents a shallow feature extraction function composed of a convolution layer, represents the output shallow feature map. Through this step, the model can maintain computational efficiency while obtaining relatively stable and representative initial features, so that the subsequent adaptive gated attention mechanism and multi-scale context modeling can more accurately focus on key areas and perform detail restoration.

[0031] In some embodiments, the deep feature modeling and multi-scale fusion module includes an adaptive gated Top-K attention mechanism, a frequency domain perception feature learning module, and an adaptive global context module, wherein, the shallow feature is processed based on the adaptive gated Top-K attention mechanism, and the processing result is input into the adaptive global context module for processing to obtain a feature ; the shallow feature is input into the frequency domain perception feature learning module for processing to obtain a feature ; and the shallow feature , the feature , and the feature are added to obtain a feature M1.

[0032] Further, the deep feature modeling and multi-scale fusion module further includes window displacement attention and SwiGLU gating units, and the feature is processed by the window displacement attention and the SwiGLU gating units, and the processing result is input into the adaptive global context module for processing to obtain a feature .

[0033] It should be noted that in the deep modeling and multi-scale fusion stages, this invention introduces an adaptive gated Top-K attention mechanism to dynamically filter and enhance high-information regions, effectively reducing redundant computation. Subsequently, an adaptive global context module is used to fuse features at different scales, taking into account both global structure and local texture. The feedforward network combines multi-scale transformation with SwiGLU gated units to improve nonlinear representation and detail recovery capabilities. Simultaneously, a frequency-domain perceptual feature learning module is introduced to enhance high-frequency texture and edge reconstruction using amplitude and phase information.

[0034] In some embodiments, in step S2, shallow features are processed based on an adaptive gated Top-K attention mechanism. The specific process for processing is as follows: After copying the input feature X into four copies, four independent attention branches are formed to focus on the input feature X. k Perform linear projection to obtain the input feature X k The corresponding query matrix Q, key matrix K, and value matrix V, k=1, 2, 3, or 4; Based on input features X k The following formula is used to express the relationship between the input feature X and the input feature X. k The corresponding generation gating weight G for each token k : ; in, and All are learning parameters. For the Sigmoid function; Gating weight G k Perform the same operation as with the value matrix V, and calculate the weighted value matrix using the following formula. : ; in, For element-wise multiplication; For input features X k The same operation is performed on the corresponding query matrix Q, key matrix K, and value matrix V, and weights are assigned to each attention branch using the following formula. : ; in, Let the attention weights be those of the k-th attention branch. For the Top-K mask function corresponding to the k-th attention branch, Let d be the attention score matrix, and d be the feature dimension. It is the softmax activation function; Based on the weight distribution results and the gating weighting factors of each attention branch, each attention branch is weighted and fused by the following formula to obtain the feature : ; wherein, is a weighted summation operation, Concat is a feature concatenation operation, Conv is a 3x3 convolution operation, is the gating weighting factor of the kth attention branch.

[0035] It should be noted that, as Figure 2 indicated, in the deep feature modeling stage, the present application introduces an adaptive gated Top-K attention mechanism, which can dynamically filter and strengthen regions with high information content, effectively reducing redundant calculations caused by complex backgrounds in remote sensing images by selectively focusing on the most critical feature tokens. Subsequently, effective fusion of multi-scale features is achieved through an adaptive global context module, which can balance global structure and local texture, ensuring that the model can recover rich details and accurate spatial structure when processing different scale features.

[0036] Further, the main purpose of the adaptive gated Top-K attention mechanism is to solve the computational redundancy and information dispersion problems faced by traditional Transformer-based super-resolution models when processing high-resolution remote sensing images. The core design of this mechanism combines adaptive Top-K ratio predictors and multi-branch attention fusion strategies, allowing it to dynamically select the most critical tokens for computation and efficiently fuse multi-branch information through a gating mechanism.

[0037] The workflow of the adaptive gated Top-K attention mechanism mainly includes four steps: feature projection, adaptive gating, multi-branch sparse attention calculation, and gated weighted fusion output. The last two steps together constitute the parallel processing and fusion process of the 4 attention branches: First, the input feature X is projected linearly to obtain the query matrix Q, key matrix K, and value matrix V required by the attention mechanism: , , ; wherein, , , are learnable weight matrices.

[0038] Subsequently, the adaptive gating mechanism generates a gating weight to evaluate its importance, to achieve the filtering of redundant information: ; wherein, and are learning parameters, is a Sigmoid function. The weight is then applied to the value matrix to generate a weighted value matrix to enhance the influence of key tokens and suppress redundant tokens. In the multi-branch sparse attention calculation stage, the adaptive gated Top-K attention mechanism designs four independent attention branches (k = 1, 2, 3, 4). In each attention branch, the attention score matrix will be sparsified by the Top-K mask function, which dynamically retains only the k most relevant attention values between each query matrix and key matrix according to the output of the adaptive Top-K ratio predictor. This ensures that each attention branch can efficiently focus on different degrees of key dependencies, thereby producing four different scale sparse attention results. Finally, in the gated weighted fusion and output stage, the sparse attention results from the four attention branches are weighted and fused through the dynamically learned gating weights . Finally, an output feature is generated through a convolution layer, so as to balance the information captured under different Top-K ratios and improve the information expression capability: ; ; In some embodiments, for the input to the adaptive global context module feature , a feature vector is generated by: ; wherein, is the nth depth convolution operation, and GAP is the global average pooling operation; The feature vector is processed by: ; wherein, is a Sigmoid activation function, GELU is a GELU activation function, is an element-wise multiplication, is the first linear transformation layer arranged in the processing order, is the second linear transformation layer arranged in the processing order.

[0039] ​It should be noted that the adaptive global context module aims to solve the problem of insufficient global context information modeling capability of the prior art when processing multi-scale targets and complex remote sensing scenes. The adaptive global context module effectively integrates feature information from different levels through an innovative deep convolution kernel and squeeze-and-excitation mechanism, thereby enhancing the model's recognition and reconstruction capability for multi-scale targets.

[0040] The core idea of the adaptive global context module is that it can dynamically aggregate global features and local features to adapt to object size changes in different scenes. As shown in Figure 3 The module first branches the input feature map, one branch remains unchanged as the input of the residual connection, and the other branch obtains global context information through the channel attention mechanism. The channel attention branch first uses three deep convolution kernel branches (convolution kernels are 3x3, 5x5 and 7x7) to extract deep features from the original feature map, capturing cross-channel and cross-spatial information in different receptive fields. Then, the squeeze-and-excitation structure is used to learn the importance of each channel, and these weights are used to weight the original features.

[0041] The calculation process of the module can be divided into two main steps. First, through three parallel deep convolution branches and subsequent global average pooling, a feature vector for generating channel attention is obtained, and the calculation process is represented as: ; wherein, is the nth deep convolution operation, and GAP is the global average pooling operation.

[0042] Next, the squeeze-and-excitation structure is used to calculate the weight W of each channel, which is used to weight the input features, and then the weighted features are added to the original input features through a residual connection, thereby enhancing the features. This process can be represented as: ; wherein, is the Sigmoid activation function, GELU is the GELU activation function, is element-wise multiplication, is the first linear transformation layer arranged in processing order, is the second linear transformation layer arranged in processing order.

[0043] As shown in Figure 3 , the processing flow of the adaptive global context module is: the input feature After being processed by parallel 3x3 depth separable convolution, 5x5 depth separable convolution and 7x7 depth separable convolution, the respective processing results are processed by a convolution operation with a convolution kernel of 1x1 to obtain feature A1, the feature A1 is subjected to squeezing and excitation operations to obtain feature A2, the feature A1 is multiplied by the feature A2 to obtain feature A3, the feature A3 is sequentially processed by convolution (convolution kernel is 1x1), Relu activation function, convolution (convolution kernel is 1x1), Sigmoid activation function to obtain feature A4, the feature A4 is added to the input feature to obtain output feature .

[0044] The operation of squeezing includes adaptive average pooling; the operation of excitation includes convolution (convolution kernel is 1x1) in series, Relu activation function, convolution (convolution kernel is 1x1), Sigmoid activation function.

[0045] In some embodiments, the processing process of the frequency domain perception feature learning module is as follows: The shallow feature is converted into a frequency domain feature map by Fourier transform, and an amplitude feature map and a phase feature map are extracted from the frequency domain feature map ; ; wherein, is a real part of the frequency domain feature map, is an imaginary part of the frequency domain feature map; The amplitude feature map A and the phase feature map are nonlinearly mapped by the following formula to obtain an enhanced amplitude feature map and an enhanced phase feature map : ; ; wherein, Conv is a depth separable convolution, and LeakyReLU is a ReLU activation function with a leakage coefficient (a learnable parameter); The enhanced amplitude feature map and the enhanced phase feature map are processed by inverse Fourier transform to obtain feature : ; wherein, is a complex exponential form composed of the enhanced phase feature map, is inverse Fourier transform.

[0046] It should be noted that by frequency domain enhancement and high resolution reconstruction, high frequency structure details in the image are recovered, thereby further improving the quality of the reconstructed image. This step mainly consists of a frequency domain perceptual feature learning module and an upsampling module for high resolution reconstruction. The frequency domain perceptual feature learning module is a key innovation point of the present application, which can directly learn and recover the detail information of the image in the frequency domain rather than the spatial domain, thereby more efficiently processing the high frequency components. Unlike the method of modeling features only in the spatial domain, the frequency domain perceptual feature learning module provides a complementary mechanism to enhance the feature representation directly at the frequency domain level. In the frequency domain, image features can be decomposed into two key components: amplitude and phase. The amplitude map represents the energy distribution of each frequency component in the image, which is closely related to visual attributes such as brightness and contrast of the image; while the phase map encodes the spatial position, structure and edge information of the image, which is crucial for accurate geometric reconstruction of the image.

[0047] The frequency domain perceptual feature learning module first converts the input spatial domain feature map into a frequency domain representation through Fourier transform, and separates the amplitude and phase information from it. The amplitude and phase can be represented as: ; ; wherein, and are the real and imaginary parts of the frequency domain complex feature, respectively.

[0048] The core of the frequency domain perceptual feature learning module is to design two independent learning branches for enhancing the amplitude and phase information, respectively. Through a learnable depth separable convolution operation, nonlinear mapping is performed on the amplitude feature map and the phase feature map, thereby realizing adaptive frequency component adjustment. This process can be represented as: ; ; wherein, Conv is a depth separable convolution (convolution kernel is 3x3), LeakyReLU is a ReLU activation function with a leakage coefficient, is the enhanced amplitude feature map, is the enhanced phase feature map.

[0049] This design ensures that the amplitude information and the phase information can be learned and corrected independently, effectively avoiding mutual interference between them in the processing process. Subsequently, the enhanced amplitude and the enhanced phase are used to reconstruct the frequency domain features back to the spatial domain through inverse Fourier transform, and finally fused with the original input features.

[0050] ; wherein, is a complex exponential form composed of enhanced phase feature maps, is an inverse Fourier transform.

[0051] As a complementary mechanism, the frequency domain perceptual feature learning module enhances the perception and recovery of high-frequency image components by directly operating in the frequency domain. Specifically, the enhancement of the amplitude by the module helps to correct the energy distribution of the image, making the brightness and contrast of the reconstructed image closer to the real high-resolution image. In addition, its ability to learn and correct the phase ensures the accurate recovery of key structural information and edge details, which is crucial for the accurate reconstruction of feature boundaries in remote sensing images.

[0052] Therefore, the frequency domain perceptual feature learning module and the spatial domain Transformer architecture of the backbone network form a complementary and synergistic relationship. The Transformer is responsible for capturing the global context and semantics of the remote sensing scene, while the frequency domain perceptual feature learning module focuses on refining high-frequency details. This spatial-frequency collaborative learning strategy enables the model to achieve excellent global consistency and accurate recovery of local details when processing complex remote sensing super-resolution tasks.

[0053] As shown in Figure 4 The operation process of the frequency domain perceptual feature learning module is as follows: based on the fast Fourier transform, the input signal is subjected to fast Fourier transform to convert it from the time domain to the frequency domain, obtaining the frequency domain representation in complex form; the real part and the imaginary part of the frequency domain signal are separated by slicing operation, and are extracted respectively; the separated real part channel and imaginary part channel are subjected to concatenation operation, obtaining the fused feature e; 1x1 convolution operation is performed on the concatenated feature e to realize channel dimension transformation or feature fusion; the convolution output is subjected to non-linear transformation by GELU activation function to enhance the expression ability of the model; the feature after non-linear transformation is again split into real part and imaginary part by slicing operation. The split real part channel and imaginary part channel are combined into complex frequency domain features in complex form. The recombined complex frequency domain features are subjected to inverse fast Fourier transform to convert them from the frequency domain back to the time domain, obtaining the final output.

[0054] Finally, in the frequency domain enhancement and high-resolution reconstruction stage, the high-resolution reconstruction module is responsible for restoring the feature maps processed by the model depth into clear and detailed high-resolution images. This process is mainly achieved through sub-pixel convolution and subsequent convolution refinement. Sub-pixel convolution is an efficient up-sampling technique that can convert channel information of feature maps into spatial resolution, thereby avoiding blurring and artifacts caused by traditional interpolation methods when enlarging images. Subsequently, a final convolution layer refines the enlarged image to ensure its visual quality and realism. That is, the high-resolution reconstruction module generates high-resolution images through residual connection and up-sampling. The present application significantly improves the detail restoration and reconstruction quality of remote sensing images while ensuring efficiency.

[0055] Further, in step S2, the number n of deep feature modeling and multi-scale fusion modules is greater than or equal to 1.

[0056] The number n is set according to actual needs, Figure 2 The "n" of the deep feature modeling and multi-scale fusion module indicates that the adaptive gating Transformer super-resolution reconstruction model has n deep feature modeling and multi-scale fusion modules in series.

[0057] The present application trains the adaptive gating Transformer super-resolution reconstruction network using peak signal-to-noise ratio (PSNR) as the evaluation function.

[0058] As shown in Figure 5 , under the ×4 super-resolution scale, the performance and model complexity evaluation results of the present application and existing methods on the AID dataset are as follows: the methods located in the upper left corner of the graph achieve a better balance between reconstruction accuracy (higher PSNR) and model complexity (fewer parameters). The AGTSR of the present application is located in the upper left corner of the coordinate system, providing superior PSNR with a relatively compact model size.

[0059] As shown in Figure 6 , on the AID dataset, the present application quantitatively compares 4x super-resolution methods, covering the performance of the most advanced CNN, Transformer, and Mamba models in all 30 scene categories. SSIM is the structural similarity, where red indicates the optimal result and blue indicates the suboptimal result.

[0060] As shown in Figure 7 , the present application quantitatively compares 4x super-resolution methods on the AID dataset, covering the performance of the most advanced CNN, Transformer, and Mamba models in all 30 scene categories. SSIM is the structural similarity, where red indicates the optimal result and blue indicates the suboptimal result.As shown, the AGTSR method proposed in the application is compared with the CNN, Transformer and Mamba methods from a visual perspective on the AID dataset at a scale of ×4: in the same local area comparison, the high-quality reconstruction result should present clearer, continuous and geometrically consistent details on small targets and regular structures, the texture is natural without obvious artifacts, and the overall subjective visual effect is closest to the real high-resolution image (GT); in the comparison results of the reconstruction images of various scenes, the application has the optimal reconstruction effect.

[0061] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, the steps described in the present disclosure can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present disclosure can be achieved, which is not limited herein.

[0062] The above specific embodiments do not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A remote sensing image super-resolution reconstruction method based on adaptive gating Transformer, characterized by: Specifically comprising the following steps: S1: constructing a training set; S2: constructing an adaptive gated Transformer super-resolution reconstruction network, and training the adaptive gated Transformer super-resolution reconstruction network using the training set to obtain an adaptive gated Transformer super-resolution reconstruction model; The adaptive gated Transformer super-resolution reconstruction network comprises a shallow feature extraction module, n deep feature modeling and multi-scale fusion modules, and a high-resolution reconstruction module; The low-resolution image input to the adaptive gated Transformer super-resolution reconstruction network is processed by the shallow feature extraction module to obtain shallow features ; shallow features After processing by the modeling and multi-scale fusion module via n deep features in series, the feature M is obtained. After processing of the feature M by the high-resolution reconstruction module, a super-resolution reconstruction image corresponding to the input image is obtained; S3: inputting a to-be-reconstructed remote sensing low-resolution image into the adaptive gated Transformer super-resolution reconstruction model to obtain a super-resolution reconstruction image.

2. The remote sensing image super-resolution reconstruction method based on adaptive gating Transformer according to claim 1, characterized in that: In step S2, the shallow feature extraction module includes a 1x1 convolutional layer, and after the low-resolution image is processed through the 1x1 convolutional layer, a shallow feature is obtained .

3. The remote sensing image super-resolution reconstruction method based on adaptive gating Transformer according to claim 1, characterized in that: The deep feature modeling and multi-scale fusion module comprises an adaptive gated Top-K attention mechanism, a frequency domain perception feature learning module, and an adaptive global context module, wherein, Based on adaptive gating top-k attention mechanism to shallow feature is processed, and the processing result is input to an adaptive global context module for processing to obtain feature ; the shallow feature is input to a frequency domain perception feature learning module for processing to obtain feature ; the shallow feature , feature and feature are added to obtain feature M1.

4. The remote sensing image super-resolution reconstruction method based on adaptive gating Transformer according to claim 3, characterized in that: In step S2, the shallow features are processed based on an adaptive gating Top-K attention mechanism The specific process of processing is as follows: After replicating the input feature X into four copies, four independent attention branches are formed to project the input feature X k k corresponding query matrix Q, key matrix K and value matrix V, k = 1, 2, 3 or 4;​ Based on the input features X k , the gating weights G k corresponding to each token are generated by the following formula: k : ； wherein, and are learning parameters, is a sigmoid function; Gating weight G k The same operation is performed on the value matrix V, and the weighted value matrix is calculated by : ； wherein is an element-wise multiplication; For the input feature X k The corresponding query matrix Q, key matrix K and value matrix V are operated in the same way, and weight distribution is made for each attention branch by the following formula : ； wherein, is the attention weight of the k-th attention branch, is the Top-K mask function corresponding to the k-th attention branch, is the attention score matrix, d is the feature dimension, is the softmax activation function; Based on the weight distribution result and the gating weighting factor of each attention branch, each attention branch is weighted and fused by the following formula to obtain the feature : ； wherein, is a weighted sum operation, Concat is a feature concatenation operation, Conv is a convolution operation, is the gating weight factor of the k-th attention branch.

5. The method of claim 3, wherein the method is based on adaptive gating Transformer. The deep feature modeling and multi-scale fusion module further comprises window displacement attention and SwiGLU gating units, and the features After processing by the window displacement attention and SwiGLU gating units, the processing result is input into the adaptive global context module for processing, to obtain features .

6. The remote sensing image super-resolution reconstruction method based on adaptive gating Transformer according to claim 3 or 5, characterized in that: generating a feature vector for input to an adaptive global context module , by the following equation : ； wherein, is the nth depth convolution operation, and GAP is a global average pooling operation. The feature vector is processed by the following formula to obtain the feature : ； wherein, is a Sigmoid activation function, and GELU is a GELU activation function, is an element-wise multiplication, is a first linear transformation layer arranged in processing order, is a second linear transformation layer arranged in processing order.

7. The remote sensing image super-resolution reconstruction method based on adaptive gating Transformer according to claim 3, characterized in that: The processing process of the frequency domain perception feature learning module is as follows: Convert the shallow features to frequency domain feature maps by Fourier transform, and extract amplitude feature maps and phase feature maps from the frequency domain feature maps ； ； wherein, is a real part of the frequency domain feature map, is an imaginary part of the frequency domain feature map; The amplitude feature map A and the phase feature map are non-linearly mapped by the following formula to obtain an enhanced amplitude feature map and an enhanced phase feature map : ； ； wherein Conv is a depth separable convolution, and LeakyReLU is a ReLU activation function with a leakage coefficient; by inverse Fourier transform on the enhanced amplitude feature map and the enhanced phase feature map to obtain features : ； wherein, is a complex exponential form composed of the enhanced phase feature map, is an inverse Fourier transform.

8. The remote sensing image super-resolution reconstruction method based on adaptive gating Transformer according to claim 1, characterized in that: In step S2, the number n of the deep feature modeling and multi-scale fusion modules is greater than or equal to 1.

9. The remote sensing image super-resolution reconstruction method based on adaptive gating Transformer according to claim 1, characterized in that: In step S1, the specific process of constructing the training set is as follows: 100 images are randomly extracted from each of 30 scene categories of an AID data set, and 3000 images obtained are taken as the training set.

Images