Remote sensing image super-resolution system and method based on adaptive mamba-attention network

Abstract

The present application belongs to the technical field of remote sensing super-resolution image, and particularly relates to a remote sensing image super-resolution system and method based on an adaptive Mamba-attention network. The system comprises a feature extraction module, a plurality of cascaded adaptive state space blocks and a reconstruction module. The feature extraction module is used for shallow feature extraction of an input low-resolution image to obtain shallow features. The plurality of cascaded adaptive state space blocks are used for processing the shallow features to obtain reconstruction features. The reconstruction module is used for mapping the reconstruction features to a target resolution space through a sub-pixel rearrangement operation to obtain a high-resolution remote sensing image. The present application utilizes a remote sensing frequency perception modulation module to cooperatively process high-frequency details and low-frequency structures in a feature space, and combines sub-pixel rearrangement up-sampling to generate a high-resolution output, thereby realizing high-quality reconstruction of a complex remote sensing scene.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing super-resolution image technology, and particularly relates to a remote sensing image super-resolution system and method based on an adaptive Mamba-attention network. Background Technology

[0002] Remote sensing image super-resolution aims to reconstruct high-resolution images from low-resolution remote sensing images, thereby improving the clarity of ground feature boundaries and texture details. It is a crucial foundational step for tasks such as land use and cover classification, target detection, change monitoring, and disaster assessment. Due to the limited optical aperture of sensors, high imaging altitude, complex atmospheric disturbances, and the influence of noise during imaging, transmission, and compression, satellite and aerial imagery generally suffer from insufficient spatial resolution. Fine information such as small buildings, road structures, and vegetation textures is difficult to accurately resolve, thus limiting the reliability and stability of subsequent high-precision remote sensing applications.

[0003] However, current deep learning methods for super-resolution of remote sensing images suffer from three main shortcomings: First, most existing prompt-based image restoration frameworks employ fixed prompt pools or uniform global prompts, using the same or only block-level varying modulation strategies for different images and even different spatial locations. Static or coarse-grained prompt designs cannot dynamically adjust prompt content and weights based on differences in local texture complexity, structural morphology, and spectral attributes within remote sensing images. In scenarios with diverse land cover categories, multiple scale targets within the same image, and uneven degradation distribution, they often exhibit insufficient adaptability and local reconstruction imbalances. Second, the reflectance responses of different channels corresponding to different bands in RGB remote sensing images contain significant material differences and land cover category information. However, many existing methods treat multi-channel features as homogeneous features after several shallow convolutions and perform uniform convolution or attention operations, lacking explicit modeling of cross-channel spectral correlations and differences. This results in the spectral distribution differences between land cover categories not being fully amplified and utilized, leading to deviations in color consistency and spectral fidelity in the reconstruction results. Furthermore, multi-head self-attention, introduced to reduce the computational complexity of self-attention, typically models independently within pre-defined fixed windows. The lack of sufficient information exchange between windows easily disrupts the structural continuity of roads, building outlines, and other structures at window boundaries, resulting in obvious boundary artifacts and structural breaks. At the same time, many networks still favor single-scale or limited-scale feature representation in their structural design, failing to simultaneously address large-scale background structures and small-scale fine textures. This makes it difficult to achieve both structural integrity and detail clarity in complex multi-scale remote sensing scenes. Summary of the Invention

[0004] In view of this, the present invention aims to provide a remote sensing image super-resolution system and method based on an adaptive Mamba-attention network to solve the problems of existing technologies in remote sensing scenes, which are difficult to fully exploit cross-channel spectral information, have limited mitigation of boundary artifacts caused by window division, and lack pixel-level cues for complex ground textures and multi-scale structures. These shortcomings result in deficiencies in improving the quality and generalization ability of remote sensing image super-resolution reconstruction. The present invention utilizes a remote sensing frequency sensing modulation module to collaboratively process high-frequency details and low-frequency structures in the feature space, and combines sub-pixel rearrangement upsampling to generate high-resolution output, thereby achieving high-quality reconstruction of complex remote sensing scenes.

[0005] To achieve the above objectives, the technical solution created by this invention is implemented as follows:

[0006] A remote sensing image super-resolution method based on an adaptive Mamba-attention network includes:

[0007] The feature extraction module performs shallow feature extraction on the input low-resolution image to obtain shallow features;

[0008] Multiple cascaded adaptive state space blocks process shallow features to obtain reconstructed features; each adaptive state space block includes a multi-head window attention module, a minimum window boundary enhancement module, an adaptive attention state space module, and a remote sensing frequency sensing modulation module, wherein:

[0009] The multi-head window attention module performs local window mapping on the input features and performs long-range dependency modeling on the mapped features to obtain the window attention output features; the minimum window boundary enhancement module performs boundary enhancement processing on the window attention output features to obtain boundary enhancement features; the input features, boundary enhancement features and window attention output features are fused to obtain boundary enhancement intermediate features; the adaptive attention state space module processes the boundary enhancement intermediate features into spectral enhancement features, and performs dynamic cue weight modulation and pixel-level adaptive selection cue processing on the spectral enhancement features to obtain deep features; the deep features are input to the remote sensing frequency sensing modulation module for explicit reconstruction in the frequency domain to obtain the output features of the current adaptive state space block;

[0010] The reconstruction module maps reconstructed features to the target resolution space through subpixel rearrangement operations to obtain high-resolution remote sensing images.

[0011] Furthermore, in the minimum window boundary enhancement module:

[0012] By performing boundary enhancement on the input features at different scales, boundary features are obtained;

[0013] By performing residual connections between the boundary features and the input features, and then processing the results of the residual connections through convolution and activation, boundary enhancement features are obtained.

[0014] Furthermore, in the minimum window boundary enhancement module:

[0015] The input features are processed through three different branches to achieve boundary enhancement at different scales, specifically:

[0016] After processing the input features with channel compression and sigmoid activation, the first type of boundary enhancement response is obtained;

[0017] Branch 2 performs convolution operations on the input features at different scales and adds the results of the convolution operations at different scales to obtain the second type of boundary enhancement response;

[0018] Branch 3 performs global average pooling on the input features, and then performs convolution and sigmoid activation operations on the global average pooling result and the input features respectively, and adds them together to obtain the third type of boundary enhancement response;

[0019] Boundary features are obtained by fusing the first type of boundary enhancement response, the second type of boundary enhancement response, and the third type of boundary enhancement response along the channel dimension.

[0020] Furthermore, the processing procedure of the adaptive attention state space module is as follows:

[0021] Channel-aware spectral attention processing is applied to the input feature F to obtain spectral enhancement features. :

[0022] ;

[0023] ;

[0024] ;

[0025] in, For grouped convolution operators, As an intermediate feature, For 1×1 convolution, Let A be the sigmoid activation function and A be the spectral attention weight map. c is the spatial location index, and 'c' is the channel.

[0026] The spectral enhancement features are input into the enhancement routing network, which performs linear mapping and activation on the spectral enhancement features and calculates the probability of selecting each dynamic cue vector at each pixel location using the Gumbel-Softmax function.

[0027] ;

[0028] ;

[0029] in, It is a non-linear activation function. For spectral enhancement features in The routing vector at that location, For spectral enhancement features in The feature vector at that location, and This is the weight matrix. For spectral enhancement features in Choose the first option The weights of each dynamic cue vector, To calculate the selection of the first The Gumbel-Softmax function is used to calculate the weights of the dynamic cue vectors.

[0030] The spectral enhancement features are input into the dynamic prompting module, which then performs adaptive two-dimensional average pooling calculation on the spectral enhancement features to obtain the global description vector.

[0031] ;

[0032] in, This is the global description vector for channel C. W is the height of the feature map, and W is the width of the feature map;

[0033] Based on the global description vector, a dynamic weight vector is generated using the following formula:

[0034] ;

[0035] in, It is a non-linear activation function. and Both are weight matrices. This is the dynamic weight vector for channel C;

[0036] Let the static prompt pool be By scaling the static cues using a dynamic weight vector, a dynamic cue pool is obtained. The expression for the k-th dynamic cue is:

[0037] ;

[0038] in, For the kth dynamic prompt, This is the kth static suggestion;

[0039] For dynamic prompt pool The dynamic cue vector corresponding to each pixel is obtained by weighted summation based on pixel-level weights. :

[0040] ;

[0041] The dynamic cue vector corresponding to each pixel The input is fed into the adaptive state space equation for calculation, and the calculation results are processed sequentially through the SGN-fold module and the linear mapping layer to obtain the output features.

[0042] Furthermore, the formula for calculating the adaptive state-space equation is as follows:

[0043] ;

[0044] ;

[0045] ;

[0046] in, This is the discretized state transition matrix. The input projection matrix is ​​the discretized form. Let be the hidden state at time i. Let i be the hidden state at time i-1. Let be the input vector at time i. Let C be the output vector at time i, C be the output projection matrix, and P be the dynamic cue vector. and These are the discretized state-space parameters.

[0047] Furthermore, in the remote sensing frequency sensing modulation module:

[0048] Frequency decomposition of the input features yields low-frequency, mid-frequency, and high-frequency features;

[0049] The low-frequency, mid-frequency, and high-frequency features are respectively subjected to feature concatenation and feature averaging operations to obtain concatenated features and mean features respectively;

[0050] After the spliced ​​features are processed by channel attention and spatial attention, feature A1 is obtained;

[0051] The concatenated features are activated by Sigmoid to generate a gated mask. The gated mask and the mean features are then used for gated fusion to obtain the fused features.

[0052] Multiply feature A1, the fused feature, and the input feature to obtain the output feature.

[0053] Furthermore, the reconstruction module includes a convolutional module and a subpixel rearrangement module, wherein:

[0054] After the convolution module performs convolution processing on the reconstructed features, it obtains intermediate features. The intermediate features are then added to the shallow features to obtain feature A2.

[0055] The subpixel rearrangement module maps feature A2 to the target resolution space to obtain a high-resolution remote sensing image.

[0056] A remote sensing image super-resolution method based on an adaptive Mamba-attention network includes:

[0057] S1: Obtain high-resolution images, perform downsampling degradation and preprocessing operations on the high-resolution images to obtain the training set;

[0058] S2: Construct a remote sensing image super-resolution system based on an adaptive Mamba-attention network;

[0059] S3: Use the training set obtained in step S1 to train the remote sensing image super-resolution system constructed in step S2 to obtain the super-resolution model;

[0060] S4: Input the low-resolution image to be reconstructed into the super-resolution model trained in step S3 to obtain the reconstructed image.

[0061] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0062] (1) The remote sensing image super-resolution system and method based on adaptive Mamba-attention network described in this invention addresses three key issues: lack of scene adaptability of static cues, insufficient utilization of spectral information, and insufficient spatial structure and multi-scale modeling capabilities. It provides an improved scheme that closely coordinates network structure and modeling mechanism, significantly improving the reconstruction quality and applicability of remote sensing image super-resolution. First, in terms of cue modeling, this invention extends static or block-level cues to pixel-level dynamic cues. Through the dynamic cue weight modulation mechanism, cue vectors related to local texture, structure, and degradation characteristics are explicitly introduced during the state space update process. This enables the model to adopt differentiated reconstruction strategies for flat areas, small targets, and complex texture regions in the same image, overcoming the coarse-grained modulation defect of existing methods where "the same cue corresponds to the global or large area," and greatly enhancing the adaptability to complex scenes and multi-scale targets.

[0063] (2) The remote sensing image super-resolution system and method based on adaptive Mamba-attention network described in this invention, in terms of spectral-spatial joint modeling, introduces a channel-aware spectral attention module to explicitly model and reweight each RGB channel. Compared with traditional networks that treat multi-channel features as homogeneous, this can better amplify the differences in spectral response of features such as buildings, vegetation, and water bodies, improve spectral separability and color fidelity, and reduce phenomena such as roof color shift and water reflection distortion, providing a spectral enhancement feature basis for subsequent cue modulation and state space modeling. In addition, by combining the minimum window boundary enhancement module and the remote sensing frequency-aware modulation module, this invention simultaneously alleviates the boundary discontinuity problem caused by window division in both spatial and frequency dimensions, and explicitly balances high-frequency details and low-frequency structures in the feature space, making the structure of areas such as roads and building edges more coherent and the texture of small targets clearer. In complex multi-scale scenes, it is easier to balance global consistency and local fineness than existing Transformer and Mamba series models.

[0064] (3) The remote sensing image super-resolution system and method based on adaptive Mamba-attention network described in this invention, in terms of comprehensive performance, experimental results on the typical AID remote sensing dataset show that in the 4x super-resolution task, compared with representative methods such as MambaIR, this invention achieves stable improvements in PSNR, SSIM and other indicators. Among them, the PSNR is improved by about 0.14dB on the AID dataset, and the number of parameters and computational overhead only increase moderately, achieving a better trade-off between reconstruction quality and efficiency. At the same time, since the dynamic cueing, spectral attention and frequency sensing modules are all designed with lightweight features, the system exhibits better generalization and robustness under different sensors, different scenes and different degradation distributions, making it more suitable for long-term deployment and promotion in actual remote sensing operations. Attached Figure Description

[0065] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0066] Figure 1 A schematic diagram of the structure of the remote sensing image super-resolution system based on an adaptive Mamba-attention network as described in an embodiment of the present invention;

[0067] Figure 2 A schematic diagram of the minimum window boundary enhancement module described in an embodiment of the present invention;

[0068] Figure 3A partial processing structure diagram of the adaptive attention state space module described in an embodiment of the present invention;

[0069] Figure 4 A schematic diagram of the structure of the remote sensing frequency sensing modulation module described in the embodiment of the present invention;

[0070] Figure 5 This is a flowchart illustrating the remote sensing image super-resolution method based on an adaptive Mamba-attention network as described in an embodiment of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0072] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0073] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0074] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0075] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0076] like Figure 1 As shown, this invention proposes a remote sensing image super-resolution system based on an adaptive Mamba-attention network, comprising: a feature extraction module for shallow feature extraction of the input low-resolution image; multiple cascaded adaptive state space blocks for processing the shallow features to obtain reconstructed features; each adaptive state space block includes a window multi-head attention module, a minimum window boundary enhancement module, an adaptive attention state space module, and a remote sensing frequency sensing modulation module, wherein: the window multi-head attention module performs local window mapping on the input features and performs long-range dependency modeling on the mapped features to obtain window attention output features; the minimum window boundary enhancement module... The window attention output features are subjected to boundary enhancement processing to obtain boundary enhancement features; the input features, boundary enhancement features and window attention output features are fused to obtain boundary enhancement intermediate features; the adaptive attention state space module processes the boundary enhancement intermediate features into spectral enhancement features, and performs dynamic cue weight modulation and pixel-level adaptive selection cue processing on the spectral enhancement features to obtain deep features. The deep features are input to the remote sensing frequency sensing modulation module for explicit reconstruction in the frequency domain to obtain the output features of the current adaptive state space block; the reconstruction module maps the reconstructed features to the target resolution space through sub-pixel rearrangement operations to obtain a high-resolution remote sensing image.

[0077] It should be noted that the adaptive state space block also includes a multilayer perceptron and a normalization layer. After normalization, the shallow features are processed sequentially by the window multi-head attention module and the minimum window boundary enhancement module to obtain boundary enhancement features. The boundary enhancement features are concatenated with the residuals of the shallow features to obtain feature B1. Feature B1 is processed sequentially by the normalization layer and the multilayer perceptron to obtain feature B2. The residuals of feature B1 and feature B2 are concatenated to obtain feature B3. Feature B3 is concatenated with the residuals of the shallow features to obtain feature B4. Feature B4 is normalized and then input to the adaptive attention state space module to obtain feature B5. The residuals of feature B4 and feature B5 are concatenated to obtain feature B6. Feature B6 is normalized and then input to the multilayer perceptron to obtain feature B7. The residuals of feature B6 and feature B7 are concatenated to obtain feature B8. The residuals of feature B4 and feature B8 are concatenated to obtain feature B9. Feature B9 is input to the remote sensing frequency sensing modulation module to obtain the output features of the current adaptive state space block.

[0078] In multiple cascaded adaptive state space blocks, the first adaptive state space block in the sorted sequence is connected to the feature extraction module, and the last one in the sorted sequence is connected to the reconstruction module.

[0079] Furthermore, this invention introduces a dynamic cue weight modulation mechanism into the Mamba state space backbone, extending static or coarse-grained cues to pixel-level content-adaptive cues, guiding the state space evolution process; simultaneously, it designs a channel-aware spectral attention module for remote sensing RGB imagery, explicitly modeling cross-channel spectral characteristics to improve spectral fidelity and ground feature differentiation capabilities; and it constructs a minimum window boundary enhancement module and a remote sensing frequency-aware modulation module to simultaneously enhance multi-scale structure modeling capabilities and boundary preservation capabilities in both spatial and frequency dimensions.

[0080] In some embodiments, the feature extraction module is a 3×3 convolution.

[0081] Let the input low-resolution remote sensing image be denoted as First, a shallow feature extraction function consisting of a 3×3 convolution layer is used. This method maps low-resolution remote sensing images from pixel space to feature space, obtaining shallow features. :

[0082] ;

[0083] in, Space dimensions and To maintain consistency, the number of channels is recorded as .

[0084] In some embodiments, in the minimum window boundary enhancement module:

[0085] By performing boundary enhancement on the input features at different scales, boundary features are obtained;

[0086] By performing residual connections between the boundary features and the input features, and then processing the results of the residual connections through convolution and activation, boundary enhancement features are obtained.

[0087] In the minimum window boundary enhancement module:

[0088] The input features are processed through three different branches to achieve boundary enhancement at different scales, specifically:

[0089] After processing the input features with channel compression and sigmoid activation, the first type of boundary enhancement response is obtained;

[0090] Branch 2 performs convolution operations on the input features at different scales and adds the results of the convolution operations at different scales to obtain the second type of boundary enhancement response;

[0091] Branch 3 performs global average pooling on the input features, and then performs convolution and sigmoid activation operations on the global average pooling result and the input features respectively, and adds them together to obtain the third type of boundary enhancement response;

[0092] Boundary features are obtained by fusing the first type of boundary enhancement response, the second type of boundary enhancement response, and the third type of boundary enhancement response along the channel dimension.

[0093] It should be noted that the minimum window boundary enhancement module takes the features at the window boundary as input and constructs three parallel branches: The first branch normalizes, compresses, and activates the features at the input feature boundary, then uses a depthwise separable convolution with a large kernel to highlight the continuous structure across the window, and obtains the first type of boundary enhancement response after Sigmoid activation; the second branch uses normalization, channel transformation, and nonlinear mapping, and then uses standard convolution to extract mesoscale edge information to obtain the second type of boundary enhancement response; the third branch performs global average pooling on the boundary region, then uses 1×1 convolution and activation to generate a gain factor biased towards the global contour, and combines it with the structure of the other branch to obtain the third type of boundary enhancement response. After the outputs of the three branches are fused in the channel dimension, they are added to the output of the window multi-head attention module in the form of residuals to obtain the boundary enhancement features, making the structure between windows more continuous and maintaining a smooth connection between building outlines and road edges across the window region.

[0094] Further, shallow features Several adaptive attention state space blocks are sequentially fed into the system for deep modeling. Within each block, normalization and multi-head self-attention of the window are first applied to model the long-range dependencies within the local window. After window self-attention, a minimum window boundary enhancement module is introduced to alleviate the boundary discontinuity problem caused by window partitioning.

[0095] like Figure 2 As shown, the detailed processing procedure of the minimum window boundary enhancement module is as follows: The input feature is processed through three different branches to obtain the first type of boundary enhancement response, the second type of boundary enhancement response, and the third type of boundary enhancement response. After normalizing the second type of boundary enhancement response, it is added to the first type of boundary enhancement response and the third type of boundary enhancement response to obtain feature C1. Feature C1 is connected to the input feature residual to obtain feature C2. Feature C2 is processed by 1×1 convolution, GELU activation function, and 1×1 convolution in sequence to obtain feature C3. Feature C3 is connected to the input feature residual to obtain the boundary enhancement feature.

[0096] Furthermore, the processing flow of branch one is as follows: the input features are sequentially processed by a 7×7 depthwise separable convolution, a GELU activation function, a 1×1 convolution, and a Sigmoid activation function to obtain the first type of boundary enhancement response; the processing flow of branch two is as follows: the input features are processed in two branches. The processing flow of one branch is as follows: the input features are sequentially processed by a 1×1 convolution, a GELU activation function, and a 1×1 convolution to obtain feature D1; the processing flow of the other branch is as follows: the input features are sequentially processed by a 3×3 convolution, a GELU activation function, and a 1×1 convolution to obtain feature D2. Adding feature D1 and feature D2 yields the second type of boundary enhancement response. The processing flow of branch three is as follows: the input features are processed in two branches. The processing flow of one branch is as follows: the input features are processed sequentially by global average pooling, 1×1 convolution, GELU activation function, 1×1 convolution, and Sigmoid activation function to obtain feature D3. The processing flow of the other branch is as follows: the input features are processed sequentially by 1×1 convolution, GELU activation function, 1×1 convolution, and Sigmoid activation function to obtain feature D4. Adding feature D3 and feature D4 yields the third type of boundary enhancement response.

[0097] like Figure 3 As shown, after completing window self-attention and minimum window boundary enhancement, this invention further explicitly models the spectral correlation between channels. Let the input features of a certain adaptive state space block be denoted as... , of which each For the first A two-dimensional feature map of each channel, where C is the total number of channels. Indicates spatial location The feature value of channel c is obtained. To explicitly model the spectral correlation between channels, this invention first obtains intermediate features through grouped convolution and 1×1 convolution. Then, a spectral attention weight map is generated using the Sigmoid activation function. It is then multiplied point-by-point with the input features to form spectral enhancement features. This process can be represented as:

[0098] ;

[0099] ;

[0100] ;

[0101] in, This represents the grouped convolution operator. Represents a 1×1 convolution. It is the Sigmoid activation function. In spatial location ,aisle The spectral weights on the spectrum, where A is the spectral weight. For spectral enhancement features in spatial location ,aisle The eigenvalues ​​on.

[0102] In obtaining spectral enhancement characteristics Subsequently, this invention implements a dynamic cue weight modulation mechanism within the same adaptive attention state space block. First, adaptive two-dimensional average pooling is used to modulate the weights in the spatial dimension. Perform global aggregation to obtain the global description vector. :

[0103]

[0104] in, Let H be the global description vector for channel C, H be the height of the feature map, and W be the width of the feature map.

[0105] Then let the global description vector The dynamic weight vector is generated by sequentially passing the weights through two 1×1 convolutions and a sigmoid activation. :

[0106] ;

[0107] in, This is the weight matrix corresponding to two 1×1 convolution layers. It is a non-linear activation function (such as GELU). Let be the number of cue vectors in the cue pool. Let the static cue pool be denoted as . Then, dynamic weights can be used to scale static prompts, resulting in a dynamic prompt pool, whose... A dynamic prompt for:

[0108] ;

[0109] To adaptively select cues at the pixel level, this invention constructs an enhanced routing network that enhances spectral features. Feature vector of each spatial location Encoding is performed. The routing vector is obtained through two layers of linear mapping and an activation function. And use the Gumbel-Softmax function to calculate the probability of selecting each cue vector at that position:

[0110] ;

[0111] ;

[0112] in, The weight matrix is ​​for a two-layer linear mapping. Indicates spatial location Choose the first option The weights of each dynamic cue vector, For the Gumbel-Softmax function. Finally, for the dynamic hint pool... The dynamic cue vector corresponding to each pixel is obtained by weighted summation based on pixel-level weights. :

[0113] ;

[0114] Two-dimensional features Expand by row or column to a length of input sequence At the same time, pixel-level prompts Simultaneously unfold into a prompt sequence This allows for the injection of cues at location into the adaptive attention state space equation, modulating state updates and outputs, thereby enabling refined adaptive modeling of different regions, textures, and spectral features in complex remote sensing scenarios.

[0115] The dynamic cue vector corresponding to each pixel The input is fed into the adaptive state-space equation for calculation, and the calculation results are output. After being processed sequentially by the SGN-fold module and the linear mapping layer, the output features are obtained.

[0116] The dynamic cue vector corresponding to each pixel Before inputting into the adaptive state-space equation, the dynamic cue vector corresponding to each pixel is... The input is processed by the SGN-unfold module, and the processing result is fed into the adaptive state-space equation. Both the SGN-unfold and SGN-fold modules were described in the paper "Mambairv2: Attentive state-space restoration" published at the Proceedings of the Computer Vision and Pattern Recognition Conference in 2025. The SGN-unfold module segments and flattens the 2D image feature map using a sliding window, transforming it into a series of vector sequences representing local regions. This sequence is then input into the selective state-space model, which dynamically adjusts its parameters based on the input and efficiently fuses global contextual information through a feedforward scanning method, thereby enhancing each sequence element. Finally, the processed enhanced sequence is reassembled back into the original 2D spatial structure through an aggregation operation, outputting the enhanced feature map.

[0117] Furthermore, the formula for calculating the adaptive state-space equation is as follows:

[0118] ;

[0119] ;

[0120] ;

[0121] In some embodiments, in the remote sensing frequency sensing modulation module:

[0122] Frequency decomposition of the input features yields low-frequency, mid-frequency, and high-frequency features;

[0123] The low-frequency, mid-frequency, and high-frequency features are respectively subjected to feature concatenation and feature averaging operations to obtain concatenated features and mean features respectively;

[0124] After the spliced ​​features are processed by channel attention and spatial attention, feature A1 is obtained;

[0125] The concatenated features are activated by Sigmoid to generate a gated mask. The gated mask and the mean features are then used for gated fusion to obtain the fused features.

[0126] Multiply feature A1, the fused feature, and the input feature to obtain the output feature.

[0127] It should be noted that this invention utilizes a remote sensing frequency sensing modulation module to explicitly reconstruct deep features in the frequency domain, such as... Figure 4As shown, this module first performs frequency decomposition on the input features, constructing smooth convolution branches, dilated convolution branches, and Laplacian convolution branches to extract features biased towards low-frequency background structures, mid-frequency local structures, and high-frequency edge textures, respectively, thus obtaining three frequency representations: low-frequency features, mid-frequency features, and high-frequency features. Subsequently, the mean feature is obtained by averaging each feature. Furthermore, the features are concatenated and then processed by a 1×1 convolution to obtain the concatenated feature. This concatenated feature undergoes channel attention and spatial attention operations to obtain feature A1. The mean feature is then processed by a 1×1 convolution and Sigmoid activation to generate a gated mask. Gated fusion is then performed on the different frequency branches to obtain the fused feature, enabling the model to adaptively balance the structural consistency of smooth regions and the detail sharpness of edge regions within a unified framework. To further enhance the ability to represent complex terrain features, feature A1, the fused feature, and the input feature are multiplied to obtain the output feature. Channel attention generates a channel attention map through global convergence and two layers of 1×1 convolutions with activation functions, emphasizing the channel responses that are more critical to remote sensing scenes. The specific operation is as follows: the input features for channel attention are processed sequentially through 1×1 convolution, GELU, 1×1 convolution, and a sigmoid activation function to obtain channel attention weights. These weights are then multiplied by the input features to obtain the channel attention. Spatial attention generates a spatial attention map through convolution and nonlinear mapping, highlighting regions with concentrated targets and rich textures in the image. The specific operation for spatial attention is as follows: the input features are processed sequentially through 1×1 convolution, ReLU, 1×1 convolution, and a sigmoid activation function to obtain spatial attention weights. These weights are then multiplied by the input features to obtain the spatial attention. The channel attention map and the spatial attention map work together on the frequency-fused features, enabling the output features to be adaptively enhanced simultaneously in both channel and spatial dimensions.

[0128] In some embodiments, the reconstruction module includes a convolution module and a subpixel rearrangement module, wherein:

[0129] After the convolution module performs convolution processing on the reconstructed features, it obtains intermediate features. The intermediate features are then added to the shallow features to obtain feature A2.

[0130] The subpixel rearrangement module maps feature A2 to the target resolution space to obtain a high-resolution remote sensing image.

[0131] It should be noted that the features processed by the remote sensing frequency sensing modulation module are fed into the upsampling reconstruction module. Through 3×3 convolution and sub-pixel rearrangement operations, the feature map is mapped to the target resolution space, further suppressing artifacts and correcting details to output a high-resolution remote sensing image. Through the close coordination of shallow feature extraction, window self-attention and minimum window boundary enhancement, channel-aware spectral attention and dynamic cue-driven adaptive attention state space modeling, and remote sensing frequency sensing modulation and upsampling reconstruction, this invention achieves high-quality super-resolution reconstruction of complex remote sensing scenes while ensuring a moderate number of parameters and computational complexity.

[0132] like Figure 5 As shown, this invention proposes a remote sensing image super-resolution method based on an adaptive Mamba-attention network, comprising:

[0133] S1: Obtain high-resolution images, perform downsampling degradation and preprocessing operations on the high-resolution images to obtain the training set;

[0134] Preprocessing specifically refers to pixel normalization preprocessing.

[0135] S2: Construct a remote sensing image super-resolution system based on an adaptive Mamba-attention network;

[0136] S3: Use the training set obtained in step S1 to train the remote sensing image super-resolution system constructed in step S2 to obtain the super-resolution model;

[0137] S4: Input the low-resolution image to be reconstructed into the super-resolution model trained in step S3 to obtain the reconstructed image.

[0138] During training, the L1 loss function is used as the core optimization objective, aiming to minimize the pixel-level difference between the predicted super-resolution output and the corresponding high-resolution target image, thereby driving the model to learn accurate image reconstruction. Training uses the Adam optimizer (β1=0.9, β2=0.99) with an initial learning rate of 1×10⁻⁶. -4 The MultiStepLR scheduler was used to halve the learning rate at iterations of 250k, 400k, 450k, and 475k to stabilize the convergence process. During training, a batch size of 4 was used, and 64×64 image patches were randomly cropped from low-resolution images as input. Data augmentation strategies such as random rotation and horizontal / vertical flipping were applied to improve the model's generalization ability. The entire training process was performed on a single NVIDIA RTX 3090 GPU, achieving efficient end-to-end super-resolution reconstruction without post-processing.

[0139] It should be noted that this method takes low-resolution remote sensing images as input. First, it constructs shallow features through convolution to extract basic texture and edge information. Then, it concatenates several adaptive attention state space blocks in the backbone, combining window self-attention with Mamba state space (which belongs to the adaptive state space module) modeling. This efficiently captures long-range dependencies while preserving fine texture locally, and enhances and mitigates structural breaks caused by window partitioning through minimum window boundary enhancement. To overcome the problems of insufficient static cues and spectral utilization, this invention introduces a dynamic cue weight modulation mechanism and channel-aware spectral attention during the state space update process. On the one hand, it applies differentiated reconstruction strategies to different regions using pixel-level adaptive cues; on the other hand, it explicitly models the spectral correlation between RGB channels to improve the spectral separability and color fidelity of land cover categories. Finally, a remote sensing frequency-aware modulation module collaboratively processes high-frequency details and low-frequency structures in the feature space, and combines sub-pixel rearrangement upsampling to generate high-resolution output, achieving high-quality reconstruction of complex remote sensing scenes with moderate computational overhead.

[0140] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0141] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A remote sensing image super-resolution system based on an adaptive Mamba-attention network, characterized in that: include: The feature extraction module performs shallow feature extraction on the input low-resolution image to obtain shallow features; Multiple cascaded adaptive state space blocks process shallow features to obtain reconstructed features; each adaptive state space block includes a multi-head window attention module, a minimum window boundary enhancement module, an adaptive attention state space module, and a remote sensing frequency sensing modulation module, wherein: The multi-head window attention module performs local window mapping on the input features and performs long-range dependency modeling on the mapped features to obtain window attention output features; the minimum window boundary enhancement module performs boundary enhancement processing on the window attention output features to obtain boundary enhancement features; the input features, boundary enhancement features and window attention output features are fused to obtain boundary enhancement intermediate features; the adaptive attention state space module processes the boundary enhancement intermediate features into spectral enhancement features, and performs dynamic cue weight modulation and pixel-level adaptive selection cue processing on the spectral enhancement features to obtain deep features; the deep features are input to the remote sensing frequency sensing modulation module for explicit reconstruction in the frequency domain to obtain the output features of the current adaptive state space block; The reconstruction module maps reconstructed features to the target resolution space through subpixel rearrangement operations to obtain high-resolution remote sensing images.

2. The remote sensing image super-resolution system based on adaptive Mamba-attention network according to claim 1, characterized in that: In the minimum window boundary enhancement module: By performing boundary enhancement on the input features at different scales, boundary features are obtained; By performing residual connections between the boundary features and the input features, and then processing the results of the residual connections through convolution and activation, boundary enhancement features are obtained.

3. The remote sensing image super-resolution system based on adaptive Mamba-attention network according to claim 2, characterized in that: In the minimum window boundary enhancement module: The input features are processed through three different branches to achieve boundary enhancement at different scales, specifically: After processing the input features with channel compression and sigmoid activation, the first type of boundary enhancement response is obtained; Branch 2 performs convolution operations on the input features at different scales and adds the results of the convolution operations at different scales to obtain the second type of boundary enhancement response; Branch 3 performs global average pooling on the input features, and then performs convolution and sigmoid activation operations on the global average pooling result and the input features respectively, and adds them together to obtain the third type of boundary enhancement response; Boundary features are obtained by fusing the first type of boundary enhancement response, the second type of boundary enhancement response, and the third type of boundary enhancement response along the channel dimension.

4. The remote sensing image super-resolution system based on adaptive Mamba-attention network according to claim 1, characterized in that: The specific processing procedure of the adaptive attention state space module is as follows: Channel-aware spectral attention processing is applied to the input feature F to obtain spectral enhancement features. : ； ； ； in, For grouped convolution operators, As an intermediate feature, For 1×1 convolution, Let A be the sigmoid activation function and A be the spectral attention weight map. c is the spatial location index, and 'c' is the channel. The spectral enhancement features are input into the enhancement routing network, which performs linear mapping and activation on the spectral enhancement features and calculates the probability of selecting each dynamic cue vector at each pixel location using the Gumbel-Softmax function. ； ； in, It is a non-linear activation function. For spectral enhancement features in The routing vector at that location, For spectral enhancement features in The feature vector at that location, and This is the weight matrix. For spectral enhancement features in Choose the first option The weights of each dynamic cue vector, To calculate the selection of the first The Gumbel-Softmax function is used to calculate the weights of the dynamic cue vectors. The spectral enhancement features are input into the dynamic prompting module, which then performs adaptive two-dimensional average pooling calculation on the spectral enhancement features to obtain the global description vector. ； in, Here, H is the global description vector for channel C, H is the height of the feature map, and W is the width of the feature map. Based on the global description vector, a dynamic weight vector is generated using the following formula: ； in, It is a non-linear activation function. and Both are weight matrices. This is the dynamic weight vector for channel C; Let the static prompt pool be By scaling the static cues using a dynamic weight vector, a dynamic cue pool is obtained. The expression for the k-th dynamic cue is: ； in, For the kth dynamic prompt, This is the kth static suggestion; For dynamic prompt pool The dynamic cue vector corresponding to each pixel is obtained by weighted summation based on pixel-level weights. : ； The dynamic cue vector corresponding to each pixel The input is fed into the adaptive state space equation for calculation, and the calculation results are processed sequentially through the SGN-fold module and the linear mapping layer to obtain the output features.

5. The remote sensing image super-resolution system based on adaptive Mamba-attention network according to claim 4, characterized in that: The formula for calculating the adaptive state-space equation is as follows: ； ； ； in, This is the discretized state transition matrix. The input projection matrix is ​​the discretized form. Let be the hidden state at time i. Let i be the hidden state at time i-1. Let be the input vector at time i. Let C be the output vector at time i, C be the output projection matrix, and P be the dynamic cue vector. and These are the discretized state-space parameters.

6. The remote sensing image super-resolution system based on adaptive Mamba-attention network according to claim 1, characterized in that: In the remote sensing frequency sensing modulation module: Frequency decomposition of the input features yields low-frequency, mid-frequency, and high-frequency features; The low-frequency, mid-frequency, and high-frequency features are respectively subjected to feature concatenation and feature averaging operations to obtain concatenated features and mean features respectively; After the spliced ​​features are processed by channel attention and spatial attention, feature A1 is obtained; The concatenated features are activated by Sigmoid to generate a gated mask. The gated mask and the mean features are then used for gated fusion to obtain the fused features. Multiply feature A1, the fused feature, and the input feature to obtain the output feature.

7. The remote sensing image super-resolution system based on adaptive Mamba-attention network according to claim 1, characterized in that: The reconstruction module includes a convolutional module and a subpixel rearrangement module, wherein: After the convolution module performs convolution processing on the reconstructed features, it obtains intermediate features. The intermediate features are then added to the shallow features to obtain feature A2. The subpixel rearrangement module maps feature A2 to the target resolution space to obtain a high-resolution remote sensing image.

8. A remote sensing image super-resolution method based on an adaptive Mamba-attention network, characterized in that: include: S1: Obtain high-resolution images, perform downsampling degradation and preprocessing operations on the high-resolution images to obtain the training set; S2: Construct a remote sensing image super-resolution system based on an adaptive Mamba-attention network as described in any one of claims 1 to 7; S3: Use the training set obtained in step S1 to train the remote sensing image super-resolution system constructed in step S2 to obtain the super-resolution model; S4: Input the low-resolution image to be reconstructed into the super-resolution model trained in step S3 to obtain the reconstructed image.

Images