Learable TransMama hybrid network for remote sensing image super-resolution

Through the LTMHN network fusion of the advantages of Mamba and Transformer, the contradiction between global modeling and computing efficiency in the super-resolution of remote sensing images is solved, efficient recovery of complex textures and multi-scale structures is achieved, and image reconstruction quality is improved.

CN120525718APending Publication Date: 2025-08-22BEIJING UNION UNIVERSITY
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202510632962.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-16
Publication Date
2025-08-22

AI Technical Summary

Technical Problem

There is a contradiction between global modeling and computing efficiency in the existing remote sensing image super-resolution method, and the problem of insufficient recovery of complex textures and multi-scale structures.

Method used

Based on the learningable TransMamba hybrid network (LTMHN), the dual-branch architecture integrates the state space model (Mamba) and self-attention mechanism (Transformer), and achieves a balance between efficient long-range dependency modeling and detail recovery.

Benefits of technology

It realizes efficient recovery of complex textures in remote sensing images and consistency of global structures, reduces computing overhead and improves image reconstruction quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120525718A_ABST
    Figure CN120525718A_ABST
Patent Text Reader

Abstract

The invention provides a dual-branch super-resolution method fusing the advantages of Mama and Transform, which realizes the super-resolution reconstruction work of a remote sensing image, and comprises the steps of preparing an AID data set and a training set and a test set of UCMerceed. The method comprises the specific steps that a provided data set is used for training a model, the training model is stored in a local folder, and if the training model is satisfied, the training model is stored as a satisfied training model which is called an optimal model; and then testing by using the optimal model to improve the resolution of the remote sensing image. According to the method, the advantages of state space modeling and a self-attention mechanism are fully fused, the calculation complexity is greatly reduced while the high-resolution reconstruction quality is kept, the LMSOCA model is used for learning information fusion strategies under different scales, the detail recovery capability of edges and complex areas is remarkably improved, and the expression capability of the model is enhanced. In addition, feature dynamic fusion is achieved through a double-branch structure and self-adaptive parameters, efficient cooperation of local details and global structure information is achieved, a channel polarization dynamic attention module is designed, and on the basis of serial space attention and channel attention branches, a channel polarization dynamic attention module is designed. And a self-adaptive channel enhancement mechanism is introduced to effectively aggregate space and channel features and suppress channel redundancy, so that the expression and generalization ability of the model to complex remote sensing textures is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a dual-branch super-resolution method that integrates the advantages of Mamba and Transformer. Background Art

[0002] Image resolution is an important performance metric for assessing the richness of image detail and reflects the imaging system's ability to restore target details. Compared to low-resolution images, high-resolution images have higher pixel density, visual credibility, and richer texture details. Super-resolution (SR) technology aims to reconstruct high-resolution images (HR) with rich details and clear structure from limited low-resolution images (LR). Remote sensing image super-resolution (RSR) utilizes computer vision and deep learning techniques to restore or reconstruct low-resolution remote sensing images into high-resolution images. Super-resolution reconstruction can significantly improve the accuracy and robustness of remote sensing images in tasks such as object recognition, target detection, land use classification, and environmental monitoring, and has important application value. Remarkable progress has been made in remote sensing image super-resolution (RSS) against the backdrop of the rapid development of deep learning [1-3], primarily including methods based on convolutional neural networks (CNNs), Transformers, and Mamba. Due to limitations in spatial resolution, sensor performance, and imaging environments, remote sensing images often struggle to meet the requirements of high-precision applications.

[0003] Super-resolution methods based on convolutional neural networks (CNNs) are one of the mainstream methods in the field of super-resolution in recent years. SRCNN proposed by Dong et al. [4] is a classic CNN-based super-resolution method. It regards the super-resolution problem as a mapping from low-resolution images to high-resolution images, and learns this mapping relationship through a multi-layer convolutional neural network. It directly learns the mapping from low-resolution images to high-resolution images in an end-to-end manner, achieving good results. Due to the remarkable achievements of SRCNN, subsequent developments have enriched the structure of CNN by introducing advanced modules (skip connections [5], residual networks [6], a large number of convolutional layers [7], and attention mechanisms [8-10], etc.), continuously improving the quality of super-resolution of remote sensing images. In addition, the DFMNet network proposed by Xiao et al.

[11] achieves deep fusion of multi-scale features through symmetric dual-path feature reuse and cross-attention mechanism, effectively improving the reconstruction quality of high-frequency details of remote sensing images. However, since CNN focuses on local information, its ability to model key long-distance dependencies is limited, which in turn makes the overall performance improvement face a bottleneck.

[0004] Transformer initially made a breakthrough in the field of natural language processing. Its core is the self-attention mechanism, which can efficiently model long-distance dependencies. Introducing it into the image field can effectively overcome the global context modeling problem that CNN is difficult to handle, thereby improving the performance of super-resolution. TransENet proposed by Lei et al.

[12] is a multi-stage enhancement structure based on Transformer. It can be combined with the traditional SR framework to fuse multi-scale high / low dimensional features. In this structure, the encoder aims to embed multi-level features into the feature extraction part, while the decoder is used to fuse these encoded embeddings. Uformer proposed by Wang et al.

[13] uses Transformer modules to construct a hierarchical encoder-decoder network. It uses non-overlapping window-based self-attention to reduce the amount of computation. At the same time, it uses deep convolution in the forward network to further improve its ability to capture local context. ARFFT proposed by Zhu et al.

[14] is an effective image SR model based on the attention-shrinkable frequency Transformer and the proposed spatial frequency fusion module. The spatial frequency fusion module is used to further expand the receptive field. CFAT, proposed by Abhisek Ray et al.

[15] , effectively alleviates the boundary distortion problem by introducing a triangular-rectangular combination window and a composite attention mechanism, significantly improving the modeling capability and reconstruction performance of image super-resolution. However, the Transformer model usually has a large model size and number of parameters, especially when processing high-resolution images, requiring a large amount of computing resources for training and inference.

[0005] Models with Mamba as the network backbone have achieved good results in image restoration tasks. The development of Mamba provides a potential solution to the contradiction between global receptive field and computational efficiency in image restoration. Zhao et al.

[32] proposed a spatial-spectral interactive fusion network (SSRFN) that combines Mamba and CNN to efficiently reconstruct high-quality fused images using HSI and MSI data, while improving the reconstruction quality and taking into account both local and global feature modeling. Xiao et al.

[33] proposed a Mamba architecture FMSR that combines frequency domain information and designed a Frequency-assistedMamba framework, which showed superior performance while maintaining low computational cost and memory consumption. Zhao et al.

[34] proposed a ResidualMambaBlock that combines state space model and residual connection, which achieved efficient long-range dependency modeling with linear complexity in image super-resolution. Mamba-based models have great potential in future super-resolution tasks, but they are still in the initial stage and need to be explored continuously.

[0006] Despite its powerful global modeling capabilities, the Transformer model suffers from insufficient scalability and excessive computational overhead when handling long-term contextual dependencies. Meanwhile, while the Mamba model exhibits excellent long-term context modeling efficiency with linear complexity, it lacks the ability to capture detailed information in high-frequency information-rich remote sensing images, making it difficult to achieve optimal performance independently. This paper proposes a new remote sensing image super-resolution architecture, LTMHN, that combines the advantages of Transformer and Mamba. By effectively capturing the relationship between global structure and local details in remote sensing images through Transformer, and efficiently and stably modeling long-term context with Mamba, the two models complement each other to create a remote sensing image super-resolution model that is both highly accurate and computationally efficient.

[0007] References:

[0008] [1] R.Zhi, X.Fan, and J.Shi, "Mambaformersr: A lightweight model for remote-sensing image super-resolution," IEEE Geoscience and Remote Sensing Letters, 2024.

[0009] [2] G. Zhao, H. Wu, D. Luo, X. Ou, and Y. Zhang, "Spatial spectral interaction super-resolution cnn-mamba network for fusion of satellite hyperspectral and multispectral image," IEEE Journal of Selected Topics in Applied EarthObservations and Remote Sensing, 2024.

[0010] [3]Y.Wang,Z.Shao,T.Lu,X.Huang,J.Wang,Z.Zhang,and X.Zuo,

[0011] "Lightweight remote sensing super-resolution with multi-scale graphattention network," Pattern Recognition, vol.160, p.111178, 2025.

[0012] [4]C.Dong,C.C.Loy,K.He and X.Tang,"Image Super-Resolution Using DeepConvolutional Networks,"in IEEE Transactions on Pattern Analysis and MachineIntelligence,vol.38,no.2,pp.295-307,1Feb.2016,doi:

[0013] 10.1109 / TPAMI.2015.2439281.

[0014] [5]Dong,C.,Change Loy,C.,and Tang,X.,“Accelerating the Super-Resolution Convolutional Neural Network”,*arXiv e-prints*,2016.doi:10.48550.

[0015] [6]Ledig,C.,“Photo-Realistic Single Image Super-Resolution Using aGenerative Adversarial Network”,arXiv e-prints,2016.doi:10.48550.

[0016] [7]Kim,J.,Lee,J.K.,and Lee,K.M.,“Accurate Image Super-ResolutionUsing Very Deep Convolutional Networks”,arXiv e-prints,2015.doi:10.48550.

[0017] [8]Zhou Z,Chao J,Gong J,et al.Enhancing real-time super resolutionwith partial convolution and efficient variance attention[C] / / Proceedings ofthe 31st ACM International Conference on Multimedia.2023:5348-5357.

[0018] [9]Y.Zhang,K.Li,K.Li,L.Wang,B.Zhong,and Y.Fu,“Image superresolutionusing very deep residual channel attention networks,”in Proceedings of theEuropean conference on computer vision(ECCV),2018,pp.286–301.

[0019]

[10] B.Niu,W.Wen,W.Ren,X.Zhang,L.Yang,S.Wang,K.Zhang,X.Cao,and H.Shen,“Single image super-resolution via a holistic attention network,”

[0020] in Computer Vision–ECCV 2020:16th European Conference,Glasgow,UK,

[0021] August 23–28,2020,Proceedings,Part XII 16.Springer,2020,pp.191–207.

[0022]

[11] Xiao H,Chen X,Luo L,et al.A dual-path feature reuse multi-scalenetwork for remote sensing image super-resolution[J].The Journal ofSupercomputing,

[0023] 2025,81(1):1-28.

[0024]

[12] S.Lei,Z.Shi and W.Mo,"Transformer-Based Multistage Enhancementfor Remote Sensing Image Super-Resolution,"in IEEE Transactions on Geoscienceand Remote Sensing,vol.60,pp.1-11,2022,Art no.5615611,doi:

[0025] 10.1109 / TGRS.2021.3136190.

[0026]

[13] Wang,Z.,Cun,X.,Bao,J.,Zhou,W.,Liu,J.,and Li,H.,“Uformer:A GeneralU-Shaped Transformer for Image Restoration”,arXiv e-prints,2021.

[0027] doi:10.48550.

[0028]

[14] Q.Zhu,P.Li and Q.Li,"Attention Retractable Frequency FusionTransformer for Image Super Resolution,"2023 IEEE / CVF Conference on ComputerVision and Pattern Recognition Workshops(CVPRW),Vancouver,

[0029] BC,Canada,2023,pp.1756-1763,doi:10.1109.

[0030]

[15] Ray A,Kumar G,Kolekar M H.Cfat:Unleashing triangular windows forimage super-resolution[C] / / Proceedings of the IEEE / CVF conference on computervision and pattern recognition.2024:26120-26129.

[0031]

[16] Ray A,Kumar G,Kolekar M H.Cfat:Unleashing triangular windows forimage super-resolution[C] / / Proceedings of the IEEE / CVF conference on computervision and pattern recognition.2024:26120-26129.

[0032]

[17] Y. Xiao, Q. Yuan, K. Jiang, Y. Chen, Q. Zhang, and C.-W. Lin,

[0033] "Frequency-assisted mamba for remote sensing image super-resolution," IEEE Transactions on Multimedia, 2024.

[0034]

[18] Zhao G,Wang F.Mamba-Based Residual Network for Remote SensingImage Super-Resolution[C] / / 20247th International Conference on PatternRecognition andArtificial Intelligence(PRAI).IEEE,2024:316-321. Summary of the Invention

[0035] To address the contradiction between global modeling and computational efficiency in existing remote sensing image super-resolution methods, as well as the problem of insufficient recovery of complex textures and multi-scale structures, this paper proposes a remote sensing image super-resolution method based on a learnable TransMamba hybrid network (LTMHN). Through a dual-branch architecture, it combines the advantages of a state-space model (Mamba) and a self-attention mechanism (Transformer), achieving a balance between efficient long-range dependency modeling and detail recovery.

[0036] The present invention proposes a remote sensing image super-resolution reconstruction method based on a learnable TransMamba hybrid network (LTMHN), which is characterized by comprising the following steps:

[0037] Step 1: Use the images in the training set to perform LTMHN network training to generate a training model;

[0038] Step 2: Save the training model to a local folder, use the images in the test set to test the effect of the training model, and if satisfactory, save the training model as a satisfactory training model;

[0039] Step 3: Save the satisfactory training model to a local folder, and use the satisfactory training model to test remote sensing image reconstruction.

[0040] Preferably, the deep feature extraction module of the LTMHN model is composed of a stack of N learnable TransMamba groups (LTMGs), each LTMG containing a learnable TransMamba block (LTMB) and a learnable multi-scale overlapping cross attention block (LMSOCAB).

[0041] In any of the above solutions, preferably, the encoder module is used to extract a feature map of the blurred image and promote the feature map to a high-dimensional space to process feature information.

[0042] Preferably, in any of the above schemes, the dynamic attention module (DAM) of the LTMB module includes a channel polarization dynamic attention (CPDA) mechanism.

[0043] In any of the above solutions, preferably, the LMSOCAB module realizes multi-scale feature interaction by calculating cross-window attention in windows of different scales and fusing multi-scale responses by averaging.

[0044] In any of the above schemes, preferably, the visual state space module (VSSM) includes linear projection and depth convolution of the input features, modeling long-range dependencies through two-dimensional selective scanning (2D-SSM); element-by-element multiplication of the state space output and the activated feature branch, and finally fusion through a linear layer.

[0045] In any of the above solutions, preferably, the training process uses an Adam optimizer.

[0046] In any of the above solutions, preferably, the training process is performed on the AID dataset for 500,000 iterations and on the UCMerced dataset for 250,000 iterations.

[0047] In any of the above solutions, preferably, the reconstruction module generates a high-resolution image by sequentially applying a convolution layer, a Pixel-Shuffle upsampling layer and a final convolution layer to the fused features.

[0048] LTMHN stands for Learnable TransMamba Hybrid Network. It achieves super-resolution reconstruction of remote sensing images, efficiently recovering complex textures (such as vegetation edges and building outlines) while maintaining global structural consistency. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Flowchart of a preferred embodiment of the dual-branch super-resolution method according to the present invention Figure 2 The overall framework of the dual-branch super-resolution method LTMHN according to the present invention.

[0050] Figure 3 Window partitioning for learnable multi-scale overlapping window attention according to the present invention.

[0051] Figure 4 Assign values ​​to the channels after passing through the PSA module and the ACEM module in the LTMHN model according to the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0053] Example 1

[0054] like Figure 1 As shown, step 100 is performed to prepare the remote sensing super-resolution datasets UCMerced and AID. The low-resolution images (LR) and the corresponding high-resolution images (HR) are divided into training and test sets. The input images are uniformly scaled to 128×128 pixels and normalized (pixel value range [0,1]). The data augmentation strategy includes random horizontal flipping and 90° rotation.

[0055] Execute step 110 to perform LTMHN network training using the images in the training set to generate a training model.

[0056] Execute step 120. After training is completed, save the training model to a local folder and use the test set to evaluate the model performance. If satisfactory, save the training model as a satisfactory training model.

[0057] Execute step 130 to save the satisfactory training model to a local folder, input a low-resolution image (such as farmland or urban area), and output a high-resolution reconstructed image.

[0058] Example 2

[0059] The present invention is a dual-branch super-resolution method that combines the advantages of Mamba and Transformer. The network is named "Learnable TransMamba Hybrid Network for Remote Sensing Image Super-Resolution" (LTMHN), a trainable end-to-end remote sensing resolution reconstruction network.

[0060] The contents of LTMHN include: the LTMHN network includes a shallow feature extraction module, a deep feature extraction module, a learnable TransMamba block, a learnable multi-scale overlapping cross attention block and a reconstruction module.

[0061] Steps to use LTMHN (implemented in Python programming language):

[0062] 1. Prepare the remote sensing super-resolution datasets UCMerced and AID, divide the low-resolution images (LR) and the corresponding high-resolution images (HR) into training and test sets. Set the image format to ".jpg" or ".png";

[0063] 2. Start the training script train_ltmhn.py to train the network. Use the Adam optimizer with a batch size of 4 for 500,000 iterations on the AID dataset and 250,000 iterations on the UCMerced dataset. Introduce an exponential moving average (EMA) smoothing parameter to stabilize the training process.

[0064] 3. Save the trained model to a local folder and use the "validate.py" file and the test set to test the effectiveness of the network training model. If PSNR ≥ 32.90 dB and SSIM ≥ 0.848 (×2 upsampling), save the model as ltmhn_final.pth. If it does not meet the standards, you can continue fine-tuning by loading the checkpoint.

[0065] 4. Save the satisfactory training model to a local folder and use "test.py" to test the remote sensing image reconstruction effect.

[0066] Example 3

[0067] Multi-scale textures (such as dense vegetation and edges of buildings) are common in remote sensing images, and traditional methods are prone to blurring or artifacts. Subsequently developed reconstruction-based methods use optimization models to restore structural information and enhance image mechanisms to mine areas with rich spatial and qualitative details, but this significantly increases computational overhead. In recent years, super-resolution methods based on convolutional neural networks (CNNs) have made significant progress in detail recovery by leveraging the powerful feature extraction capabilities of deep learning. However, due to the inherent local receptive field of CNNs, these methods find it difficult to effectively capture global contextual information. To address these challenges, a two-branch super-resolution method called LTMHN that combines the advantages of Mamba and Transformer is proposed. LTMHN addresses this through the following strategies:

[0068] Dynamic multi-scale modeling: The LMSOCAB module dynamically adjusts the window size through a learnable factor, such as Figure 3 As shown, small windows capture fine-grained edges, while large windows model macrostructures;

[0069] Channel redundancy suppression: The ACEM module reduces the activation intensity of redundant channels by 47% through global average pooling and fully connected layers, focusing on key feature channels (such as edges and corners);

[0070] Long-range dependency modeling: The VSSM branch uses 2D-SSM to model the global structure with linear complexity (O(L)), reducing the computational overhead by 93% compared to Transformer (O(L2)).

[0071] like Figure 2As shown, the overall network architecture is an end-to-end network designed specifically for remote sensing image super-resolution. Its pipeline can be divided into three stages: shallow feature extraction, deep feature extraction, and high-resolution reconstruction. The shallow feature extraction layer rapidly extracts low-level features from the input low-resolution remote sensing image through a 3×3 convolutional layer, which serves as the initial features for subsequent processing. The deep feature extraction layer consists of multiple stacked learnable TransMamba groups (LTMGs). Each LTMG contains a learnable TransMamba block (LTMB). This layer uses a channel-polarized dynamic attention (CPDA) mechanism to filter key regions in the spatial and channel dimensions and suppress redundant features. A Mamba-based two-dimensional selective scan (2D-SSM) models global structure with linear complexity, reducing computational overhead. The outputs of the two branches are dynamically fused using learnable weights to balance local details with global information, achieving adaptive fusion. The model also dynamically adjusts the size of multi-scale windows and enhances the model's ability to recover multi-scale textures through cross-window attention interactions. High-resolution reconstruction adds deep features to shallow features, preserving underlying details while integrating deep semantic information; through the Pixel-Shuffle operation, it gradually upsamples and ultimately generates a high-resolution image, restoring clear texture and structure.

[0072] The work of each module is as follows:

[0073] 1. Learnable TransMamba Blocks

[0074] The structure of Learnable TransMamba Block (LTMB) is as follows Figure 2 As shown in Figure 2, LTMB, the core component of LTMG, consists of two parallel branches, each of which is composed of multiple cascaded modules of different types. The upper branch contains a series of dynamic attention modules (DAMs), while the lower branch contains a series of visual state space modules (VSSMs).

[0075] 2. Dynamic Attention Module

[0076] 1) Polarized Self-Attention Mechanism: To compensate for the loss of high-resolution information caused by pooling and downsampling in traditional neural networks, the PSA module retains the highest internal resolution among existing attention mechanisms. While maintaining low computational overhead, it effectively strengthens the modeling of long-range dependencies on high-resolution inputs, significantly improving the performance of pixel-level regression tasks.

[0077] 2) Adaptive Channel Enhancement Module: Different channels encode different amounts of information, and dynamic attention is implemented with channel polarization. This module not only adaptively assigns different attention weights to each channel through the Adaptive Channel Attention Group (ACAG) module, thereby highlighting important feature channels and suppressing less relevant or redundant channels, but also encodes multi-scale and structured information. By combining multiple attention mechanisms, DAM significantly improves the performance and robustness of pixel-level tasks. This improved performance and robustness enables it to learn optimal channel combinations and enhance its ability to capture key features. In practice, ACAG obtains global channel statistics through global average pooling (GAP). It then passes through two fully connected layers, a GELU activation function, dropout, a residual connection, and finally a sigmoid function to generate per-channel weighting coefficients. These coefficients are then multiplied by the original feature map channel by channel to re-rank the channel importance. Through this mechanism, the network can better focus on the channels most relevant to the task, significantly improving performance and generalization.

[0078] 3) Channel-polarized dynamic attention: This study introduces the polarized self-attention (PSA) module, whose lightweight channel attention mechanism is still limited in capacity. Figure 4 As shown in (a), after PSA, the channel activation values ​​are roughly in the range of [0.07, 0.13], indicating that there is significant channel redundancy. In order to further enhance the model's attention to key channels in remote sensing images, we introduced an adaptive channel enhancement module (ACEM) outside the PSA module. First, the global channel weight is applied to the input feature X through channel attention; then, the adaptive channel attention gating (ACAG) submodule adaptively adjusts and selectively suppresses the preliminary channel weight to achieve nonlinear channel reconstruction. Figure 4 As shown in (b), after ACEM, the channel activation values ​​shift to approximately [0.48, 0.53], showing higher amplitudes and tighter distributions. Therefore, ACEM, as a compensatory enhancement to PSA, significantly improves the network's sensitivity to fine structural details in remote sensing images (such as weak edges, vegetation textures, and building boundaries).

[0079] 3. Visual State Space Module

[0080] By leveraging an efficient state transfer mechanism, SSM can effectively capture long-range dependencies in image data, enabling a deep understanding of global image structure while maintaining computational efficiency. In the context of image super-resolution, Vmamba introduced the Visual State Space Module (VSSM), which leverages the linear complexity and long-range modeling advantages of SSM to more accurately represent the deep structural information of the image. Specifically, VSSM consists of two parallel branches. The first branch processes the input feature X through a series of operations such as linear projection, depthwise convolution, SiLU activation, two-dimensional selective sweep (2D-SSM) and LayerNorm, followed by residual channel polarization dynamic attention (CPCA) to obtain X1. The second branch applies linear projection to the input feature, followed by SiLU activation, to obtain X2. Finally, the outputs X1 and X2 of the two branches are fused through element-wise multiplication and passed through a linear layer to produce the final output of the module.

[0081] 4. Learnable Multi-Scale Overlapping Cross-Attention

[0082] In the remote sensing image super-resolution task, modeling local spatial relationships and contextual information is crucial for recovering fine textures. To this end, we first adopt the Overlapping Cross-Attention (OCA) mechanism, which uses fixed-size overlapping windows to establish direct connections across windows, thereby enhancing the receptive field and local context modeling. To further improve the multi-scale representation capability of the model, we propose a Learnable Multi-Scale Overlapping Cross Attention (LMSOCA) module. The core idea is to introduce a learnable multi-scale receptive field in the attention calculation, enhance fine-grained feature modeling and channel importance, and thus achieve global context fusion of detail information from small to large scales. Different from the fixed window overlap of OCA, in order to dynamically adjust the size of each local feature region, the optimal receptive field is automatically learned.

Claims

1. A method for restoring or reconstructing a low-resolution remote sensing image into a high-resolution image using the LTMHN model. The method is characterized in that: The following steps are involved: Step 1: Prepare remote sensing super-resolution training datasets (UCMerced and AID) and preprocess the low-resolution images in the training datasets; Step 2: Constructing the LTMHN model, which includes a shallow feature extraction module, a deep feature extraction module composed of multiple learnable TransMamba groups (LTMGs), and a reconstruction module; Step 3: Perform end-to-end training on the LTMHN model using the training set to generate a training model; Step 4: Save the trained model locally and use low-resolution images in the test set to verify the model performance. If the reconstruction effect meets the preset indicators, save it as the final model; Step 5: Deploy the final model to the remote sensing image processing system, input the low-resolution remote sensing image, and output the high-resolution reconstruction result.

2. The remote sensing image super-resolution reconstruction method according to claim 1, wherein: The deep feature extraction module of the LTMHN model is composed of N stacked learnable TransMamba groups (LTMGs).

3. The remote sensing image super-resolution reconstruction method according to claim 2, wherein: The dynamic attention module (DAM) of the LTMB module includes a channel polarization dynamic attention (CPDA) mechanism, which is implemented by the following steps: (a) Generate attention weights in spatial and channel dimensions based on polarized self-attention (PSA); (b) Adaptive Channel Enhancement Module (ACEM) is introduced to dynamically adjust channel importance through global average pooling and fully connected layers to suppress redundant features.

4. The remote sensing image super-resolution reconstruction method according to claim 3, wherein: The visual state space module (VSSM) contains the following operations: (a) Linear projection and depthwise convolution are performed on the input features to model long-range dependencies through two-dimensional selective scanning (2D-SSM); (b) The state space output is element-wise multiplied with the activated feature branch and finally fused through a linear layer.

5. The remote sensing image super-resolution reconstruction method according to claim 4, wherein: The LMSOCAB module implements multi-scale feature interaction through the following steps: (a) Define the learnable scaling factor {γ i }, dynamically adjust the local window size; (b) Cross-window attention is calculated within windows of different scales, and multi-scale responses are fused by averaging.

6. The remote sensing image super-resolution reconstruction method according to claim 5, characterized in that: The training process adopts the following strategy: (a) Using Adam optimizer, the initial learning rate is 2×10 -4 , and introduce the exponential moving average (EMA) smoothing parameter ema_decay=0.999; (b) For different upsampling factors (×2, ×3, ×4), we train the model for 500,000 iterations on the AID dataset and 250,000 iterations on the UCMerced dataset.

7. The remote sensing image super-resolution reconstruction method according to claim 6, wherein: The reconstruction module generates a high-resolution image by the following steps: (a) The deep feature F D Added to the shallow feature F0, the fusion feature F is obtained Rec ; (b) For F Rec Apply the convolution layer, Pixel-Shuffle upsampling layer and final convolution layer in sequence to output a high-resolution image I HQ .

Citation Information

Cited By

  • Image super-resolution method and device based on bidirectional focusing enhancement

    CN120912438A

  • Remote sensing image super-resolution system and method based on adaptive Mamba-attention network

    CN121685267A

  • Remote sensing image super-resolution system and method based on adaptive mamba-attention network

    CN121685267B