Dual-domain fusion panchromatic sharpening method and system based on wavelet transform and Mama

Abstract

The invention discloses a dual-domain fusion panchromatic sharpening method and system based on wavelet transform and Mama, and relates to the technical field of image processing, and the method comprises the steps: employing two-stage wavelet decomposition at a frequency domain branch to obtain a low-frequency main body and high-frequency details for injection; a spatial Mama path and a spectral Mama path are arranged in a spatial domain branch, and spatial-spectral information deep fusion is realized through interaction of Mama; and in cooperation with progressive fusion up-sampling and channel attention reconstruction, a high-resolution multispectral image is finally output. The system comprises a data alignment module, a feature extraction module, a frequency domain decomposition module, a Mama interaction module, a fusion up-sampling module, a reconstruction module and the like. On QuickBird and IKONOS data sets, the method provided by the invention is superior to a comparison method in subjective vision and objective indexes such as MS-SSIM, PSNR and SAM, and is high in reasoning efficiency and friendly in deployment.

Description

Technical Field

[0001] This invention belongs to the field of image processing, specifically relating to a dual-domain fusion full-color sharpening method and system based on wavelet transform and Mamba. Background Technology

[0002] With the rapid development of satellite remote sensing sensors and Earth observation technologies, multispectral remote sensing imagery has been widely used in scenarios such as land natural resource surveys, marine environmental monitoring, and meteorological disaster early warning. Spatial resolution and spectral resolution, as two key attributes of imagery, directly affect interpretation accuracy and subsequent applications. However, due to limitations such as imaging mechanisms and payload design, a single sensor cannot simultaneously acquire high spatial and high spectral resolution data. Therefore, fusing panchromatic (PAN) and high-resolution LRMS (LRMS) images to reconstruct high-resolution multispectral (HRMS) images has become an important approach, known as panchromatic sharpening. Its core is to enhance the spatial details of LRMS using high spatial resolution PAN while maintaining its spectral consistency as much as possible.

[0003] Current full-color sharpening methods mainly fall into two categories: traditional methods (such as component substitution CS: IHS, GS, PCA; and multi-resolution analysis MRA: Laplacian pyramid, wavelet transform, etc.) inject spatial details with low computational cost, but are prone to introducing spectral distortion or detail loss; model-based methods (such as Bayesian, sparse representation, variational optimization) model fusion as a reconstruction problem, which is theoretically controllable but difficult to characterize complex nonlinear relationships and complex to implement in engineering; deep learning methods have significantly improved fusion quality, from early CNN / residual networks to further improvements with the introduction of high-frequency supervision and multi-scale convolution, and then to GANs that improve realism and detail through adversarial training, while taking into account both spectral and spatial discrimination; to overcome the limited receptive field of CNNs, Transformer relies on self-attention to achieve global modeling, but at the cost of high computational cost; in recent years, Mamba based on the state-space model (SSM) has characterized long-range dependencies with linear complexity, becoming a powerful alternative that balances global modeling and efficiency, and has begun to be used for joint spatial-spectral feature extraction and cross-modal interaction.

[0004] Chinese invention patent application CN116402700A proposes a panchromatic sharpening method for remote sensing images based on multi-scale dual-domain information fusion technology. It focuses on image acquisition and preprocessing, model construction and training, and result generation, aiming to alleviate the problems of artifacts, color distortion, and poor generalization, and improve fusion accuracy and color performance. Chinese invention patent CN119671864B proposes a dual-domain aligned non-paired hyperspectral and SAR image spatiotemporal spectral fusion method. It achieves cross-modal alignment and improves spatiotemporal spectral fusion accuracy through steps such as radiation distribution approximation and spatial geometry adjustment. Chinese invention patent application CN117437138A proposes a remote sensing image fusion method and system based on a dual-domain dynamic fusion network. It extracts high-frequency and spatial-spectral features in the high-pass and spatial domains respectively, fuses and reconstructs them, and then adds them to low-resolution multispectral data to obtain HRMS, thereby improving spatial details and fusion effects. Chinese invention patent CN119515706B proposes a KAN-based spatial-frequency dual-domain implicit guided sampling remote sensing image fusion method. It employs a pure KAN architecture to perform implicit sampling and feature fusion in the spatial and frequency domains, achieving lightweight spatial-frequency dual-domain fusion. These technologies indicate that the "dual-domain / multi-domain" approach and strategies such as cross-modal alignment and upsampling reconstruction have become trends, but a unified framework for efficient global modeling and spatial-spectral interaction is still evolving.

[0005] Existing technologies still have the following shortcomings: traditional CS / MRA methods often sacrifice spectral consistency or introduce artifacts when enhancing spatial details; model-based methods are difficult to construct accurate priors and robust optimizations in real and complex scenarios; deep CNN methods are limited by local receptive fields and cannot fully model long-distance spatial-spectral dependencies; although Transformer has global modeling capabilities, it has high computational and memory overhead and high deployment costs, and its inference efficiency is limited when facing high-resolution remote sensing images; most existing dual-domain fusion methods are only coupled at the filtering / pyramid or simple frequency domain injection level, which is insufficient for modeling the interaction between low / high frequency subbands and spatial-spectral representations, and the coordination between cross-domain / cross-modal detail compensation and reconstruction stages is often not tight enough. Summary of the Invention

[0006] To address the problems of spectral distortion, insufficient spatial detail, low global modeling efficiency, and poor robustness in complex scenes in existing pancolor sharpening methods, this invention proposes a dual-domain fusion pancolor sharpening method and system based on wavelet transform and Mamba. By constructing a two-level wavelet decomposition in the frequency domain to obtain low-frequency main elements and high-frequency details, and introducing spatial Mamba and spectral Mamba in the spatial domain, long-range dependency modeling of spatial and spectral features and cross-modal feature fusion are achieved through interactive Mamba. High-frequency detail residual injection and upsampling fusion modules are combined for collaborative reconstruction, and channel attention reconstruction and upsampling LRMS global residual output ensure spectral consistency and detail sharpness. This solves the problems of distortion in traditional CS / MRA methods, limited receptive field of CNNs, high computational overhead of Transformers, and loose coupling in existing dual-domain methods. The dual-domain fusion pancolor sharpening method based on wavelet transform and Mamba described in this invention includes the following steps: S1: Acquire the low-resolution multispectral image LRMS and the corresponding panchromatic image PAN, perform bicubic interpolation upsampling on LRMS to obtain an upsampled image XLRMS↑ with the same resolution as PAN, and retain the original PAN; S2: Construct frequency domain branch and feature extraction. Extract features from XLRMS↑ and PAN in step S1 through a feature extraction network, and concatenate them along the channel dimension to form frequency domain input features; the specific calculation formula is as follows: (1) in, This indicates the input LRMS image or PAN image. This represents a two-dimensional convolution operation. This means dividing the input equally along the channel dimension into and Two parts, This represents the LeakyReLU activation function. This indicates the instance normalization operation. This indicates a concatenation operation on the channel dimension of features. Indicates feature extraction operation; S3: Perform wavelet decomposition on the frequency domain input features from step S2, using discrete wavelet transform for two consecutive levels of decomposition, successively obtaining the first-level low-frequency main component and the first-level high-frequency detail component, and the second-level low-frequency main component and the second-level high-frequency detail component; the specific calculation formula is as follows: (2) Among them, superscript Indicates an upsampling operation. Represents wavelet transform, express Feature maps of different sub-bands after wavelet decomposition; specifically, express Low-frequency sub-band feature map after wavelet decomposition. , and They represent High-frequency sub-band feature maps in the horizontal, vertical, and diagonal directions after wavelet decomposition; S4: Construct a spatial domain branch aligned with the input, downsample the PAN to obtain XPAN↓ with the same resolution as the original LRMS; input the LRMS and XPAN↓ into the feature extraction network respectively to obtain spectral features and spatial structure features, and connect them in the channel dimension to form the initial features of the spatial domain; S5: Perform Mamba feature extraction and cross-modal interaction in the spatial domain branch, specifically including the following steps: S5-1: Spatial Mamba: Expand the initial spatial domain features from step S4 into a sequence input Mamba module by channel to obtain spatial structure representation and residual output; S5-2: Spectral Mamba: The initial spatial domain features from step S4 are grouped along the channels and sequentially input into the Mamba module to obtain the spectral correlation characterization and output the residuals. S5-3: Interactive Mamba: Taking the outputs of spatial Mamba and spectral Mamba as inputs, after layer normalization and linear projection, the spatial features and spectral features are fused through one-dimensional convolution with activation and state space modeling in sequence to obtain interactive features F_CM¹. S6: High-frequency detail residual injection and fusion upsampling: The second-level high-frequency detail component obtained in step S3 of the frequency domain branch is injected into F_CM¹ in a residual manner to form a compensation feature; the compensation feature is input into the fusion module Fuse to perform upsampling and feature recombination to obtain the upsampled feature F_FUSE¹; F_FUSE¹ is integrated with the first-level low-frequency main component of the frequency domain branch; S7: Multi-level iteration and resolution restoration. Perform at least one iteration on the process of steps S4-S5 to obtain higher resolution interactive features and fusion results; when the feature map resolution is restored to the PAN level, output the final interactive features; S8: Reconstruction and Output. The interactive features output from the last-level interactive Mamba block are input into the reconstruction module to generate reconstruction features F_RE. The F_RE is then connected with the global residuals of XLRMS↑ by pixel-by-pixel addition to output the target high-resolution multispectral image HRMS.

[0007] As a preferred embodiment of the present invention, the processing steps of step S5-1 include: expanding the input features into a one-dimensional sequence along each channel dimension and inputting it into the Mamba module; then grouping and normalizing the output of the Mamba module and activating it with SiLU; finally, adding the residuals with the original input. The specific calculation formula is as follows: (3) in, Indicates input up to the number The first space Mamba block and the first Features of the Mamba spectral block , Indicates the first The output of each space Mamba block, This indicates a feature flattening operation. This indicates a Mamba module. This indicates the output of the Mamba module within the Mamba space block. This indicates a grouping normalization operation. This represents the SiLU activation function.

[0008] As a preferred embodiment of the present invention, the processing steps of step S5-2 include: grouping the input features along the channel dimension, flattening them respectively, and inputting them into the Mamba module; then, normalizing each group of outputs and activating them with SiLU; and finally adding them to the original input as residuals. The specific calculation formula is as follows: (4) in, This indicates that the input features are grouped along the channel dimension. This represents the output of the Mamba module in the spectral Mamba block. Indicates the first The output of a spectral Mamba block.

[0009] As a preferred embodiment of the present invention, the specific processing procedure of step S5-3 includes: using the output of spatial Mamba With the output of the Mamba spectrum As input, after layer normalization and linear projection, it is sequentially processed through a two-branch system of one-dimensional convolution with SiLU activation and state space modeling to obtain... and At the same time The intermediate feature Y is obtained by layer normalization, linear projection, and SiLU activation. Y is then compared with the features output from the two branches. and Multiply and sum, then project linearly, and then... Add the residual branches to obtain the interactive output. The specific calculation process is as follows: (5) in, Features that represent input to an interactive Mamba block; Presentation layer normalization operation, This represents a linear projection operation. This represents a one-dimensional convolution operation. Representing a state-space model; Indicates the output of SSM; Indicates the first The output of an interactive Mamba block.

[0010] As a preferred embodiment of the present invention, the fusion module in step S6 uses sub-pixel convolution with 2x upsampling to improve the spatial resolution of the feature map. Specifically, it includes the following steps: convolution and batch normalization are performed on the input features sequentially; after LeakyReLU activation, PixelShuffle is used to achieve 2x upsampling; finally, 1×1 convolution is performed to reconstruct the features, and the output is... The specific calculation formula is as follows: (6) in, Indicates the first The input of each fusion module, This represents the subpixel convolution PixelShuffle operation. This indicates a batch normalization operation. Indicates the first The output of each fusion module.

[0011] As a preferred embodiment of the present invention, the reconstruction module in step S8 adopts a channel attention-based reconstruction structure. It performs deep convolutional feature extraction and adaptive average pooling on the input features to generate channel attention weights ω. ω is then applied to the features according to their channels and added to the identity mapping branch to achieve residual enhancement. Finally, the reconstructed features are obtained through convolution. The specific calculation formula is as follows: (7) in, This represents the output of the third interactive Mamba block. This indicates an adaptive average pooling operation. Represents the ReLU activation function. This represents the Sigmoid activation function. This represents the attention weight coefficient. This represents the operation of multiplying the feature map of each channel by its corresponding weight coefficient. This indicates the output of the reconstruction module.

[0012] As a preferred embodiment of the present invention, step S8 further includes employing global residual connection to add the reconstructed image obtained by the reconstruction module to the upsampled image obtained by bicubic interpolation upsampling of LRMS in step S1 pixel by pixel to obtain the final high-resolution multispectral image HRMS. The specific calculation formula is as follows: (8) in, This refers to the result after upsampling LRMS by 4 times. The image output by the reconstruction module.

[0013] As a preferred embodiment of the present invention, the method further includes a training phase, in which L1 norm loss is used to optimize and constrain the computation process corresponding to steps S1 to S8, and the loss function is defined as: (9) in, Represents the true value. Representing LRMS images With PAN Image The fusion result obtained by the WMDF-Net model designed in this invention represent Norm.

[0014] A dual-domain fusion full-color sharpening system based on wavelet transform and Mamba, used to deploy and execute the above method, includes the following modules: T1: Data Acquisition and Interface Module: Used in step S1 to acquire the low-resolution multispectral image LRMS and the corresponding panchromatic image PAN, and output the upsampled image XLRMS↑ with the same resolution as PAN; T2: Frequency domain feature extraction module: used in step S2 to extract features from XLRMS↑ and PAN respectively through the feature extraction network, and to connect the two in the channel dimension to form frequency domain input features; T3: Wavelet decomposition module: Used in step S3 to perform discrete wavelet transform on the frequency domain input features to perform continuous two-level decomposition, obtaining first-level low-frequency main component, first-level high-frequency detail component, and second-level low-frequency main component and second-level high-frequency detail component; T4: Spatial Domain Alignment and Feature Construction Module: Used in step S4 to downsample PAN to obtain XPAN↓ with the same resolution as the original LRMS, and input LRMS and XPAN↓ into the feature extraction network to obtain spectral features and spatial structure features respectively, and connect them in the channel dimension to form the initial features of the spatial domain. T5: Spatial Mamba Module: Used in step S5-1 to expand the initial features of the spatial domain into a sequence input Mamba module by channel, obtain the spatial structure representation and output the residual; T6: Spectral Mamba Module: Used in step S5-2 to group and serialize the initial features of the spatial domain along the channels and input them into the Mamba module to obtain the spectral correlation characterization and output the residuals. T7: Interactive Mamba module: In step S5-3, spatial Mamba output and spectral Mamba output are used as inputs. After layer normalization and linear projection, one-dimensional convolution and state space modeling, the interactive fusion of spatial features and spectral features is completed to obtain interactive features F_CM¹. T8: High-frequency residual injection and fusion upsampling module: used in step S6 to inject the second-level high-frequency detail component into F_CM¹ in a residual manner to form a compensation feature, and input it into the fusion module Fuse to perform upsampling and feature recombination to obtain the upsampled feature F_FUSE¹, which is also integrated with the first-level low-frequency main component. T9: Multi-level iteration and resolution recovery control module: used to perform at least one iteration of the process in steps S4-S5 in step S7 until the feature map resolution is restored to the PAN level and the final interactive feature is output; T10: Reconstruction and Output Module: Used in step S8 to reconstruct the final interactive features to obtain F_RE, and perform a pixel-by-pixel addition global residual connection with XLRMS↑ to output the high-resolution multispectral image HRMS; T11: Computation and storage unit and scheduling management: Used to carry and schedule the method steps executed by modules T1~T10, store model parameters and complete batch processing and inference processes.

[0015] Compared with the relevant prior art, the beneficial effects of the present invention are: Better spectral consistency: Low-frequency mains are explicitly separated from high-frequency details through wavelet domain, and a global residual connection with upsampled LRMS is used at the output, which significantly reduces color shift and spectral distortion; Clearer spatial details: Residual injection of high-frequency details in the frequency domain and fusion upsampling (PixelShuffle) are used to reconstruct the image, preserving edges and textures while suppressing oversmoothing and artifacts; The spatial-spectral coupling is tighter: spatial Mamba, spectral Mamba and interactive Mamba work together and interact deeply. Compared with the simple parallel or post-concatenation dual-domain method, it can more fully model cross-modal correlation. Global modeling and computationally efficient: Mamba's state space modeling has linear complexity, which is significantly more computationally and memory-efficient than Transformer-type methods while maintaining the ability to model global dependencies, making it more suitable for high-resolution remote sensing images. More comprehensive multi-scale representation: The combination of two-level wavelet decomposition and multi-level iterative fusion path not only covers large-scale structures, but also enhances fine-grained details, improving the ability to represent complex land features (urban construction, agriculture and forestry, shorelines, etc.). More stable training and convergence: Residualization design is used throughout the frequency domain injection and reconstruction stages. Combined with L1 loss and normalization / activation strategies, it reduces the risk of gradient oscillation and mode collapse, resulting in smoother convergence. Attached Figure Description

[0016] Figure 1 A flowchart of a dual-domain fusion full-color sharpening method based on wavelet transform and Mamba provided by the present invention; Figure 2 A framework diagram of a dual-domain fusion full-color sharpening system based on wavelet transform and Mamba is provided for an embodiment of the present invention; Figure 3 A specific network structure diagram of the feature extraction module provided in an embodiment of the present invention is shown. Figure 4 A specific network structure diagram of the spatial Mamba block is provided for an embodiment of the present invention; Figure 5 A specific network structure diagram of the spectral Mamba block is provided for an embodiment of the present invention; Figure 6 A specific network structure diagram of the interactive Mamba block is provided for an embodiment of the present invention; Figure 7 A specific network structure diagram of the fusion module is provided for an embodiment of the present invention; Figure 8 A specific network structure diagram of the reconstruction module is provided for an embodiment of the present invention; Figure 9 A specific network structure diagram of the Mamba module is provided for an embodiment of the present invention; Figure 10 A fusion result diagram of different methods in embodiments of the present invention is provided; Figure 11 A general framework diagram of the ablation model (Net1) provided for embodiments of the present invention; Figure 12 The figure shows the fusion result of Net1 and WMDF-Net in an embodiment of the present invention. Detailed Implementation

[0017] The solutions provided by the present invention will be further described below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways and should not be construed as limited to the embodiments shown; rather, these embodiments provide those skilled in the art with implementation methods that meet applicable legal requirements.

[0018] Example 1: As Figure 1 As shown, this embodiment provides a dual-domain fusion panchromatic sharpening method based on wavelet transform and Mamba: First, LRMS and PAN are acquired. Bicubic interpolation is performed on LRMS to obtain XLRMS↑ with the same resolution as PAN. In the frequency domain branch, XLRMS↑ and PAN are concatenated in the channel dimension after feature extraction, and two-level discrete wavelet decomposition is performed to explicitly obtain low-frequency main body and high-frequency details. In the spatial domain branch, PAN is downsampled and aligned with LRMS, and initial spatial / spectral features are obtained through feature extraction. The structure and spectral characterization are extracted sequentially through spatial Mamba and spectral Mamba, and then cross-modal fusion is completed through interactive Mamba to obtain interactive features. Subsequently, the high-frequency details of the frequency domain branch are injected in a residual manner, and progressive upsampling and feature reconstruction are performed through the Fuse module, while integrating with the low-frequency main body component in the frequency domain. The above interaction and fusion are iterated in stages until the feature resolution is restored to the PAN level. Finally, the interactive features are input into the channel attention reconstruction module to obtain reconstructed features, and global residual connection is performed with XLRMS↑ to output high-resolution multispectral image HRMS.

[0019] This embodiment balances spectral consistency and spatial detail, offering efficient global modeling and good deployability; it includes the following steps: S1: Acquire the low-resolution multispectral image LRMS and the corresponding panchromatic image PAN, perform bicubic interpolation upsampling on LRMS to obtain an upsampled image XLRMS↑ with the same resolution as PAN, and retain the original PAN; S2: Construct frequency domain branch and feature extraction. Extract features from XLRMS↑ and PAN in step S1 through a feature extraction network, and concatenate them along the channel dimension to form frequency domain input features; the specific calculation formula is as follows: (1) in, This indicates the input LRMS image or PAN image. This represents a two-dimensional convolution operation. This means dividing the input equally along the channel dimension into and Two parts, This represents the LeakyReLU activation function. This indicates the instance normalization operation. This indicates a concatenation operation on the channel dimension of features. Indicates feature extraction operation; S3: Perform wavelet decomposition on the frequency domain input features from step S2, using discrete wavelet transform for two consecutive levels of decomposition, successively obtaining the first-level low-frequency main component and the first-level high-frequency detail component, and the second-level low-frequency main component and the second-level high-frequency detail component; the specific calculation formula is as follows: (2) Among them, superscript Indicates an upsampling operation. Represents wavelet transform, express Feature maps of different sub-bands after wavelet decomposition; specifically, express Low-frequency sub-band feature map after wavelet decomposition. , and They represent High-frequency sub-band feature maps in the horizontal, vertical, and diagonal directions after wavelet decomposition; S4: Construct a spatial domain branch aligned with the input, downsample the PAN to obtain XPAN↓ with the same resolution as the original LRMS; input the LRMS and XPAN↓ into the feature extraction network respectively to obtain spectral features and spatial structure features, and connect them in the channel dimension to form the initial features of the spatial domain; S5: Perform Mamba feature extraction and cross-modal interaction in the spatial domain branch, specifically including the following steps: S5-1: Spatial Mamba: The initial spatial domain features from step S4 are expanded into a sequence along each channel and input into the Mamba module to obtain a spatial structure representation and residual output. The processing includes: expanding the input features into a one-dimensional sequence along each channel dimension and inputting it into the Mamba module; then, grouping and normalizing the output of the Mamba module and activating it with SiLU; finally, adding the residuals with the original input. The specific calculation formula is as follows: (3) in, Indicates input up to the number The first space Mamba block and the first Features of the Mamba spectral block , Indicates the first The output of each space Mamba block, This indicates a feature flattening operation. This indicates a Mamba module. This indicates the output of the Mamba module within the Mamba space block. This indicates a grouping normalization operation. Indicates the SiLU activation function; S5-2: Spectral Mamba: The initial spatial domain features from step S4 are grouped along the channels and sequentially input into the Mamba module to obtain spectral correlation representations and residual outputs. The processing includes: grouping the input features along the channel dimension, flattening them, and inputting them into the Mamba module. Then, the outputs of each group are grouped, normalized, activated by SiLU, and added to the original input as residuals. The specific calculation formula is as follows: (4) in, This indicates that the input features are grouped along the channel dimension. This represents the output of the Mamba module in the spectral Mamba block. Indicates the first The output of each spectral Mamba block; S5-3: Interactive Mamba: Taking the outputs of spatial Mamba and spectral Mamba as inputs, after layer normalization and linear projection, the spatial and spectral features are sequentially fused through one-dimensional convolution with activation and state space modeling to obtain the interactive feature F_CM¹; the specific processing includes: taking the outputs of spatial Mamba... With the output of the Mamba spectrum As input, after layer normalization and linear projection, it is sequentially processed through a two-branch system of one-dimensional convolution with SiLU activation and state space modeling to obtain... and At the same time The intermediate feature Y is obtained by layer normalization, linear projection, and SiLU activation. Y is then compared with the features output from the two branches. and Multiply and sum, then project linearly, and then... Add the residual branches to obtain the interactive output. The specific calculation process is as follows: (5) in, Features that represent input to an interactive Mamba block; Presentation layer normalization operation, This represents a linear projection operation. This represents a one-dimensional convolution operation. Representing a state-space model; Indicates the output of SSM; Indicates the first The output of each interactive Mamba block; S6: High-frequency detail residual injection and fusion upsampling. The second-level high-frequency detail component obtained in step S3 of the frequency domain branch is injected into F_CM¹ in a residual manner to form a compensation feature; the compensation feature is input into the fusion module Fuse to perform upsampling and feature recombination to obtain the upsampled feature F_FUSE¹; F_FUSE¹ is integrated with the first-level low-frequency main component of the frequency domain branch; the fusion module uses sub-pixel convolution with 2x upsampling to improve the spatial resolution of the feature map, specifically including the following operation process: the input feature is subjected to convolution and batch normalization processing, activated by LeakyReLU and then upsampled by PixelShuffle, and finally reconstructed by 1×1 convolution to output the feature. The specific calculation formula is as follows: (6) in, Indicates the first The input of each fusion module, This represents the subpixel convolution PixelShuffle operation. This indicates a batch normalization operation. Indicates the first The output of each fusion module; S7: Multi-level iteration and resolution restoration. Perform at least one iteration on the process of steps S4-S5 to obtain higher resolution interactive features and fusion results; when the feature map resolution is restored to the PAN level, output the final interactive features; S8: Reconstruction and Output. The interactive features output from the last-level interactive Mamba block are input into the reconstruction module to generate the reconstruction features F_RE. F_RE is then connected to XLRMS↑ via a pixel-by-pixel global residual concatenation to output the target high-resolution multispectral image HRMS. The reconstruction module employs a channel attention-based reconstruction structure: deep convolutional feature extraction and adaptive average pooling are performed on the input features to generate channel attention weights ω. ω is then applied to the features according to each channel and added to the identity mapping branch to achieve residual enhancement. Finally, the reconstructed features are obtained through convolution. The specific calculation formula is as follows: (7) in, This represents the output of the third interactive Mamba block. This indicates an adaptive average pooling operation. Represents the ReLU activation function. This represents the Sigmoid activation function. This represents the attention weight coefficient. This represents the operation of multiplying the feature map of each channel by its corresponding weight coefficient. This represents the output of the reconstruction module. Using global residual connections, the reconstructed image obtained from the reconstruction module is added pixel-by-pixel to the upsampled image obtained from the bicubic interpolation upsampling of LRMS in step S1, resulting in the final high-resolution multispectral image HRMS. The specific calculation formula is as follows: (8) in, This refers to the result after upsampling LRMS by 4 times. For the image output by the reconstruction module, this global residual is used to enhance the spectral consistency of the output; The method also includes a training phase, during which L1 norm loss is used to optimize and constrain the computation process corresponding to steps S1 to S8. The loss function is defined as follows: (9) in, Represents the true value. Representing LRMS images With PAN Image The fusion result obtained by the WMDF-Net model designed in this invention represent Norm.

[0020] Example 2: Figure 2 As shown, this method proposes a general framework for panchromatic sharpening of remote sensing images, WMDF-Net. WMDF-Net is generally composed of a spatial domain based on Mamba and a frequency domain based on wavelet transform, specifically including the following modules: Frequency domain branching: such as Figure 1 As shown, firstly, bicubic interpolation upsampling is performed on the LRMS image to match its spatial resolution with the PAN image; simultaneously, the original PAN image is preserved to maintain its rich spatial detail. Subsequently, a feature extraction network (specific structure shown in...) is used... Figure 3 (As shown) Features of the upsampled MS and PAN images are extracted respectively, and then concatenated and integrated along the channel dimension. Finally, the frequency domain information is decomposed through two cascaded wavelet transform modules. Specifically, the execution processes of the feature extraction network and wavelet decomposition are detailed below: Feature extraction networks: such as Figure 3 As shown, in the feature extraction network, the input image first undergoes nonlinear transformation through convolutional layers and activation functions, then undergoes semi-instance normalization (InstanceNorm) processing, and finally completes the feature extraction operation through convolutional layers and activation functions.

[0021] Wavelet decomposition: such as Figure 2 As shown, after channel-dimensional concatenation and integration of the features output by the feature extraction network, the features are input into two cascaded wavelet transform modules to achieve frequency domain information decomposition.

[0022] Spatial domain branches: such as Figure 2 As shown, the PAN image is first downsampled to match the spatial resolution of the original LRMS image while preserving its original spectral characteristics. Then, through methods such as... Figure 3 The feature extraction network shown extracts the spatial structure features of the downsampled PAN image and the spectral features of the LRMS image, respectively, and then concatenates and integrates them along the channel dimension. Subsequently, the integrated features are simultaneously processed through a Spatial Mamba block (SpaMamba, specific structure as shown below). Figure 4 (as shown) and the spectral Mamba block (SpeMamba, specific structure as shown) Figure 5 After that (as shown), it is then processed through a cross-modal Mamba block (Cross-Mamba, specific structure as shown). Figure 6 As shown in the diagram, this enables the extraction and interaction of deep features. Simultaneously, high-frequency detail information obtained from the frequency domain branch through second-level wavelet decomposition is injected as residuals to complete cross-domain feature compensation. Next, the compensated features are input to the fusion module (Fuse, specific structure as shown in the diagram). Figure 7 (As shown). After processing by the Fuse module, the reconstructed features are integrated with the low-frequency subject information obtained from first-order wavelet decomposition. Then, the above feature extraction and interaction process is repeated, i.e., spatial Mamba block, spectral Mamba block, and interactive Mamba block operations are performed. Subsequently, first-order wavelet decomposition is integrated to obtain high-frequency detail components, and the Fuse module operation is executed again. When the feature map resolution is restored to the PAN image level, the final spatial and spectral information interaction is performed (spatial Mamba block, spectral Mamba block, and interactive Mamba block operations are performed), and then input into the Reconstruction module (specific structure as shown). Figure 8 (As shown).

[0023] Fuse modules: such as Figure 7 As shown, the fusion module (Fuse) introduces a 2x upsampling operation, which improves the spatial resolution of the feature map through PixelShuffle subpixel convolution. Specifically, the Fuse module first performs convolution and batch normalization on the input features, then uses LReLu for activation and performs 2x upsampling, and finally reassembles the features through 1×1 convolution.

[0024] Reconstruction module: such as Figure 2As shown, to effectively preserve spatial details and spectral features, this invention designs a feature reconstruction module based on a channel attention mechanism, aiming to achieve accurate mapping from the feature space to high-resolution multispectral image HRMS. Specifically, as... Figure 8 As shown, this module first performs deep feature extraction on the input features, then uses adaptive average pooling and introduces convolutional layers to adjust the channel dimensions, thereby generating channel attention weights. Subsequently, these weights are multiplied channel-by-channel with the original feature map to achieve feature enhancement, and residual connections are introduced to preserve the original feature information.

[0025] Finally, as Figure 2 As shown, the output of the reconstruction module is globally residually coupled with the LRMS image that has undergone bicubic interpolation upsampling to further improve the spectral consistency of the reconstruction results.

[0026] Example 3: This example uses publicly available QuickBird and IKONOS urban scene data to comprehensively verify the proposed dual-domain fusion full-color sharpening method based on wavelet transform and Mamba. Both types of data generally include typical features such as densely built-up areas, road networks, green belts, and water bodies, exhibiting complex textures, significant differences in features, and rich spectral components. This allows for testing the algorithm's comprehensive ability to restore spatial details and maintain spectral consistency under realistic and demanding conditions. The samples cover various shooting conditions and feature combinations, emphasizing both the discernibility of edge contours and texture levels, as well as the stability of color relationships between channels, thus providing sufficient scene diversity and statistical reliability for subsequent subjective and objective evaluations.

[0027] The data used in this method comes from the publicly available QuickBird (QB) and IKONOS (IK) datasets. These datasets primarily focus on urban areas, especially buildings, and cover various land cover types such as trees and roads. During the experiments, approximately 90% of the data was used for model training, and the remaining 10% was used to test model performance. Since the ground truth of the sharpening results is unavailable, this method generates a training set according to the Wald protocol. Specifically, the source LRMS image is used as a reference and compared with the sharpened HRMS image, where the size of the source PAN image is [size missing]. The size of the source LRMS image is The experimental PAN and LRMS images generated according to the Wald protocol have the following dimensions: and The dimensions of both the reference image and the generated HRMS image are... Using this method, a total of 9000 image pairs were obtained for training and 200 image pairs for testing. During the training phase, the Adam optimizer was used for model optimization, with an initial learning rate set to a certain value, and the learning rate was reduced by a factor of 10 every 200 epochs. The entire training process lasted 700 epochs to ensure that the model could fully learn and optimize its parameters. Furthermore, all training and testing tasks were performed using the PyTorch framework on a computing platform equipped with an NVIDIA GeForce RTX 4090 graphics card.

[0028] To evaluate the performance of the proposed method, we compared it with two traditional panchromatic and multispectral remote sensing image fusion techniques (GS and PCA) and nine recently proposed deep learning-based methods (MSDCNN, DARN, MUCNN, LAG, UCGAN, Restormer, LFormer, SFINet, and SMCET).

[0029] To subjectively demonstrate the fusion effects of different methods, this method selects two sets of fusion results from PAN and LRMS images in the test data for presentation, such as... Figure 10 As shown. In particular, for easier observation and comparison, the local areas shown in the red boxes in the images have been magnified and placed in the lower left or upper left corner of each image.

[0030] Depend on Figure 10It can be seen that both traditional algorithms and deep learning methods have achieved certain results in improving spatial detail information. However, compared with the reference image (GT), it was found that the HRMS images generated by traditional methods are inferior to those generated by deep learning methods in terms of spatial and spectral detail quality. Specifically, the two traditional methods, GS and PCA, exhibited varying degrees of spectral distortion in both sets of images, and noise appeared at the edge contours. Among the deep learning methods, MSDCNN and MUCNN suffered from shallow spectral density and blurred contours; DARN and LAG methods showed relatively good spectral information representation in the first set of fusion results, but distorted texture information and spectral information in the second set; the UCGAN method produced noise in the fusion results, leading to distortion of building edge contour information; the Restormer and LFormer methods caused building contour blurring due to artifacts; the SFINet method preserved spatial and spectral information well in the first set of fusion results, but showed building blurring and texture distortion in the second set of fusion results; the SMCET method performed relatively well, but still lost some spectral information in the second set of fusions. In comparison, the proposed method demonstrates a more significant effect in the fusion of panchromatic and multispectral images. This method not only effectively preserves the spatial information of PAN images but also maintains the spectral characteristics of LRMS images well.

[0031] To further evaluate the fusion performance of different methods, this paper selects seven commonly used evaluation metrics, including Multiscale Structural Similarity (MS-SSIM), Peak Signal-to-Noise Ratio (PSNR), Spectral Angle Mapper (SAM), Root Mean Square Error (RMSE), Relative Global Error in Synthesis (ERGAS), Spatial Correlation Coefficient (SCC), and Universal Image Quality Index (UIQI), to objectively evaluate the fusion results of different methods. A higher PSNR value indicates better fusion performance. A higher MS-SSIM value indicates greater structural similarity between the fused image and the ground truth (GT). ERGAS and SAM are related to spectral distortion; lower values ​​indicate higher image quality. A lower RMSE value indicates smaller differences between the fused image and the GT. A higher SCC value indicates greater spatial correlation between the fused image and the GT. The higher the UIQI value, the richer the information contained in the fused image. The average evaluation indexes of the fusion results obtained by different methods are shown in Table 1, where the bolded values ​​indicate the best performance and the underlined values ​​indicate the second best performance.

[0032] Table 1: Average evaluation metrics obtained by different methods when fusing 200 sets of images

[0033] As shown in Table 1, SMCET ranks first in both ERGAS and UIQI metrics, outperforming the proposed model by 1.63% and 0.49% respectively, and outperforming the third-ranked Restormer method by 6.29% and 0.61% respectively. Furthermore, the fusion results of this method are optimal in all five metrics: MS-SSIM, PSNR, SAM, RMSE, and SCC. This indicates that, compared to the comparative methods, the fused image obtained by this method possesses richer spectral information and clearer spatial information.

[0034] To more comprehensively evaluate different algorithms, this method further conducts comparative experiments from the perspectives of model parameter count (Params), model complexity (FLOPs), and inference speed (Time). The experimental results are shown in Table 2. Bold values ​​indicate the best algorithm, and underlined values ​​indicate the second best.

[0035] Table 2: Comparison of Params, FLOPs, and Time for Different Methods

[0036] In terms of data organization, a strategy of using approximately 90% of the samples for training and the remaining 10% for independent testing was adopted to ensure sufficient sample coverage during the learning phase and representativeness and invisibility during the evaluation phase. Considering the lack of ground truth values ​​for sharpened high-resolution multispectral images in real-world scenes, this embodiment follows the Wald protocol to construct training and validation pairs. The core idea is to perform consistent downscaling and matching processing on the original panchromatic and multispectral images across multiple scales, using the source multispectral image as a reference to form a "supervised" evaluation loop. This protocol effectively circumvents the problem of unavailable ground truth values ​​and keeps the training objective and evaluation criteria consistent, thereby ensuring the credibility of the supervision signal and the interpretability of the experimental conclusions. Ultimately, approximately 9,000 training pairs and approximately 200 test pairs were obtained, which is sufficient in scale and reasonably distributed.

[0037] As shown in Table 2, the proposed model significantly outperforms most of the comparative methods in terms of computational efficiency. Its model complexity is 6.541G, 3.2 times that of the best method, LAG, but its inference speed is 27.7 times faster than LAG; the number of parameters is 0.116M, only 0.9% of the number of parameters required by the SMCET method; and the inference time is 0.02s, a 45.9% speedup compared to the second-best comparative method, DARN.

[0038] Example 4: To verify the effectiveness of injecting frequency domain information in improving image fusion in the proposed method, an ablation experiment was conducted on the frequency domain branch. In the ablation experiment, the frequency domain branch was removed from the overall framework, resulting in a simplified network structure, which was named Net1. Figure 11 As shown. By Figure 11 As can be seen, after the frequency domain branch is removed, the spatial domain branch no longer receives information compensation from the frequency domain wavelet decomposition, but relies solely on feature extraction, spatial and spectral information extraction and interaction, the fusion module, and the reconstruction module in the spatial domain to complete the fusion task. Furthermore, the ablation model maintains consistency with the proposed method in terms of network structure design, loss function, and parameter settings during model training. The fusion result of the ablation model (Net1) and the proposed method (WMDF-Net) and its magnified local region are shown below. Figure 12 As shown.

[0039] As shown in the magnified area marked by the red box, the edge contour clarity of the fusion result obtained by this method is better than that of the ablation model Net1. This is mainly due to the compensation of frequency domain information for the spatial domain. To further compare the performance of the ablation model and this method, the specific performance of the two methods on seven objective evaluation indicators was calculated, as shown in Table 3. Bold values ​​indicate the superior result.

[0040] Table 3: Average evaluation metrics achieved by different models when fusing 200 sets of images. method MS-SSIM↑ PSNR↑ SAM↓ RMSE↓ ERGAS↓ SCC↑ UIQI↑ WMDF-Net 0.9494 22.5074 0.1955 0.1328 6.3126 0.9313 0.9499 Net1 0.8958 19.2048 0.2822 0.193 8.7722 0.8934 0.9238 As shown in Table 3, the fusion results of the ablation model are inferior to those of the model proposed in this invention across all seven objective evaluation metrics. Specifically, compared to the ablation model without a frequency domain branch, the model proposed in this method, after adding a frequency domain branch, achieves improvements in MS-SSIM by 5.98%, PSNR by 17.20%, SAM by 30.72%, RMSE by 31.19%, ERGAS by 28.04%, SCC by 4.24%, and UIQI by 2.82%. This indicates that the information compensation of the frequency domain branch proposed in this method plays a crucial role in improving panchromatic sharpening performance.

[0041] This embodiment proposes a frequency-spatial domain dual-domain collaborative deep fusion network for panchromatic and multispectral remote sensing image fusion, and employs a progressive fusion strategy to complete the reconstruction from low resolution to high resolution. Specifically, wavelet multi-scale decomposition is introduced in the frequency domain branch to explicitly obtain the high and low frequency components of PAN and LRMS and extract their features. Then, high-frequency details and low-frequency main elements are injected into the spatial domain in stages. In the spatial domain branch, a dual path of spatial Mamba and spectral Mamba is constructed, focusing on spatial structure and spectral relationship modeling respectively, and deep coupling of spatial-spectral information is achieved through interactive Mamba. Subsequently, the resolution is progressively improved using the fusion module, and finally, a reconstruction module with channel attention is used to generate HRMS. Extensive experiments based on the QB and IK standard datasets show that the proposed method has clearer edges and textures and more stable colors in subjective vision. It outperforms many traditional and deep learning comparison methods in objective indicators such as MS-SSIM, PSNR, SAM, RMSE, ERGAS, SCC, and UIQI, while maintaining high computational efficiency and good deployment friendliness.

[0042] The above embodiments merely illustrate implementation methods of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A dual-domain fusion full-color sharpening method based on wavelet transform and Mamba, characterized in that: Includes the following steps: S1: Acquire the low-resolution multispectral image LRMS and the corresponding panchromatic image PAN, perform bicubic interpolation upsampling on LRMS to obtain an upsampled image XLRMS↑ with the same resolution as PAN, and retain the original PAN; S2: Construct frequency domain branch and feature extraction. Extract features from XLRMS↑ and PAN in step S1 through the feature extraction network, and connect them in the channel dimension to form frequency domain input features. The specific calculation formula is as follows: （1） in, This indicates the input LRMS image or PAN image. This represents a two-dimensional convolution operation. This means dividing the input equally along the channel dimension into and Two parts, This represents the LeakyReLU activation function. This indicates the instance normalization operation. This indicates a concatenation operation on the channel dimension of features. Indicates feature extraction operation; S3: Perform wavelet decomposition on the frequency domain input features from step S2, using discrete wavelet transform for two consecutive levels of decomposition, successively obtaining the first-level low-frequency main component and the first-level high-frequency detail component, and the second-level low-frequency main component and the second-level high-frequency detail component; the specific calculation formula is as follows: （2） Among them, superscript Indicates an upsampling operation. Represents wavelet transform, express Feature maps of different sub-bands after wavelet decomposition; specifically, express Low-frequency sub-band feature map after wavelet decomposition. , and They represent High-frequency sub-band feature maps in the horizontal, vertical, and diagonal directions after wavelet decomposition; S4: Construct a spatial domain branch aligned with the input, downsample the PAN to obtain XPAN↓ with the same resolution as the original LRMS; input the LRMS and XPAN↓ into the feature extraction network respectively to obtain spectral features and spatial structure features, and connect them in the channel dimension to form the initial features of the spatial domain; S5: Perform Mamba feature extraction and cross-modal interaction in the spatial domain branch, specifically including the following steps: S5-1: Spatial Mamba: Expand the initial spatial domain features from step S4 into a sequence input Mamba module by channel to obtain spatial structure representation and residual output; S5-2: Spectral Mamba: The initial spatial domain features from step S4 are grouped along the channels and sequentially input into the Mamba module to obtain the spectral correlation characterization and output the residuals. S5-3: Interactive Mamba: Taking the outputs of spatial Mamba and spectral Mamba as inputs, after layer normalization and linear projection, the spatial features and spectral features are fused through one-dimensional convolution with activation and state space modeling in sequence to obtain interactive features F_CM¹. S6: High-frequency detail residual injection and fusion upsampling: The second-level high-frequency detail component obtained in step S3 of the frequency domain branch is injected into F_CM¹ in a residual manner to form a compensation feature; the compensation feature is input into the fusion module Fuse to perform upsampling and feature recombination to obtain the upsampled feature F_FUSE¹; F_FUSE¹ is integrated with the first-level low-frequency main component of the frequency domain branch; S7: Multi-level iteration and resolution restoration. Perform at least one iteration on the process of steps S4-S5 to obtain higher resolution interactive features and fusion results; when the feature map resolution is restored to the PAN level, output the final interactive features; S8: Reconstruction and Output. The interactive features output from the last-level interactive Mamba block are input into the reconstruction module to generate reconstruction features F_RE. The F_RE is then connected with the global residuals of XLRMS↑ by pixel-by-pixel addition to output the target high-resolution multispectral image HRMS.

2. The dual-domain fusion full-color sharpening method based on wavelet transform and Mamba as described in claim 1, characterized in that: Step S5-1 includes the following processing steps: expanding the input features into a one-dimensional sequence along each channel dimension and inputting it into the Mamba module; then, grouping and normalizing the output of the Mamba module and activating it with SiLU; finally, adding the residuals to the original input. The specific calculation formula is as follows: （3） in, Indicates input up to the number The first space Mamba block and the first Features of the Mamba spectral block , Indicates the first The output of each space Mamba block, This indicates a feature flattening operation. This indicates a Mamba module. This indicates the output of the Mamba module within the Mamba block. This indicates a grouping normalization operation. This represents the SiLU activation function.

3. The dual-domain fusion full-color sharpening method based on wavelet transform and Mamba as described in claim 1, characterized in that: Step S5-2 includes the following processing steps: The input features are grouped along the channel dimension, flattened, and input into the Mamba module. Then, the outputs of each group are grouped, normalized, activated by SiLU, and added to the original input as residuals. The specific calculation formula is as follows: （4） in, This indicates that the input features are grouped along the channel dimension. This represents the output of the Mamba module in the spectral Mamba block. Indicates the first The output of a spectral Mamba block.

4. The dual-domain fusion full-color sharpening method based on wavelet transform and Mamba as described in claim 1, characterized in that: The specific processing steps in step S5-3 include: using the output of spatial Mamba... With the output of the Mamba spectrum As input, after layer normalization and linear projection, it is sequentially processed through a two-branch system of one-dimensional convolution with SiLU activation and state space modeling to obtain... and At the same time The intermediate feature Y is obtained by layer normalization, linear projection, and SiLU activation. Y is then compared with the features output from the two branches. and Multiply and sum, then project linearly, and then... Add the residual branches to obtain the interactive output. The specific calculation process is as follows: （5） in, Features that represent input to an interactive Mamba block; Presentation layer normalization operation, This represents a linear projection operation. This represents a one-dimensional convolution operation. Representing a state-space model; Indicates the output of SSM; Indicates the first The output of an interactive Mamba block.

5. The dual-domain fusion full-color sharpening method based on wavelet transform and Mamba as described in claim 1, characterized in that: The fusion module in step S6 uses sub-pixel convolution with 2x upsampling to improve the spatial resolution of the feature map. Specifically, it includes the following steps: The input features are sequentially convolved and batch normalized; after LeakyReLU activation, they are upsampled by PixelShuffle; finally, a 1×1 convolution is performed to reconstruct the features, resulting in the output. The specific calculation formula is as follows: （6） in, Indicates the first The input of each fusion module, This represents the subpixel convolution PixelShuffle operation. This indicates a batch normalization operation. Indicates the first The output of each fusion module.

6. The dual-domain fusion full-color sharpening method based on wavelet transform and Mamba as described in claim 1, characterized in that: The reconstruction module in step S8 adopts a channel attention-based reconstruction structure. It performs deep convolutional feature extraction and adaptive average pooling on the input features to generate channel attention weights ω. ω is then applied to the features according to their channels and added to the identity mapping branch to achieve residual enhancement. Finally, the reconstructed features are obtained through convolution. The specific calculation formula is as follows: （7） in, This represents the output of the third interactive Mamba block. This indicates an adaptive average pooling operation. Represents the ReLU activation function. This represents the Sigmoid activation function. This represents the attention weight coefficient. This represents the operation of multiplying the feature map of each channel by its corresponding weight coefficient. This indicates the output of the reconstruction module.

7. The dual-domain fusion panchromatic sharpening method based on wavelet transform and Mamba as described in claim 1, characterized in that: Step S8 also includes using global residual connections to add the reconstructed image obtained by the reconstruction module to the upsampled image obtained by bicubic interpolation upsampling of LRMS in step S1 pixel by pixel to obtain the final high-resolution multispectral image HRMS. The specific calculation formula is as follows: （8） in, This refers to the result after upsampling LRMS by 4 times. The image output by the reconstruction module.

8. The dual-domain fusion full-color sharpening method based on wavelet transform and Mamba as described in claim 1, characterized in that: The method also includes a training phase, during which L1 norm loss is used to optimize and constrain the computational process corresponding to steps S1 to S8. The loss function is defined as follows: （9） in, Represents the true value. Representing LRMS images With PAN Image The fusion result obtained by the WMDF-Net model designed in this invention represent Norm.

9. A dual-domain fusion full-color sharpening system based on wavelet transform and Mamba, used to deploy and execute the method described in any one of claims 1 to 8, characterized in that: Includes the following modules: T1: Data Acquisition and Interface Module: Used in step S1 to acquire the low-resolution multispectral image LRMS and the corresponding panchromatic image PAN, and output the upsampled image XLRMS↑ with the same resolution as PAN; T2: Frequency domain feature extraction module: used in step S2 to extract features from XLRMS↑ and PAN respectively through the feature extraction network, and to connect the two in the channel dimension to form frequency domain input features; T3: Wavelet decomposition module: Used in step S3 to perform discrete wavelet transform on the frequency domain input features to perform continuous two-level decomposition, obtaining first-level low-frequency main component, first-level high-frequency detail component, and second-level low-frequency main component and second-level high-frequency detail component; T4: Spatial Domain Alignment and Feature Construction Module: Used in step S4 to downsample PAN to obtain XPAN↓ with the same resolution as the original LRMS, and input LRMS and XPAN↓ into the feature extraction network to obtain spectral features and spatial structure features respectively, and connect them in the channel dimension to form the initial features of the spatial domain. T5: Spatial Mamba Module: Used in step S5-1 to expand the initial features of the spatial domain into a sequence input Mamba module by channel, obtain the spatial structure representation and output the residual; T6: Spectral Mamba Module: Used in step S5-2 to group and serialize the initial features of the spatial domain along the channels and input them into the Mamba module to obtain the spectral correlation characterization and output the residuals. T7: Interactive Mamba module: In step S5-3, spatial Mamba output and spectral Mamba output are used as inputs. After layer normalization and linear projection, one-dimensional convolution and state space modeling, the interactive fusion of spatial features and spectral features is completed to obtain interactive features F_CM¹. T8: High-frequency residual injection and fusion upsampling module: used in step S6 to inject the second-level high-frequency detail component into F_CM¹ in a residual manner to form a compensation feature, and input it into the fusion module Fuse to perform upsampling and feature recombination to obtain the upsampled feature F_FUSE¹, which is also integrated with the first-level low-frequency main component. T9: Multi-level iteration and resolution recovery control module: used to perform at least one iteration of the process in steps S4-S5 in step S7 until the feature map resolution is restored to the PAN level and the final interactive feature is output; T10: Reconstruction and Output Module: Used in step S8 to reconstruct the final interactive features to obtain F_RE, and perform a pixel-by-pixel addition global residual connection with XLRMS↑ to output the high-resolution multispectral image HRMS; T11: Computation and storage unit and scheduling management: Used to carry and schedule the method steps executed by modules T1~T10, store model parameters and complete batch processing and inference processes.

Images