Remote sensing image panchromatic sharpening method based on wavelet prior guided state space model

Abstract

The application relates to the technical field of image processing, and discloses a remote sensing image panchromatic sharpening method based on a wavelet prior guided state space model. The method acquires a multispectral image and a panchromatic image to be fused, and up-samples the multispectral image to the same spatial scale as the panchromatic image; a spatial prior tensor and a wavelet frequency domain prior tensor are respectively extracted; a hierarchical local feature injection network is used to extract deep multispectral features of the multispectral image; a shallow layer encoder is used to extract shallow panchromatic features of the panchromatic image; the deep multispectral features and the shallow panchromatic features are input into a multi-layer stacked frequency domain perception gated state space model, and multispectral fusion features and panchromatic fusion features are output; the two kinds of features are aggregated and optimized, and combined with the up-sampled multispectral image to perform global residual reconstruction, thereby generating a high-resolution multispectral image. The application guarantees that local subtle features are not lost, and the fusion quality and the calculation efficiency of the remote sensing image fusion model are considered.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method for panchromatic sharpening of remote sensing images based on a wavelet prior-guided state-space model, as well as a computer terminal and a computer-readable storage medium for applying this method. Background Technology

[0002] With the rapid development of satellite technology, the demand for high-resolution multispectral imagery (HRMS) is increasing in fields such as agriculture, urban planning, and environmental monitoring. However, due to current technological limitations, directly acquiring high-resolution multispectral images remains challenging. To address this issue, satellites are typically equipped with two types of sensors to simultaneously capture low-resolution multispectral (LRMS) images and high-resolution, texture-rich panchromatic (PAN) images. Image fusion technology (i.e., panchromatic sharpening) that combines these two data sources to obtain HRMS images has become a crucial solution.

[0003] Early pancolor sharpening primarily relied on traditional algorithms categorized into three types: component substitution (CS), multi-resolution analysis (MRA), and variational optimization (VO). However, these traditional methods often yielded unsatisfactory results due to their reliance on handcrafted features and insufficient integration of prior knowledge. In recent years, the rapid development of deep learning has led to the widespread application of convolutional neural networks (CNNs) in pancolor sharpening. Research indicates that effective global modeling and efficient extraction of high-frequency information from pancolor images are crucial for addressing the complex challenges of pancolor sharpening. Existing CNN and Transformer architectures often overfit low-frequency information, resulting in unclear structural details and high computational complexity. Furthermore, while a range of Transformer-based models excel in capturing long-range dependencies, the secondary computational complexity in their query, key, and value interactions remains a significant concern. This remains a fundamental bottleneck in high-resolution remote sensing image processing.

[0004] Recently, the emergence of state-space models (SSMs), represented by Mamba, has provided a paradigm-innovative alternative for global context modeling. Mamba captures long-range dependencies while maintaining the linear complexity of sequence modeling. This breakthrough overcomes the inherent computational bottleneck of the self-attention mechanism. However, pure state-space models have a natural disadvantage in capturing fine-grained local features (such as corners and subtle textures). Therefore, how to effectively compensate for the shortcomings of state-space models in extracting microscopic geometric features, while maintaining their linear computational efficiency advantage, and achieving high-quality global context modeling and accurate fusion of local high-frequency details, is an urgent problem to be solved in the current field of panchromatic sharpening of remote sensing images. Summary of the Invention

[0005] To address the technical problems existing in the prior art, this invention provides a remote sensing image panchromatic sharpening method based on a wavelet prior-guided state space model, which balances the fusion quality and computational efficiency of the remote sensing image fusion model while ensuring that local subtle features are not lost.

[0006] To achieve the above objectives, the present invention provides the following technical solution: This invention discloses a method for panchromatic sharpening of remote sensing images based on a wavelet prior-guided state-space model, comprising steps S1 to S5.

[0007] S1. Acquire the low-resolution multispectral image and the high-resolution panchromatic image to be fused, and upsample the multispectral image to the same spatial scale as the panchromatic image; S2. Based on the upsampled multispectral image and the original panchromatic image, extract the spatial prior tensor used to construct the physical structure substrate and the wavelet frequency domain prior tensor used for frequency domain guidance, respectively. S3. A hierarchical local feature injection network is used to extract deep multispectral features from the multispectral image. During the extraction process, the spatial prior tensor and the wavelet frequency domain prior tensor are explicitly injected layer by layer. At the same time, a shallow encoder is used to extract shallow panchromatic features from the panchromatic image. S4. Flatten and interleave the deep multispectral features and shallow panchromatic features, and input them into the multi-layered stacked frequency domain sensing gated state space model. Under the explicit guidance of the wavelet frequency domain prior tensor, dynamically adjust the dynamic parameters of the frequency domain sensing gated state space model to perform multimodal feature interaction and manifold alignment, and output multispectral fusion features and panchromatic fusion features. S5. Perform element-wise aggregation and spatial projection optimization on the multispectral fusion features and panchromatic fusion features, and combine them with the upsampled multispectral image to perform global residual reconstruction to generate a high-resolution multispectral image.

[0008] As a further improvement to the above scheme, step S2 specifically includes: S21. The upsampled multispectral image and the original panchromatic image are stitched together along the channel dimension to construct the spatial prior tensor that preserves the physical topological form; S22. The upsampled multispectral image is decomposed using discrete wavelet transform to extract low-frequency approximate components, and the panchromatic image is decomposed to extract high-frequency detail components including horizontal, vertical and diagonal directions; the low-frequency approximate components and high-frequency detail components are concatenated in the channel dimension to construct the wavelet frequency domain prior tensor.

[0009] As a further improvement to the above scheme, in step S3, the hierarchical local feature injection network is a U-shaped structure containing an encoder and a decoder, and the extraction process specifically includes: In the downsampling stage of the encoder, features are extracted step by step using stacked feature modulation injection blocks, and the spatial prior tensor is explicitly injected at each layer to establish a physical structure basis. Deep within the encoder, a multi-head spatial self-attention mechanism is introduced to aggregate global contextual features; During the upsampling phase of the decoder, the current layer features are concatenated with the corresponding skip connection features from the encoder, and wavelet frequency domain prior tensors adapted to the current resolution are explicitly injected step by step through feature modulation injection blocks to recover high-frequency details.

[0010] As a further improvement to the above scheme, the feature injection process of the feature modulation injection block specifically includes: The input spatial prior tensor or wavelet frequency domain prior tensor is mapped through a convolutional network to generate scaling and translation factors aligned in both spatial and channel dimensions. Affine modulation of the main feature stream is performed using scaling and translation factors to inject prior information; The affine modulated main features are sequentially processed through group normalization, activation function, and double-layer convolutional residual path for local texture extraction, and the modulated features are output through residual connection.

[0011] As a further improvement to the above scheme, step S4, which involves inputting deep multispectral features and shallow panchromatic features into the frequency domain sensing gated state space model, includes the following specific steps: Deep multispectral features and shallow panchromatic features are flattened into one-dimensional sequences in the spatial dimension and then arranged in an alternating manner according to their spatial positions to construct a spectral-spatial hybrid feature sequence. Flatten the wavelet frequency domain prior tensor in the spatial dimension and copy and interleave it to generate a hybrid frequency domain prior sequence; A set of learnable global context registers is appended to the end of the spectral-spatial hybrid feature sequence to generate an amplified feature sequence. By absorbing the global statistical information in the global context registers, the frequency domain-aware gated state space model is transitioned to a non-zero steady state, thus alleviating the zero-state initialization bias of the frequency domain-aware gated state space model. Configure learnable virtual frequency domain conditions for the global context register and append the virtual frequency domain conditions to the end of the hybrid frequency domain prior sequence to generate the amplified frequency domain prior sequence.

[0012] As a further improvement to the above scheme, in step S4, the specific process of dynamically adjusting the dynamic parameters of the frequency-domain sensing gated state-space model under the explicit guidance of the wavelet frequency-domain prior tensor includes: A multi-directional scanning strategy is applied to the amplified feature sequence and the amplified frequency domain prior sequence to obtain directional feature sequences in different directions. and directional frequency domain prior sequences ; For each scanning direction, the gating input is calculated using the direction frequency domain prior sequence as an explicit regulator. The calculation formula is:

[0013] In the formula, A learnable adaptive scaling factor. This represents a linear mapping operation; Gated input The core dynamic parameters for generating a frequency-domain sensing gated state-space model include the discretization step size. and projection matrix and When scanning to a smooth multispectral node, the discretization step size is adaptively amplified by the guidance of the high-frequency detail components of the panchromatic image contained in the wavelet frequency domain prior tensor. ; The system parameters are discretized and the state is updated using the core dynamic parameters, and the sequential response in each direction is output. The sequence responses in all directions are inversely transformed and aggregated, and then the multispectral response sequence and panchromatic response sequence are separated. After being reshaped into a two-dimensional spatial grid, the multispectral features and panchromatic features are updated respectively through residual connections to obtain the multispectral fusion features and the panchromatic fusion features.

[0014] As a further improvement to the above scheme, the processing steps S3 to S5 are integrated into a full-color sharpening network model; in the end-to-end training phase of the full-color sharpening network model, a composite loss function is used for joint constraints and parameter updates, and the calculation formula of the composite loss function is as follows:

[0015] In the formula, It is a composite loss function; This is used to constrain the pixel-level reconstruction loss for spectral numerical fidelity; For structural similarity loss used to enhance local structure perception; This is used for gradient preservation loss in explicit regularization of high-frequency details; These are hyperparameters used to balance the contributions of different loss components.

[0016] As a further improvement to the above scheme, pixel-level reconstruction loss The calculation formula is:

[0017] In the formula, This represents the total number of pixels contained in the image. To fuse the image in the first The pixel value at each position; For the first in the real image The pixel value at each position; It is an L1 norm; Structural similarity loss The calculation formula is:

[0018] In the formula, The number of image patches; An evaluation function for calculating the brightness, contrast, and structural similarity between two image patches; The first cropped image from the merged image Image blocks; The first cropped from the real image Image blocks in each region; Gradient Preservation Loss The calculation formula is:

[0019] In the formula, This is the Sobel operator, used to extract gradient maps in the horizontal and vertical directions.

[0020] The present invention also discloses a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the remote sensing image panchromatic sharpening method based on a wavelet prior guided state space model as described above.

[0021] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, it implements the steps of the remote sensing image panchromatic sharpening method based on a wavelet prior-guided state space model as described above.

[0022] Compared with the prior art, the beneficial effects of the present invention are: This invention proposes a panchromatic sharpening method for remote sensing images based on a wavelet prior-guided state-space model, which abandons the traditional approach of pure spatial domain stitching. By explicitly injecting wavelet high-frequency priors into the generation process of the dynamic parameters of the state-space model, the model can maintain a high response to high-frequency edges when processing smoothed multispectral nodes, effectively preventing the high-frequency texture details of the panchromatic image from being rapidly attenuated or "diluted" by smoothed nodes. This allows for the full fusion of features contained in the image in both the spatial and temporal domains.

[0023] This invention designs a hierarchical local feature injection module as the basic backbone. This module explicitly recovers multi-scale spatial details before deep global fusion occurs, decoupling "local texture extraction" from "global spectral modeling." This not only retains the advantages of the state-space model in terms of linear computational complexity but also compensates for the accuracy limitations of the pure state-space model architecture in recovering microscopic geometric features (such as corners and tiny edges).

[0024] Furthermore, this invention introduces a global context anchor learning mechanism. By appending a learnable global context register to the end of the mixed sequence, the state space is preheated before scanning the actual multimodal sequence, enabling the system to quickly transition to a non-zero steady state containing global statistical information. This eliminates the context vacuum of the initial state and enhances the stability of model inference. Attached Figure Description

[0025] Figure 1 This is a flowchart of the remote sensing image panchromatic sharpening method based on a wavelet prior-guided state space model in Embodiment 1 of the present invention.

[0026] Figure 2 This is a schematic diagram of the remote sensing image panchromatic sharpening training framework based on a wavelet prior-guided state space model in Embodiment 1 of the present invention.

[0027] Figure 3 This is a visualization comparison of the visible light results of the method of the present invention in Embodiment 1 of the present invention and several existing mainstream methods.

[0028] Figure 4 This is a visualization comparison of the mean square error of the real image between the method of the present invention in Embodiment 1 and several existing mainstream methods (the color bars in the figure represent the mean square error values, and the higher the pixel color in the figure, the greater the difference between the result and the real image).

[0029] Figure 5 This is a schematic diagram of the structure of the computer terminal in Embodiment 2 of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Example 1

[0032] Please see Figures 1 to 2This embodiment provides a panchromatic sharpening method for remote sensing images based on the wavelet prior guided state space model (WPG-Mamba), the overall process of which can include the following five steps: The first step is multi-source image acquisition and preprocessing: acquire low-resolution multispectral images and high-resolution panchromatic images, and complete the scale alignment of the images and explicit decoupling and extraction of multimodal prior information.

[0033] The second step is to build the WPG-Mamba model: construct a full-color sharpening network that includes a core architecture such as the Hierarchical Local Feature Injection Module (HLFI) and the Frequency Domain Aware Gated Mamba Module (FAGMamba); The third step is end-to-end model training: set up a composite loss function that includes pixel-level, structural-level, and gradient-level loss functions, and use the training set to perform end-to-end iterative optimization and parameter updates on the constructed model. The fourth step is to acquire the images to be processed: acquire the low-resolution multispectral and high-resolution panchromatic images to be fused in the actual application scenario, and perform preprocessing. Step 5, output the fusion result: input the preprocessed image to be processed and the prior tensor into the trained WPG-Mamba model to output a high-fidelity, high-resolution multispectral fused image.

[0034] The specific implementation method of the first step, the acquisition and prior extraction of panchromatic and multispectral images, is as follows: (1) Image acquisition and scale alignment First, acquire low-resolution multispectral images (LRMS) and high-resolution panchromatic images (PAN) of the same area from satellite or airborne sensors. Since the two modalities differ in spatial resolution, the low-resolution multispectral image needs to be upsampled using a bicubic interpolation (or bilinear interpolation) algorithm to perfectly align its spatial resolution scale with that of the high-resolution panchromatic image, resulting in the upsampled multispectral image.

[0035] (2) Explicit decoupling of multimodal priors

[0036] To fully utilize the physical and frequency domain characteristics of multi-source images, feature decoupling is performed on the upsampled multispectral and panchromatic images, and spatial priors and wavelet frequency domain priors are constructed respectively: (2.1) Spatial Prior Construction: The upsampled multispectral image and the original panchromatic image are directly concatenated along the channel dimension to construct the spatial prior tensor. This tensor preserves the complete physical topology and photometric information, and is used to provide basic physical structure guidance for the network.

[0037] (2.2) Wavelet Frequency Domain Prior Construction: Discrete Wavelet Transform (DWT) was used to decompose the images of the two modalities in the frequency domain. Through DWT, the low-frequency approximation component (LL) of the upsampled multispectral image was extracted to obtain its global spectral tone; simultaneously, the high-frequency detail components of the panchromatic image were extracted, including the horizontal (LH), vertical (HL), and diagonal (HH) directions, to obtain sharp edges and local texture details. Finally, the extracted multispectral low-frequency components and the panchromatic high-frequency components were concatenated along the channel dimension to construct the wavelet frequency domain prior tensor. This tensor will serve as the frequency domain guiding signal for the cross-modal dynamic gating mechanism in the core evolution module of the subsequent model.

[0038] The second step is the construction of the wavelet prior-guided panchromatic sharpening model (WPG-Mamba): Based on the acquired multi-source images and prior information, a deep learning network model for image fusion is constructed. The specific construction process includes: (1) Overall network architecture setting The panchromatic sharpening model comprises four main parts: a shallow feature extraction module, a hierarchical local feature injection module, a frequency-aware gated Mamba module, and a feature aggregation and global residual reconstruction module. Specifically, the shallow feature extraction module extracts shallow structural features from the panchromatic image; the hierarchical local feature injection module corresponds to the multispectral branch and compensates for the lack of local inductive bias in micro-texture restoration in the pure state-space model; the frequency-aware gated Mamba module serves as the core fusion engine, utilizing explicit wavelet priors to adjust system dynamics; and the global residual reconstruction module outputs the final high-fidelity fused image.

[0039] (2) Shallow feature extraction module

[0040] The shallow feature extraction module is configured to consist of a series of stacked semi-instance normalized residual blocks. When a high-resolution panchromatic image is input into this module, semi-instance normalization effectively separates and filters out interference information such as illumination changes, while extracting robust shallow local spatial structures and high-frequency texture features, and outputting a shallow feature representation of the panchromatic image.

[0041] (3) Hierarchical Local Feature Injection Module

[0042] For multispectral image processing, a hierarchical local feature injection module is constructed. This module uses the U-Net architecture as the backbone network and includes an encoder, intermediate layers, and a decoder.

[0043] To achieve effective fusion of local priors, this module employs an asymmetric dual-guidance strategy: during the encoder stage, features are downsampled layer by layer and the constructed spatial prior tensor is continuously and explicitly injected. A stable physical structure foundation is established through physical topology and photometric information; in the decoder stage, the skip connection features of the corresponding level of the encoder are combined to upsample layer by layer and explicitly inject wavelet frequency domain prior tensors. The network is guided by panchromatic high-frequency components to recover high-resolution edge and high-frequency details.

[0044] The basic unit of this module is defined as the Feature Modulation Injection Block (FMIB). Unlike traditional direct channel concatenation, FMIB maps the input prior conditions through a lightweight convolutional network, transforming them into spatial scaling and shifting factors in an affine transformation. When the main feature stream passes through this module, it is processed using the formula... Modulation is performed to force the injection of local prior information and achieve non-intrusive alignment. The modulated features are then further processed by group normalization (GroupNorm), the Swish activation function, and a two-layer convolutional residual path to extract local texture, thus constructing a high-quality feature sequence.

[0045] (4) Frequency Domain Aware Gated Mamba Module

[0046] The extracted multispectral deep features and panchromatic shallow features are then fed into a multi-layered frequency domain-aware gated Mamba module for deep global fusion.

[0047] First, a cross-modal interleaved scan is performed: multispectral and panchromatic features are flattened into a one-dimensional sequence in the spatial dimension and then interleaved according to their spatial positions to construct a spectral-spatial mixed sequence. This arrangement allows the state-space model to process the multispectral basis and panchromatic space details sequentially in adjacent time steps.

[0048] Secondly, a global context register mechanism is introduced to address the inherent zero-state initialization bias problem in the standard state-space model (SSM). At the end of the interleaved mixed sequence, a set of learnable global context register tokens is appended, and virtual frequency domain conditions corresponding to the specified dimensions are configured for them. Before scanning the actual image sequence, the SSM first absorbs these register tokens, rapidly warming up the originally zero hidden states and smoothly transitioning them to non-zero steady states containing global statistical distributions. This effectively alleviates feature divergence and long-sequence forgetting in the early stages of the sequence.

[0049] Subsequently, an explicit frequency-domain dynamic gating mechanism is implemented: during the sequence scanning of the SSM, in order to overcome the "spectral dilution" phenomenon caused by the attenuation of high-frequency features due to the small discretization step size when directly scanning smoothed multispectral nodes, the aligned and flattened wavelet frequency-domain prior sequence is injected as an explicit gating signal into the system dynamic parameters of the SSM. In the generative network, when scanning to a smooth multispectral node, the gated network can adaptively amplify the discretization step size by relying on the dynamic adjustment of the panchromatic high-frequency wavelet prior. This forces the hidden states to maintain a high response to high-frequency edges, preventing texture decay and thus achieving manifold alignment of multimodal features in the hidden state space. This evolution process combines a four-directional scanning strategy of top-left to bottom-right, top-right to bottom-left, and its transpose, which captures long-range dependencies across the entire image while completing gradient synchronization and depth interaction of different modal features.

[0050] (5) Feature aggregation and reconstruction

[0051] After deep evolution interaction through multi-layer frequency-domain aware gated Mamba modules, aligned multispectral and panchromatic dual-stream features are output. These two features are then element-wise summed to achieve final feature aggregation. The aggregated features are input to the restoration and optimization module for channel dimensionality reduction and image spatial projection. To further ensure the stability of network reconstruction and accelerate convergence, a global residual connection mechanism is introduced, directly superimposing the initially upsampled multispectral image onto the optimized features output by the restoration module. This forces the network to focus solely on learning high-frequency spatial details and spectral residuals during deep processing, ultimately outputting high-fidelity, high-resolution multispectral image fusion features.

[0052] The third step is training the full-color sharpening model: using the training set to perform end-to-end training on the constructed wavelet prior-guided full-color sharpening model (WPG-Mamba).

[0053] To simultaneously ensure spectral fidelity and accurate reconstruction of spatial structure, this invention employs a multi-component composite loss function. Joint constraints are applied, which include pixel-level reconstruction loss. Structural similarity loss and gradient preservation loss The formula for calculating the total loss function is:

[0054] in, These are hyperparameters used to balance the contributions of different loss components.

[0055] The specific calculation formulas and principles for each of the above loss components are as follows: (1) Pixel-level reconstruction loss is adopted Fusion image generated by norm computation model ground truth image The absolute error between them is used to constrain spectral values ​​to achieve basic spectral fidelity:

[0056] In the formula, This represents the total number of pixels contained in the image. To fuse the image in the first The pixel value at each position; For the first in the real image The pixel value at each position; It is an L1 norm.

[0057] (2) Structural similarity loss is used to enhance the quality of local structure perception, forcing the model to learn the local structural correlations of the image. Its formula is:

[0058] In the formula, The number of image patches; An evaluation function for calculating the brightness, contrast, and structural similarity between two image patches; The first cropped image from the merged image Image blocks; The first cropped from the real image Image blocks of a region.

[0059] (3) Gradient preservation loss is achieved through the Sobel operator. Extracting gradient maps in the horizontal and vertical directions, and explicitly regularizing high-frequency details, echoes the principles of gradient synchronization and manifold alignment in the hidden state space:

[0060] During optimization, the training set data is input into the WPG-Mamba model for forward propagation. The aforementioned composite loss function is calculated, and all weight parameters of the network are updated using backpropagation and gradient descent algorithms, including the system dynamics parameters within the FAGMamba module and the newly introduced learnable global context register parameters. After each training epoch, the model's fusion performance is monitored using a validation set. When the evaluation metric on the validation set reaches its optimum or the loss no longer decreases, training is stopped, and the optimal weight file at this point is saved, thus obtaining the trained WPG-Mamba model.

[0061] The fourth step is the acquisition of the image to be sharpened: In the actual inference and testing phase, low-resolution multispectral images and high-resolution panchromatic images that require high-resolution fusion in the actual scene are acquired. After acquiring the images, the original input images undergo the same preprocessing operations as during the training phase, namely, the multispectral images are scaled to the scale of the panchromatic images using interpolation algorithms, and the corresponding low-frequency and high-frequency components are explicitly extracted using discrete wavelet transform to construct the spatial prior tensor and wavelet frequency domain prior tensor used for inference testing.

[0062] The fifth step is obtaining the high-resolution multispectral image (HRMS): The preprocessed multispectral image to be fused, the panchromatic image, and the corresponding spatial prior tensor and wavelet frequency domain prior tensor are input into the trained WPG-Mamba model with optimal weights. After hierarchical feature injection and frequency domain gated interleaved forward propagation calculations within the model, the final high-resolution multispectral image (HRMS) is directly output end-to-end. This fusion result preserves the low-frequency spectral color information of the multispectral image while faithfully reproducing the clear physical texture and microscopic edge details of the panchromatic image.

[0063] To verify the effectiveness of the remote sensing image panchromatic sharpening method proposed in this invention, this embodiment conducted a large number of experiments on multiple datasets, and the experimental results are as follows.

[0064] Table 1-3 presents the quantitative comparison results on WorldView-II, GaoFen2, and WorldView-III. Bold and underlined lines represent the best and second-best results, respectively. Upward arrows indicate that higher values ​​are better, and downward arrows indicate that lower values ​​are better. Evaluation metrics include: Peak signal-to-noise ratio (PSNR): Measures the difference between the reconstructed image and the target image; Structural similarity (SSIM): assesses the similarity between images based on three aspects: brightness, contrast, and structure. Spectral angle mapping ( ): Evaluate the angular difference between spectral vectors of images; Overall relative global error ( ): Measures the relative error between the reconstructed image and the target image in different bands.

[0065] Table 1: Quantitative Results Based on the WorldView-II Dataset

[0066] As shown in Table 1, in the simulation experiments on the WorldView-II dataset, the method of this invention achieved the best performance on all evaluation metrics. Specifically, the model's PSNR reached 42.5597 dB, an improvement of 0.2477 dB compared to the suboptimal algorithm MMMamba; meanwhile, SSIM reached 0.9734, and SAM and ERGAS decreased to 0.0205 and 0.8498, respectively. This indicates that the method can effectively recover high-frequency spatial details while stably maintaining the spectral fidelity of multispectral images, verifying the effectiveness of the frequency domain gating mechanism.

[0067] Table 2: Quantitative results based on the GaoFen2 dataset

[0068] As shown in Table 2, the proposed method still demonstrates superior generalization ability in the GaoFen2 dataset test. In terms of PSNR, this method achieves the best result at 48.3120 dB, an improvement of approximately 0.32 dB compared to MMMamba. SAM (0.0096) and ERGAS (0.4936) both reached the lowest values ​​among the compared methods, while SSIM (0.9902) tied for first place with MMMamba. These results confirm that the model can robustly achieve manifold alignment of spatial and spectral features when processing data from different sensors.

[0069] Table 3: Quantitative Results Based on the WorldView-III Dataset

[0070] As shown in Table 3, the method of this invention performs competitively in the evaluation of the WorldView-III dataset, but it does not achieve optimal performance on some metrics. Specifically, the method achieves the best ERGAS score of 2.8834, indicating the smallest global fusion error for the overall image; however, it is slightly inferior to MMMamba in PSNR (31.1828 dB), SSIM (0.9280), and SAM (0.0691), ranking second best. This objectively existing performance gap may stem from the more complex spatial distribution or noise characteristics of the WorldView-III dataset itself, which limits the gain of the model when extracting high-frequency wavelet priors. However, its overall performance is still significantly better than the basic PanMamba and traditional CNN models.

[0071] Table 4: Quantitative comparison with existing methods on the full-resolution WorldView-II dataset

[0072] As shown in Table 4, in the evaluation without a reference image, the method of the present invention achieves the best performance in terms of spectral distortion index ( ). It reached an extremely low value of 0.0920 on the spatial distortion index (SDI). The spectral density (SDR) reached 0.1252, and the overall fusion quality index (QNR) reached the best of 0.7968. This indicates that the proposed method has extremely strong spectral preservation, spatial detail fidelity, and overall visual quality in real-world no-reference scenarios.

[0073] Please see Figures 3 to 4As shown in the figure, compared to other mainstream methods that are prone to edge blurring, artifacts, or spectral distortion, the method of this invention has significant advantages in both subjective visual perception and objective error performance. By introducing wavelet frequency domain priors and dynamic gating mechanisms, spectral dilution during multimodal fusion is effectively avoided. This method not only ensures high fidelity of the global spectrum but also accurately restores the microscopic textures and high-frequency details of buildings, roads, etc.

[0074] Example 2

[0075] This embodiment provides a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the remote sensing image panchromatic sharpening method based on a wavelet prior-guided state space model as described in Embodiment 1.

[0076] like Figure 5 As shown, the computer terminal provided in this embodiment includes: at least one processor 101, and a memory 102 connected to at least one processor 101. This embodiment does not limit the specific connection medium between the processor 101 and the memory 102. Figure 5 The example shown is the connection between processor 101 and memory 102 via bus 100. Bus 100 is... Figure 5 The connections between other components are shown in bold lines and are for illustrative purposes only, not as limiting information. Bus 100 can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 5 The bus is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. Alternatively, the processor 101 may also be called a controller; there is no restriction on the name.

[0077] In this embodiment, the memory 102 stores instructions that can be executed by at least one processor 101. The at least one processor 101 can execute the aforementioned method by executing the instructions stored in the memory 102.

[0078] The processor 101 is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 102 and calling data stored in memory 102, the processor can perform various functions and process data, thereby monitoring the device as a whole.

[0079] In one possible design, processor 101 may include one or more processing units. Processor 101 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 101. In some embodiments, processor 101 and memory 102 may be implemented on the same chip; in some embodiments, they may also be implemented on separate chips.

[0080] Processor 101 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the remote sensing image panchromatic sharpening method based on a wavelet prior-guided state-space model disclosed in Embodiment 1 can be directly implemented by the hardware processor, or implemented by a combination of hardware and software modules in processor 101.

[0081] Memory 102, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 102 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 102 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In this embodiment, memory 102 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0082] By designing and programming the processor 101, the code corresponding to the remote sensing image panchromatic sharpening method based on the wavelet prior guided state space model described in the foregoing embodiments can be embedded into the chip, thereby enabling the chip to execute the code during runtime. Figure 1 The steps of the remote sensing image panchromatic sharpening method based on a wavelet prior-guided state-space model are shown. How to design and program the processor 101 is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0083] Example 3

[0084] This embodiment provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it implements the steps of the remote sensing image panchromatic sharpening method based on a wavelet prior-guided state space model as described in Embodiment 1.

[0085] The computer-readable storage medium may include flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage medium may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc., provided on the computer device. Of course, the storage medium may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is typically used to store the operating system and various application software installed on the computer device. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.

[0086] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for panchromatic sharpening of remote sensing images based on a wavelet prior-guided state-space model, characterized in that, Including the following steps: S1. Acquire the low-resolution multispectral image and the high-resolution panchromatic image to be fused, and upsample the multispectral image to the same spatial scale as the panchromatic image; S2. Based on the upsampled multispectral image and the original panchromatic image, extract the spatial prior tensor used to construct the physical structure substrate and the wavelet frequency domain prior tensor used for frequency domain guidance, respectively. S3. A hierarchical local feature injection network is used to extract deep multispectral features from the multispectral image. During the extraction process, the spatial prior tensor and the wavelet frequency domain prior tensor are explicitly injected layer by layer. At the same time, a shallow encoder is used to extract shallow panchromatic features from the panchromatic image. S4. Flatten and interleave the deep multispectral features and shallow panchromatic features, and input them into the multi-layered stacked frequency domain sensing gated state space model. Under the explicit guidance of the wavelet frequency domain prior tensor, dynamically adjust the dynamic parameters of the frequency domain sensing gated state space model to perform multimodal feature interaction and manifold alignment, and output multispectral fusion features and panchromatic fusion features. S5. Perform element-wise aggregation and spatial projection optimization on the multispectral fusion features and panchromatic fusion features, and combine them with the upsampled multispectral image to perform global residual reconstruction to generate a high-resolution multispectral image.

2. The remote sensing image panchromatic sharpening method based on a wavelet prior-guided state-space model according to claim 1, characterized in that, Step S2 specifically includes: S21. The upsampled multispectral image and the original panchromatic image are stitched together along the channel dimension to construct the spatial prior tensor that preserves the physical topological form; S22. The upsampled multispectral image is decomposed using discrete wavelet transform to extract low-frequency approximate components, and the panchromatic image is decomposed to extract high-frequency detail components including horizontal, vertical and diagonal directions; the low-frequency approximate components and high-frequency detail components are concatenated in the channel dimension to construct the wavelet frequency domain prior tensor.

3. The remote sensing image panchromatic sharpening method based on a wavelet prior-guided state-space model according to claim 1, characterized in that, In step S3, the hierarchical local feature injection network is a U-shaped structure containing an encoder and a decoder, and the extraction process specifically includes: In the downsampling stage of the encoder, features are extracted step by step using stacked feature modulation injection blocks, and the spatial prior tensor is explicitly injected at each layer to establish a physical structure basis. Deep within the encoder, a multi-head spatial self-attention mechanism is introduced to aggregate global contextual features; During the upsampling phase of the decoder, the current layer features are concatenated with the corresponding skip connection features from the encoder, and wavelet frequency domain prior tensors adapted to the current resolution are explicitly injected step by step through feature modulation injection blocks to recover high-frequency details.

4. The remote sensing image panchromatic sharpening method based on a wavelet prior-guided state-space model according to claim 3, characterized in that, The feature injection process of the feature modulation injection block specifically includes: The input spatial prior tensor or wavelet frequency domain prior tensor is mapped through a convolutional network to generate scaling and translation factors aligned in both spatial and channel dimensions. Affine modulation of the main feature stream is performed using scaling and translation factors to inject prior information; The affine modulated main features are sequentially processed through group normalization, activation function, and double-layer convolutional residual path for local texture extraction, and the modulated features are output through residual connection.

5. The remote sensing image panchromatic sharpening method based on a wavelet prior-guided state-space model according to claim 1, characterized in that, In step S4, the specific process of inputting deep multispectral features and shallow panchromatic features into the frequency domain sensing gated state space model includes: Deep multispectral features and shallow panchromatic features are flattened into one-dimensional sequences in the spatial dimension and then arranged in an alternating manner according to their spatial positions to construct a spectral-spatial hybrid feature sequence. Flatten the wavelet frequency domain prior tensor in the spatial dimension and copy and interleave it to generate a hybrid frequency domain prior sequence; A set of learnable global context registers is appended to the end of the spectral-spatial hybrid feature sequence to generate an amplified feature sequence. By absorbing the global statistical information in the global context registers, the frequency domain-aware gated state space model is transitioned to a non-zero steady state, thus alleviating the zero-state initialization bias of the frequency domain-aware gated state space model. Configure learnable virtual frequency domain conditions for the global context register and append the virtual frequency domain conditions to the end of the hybrid frequency domain prior sequence to generate the amplified frequency domain prior sequence.

6. The remote sensing image panchromatic sharpening method based on a wavelet prior-guided state-space model according to claim 5, characterized in that, In step S4, the specific process of dynamically adjusting the dynamic parameters of the frequency-domain sensing gated state-space model under the explicit guidance of the wavelet frequency-domain prior tensor includes: A multi-directional scanning strategy is applied to the amplified feature sequence and the amplified frequency domain prior sequence to obtain directional feature sequences in different directions. and directional frequency domain prior sequences ; For each scanning direction, the gating input is calculated using the direction frequency domain prior sequence as an explicit regulator. The calculation formula is: In the formula, A learnable adaptive scaling factor. This represents a linear mapping operation; Gated input The core dynamic parameters for generating a frequency-domain sensing gated state-space model include the discretization step size. and projection matrix and When scanning to a smooth multispectral node, the discretization step size is adaptively amplified by the guidance of the high-frequency detail components of the panchromatic image contained in the wavelet frequency domain prior tensor. ; The system parameters are discretized and the state is updated using the core dynamic parameters, and the sequential response in each direction is output. The sequence responses in all directions are inversely transformed and aggregated, and then the multispectral response sequence and panchromatic response sequence are separated. After being reshaped into a two-dimensional spatial grid, the multispectral features and panchromatic features are updated respectively through residual connections to obtain the multispectral fusion features and the panchromatic fusion features.

7. The remote sensing image panchromatic sharpening method based on a wavelet prior-guided state-space model according to claim 1, characterized in that, The processing steps S3 to S5 are integrated into a full-color sharpening network model; during the end-to-end training phase of the full-color sharpening network model, a composite loss function is used for joint constraints and parameter updates. The formula for calculating the composite loss function is as follows: In the formula, It is a composite loss function; This is used to constrain the pixel-level reconstruction loss for spectral numerical fidelity; For structural similarity loss used to enhance local structure perception; This is used for gradient preservation loss in explicit regularization of high-frequency details; These are hyperparameters used to balance the contributions of different loss components.

8. The remote sensing image panchromatic sharpening method based on a wavelet prior-guided state-space model according to claim 7, characterized in that, Pixel-level reconstruction loss The calculation formula is: In the formula, This represents the total number of pixels contained in the image. To fuse the image in the first The pixel value at each position; For the first in the real image The pixel value at each position; It is an L1 norm; Structural similarity loss The calculation formula is: In the formula, The number of image patches; An evaluation function for calculating the brightness, contrast, and structural similarity between two image patches; The first cropped image from the merged image Image blocks; The first cropped from the real image Image blocks in each region; Gradient Preservation Loss The calculation formula is: In the formula, This is the Sobel operator, used to extract gradient maps in the horizontal and vertical directions.

9. A computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the remote sensing image panchromatic sharpening method based on a wavelet prior-guided state space model as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the remote sensing image panchromatic sharpening method based on a wavelet prior-guided state space model as described in any one of claims 1 to 8.

Images