Remote sensing image panchromatic sharpening method based on multi-scale feature interaction

Abstract

The invention provides a remote sensing image panchromatic sharpening method based on multi-scale feature interaction, and relates to the field of image processing, and the method comprises the steps: obtaining a remote sensing image, and carrying out the preprocessing of the remote sensing image; constructing a panchromatic sharpening model based on multi-scale feature interaction; training a panchromatic sharpening model through the preprocessed remote sensing image; obtaining a remote sensing image to be sharpened; and inputting a remote sensing image to be sharpened into the trained panchromatic sharpening model to realize panchromatic sharpening of the remote sensing image. According to the technical scheme of the invention, the lightweight structure with the efficient parallel modeling capability is introduced into the panchromatic sharpening model, so that the modeling efficiency and the detail reduction capability of the fusion network are improved.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a method for panchromatic sharpening of remote sensing images based on multi-scale feature interaction. Background Technology

[0002] Multi-source remote sensing plays an increasingly important role in key fields such as precision agriculture, resource surveys, environmental monitoring, national defense, and urban planning. Many remote sensing satellites have been launched worldwide, such as Kuaibird, Worldview, Gaofen-1, and Gaofen-2, which have acquired images with different resolutions and spectral bands. However, limited by the satellite's own hardware, remote sensing images are currently generally divided into high spatial resolution panchromatic images (PAN) and low spatial resolution but high spectral resolution multispectral images (MS). To obtain remote sensing images with both high spatial and high spectral resolution, it is usually necessary to fuse the two types of images; this processing method is commonly referred to as panchromatic sharpening of remote sensing images.

[0003] Panchromatic sharpening is a classic image fusion technique for solving this problem. It essentially utilizes the complementary information between panchromatic (PAN) and multispectral (MS) images. This technique combines spatial and spectral information to obtain an ideal high-resolution image. Traditional methods include Brovey, HPF, MTF-GLP, etc. They are widely used and efficient, but they have inherent limitations: (1) they cannot effectively fuse the multi-scale features between the two images, resulting in poor spatial details and spectral information in the fused image; (2) they are difficult to fully preserve the rich texture and high-frequency detail information in the original image.

[0004] Traditional panchromatic sharpening methods for remote sensing images mostly rely on convolutional neural networks (CNNs) to extract image features. While these methods have improved the fusion effect to some extent, they are generally limited to the extraction and fusion of features at a single scale.

[0005] Panchromatic sharpening employs various methods, including convolutional neural networks, generative adversarial networks, and multi-scale delayed-channel attention networks, which to some extent overcomes the limitations of traditional and deep learning-based methods. However, some areas for improvement remain. Firstly, the module's sequential processing of feature transformation, state modeling, and activation operations results in low feature extraction efficiency and high computational resource consumption, making it difficult to meet the real-time inference requirements of high-resolution remote sensing images. Secondly, the single feature interaction path fails to fully integrate information from different scales and multiple modalities, limiting the model's expressive power. While spatial resolution is improved, there is still room for improvement in maintaining spectral consistency, with slight deviations in spectral fidelity in the fused image. Summary of the Invention

[0006] The purpose of this invention is to provide a remote sensing image pancolor sharpening method based on multi-scale feature interaction, in order to solve the problem that existing pancolor sharpening methods are prone to spatial detail loss or spectral information distortion.

[0007] The above-mentioned objective of this application is achieved through the following technical solution: S1: Acquire remote sensing images and perform preprocessing; S2: Construct a full-color sharpening model based on multi-scale feature interaction; S3: Train the panchromatic sharpening model using preprocessed remote sensing images; S4: Obtain the remote sensing image to be sharpened; input the remote sensing image to be sharpened into the trained pancolor sharpening model to achieve pancolor sharpening of the remote sensing image.

[0008] Optionally, step S1 includes: Remote sensing images are physically cropped to corresponding scales; remote sensing images include PAN images and MS images; The cropped MS image is upsampled using bicubic interpolation to obtain an LRMS image; Size matching, feature extraction, and image enhancement were performed on LRMS images and PAN images.

[0009] Optionally, step S2 includes: The full-color sharpening model includes: a multi-scale attention feature extraction module, a multi-scale attention cross-layer fusion module, a cross-scale feature fusion enhancement module, and an image reconstruction module; The multi-scale attention feature extraction module, the multi-scale attention cross-layer fusion module, the cross-scale feature fusion enhancement module, and the image reconstruction module are connected in sequence.

[0010] Optionally, step S2 includes: The image reconstruction module adopts a top-down decoding structure, consisting of multiple decoding units, to gradually restore the spatial resolution and detail texture of the image. Each decoding unit mainly consists of the following three parts: convolution operation, BatchNorm image processing domain normalization layer, and ReLU activation function; The decoding unit includes multiple downsampled deconvolution channels in the image processing domain s. Each channel contains a kernel size of k, n channels, a stride of s, and a display dimension of s. .

[0011] Optionally, the multi-scale attention feature extraction module includes: a state space modeling module, a bidirectional interaction module, a residual unit, a feature splicing unit, and a wavelet transform unit; Feature extraction paths were designed for PAN and LRMS images respectively, and input features were processed accordingly. After initial extraction of shallow features, denoted as and , corresponding to local feature branches and global feature branches; We extract detailed features from the regular spatial context and capture structural semantic information over a wider range. The features extracted from branches at different scales are then fused using a channel-dimensional concatenation operation, as follows:

[0012] in, , These represent feature branches at different scales. Extracting regular spatial context, Used to model global information across channels; Fusion features The input state space modeling module further enhances the ability to model spatiotemporal dependencies between features across channels and receptive fields. Its state update formula is as follows:

[0013]

[0014] in, Input features for the current step, It is in a hidden state. The state output results, A, B, and C are learnable parameter matrices; The residual connection unit is used to integrate the input features of the multi-scale attention feature extraction module. With output features Adding them together yields the final feature. .

[0015] Optionally, the multi-scale attention cross-layer fusion module includes: an attention transformation function S and a fusion unit; The multi-scale attention cross-layer fusion module is used to integrate input features. The input features are obtained through the attention transformation function S and the multi-scale attention cross-layer fusion module. Relationship modeling is performed to obtain the fused feature representation. ,as follows:

[0016] in and These represent feature weighting functions for different channels or spatial dimensions. It is an identity matrix used to preserve the original feature information in the main channel; Representing input features This is represented as a hidden state during computation; Introducing a feature fusion enhancement mechanism to... Residual enhancement is performed, and the final intermediate state features are constructed as follows:

[0017] in I is a linear compression transformation function used to further compress the channel dimension and improve the fusion expression capability; I is also an identity mapping matrix used to compensate for features.

[0018] Optionally, the cross-scale feature fusion enhancement module includes: a normalization module, a multi-layer neural network module, a state space module, an activation function unit, a one-dimensional convolution module, a splicing unit, a reshaping unit, a depthwise separable convolution, and a flattening unit; The multilayer neural network module includes: a first multilayer neural network, a second multilayer neural network, a third multilayer neural network, and a fourth multilayer neural network; the one-dimensional convolution module includes: a first one-dimensional convolution and a second one-dimensional convolution; the state space module includes: a first state space model and a second state space model; and the normalization module connects to the multilayer neural network module. The first multi-layer neural network, the first one-dimensional convolution, and the first state space model are connected sequentially. The second multilayer neural network connects the activation function unit; The third multi-layer neural network, the second one-dimensional convolution, and the second state space model are connected sequentially. The activation function unit, the state space module, and the fourth multilayer neural network connection and splicing unit are connected sequentially; the reshaping unit, the depthwise separable convolution, and the flattening unit are connected in sequence. The fourth multilayer neural network connects the reshaping units through splicing units; Introducing structure preservation function The residual connection improves the cross-scale feature fusion enhancement module, resulting in the final fused features. .

[0019] An electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to enable the electronic device to perform a remote sensing image panchromatic sharpening method based on multi-scale feature interaction.

[0020] A computer-readable storage medium storing instructions that, when executed, perform a remote sensing image panchromatic sharpening method based on multi-scale feature interaction.

[0021] The beneficial effects of the technical solution provided in this application are: This invention centers on multi-scale feature interaction and incorporates a more efficient spatial sequence modeling mechanism. It primarily leverages the dynamic modeling capabilities of parallel structures to synchronously perform state modeling, convolution extraction, and gating mechanisms within feature paths, significantly improving inference speed. Combined with a dual-path information exchange strategy involving both channels and space, it achieves cross-scale fusion and structural alignment between images of different resolutions, enhancing information consistency among multimodal images. An edge-aware mechanism is introduced during the fusion stage to strengthen the representation of image details, edges, and textures, improving the spatial resolution and spectral fidelity of the fused image. While maintaining the advantages of the original multi-scale feature interaction structure, the introduction of a more efficient feature modeling module and information fusion strategy effectively alleviates problems such as low efficiency of serial modeling, insufficient multi-scale information interaction, and weak high-frequency recovery capabilities, significantly improving the overall performance in remote sensing image panchromatic sharpening tasks. Attached Figure Description

[0022] The present application will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a step diagram of an embodiment of this application; Figure 2 This is an overall framework diagram of the embodiments of this application; Figure 3 This is a diagram of the multi-scale attention feature extraction module in an embodiment of this application; Figure 4 This is a module structure diagram in the embodiments of this application; Figure 5 This is a schematic diagram of the electronic device structure in the embodiments of this application. Detailed Implementation

[0023] To provide a clearer understanding of the technical features, objectives, and effects of this application, the specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0024] The embodiments of this application provide a method for panchromatic sharpening of remote sensing images based on multi-scale feature interaction.

[0025] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating the steps of a remote sensing image panchromatic sharpening method based on multi-scale feature interaction in an embodiment of this application, including: S1: Acquire remote sensing images and perform preprocessing; S2: Construct a full-color sharpening model based on multi-scale feature interaction; S3: Train the panchromatic sharpening model using preprocessed remote sensing images; S4: Obtain the remote sensing image to be sharpened; input the remote sensing image to be sharpened into the trained pancolor sharpening model to achieve pancolor sharpening of the remote sensing image.

[0026] This application provides an embodiment as follows: the present invention uses L1 loss as the main training objective to directly constrain the pixel-level differences between the generated high-resolution fused MS image and the real label. Finally, the discriminator loss of the network is as follows:

[0027] in The total number of samples, Indicates the first High-resolution MS image labels for each sample; Indicates the network output of the first A high-resolution fused image. This loss function can directly optimize the reconstruction error between the fused image and the true label, helping to improve the subjective and objective quality of the fused image.

[0028] This invention proposes a multi-scale feature interaction-based full-color sharpening network architecture to address the shortcomings of existing technologies that often employ single-scale or fixed receptive field image modeling, neglecting the coupling between spatial structure and contextual semantics at different scales. Compared to existing methods that rely solely on shallow feature alignment or fixed transformations for information fusion, this invention uses a multi-branch, multi-scale attention feature extraction module to segment and heterogeneously model the input MS and PAN images. It introduces local (small convolution) and global (large convolution) dual-scale convolutional branches, effectively enhancing the model's ability to perceive spatial details and edge textures. Furthermore, it introduces a cross-layer state modeling mechanism, combining spatial weight guidance and cross-scale feature interaction to mitigate interference from shallow high-frequency information and enhance the discriminative stability of deep semantic information. This module can improve the image contextual relationship modeling ability while preserving the original structural information, thereby obtaining more consistent and stable intermediate representation features. This invention designs a fusion enhancement mechanism to uniformly represent and model multi-source heterogeneous features, and further compresses channel redundancy through feature residual connections and a structure preservation module, improving spatial structure fidelity. Finally, the proposed image reconstruction module employs a combination of multi-layer separable convolutions and skip connections, which can better recover detailed regions and achieve a clearer and sharper edge restoration effect. In summary, the multi-stage panchromatic method proposed in this invention has stronger feature representation capabilities and cross-scale modeling performance, and can obtain better detail clarity, spatial consistency, and structural restoration quality in the process of panchromatic image reconstruction. It has significant advantages and broad application prospects in remote sensing image fusion tasks.

[0029] Step S1 includes: Remote sensing images are physically cropped to corresponding scales; remote sensing images include PAN images and MS images; As one example, the original remote sensing images are very large. For example, the PAN and MS images of Gaofen-2 are approximately 30000×30000 and 7500×7500 pixels, respectively. The size is too large to be directly input into the neural network for training. In the preprocessing stage, these images are first cropped into small patches. The MS image is cropped to 64×64, and the PAN image is cropped to 256×256.

[0030] The cropped MS image is upsampled using bicubic interpolation to obtain an LRMS image; Size matching, feature extraction, and image enhancement were performed on LRMS images and PAN images.

[0031] As one example, feature extraction is performed on LRMS and PAN images through convolutional layers, and preprocessing is carried out using some basic image enhancement techniques such as histogram equalization, thereby improving image quality and enhancing the fusion effect in subsequent stages.

[0032] Step S2 includes: The full-color sharpening model includes: a multi-scale attention feature extraction module, a multi-scale attention cross-layer fusion module, a cross-scale feature fusion enhancement module, and an image reconstruction module; The multi-scale attention feature extraction module, the multi-scale attention cross-layer fusion module, the cross-scale feature fusion enhancement module, and the image reconstruction module are connected in sequence.

[0033] As one example, from an overall structural perspective, the network employs a multi-scale parallel feature extraction structure, performing deep feature representation on both the input PAN image and the upsampled MS image to more fully preserve spatial details and spectral characteristics. The feature extraction stage uses a multi-scale feature extraction module, guiding the network to simultaneously model both detailed and semantic features through combinations of convolutional kernels at different scales, thereby enhancing image representation capabilities.

[0034] In one embodiment, the feature fusion stage processes channel features at different scales into multiple branches, with each branch employing different receptive field sizes and convolution combinations to improve the modeling effect of image elements such as texture, edges, and contours. The fused features are then integrated and passed to the reconstruction module.

[0035] Step S2 includes: The image reconstruction module adopts a top-down decoding structure, consisting of multiple decoding units, to gradually restore the spatial resolution and detail texture of the image. Each decoding unit mainly consists of the following three parts: convolution operation, BatchNorm image processing domain normalization layer, and ReLU activation function; The decoding unit includes multiple downsampled deconvolution channels in the image processing domain s. Each channel contains a kernel size of k, n channels, a stride of s, and a display dimension of s. .

[0036] As one embodiment, the multi-scale feature maps output by the image reconstruction module are integrated and then input into the image reconstruction module. The image reconstruction module adopts a top-down decoding structure, with multiple decoding units gradually restoring the spatial resolution and detailed texture of the image.

[0037] Each decoding unit mainly consists of three parts: convolution operations, a BatchNorm image processing domain normalization layer, and a ReLU activation function. Each decoding unit achieves spatial reconstruction through progressive upsampling operations and preserves shallow structural information using residual connections. The decoding module includes multiple downsampled deconvolution channels, each containing a kernel size of k, n channels, a stride of s, and a display dimension of s. This structure effectively enhances the expressive power of the reconstruction stage while controlling the computational complexity of the network. The final reconstruction module progressively restores the multi-scale fused features into a high-resolution image output, providing a structural foundation for subsequent high-quality image generation.

[0038] The multi-scale attention feature extraction module includes: a state space modeling module, a bidirectional interaction module, a residual unit, a feature splicing unit, and a wavelet transform unit; The multi-scale attention feature extraction module aims to fully exploit the spatial details in remote sensing images. Balancing efficiency and modeling capabilities, it achieves effective feature encoding through an improved multi-scale state-space model. The module employs a main branch structure, using convolution and state-space modeling operations to extract local features in the main branch. Simultaneously, a cross-scale receptive field structure is introduced to fuse image feature information from different scales, thereby enhancing the model's expressive power.

[0039] Feature extraction paths were designed for PAN images and LRMS images respectively, and input features were processed. After initial extraction of shallow features, denoted as and Corresponding to local and global feature branches, they respectively extract detailed features from the general spatial context and capture a wider range of structural semantic information. Internally, the features extracted from branches at different scales are fused through channel-dimensional concatenation, as follows:

[0040] in, , These represent feature branches at different scales. Extracting regular spatial context, Used to model global information across channels; Fusion features The input state space modeling module further enhances the ability to model spatiotemporal dependencies between features across channels and receptive fields. Its state update formula is as follows:

[0041]

[0042] in, Input features for the current step, It is in a hidden state. The state output results, A, B, and C are learnable parameter matrices; The residual connection unit is used to integrate the input features of the multi-scale attention feature extraction module. With output features Adding them together yields the final feature. .

[0043] In one specific embodiment of this application, the state-space modeling module possesses efficient long-dependency modeling capabilities, making it particularly suitable for perceiving complex textures and structures in remote sensing images. The model also employs a residual connection strategy, that is, input features... With output features Adding them together yields the final feature. This enhances the stability of information expression. The output will also serve as input, passed to subsequent modules (such as a bidirectional interaction module) to achieve further fusion processing of multimodal features.

[0044] The multi-scale attention cross-layer fusion module includes: an attention transformation function S and a fusion unit; In one specific embodiment of this application, to achieve effective cross-module transfer of deep features and enhance the model's ability to integrate image spatial structure and semantic information, this invention introduces a multi-scale attention cross-layer fusion module. This module primarily uses Local-Global Joint Modeling to effectively connect and update the multi-scale feature maps obtained during the encoding stage. The module's design draws on the information weighting concept in attention mechanisms, while also incorporating structural similarity information between features to improve spatial context modeling capabilities.

[0045] The multi-scale attention cross-layer fusion module is used to integrate input features. The input features are obtained through the attention transformation function S and the multi-scale attention cross-layer fusion module. Relationship modeling is performed to obtain the fused feature representation. ,as follows:

[0046] in and These represent feature weighting functions for different channels or spatial dimensions. It is an identity matrix used to preserve the original feature information in the main channel; Representing input features This is represented as a hidden state during computation; In one specific embodiment of this application, the multi-scale attention cross-layer fusion module itself is an iterative structure that receives different input features at different time steps (t=1,2,…). and ), and output the fused features. . It is the input features of the multi-scale attention cross-layer fusion module at the current time, which come from other parts of the same network (the output of the previous layer or different branches of the same layer).

[0047] This application provides an embodiment as follows: to further enhance the stability of feature representation and the ability to model context structure, this paper introduces a feature fusion enhancement mechanism.

[0048] Introducing a feature fusion enhancement mechanism to... Residual enhancement is performed, and the final intermediate state features are constructed as follows:

[0049] in I is a linear compression transformation function used to further compress the channel dimension and improve the fusion expression capability; I is also an identity mapping matrix used to compensate for features.

[0050] This application provides an embodiment as follows: the final generated features will be used as input to the cross-channel modeling and fusion module in the subsequent module to complete the information unification of deep and shallow features and the alignment of context structure, thereby providing more robust feature support for image reconstruction.

[0051] The cross-scale feature fusion enhancement module includes: a normalization module, a multi-layer neural network module, a state space module, an activation function unit, a one-dimensional convolution module, a splicing unit, a reshaping unit, a depthwise separable convolution, and a flattening unit; The multilayer neural network module includes: a first multilayer neural network, a second multilayer neural network, a third multilayer neural network, and a fourth multilayer neural network; the one-dimensional convolution module includes: a first one-dimensional convolution and a second one-dimensional convolution; the state space module includes: a first state space model and a second state space model; and the normalization module connects to the multilayer neural network module. The first multi-layer neural network, the first one-dimensional convolution, and the first state space model are connected sequentially. The second multilayer neural network connects the activation function unit; The third multi-layer neural network, the second one-dimensional convolution, and the second state space model are connected sequentially. The activation function unit, the state space module, and the fourth multilayer neural network connection and splicing unit are connected sequentially; the reshaping unit, the depthwise separable convolution, and the flattening unit are connected in sequence. The fourth multilayer neural network connects the reshaping units through splicing units; This application provides an embodiment as follows: To achieve a unified representation of features at different scales and enhance the model's comprehensive modeling ability for image structural details and contextual semantics, this paper designs a cross-scale feature fusion enhancement module. This mechanism comprehensively considers the balance between local convolutional receptive fields and global modeling, effectively improving the unified representation ability of cross-scale features.

[0052] Multi-scale attention cross-layer fusion module for feature representation at different scales or different levels Perform cascading operations, and through 1 Convolution compresses the channel dimension to generate fused features. as follows:

[0053] Let represent the intermediate state features extracted from the i-th branch, and let represent the cascading operation performed along the channel dimension. This indicates a cascading operation at the channel level. To compress the channel dimension The product operation is used to fuse multi-scale information and suppress redundancy.

[0054] As one embodiment, in order to further enhance the ability of fused features to express edge structural information while suppressing structural degradation in blurred regions, this invention introduces a structure preservation function. Furthermore, residual connection methods are employed to enhance feature reconstruction capabilities.

[0055] Introducing structure preservation function The residual connection improves the cross-scale feature fusion enhancement module, resulting in the final fused features. .

[0056] As one example, This represents a set of structure-preserving modules with lightweight convolutional properties, used to enhance the expressive power of image texture and edge structures. These modules can be optimized based on attention mechanisms, thereby improving the discriminative power and accuracy of the fused features. Finally, the fused features... It will serve as input to subsequent image reconstruction modules, providing effective support for generating high-resolution images with consistent structure and semantic coherence.

[0057] As shown in Table 1, our method was tested on the WorldView-II dataset. Compared with existing methods (SFIM, PanNet, MSDCNN, and PanMamba), our method achieved the best results in all four key metrics. It achieved a PSNR of 42.8752, higher than PanMamba's 42.2354; a PSNR of 0.9758, higher than PanMamba's 0.9726; a PSNR of 0.0198, lower than PanMamba's 0.0212; and a PSNR of 0.9337, higher than PanMamba's 0.9229. These data fully demonstrate the superiority of our proposed method in preserving spatial details and spectral information, especially in the design of the multi-scale feature interaction and structure-preserving module G(·), which effectively improves the fusion effect. Table 1

[0058] This application also discloses an electronic device. (See reference...) Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0059] The communication bus 502 is used to enable communication between these components.

[0060] The user interface 503 may include a display screen, and optionally, the user interface 503 may also include a standard wired interface or a wireless interface.

[0061] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0062] This application also discloses a computer-readable storage medium storing multiple instructions adapted for loading by a processor to execute the aforementioned method for panchromatic sharpening of remote sensing images based on multi-scale feature interaction.

[0063] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure.

[0064] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for panchromatic sharpening of remote sensing images based on multi-scale feature interaction, characterized in that, The method includes the following steps: S1: Acquire remote sensing images and perform preprocessing; S2: Construct a full-color sharpening model based on multi-scale feature interaction; S3: Train the panchromatic sharpening model using preprocessed remote sensing images; S4: Obtain the remote sensing image to be sharpened; input the remote sensing image to be sharpened into the trained pancolor sharpening model to achieve pancolor sharpening of the remote sensing image.

2. The remote sensing image panchromatic sharpening method based on multi-scale feature interaction as described in claim 1, characterized in that, Step S1 includes: Remote sensing images are physically cropped to corresponding scales; remote sensing images include PAN images and MS images; The cropped MS image is upsampled using bicubic interpolation to obtain an LRMS image; Size matching, feature extraction, and image enhancement were performed on LRMS images and PAN images.

3. The remote sensing image panchromatic sharpening method based on multi-scale feature interaction as described in claim 2, characterized in that, Step S2 includes: The full-color sharpening model includes: a multi-scale attention feature extraction module, a multi-scale attention cross-layer fusion module, a cross-scale feature fusion enhancement module, and an image reconstruction module; The multi-scale attention feature extraction module, the multi-scale attention cross-layer fusion module, the cross-scale feature fusion enhancement module, and the image reconstruction module are connected in sequence.

4. The remote sensing image panchromatic sharpening method based on multi-scale feature interaction as described in claim 3, characterized in that, Step S2 includes: The image reconstruction module adopts a top-down decoding structure, consisting of multiple decoding units, to gradually restore the spatial resolution and detail texture of the image. Each decoding unit mainly consists of the following three parts: convolution operation, BatchNorm image processing domain normalization layer, and ReLU activation function; The decoding unit includes multiple downsampled deconvolution channels in the image processing domain s. Each channel contains a kernel size of k, n channels, a stride of s, and a display dimension of s. .

5. The remote sensing image panchromatic sharpening method based on multi-scale feature interaction as described in claim 3, characterized in that, The multi-scale attention feature extraction module includes: a state space modeling module, a bidirectional interaction module, a residual unit, a feature splicing unit, and a wavelet transform unit; Feature extraction paths were designed for PAN and LRMS images respectively, and input features were processed accordingly. After initial extraction of shallow features, denoted as and , corresponding to local feature branches and global feature branches; We extract detailed features from the regular spatial context and capture structural semantic information over a wider range. The features extracted from branches at different scales are then fused using a channel-dimensional concatenation operation, as follows: in, , These represent feature branches at different scales. Extracting regular spatial context, Used to model global information across channels; Fusion features The input state space modeling module further enhances the ability to model spatiotemporal dependencies between features across channels and receptive fields. Its state update formula is as follows: in, Input features for the current step, It is in a hidden state. The state output results, A, B, and C are learnable parameter matrices; The residual connection unit is used to integrate the input features of the multi-scale attention feature extraction module. With output features Adding them together yields the final feature. .

6. The remote sensing image panchromatic sharpening method based on multi-scale feature interaction as described in claim 3, characterized in that, The multi-scale attention cross-layer fusion module includes: an attention transformation function S and a fusion unit; The multi-scale attention cross-layer fusion module is used to integrate input features. The input features are obtained through the attention transformation function S and the multi-scale attention cross-layer fusion module. Relationship modeling is performed to obtain the fused feature representation. ,as follows: in and These represent feature weighting functions for different channels or spatial dimensions. It is an identity matrix used to preserve the original feature information in the main channel; Representing input features This is represented as a hidden state during computation; Introducing a feature fusion enhancement mechanism to... Residual enhancement is performed, and the final intermediate state features are constructed as follows: in I is a linear compression transformation function used to further compress the channel dimension and improve the fusion expression capability; I is also an identity mapping matrix used to compensate for features.

7. The remote sensing image panchromatic sharpening method based on multi-scale feature interaction as described in claim 3, characterized in that, The cross-scale feature fusion enhancement module includes: a normalization module, a multi-layer neural network module, a state space module, an activation function unit, a one-dimensional convolution module, a splicing unit, a reshaping unit, a depthwise separable convolution, and a flattening unit; The multilayer neural network module includes: a first multilayer neural network, a second multilayer neural network, a third multilayer neural network, and a fourth multilayer neural network; the one-dimensional convolution module includes: a first one-dimensional convolution and a second one-dimensional convolution; the state space module includes: a first state space model and a second state space model; and the normalization module connects to the multilayer neural network module. The first multi-layer neural network, the first one-dimensional convolution, and the first state space model are connected sequentially. The second multilayer neural network connects the activation function unit; The third multi-layer neural network, the second one-dimensional convolution, and the second state space model are connected sequentially. The activation function unit, the state space module, and the fourth multilayer neural network connection and splicing unit are connected sequentially; the reshaping unit, the depthwise separable convolution, and the flattening unit are connected in sequence. The fourth multilayer neural network connects the reshaping units through splicing units; Introducing structure preservation function The residual connection improves the cross-scale feature fusion enhancement module, resulting in the final fused features. .

8. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a computer, perform the method as described in any one of claims 1-7.

Images