Remote sensing image panchromatic sharpening method based on spatial-spectral information enhancement

Abstract

The invention provides a remote sensing image panchromatic sharpening method based on spatial-spectral information enhancement, and the method comprises the steps: carrying out the up-sampling operation of a low-resolution multispectral image, obtaining an up-sampled multispectral image, carrying out the feature extraction of the up-sampled multispectral image and a panchromatic image through dynamic deformable convolution, and carrying out the feature extraction of the panchromatic image; the spectral features of the low-resolution multispectral image and the spatial features of the panchromatic image are respectively obtained, and refined spectral features and refined spatial features are respectively obtained through fine-tuning triple attention mechanism and residual network block processing. According to the method, the shape and the weight of the convolution kernel can be dynamically adjusted according to the input image through dynamic deformable convolution, so that the adaptability of the network to different scale features is improved in the feature extraction and fusion process, the adaptability enables the network to more effectively capture image spectrum and spatial features, and the generalization ability of the network is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a remote sensing image panchromatic sharpening method based on spatial-spectral information enhancement. Background Art

[0002] Pan-sharpening of remote sensing images is a key technique in remote sensing image processing. Its purpose is to fuse multispectral and panchromatic images acquired by remote sensing satellites to produce a high-resolution, spectrally rich multispectral image. This technique is widely used in remote sensing Earth observations, including monitoring land loss, geological disasters, and national defense security. Despite significant progress in recent years, various issues remain to be addressed due to existing methods, such as spectral distortion and spatial deformation.

[0003] Deep learning methods based on neural networks have demonstrated superior performance in pan-sharpening remote sensing images. This proposed neural network-based method achieves dual fidelity in pan-sharpening remote sensing images, preserving both spectral and spatial information. First, a neural network is used to extract the spectral features of multispectral images and the spatial features of panchromatic images. Then, based on these features and a loss function, a high-resolution multispectral image is learned.

[0004] While deep learning methods based on neural networks have made significant progress in pan-sharpening remote sensing images, they still face limitations: These methods often overlook the impact of image scale variations and complex texture features. In practical applications, the diversity of object sizes and the complex texture structures in scenes (such as densely packed similar objects and irregular surface textures) require the network to dynamically adjust feature extraction and fusion. Without this adaptive mechanism, the network will struggle to accurately capture feature variations at different scales, significantly reducing the model's generalization performance in cross-modal dataset applications.

[0005] Specifically, these limitations are manifested in three main aspects: First, traditional bilinear / bicubic interpolation methods ignore panchromatic image gradient information, resulting in blurred edges in the reconstructed image, and fixed-scale convolution kernels are difficult to adapt to the needs of multi-scale panchromatic sharpening. Second, static feature fusion strategies cannot dynamically adjust feature weights based on texture complexity. Finally, existing dual-branch architectures use simple addition or splicing operations, which cannot dynamically balance the contributions of spectral and spatial features, resulting in spectral distortion in homogeneous areas such as farmland. These factors jointly restrict the model's robustness in real-world complex scenes. Summary of the Invention

[0006] In view of the above situation, the main purpose of the present invention is to propose a remote sensing image panchromatic sharpening method based on spatial-spectral information enhancement to solve the above technical problems.

[0007] The present invention proposes a remote sensing image pan-sharpening method based on spatial-spectral information enhancement, which comprises the following steps: Step 1: A feature extraction module is constructed based on dynamic deformable convolution, normalization, and ReLU activation function. A coupling mechanism is introduced into the triple attention mechanism to obtain a fine-tuned triple attention mechanism. A feature fusion module is constructed based on the residual dense block, gating mechanism, and Sobel operator. The feature extraction module, the fine-tuned triple attention mechanism, the residual network block, and the feature fusion module constitute a remote sensing image enhancement fusion model. Step 2: Acquire a remote sensing image, perform an upsampling operation on the low-resolution multispectral image in the remote sensing image to obtain an upsampled multispectral image, perform feature extraction on the upsampled multispectral image and the panchromatic image in the remote sensing image, respectively, and stitch them together to obtain a feature map of the low-resolution multispectral image and the panchromatic image; Step 3: Using a feature extraction module to perform feature extraction on the upsampled multispectral image and the panchromatic image, respectively, to obtain the spectral features of the low-resolution multispectral image and the spatial features of the panchromatic image; Step 4: The feature maps of the concatenated low-resolution multispectral image and the panchromatic image are processed by the fine-tuned triple attention mechanism and residual network block to obtain refined spectral features and refined spatial features respectively; Step 5: Process the refined spectral features and refined spatial features using a feature fusion module to obtain a high-resolution multispectral image; Step 6. Based on the high-resolution multispectral image, the perception loss function and the pixel loss function are constructed respectively. The remote sensing image enhancement fusion model is optimized using the perception loss function and the pixel loss function to obtain the optimized remote sensing image enhancement fusion model. The remote sensing image is input into the optimized remote sensing image enhancement fusion model for processing to obtain the final high-resolution multispectral image.

[0008] Compared with the prior art, the present invention has the following beneficial effects: 1. The dynamic deformable convolution constructed by the present invention can dynamically adjust the shape and weight of the convolution kernel according to the input image, thereby improving the network's adaptability to features of different scales during feature extraction and fusion. This adaptability enables the network to more effectively capture image details and edge features, thereby improving its generalization ability; 2. This invention trains the model by constructing a pixel loss function and a perceptual loss function, enabling the model to better capture the structure and content of the image at the feature level while fine-tuning the image pixel details. This approach enhances the model's adaptability to different types of remote sensing images and reduces the risk of overfitting. 3. The remote sensing image enhancement fusion model proposed in the present invention enhances the spatial and spectral information of the remote sensing image separately before image fusion, and then fuses the enhanced information, thereby improving the generalization ability of the model. At the same time, by utilizing the full-color image gradient to enhance the high-frequency details of the fused features, it overcomes the edge blur problem in traditional image reconstruction and significantly improves the image edge clarity.

[0009] 4. By introducing a coupling mechanism into the triple attention mechanism, the present invention can accurately distinguish subtle differences between different channels, especially in areas with similar space but weak spectral differences. At the same time, it effectively reduces the excessive attention of a single dimension to specific features, suppresses background interference and artifacts, and effectively maintains the spectrum of the source image, thereby improving the accuracy and generalization ability of feature expression.

[0010] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flowchart of the steps of a remote sensing image pan-sharpening method based on spatial-spectral information enhancement proposed by the present invention; Figure 2 This is a method architecture diagram of a remote sensing image panchromatic sharpening method based on spatial-spectral information enhancement proposed in the present invention. DETAILED DESCRIPTION

[0012] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0013] These and other aspects of the embodiments of the present invention will become clear with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to illustrate some ways of implementing the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0014] See also Figure 1This embodiment provides a remote sensing image pan-sharpening method based on spatial-spectral information enhancement, the method comprising the following steps: Step 1: A feature extraction module is constructed based on dynamic deformable convolution, normalization, and ReLU activation function. A coupling mechanism is introduced into the triple attention mechanism to obtain a fine-tuned triple attention mechanism. A feature fusion module is constructed based on the residual dense block, gating mechanism, and Sobel operator. The feature extraction module, the fine-tuned triple attention mechanism, the residual network block, and the feature fusion module constitute a remote sensing image enhancement fusion model. It should be noted that if Figure 2 As shown in the figure, the remote sensing image enhancement fusion model proposed in this paper extracts and fuses features through spectral enhancement branches and spatial enhancement branches, respectively. This branch structure enables the model to effectively utilize spatial and spectral information in low-resolution multispectral images and panchromatic images, thereby enhancing the model's expressiveness and adaptability. In the spectral enhancement branch, a fine-tuned triple attention mechanism is employed to enable the model to adaptively focus on important regions and channels of features, improving the effectiveness and accuracy of feature selection and adapting it to different scenarios. The residual network block used in the spatial enhancement branch effectively fuses features, reduces information loss, and promotes deep feature learning.

[0015] In addition, since the convolution kernel setting of dynamic deformable convolution is more flexible and can adapt to different pixel and image features, this paper uses dynamic deformable convolution to extract image features in the spectral and spatial enhancement branches, making the model more robust when processing images of different scales.

[0016] Step 2: Acquire a remote sensing image, perform an upsampling operation on the low-resolution multispectral image in the remote sensing image to obtain an upsampled multispectral image, perform feature extraction on the upsampled multispectral image and the panchromatic image in the remote sensing image, respectively, and stitch them together to obtain a feature map of the low-resolution multispectral image and the panchromatic image.

[0017] See also Figure 2 In step 2, a remote sensing image is acquired, and an upsampling operation is performed on the low-resolution multispectral image in the remote sensing image to obtain an upsampled multispectral image. Feature extraction is performed on the upsampled multispectral image and the panchromatic image in the remote sensing image, and the two images are stitched together to obtain a feature map of the low-resolution multispectral image and the panchromatic image. The specific steps include the following: Obtain a remote sensing image, and process the low-resolution multispectral image in the remote sensing image through deconvolution operation and ReLU activation function in sequence to obtain an upsampled multispectral image. The following relationship exists in the corresponding process: ; in, represents the upsampled multispectral image, Indicates that it has been processed by the ReLU activation function. Indicates that after deconvolution operation, Represents a low-resolution multispectral image; The upsampled multispectral image is processed by convolution operation and ReLU activation function in sequence to obtain the initial spectral features of the low-resolution multispectral image. The following relationship exists in the corresponding process: ; in, represents the initial spectral characteristics of the low-resolution multispectral image, Indicates that it has undergone convolution operation; The full-color image in the remote sensing image is processed by convolution operation and ReLU activation function in sequence to obtain the initial spatial features of the full-color image. The following relationship exists in the corresponding process: ; in, represents the initial spatial features of the full-color image, represents a full-color image; The initial spectral features of the low-resolution multispectral image are spliced ​​with the initial spatial features of the panchromatic image to obtain the feature map of the low-resolution multispectral image and the panchromatic image. The following relationship exists in the corresponding process: ; in, Represents the feature map after the low-resolution multispectral image and the panchromatic image are stitched together, Indicates that a splicing operation has been performed.

[0018] It should be noted that in Figure 2 middle, Indicates addition.

[0019] Step 3: Use the feature extraction module to perform feature extraction on the upsampled multispectral image and the panchromatic image respectively, so as to obtain the spectral features of the low-resolution multispectral image and the spatial features of the panchromatic image respectively.

[0020] In step 3, feature extraction is performed on the upsampled multispectral image and the panchromatic image using a feature extraction module to obtain the spectral features of the low-resolution multispectral image and the spatial features of the panchromatic image, respectively. The steps include the following: The upsampled multispectral image is processed sequentially through dynamic deformable convolution operation, normalization operation and ReLU activation function to obtain the spectral characteristics of the low-resolution multispectral image. The following relationship exists in the corresponding process: ; in, represents the spectral characteristics of low-resolution multispectral images, After normalization operation, Indicates that it has undergone dynamic deformable convolution operation; The full-color image is processed by dynamic deformable convolution operation, normalization operation and ReLU activation function in sequence to obtain the spatial features of the full-color image. The following relationship exists in the corresponding process: ; in, Represents the spatial characteristics of a full-color image.

[0021] Furthermore, the dynamic deformable convolution operation includes the following sub-steps: Generate a dynamic offset based on the spatial structure of the input feature map. Use a learnable offset field to adaptively adjust the sampling position according to the dynamic offset to obtain the offset convolution position. Use bilinear interpolation to achieve continuous sampling and obtain the weight of the offset position. The following relationship exists in the corresponding process: ; in, Indicates the pixel value of the offset position, represents the convolution position after the shift, represents the center position of the convolution kernel, Represents the first sampling locations, Indicates the index of the sampling point, Represents the first Dynamic offset of positions, represents the interpolation weight, Indicates the first neighboring pixels, Indicates that the input feature map is at position Pixel value of The softmax activation function is used to calculate the weight of the convolution kernel. The following relationship exists in the corresponding process: ; in, Indicates the The weight of the convolution kernel, represents the index of the convolution kernel, Represents the weight parameters generated by the dynamic perception mechanism; The output of the convolution kernel is weighted and fused with the weight of the convolution kernel to obtain the output feature map. The following relationship exists in the corresponding process: ; in, Indicates that the output feature map is at position The pixel value at Represents the number of sampling points in the convolution kernel, Indicates the The convolution kernel is The weight value of each sampling point.

[0022] It should be noted that the present invention uses a convolution kernel of size 5*5 when performing deconvolution operation, convolution operation and dynamic deformable convolution operation.

[0023] Step 4: The feature maps obtained by concatenating the low-resolution multispectral image and the panchromatic image are processed by the fine-tuned triple attention mechanism and residual network block to obtain refined spectral features and refined spatial features respectively.

[0024] In step 4, the feature maps of the concatenated low-resolution multispectral image and the panchromatic image are processed by the fine-tuned triple attention mechanism and residual network block to obtain refined spectral features and refined spatial features, respectively. The specific steps include the following: The feature map of the concatenated low-resolution multispectral image and the panchromatic image is processed by a fine-tuned triple attention mechanism, added to the spectral features of the low-resolution multispectral image, and processed again by a fine-tuned triple attention mechanism to obtain the refined spectral features. The following relationship exists in the corresponding process: ; in, represents the refined spectral features, Represents the fine-tuned triple attention mechanism processing; The feature map of the low-resolution multispectral image and the panchromatic image is processed by the residual network block, added to the spatial features of the panchromatic image, and processed again by the residual network block to obtain the refined spatial features. The following relationship exists in the corresponding process: ; in, Represents refined spatial features, Indicates that it has been processed by the residual network block.

[0025] Furthermore, the fine-tuning triple attention mechanism specifically includes the following sub-steps: The original feature map is reordered by channel-height and channel-width to obtain a height-reordered feature map and a width-reordered feature map respectively; The original feature map is subjected to the maximum pooling operation and the average pooling operation respectively to obtain the first maximum pooling feature map and the first average pooling feature map respectively. After the first maximum pooling feature map and the first average pooling feature map are feature-concatenated, they are sequentially processed by the convolution operation and the Sigmoid activation function to obtain the first attention map. The following relationship exists in the corresponding process: ; in, represents the first maximum pooling feature map, Indicates that it has undergone maximum pooling operation; represents the original feature map, and ; represents the batch size, Indicates the number of channels, Indicates height, Indicates width, represents the first average pooling feature map, Indicates that after the average pooling operation, represents the first attention map, Indicates that it has been processed by the Sigmoid activation function; The maximum pooling operation and the average pooling operation are performed on the highly rearranged feature map to obtain the second maximum pooling feature map and the second average pooling feature map respectively. After the second maximum pooling feature map and the second average pooling feature map are feature-concatenated, they are sequentially processed by the convolution operation and the Sigmoid activation function to obtain the second attention map. The following relationship exists in the corresponding process: ; in, represents the second maximum pooling feature map, Represents the second average pooling feature map; represents a highly rearranged feature map, and ; represents the second attention map; The width-rearranged feature map is subjected to maximum pooling and average pooling operations respectively to obtain the third maximum pooling feature map and the third average pooling feature map respectively. After feature splicing of the third maximum pooling feature map and the third average pooling feature map, the third attention map is obtained by convolution operation and Sigmoid activation function processing. The following relationship exists in the corresponding process: ; in, represents the third maximum pooling feature map, Represents the third average pooling feature map; represents the width-rearranged feature map, and ; represents the third attention map; The first attention map, the second attention map, and the third attention map are processed respectively using the coupling mechanism to obtain the enhanced first attention map, the enhanced second attention map, and the enhanced third attention map, respectively. The following relationship exists in the corresponding process: ; in, represents the enhanced first attention map, represents the enhanced second attention map, represents the enhanced third attention map; The enhanced first attention map is used to weight the original feature map element by element and perform a dimensional inverse permutation operation to obtain an enhanced original spectral feature map; the enhanced second attention map is used to weight the height rearranged feature map element by element and perform a dimensional inverse permutation operation to obtain an enhanced height spectral feature map; the enhanced third attention map is used to weight the width rearranged feature map element by element and perform a dimensional inverse permutation operation to obtain an enhanced width spectral feature map. The following relationship exists in the corresponding process: ; in, represents the enhanced original spectral feature map, represents the enhanced height spectral feature map, represents the enhanced width spectral feature map, Indicates that after the dimension inverse permutation operation, represents element-wise multiplication; The enhanced original spectral feature map, the enhanced height spectral feature map and the enhanced width spectral feature map are weighted averaged and fused to obtain the final output feature map. The following relationship exists in the corresponding process: ; in, represents the final output feature map, Indicates the An enhanced spectral feature map.

[0026] Step 5: Use the feature fusion module to process the refined spectral features and refined spatial features to obtain a high-resolution multispectral image.

[0027] In step 5, the refined spectral features and refined spatial features are processed using a feature fusion module to obtain a high-resolution multispectral image, which specifically includes the following sub-steps: The refined spectral features are input into the residual dense block for feature extraction to obtain the residual features. The following relationship exists in the corresponding process: ; in, represents the residual feature, Indicates that it has been processed by the residual dense block; The residual features are spliced ​​with the refined spatial features to obtain the gated features. The following relationship exists in the corresponding process: ; in, represents the gated feature; The gated features are processed by convolution operation and Sigmoid function in sequence to obtain the gated weights. The following relationship exists in the corresponding process: ; in, represents the gate weight; Based on the gating weight, the residual features and the refined spatial features are weightedly fused to obtain the fused features. The following relationship exists in the corresponding process: ; in, represents fusion features; The fused features are subjected to convolution operations to obtain convolution features. The following relationship exists in the corresponding process: ; in, Represents convolutional features; The Sobel operator is used to process the full-color image to obtain a gradient map. The following relationship exists in the corresponding process: ; in, represents the gradient map, It means processed by Sobel operator; The convolution feature is multiplied element-by-element with the gradient map to obtain high-frequency detail features. The following relationship exists in the corresponding process: ; in, Represents high-frequency detail features; The high-frequency detail features are added to the fusion features and convolution is performed to obtain a high-resolution multispectral image. The following relationship exists in the corresponding process: ; in, Represents a high-resolution multispectral image.

[0028] Step 6. Based on the high-resolution multispectral image, the perception loss function and the pixel loss function are constructed respectively. The remote sensing image enhancement fusion model is optimized using the perception loss function and the pixel loss function to obtain the optimized remote sensing image enhancement fusion model. The remote sensing image is input into the optimized remote sensing image enhancement fusion model for processing to obtain the final high-resolution multispectral image.

[0029] In step 6, the perception loss function and pixel loss function are constructed based on the high-resolution multispectral image, and the remote sensing image enhancement fusion model is optimized using the perception loss function and the pixel loss function to obtain the optimized remote sensing image enhancement fusion model. The remote sensing image is input into the optimized remote sensing image enhancement fusion model for processing to obtain the final high-resolution multispectral image. The expression of the perception loss function is: ; in, represents the perceived loss, represents the dimension of the feature layer, represents the target image, Indicates that it has been processed by the VGG16 network. Indicates taking the two norm; The expression of pixel loss function is: ; in, represents pixel loss, Indicates the total number of pixels in the image; Furthermore, the expression of the total loss function is: ; in, represents the total loss, Represents a hyperparameter that controls the weights of perceptual loss and pixel loss.

[0030] It should be understood that, although the various steps in the flow chart of each embodiment of the present invention are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in order, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0031] It should be understood that various components of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0032] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0033] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A remote sensing image pan-sharpening method based on spatial-spectral information enhancement, characterized in that: The method comprises the following steps: Step 1: Construct a feature extraction module based on dynamic deformable convolution, normalization, and ReLU activation function. Introduce a coupling mechanism into the triple attention mechanism to obtain a fine-tuned triple attention mechanism. Construct a feature fusion module based on residual dense blocks, gating mechanism, and Sobel operator. The feature extraction module, the fine-tuned triple attention mechanism, the residual network block and the feature fusion module constitute the remote sensing image enhancement fusion model; Step 2: Acquire a remote sensing image, perform an upsampling operation on the low-resolution multispectral image in the remote sensing image to obtain an upsampled multispectral image, perform feature extraction on the upsampled multispectral image and the panchromatic image in the remote sensing image, respectively, and stitch them together to obtain a feature map of the low-resolution multispectral image and the panchromatic image; Step 3: Using a feature extraction module to perform feature extraction on the upsampled multispectral image and the panchromatic image, respectively, to obtain the spectral features of the low-resolution multispectral image and the spatial features of the panchromatic image; Step 4: The feature maps of the concatenated low-resolution multispectral image and the panchromatic image are processed by the fine-tuned triple attention mechanism and residual network block to obtain refined spectral features and refined spatial features respectively; Step 5: Process the refined spectral features and refined spatial features using a feature fusion module to obtain a high-resolution multispectral image; Step 6. Based on the high-resolution multispectral image, the perception loss function and the pixel loss function are constructed respectively. The remote sensing image enhancement fusion model is optimized using the perception loss function and the pixel loss function to obtain the optimized remote sensing image enhancement fusion model. The remote sensing image is input into the optimized remote sensing image enhancement fusion model for processing to obtain the final high-resolution multispectral image.

2. The remote sensing image pan-sharpening method based on spatial-spectral information enhancement according to claim 1, characterized in that: In the step 2, a remote sensing image is acquired, a low-resolution multispectral image in the remote sensing image is upsampled to obtain an upsampled multispectral image, and feature extraction is performed on the upsampled multispectral image and the panchromatic image in the remote sensing image, respectively, and the two are spliced ​​to obtain a feature map of the low-resolution multispectral image and the panchromatic image. The step 2 specifically includes the following sub-steps: Acquire a remote sensing image, and process the low-resolution multispectral image in the remote sensing image through deconvolution operation and ReLU activation function in sequence to obtain an upsampled multispectral image; The upsampled multispectral image is processed by convolution operation and ReLU activation function in sequence to obtain the initial spectral features of the low-resolution multispectral image; The full-color image in the remote sensing image is processed by convolution operation and ReLU activation function in sequence to obtain the initial spatial features of the full-color image; The initial spectral features of the low-resolution multispectral image are spliced ​​with the initial spatial features of the panchromatic image to obtain a feature map of the spliced ​​low-resolution multispectral image and the panchromatic image.

3. The remote sensing image pan-sharpening method based on spatial-spectral information enhancement according to claim 2, characterized in that: In the step of acquiring a remote sensing image and sequentially subjecting the low-resolution multispectral image in the remote sensing image to deconvolution operations and ReLU activation functions to obtain an upsampled multispectral image, the following relationship exists: ； in, represents the upsampled multispectral image, Indicates that it has been processed by the ReLU activation function. Indicates that after deconvolution operation, Represents a low-resolution multispectral image; In the step of sequentially processing the upsampled multispectral image through convolution operations and ReLU activation functions to obtain the initial spectral features of the low-resolution multispectral image, the following relationship exists: ； in, represents the initial spectral characteristics of the low-resolution multispectral image, Indicates that it has undergone convolution operation; In the step of sequentially processing the full-color image in the remote sensing image through convolution operations and ReLU activation functions to obtain the initial spatial features of the full-color image, the following relationship exists: ； in, represents the initial spatial features of the full-color image, represents a full-color image; In the step of splicing the initial spectral features of the low-resolution multispectral image with the initial spatial features of the panchromatic image to obtain a feature map of the spliced ​​low-resolution multispectral image and the panchromatic image, the following relationship exists: ； in, Represents the feature map after the low-resolution multispectral image and the panchromatic image are stitched together, Indicates that a splicing operation has been performed.

4. The remote sensing image pan-sharpening method based on spatial-spectral information enhancement according to claim 3, characterized in that: In step 3, feature extraction is performed on the upsampled multispectral image and the panchromatic image using a feature extraction module to obtain spectral features of the low-resolution multispectral image and spatial features of the panchromatic image, respectively. The steps specifically include the following sub-steps: The upsampled multispectral image is processed sequentially through dynamic deformable convolution operation, normalization operation and ReLU activation function to obtain the spectral characteristics of the low-resolution multispectral image. The following relationship exists in the corresponding process: ； in, represents the spectral characteristics of low-resolution multispectral images, After normalization operation, Indicates that it has undergone dynamic deformable convolution operation; The full-color image is processed by dynamic deformable convolution operation, normalization operation and ReLU activation function in sequence to obtain the spatial features of the full-color image. The following relationship exists in the corresponding process: ； in, Represents the spatial characteristics of a full-color image.

5. The remote sensing image pan-sharpening method based on spatial-spectral information enhancement according to claim 4, characterized in that: In step 4, the feature maps obtained by splicing the low-resolution multispectral image and the panchromatic image are processed by the fine-tuned triple attention mechanism and the residual network block to obtain refined spectral features and refined spatial features, respectively. The specific steps include the following: The feature map of the concatenated low-resolution multispectral image and the panchromatic image is processed by a fine-tuned triple attention mechanism, added to the spectral features of the low-resolution multispectral image, and processed again by a fine-tuned triple attention mechanism to obtain the refined spectral features. The following relationship exists in the corresponding process: ； in, represents the refined spectral features, Represents the fine-tuned triple attention mechanism processing; The feature map of the low-resolution multispectral image and the panchromatic image is processed by the residual network block, added to the spatial features of the panchromatic image, and processed again by the residual network block to obtain the refined spatial features. The following relationship exists in the corresponding process: ； in, Represents refined spatial features, Indicates that it has been processed by the residual network block.

6. The remote sensing image pan-sharpening method based on spatial-spectral information enhancement according to claim 5, characterized in that: The fine-tuned triple attention mechanism specifically includes the following sub-steps: The original feature map is reordered by channel-height and channel-width to obtain a height-reordered feature map and a width-reordered feature map respectively; Performing maximum pooling and average pooling operations on the original feature map to obtain the first maximum pooling feature map and the first average pooling feature map respectively. After feature splicing, the first maximum pooling feature map and the first average pooling feature map are sequentially processed by convolution operation and Sigmoid activation function to obtain the first attention map; Performing maximum pooling and average pooling operations on the highly rearranged feature map to obtain a second maximum pooling feature map and a second average pooling feature map, respectively. After feature splicing, the second maximum pooling feature map and the second average pooling feature map are sequentially processed by convolution operation and Sigmoid activation function to obtain a second attention map. Performing maximum pooling and average pooling operations on the width-rearranged feature map to obtain a third maximum pooling feature map and a third average pooling feature map, respectively. After feature splicing, the third maximum pooling feature map and the third average pooling feature map are sequentially processed by convolution operation and Sigmoid activation function to obtain a third attention map. The first attention map, the second attention map, and the third attention map are processed respectively by using a coupling mechanism to obtain an enhanced first attention map, an enhanced second attention map, and an enhanced third attention map respectively; The enhanced first attention map is used to weight the original feature map element by element and perform a dimension inverse permutation operation to obtain an enhanced original spectral feature map; the enhanced second attention map is used to weight the height rearranged feature map element by element and perform a dimension inverse permutation operation to obtain an enhanced height spectral feature map; the enhanced third attention map is used to weight the width rearranged feature map element by element and perform a dimension inverse permutation operation to obtain an enhanced width spectral feature map; The enhanced original spectral feature map, the enhanced height spectral feature map and the enhanced width spectral feature map are weighted averaged and fused to obtain the final output feature map.

7. The remote sensing image pan-sharpening method based on spatial-spectral information enhancement according to claim 6, characterized in that: In the step of performing the maximum pooling operation and the average pooling operation on the original feature map to obtain the first maximum pooling feature map and the first average pooling feature map respectively, and then performing feature splicing on the first maximum pooling feature map and the first average pooling feature map, and then sequentially performing convolution operation and Sigmoid activation function processing to obtain the first attention map, the following relationship exists: ； in, represents the first maximum pooling feature map, Indicates that after the maximum pooling operation, represents the original feature map, represents the first average pooling feature map, Indicates that after the average pooling operation, represents the first attention map, Indicates that it has been processed by the Sigmoid activation function; In the step of performing maximum pooling and average pooling operations on the highly rearranged feature map to obtain the second maximum pooling feature map and the second average pooling feature map respectively, and then performing feature splicing on the second maximum pooling feature map and the second average pooling feature map, and then sequentially performing convolution operations and Sigmoid activation function processing to obtain the second attention map, the following relationship exists: ； in, represents the second maximum pooling feature map, Represents the second average pooling feature map; represents a highly rearranged feature map, represents the second attention map; In the step of performing maximum pooling operations and average pooling operations on the width-rearranged feature map to obtain a third maximum pooling feature map and a third average pooling feature map, respectively, concatenating the third maximum pooling feature map and the third average pooling feature map, and then sequentially performing convolution operations and Sigmoid activation function processing to obtain the third attention map, the following relationship exists: ； in, represents the third maximum pooling feature map, Represents the third average pooling feature map; represents the width rearrangement feature map, represents the third attention map; In the step of using the coupling mechanism to process the first attention map, the second attention map, and the third attention map respectively to obtain the enhanced first attention map, the enhanced second attention map, and the enhanced third attention map, the following relationship exists: ； in, represents the enhanced first attention map, represents the enhanced second attention map, represents the enhanced third attention map; In the steps of weighting the original feature map element by element using the enhanced first attention map and performing a dimensional inverse permutation operation to obtain an enhanced original spectral feature map; weighting the height rearranged feature map element by element using the enhanced second attention map and performing a dimensional inverse permutation operation to obtain an enhanced height spectral feature map; and weighting the width rearranged feature map element by element using the enhanced third attention map and performing a dimensional inverse permutation operation to obtain an enhanced width spectral feature map, the following relationship exists: ； in, represents the enhanced original spectral feature map, represents the enhanced height spectral feature map, represents the enhanced width spectral feature map, Indicates that after the dimension inverse permutation operation, represents element-wise multiplication; In the step of performing weighted averaging on the enhanced original spectral feature map, the enhanced height spectral feature map, and the enhanced width spectral feature map to obtain the final output feature map, the following relationship exists: ； in, represents the final output feature map, Indicates the An enhanced spectral feature map.

8. The remote sensing image pan-sharpening method based on spatial-spectral information enhancement according to claim 7, characterized in that: In step 5, the refined spectral features and the refined spatial features are processed using a feature fusion module to obtain a high-resolution multispectral image, which specifically includes the following sub-steps: The refined spectral features are input into the residual dense block for feature extraction to obtain residual features; Concatenate the residual features with the refined spatial features to obtain the gated features; The gated features are processed by convolution operation and Sigmoid function in sequence to obtain the gated weights; The residual features and the refined spatial features are weightedly fused based on the gating weights to obtain the fused features; The fused features are subjected to convolution operation to obtain convolution features; The full-color image is processed using the Sobel operator to obtain a gradient map; Multiply the convolution feature with the gradient map element by element to obtain high-frequency detail features; The high-frequency detail features are added to the fusion features and a convolution operation is performed to obtain a high-resolution multispectral image.

9. The remote sensing image pan-sharpening method based on spatial-spectral information enhancement according to claim 8, characterized in that: In the step of inputting the refined spectral features into the residual dense block for feature extraction to obtain the residual features, the following relationship exists: ； in, represents the residual feature, Indicates that it has been processed by the residual dense block; In the step of concatenating the residual features with the refined spatial features to obtain the gated features, the following relationship exists: ； in, represents the gated feature; In the step of processing the gated features through convolution operations and Sigmoid functions in sequence to obtain the gated weights, the following relationship exists: ； in, represents the gate weight; In the step of weighted fusion of the residual features and the refined spatial features based on the gating weights to obtain the fused features, the following relationship exists: ； in, represents fusion features; In the step of performing a convolution operation on the fused features to obtain the convolution features, the following relationship exists: ； in, Represents convolutional features; In the step of processing the full-color image using the Sobel operator to obtain a gradient map, the following relationship exists: ； in, represents the gradient map, It means processed by Sobel operator; In the step of multiplying the convolution feature and the gradient map element by element to obtain high-frequency detail features, the following relationship exists: ； in, Represents high-frequency detail features; In the step of adding high-frequency detail features to fusion features and performing convolution operations to obtain a high-resolution multispectral image, the following relationship exists: ； in, Represents a high-resolution multispectral image.

10. The remote sensing image pan-sharpening method based on spatial-spectral information enhancement according to claim 9, characterized in that: In step 6, a perception loss function and a pixel loss function are constructed based on the high-resolution multispectral image, and the remote sensing image enhancement fusion model is optimized using the perception loss function and the pixel loss function to obtain an optimized remote sensing image enhancement fusion model. The remote sensing image is input into the optimized remote sensing image enhancement fusion model for processing to obtain the final high-resolution multispectral image, wherein the expression of the perception loss function is: ； in, represents the perceived loss, represents the dimension of the feature layer, represents the target image, Indicates that it has been processed by the VGG16 network. Indicates taking the two norm; Among them, the expression of the pixel loss function is: ； in, represents pixel loss, Indicates the total number of pixels in the image.

Images