A panchromatic sharpening method based on progressive space-frequency integrated fusion network

Abstract

The present application relates to the technical field of image processing, and more particularly to a panchromatic sharpening method based on a progressive space-frequency integrated fusion network, which comprises extracting multi-scale spatial features of a PAN image and multi-scale spectral features of an LRMS image; splicing the minimum-scale spatial features and the minimum-scale spectral features to obtain minimum-scale intermediate fusion features and inputting the minimum-scale intermediate fusion features, the spatial features of the next scale and the spectral features of the next scale into a space-frequency domain fusion module for feature enhancement; after the next-scale enhanced features are subjected to convolution and up-sampling, the next-scale enhanced features are subjected to feature enhancement with the multi-scale spatial features and the multi-scale spectral features through multiple space-frequency domain fusion modules to obtain maximum-scale enhanced features; and the maximum-scale enhanced features are subjected to residual processing and added to the LRMS image to obtain an HRMS image. The present application improves the spectral fidelity and spatial detail restoration in panchromatic sharpening of remote sensing images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a full-color sharpening method based on a progressive space-frequency integrated fusion network. Background Technology

[0002] High-spatial-resolution multispectral (HRMS) images have wide applications in remote sensing, including image classification, environmental monitoring, and land surface detection. However, due to the physical limitations of optical imaging sensors, monospectral imaging systems often require trade-offs: low-spatial-resolution multispectral (LRMS) images capture rich spectral information but have limited spatial resolution; conversely, high-spatial-resolution panchromatic (PAN) images provide rich spatial detail but lack sufficient spectral information. To meet the explicit needs of practical applications, these two imaging modalities must be integrated to generate HRMS images; this specific fusion process is formally known as panchromatic sharpening. Existing panchromatic sharpening methods can be mainly divided into two categories: traditional algorithms and deep learning-based methods. Traditional algorithms typically include methods based on component substitution, multiresolution analysis, and variational optimization. In recent years, due to the powerful feature extraction capabilities of neural networks, deep learning-based panchromatic sharpening methods have become the dominant paradigm.

[0003] Traditional algorithms are limited by hand-designed feature extraction mechanisms and empirically designed fusion rules. These limitations usually result in suboptimal feature integration, inevitably leading to significant spectral and spatial distortions in the fusion results.

[0004] Despite significant progress in deep learning-based panchromatic sharpening methods, most existing approaches focus primarily on spatial domain feature representations while neglecting frequency domain information. This neglect leads to insufficient extraction of crucial high- and low-frequency components, inevitably causing spatial and spectral distortion.

[0005] Although some recent methods have introduced dual-domain interaction or dedicated spatial and frequency domain branches, these methods typically directly stitch together the amplitude and phase components of LRMS and PAN images without explicitly distinguishing and decoupling high-frequency and low-frequency information. This limits the in-depth extraction of specific frequency components and their optimal fusion with spatial features. Summary of the Invention

[0006] This invention aims to at least solve one of the technical problems existing in related technologies. To this end, this invention provides a pancolor sharpening method based on a progressive spatial-frequency integrated fusion network, which improves the spectral fidelity and spatial detail reproduction in pancolor sharpening of remote sensing images.

[0007] This invention provides a full-color sharpening method based on a progressive space-frequency integrated fusion network, comprising: S1: Acquire PAN and LRMS images, extract multi-scale spatial features of PAN images through the PAN spatial feature extraction branch, and extract multi-scale spectral features of LRMS images through the LRMS spectral feature extraction branch; S2: The minimum scale spatial features and minimum scale spectral features are spliced ​​together to obtain the minimum scale intermediate fused features; S3: Input the minimum scale intermediate fusion features, the next scale spatial features, and the next scale spectral features into the spatial-frequency domain fusion module for feature enhancement to obtain the next scale enhanced features; S4: The next scale enhanced features are convolved and upsampled, and then enhanced by multiple spatial and frequency domain fusion modules with multi-scale spatial features and multi-scale spectral features to obtain the maximum scale enhanced features. S5: The maximum scale enhancement features are processed by residuals and added to the LRMS image to obtain the HRMS image.

[0008] Furthermore, the multi-scale spatial features of the PAN image are extracted through the PAN spatial feature extraction branch, including: The PAN image is processed with residuals to obtain the maximum scale spatial features; The spatial features at the largest scale are downsampled and processed by residuals to obtain the spatial features at the first scale. The spatial features at the first scale are downsampled and processed with residuals to obtain the spatial features at the next scale. The next scale spatial features are processed by residuals to obtain the minimum scale spatial features.

[0009] Furthermore, the multi-scale spectral features of LRMS images extracted through the LRMS spectral feature extraction branch include: LRMS images are processed using residuals to obtain minimum-scale spectral features; The minimum-scale spectral features are processed by residuals to obtain the next-scale spectral features; The spectral features at the next scale are then upsampled and processed using residuals to obtain the spectral features at the first scale. The first-scale spectral features are then upsampled and processed with residuals to obtain the maximum-scale spectral features.

[0010] Furthermore, step S4 includes: The next-scale enhanced features are convolved and upsampled, and then input into the spatial-frequency domain fusion module along with the first-scale spatial features and the first-scale spectral features to obtain the first-scale enhanced features. The first-scale enhancement features are convolved and upsampled, then input into the spatial-frequency domain fusion module along with the maximum-scale spatial features and the maximum-scale spectral features to obtain the maximum-scale enhancement features.

[0011] Furthermore, the spatial-frequency domain fusion module includes an adaptive high- and low-frequency learning block, a global-local spatial feature extraction block, and a hybrid information feature extraction block.

[0012] Furthermore, the adaptive high- and low-frequency learning blocks include: pass The convolutional layers map multi-scale spatial features to the frequency domain and obtain the amplitude and phase maps of the PAN image through Fourier transform. pass The convolutional layers map multi-scale spectral features to the frequency domain and obtain the amplitude and phase maps of the LRMS image through Fourier transform. Attention calculations are applied to the amplitude map of the PAN image to obtain the attention map of the PAN image; attention calculations are applied to the amplitude map of the LRMS image to obtain the attention map of the LRMS image. High- and low-frequency adaptive weights are generated by calculating the difference between the attention maps of the PAN image and the LRMS image. The high and low frequency adaptive weights are multiplied position by position with the amplitude maps of the PAN image and the LRMS image to obtain the enhanced amplitude after high and low frequency learning. The phase maps of the PAN image and the LRMS image are concatenated and convolved to obtain the enhanced phase after high and low frequency learning; The enhanced amplitude and enhanced phase after high and low frequency learning are subjected to inverse discrete Fourier transform and added to the intermediate fusion features to obtain the frequency domain enhanced output features.

[0013] Furthermore, the expression for calculating the intermediate fusion feature is as follows: in, For the smallest scale intermediate fusion feature, For splicing operations, The kernel size is convolutional layers, For a 2x upsampling operation, For the first Scale-intermediate fusion features For the first Scale-enhanced features For the smallest scale spectral features, It represents the smallest scale spatial feature.

[0014] Furthermore, the global-local spatial feature extraction block includes: The spatial features of the PAN image and the spectral features of the LRMS image are added together, and a global feature weight map is obtained by global average pooling and convolutional layers. The spatial features of the PAN image and the spectral features of the LRMS image are added together, and a local feature weight map is obtained through successive convolutional layers. The global feature weight map and the local feature weight map are added pixel by pixel, and then multiplied position by position with the spatial features of the PAN image and the spectral features of the LRMS image to obtain the enhanced spatial features. The enhanced spatial features are concatenated with the intermediate fused features, then convolved and added to the intermediate fused features to obtain global-local spatial features.

[0015] Furthermore, the calculation expression for the hybrid information feature extraction block is as follows: in, For the first Scale-based attention enhancement features For convolutional block attention modules, For splicing operations, For the first Features enhanced by scale in the frequency domain For the first Global-local spatial features at scale The kernel size is convolutional layers, For AHLFB branch attention enhancement features, For attention enhancement features of the GLFEB branch, For the first Scale-mixed weighted graph This is the Sigmoid activation function.

[0016] Furthermore, the computational expression for the convolutional block attention module is: in, For the first Scale intermediate features For the first Scale-channel attention-weighted features For two consecutive convolutional kernels with a size of A shared multilayer perceptron consisting of convolutional layers and ReLU activation functions. For max pooling, For average pooling, for convolutional layers, This involves multiplying each position sequentially.

[0017] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: This invention explicitly captures and refines the frequency domain representation through an Adaptive High-Low Frequency Block (AHLFB), and captures contextual dependencies and fine-grained spatial texture through a Global-Local Feature Extraction Block (GLEFEB). Cross-domain integration is achieved through a Hybrid Information Feature Extraction Block (HIFEEB), which adaptively calibrates spatial and frequency features through dynamic weight allocation, thereby maximizing overall fusion fidelity.

[0018] This invention enhances spatial details while maintaining the spectral consistency of the original multispectral image, thus improving the problem of balancing spectral fidelity and geometric fidelity and enhancing the overall fusion effect.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a full-color sharpening method based on a progressive space-frequency integrated fusion network provided by the present invention.

[0022] Figure 2 This is a schematic diagram of the full-color sharpening network based on the progressive space-frequency integrated fusion network provided by the present invention.

[0023] Figure 3This is a schematic diagram of the structure of the adaptive high- and low-frequency learning block provided by the present invention.

[0024] Figure 4 This is a schematic diagram of the structure of the global-local spatial feature extraction block provided by the present invention.

[0025] Figure 5 This is a schematic diagram of the structure of the hybrid information feature extraction block provided by the present invention.

[0026] Figure 6 This is a schematic diagram illustrating the experimental results of quantitative indicators in the Pléiades dataset of this invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but cannot be used to limit the scope of this invention.

[0028] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0029] The following is combined Figures 1 to 6 This invention describes a full-color sharpening method based on a progressive space-frequency integrated fusion network.

[0030] like Figure 2As shown, the Progressive Spatial-Frequency Integration Fusion Network (PSFIFusion) for full-color sharpening includes a PAN spatial feature extraction branch, an LRMS spectral feature extraction branch, and a spatial-spectral feature fusion branch. The spatial-spectral feature fusion branch constructs a spatial-frequency domain integrated block (SDIB) at each processing scale. In the SDIB, an adaptive high- and low-frequency learning block is designed in the frequency domain to explicitly capture and refine the frequency domain representation. In the spatial domain, a global-local spatial feature extraction block is designed to meticulously preserve texture and detail information. Additionally, a hybrid information feature extraction block is constructed to adaptively calibrate and enhance features from both the spatial and frequency domains. This module employs a convolutional block attention module (CBAM) to enhance channel and spatial features and uses residual connections and a sigmoid activation function to generate a weight map that integrates spatial-frequency information. The weighted map is applied to features in the frequency and spatial domains respectively. The enhanced features from these two domains are then concatenated, convolved, and upsampled to achieve optimal cross-domain integration. Finally, the residual is processed and added to the LRMS image to obtain the final HRMS image result.

[0031] like Figure 1 A full-color sharpening method based on a progressive space-frequency integrated fusion network includes: S1: Acquire PAN and LRMS images, extract multi-scale spatial features of PAN images through the PAN spatial feature extraction branch, and extract multi-scale spectral features of LRMS images through the LRMS spectral feature extraction branch; The multi-scale spatial features of PAN images extracted through the PAN spatial feature extraction branch include: The PAN image is processed with residuals to obtain the maximum scale spatial features; The spatial features at the largest scale are downsampled and processed by residuals to obtain the spatial features at the first scale. The spatial features at the first scale are downsampled and processed with residuals to obtain the spatial features at the next scale. The next-scale spatial features are processed by residuals to obtain the minimum-scale spatial features; The calculation expression is: in, For the largest scale spatial features, For residual blocks, It is a panchromatic image. , For dimension Image set, For the next scale of spatial features, For the first Scale spatial characteristics, For the first Scale spatial characteristics, This is a 2x downsampling operation. For the smallest scale spatial features, This represents the first-scale spatial feature; The multi-scale spectral features of LRMS images extracted through the LRMS spectral feature extraction branch include: LRMS images are processed using residual blocks to obtain minimum-scale spectral features; The minimum-scale spectral features are processed by residuals to obtain the next-scale spectral features; The spectral features at the next scale are then upsampled and processed using residuals to obtain the spectral features at the first scale. The first-scale spectral features are upsampled and processed using residuals to obtain the maximum-scale spectral features. The calculation expression is: in, For the smallest scale spectral features, For low-resolution multispectral images, , For dimension Image set, The height of the image. The width of the image. This represents the number of spectral bands in a low-resolution multispectral image. For the first Scale spectral characteristics, For the first Scale spectral characteristics, For a 2x upsampling operation, For the next scale spectral features, The first-scale spectral features, This represents the largest-scale spectral characteristics.

[0032] S2: The minimum scale spatial features and minimum scale spectral features are spliced ​​together to obtain the minimum scale intermediate fused features; The calculation expression is: in, For the smallest scale intermediate fusion feature, For splicing operations, The kernel size is The convolutional layer.

[0033] S3: Input the minimum-scale intermediate fusion features, the next-scale spatial features, and the next-scale spectral features into the spatial-frequency domain fusion module for feature enhancement to obtain the next-scale enhanced features. The calculation expression is as follows: in, To enhance features at the next scale, This is a spatial frequency domain fusion module.

[0034] S4: The next scale enhanced features are convolved and upsampled, and then enhanced by multiple spatial and frequency domain fusion modules with multi-scale spatial features and multi-scale spectral features to obtain the maximum scale enhanced features. The next-scale enhanced features are convolved and upsampled, and then input into the spatial-frequency domain fusion module along with the first-scale spatial features and the first-scale spectral features to obtain the first-scale enhanced features. After convolution and upsampling, the first-scale enhancement feature is input into the spatial-frequency domain fusion module along with the maximum-scale spatial feature and the maximum-scale spectral feature to obtain the maximum-scale enhancement feature. in, For first-scale enhanced features, For spatial-frequency domain fusion module, This is a feature enhanced at the maximum scale.

[0035] The spatial-frequency fusion module includes an adaptive high- and low-frequency learning block, a global-local spatial feature extraction block, and a hybrid information feature extraction block.

[0036] like Figure 3 As shown, the adaptive high- and low-frequency learning block includes: pass The convolutional layer maps multi-scale spatial features to the frequency domain and obtains the amplitude and phase maps of the PAN image through Fourier transform. The calculation expression is as follows: in, For the first Amplitude plot of the scaled PAN image. For the first Phase map of a scaled PAN image. For Discrete Fourier Transform, The kernel size is Convolutional layers; pass The convolutional layer maps multi-scale spectral features to the frequency domain and obtains the amplitude and phase maps of the LRMS image through Fourier transform; the calculation expression is: in, For the first Amplitude plot of scaled LRMS image. For the first Phase map of scaled LRMS image.

[0037] Attention calculations are applied to the amplitude map of the PAN image to obtain the attention map of the PAN image; attention calculations are applied to the amplitude map of the LRMS image to obtain the attention map of the LRMS image. in, The attention map for the PAN image. It is the Sigmoid activation function. For two consecutive convolutional layers, the kernel size is... Convolutional layers, For splicing operations, For max pooling, For average pooling, for convolution, This is the attention map for the LRMS image.

[0038] High- and low-frequency adaptive weights are generated by calculating the difference between the attention maps of the PAN image and the LRMS image. The high- and low-frequency adaptive weights are multiplied position-by-position with the amplitude maps of the PAN image and the LRMS image to obtain the enhanced amplitude after high- and low-frequency learning. The calculation expression is as follows: in, The enhanced amplitude after high and low frequency learning, This involves multiplying each position sequentially.

[0039] The phase maps of the PAN image and the LRMS image are concatenated and convolved to obtain the enhanced phase after high- and low-frequency learning. The calculation expression is as follows: in, This is the enhanced phase after high and low frequency learning.

[0040] Perform inverse discrete Fourier transform on the enhanced amplitude and enhanced phase after high- and low-frequency learning, and add them to the intermediate fused features to obtain the frequency-domain enhanced output features. The calculation expression is as follows: in, For the first Features enhanced by scale in the frequency domain This is the inverse Fourier transform operation. For the first Scale-intermediate fusion features; In the spatial domain, GLFEB is used to enhance the detailed texture information of the fusion result, such as... Figure 4 As shown, the global-local spatial feature extraction block includes: The spatial features of the PAN image and the spectral features of the LRMS image are added together, and a global feature weight map is obtained through global average pooling and convolutional layers. The calculation expression is as follows: in, This is a global feature weight map. This is global average pooling.

[0041] The spatial features of the PAN image and the spectral features of the LRMS image are added together, and a local feature weight map is obtained by passing them through successive convolutional layers. in, This is a local feature weight map. For two consecutive convolutional layers, the kernel size is... The convolutional layer.

[0042] The global feature weight map and the local feature weight map are added pixel by pixel, and then multiplied position by position with the spatial features of the PAN image and the spectral features of the LRMS image to obtain the enhanced spatial domain features. The calculation expression is as follows: in, To enhance the spatial characteristics, This involves multiplying each position sequentially.

[0043] The enhanced spatial features are concatenated with the fused features, then convolved and added to the fused features to obtain global-local spatial features. The calculation expression is as follows: in, For the first Global-local spatial features at scale.

[0044] To fully explore the characteristic information in the spatial and frequency domains, such as Figure 5 As shown, the hybrid information feature extraction block introduces a convolutional block attention module (CBAM) to enhance channel and spatial features. Then, it fuses intermediate features using residual connections and generates a hybrid weight map through a sigmoid activation function. The calculation expression is as follows: in, For the first Scale-based attention enhancement features For convolutional block attention modules, For AHLFB branch attention enhancement features, For attention enhancement features of the GLFEB branch, For the first Scale-mixed weighted graph This is the Sigmoid activation function.

[0045] CBAM infers the attention weight map sequentially along two independent dimensions, channel and space, to achieve adaptive dynamic weight allocation and calibration of spatial and frequency domain features; make The specific operation of CBAM is as follows: in, For the first Scale intermediate features For two consecutive convolutional kernels with a size of A shared multilayer perceptron consisting of convolutional layers and ReLU activation functions. for Convolutional layers, For the first Scale-channel attention-weighted features.

[0046] Will Acting on respectively and Then, the data is concatenated, and convolution and upsampling operations are performed sequentially: in, For the first Scale-enhanced features For the first Scale-intermediate fusion features.

[0047] S5: The maximum scale enhancement feature is processed by residual processing and added to the LRMS image to obtain the HRMS image. The calculation expression is: in, This is an HRMS image.

[0048] A full-color sharpening network based on a progressive space-frequency fusion network is trained using mean absolute error loss. The loss function is defined as follows: in, For loss function, For the number of training sample logs, For the first High-resolution multispectral images output by the network. For the first A real image.

[0049] This invention explicitly captures and refines the frequency domain representation through an Adaptive High-Low Frequency Block (AHLFB), and captures contextual dependencies and fine-grained spatial texture through a Global-Local Feature Extraction Block (GLEFEB). Cross-domain integration is achieved through a Hybrid Information Feature Extraction Block (HIFEEB), which adaptively calibrates spatial and frequency features through dynamic weight allocation, thereby maximizing overall fusion fidelity.

[0050] This invention enhances spatial details while maintaining the spectral consistency of the original multispectral image, thus improving the problem of balancing spectral fidelity and geometric fidelity and enhancing the overall fusion effect.

[0051] To verify the effectiveness of this invention, extensive experiments were conducted on the IKONOS, Pléiades, and WorldView-3 datasets (covering both simulated and real datasets). The experiments demonstrate that, compared to traditional GSA methods and state-of-the-art deep learning methods such as PanNet, ADKNet, PanFormer, BiMPan, DISPNet, and WFANet, this invention (PSFIFusion) achieves the best quantitative metrics (such as ERGAS, CC, UIQI, RMSE, SSIM, PSNR, and QNR). In terms of subjective visual results, this invention provides enhanced spatial resolution, sharp edge reconstruction, and faithful terrain texture preservation while maintaining the spectral consistency of the original multispectral features, thus resolving the fundamental balance between spectral and geometric fidelity.

[0052] To verify the superior performance of this invention, quantitative and subjective comparison experiments were conducted on the public dataset Pléiades, comparing it with several state-of-the-art international algorithms. To better observe the differences between the fusion results, the mean absolute error (MAE) plot between the fusion result and the true values ​​(GTs) was calculated and displayed, with local regions magnified for easier observation. These images show that the MAE plot of this invention contains more dark blue areas, indicating the smallest fusion error. The quantitative results also demonstrate that the quantitative performance of this invention is superior to that of advanced international algorithms in recent years. Experimental results are shown in Table 1 and... Figure 6 As shown.

[0053] Table 1. Comparison of quantitative index results for the Pléiades dataset.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A full-color sharpening method based on a progressive space-frequency integrated fusion network, characterized in that, include: S1: Acquire PAN and LRMS images, extract multi-scale spatial features of PAN images through the PAN spatial feature extraction branch, and extract multi-scale spectral features of LRMS images through the LRMS spectral feature extraction branch; S2: The minimum scale spatial features and minimum scale spectral features are spliced ​​together to obtain the minimum scale intermediate fused features; S3: Input the minimum scale intermediate fusion features, the next scale spatial features, and the next scale spectral features into the spatial-frequency domain fusion module for feature enhancement to obtain the next scale enhanced features; S4: The next scale enhanced features are convolved and upsampled, and then enhanced by multiple spatial and frequency domain fusion modules with multi-scale spatial features and multi-scale spectral features to obtain the maximum scale enhanced features. S5: The maximum scale enhancement features are processed by residuals and added to the LRMS image to obtain the HRMS image.

2. The full-color sharpening method based on a progressive space-frequency integrated fusion network according to claim 1, characterized in that, The multi-scale spatial features of PAN images extracted through the PAN spatial feature extraction branch include: The PAN image is processed with residuals to obtain the maximum scale spatial features; The spatial features at the largest scale are downsampled and processed by residuals to obtain the spatial features at the first scale. The spatial features at the first scale are downsampled and processed with residuals to obtain the spatial features at the next scale. The next scale spatial features are processed by residuals to obtain the minimum scale spatial features.

3. The full-color sharpening method based on a progressive space-frequency integrated fusion network according to claim 2, characterized in that, The multi-scale spectral features of LRMS images extracted through the LRMS spectral feature extraction branch include: LRMS images are processed using residuals to obtain minimum-scale spectral features; The minimum-scale spectral features are processed by residuals to obtain the next-scale spectral features; The spectral features at the next scale are then upsampled and processed using residuals to obtain the spectral features at the first scale. The first-scale spectral features are then upsampled and processed with residuals to obtain the maximum-scale spectral features.

4. The full-color sharpening method based on a progressive space-frequency integrated fusion network according to claim 3, characterized in that, Step S4 includes: The next-scale enhanced features are convolved and upsampled, and then input into the spatial-frequency domain fusion module along with the first-scale spatial features and the first-scale spectral features to obtain the first-scale enhanced features. The first-scale enhancement features are convolved and upsampled, then input into the spatial-frequency domain fusion module along with the maximum-scale spatial features and the maximum-scale spectral features to obtain the maximum-scale enhancement features.

5. The full-color sharpening method based on a progressive space-frequency integrated fusion network according to claim 1, characterized in that, The spatial-frequency domain fusion module includes an adaptive high- and low-frequency learning block, a global-local spatial feature extraction block, and a hybrid information feature extraction block.

6. The full-color sharpening method based on a progressive space-frequency integrated fusion network according to claim 5, characterized in that, The adaptive high- and low-frequency learning blocks include: pass The convolutional layers map multi-scale spatial features to the frequency domain and obtain the amplitude and phase maps of the PAN image through Fourier transform. pass The convolutional layers map multi-scale spectral features to the frequency domain and obtain the amplitude and phase maps of the LRMS image through Fourier transform. Attention calculations are applied to the amplitude map of the PAN image to obtain the attention map of the PAN image; attention calculations are applied to the amplitude map of the LRMS image to obtain the attention map of the LRMS image. High- and low-frequency adaptive weights are generated by calculating the difference between the attention maps of the PAN image and the LRMS image. The high and low frequency adaptive weights are multiplied position by position with the amplitude maps of the PAN image and the LRMS image to obtain the enhanced amplitude after high and low frequency learning. The phase maps of the PAN image and the LRMS image are concatenated and convolved to obtain the enhanced phase after high and low frequency learning; The enhanced amplitude and enhanced phase after high and low frequency learning are subjected to inverse discrete Fourier transform and added to the intermediate fusion features to obtain the frequency domain enhanced output features.

7. The full-color sharpening method based on a progressive space-frequency integrated fusion network according to claim 6, characterized in that, The intermediate fusion feature calculation expression is as follows: in, For the smallest scale intermediate fusion feature, For splicing operations, The kernel size is Convolutional layers, For a 2x upsampling operation, For the first Scale-intermediate fusion features For the first Scale-enhanced features For the smallest scale spectral features, It represents the smallest scale spatial feature.

8. The full-color sharpening method based on a progressive space-frequency integrated fusion network according to claim 7, characterized in that, Global-local spatial feature extraction blocks include: The spatial features of the PAN image and the spectral features of the LRMS image are added together, and a global feature weight map is obtained by global average pooling and convolutional layers. The spatial features of the PAN image and the spectral features of the LRMS image are added together, and a local feature weight map is obtained through successive convolutional layers. The global feature weight map and the local feature weight map are added pixel by pixel, and then multiplied position by position with the spatial features of the PAN image and the spectral features of the LRMS image to obtain the enhanced spatial features. The enhanced spatial features are concatenated with the intermediate fused features, then convolved and added to the intermediate fused features to obtain global-local spatial features.

9. The full-color sharpening method based on a progressive space-frequency integrated fusion network according to claim 5, characterized in that, The calculation expression for the hybrid information feature extraction block is: in, For the first Scale-based attention enhancement features For convolutional block attention modules, For splicing operations, For the first Features enhanced by scale in the frequency domain For the first Global-local spatial features at scale The kernel size is Convolutional layers, For AHLFB branch attention enhancement features, For attention enhancement features of the GLFEB branch, For the first Scale-mixed weighted graph This is the Sigmoid activation function.

10. A full-color sharpening method based on a progressive space-frequency integrated fusion network according to claim 9, characterized in that, The computational expression for the convolutional block attention module is: in, For the first Scale intermediate features For the first Scale-channel attention-weighted features For two consecutive convolutional kernels with a size of A shared multilayer perceptron consisting of convolutional layers and ReLU activation functions. For max pooling, For average pooling, for Convolutional layers, This involves multiplying each position sequentially.

Images