A hyperspectral fusion imaging method based on hierarchical gradient guidance

Abstract

This invention relates to the field of hyperspectral and multispectral fusion imaging. Addressing the challenge of acquiring high spatial and hyperspectral resolution simultaneously using a single sensor, this invention fully leverages the complementary characteristics of hyperspectral and multispectral images in both spatial and spectral dimensions, proposing a hierarchical gradient-guided hyperspectral fusion imaging method. The method and system construct a multi-scale fusion framework, introducing a hierarchical gradient interaction module to extract and refine directional gradient information from hyperspectral and multispectral features. Gradient residuals are used as structural priors to guide feature fusion, enhancing the preservation of spatial structures such as edges and textures. Simultaneously, an adaptive cross-modal gating fusion module dynamically adjusts the spatial-spectral feature contributions of different modalities, improving cross-modal consistency. This method and system effectively alleviate the problems of weakened spatial details, structural shift, and spectral distortion in complex scenes, improving the spatial clarity and spectral fidelity of the fused image.

Description

Technical Field

[0001] This invention belongs to the field of computational imaging and remote sensing image processing, and specifically relates to a hyperspectral fusion imaging method based on hierarchical gradient guidance. Background Technology

[0002] Hyperspectral imaging technology can capture the reflectance characteristics of ground objects in hundreds of consecutive narrow spectral bands, playing a crucial role in advanced visual tasks such as target detection, classification, and segmentation. However, due to limitations in imaging principles, hyperspectral images (HSI) have relatively low spatial resolution; while multispectral images (MSI) have high spatial resolution but limited spectral information. Therefore, hyperspectral and multispectral image fusion has become a key technology for improving the utilization rate of remote sensing images, with the goal of reconstructing high-resolution hyperspectral images (HR-HSI) that combine high spatial resolution and high spectral fidelity.

[0003] In existing technologies, traditional model-driven fusion methods rely on simplified degenerate models and manually designed priors, making them difficult to adapt to complex real-world scenarios. While deep learning methods possess nonlinear modeling capabilities, they still suffer from two major drawbacks: first, insufficient extraction of high-frequency spatial structure information such as edges and textures leads to the accumulation of spatial structure biases; second, an overemphasis on spatial information at the expense of spectral information during cross-modal fusion, resulting in an imbalance in spatial-spectral modeling and an inability to maintain spectral consistency. These issues make it difficult for existing fusion methods to simultaneously guarantee the accuracy of spatial detail reconstruction and spectral fidelity, limiting their application in practical remote sensing scenarios. Summary of the Invention

[0004] The technical problem to be solved by this invention is to address the issues of spatial structure reconstruction deviation and spatial-spectral modeling imbalance leading to poor spectral consistency in existing hyperspectral fusion imaging methods. This invention provides a hierarchical gradient-guided hyperspectral fusion imaging method that enhances edge-aware structural representation and cross-modal feature complementarity, thereby achieving a balance between spatial detail and spectral fidelity.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A hierarchical gradient-guided hyperspectral fusion imaging method includes:

[0007] 1) Perform spatial and spectral dimension alignment preprocessing on low-resolution hyperspectral images (LR-HSI) and high-resolution multispectral images (HR-MSI);

[0008] 2) In the encoding stage, the features of the hyperspectral image (HSI) and multispectral image (MSI) are downsampled layer by layer. The downsampled HSI and MSI features are first extracted through a window attention module to achieve multi-scale spatial-spectral feature extraction. Then, the extracted features are processed through an adaptive cross-modal gating fusion module composed of a hybrid attention mechanism and a gating fusion mechanism to achieve cross-modal feature interaction, and through a hierarchical gradient interaction module composed of gradient calculation and gradient refinement mechanisms to achieve structural gradient extraction and fusion. Finally, the features obtained from the two modules are fused to obtain a multi-scale encoded feature sequence.

[0009] 3) In the decoding stage, the features at each scale of the encoding end are upsampled layer by layer; then, the same window attention, adaptive cross-modal gating fusion and hierarchical gradient interaction operations as in the encoding stage are performed in each decoding layer to obtain a multi-scale decoded feature sequence; and then it is concatenated with the corresponding scale encoded features to obtain the multi-scale fusion result.

[0010] 4) The multi-scale fusion results and shallow gradient priors are aggregated step by step through the gradient-aware fusion module; finally, the final fusion result is obtained by convolution and residual correction with upsampled LR-HSI to reconstruct a high-resolution hyperspectral image; the gradient-aware fusion module is the process of stitching and fusing the multi-scale fusion results one by one, and finally fusing them with the shallow gradient priors.

[0011] Optionally, the preprocessing in step 1) includes: spatially upsampling the LR-HSI to match the resolution of the HR-MSI; and performing convolutional embedding on the two modal images respectively to align the spectral dimensions and thus construct an initial feature space of uniform dimension.

[0012] Optionally, the detailed steps in step 2) include:

[0013] 2.1) During the encoding stage, HSI and MSI features are downsampled layer by layer respectively;

[0014] 2.2) The downsampled HSI and MSI features are used to extract multi-scale spatial-spectral features through a window attention module;

[0015] 2.3) The HSI and MSI features extracted by the window attention module are used to achieve cross-modal feature interaction through the adaptive cross-modal gating fusion module;

[0016] 2.4) The HSI and MSI features extracted by the window attention module are processed through the hierarchical gradient interaction module to achieve structural gradient extraction and fusion;

[0017] 2.5) The features obtained through the adaptive cross-modal gating fusion module and the hierarchical gradient interaction module are fused to obtain a multi-scale encoded feature sequence.

[0018] Optionally, in step 2.1), during the encoding stage, the function expressions for downsampling the HSI and MSI features layer by layer are as follows:

[0019] ,

[0020] ,

[0021] In the above formula, and The HSI and MSI features are input separately. and HSI and MSI features after downsampling, respectively. This is a downsampling operation;

[0022] Optionally, in step 2.2), the downsampled HSI and MSI features are processed through a window attention module to achieve the multi-scale spatial-spectral feature extraction function expression, which is:

[0023] ,

[0024] ,

[0025] In the above formula, and The HSI and MSI features are obtained after passing through the window attention module, respectively. For window attention module;

[0026] Optionally, in step 2.3), the HSI and MSI features extracted by the window attention module are processed through an adaptive cross-modal gating fusion module. The function expression for achieving cross-modal feature interaction is as follows:

[0027] ,

[0028] Detailed implementation steps of the adaptive cross-modal gating fusion module: First, the input HSI and MSI features are processed through a hybrid attention mechanism to obtain a hybrid attention map:

[0029] ,

[0030] ,

[0031] Subsequently, gating weights are generated through a gating mechanism.

[0032] ,

[0033] ,

[0034] ,

[0035] Finally, the input features are fused to achieve cross-modal feature interaction:

[0036] ,

[0037] ,

[0038] In the above formula, Channel attention map Spatial attention map Element-wise multiplication yields the joint attention map. This is an element-wise multiplication operation. For channel attention, Spatial attention; This is the Sigmoid activation function. and They are respectively and Features obtained after flattening; For gating weights, Decomposed along the channel dimension and Features and The corresponding weights; For splicing operations; A multilayer perceptron consists of two fully connected layers and LeakyReLU and a sigmoid activation function. For flattening operation, This is a reshaping operation. Among them... and Features and The weight, The features output by the adaptive cross-modal gating fusion module, For adaptive cross-modal gating fusion module;

[0039] Optionally, in step 2.4), the HSI and MSI features extracted by the window attention module are processed through the hierarchical gradient interaction module to achieve structural gradient extraction and fusion. The function expression is as follows:

[0040] ,

[0041] Detailed implementation steps of the hierarchical gradient interaction module:

[0042] First, gradient calculation is performed on the input HSI and MSI feature maps:

[0043] , , , ,

[0044] Subsequently, the gradient features are concatenated and fed into the gradient refining module to achieve structural gradient extraction and fusion:

[0045] ,

[0046] ,

[0047] In the above formula, The features output by the hierarchical gradient interaction module, This is a hierarchical gradient interaction module. and They are respectively Gradients along the horizontal and vertical directions, and respectively Let be the gradient along the horizontal and vertical directions, where yes and Features after splicing yes and Features after splicing; The gradient refining module consists of convolutional layers and channel attention, and is used to enhance and filter key gradient information.

[0048] Optionally, in step 2.5), the features obtained through the adaptive cross-modal gating fusion module and the hierarchical gradient interaction module are fused to obtain the function expression of the multi-scale encoded feature sequence as follows:

[0049] ,

[0050] In the above formula, For multi-scale encoding features, This is for splicing operations.

[0051] Optionally, the detailed steps in step 3) include:

[0052] 3.1) During the decoding stage, the features at each scale of the encoding end are upsampled layer by layer;

[0053] 3.2) Subsequently, the same window attention, adaptive cross-modal gating fusion, and hierarchical gradient interaction operations as in the encoding stage are repeated in each decoding layer to obtain a multi-scale decoding feature sequence;

[0054] 3.3) The obtained multi-scale decoded feature sequence is concatenated with the corresponding scale encoded features to obtain the multi-scale fusion result.

[0055] Optionally, in step 3.1), during the decoding stage, the function expression for upsampling the features at each scale layer at the encoding end is:

[0056] ,

[0057] ,

[0058] In the above formula, and HSI and MSI features at the encoding end respectively. and HSI and MSI features, respectively, after upsampling This is an upsampling operation;

[0059] Optionally, in step 3.2), the same window attention, adaptive cross-modal gating fusion, and hierarchical gradient interaction operations as in the encoding stage are subsequently repeated at each decoding layer, resulting in the functional expression of the multi-scale decoded feature sequence as follows:

[0060] ,

[0061] ,

[0062] ,

[0063] ,

[0064] ,

[0065] In the above formula, and The HSI and MSI features are obtained after passing through the window attention module, respectively. The features output by the adaptive cross-modal gating fusion module, The features output by the hierarchical gradient interaction module, For multi-scale decoding features;

[0066] Optionally, in step 3.3), the function expression for concatenating the obtained multi-scale decoded feature sequence with the corresponding scale encoded features to obtain the multi-scale fusion result is as follows:

[0067] ,

[0068] In the above formula, To combine and The multi-scale fusion results and These are the corresponding scale-encoded and decoded features. For the first Layer encoding or decoding layer.

[0069] Optionally, the detailed steps in step 4) include:

[0070] 4.1) The multi-scale fusion results and shallow gradient priors are aggregated step by step through the gradient-aware fusion module to obtain the enhanced fusion result;

[0071] 4.2) The fusion result obtained through the gradient-aware fusion module is convolved and residual correction is performed with the upsampled LR-HSI to obtain the final fusion result, so as to reconstruct a high-resolution hyperspectral image.

[0072] Optionally, the function expression for aggregating the multi-scale fusion results and shallow gradient priors step by step through the gradient-aware fusion module in step 4.1) is as follows:

[0073] ,

[0074] In the above formula, The result is a multi-scale fusion. It is a shallow gradient prior. These are features that have been aggregated step by step by the gradient-aware fusion module. This is the gradient-aware fusion module.

[0075] In step 4.2), the fusion result obtained through the gradient-aware fusion module is convolved and residual correction is performed with the upsampled LR-HSI to obtain the final fusion result. The function expression is as follows:

[0076] ,

[0077] In the above formula, For upsampled LR-HSI, The final fusion result is a high-resolution hyperspectral image, where Conv is the convolution operation.

[0078] Furthermore, the present invention also provides a hierarchical gradient-guided hyperspectral fusion imaging method, comprising an interconnected microprocessor and a memory, wherein the microprocessor is programmed or configured to execute the hierarchical gradient-guided hyperspectral fusion imaging method.

[0079] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the hierarchical gradient-guided hyperspectral fusion imaging method by a processor.

[0080] In addition, the present invention also provides a computer program product, including a computer program or instructions that are programmed or configured to execute the hierarchical gradient-guided hyperspectral fusion imaging method via a processor.

[0081] Compared with existing technologies, this invention has the following advantages: The method of this invention integrates high-resolution hyperspectral spectral information with high-quality multispectral spatial information, introduces a hierarchical gradient interaction module to explicitly model directional gradients and structural residual information, effectively suppressing multi-layered cumulative structural bias during the fusion process; through an adaptive cross-modal gating fusion module, it comprehensively utilizes hybrid attention and learnable gating mechanisms to dynamically adjust the spectral-spatial contributions of different modalities, improving cross-modal complementarity and significantly alleviating the spectral distortion problem that easily occurs in traditional methods. Furthermore, this invention, through a multi-scale fusion decoding framework, jointly constructs cross-scale spatial-spectral associations at the encoding and decoding ends, enhancing the global dependency modeling capability of features, enabling the network to maintain structural consistency at different resolutions; combined with a gradient-aware fusion module, it uniformly aggregates multi-scale reconstruction results, allowing the output side to selectively emphasize key structural regions while maintaining the smoothness of the spectral curve. The method of this invention possesses excellent noise robustness and scene adaptability, maintaining stable fusion quality under real complex terrain, mixed illumination, and cross-sensor data conditions, significantly improving classification, detection, and fine analysis capabilities in remote sensing applications, and providing reliable data support for high-precision remote sensing monitoring. Attached Figure Description

[0082] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0083] Figure 2 This is a schematic diagram of the overall framework in an embodiment of the present invention.

[0084] Figure 3 This is a schematic diagram of the adaptive cross-modal gating fusion module in an embodiment of the present invention.

[0085] Figure 4 This is a schematic diagram of the hierarchical gradient interaction module in an embodiment of the present invention.

[0086] Figure 5 This is a schematic diagram of the gradient perception fusion module in an embodiment of the present invention. Detailed Implementation

[0087] like Figure 1 and Figure 2 As shown, a hierarchical gradient-guided hyperspectral fusion imaging method is characterized by comprising:

[0088] 1) Perform spatial and spectral dimension alignment preprocessing on low-resolution hyperspectral images LR-HSI and high-resolution multispectral images HR-MSI;

[0089] 2) In the encoding stage, HSI and MSI features are downsampled layer by layer; and then passed through the window attention module, the adaptive cross-modal gating fusion module, and the hierarchical gradient interaction module in sequence to achieve multi-scale spatial-spectral feature extraction, cross-modal feature interaction, structural gradient extraction and fusion, resulting in a multi-scale encoded feature sequence.

[0090] 3) In the decoding stage, the features at each scale of the encoding end are upsampled layer by layer; then, the window attention module, the adaptive cross-modal gating fusion module, and the hierarchical gradient interaction module are executed sequentially in each decoding layer to obtain a multi-scale decoded feature sequence; and then concatenated with the corresponding scale encoded features to obtain the multi-scale fusion result.

[0091] 4) The multi-scale fusion results and shallow gradient priors are aggregated step by step through the gradient-aware fusion module; finally, the fusion results are obtained by convolution and residual correction with upsampled LR-HSI to reconstruct a high-resolution hyperspectral image.

[0092] See Figure 2 It can be seen that step 1) is the data preprocessing process; step 2) is the feature decoding process at the encoding end; step 3) is the feature decoding process at the decoding end; and step 4) is the process of reconstructing a high-resolution hyperspectral image.

[0093] like Figure 2 As shown, step 1) involves preprocessing the low-resolution hyperspectral image and the high-resolution multispectral image, including: spatial upsampling of the LR-HSI to match the resolution of the HR-MSI; and convolutional embedding of the two modal images to align the spectral dimensions and thus construct an initial feature space of uniform dimension.

[0094] like Figure 2As shown, in step 2), during the encoding stage, HSI and MSI features are downsampled layer by layer; and sequentially passed through the window attention module, the adaptive cross-modal gating fusion module, and the hierarchical gradient interaction module; to achieve multi-scale spatial-spectral feature extraction, cross-modal feature interaction, structural gradient extraction and fusion, resulting in a multi-scale encoded feature sequence; in this embodiment, the detailed steps in step 2) include:

[0095] 2.1) During the encoding stage, HSI and MSI features are downsampled layer by layer respectively;

[0096] 2.2) The downsampled HSI and MSI features are used to extract multi-scale spatial-spectral features through a window attention module;

[0097] 2.3) The HSI and MSI features extracted by the window attention module are used to achieve cross-modal feature interaction through the adaptive cross-modal gating fusion module;

[0098] 2.4) The HSI and MSI features extracted by the window attention module are processed through the hierarchical gradient interaction module to achieve structural gradient extraction and fusion;

[0099] 2.5) The features obtained through the adaptive cross-modal gating fusion module and the hierarchical gradient interaction module are fused to obtain a multi-scale encoded feature sequence.

[0100] In this embodiment, in step 2.1), during the encoding stage, the function expression for downsampling the HSI and MSI features layer by layer is as follows:

[0101] ,

[0102] ,

[0103] In the above formula, and The HSI and MSI features are input separately. and HSI and MSI features after downsampling, respectively. This is a downsampling operation;

[0104] In this embodiment, the downsampled HSI and MSI features in step 2.2) are processed through a window attention module to achieve the multi-scale spatial-spectral feature extraction function expression, which is:

[0105] ,

[0106] ,

[0107] In the above formula, and The HSI and MSI features are obtained after passing through the window attention module, respectively. For window attention module;

[0108] In this embodiment, as Figure 3 As shown in step 2.3), the function expression for achieving cross-modal feature interaction by passing the HSI and MSI features extracted by the window attention module through the adaptive cross-modal gating fusion module is as follows:

[0109] ,

[0110] The detailed calculation formula for the adaptive cross-modal gating fusion module is as follows:

[0111] HSI and MSI features are first processed through a mixed attention process to obtain a mixed attention map:

[0112] ,

[0113] ,

[0114] Subsequently, to further enhance cross-modal selectivity, a gating mechanism is introduced. The input features are concatenated, flattened, and then fed into a multilayer perceptron to generate gating weights.

[0115] ,

[0116] ,

[0117] ,

[0118] Finally, the input features are fused to achieve cross-modal feature interaction:

[0119] ,

[0120] ,

[0121] In the above formula, The features output by the adaptive cross-modal gating fusion module, and The HSI and MSI features are obtained after passing through the window attention module, respectively. Channel attention map Spatial attention map Element-wise multiplication yields the joint attention map. This is an element-wise multiplication operation. For channel attention, Spatial attention; This is the Sigmoid activation function. and They are respectively and Features obtained after flattening; For gating weights, Decomposed along the channel dimension and Features and The corresponding weights; A multilayer perceptron consists of two fully connected layers and LeakyReLU and a sigmoid activation function. For flattening operation, This is a reshaping operation. Among them... and Features and The weight, For adaptive cross-modal gating fusion module;

[0122] In this embodiment, as Figure 4 As shown in step 2.4), the function expression for extracting and fusing the HSI and MSI features extracted by the window attention module through the hierarchical gradient interaction module is as follows:

[0123] ,

[0124] The detailed calculation formula for the hierarchical gradient interaction module is as follows:

[0125] First, gradient calculation is performed on the input HSI and MSI feature maps:

[0126] , , , ,

[0127] Subsequently, the gradient features are concatenated and fed into the gradient refining module to achieve structural gradient extraction and fusion:

[0128] ,

[0129] ,

[0130] In the above formula, The features output by the hierarchical gradient interaction module, This is a hierarchical gradient interaction module. This represents the feature map of the input HSI. This is represented as the feature map of the input MSI. and They are respectively Gradients along the horizontal and vertical directions, and respectively Let be the gradient along the horizontal and vertical directions, where yes and Features after splicing yes and Features after splicing; The gradient refining module consists of convolutional layers and channel attention, and is used to enhance and filter key gradient information.

[0131] In this embodiment, step 2.5) fuses the features obtained through the adaptive cross-modal gating fusion module and the hierarchical gradient interaction module to obtain the function expression of the multi-scale encoded feature sequence:

[0132] ,

[0133] In the above formula, It is a multi-scale encoded feature.

[0134] like Figure 2 As shown, in step 3), during the decoding stage, the features at each scale of the encoding end are upsampled layer by layer; then, in each decoding layer, the window attention module, the adaptive cross-modal gating fusion module, and the hierarchical gradient interaction module are executed sequentially to obtain a multi-scale decoded feature sequence; and this sequence is concatenated with the corresponding scale's encoded features to obtain a multi-scale fusion result; in this embodiment, the detailed steps in step 3) include:

[0135] 3.1) During the decoding stage, the features at each scale of the encoding end are upsampled layer by layer;

[0136] 3.2) Subsequently, the window attention module, the adaptive cross-modal gating fusion module, and the hierarchical gradient interaction module are repeatedly executed in each decoding layer to obtain a multi-scale decoding feature sequence;

[0137] 3.3) The obtained multi-scale decoded feature sequence is concatenated with the corresponding scale encoded features to obtain the multi-scale fusion result.

[0138] In this embodiment, in step 3.1), during the decoding stage, the function expression for upsampling the features at each scale of the encoding end layer by layer is:

[0139] ,

[0140] ,

[0141] In the above formula, and HSI and MSI features at the encoding end respectively. and HSI and MSI features, respectively, after upsampling This is an upsampling operation;

[0142] In this embodiment, in step 3.2), the window attention module, the adaptive cross-modal gating fusion module, and the hierarchical gradient interaction module are subsequently executed repeatedly in each decoding layer to obtain the function expression of the multi-scale decoded feature sequence:

[0143] ,

[0144] ,

[0145] ,

[0146] ,

[0147] ,

[0148] In the above formula, and The HSI and MSI features are obtained after passing through the window attention module, respectively. The features output by the adaptive cross-modal gating fusion module, The features output by the hierarchical gradient interaction module, For multi-scale decoding features;

[0149] In this embodiment, the function expression for concatenating the obtained multi-scale decoded feature sequence with the corresponding scale encoded features in step 3.3) to obtain the multi-scale fusion result is as follows:

[0150] ,

[0151] In the above formula, To combine and The multi-scale fusion results and These are the corresponding scale-encoded and decoded features. For the first Layer encoding or decoding layer.

[0152] like Figure 5 As shown, step 4) aggregates the multi-scale fusion result and shallow gradient prior through a gradient-aware fusion module step by step; finally, it is convolved and residually corrected with upsampled LR-HSI to obtain the final fusion result, reconstructing a high-resolution hyperspectral image; the detailed steps in step 4) include:

[0153] 4.1) The multi-scale fusion results and shallow gradient priors are aggregated step by step through the gradient-aware fusion module to obtain the enhanced fusion result;

[0154] 4.2) The fusion result obtained through the gradient-aware fusion module is convolved and residual correction is performed with the upsampled LR-HSI to obtain the final fusion result, so as to reconstruct a high-resolution hyperspectral image.

[0155] In this embodiment, the function expression for aggregating the multi-scale fusion results and shallow gradient priors step by step through the gradient-aware fusion module in step 4.1) is as follows:

[0156] ,

[0157] ,

[0158] ,

[0159] ,

[0160] ,

[0161] In the above formula, The result is a multi-scale fusion. To concatenate results from one or more scales to obtain features; It is a module consisting of convolutional layers, batch normalization layers, and modified linear unit activation functions. and These are shallow HSI features. Gradients in the horizontal and vertical directions, and These are shallow MSI features. Gradients in the horizontal and vertical directions, It is a shallow gradient prior. These are features that have been aggregated step by step by the gradient-aware fusion module. This is the gradient-aware fusion module.

[0162] In this embodiment, the function expression for obtaining the final fusion result by performing convolution and residual correction with the upsampled LR-HSI after obtaining the fusion result obtained through the gradient-aware fusion module in step 4.2) is as follows:

[0163] ,

[0164] In the above formula, For upsampled LR-HSI, The final fusion result is a high-resolution hyperspectral image.

[0165] To verify the effectiveness of this invention, the proposed model was comprehensively evaluated on the CAVE dataset. The CAVE dataset contains high-resolution multispectral images covering a variety of natural and synthetic materials, encompassing 31 spectral bands within the 400-700 nm wavelength range, with a spatial resolution of 512×512 pixels for each scene. Four quantitative evaluation metrics were used to assess the fusion performance: Peak Signal-to-Noise Ratio (PSNR), Spectral Angle Mapper (SAM), Root Mean Square Error (RMSE), and Dimensionless Relative Global Error (ERGAS).For comparison with the method in this embodiment: PSRT (see Deng SQ, Deng LJ, Wu X, et al. PSRT: Pyramid shuffle-and-reshuffle transformer for multispectral and hyperspectral image fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-15.); MIMO (see Fang J, Yang J, Khader A, et al. MIMO-SST: Multi-input multi-output spatial-spectral transformer for hyperspectral and multispectral image fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 1-20.); SFIG (see Zhu C, Song X, Li Y, et al. Aspatial-frequency dual-domain implicit guidance method for hyperspectral and multispectral remote sensing image fusion based on Kolmogorov-Arnold Network[J]. Information Fusion, 2025: 103261.); The comparison method is PSTUN (see Wang B, Xiong X, Lian Y, et al. Perceptive spectral transformer unfolding network with multiscale mixed training for arbitrary-scale hyperspectral and multispectral image fusion[J]. Information Fusion, 2025, 122: 103166.). Table 1 shows the quantitative comparison results of the method of this invention with four advanced algorithms on the CAVE dataset.

[0166] Table 1 compares the experimental results of the method in this embodiment with those of four advanced algorithms on the CAVE dataset.

[0167] method PSNR RMSE SAM ERGAS PSRT 45.9228 1.6614 3.1022 0.7010 MIMO 48.2850 1.2588 2.5157 0.5477 SFIG 45.6145 1.7325 2.8083 0.7458 PSTUN 45.2436 1.7942 2.8120 0.7682 Method of this embodiment 48.8443 1.1407 2.2946 0.5058 .

[0168] As shown in Table 1, the method of this invention outperforms the comparative methods in all four evaluation metrics—PSNR, RMSE, SAM, and ERGAS—on the CAVE dataset. This demonstrates that it can effectively recover spatial details while maintaining spectral fidelity.

[0169] In summary, this invention extracts and refines directional gradients through a hierarchical gradient interaction module, enhancing the structural consistency between edges and textures. Simultaneously, it designs an adaptive cross-modal gating fusion module that dynamically balances spatial-spectral contributions through a hybrid attention and gating mechanism, strengthening the complementary information of the two modalities. Furthermore, it integrates spatial-spectral features at different scales using a multi-scale fusion strategy, effectively improving the spatial clarity and spectral fidelity of the fused image. Extensive experimental verification demonstrates that this method exhibits excellent performance in scenarios with complex geometric deformations and spectral variations, providing reliable technical support for practical applications such as remote sensing monitoring and environmental perception.

[0170] Furthermore, this embodiment also provides a hierarchical gradient-guided hyperspectral fusion imaging method, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the hierarchical gradient-guided hyperspectral fusion imaging method.

[0171] Furthermore, this embodiment also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the hierarchical gradient-guided hyperspectral fusion imaging method via a processor.

[0172] Furthermore, this embodiment also provides a computer program product, including a computer program or instructions that are programmed or configured to execute the hierarchical gradient-guided hyperspectral fusion imaging method via a processor.

[0173] Those skilled in the art will understand that the solutions of this application can be implemented as a method flow, constructed as a system architecture, or provided in the form of a computer program product. Specifically, this technology can be implemented in pure hardware, executed by software, or implemented using a hardware-software co-operation approach. Furthermore, the methods of this application can also be encapsulated as program code stored on a computer-readable medium, which can be a magnetic storage medium, an optical disc medium, or other storage media capable of storing instructions. The descriptions of method steps, system components, and computer program products in this specification can be considered as corresponding to functional modules implemented when a processor executes corresponding instructions. These instructions can be written into a computer-readable medium to drive a computing device to operate according to specific logic, thereby forming an instruction set product that performs the functions described. The instructions can also be loaded and run on a computer or other programmable device to implement the various functions defined in the flowchart or block diagram through a series of operations.

[0174] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A hyperspectral fusion imaging method based on hierarchical gradient guidance, characterized in that, include: 1) Perform spatial and spectral dimension alignment preprocessing on low-resolution hyperspectral images (LR-HSI) and high-resolution multispectral images (HR-MSI); 2) In the encoding stage, the features of the hyperspectral image (HSI) and the multispectral image (MSI) are downsampled layer by layer. The downsampled HSI and MSI features are first extracted through a window attention module to achieve multi-scale spatial-spectral feature extraction. Then, the extracted features are processed through an adaptive cross-modal gating fusion module composed of a hybrid attention mechanism and a gating fusion mechanism to achieve cross-modal feature interaction, and through a hierarchical gradient interaction module composed of gradient calculation and gradient refinement mechanisms to achieve structural gradient extraction and fusion. Finally, the features obtained from the two modules are fused to obtain a multi-scale encoded feature sequence. 3) During the decoding stage, the features at each scale of the encoding end are upsampled layer by layer; Subsequently, the same window attention, adaptive cross-modal gating fusion, and hierarchical gradient interaction operations as in the encoding stage are performed in each decoding layer to obtain a multi-scale decoded feature sequence; Then, it is concatenated with the encoded features of the corresponding scale to obtain the multi-scale fusion result; 4) The multi-scale fusion results and shallow gradient priors are aggregated step by step through the gradient-aware fusion module; finally, the fusion results are obtained by convolution and residual correction with upsampled LR-HSI to reconstruct a high-resolution hyperspectral image; The gradient-aware fusion module is a process of stitching and fusing multi-scale fusion results one after another, and finally fusing them with shallow gradient priors.

2. The method of claim 1, wherein, The preprocessing described in step 1) includes: spatially upsampling the LR-HSI to match the resolution of the HR-MSI; and performing convolutional embedding on the two modal images to align the spectral dimensions, thereby constructing an initial feature space of uniform dimension.

3. The method of claim 1, wherein, Step 2) includes the following detailed steps: 1) During the encoding stage, HSI and MSI features are downsampled layer by layer respectively; 2) The downsampled HSI and MSI features are used to extract multi-scale spatial-spectral features through the window attention module; 3) The HSI and MSI features extracted by the window attention module are used to achieve cross-modal feature interaction through the adaptive cross-modal gating fusion module; 4) The HSI and MSI features extracted by the window attention module are processed through the hierarchical gradient interaction module to achieve structural gradient extraction and fusion; 5) The features obtained through the adaptive cross-modal gating fusion module and the hierarchical gradient interaction module are fused to obtain a multi-scale encoded feature sequence.

4. The method of claim 1, wherein, In step 1), during the encoding stage, the function expressions for downsampling the HSI and MSI features layer by layer are as follows: ， ， In the above formula, and respectively inputted HSI and MSI features, and respectively down-sampled HSI and MSI features, is a down-sampling operation; In step 2), the downsampled HSI and MSI features are processed through a window attention module to achieve multi-scale spatial-spectral feature extraction. The functional expression is as follows: ， ， In the above formula, and The HSI and MSI features are obtained after passing through the window attention module, respectively. For window attention module; In step 3), the HSI and MSI features extracted by the window attention module are processed through an adaptive cross-modal gating fusion module to achieve cross-modal feature interaction. The function expression is as follows: ， Detailed implementation steps of the adaptive cross-modal gating fusion module: First, the input HSI and MSI features are processed through a hybrid attention mechanism to obtain a hybrid attention map: ， ， Subsequently, gating weights are generated through a gating mechanism. ， ， ， Finally, the input features are fused to achieve cross-modal feature interaction: ， ， In the above formula, For channel attention diagram Spatial attention diagram Element-wise multiplication yields the joint attention map. This is an element-wise multiplication operation. For channel attention, Spatial attention; Here, the Sigmoid activation function is used; and They are respectively and Features obtained after flattening; For gating weights, Decomposed along the channel dimension and Features and The corresponding weights; For splicing operations; A multilayer perceptron consists of two fully connected layers and LeakyReLU and a sigmoid activation function. For flattening operation, For reshaping operations; where and Features and The weight, The features output by the adaptive cross-modal gating fusion module, For adaptive cross-modal gating fusion module; In step 4), the HSI and MSI features extracted by the window attention module are processed through the hierarchical gradient interaction module to achieve structural gradient extraction and fusion. The function expression is as follows: ， Detailed implementation steps of the hierarchical gradient interaction module: First, gradient calculation is performed on the input HSI and MSI feature maps: , , , ， Subsequently, the gradient features are concatenated and fed into the gradient refining module to achieve structural gradient extraction and fusion: ， ， In the above formula, The features output by the hierarchical gradient interaction module, This is a hierarchical gradient interaction module. and They are respectively Gradients along the horizontal and vertical directions, and They are respectively Gradients along the horizontal and vertical directions, where yes and Features after splicing yes and Features after splicing; The gradient refining module consists of convolutional layers and channel attention, and is used to enhance and filter key gradient information. In step 5), the features obtained through the adaptive cross-modal gating fusion module and the hierarchical gradient interaction module are fused to obtain the function expression of the multi-scale encoded feature sequence: ， In the above formula, For multi-scale encoding features, This is for splicing operations.

5. The hyperspectral fusion imaging method according to claim 1, characterized in that, Step 3) includes the following detailed steps: 1) During the decoding stage, the features at each scale of the encoding end are upsampled layer by layer; 2) Subsequently, the same window attention, adaptive cross-modal gating fusion, and hierarchical gradient interaction operations as in the encoding stage are repeated in each decoding layer to obtain a multi-scale decoded feature sequence; 3) The obtained multi-scale decoded feature sequence is concatenated with the corresponding scale encoded features to obtain the multi-scale fusion result.

6. The hyperspectral fusion imaging method according to claim 5, characterized in that, In step 1), during the decoding stage, the function expression for upsampling the features at each scale layer at the encoding end is: ， ， In the above formula, and HSI and MSI features at the encoding end respectively. and HSI and MSI features, respectively, after upsampling This is an upsampling operation; In step 2), the same window attention, adaptive cross-modal gating fusion, and hierarchical gradient interaction operations as in the encoding stage are then repeated at each decoding layer, resulting in the following functional expression for the multi-scale decoded feature sequence: ， ， ， ， ， In the above formula, and The HSI and MSI features are obtained after passing through the window attention module, respectively. The features output by the adaptive cross-modal gating fusion module, The features output by the hierarchical gradient interaction module, For multi-scale decoding features; In step 3), the function expression for concatenating the obtained multi-scale decoded feature sequence with the corresponding scale encoded features to obtain the multi-scale fusion result is as follows: ， In the above formula, To combine and The multi-scale fusion results and These are the corresponding scale-encoded and decoded features. For the first Layer encoding or decoding layer.

7. The hyperspectral fusion imaging method according to claim 1, characterized in that, Step 4) includes the following detailed steps: 1) The multi-scale fusion results and shallow gradient priors are aggregated step by step through the gradient-aware fusion module to obtain the enhanced fusion result; 2) The fusion result obtained through the gradient-aware fusion module is convolved and residual correction is performed with the upsampled LR-HSI to obtain the final fusion result, so as to reconstruct a high-resolution hyperspectral image.

8. The hyperspectral fusion imaging method according to claim 7, characterized in that, In step 1), the function expression for the step-by-step aggregation of the multi-scale fusion results and shallow gradient priors through the gradient-aware fusion module is as follows: ， In the above formula, The result is a multi-scale fusion. It is a shallow gradient prior. These are features that have been aggregated step by step by the gradient-aware fusion module. This is a gradient-aware fusion module; In step 2), the fusion result obtained through the gradient-aware fusion module is convolved and residual correction is performed with the upsampled LR-HSI to obtain the final fusion result. The function expression is as follows: ， In the above formula, For upsampled LR-HSI, The final fusion result is a high-resolution hyperspectral image, where Conv is the convolution operation.

9. A hierarchical gradient-guided hyperspectral fusion imaging method, comprising an interconnected microprocessor and a memory, characterized in that, The microprocessor is programmed or configured to execute the hierarchical gradient-guided hyperspectral fusion imaging method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the hierarchical gradient-guided hyperspectral fusion imaging method according to any one of claims 1 to 7 via a processor.

11. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the hierarchical gradient-guided hyperspectral fusion imaging method according to any one of claims 1 to 7 via a processor.

Images