Hyperspectral image fusion method and system based on non-uniform features and dynamic convolution

Abstract

The present invention relates to the technical field of hyperspectral image processing, and provides a hyperspectral image fusion method and system based on non-uniform features and dynamic convolution. The method comprises extracting image features of a PAN image and an upsampled LRHS image through a convolution module; extracting features of the PAN image through spatial curvature dynamic convolution, and extracting features of the upsampled LRHS image through spectral curvature dynamic convolution; performing feature fusion through a non-uniform feature integration unit, fusing output features with spectral convolution features to obtain current-scale fusion features; performing multi-scale feature extraction and fusion of current-scale fusion features and spatial convolution features through a non-uniform fusion module and a jump connection; and performing a residual operation on the multi-scale fusion features to obtain a hyperspectral image. The present invention significantly improves the spatial resolution of hyperspectral images, and provides technical support for the refined analysis and practical application of hyperspectral images.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image processing, and in particular to a hyperspectral image fusion method and system based on non-uniform features and dynamic convolution. Background Art

[0002] Hyperspectral (HS) images are obtained by sampling hundreds of continuous narrow spectral bands using a spectral imaging system. They provide rich spectral information and are commonly used to study the spectral differences in materials. However, due to the physical design limitations of optical imaging systems, the imaging results require a compromise between spectral resolution and spatial resolution. Single-spectral imaging systems are unable to obtain high-spatial-resolution hyperspectral (HRHS) images, and their imaging results often produce low-spatial-resolution hyperspectral (LRHS) images. This limits the application of HS images in fields such as mineral detection, ecosystem monitoring, and agricultural testing. Panchromatic (PAN) imaging systems, which only output single-band PAN images, have high spatial resolution but low spectral resolution.

[0003] By analyzing the feature distributions of PAN, LRHS, and ground truth (GT) images, a "non-uniformity phenomenon" was discovered, which can lead to inaccurate fusion features. This phenomenon, previously unnoticed in hyperspectral panchromatic tasks, also results in localized information loss in the final fusion results of previous methods. This non-uniformity phenomenon suggests that image fusion requires fine-grained information fusion in localized regions to avoid information injection errors, which can lead to spatial and spectral distortions in the fusion results. Previous deep learning-based methods for fusing HS and PAN images have neglected the physical interpretability of the fusion process, often employing a "black-box" approach, which limits the reliability of network applications. Hyperspectral images, due to their large number of spectral channels, present a high computational complexity for data processing. When processing large-scale data, methods such as matrix decomposition and sparse representation often involve complex iterative optimization processes, resulting in high computational costs and difficulty meeting real-time processing requirements. Existing deep learning-based models have large parameter counts, high resource requirements for training and inference, and demanding hardware performance, making them difficult to apply in resource-constrained scenarios. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a hyperspectral image fusion method and system based on non-uniform features and dynamic convolution. The method uses spatial curvature-guided dynamic convolution (SpaC-DConv) and spectral curvature-guided dynamic convolution (SpeC-DConv) to extract and enhance the features of hyperspectral images and panchromatic images, and constructs a nonuniform feature integration unit (NFIU) to achieve the fusion of the two image features at each scale. The present invention provides an efficient and physically interpretable fusion method that can significantly improve the spatial resolution of hyperspectral images while maintaining their spectral consistency, providing technical support for the refined analysis and practical application of hyperspectral images.

[0005] The present invention provides a hyperspectral image fusion method based on non-uniform features and dynamic convolution, comprising:

[0006] S1: extract the image features of the PAN image through the convolution module to obtain the PAN image features; extract the image features of the upsampled LRHS image through the convolution module to obtain the upsampled LRHS image features;

[0007] S2: Extract the PAN image features through spatial curvature dynamic convolution to obtain spatial convolution features, and extract the upsampled LRHS image features through spectral curvature dynamic convolution to obtain spectral convolution features;

[0008] S3: The upsampled LRHS image features, PAN image features, spatial convolution features, and spectral convolution features are fused through the non-uniform feature integration unit to obtain non-uniform fusion features;

[0009] S4: Fuse the non-uniform fusion features and the spectral convolution features to obtain the current scale fusion features;

[0010] S5: Multi-scale feature extraction and fusion of the current scale fusion feature and spatial convolution feature are performed through the non-uniform fusion module and jump connection to obtain multi-scale fusion features;

[0011] S6: Perform residual operation on the multi-scale fusion features to obtain a hyperspectral image.

[0012] Furthermore, in step S2, obtaining spatial convolution features includes:

[0013] S211: Calculating the curvature value of the PAN image feature by using a spatial curvature function to obtain a spatial curvature map;

[0014] S212: using the local patch of the spatial curvature map as a dynamic weight to adjust the spatial curvature convolution kernel to obtain a dynamic spatial curvature convolution kernel;

[0015] S213: Performing a convolution operation on the PAN image features through a dynamic spatial curvature convolution kernel to obtain spatial convolution features.

[0016] Furthermore, in step S2, obtaining spectral convolution features includes:

[0017] S221: Calculate the curvature value of the upsampled LRHS image feature in the channel dimension using the spectral curvature function to obtain a spectral curvature map;

[0018] S222: using the local patch of the spectral curvature map as a dynamic weight-adjusted spectral curvature convolution kernel to obtain a dynamic spectral curvature convolution kernel;

[0019] S223: Perform a convolution operation on the upsampled LRHS image features through a dynamic spectral curvature convolution kernel to obtain spectral convolution features.

[0020] Furthermore, the non-uniform feature integration unit includes a feature selection fusion module and a detail fusion module.

[0021] The feature selection and fusion module selects and fuses the non-uniform features of the PAN image and the upsampled LRHS image through fuzzy logic fusion rules to obtain the fusion features of the feature selection and fusion module;

[0022] The detail fusion module obtains a detail weight map by fusing the spatial curvature map and the spectral curvature map, and weights the fusion features of the feature selection fusion module through the detail weight map.

[0023] Furthermore, the feature selection fusion module includes:

[0024] Calculate the structural similarity between local blocks in the PAN image and the upsampled LRHS image to obtain the first similarity feature;

[0025] Calculate the similarity between the local blocks in the Gaussian blurred PAN image and the upsampled LRHS image to obtain the second similarity feature;

[0026] defining a first fuzzy logic soft threshold and a second fuzzy logic soft threshold according to the first similarity feature and the second similarity feature;

[0027] formulating a fuzzy logic fusion rule according to the first fuzzy logic soft threshold and the second fuzzy logic soft threshold;

[0028] The non-uniform features of PAN image and upsampled LRHS image are obtained according to fuzzy logic fusion rules.

[0029] Furthermore, the fuzzy logic fusion rule is:

[0030] Set the critical fuzzy logic soft threshold,

[0031] When the first fuzzy logic soft threshold is greater than or equal to the critical fuzzy logic soft threshold, the upsampled LRHS image feature is selected;

[0032] When the first fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold and the second fuzzy logic soft threshold is greater than or equal to the critical fuzzy logic soft threshold, selecting the PAN image feature;

[0033] When the first fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold and the second fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold, the upsampled LRHS image feature is selected.

[0034] Furthermore, the calculation expression of the detail fusion module is:

[0035]

[0036] in, for scale-detail weight map, for The local entropy of the scale spectral curvature, for Local entropy of scale-space curvature, is the spectral curvature diagram, is the spatial curvature graph, This is pixel-by-pixel multiplication.

[0037] Furthermore, the non-uniform fusion module includes spatial curvature dynamic convolution, spectral curvature dynamic convolution and a non-uniform feature integration unit.

[0038] Furthermore, the current scale fusion feature is downsampled, and the downsampled current scale fusion feature is used as the input feature of the next scale layer.

[0039] The present invention also provides a hyperspectral image fusion system based on non-uniform features and dynamic convolution, which is used to execute any of the above-mentioned hyperspectral image fusion methods based on non-uniform features and dynamic convolution, comprising:

[0040] a first feature extraction module, wherein the first feature extraction module extracts image features of the PAN image through the convolution module to obtain PAN image features; and extracts image features of the upsampled LRHS image through the convolution module to obtain upsampled LRHS image features;

[0041] a second feature extraction module, wherein the second feature extraction module extracts features of the PAN image by spatial curvature dynamic convolution to obtain spatial convolution features, and extracts features of the upsampled LRHS image by spectral curvature dynamic convolution to obtain spectral convolution features;

[0042] A non-uniform integration module, wherein the non-uniform integration module performs feature fusion on the upsampled LRHS image features, PAN image features, spatial convolution features, and spectral convolution features through a non-uniform feature integration unit to obtain non-uniform fusion features;

[0043] A current scale fusion module, wherein the current scale fusion module fuses the non-uniform fusion feature and the spectral convolution feature to obtain the current scale fusion feature;

[0044] A multi-scale fusion module, which extracts and fuses multi-scale features of the current scale fusion feature and the spatial convolution feature through a non-uniform fusion module and a jump connection to obtain a multi-scale fusion feature;

[0045] The residual module performs a residual operation on the multi-scale fusion features to obtain a hyperspectral image.

[0046] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0047] This paper extracts and enhances the features of hyperspectral and panchromatic images through dynamic convolution of spatial and spectral curvature. By constructing a non-uniform feature integration unit, it achieves the fusion of the two image features at each scale. This paper provides an efficient and physically interpretable fusion method that can significantly improve the spatial resolution of hyperspectral images while maintaining their spectral consistency, providing technical support for the refined analysis and practical application of hyperspectral images.

[0048] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1This is a flow chart of a hyperspectral image fusion method based on non-uniform features and dynamic convolution provided by the present invention.

[0051] Figure 2 This is a schematic diagram of the network structure of a hyperspectral image fusion method based on non-uniform features and dynamic convolution provided by the present invention.

[0052] Figure 3 This is a comparison diagram of different images and real pixel values as the row coordinates of the image blocks change according to the embodiment of the present invention.

[0053] Figure 4 This is a schematic diagram of the spatial curvature dynamic convolution and spectral curvature dynamic convolution structure of a hyperspectral image fusion method based on non-uniform features and dynamic convolution provided by the present invention.

[0054] Figure 5 This is a schematic diagram of the structure of a non-uniform feature integration unit of a hyperspectral image fusion method based on non-uniform features and dynamic convolution provided by the present invention.

[0055] Figure 6 It is a structural schematic diagram of a hyperspectral image fusion system based on non-uniform features and dynamic convolution provided by the present invention.

[0056] Reference numerals:

[0057] 101. First feature extraction module; 102. Second feature extraction module; 103. Non-uniform integration module; 104. Current scale fusion module; 105. Multi-scale fusion module; 106. Residual module. DETAILED DESCRIPTION

[0058] To make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Obviously, the embodiments described are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0059] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0060] The following combination Figures 1 to 6 The present invention describes a hyperspectral image fusion method and system based on non-uniform features and dynamic convolution.

[0061] like Figure 1 As shown, a hyperspectral image fusion method based on non-uniform features and dynamic convolution includes:

[0062] S1: extract the image features of the PAN image through the convolution module to obtain the PAN image features; extract the image features of the upsampled LRHS image through the convolution module to obtain the upsampled LRHS image features;

[0063] The calculation expression is:

[0064]

[0065]

[0066] in, is the first scale PAN image feature, is a convolution operation with a convolution kernel of 3×3. For PAN images, is the first scale upsampled LRHS image feature, is the upsampled LRHS image.

[0067] like Figure 2 As shown in FIG, the present invention adopts a dual-branch U-type network architecture, which is an encoder-decoder structure, including a spectral U-type network and a spatial U-type network. The spectral U-type network is used to extract features of different scales from the upsampled LRHS image, and the spatial U-type network is used to extract features of different scales from the PAN image. and As the first-scale input features of the encoder in the spectral U-network and spatial U-network.

[0068] S2: Extract the PAN image features through spatial curvature dynamic convolution to obtain spatial convolution features, and extract the upsampled LRHS image features through spectral curvature dynamic convolution to obtain spectral convolution features;

[0069] like Figure 4 As shown in Figure (a), obtaining spatial convolution features includes:

[0070] S211: Calculating the curvature value of the PAN image feature by using a spatial curvature function to obtain a spatial curvature map;

[0071] S212: using the local patch of the spatial curvature map as a dynamic weight to adjust the spatial curvature convolution kernel to obtain a dynamic spatial curvature convolution kernel;

[0072] S213: Performing a convolution operation on the PAN image features through a dynamic spatial curvature convolution kernel to obtain spatial convolution features.

[0073] Through the space curvature function calculate Scaled PAN image features The curvature value of , obtain the spatial curvature map , , is the number of channels of the feature map, is the height of the feature map, is the width of the feature map, is a set of real vectors, and the space curvature graph Local patches in Used as dynamic weights, , to adjust the dynamic spatial curvature convolution kernel , ,in is the number of channels of the output feature map, is the size of the convolution kernel in the two-dimensional plane, and the dynamic spatial curvature convolution kernel is Convolution operations are performed on the kernel to extract accurate features. The specific operation of spatial curvature dynamic convolution is expressed as:

[0074]

[0075]

[0076] in, for Scale PAN image features, In space coordinates A local patch centered at the eigenvalue at , is the spatial coordinate The convolution result at , , is the convolution operation, This is pixel-by-pixel multiplication.

[0077] Figure 4 In Figure (a) for Scaled PAN image convolution results.

[0078] because By effectively representing the structural information of the image, SpaC-DConv can more accurately extract features from different regions, which enhances the model's ability to process images with different complex contents.

[0079] like Figure 4 As shown in Figure (b), the spectral convolution features obtained include:

[0080] S221: Calculate the curvature value of the upsampled LRHS image feature in the channel dimension using the spectral curvature function to obtain a spectral curvature map;

[0081] S222: using the local patch of the spectral curvature map as a dynamic weight-adjusted spectral curvature convolution kernel to obtain a dynamic spectral curvature convolution kernel;

[0082] S223: Perform a convolution operation on the upsampled LRHS image features through a dynamic spectral curvature convolution kernel to obtain spectral convolution features.

[0083] Aiming at the rich spectral information in upsampled LRHS images, SpeC-DConv aims to dynamically adjust model parameters by calculating the spectral curvature weight map in the channel dimension to extract and enhance the features of low-resolution hyperspectral images.

[0084] Through the spectral curvature function Computing in the channel dimension Upsampling LRHS image features , The curvature value of the spectrum is obtained , , taking into account the spectral correlation between adjacent channels, the spectral curvature map Local patches in Used as dynamic weights, , to adjust the convolution kernel , , convolution in Convolution operations are performed on the network to extract accurate features. The specific operation of SpeC-DConv is expressed as:

[0085]

[0086]

[0087] in, for Scale upsampled LRHS image features, For the coordinates A local patch centered at the eigenvalue at , is the spatial coordinate The convolution result at , .

[0088] Figure 4 In Figure (b) is the convolution result of the upsampled LRHS image.

[0089] SpeC-DConv can enhance some important spectral features and better capture subtle changes between spectral channels.

[0090] S3: The upsampled LRHS image features, PAN image features, spatial convolution features, and spectral convolution features are fused through the non-uniform feature integration unit to obtain non-uniform fusion features;

[0091] The non-uniform feature integration unit (NFIU) is used to fuse the features of PAN images and upsampled LRHS images at different scales.

[0092] like Figure 5 As shown in Figure 1, the non-uniform feature integration unit includes a feature selection fusion block (FSFB) and a detail fusion block (DFB). The feature selection fusion block selects and fuses the non-uniform features of the PAN image and the upsampled LRHS image through fuzzy logic fusion rules to obtain the fusion features of the feature selection fusion block. The detail fusion block obtains the detail weight map by fusing the spatial curvature map and the spectral curvature map. The fusion features of the feature selection fusion block are weighted by the detail weight map to enhance the detail information. The calculation expression is:

[0093]

[0094]

[0095] in, is the output of the non-uniform feature integration unit at the s-th scale, For FSFB operation, For DFB operation, The fusion features of the feature selection fusion module, is the detail weight map.

[0096] Due to the presence of non-uniformity, when fusing PAN image features with upsampled LRHS image features, it is important to focus on features that are more similar to those in the GT image to reduce inaccurate fused features caused by feature aliasing. Based on this idea, this paper utilizes fuzzy logic theory to design a feature selection and fusion module for selecting and fusing the two image features. The key to this feature selection and fusion module lies in the definition of fuzzy logic fusion rules.

[0097] like Figure 3 As shown in Figure 2, the local structures of the three images have the following characteristics, which can guide feature selection during the fusion process:

[0098] like Figure 3 In the purple dashed box, some regions in the three images have similar structural information. For these regions, in order to ensure spectral fidelity, the upsampled LRHS image features are emphasized in feature fusion.

[0099] like Figure 3 In the black dotted box, there are differences in the structural information of some areas in the upsampled LRHS image and the PAN image, while the structures of these areas in the upsampled LRHS image and the Gaussian blurred PAN image are similar, which indicates that these areas in the PAN image may contain rich detail information. In order to integrate the rich detail information, the features of the PAN image are emphasized in feature fusion.

[0100] like Figure 3 In the middle green dashed box, the structural information of some regions in the three images is significantly different. In order to reduce the spatial structure distortion caused by feature aliasing, the upsampled LRHS image features are emphasized in feature fusion to ensure spectral fidelity.

[0101] The feature selection fusion module includes:

[0102] Calculate the structural similarity between local blocks in the PAN image and the upsampled LRHS image to obtain the first similarity feature;

[0103] Calculate the similarity between the local blocks in the Gaussian blurred PAN image and the upsampled LRHS image to obtain the second similarity feature;

[0104] defining a first fuzzy logic soft threshold and a second fuzzy logic soft threshold according to the first similarity feature and the second similarity feature;

[0105] Formulate a fuzzy logic fusion rule according to the first fuzzy logic soft threshold, the second fuzzy logic soft threshold and the image feature;

[0106] The non-uniform features of PAN image and upsampled LRHS image are obtained according to fuzzy logic fusion rules.

[0107] The present invention defines a fuzzy logic fusion rule, in which the definition of the fuzzy logic soft threshold is the key to feature selection. Structural similarity (SSIM) is used to measure the feature similarity between local blocks in two images. It is necessary to measure the similarity between local blocks in the full-color image and the upsampled LRHS image; measure the similarity between local blocks in the Gaussian blurred PAN image and the upsampled LRHS image, and define the first fuzzy logic soft threshold. and the second fuzzy logic soft threshold , and They are all monotonically increasing functions of the SSIM value of the local block, and the calculation expression is:

[0108]

[0109]

[0110] in, is the spatial position coordinate in the feature map, is an exponential function, is the spatial coordinate Upsampled LRHS image features at is the spatial coordinate PAN image features at is the SSIM calculation for local feature patches, is the Gaussian blur operation, is a hyperparameter used to adjust the impact of the SSIM value on the function change trend.

[0111] The fuzzy logic fusion rules are:

[0112] Set the critical fuzzy logic soft threshold,

[0113] When the first fuzzy logic soft threshold is greater than or equal to the critical fuzzy logic soft threshold, the upsampled LRHS image feature is selected;

[0114] When the first fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold and the second fuzzy logic soft threshold is greater than or equal to the critical fuzzy logic soft threshold, selecting the PAN image feature;

[0115] When the first fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold and the second fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold, the upsampled LRHS image feature is selected.

[0116] In some specific embodiments of the present invention, the critical fuzzy logic soft threshold is 0.5.

[0117] The calculation expression of fuzzy logic fusion rule is:

[0118]

[0119] in, The fused features are selected for the feature fusion module.

[0120] In the non-uniform feature integration unit, in order to enhance the image detail information while maintaining the spectral fidelity, the present invention designs a detail fusion module based on the local entropy theory. DFB fuses the spatial curvature map obtained by SpaC-DConv and SpeC-DConv. and spectral curvature diagram To get the detail weight map Because local entropy can reflect the information richness of the local area of the image, the local entropy of the local area is calculated to achieve and For the selection of two features, the detail weight map can be obtained by the following formula:

[0121]

[0122]

[0123] in, for Scale local entropy, is the curvature graph, for Scale curvature map, for The probability of , To find the local entropy function, for Scale curvature map In spatial coordinates Local patches at is the detail weight map, for The local entropy of the scale spectral curvature, for Local entropy of scale-space curvature, is the spectral curvature diagram, is the spatial curvature graph, For pixel-by-pixel multiplication, For space, For the spectrum.

[0124] By comprehensively considering spatial and spectral features, higher quality image fusion results can be obtained.

[0125] S4: Fuse the non-uniform fusion features and the spectral convolution features to obtain the current scale fusion features;

[0126] The non-uniform fusion features are fused with the spectral convolution features through addition operation to obtain the current scale fusion features.

[0127] S5: Multi-scale feature extraction and fusion of the current scale fusion feature and spatial convolution feature are performed through the non-uniform fusion module and jump connection to obtain multi-scale fusion features;

[0128] The non-uniform fusion module includes spatial curvature dynamic convolution, spectral curvature dynamic convolution and non-uniform feature integration unit.

[0129] The current scale fusion feature is downsampled and used as the input feature of the next scale layer of the spectral U-type network.

[0130] The spatial convolution features are downsampled and used as the input features of the next scale layer of the spatial U-shaped network.

[0131] Both the spectral U-net and the spatial U-net are encoder-decoder structures. In the decoder, taking the decoder's s-th scale calculation as an example, the same operations as the encoder at the same scale are performed at each scale. At the same time, according to the calculation of the standard U-net structure, the output of the corresponding encoder s stage is added to the input features of the decoder at the s-th scale through a jump connection to obtain multi-scale fusion features.

[0132] S6: Perform residual operation on the multi-scale fusion features to obtain the hyperspectral image. The calculation expression is:

[0133]

[0134] in, For high spatial resolution hyperspectral images, is the residual network operation, It is a multi-scale fusion feature.

[0135] like Figure 6 As shown, a hyperspectral image fusion system based on non-uniform features and dynamic convolution is used to perform a hyperspectral image fusion method based on non-uniform features and dynamic convolution, including:

[0136] The first feature extraction module 101 extracts the image features of the PAN image through the convolution module to obtain the PAN image features; extracts the image features of the upsampled LRHS image through the convolution module to obtain the upsampled LRHS image features;

[0137] The second feature extraction module 102 extracts the PAN image features by spatial curvature dynamic convolution to obtain spatial convolution features, and extracts the upsampled LRHS image features by spectral curvature dynamic convolution to obtain spectral convolution features;

[0138] The non-uniform integration module 103 performs feature fusion on the upsampled LRHS image features, PAN image features, spatial convolution features and spectral convolution features through a non-uniform feature integration unit to obtain non-uniform fusion features;

[0139] The current scale fusion module 104 performs feature fusion on the non-uniform fusion feature and the spectral convolution feature to obtain the current scale fusion feature;

[0140] The multi-scale fusion module 105 extracts and fuses the current scale fusion feature and the spatial convolution feature through the non-uniform fusion module and the jump connection to obtain the multi-scale fusion feature;

[0141] The residual module 106 performs a residual operation on the multi-scale fusion features to obtain a hyperspectral image.

[0142] Through the collaborative work of the above modules, spatial curvature dynamic convolution and spectral curvature dynamic convolution realize feature extraction and enhancement of hyperspectral images and panchromatic images, and the fusion of the two image features is realized at each scale through the non-uniform feature integration unit. The present invention aims to provide an efficient and physically interpretable fusion method that can significantly improve the spatial resolution of hyperspectral images while maintaining their spectral consistency, providing technical support for the refined analysis and practical application of hyperspectral images.

[0143] This method was compared with several state-of-the-art algorithms on the public Pavia Center dataset using quantitative and subjective metrics. To better visualize the differences between the fusion results, the root mean square error (RMSE) between the fusion results and the ground truth (GT) was calculated and displayed. The state-of-the-art algorithms used were Refiner (2023), TreeNet (2024), DFCFN (2025), and FPFNet (2025). Refiner (2023) is the 2023 version of the Refined Hyperspectral Pan-spectral Network, TreeNet (2024) is the 2024 version of the Tree-structured Neural Network, DFCFN (2025) is the 2025 version of the Dual-stage Feature Correction Fusion Network, and FPFNet (2025) is the 2025 version of the Feature Pyramid Fusion Network. As shown in Table 1, the quantitative metrics of the proposed algorithm demonstrate that they outperform those of recent state-of-the-art algorithms. The experimental results are summarized in Table 1.

[0144] Table 1 Comparative experiment results of quantitative indicators of Pavia center dataset

[0145]

[0146] Among them, CC is the correlation coefficient, SAM is the spectral angle mapping, RMSE is the root mean square error, ERGAS is the dimensionless global relative error, PSNR is the peak signal-to-noise ratio, "↑" means the higher the value, the better, and "↓" means the lower the value, the better.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A hyperspectral image fusion method based on non-uniform features and dynamic convolution, characterized in that: include: S1: extract the image features of the PAN image through the convolution module to obtain the PAN image features; extract the image features of the upsampled LRHS image through the convolution module to obtain the upsampled LRHS image features; S2: Extract the PAN image features through spatial curvature dynamic convolution to obtain spatial convolution features, and extract the upsampled LRHS image features through spectral curvature dynamic convolution to obtain spectral convolution features; S3: The upsampled LRHS image features, PAN image features, spatial convolution features, and spectral convolution features are fused through the non-uniform feature integration unit to obtain non-uniform fusion features; The non-uniform feature integration unit includes a feature selection fusion module and a detail fusion module. The feature selection and fusion module selects and fuses the non-uniform features of the PAN image and the upsampled LRHS image through fuzzy logic fusion rules to obtain the fusion features of the feature selection and fusion module; The detail fusion module obtains a detail weight map by fusing the spatial curvature map and the spectral curvature map, and weights the fusion features of the feature selection fusion module through the detail weight map; S4: Fuse the non-uniform fusion features and the spectral convolution features to obtain the current scale fusion features; S5: Multi-scale feature extraction and fusion of the current scale fusion feature and spatial convolution feature are performed through the non-uniform fusion module and jump connection to obtain multi-scale fusion features; S6: Perform residual operation on the multi-scale fusion features to obtain a hyperspectral image.

2. The hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 1, characterized in that: In step S2, the spatial convolution features are obtained including: S211: Calculating the curvature value of the PAN image feature by using a spatial curvature function to obtain a spatial curvature map; S212: using the local patch of the spatial curvature map as a dynamic weight to adjust the spatial curvature convolution kernel to obtain a dynamic spatial curvature convolution kernel; S213: Performing a convolution operation on the PAN image features through a dynamic spatial curvature convolution kernel to obtain spatial convolution features.

3. The hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 1 is characterized in that: In step S2, the spectral convolution features are obtained including: S221: Calculate the curvature value of the upsampled LRHS image feature in the channel dimension using the spectral curvature function to obtain a spectral curvature map; S222: using the local patch of the spectral curvature map as a dynamic weight-adjusted spectral curvature convolution kernel to obtain a dynamic spectral curvature convolution kernel; S223: Perform a convolution operation on the upsampled LRHS image features through a dynamic spectral curvature convolution kernel to obtain spectral convolution features.

4. The hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 1, characterized in that: The feature selection fusion module includes: Calculate the structural similarity between local blocks in the PAN image and the upsampled LRHS image to obtain the first similarity feature; Calculate the similarity between the local blocks in the Gaussian blurred PAN image and the upsampled LRHS image to obtain the second similarity feature; defining a first fuzzy logic soft threshold and a second fuzzy logic soft threshold according to the first similarity feature and the second similarity feature; formulating a fuzzy logic fusion rule according to the first fuzzy logic soft threshold and the second fuzzy logic soft threshold; The non-uniform features of PAN image and upsampled LRHS image are obtained according to fuzzy logic fusion rules.

5. The hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 4 is characterized in that: The fuzzy logic fusion rules are: Set the critical fuzzy logic soft threshold, When the first fuzzy logic soft threshold is greater than or equal to the critical fuzzy logic soft threshold, the upsampled LRHS image feature is selected; When the first fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold and the second fuzzy logic soft threshold is greater than or equal to the critical fuzzy logic soft threshold, selecting the PAN image feature; When the first fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold and the second fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold, the upsampled LRHS image feature is selected.

6. The hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 1, characterized in that: The calculation expression of the detail fusion module is: in, for scale-detail weight map, for The local entropy of the scale spectral curvature, for The local entropy of scale-space curvature, is the spectral curvature diagram, is the space curvature graph, This is pixel-by-pixel multiplication.

7. The hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 1, characterized in that: The non-uniform fusion module includes spatial curvature dynamic convolution, spectral curvature dynamic convolution and a non-uniform feature integration unit.

8. The hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 1, characterized in that: The current scale fusion feature is downsampled and used as the input feature of the next scale layer.

9. A hyperspectral image fusion system based on non-uniform features and dynamic convolution, characterized in that: The method for performing a hyperspectral image fusion method based on non-uniform features and dynamic convolution as claimed in any one of claims 1 to 8 comprises: a first feature extraction module, wherein the first feature extraction module extracts image features of the PAN image through the convolution module to obtain PAN image features; and extracts image features of the upsampled LRHS image through the convolution module to obtain upsampled LRHS image features; a second feature extraction module, wherein the second feature extraction module extracts features of the PAN image by spatial curvature dynamic convolution to obtain spatial convolution features, and extracts features of the upsampled LRHS image by spectral curvature dynamic convolution to obtain spectral convolution features; A non-uniform integration module, wherein the non-uniform integration module performs feature fusion on the upsampled LRHS image features, PAN image features, spatial convolution features, and spectral convolution features through a non-uniform feature integration unit to obtain non-uniform fusion features; A current scale fusion module, wherein the current scale fusion module fuses the non-uniform fusion feature and the spectral convolution feature to obtain the current scale fusion feature; A multi-scale fusion module, which extracts and fuses multi-scale features of the current scale fusion feature and the spatial convolution feature through a non-uniform fusion module and a jump connection to obtain a multi-scale fusion feature; The residual module performs a residual operation on the multi-scale fusion features to obtain a hyperspectral image.

Images