Hyperspectral image fusion method and system based on non-uniform features and dynamic convolution

Abstract

The invention relates to the technical field of hyperspectral image processing, and provides a hyperspectral image fusion method and system based on non-uniform features and dynamic convolution, and the method comprises the steps: extracting the image features of a PAN image and an up-sampling LRHS image through a convolution module; performing feature extraction on PAN image features through spatial curvature dynamic convolution, and performing feature extraction on up-sampling LRHS image features through spectral curvature dynamic convolution; performing feature fusion through a non-uniform feature integration unit, and performing feature fusion on the output feature and the spectral convolution feature to obtain a current scale fusion feature; performing multi-scale feature extraction and fusion on the current scale fusion features and the spatial convolution features through a non-uniform fusion module and jump connection; and performing residual operation on the multi-scale fusion features to obtain a hyperspectral image. According to the method, the spatial resolution of the hyperspectral image is remarkably improved, and technical support is provided for fine analysis and practical application of the hyperspectral image.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image processing, and particularly to a hyperspectral image fusion method and system based on non-uniform features and dynamic convolution. Background Art

[0002] Hyperspectral (HS) images are obtained by sampling hundreds of continuous narrow spectral bands using a spectral imaging system, and can provide rich spectral information, which is often used for the research of spectral difference characteristics of materials. However, due to the physical design limitations of the optical imaging system, the imaging result needs to be considered as a trade-off between spectral resolution and spatial resolution. A single-spectral imaging system cannot obtain a high-resolution hyperspectral (HRHS) image, and its imaging result often obtains a low-resolution hyperspectral (LRHS) image, which limits the application of HS images in fields such as mineral detection, ecosystem monitoring, and agricultural detection. The panchromatic (PAN) imaging system only outputs a single-band imaging PAN image, which has a high spatial resolution but a low spectral resolution.

[0003] By analyzing the feature distributions of PAN, LRHS, and ground truth (GT) images, a "non-uniformity phenomenon" that may lead to inaccurate fusion features is found, which has not been noticed in the hyperspectral panchromatic task, and this also causes local information loss in the final fusion result of previous methods. The existence of the non-uniformity phenomenon indicates that when performing image fusion, fine information fusion needs to be carried out in the local area to avoid incorrect information injection, resulting in spatial and spectral distortion of the fusion result; previous deep learning-based methods ignored the physical interpretability of the fusion process when fusing HS and PAN images, and often used a "black box" method for fusion, which limits the credibility of network applications. Due to the large number of spectral channels in hyperspectral images, the computational complexity of data processing is relatively high. When dealing with large-scale data, methods such as matrix decomposition and sparse representation usually involve complex iterative optimization processes, with high computational costs and difficulty in meeting real-time processing requirements. Existing deep learning-based models have a large number of parameters, high resource requirements for the training and inference processes, and high requirements for hardware performance, making it difficult to apply to resource-constrained scenarios. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a hyperspectral image fusion method and system based on non-uniform features and dynamic convolution. Through spatial curvature-guided dynamic convolution (SpaC-DConv) and spectral curvature-guided dynamic convolution (SpeC-DConv), feature extraction and enhancement of hyperspectral images and panchromatic images are realized. By constructing a non-uniform feature integration unit (NFIU), fusion of the features of the two images is achieved at each scale. The present invention provides an efficient and physically interpretable fusion method, which can significantly improve the spatial resolution of hyperspectral images while maintaining their spectral consistency, providing technical support for the refined analysis and practical application of hyperspectral images.

[0005] The present invention provides a hyperspectral image fusion method based on non-uniform features and dynamic convolution, including: S1: Extract the image features of the PAN image through a convolution module to obtain PAN image features; extract the image features of the upsampled LRHS image through a convolution module to obtain upsampled LRHS image features; S2: Extract features from the PAN image features through spatial curvature dynamic convolution to obtain spatial convolution features, and extract features from the upsampled LRHS image features through spectral curvature dynamic convolution to obtain spectral convolution features; S3: Perform feature fusion on the upsampled LRHS image features, PAN image features, spatial convolution features, and spectral convolution features through a non-uniform feature integration unit to obtain non-uniform fusion features; S4: Perform feature fusion on the non-uniform fusion features and the spectral convolution features to obtain the current scale fusion features; S5: Perform multi-scale feature extraction and fusion on the current scale fusion features and the spatial convolution features through a non-uniform fusion module and a skip connection to obtain multi-scale fusion features; S6: Perform a residual operation on the multi-scale fusion features to obtain a hyperspectral image.

[0006] Further, in step S2, obtaining the spatial convolution features includes: S211: Calculate the curvature value of the PAN image features through a spatial curvature function to obtain a spatial curvature map; S212: Use the local patches of the spatial curvature map as dynamic weights to adjust the spatial curvature convolution kernel to obtain a dynamic spatial curvature convolution kernel; S213: Convolve the PAN image features through a dynamic spatial curvature convolution kernel to obtain spatial convolution features.

[0007] Further, in step S2, obtaining the spectral convolution features includes: S221: Calculate the curvature values of the upsampled LRHS image features in the channel dimension through a spectral curvature function to obtain a spectral curvature map; S222: Use the local patches of the spectral curvature map as dynamic weights to adjust the spectral curvature convolution kernel to obtain a dynamic spectral curvature convolution kernel; S223: Convolve the upsampled LRHS image features through the dynamic spectral curvature convolution kernel to obtain spectral convolution features.

[0008] Further, the non-uniform feature integration unit includes a feature selection and fusion module and a detail fusion module. The feature selection and fusion module selects and fuses the non-uniform features of the PAN image and the upsampled LRHS image through a fuzzy logic fusion rule to obtain the fusion features of the feature selection and fusion module; The detail fusion module obtains a detail weight map by fusing the spatial curvature map and the spectral curvature map, and weights the fusion features of the feature selection and fusion module through the detail weight map.

[0009] Further, the feature selection and fusion module includes: Calculate the structural similarity between local blocks in the PAN image and the upsampled LRHS image to obtain a first similarity feature; Calculate the similarity between local blocks in the Gaussian-blurred PAN image and the upsampled LRHS image to obtain a second similarity feature; Define a first fuzzy logic soft threshold and a second fuzzy logic soft threshold according to the first similarity feature and the second similarity feature; Formulate a fuzzy logic fusion rule according to the first fuzzy logic soft threshold and the second fuzzy logic soft threshold; Obtain the non-uniform features of the PAN image and the upsampled LRHS image according to the fuzzy logic fusion rule.

[0010] Further, the fuzzy logic fusion rule is: Set a critical fuzzy logic soft threshold. When the first fuzzy logic soft threshold is greater than or equal to the critical fuzzy logic soft threshold, select the upsampled LRHS image features; When the first fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold and the second fuzzy logic soft threshold is greater than or equal to the critical fuzzy logic soft threshold, select the PAN image features; When the first fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold and the second fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold, the upsampled LRHS image features are selected.

[0011] Furthermore, the calculation expression of the detail fusion module is: Where, is the scale detail weight map, is the scale spectral curvature local entropy, is the scale space curvature local entropy, is the spectral curvature map, is the space curvature map, is pixel-by-pixel multiplication.

[0012] Furthermore, the non-uniform fusion module includes a space curvature dynamic convolution, a spectral curvature dynamic convolution, and a non-uniform feature integration unit.

[0013] Furthermore, downsample the current scale fusion feature and use the downsampled current scale fusion feature as the input feature for the next scale layer.

[0014] The present invention also provides a hyperspectral image fusion system based on non-uniform features and dynamic convolution for performing any one of the above-mentioned hyperspectral image fusion methods based on non-uniform features and dynamic convolution, including: A first feature extraction module, which extracts the image features of the PAN image through a convolution module to obtain PAN image features; and extracts the image features of the upsampled LRHS image through a convolution module to obtain upsampled LRHS image features; A second feature extraction module, which extracts features from the PAN image features through a space curvature dynamic convolution to obtain spatial convolution features, and extracts features from the upsampled LRHS image features through a spectral curvature dynamic convolution to obtain spectral convolution features; A non-uniform integration module, which performs feature fusion on the upsampled LRHS image features, PAN image features, spatial convolution features, and spectral convolution features through a non-uniform feature integration unit to obtain non-uniform fusion features; A current scale fusion module, which performs feature fusion on the non-uniform fusion features and spectral convolution features to obtain current scale fusion features; A multi-scale fusion module, which performs multi-scale feature extraction and fusion on the current scale fusion features and spatial convolution features through a non-uniform fusion module and a skip connection to obtain multi-scale fusion features; A residual module that performs a residual operation on the multi-scale fusion features to obtain a hyperspectral image.

[0015] One or more of the above technical solutions in the embodiments of the present invention have at least one of the following technical effects: The present invention uses spatial curvature dynamic convolution and spectral curvature dynamic convolution to achieve feature extraction and enhancement of hyperspectral images and panchromatic images, and constructs a non-uniform feature integration unit to achieve the fusion of the two types of image features at each scale. The present invention provides an efficient and physically interpretable fusion method, which can significantly improve the spatial resolution of hyperspectral images while maintaining their spectral consistency, providing technical support for the refined analysis and practical application of hyperspectral images.

[0016] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a schematic flowchart of a hyperspectral image fusion method based on non-uniform features and dynamic convolution provided by the present invention.

[0019] Figure 2 It is a schematic network structure diagram of a hyperspectral image fusion method based on non-uniform features and dynamic convolution provided by the present invention.

[0020] Figure 3 It is a comparison diagram of different images and true pixel values with the change of the row coordinates of the image block in the embodiments of the present invention.

[0021] Figure 4 It is a schematic structure diagram of the spatial curvature dynamic convolution and spectral curvature dynamic convolution of a hyperspectral image fusion method based on non-uniform features and dynamic convolution provided by the present invention.

[0022] Figure 5 It is a schematic structure diagram of the non-uniform feature integration unit of a hyperspectral image fusion method based on non-uniform features and dynamic convolution provided by the present invention.

[0023] Figure 6 It is a schematic structure diagram of a hyperspectral image fusion system based on non-uniform features and dynamic convolution provided by the present invention.

[0024] Reference numerals: 101, the first feature extraction module; 102, the second feature extraction module; 103, the non-uniform integration module; 104, the current scale fusion module; 105, the multi-scale fusion module; 106, the residual module. Detailed implementation manners

[0025] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without making creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0026] In the description of this specification, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of the present invention. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without conflict, those skilled in the art can combine the different embodiments or examples described in this specification and the features of the different embodiments or examples.

[0027] Next, in combination with Figures 1 to 6 Describe a hyperspectral image fusion method and system based on non-uniform features and dynamic convolution of the present invention.

[0028] As Figure 1 shown, a hyperspectral image fusion method based on non-uniform features and dynamic convolution includes: S1: Extract the image features of the PAN image through a convolution module to obtain the PAN image features; extract the image features of the upsampled LRHS image through a convolution module to obtain the upsampled LRHS image features; The calculation expression is: Wherein, is the first-scale PAN image feature, is the convolution operation with a convolution kernel of 3×3, is the PAN image, Upsample the LRHS image features at the first scale, to upsample the LRHS image.

[0029] As Figure 2 shown, the present invention adopts a dual-branch U-shaped network architecture, which is an encoder-decoder structure, including a spectral U-shaped network and a spatial U-shaped network. The spectral U-shaped network is used to extract features of different scales from the upsampled LRHS image, and the spatial U-shaped network is used to extract features of different scales from the PAN image. and serve as the first-scale input features of the encoders in the spectral U-shaped network and the spatial U-shaped network.

[0030] S2: Extract features from the PAN image features through spatial curvature dynamic convolution to obtain spatial convolution features, and extract features from the upsampled LRHS image features through spectral curvature dynamic convolution to obtain spectral convolution features; As Figure 4 shown in Figure (a) of obtaining the spatial convolution features includes: S211: Calculate the curvature value of the PAN image features through the spatial curvature function to obtain the spatial curvature map; S212: Use the local patches of the spatial curvature map as dynamic weights to adjust the spatial curvature convolution kernel to obtain the dynamic spatial curvature convolution kernel;

[0031] Through the spatial curvature function calculate the curvature value of the scale PAN image features to obtain the spatial curvature map , , , where is the number of channels of the feature map, is the height of the feature map, is the width of the feature map, is the set of real number vectors. Use the local patches in the spatial curvature map as dynamic weights , to adjust the dynamic spatial curvature convolution kernel where is the number of channels of the output feature map, is the size of the convolution kernel in the two-dimensional plane. The dynamic spatial curvature convolution kernel performs convolution operations on Among them, is the scale PAN image feature, is the local patch centered at the eigenvalue at the spatial coordinate , , is the convolution result at the spatial coordinate , , is the convolution operation, is the per-pixel multiplication.

[0032] Figure 4 In the (a) figure of is the convolution result of the scale PAN image.

[0033] Since effectively represents the structural information of the image, SpaC-DConv can extract features more accurately from different regions, which enhances the model's ability to process images with different complex contents.

[0034] As Figure 4 shown in the (b) figure of , obtaining the spectral convolution features includes: S221: Calculate the curvature value of the upsampled LRHS image feature in the channel dimension through the spectral curvature function to obtain the spectral curvature map; S222: Use the local patch of the spectral curvature map as the dynamic weight to adjust the spectral curvature convolution kernel to obtain the dynamic spectral curvature convolution kernel; S223: Perform a convolution operation on the upsampled LRHS image feature through the dynamic spectral curvature convolution kernel to obtain the spectral convolution features.

[0035] Aiming at the rich spectral information in the upsampled LRHS image, SpeC-DConv aims to dynamically adjust the model parameters by calculating the spectral curvature weight map in the channel dimension in order to extract and enhance the features of the low-resolution hyperspectral image.

[0036] Through the spectral curvature function calculate in the channel dimension the curvature value of the scale upsampled LRHS image feature , to obtain the spectral curvature map , , considering the spectral correlation between adjacent channels, use the local patch in the spectral curvature map as the dynamic weight, , to adjust the convolution kernel , The convolution is performed on to perform a convolution operation to extract accurate features. The specific operation of SpeC-DConv is expressed as: where is the upsampled LRHS image feature at scale , is the local patch centered at the feature value at coordinate , , is the spatial coordinate where the convolution result is .

[0037] Figure 4 In the figure (b) of is the convolution result of the upsampled LRHS image.

[0038] SpeC-DConv can enhance some important spectral features and better capture the subtle changes between spectral channels.

[0039] S3: Feature fusion of the upsampled LRHS image features, PAN image features, spatial convolution features, and spectral convolution features is performed through a non-uniform feature integration unit to obtain non-uniform fusion features; The non-uniform feature integration unit (NFIU) is used to fuse the features of the PAN image and the upsampled LRHS image at different scales.

[0040] As Figure 5 shown, the non-uniform feature integration unit includes a feature selection and fusion module (Feature SelectionFusion Block, FSFB) and a detail fusion module (Detail Fusion Block, DFB). The feature selection and fusion module selects and fuses the non-uniform features of the PAN image and the upsampled LRHS image through a fuzzy logic fusion rule to obtain the fusion feature of the feature selection and fusion module; the detail fusion module obtains a detail weight map by fusing the spatial curvature map and the spectral curvature map, and weights the fusion feature of the feature selection and fusion module through the detail weight map to enhance the detail information; the calculation expression is: where is the output of the non-uniform feature integration unit at the s-th scale, is the FSFB operation, is the DFB operation, The fused features for the feature selection and fusion module is the detail weight map.

[0041] Due to the existence of non-uniformity phenomena, when fusing the PAN image features and the upsampled LRHS image features, it is necessary to focus on the features that are more similar to the GT image features to reduce the inaccurate fused features caused by feature aliasing. Based on this idea, the present invention designs a feature selection and fusion module using fuzzy logic theory for selecting and fusing the two types of image features. The key of the feature selection and fusion module lies in the definition of the fuzzy logic fusion rules.

[0042] As Figure 3 shown, the local structures of the three images have the following features, which can guide feature selection during the fusion process: As Figure 3 in the purple dashed box in the middle, some regions in the three images have similar structural information. For these regions, in order to ensure spectral fidelity, the upsampled LRHS image features are emphasized in feature fusion.

[0043] As Figure 3 in the black dashed box in the middle, there are differences in the structural information of some regions between the upsampled LRHS image and the PAN image, while the structures of these regions in the upsampled LRHS image and the Gaussian-blurred PAN image are similar, which indicates that these regions in the PAN image may contain rich detail information. In order to integrate the rich detail information, the features of the PAN image are emphasized in feature fusion.

[0044] As Figure 3 in the green dashed box in the middle, there are significant differences in the structural information of some regions among the three images. In order to reduce the spatial structure distortion caused by feature aliasing, the upsampled LRHS image features are emphasized in feature fusion to ensure spectral fidelity.

[0045] The feature selection and fusion module includes: Calculating the structural similarity between local blocks in the PAN image and the upsampled LRHS image to obtain the first similarity feature; Calculating the similarity between local blocks in the Gaussian-blurred PAN image and the upsampled LRHS image to obtain the second similarity feature; Defining the first fuzzy logic soft threshold and the second fuzzy logic soft threshold according to the first similarity feature and the second similarity feature; Formulating fuzzy logic fusion rules according to the first fuzzy logic soft threshold, the second fuzzy logic soft threshold and the image features; Obtaining the non-uniform features of the PAN image and the upsampled LRHS image according to the fuzzy logic fusion rules.

[0046] The present invention defines a fuzzy logic fusion rule, where the definition of the fuzzy logic soft threshold is the key to achieving feature selection. Structural similarity (SSIM) is used to measure the feature similarity between local blocks in two images. It is necessary to measure the similarity between local blocks in the panchromatic image and the upsampled LRHS image; measure the similarity between local blocks in the PAN image after Gaussian blur and the upsampled LRHS image, and define the first fuzzy logic soft threshold and the second fuzzy logic soft threshold , and are both monotonically increasing functions of the SSIM value of the local block, and the calculation expression is: where are the spatial position coordinates in the feature map, is the exponential function, is the spatial coordinate the feature of the upsampled LRHS image at, is the spatial coordinate the feature of the PAN image at, is the SSIM calculation of the local feature patch, is the Gaussian blur operation, is a hyperparameter used to adjust the influence of the SSIM value on the function change trend.

[0047] The fuzzy logic fusion rule is: Set the critical fuzzy logic soft threshold, When the first fuzzy logic soft threshold is greater than or equal to the critical fuzzy logic soft threshold, select the feature of the upsampled LRHS image; When the first fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold and the second fuzzy logic soft threshold is greater than or equal to the critical fuzzy logic soft threshold, select the feature of the PAN image; When the first fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold and the second fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold, select the feature of the upsampled LRHS image.

[0048] In some specific embodiments of the present invention, the critical fuzzy logic soft threshold is 0.5.

[0049] The calculation expression of the fuzzy logic fusion rule is: where is the fusion feature of the feature selection fusion module.

[0050] In the non-uniform feature integration unit, in order to enhance the image detail information while maintaining the spectral fidelity, the present invention designs a detail fusion module based on the local entropy theory. The DFB obtains the detail weight map by fusing the spatial curvature map and the spectral curvature map . Since the local entropy can reflect the information richness of the local area of the image, the selection of the two features in and is realized by calculating the local entropy of the local area. The detail weight map can be obtained by the following formula: where is the scale local entropy, is the curvature map, is the scale curvature map, is the probability of , is the local entropy function, , is the scale curvature map at the spatial coordinate is the detail weight map, is the scale spectral curvature local entropy, is the scale spatial curvature local entropy, is the spectral curvature map, is the spatial curvature map, is element-wise multiplication, is the space, is the spectrum.

[0051] By comprehensively considering the spatial and spectral features, a higher-quality image fusion result can be obtained.

[0052] S4: Fuse the non-uniform fusion feature and the spectral convolution feature to obtain the current-scale fusion feature; The non-uniform fusion feature is fused with the spectral convolution feature through an addition operation to obtain the current-scale fusion feature.

[0053] S5: Perform multi-scale feature extraction and fusion on the current-scale fusion feature and the spatial convolution feature through the non-uniform fusion module and the skip connection to obtain the multi-scale fusion feature; The non-uniform fusion module includes a spatial curvature dynamic convolution, a spectral curvature dynamic convolution, and a non-uniform feature integration unit. Downsample the current-scale fusion feature, and use the downsampled current-scale fusion feature as the input feature for the next scale layer of the spectral U-Net.

[0054] Downsample the spatial convolution feature, and use the downsampled spatial convolution feature as the input feature for the next scale layer of the spatial U-Net.

[0055] Both the spectral U-Net and the spatial U-Net are encoder-decoder structures. In the decoder, taking the calculation at the s-th scale of the decoder as an example, the same operations as those at the same scale in the encoder are performed at each scale. At the same time, according to the calculation of the standard U-Net structure, the output of the s-th stage of the corresponding encoder is added to the input feature at the s-th scale of the decoder through a skip connection to obtain a multi-scale fusion feature.

[0056] S6: Perform a residual operation on the multi-scale fusion feature to obtain a hyperspectral image. The calculation expression is: where is the hyperspectral image with high spatial resolution, is the residual network operation, is the multi-scale fusion feature.

[0057] As Figure 6 shown, a hyperspectral image fusion system based on non-uniform features and dynamic convolution is used to execute a hyperspectral image fusion method based on non-uniform features and dynamic convolution, including: The first feature extraction module 101 extracts the image features of the PAN image through the convolution module to obtain the PAN image features; extracts the image features of the upsampled LRHS image through the convolution module to obtain the upsampled LRHS image features. The second feature extraction module 102 extracts features from the PAN image features through the spatial curvature dynamic convolution to obtain spatial convolution features, and extracts features from the upsampled LRHS image features through the spectral curvature dynamic convolution to obtain spectral convolution features. The non-uniform integration module 103 performs feature fusion on the upsampled LRHS image features, PAN image features, spatial convolution features, and spectral convolution features through the non-uniform feature integration unit to obtain non-uniform fusion features. The current-scale fusion module 104 performs feature fusion on the non-uniform fusion features and the spectral convolution features to obtain the current-scale fusion features. The multi-scale fusion module 105 performs multi-scale feature extraction and fusion on the current-scale fusion feature and the spatial convolution feature through the non-uniform fusion module and the skip connection to obtain the multi-scale fusion feature; The residual module 106 performs a residual operation on the multi-scale fusion feature to obtain the hyperspectral image.

[0058] Through the collaborative work of the above-mentioned modules, the spatial curvature dynamic convolution and the spectral curvature dynamic convolution achieve the feature extraction and enhancement of the hyperspectral image and the panchromatic image, and the non-uniform feature integration unit realizes the fusion of the features of the two images at each scale. The present invention aims to provide an efficient and physically interpretable fusion method, which can significantly improve the spatial resolution of the hyperspectral image while maintaining its spectral consistency, and provide technical support for the refined analysis and practical application of the hyperspectral image.

[0059] The present invention conducts quantitative index result comparison experiments and subjective comparison experiments with a variety of internationally advanced algorithms on the public dataset Pavia center. To better observe the differences between the fusion results, the root mean square error (RMSE) between the fusion results and the ground truth (GT) is calculated and shown. The internationally advanced algorithms are Refiner(2023), TreeNet(2024), DFCFN(2025), and FPFNet(2025). Refiner(2023) is the refined hyperspectral panspectral network version 2023, TreeNet(2024) is the tree-structured neural network version 2024, DFCFN(2025) is the two-stage feature correction fusion network version 2025, and FPFNet(2025) is the feature pyramid fusion network version 2025. As shown in Table 1, it can be seen from the quantitative index results that the quantitative result indexes of the algorithm of the present invention are better than those of the internationally advanced algorithms in recent years. The experimental results are shown in Table 1 below.

[0060] Table 1 Comparison experiment of quantitative index results on the Pavia center dataset Among them, CC is the correlation coefficient, SAM is the spectral angle mapping, RMSE is the root mean square error, ERGAS is the dimensionless global relative error, PSNR is the peak signal-to-noise ratio, "↑" indicates that the higher the value, the better, and "↓" indicates that the lower the value, the better.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A hyperspectral image fusion method based on non-uniform features and dynamic convolution, characterized in that Including: S1: Extract the image features of the PAN image through a convolution module to obtain PAN image features; extract the image features of the upsampled LRHS image through a convolution module to obtain upsampled LRHS image features. S2: Extract features from the PAN image features through spatial curvature dynamic convolution to obtain spatial convolution features, and extract features from the upsampled LRHS image features through spectral curvature dynamic convolution to obtain spectral convolution features. S3: Feature fusion of the upsampled LRHS image features, PAN image features, spatial convolution features, and spectral convolution features is performed through a non-uniform feature integration unit to obtain non-uniform fusion features. S4: Feature fusion of the non-uniform fusion features and spectral convolution features is performed to obtain the current scale fusion features. S5: Multi-scale feature extraction and fusion of the current scale fusion features and spatial convolution features are performed through a non-uniform fusion module and skip connection to obtain multi-scale fusion features. S6: Residual operation is performed on the multi-scale fusion features to obtain the hyperspectral image.

2. A hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 1, characterized in that, In step S2, obtaining the spatial convolution features includes: S211: Calculate the curvature value of the PAN image features through a spatial curvature function to obtain a spatial curvature map. S212: Use the local patches of the spatial curvature map as dynamic weights to adjust the spatial curvature convolution kernel to obtain a dynamic spatial curvature convolution kernel. S213: Perform a convolution operation on the PAN image features through the dynamic spatial curvature convolution kernel to obtain spatial convolution features.

3. A hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 1, characterized in that, In step S2, obtaining the spectral convolution features includes: S221: Calculate the curvature value of the upsampled LRHS image features in the channel dimension through a spectral curvature function to obtain a spectral curvature map. S222: Use the local patches of the spectral curvature map as dynamic weights to adjust the spectral curvature convolution kernel to obtain a dynamic spectral curvature convolution kernel. S223: Perform a convolution operation on the upsampled LRHS image features through the dynamic spectral curvature convolution kernel to obtain spectral convolution features.

4. A hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 1, characterized in that, The non-uniform feature integration unit includes a feature selection and fusion module and a detail fusion module. The feature selection and fusion module selects and fuses the non-uniform features of the PAN image and the upsampled LRHS image through a fuzzy logic fusion rule to obtain the fusion features of the feature selection and fusion module. The detail fusion module obtains a detail weight map by fusing the spatial curvature map and the spectral curvature map, and weights the fusion features of the feature selection and fusion module through the detail weight map.

5. A hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 4, characterized in that The feature selection and fusion module includes: Calculate the structural similarity between local blocks in the PAN image and the upsampled LRHS image to obtain the first similarity feature. Calculate the similarity between local blocks in the Gaussian blurred PAN image and the upsampled LRHS image to obtain the second similarity feature. Define the first fuzzy logic soft threshold and the second fuzzy logic soft threshold according to the first similarity feature and the second similarity feature. Formulate a fuzzy logic fusion rule according to the first fuzzy logic soft threshold and the second fuzzy logic soft threshold. Obtain the non-uniform features of the PAN image and the upsampled LRHS image according to the fuzzy logic fusion rule.

6. A hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 5, characterized in that The fuzzy logic fusion rule is: Set the critical fuzzy logic soft threshold. When the first fuzzy logic soft threshold is greater than or equal to the critical fuzzy logic soft threshold, the upsampled LRHS image features are selected; When the first fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold and the second fuzzy logic soft threshold is greater than or equal to the critical fuzzy logic soft threshold, the PAN image features are selected; When the first fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold and the second fuzzy logic soft threshold is less than the critical fuzzy logic soft threshold, the upsampled LRHS image features are selected.

7. A hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 4, characterized in that The calculation expression of the detail fusion module is: Among them, is the scale detail weight map, is the scale spectral curvature local entropy, is the scale space curvature local entropy, is the spectral curvature map, is the space curvature map, is the per-pixel multiplication.

8. A hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 1, characterized in that, The non-uniform fusion module includes a spatial curvature dynamic convolution, a spectral curvature dynamic convolution, and a non-uniform feature integration unit.

9. A hyperspectral image fusion method based on non-uniform features and dynamic convolution according to claim 1, wherein, Downsample the current scale fusion features and use the downsampled current scale fusion features as the input features of the next scale layer.

10. A hyperspectral image fusion system based on non-uniform features and dynamic convolution, characterized in that, A hyperspectral image fusion method based on non-uniform features and dynamic convolution as claimed in any one of claims 1 to 9, comprising: A first feature extraction module, which extracts the image features of the PAN image through a convolution module to obtain PAN image features; and extracts the image features of the upsampled LRHS image through a convolution module to obtain upsampled LRHS image features; A second feature extraction module, which extracts features from the PAN image features through a spatial curvature dynamic convolution to obtain spatial convolution features, and extracts features from the upsampled LRHS image features through a spectral curvature dynamic convolution to obtain spectral convolution features; A non-uniform integration module, which fuses the upsampled LRHS image features, PAN image features, spatial convolution features, and spectral convolution features through a non-uniform feature integration unit to obtain non-uniform fusion features; A current scale fusion module, which fuses the non-uniform fusion features and spectral convolution features to obtain current scale fusion features; A multi-scale fusion module, which performs multi-scale feature extraction and fusion on the current scale fusion features and spatial convolution features through a non-uniform fusion module and a skip connection to obtain multi-scale fusion features; A residual module, which performs a residual operation on the multi-scale fusion features to obtain a hyperspectral image.

Images