Hyperspectral image super-resolution method based on unmixing

Abstract

The application discloses a hyperspectral image super-resolution method based on unmixing, belongs to computer vision and image processing technology, and is used for hyperspectral image resolution. The method comprises the following steps: constructing a hyperspectral image super-resolution framework, performing neural network training, outputting a hyperspectral image super-resolution framework processing result based on a composite loss function or returning neural network training, and the like. The hyperspectral image super-resolution framework comprises a material component guide model and a super-resolution model. Low-resolution hyperspectral images are respectively input into the material component guide model and the super-resolution model. An enhanced hyperspectral image is output after the achievements of the material component guide model and the super-resolution model are fused. Through the double-model collaborative framework, the spatial resolution accuracy of the hyperspectral image and the physical reliability of the material component analysis are simultaneously improved, and the joint optimization of spatial enhancement and spectral decoupling is realized.

Description

Technical Field

[0001] This invention discloses a super-resolution method for hyperspectral images based on unmixing, belonging to the field of computer vision and image processing technology. Background Technology

[0002] Existing hyperspectral super-resolution methods often prioritize enhancing spatial resolution based on human visual preferences, neglecting the accurate recovery of material composition information. Current methods primarily focus on enhancing spatial resolution by utilizing spatial spectral information, largely ignoring the crucial role of material composition. From a material composition perspective, compositional information provides essential guidance for accurate spatial reconstruction. However, the lack of material composition data in current research results in high-resolution images failing to accurately represent the sub-pixel mixing of multiple materials, leading to lower reconstruction accuracy and an inability to faithfully reflect the true distribution of surface materials.

[0003] Most existing hyperspectral super-resolution methods follow a visual enhancement paradigm designed for natural images, with optimization goals typically limited to pixel-level fidelity or perceptual quality metrics (such as PSNR and SSIM). While these methods can improve the visual sharpness of images, they fail to embed the unique physical generation mechanism of hyperspectral data—that is, pixel spectra are the result of a linear mixture of a finite number of end-member substances according to their spatial abundance. Due to the lack of explicit modeling of the decomposition of mixed pixels and the distribution patterns of substances, existing methods tend to blur or distort the spectral features and boundaries between different land features when improving resolution. This results in reconstructed images that "look clear" but introduce errors in subsequent quantitative inversions (such as land feature classification, composition estimation, and target detection). Therefore, how to maintain or even enhance the feasibility of spectral unmixing while improving spatial detail has become a key bottleneck for the practical application of hyperspectral super-resolution. Summary of the Invention

[0004] The purpose of this invention is to provide a hyperspectral image super-resolution method based on unmixing, in order to solve the problems in the prior art, which are visually clear but physically distorted due to neglecting the physical generation mechanism, and which improve the spatial details of the reconstructed image but destroy the true proportion of material components within the sub-pixel, thus restricting its application in quantitative remote sensing.

[0005] Hyperspectral image super-resolution methods based on unmixing include:

[0006] Prepare a dataset, construct a hyperspectral image super-resolution framework based on the unmixing prior of material composition and the parallel connection of the super-resolution model, train the neural network, introduce a composite loss function to optimize the network parameters, and verify the composite loss function after training. If the composite loss function is less than the preset threshold, output the processing result of the hyperspectral image super-resolution framework; if the composite loss function is greater than or equal to the preset threshold, return to neural network training.

[0007] The hyperspectral image super-resolution framework takes a low-resolution hyperspectral image as input and outputs a super-resolution hyperspectral image. The hyperspectral image super-resolution framework includes a material composition-guided model and a super-resolution model. The low-resolution hyperspectral image is input into the material composition-guided model and the super-resolution model, respectively. After the super-resolution model is processed, it splits into two branches. The first branch inputs the processing result of the super-resolution model into the material composition-guided model. The second branch adds the processing result of the super-resolution model and the processing result of the material composition-guided model element by element to output the super-resolution hyperspectral image.

[0008] The super-resolution model consists of a super-resolution network, which includes a spatial information reconstruction module, an edge refinement module, and a spectral information reconstruction module.

[0009] Low-resolution hyperspectral images Input the super-resolution network, then split it into two branches. The first branch... Upsampling, the second branch will The input is processed by the spatial information reconstruction module, and the result is then input into the edge refinement module, and finally into the spectral information reconstruction module. The upsampling result and the result from the spectral information reconstruction module are added element-wise to output the super-resolution hyperspectral features. :

[0010] ;

[0011] In the formula, For low-resolution hyperspectral images, , Image height, Image width, For image bands, It is a super-resolution network.

[0012] Upsampling is performed using subpixel convolutional layers;

[0013] The spatial information reconstruction module first utilizes right Preliminary feature encoding is performed, and the result is input into a ConvLSTM skip connection layer, then upsampled, and finally passed through... Fuse features to output a spatial feature cube;

[0014] The edge refinement module takes a spatial feature cube as input and extracts multi-scale edge information using both the Laplacian and Canny operators. The edge information is obtained by element-wise summing the results of the Laplacian and Canny operators. The fusion is performed, and the fusion result is input into the edge attention layer and the ConvLSTM skip connection layer respectively. The processing results of the edge attention layer and the ConvLSTM skip connection layer are multiplied element-wise. The multiplication result is added element-wise to the spatial feature cube. The addition result is subtracted element-wise from the edge information to obtain the edge features.

[0015] The spectral information reconstruction module takes edge features as input, first learning spatiotemporal features through a first ConvLSTM skip connection layer, and then... The activation function introduces nonlinearity, and then the second ConvLSTM skip connection layer is used for deep feature modeling. The modeling result is input into the spectral attention layer, and the result of the spectral attention layer is multiplied element-wise with the edge features to obtain the spectral features.

[0016] The ConvLSTM skip connection layer is a residual unit that receives input features and splits them into two branches. The first branch passes through the ConLSTM layer and outputs the updated features. The second branch then passes through... and The activation function sums the results of the two branches element by element and outputs the feature fusion result.

[0017] The material composition-guided model includes unmixing networks, correction networks, and reconstruction networks;

[0018] Will and Two demixing networks are input separately, the first demixing network is... As input, output a low-resolution abundance map. and low-resolution reconstruction features, for Upsampling is performed, and the upsampling result is input into the correction network; the second demixing network uses... As input, output the abundance map of the super-resolution image. and super-resolution reconstructed features, then Input the super-resolution reconstructed features into the correction network; the correction network outputs the corrected abundance features, which are then input into the reconstruction network; the reconstruction network outputs the reconstructed features.

[0019] The unmixing network is an abundance-guided spectral-spatial network, which includes a spectral module, a spatial module, and an abundance-guided attention unmixing module. Hyperspectral image data is input into the spectral module and the spatial module, respectively. The results of the spectral module and the spatial module are input into the abundance-guided attention unmixing module, which outputs abundance features and reconstructed features.

[0020] The spectral module takes hyperspectral images as input and processes them sequentially. Normalization layer Activation function and The process is iterated four times, with each iteration's result added element-wise to the previous iteration's result, ultimately outputting the spectral characteristics. ;

[0021] The space module takes hyperspectral images as input and passes through sequentially... Normalization layer Activation function and The process is iterated four times, with each iteration's result added element-wise to the previous iteration's result, ultimately outputting the spatial features. .

[0022] Abundance-guided attention unmixing module will and Fusion, resulting in fusion characteristics :

[0023] ;

[0024] In the formula, for Activation function for Convolution operation;

[0025] use Constructing the key matrix in the attention mechanism Sum matrix and using coarse abundance maps Constructing the query matrix in the attention mechanism ,set up for and Dimensional value, yes Dimensional values:

[0026] ;

[0027] ;

[0028] ;

[0029] ;

[0030] Will , from Remodeling into sequence form ,Will from Remodeling into sequence form , For the number of pixels, ;

[0031] Weighted features for:

[0032] ;

[0033] In the formula, For normalized exponential functions, It is the transpose symbol;

[0034] Final abundance map for:

[0035] ;

[0036] If the input to the demixing network is , for If the input to the demixing network is , for .

[0037] The calibration network is used to construct a linear weighted fusion model:

[0038] ;

[0039] In the formula, For upsampling, To add element by element, The abundance information has been corrected.

[0040] Will Transpose the matrix and perform matrix multiplication with the corresponding hyperspectral features to obtain the endmember feature matrix. .

[0041] Rebuild network utilization Reconstructing hyperspectral image features, including estimating abundance distribution maps, extracting endmember features, and image reconstruction;

[0042] Estimated abundance distribution maps include those based on and Construct a similarity matrix :

[0043] ;

[0044] In the formula, It is the transpose symbol. Follows the sum-of-ones constraint and the non-negativity constraint;

[0045] Endmember feature extraction and image reconstruction include the reconstruction network output expression:

[0046] ;

[0047] In the formula, To reconstruct hyperspectral image features;

[0048] Employing a multi-source information fusion strategy, and Perform feature-level fusion:

[0049] ;

[0050] In the formula, The result is the super-resolution image after fusion. and is a learnable hyperparameter matrix.

[0051] Establish super-resolution loss functions, including L1 loss, spatial loss, and spectral loss;

[0052] L1 loss for:

[0053] ;

[0054] In the formula, The original hyperspectral image, For the first The original hyperspectral images of each sample, For the first Super-resolution results after fusion of individual samples. To calculate the absolute difference between two images on a pixel-by-pixel basis, The number of training samples. For training sample index, ;

[0055] Space loss for:

[0056] ;

[0057] ;

[0058] ;

[0059] In the formula, , To replace the variable, For image band indexing, , For pixel row index, , Indexed by pixel column, , for In the The image band, the first line, number The pixel values ​​of the column;

[0060] Spectral loss for:

[0061] ;

[0062] Super-resolution loss for:

[0063] ;

[0064] Establish unmixing loss, including reconstruction loss and abundance loss, reconstruction loss for:

[0065] ;

[0066] In the formula, For pixel index, , For the first The true endmember hyperspectral features of each pixel For the first Reconstructed hyperspectral features of each pixel It is an L2 norm.

[0067] Abundance loss includes summation of one-constraint loss and non-negativity constraint loss, and summation of one-constraint loss. for:

[0068] ;

[0069] In the formula, The number of endmembers, For endmember index, , for No. The first pixel Each terminal element;

[0070] Change the abundance matrix to:

[0071] ;

[0072] Non-negative constraint loss for:

[0073] ;

[0074] Abundance loss is:

[0075] ;

[0076] Unmixing loss for:

[0077] ;

[0078] Composite loss function for:

[0079] ;

[0080] In the formula, for The weight hyperparameter.

[0081] Compared with existing technologies, this invention has the following advantages: by embedding the material composition unmixing prior into the network, the output simultaneously satisfies the constraints of visual quality and linear mixture model, ensuring that the reconstruction results strictly maintain the physical interpretability of pixel spectra while improving spatial details; the reconstructed image has a more accurate representation of material composition, significantly improving the accuracy and reliability of subsequent quantitative remote sensing tasks such as land cover classification, target detection, and composition inversion; through a parallel dual-branch architecture and a composite loss function, the synergistic optimization of spatial spectral features and material composition priors is achieved, providing a new paradigm for hyperspectral image enhancement that balances perceptual quality and physical consistency. Attached Figure Description

[0082] Figure 1 This is a super-resolution framework diagram of hyperspectral images in this invention;

[0083] Figure 2 This is a diagram of the material-guided model architecture;

[0084] Figure 3 This is a diagram of a super-resolution network architecture;

[0085] Figure 4 This is a diagram of the demixed network architecture;

[0086] Figure 5 This is a diagram of the spectral module architecture;

[0087] Figure 6 This is a diagram of the spatial module architecture;

[0088] Figure 7 It is a spatial information reconstruction module diagram;

[0089] Figure 8 This is the architecture diagram of the edge refinement module;

[0090] Figure 9 This is a diagram of the spectral information reconstruction module architecture;

[0091] Figure 10 This is a diagram of the ConvLSTM skip connection layer architecture;

[0092] Figure 11 This is the spectral curve of the sampling points in the first dataset;

[0093] Figure 12 This is the spectral curve of the sampling points in the second dataset. Detailed Implementation

[0094] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention are described clearly and completely below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0095] Hyperspectral image super-resolution methods based on unmixing include:

[0096] Prepare a dataset, construct a hyperspectral image super-resolution framework based on the unmixing prior of material composition and the parallel connection of the super-resolution model, train the neural network, introduce a composite loss function to optimize the network parameters, and verify the composite loss function after training. If the composite loss function is less than the preset threshold, output the processing result of the hyperspectral image super-resolution framework; if the composite loss function is greater than or equal to the preset threshold, return to neural network training.

[0097] The hyperspectral image super-resolution framework takes a low-resolution hyperspectral image as input and outputs a super-resolution hyperspectral image. The hyperspectral image super-resolution framework includes a material composition-guided model and a super-resolution model. The low-resolution hyperspectral image is input into the material composition-guided model and the super-resolution model, respectively. After the super-resolution model is processed, it splits into two branches. The first branch inputs the processing result of the super-resolution model into the material composition-guided model. The second branch adds the processing result of the super-resolution model and the processing result of the material composition-guided model element by element to output the super-resolution hyperspectral image.

[0098] The super-resolution model consists of a super-resolution network, which includes a spatial information reconstruction module, an edge refinement module, and a spectral information reconstruction module.

[0099] Low-resolution hyperspectral images Input the super-resolution network, then split it into two branches. The first branch... Upsampling, the second branch will The input is processed by the spatial information reconstruction module, and the result is then input into the edge refinement module, and finally into the spectral information reconstruction module. The upsampling result and the result from the spectral information reconstruction module are added element-wise to output the super-resolution hyperspectral features. :

[0100] ;

[0101] In the formula, For low-resolution hyperspectral images, , Image height, Image width, For image bands, It is a super-resolution network.

[0102] Upsampling is performed using subpixel convolutional layers;

[0103] The spatial information reconstruction module first utilizes right Preliminary feature encoding is performed, and the result is input into a ConvLSTM skip connection layer, then upsampled, and finally passed through... Fuse features to output a spatial feature cube;

[0104] The edge refinement module takes a spatial feature cube as input and extracts multi-scale edge information using both the Laplacian and Canny operators. The edge information is obtained by element-wise summing the results of the Laplacian and Canny operators. The fusion is performed, and the fusion result is input into the edge attention layer and the ConvLSTM skip connection layer respectively. The processing results of the edge attention layer and the ConvLSTM skip connection layer are multiplied element-wise. The multiplication result is added element-wise to the spatial feature cube. The addition result is subtracted element-wise from the edge information to obtain the edge features.

[0105] The spectral information reconstruction module takes edge features as input, first learning spatiotemporal features through a first ConvLSTM skip connection layer, and then... The activation function introduces nonlinearity, and then the second ConvLSTM skip connection layer is used for deep feature modeling. The modeling result is input into the spectral attention layer, and the result of the spectral attention layer is multiplied element-wise with the edge features to obtain the spectral features.

[0106] The ConvLSTM skip connection layer is a residual unit that receives input features and splits them into two branches. The first branch passes through the ConLSTM layer and outputs the updated features. The second branch then passes through... and The activation function sums the results of the two branches element by element and outputs the feature fusion result.

[0107] The material composition-guided model includes unmixing networks, correction networks, and reconstruction networks;

[0108] Will and Two demixing networks are input separately, the first demixing network is... As input, output a low-resolution abundance map. and low-resolution reconstruction features, for Upsampling is performed, and the upsampling result is input into the correction network; the second demixing network uses... As input, output the abundance map of the super-resolution image. and super-resolution reconstructed features, then Input the super-resolution reconstructed features into the correction network; the correction network outputs the corrected abundance features, which are then input into the reconstruction network; the reconstruction network outputs the reconstructed features.

[0109] The unmixing network is an abundance-guided spectral-spatial network, which includes a spectral module, a spatial module, and an abundance-guided attention unmixing module. Hyperspectral image data is input into the spectral module and the spatial module, respectively. The results of the spectral module and the spatial module are input into the abundance-guided attention unmixing module, which outputs abundance features and reconstructed features.

[0110] The spectral module takes hyperspectral images as input and processes them sequentially. Normalization layer Activation function and The process is iterated four times, with each iteration's result added element-wise to the previous iteration's result, ultimately outputting the spectral characteristics. ;

[0111] The space module takes hyperspectral images as input and passes through sequentially... Normalization layer Activation function and The process is iterated four times, with each iteration's result added element-wise to the previous iteration's result, ultimately outputting the spatial features. .

[0112] Abundance-guided attention unmixing module will and Fusion, resulting in fusion characteristics :

[0113] ;

[0114] In the formula, for Activation function for Convolution operation;

[0115] use Constructing the key matrix in the attention mechanism Sum matrix and using coarse abundance maps Constructing the query matrix in the attention mechanism ,set up for and Dimensional value, yes Dimensional values:

[0116] ;

[0117] ;

[0118] ;

[0119] ;

[0120] Will , from Remodeling into sequence form ,Will from Remodeling into sequence form , For the number of pixels, ;

[0121] Weighted features for:

[0122] ;

[0123] In the formula, For normalized exponential functions, It is the transpose symbol;

[0124] Final abundance map for:

[0125] ;

[0126] If the input to the demixing network is , for If the input to the demixing network is , for .

[0127] The calibration network is used to construct a linear weighted fusion model:

[0128] ;

[0129] In the formula, For upsampling, To add element by element, The abundance information has been corrected.

[0130] Will Transpose the matrix and perform matrix multiplication with the corresponding hyperspectral features to obtain the endmember feature matrix. .

[0131] Rebuild network utilization Reconstructing hyperspectral image features, including estimating abundance distribution maps, extracting endmember features, and image reconstruction;

[0132] Estimated abundance distribution maps include those based on and Construct a similarity matrix :

[0133] ;

[0134] In the formula, It is the transpose symbol. Follows the sum-of-ones constraint and the non-negativity constraint;

[0135] Endmember feature extraction and image reconstruction include the reconstruction network output expression:

[0136] ;

[0137] In the formula, To reconstruct hyperspectral image features;

[0138] Employing a multi-source information fusion strategy, and Perform feature-level fusion:

[0139] ;

[0140] In the formula, The result is the super-resolution image after fusion. and is a learnable hyperparameter matrix.

[0141] Establish super-resolution loss functions, including L1 loss, spatial loss, and spectral loss;

[0142] L1 loss for:

[0143] ;

[0144] In the formula, The original hyperspectral image, For the first The original hyperspectral images of each sample, For the first Super-resolution results after fusion of individual samples. To calculate the absolute difference between two images on a pixel-by-pixel basis, The number of training samples. For training sample index, ;

[0145] Space loss for:

[0146] ;

[0147] ;

[0148] ;

[0149] In the formula, , To replace the variable, For image band indexing, , For pixel row index, , Indexed by pixel column, , for In the The image band, the first line, number The pixel values ​​of the column;

[0150] Spectral loss for:

[0151] ;

[0152] Super-resolution loss for:

[0153] ;

[0154] Establish unmixing loss, including reconstruction loss and abundance loss, reconstruction loss for:

[0155] ;

[0156] In the formula, For pixel index, , For the first The true endmember hyperspectral features of each pixel For the first Reconstructed hyperspectral features of each pixel It is an L2 norm.

[0157] Abundance loss includes summation of one-constraint loss and non-negativity constraint loss, and summation of one-constraint loss. for:

[0158] ;

[0159] In the formula, The number of endmembers, For endmember index, , for No. The first pixel Each terminal element;

[0160] Change the abundance matrix to:

[0161] ;

[0162] Non-negative constraint loss for:

[0163] ;

[0164] Abundance loss is:

[0165] ;

[0166] Unmixing loss for:

[0167] ;

[0168] Composite loss function for:

[0169] ;

[0170] In the formula, for The weight hyperparameter.

[0171] The following description, in conjunction with the accompanying figures, further illustrates this point. Hyperspectral unmixing is a core technology for providing material composition information. This technology unmixes the observed spectrum into two key components: endmember reflectance spectra (representing the inherent spectral characteristics of a pure material) and abundance maps (describing the sub-pixel material composition of each material). Embedding the unmixing mechanism into the super-resolution framework provides the necessary prior knowledge—the spatial consistency of the abundance distribution—while enhancing spatial resolution and spectral fidelity. However, a well-designed framework alone cannot guarantee strong performance in practical applications; its specific implementation is equally important. Hyperspectral image super-resolution involves multi-level information processing, and the effectiveness of each component directly affects the final reconstruction quality. Therefore, this invention systematically describes the implementation of four key components to ensure the feasibility and efficiency of the proposed framework in real-world scenarios. The material composition-guided model consists of three networks: an unmixing network, a correction network, and a reconstruction network. The super-resolution model consists of a super-resolution network with a ConvLSTM skip connection structure. The super-resolution network improves spatial resolution. Specifically, two unmixing networks with identical structures are applied to the input image and the super-resolution features, respectively, to extract their corresponding abundance information. The correction network uses abundance information from the input image to correct the abundance information of the super-resolution features. Next, the reconstruction network processes the corrected abundance features to generate reconstructed features containing accurate material composition. For example... Figure 1 As shown, the hyperspectral image super-resolution framework takes a low-resolution hyperspectral image as input and outputs a super-resolution hyperspectral image. The hyperspectral image super-resolution framework includes a material composition-guided model and a super-resolution model. The low-resolution hyperspectral image is input into the material composition-guided model and the super-resolution model, respectively. The super-resolution model outputs super-resolution hyperspectral features, which are then divided into two branches. The first branch inputs the super-resolution hyperspectral features into the material composition-guided model, and the second branch adds the super-resolution hyperspectral features to the reconstructed features output by the material composition-guided model element by element to output the super-resolution hyperspectral image.

[0172] The material-guided model architecture and processing of this invention are as follows: Figure 2 As shown, the low-resolution hyperspectral image and the super-resolution hyperspectral features are input into two unmixing networks, respectively. The unmixing network outputs the low-resolution abundance features and low-resolution reconstruction features of the low-resolution hyperspectral image. The low-resolution abundance features are upsampled, and the upsampling result is input into the calibration network. The unmixing network outputs the super-resolution abundance features and super-resolution reconstruction features of the super-resolution hyperspectral image. The super-resolution abundance features are input into the calibration network, and the super-resolution reconstruction features are input into the reconstruction network. The calibration network outputs the corrected abundance features, which are input into the reconstruction network. The reconstruction network outputs the reconstructed features.

[0173] The implementation of super-resolution networks directly impacts the effectiveness of the framework in enhancing baseline spatial quality. In hyperspectral image super-resolution, features from different receptive fields and layers represent different reconstruction patterns. Indiscriminate fusion of these features can lead to spatial inconsistencies and redundant information, ultimately degrading reconstruction quality. To address this issue, the ConvLSTM skip connection structure optimizes the feature fusion strategy, effectively enhancing spatial consistency while reducing redundancy, thereby improving super-resolution reconstruction performance. The super-resolution network architecture and processing procedure of this invention are as follows: Figure 3 As shown, a low-resolution hyperspectral image is input into a super-resolution network, which then splits into two branches. The first branch upsamples the low-resolution hyperspectral image, while the second branch first inputs the low-resolution hyperspectral image into a spatial information reconstruction module, then into an edge refinement module, and finally into a spectral information reconstruction module. The upsampling result and the result from the spectral information reconstruction module are then added element-wise to output the super-resolution hyperspectral features. The hyperspectral image super-resolution network with a ConvLSTM skip connection structure effectively fuses spatial information from different receiving fields by optimizing a multi-layer feature fusion strategy, suppressing redundant features and improving spatial consistency, thereby significantly improving the super-resolution reconstruction effect.

[0174] The demixing network architecture and processing procedure of this invention are as follows: Figure 4 As shown, hyperspectral image data is input into the spectral module and the spatial module, respectively. The results from the spectral and spatial modules are then input into the abundance-guided attention unmixing module, which outputs abundance features and reconstructed features. The abundance-guided spectral-spatial network consists of three modules: the spectral module, the spatial module, and the abundance-guided attention unmixing module. Specifically, the spectral module extracts pixel-by-pixel spectral features from the input hyperspectral image. This pixel-by-pixel processing preserves the original spatial structure while capturing spectral features. Simultaneously, the spatial module focuses on learning the local spatial context. The obtained spectral features, along with spatial cues, are then passed to the abundance-guided attention unmixing module, which integrates a self-attention mechanism and an abundance-guided strategy. It uses an abundance matrix to reweight the features, enhancing their correlation with material proportions. To reduce computational complexity, the matrix multiplication order is optimized. The final output is a highly accurate abundance map. It is worth emphasizing that this module has a dual role: acting as an auxiliary feature extractor while providing an integrated method for endmember recognition and abundance prediction, which significantly improves unmixing efficiency. Abundance-guided spectral spatial networks achieve more accurate and robust spectral unmixing by directly extracting endmember features from the network, bypassing the dependence on external endmember initialization.

[0175] Since the material composition information in super-resolution features is generated by a super-resolution network, it inevitably deviates to some extent from the actual distribution of surface materials. To accurately recover the material composition, it is necessary to correct these deviations in the super-resolution features. Therefore, the correction network of this invention uses abundance information extracted from the input image as prior knowledge to guide the correction of material differences. The correction network of this invention is based on the fact that the material information in the input image comes from real observations, providing a reliable representation of the actual proportions and a reliable reference for abundance correction in super-resolution features. The correction network of this invention is a correction network based on super-resolution abundance information fusion, aiming to optimize the accuracy of the material composition representation of the reconstructed image through prior knowledge from real observation data. Specifically, the abundance information of the input image originates from measured spectral data and possesses strict physical authenticity. However, the abundance information of the super-resolution reconstructed image is limited by model reconstruction errors and noise interference, resulting in spectral distortion or spatial distribution deviations. A linear weighted fusion model is constructed to balance the reliability of the observation data with the spatial detail of the reconstruction results. The correction network of this invention is used to suppress image detail distortion caused by model reconstruction errors and noise. This network effectively improves the accuracy of land cover composition information recovery and the ability to represent spatial details in abundance maps by using physically reliable material composition representations.

[0176] The reconstruction network of this invention is a hyperspectral image reconstruction network specifically designed for demixing. This network utilizes corrected abundance information to reconstruct hyperspectral images with higher spectral purity and stronger physical interpretability. This method effectively mitigates spectral mixing effects, providing strong support for improving the accuracy and reliability of material proportion identification. The reconstruction network integrates attention in a principled manner, thereby enhancing model interpretability, and introduces physical constraints, significantly improving the robustness and reliability of the reconstruction results.

[0177] The spectral module architecture of this invention is as follows: Figure 5 As shown, the spectral module takes a hyperspectral image as input and processes it sequentially. Batch normalization layer, ReLU activation function and The process is iterated four times, with each iteration's result added element-wise to the previous iteration's result, ultimately outputting the spectral characteristics. The spatial module architecture of this invention is as follows: Figure 6 As shown, the space module takes hyperspectral images as input and passes through sequentially... Batch normalization layer, ReLU activation function and The process is iterated four times, with each iteration's result added element-wise to the previous iteration's result, ultimately outputting the spatial features. .

[0178] The spatial information reconstruction module architecture of this invention is as follows: Figure 7 As shown, the spatial information reconstruction module first utilizes right Preliminary feature encoding is performed, and the input is fed into a ConvLSTM skip connection layer, followed by upsampling, and finally... The fusion features output a spatial feature cube; the edge refinement module architecture of this invention is as follows: Figure 8 As shown, the edge refinement module takes a spatial feature cube as input and extracts multi-scale edge information using the Laplacian and Canny operators respectively. The results of the Laplacian and Canny operators are added element-wise to obtain the edge information, which is then input into... The components are fused and then input into the edge attention layer and the ConvLSTM skip connection layer respectively. The outputs of the edge attention layer and the ConvLSTM skip connection layer are multiplied element-wise. The multiplication result is added element-wise to the spatial feature cube. The addition result is subtracted element-wise from the edge information to obtain the edge features. The architecture of the spectral information reconstruction module of this invention is as follows: Figure 9 As shown, the spectral information reconstruction module takes edge features as input. First, it learns spatiotemporal features through a first ConvLSTM skip connection layer. Then, it introduces nonlinearity through a ReLU activation function, followed by deep feature modeling through a second ConvLSTM skip connection layer. This result is then input to the spectral attention layer, where the result is multiplied element-wise with the edge features to obtain the spectral features. The ConvLSTM skip connection layer architecture of this invention is as follows: Figure 10 As shown, the ConvLSTM skip connection layer is a residual unit that receives input features and splits them into two branches. The first branch passes through the ConLSTM layer and outputs the updated features, while the second branch passes through... The ReLU activation function is used to sum the outputs of the two branches element by element, resulting in the feature fusion output.

[0179] This invention uses multi-source hyperspectral data to verify the effectiveness of the method. The selected datasets cover typical urban and natural geographical scenarios. The first group is an urban area dataset, and the second group is a composite landform dataset. The threshold for the composite loss function is set to 0.0001.

[0180] The first dataset, recorded on February 16, 2017, primarily contains spatial information about the university campus and surrounding urban area. This data has a spatial resolution of 1 meter, a spectral range covering 380 to 1050 nanometers, and includes 48 continuous spectral bands. The image size is 4172×1202 pixels, fully presenting the distribution characteristics of urban buildings, roads, and vegetation.

[0181] The second dataset, collected on April 10, 2019, covers landforms including plains, mountains, and marine areas. The satellite sensor acquired 32 effective bands, covering a spectral range of 400 to 1000 nanometers. The image spatial resolution is 10 meters, and each scene data point is 5056×5056 pixels, comprehensively recording the spectral characteristics of different land cover types.

[0182] A multi-faceted evaluation framework with eight standardized metrics is employed to assess spatial reconstruction accuracy, spectral fidelity, and perceptual quality. The Fraser Initiation Distance (FID) measures the distributional difference between the generated and reference images using covariance statistics; Peak Signal-to-Noise Ratio (PSNR) quantifies pixel-level accuracy using logarithmic-scale SNR; Structural Similarity Index (SSIM) evaluates structure preservation based on brightness, contrast, and spatial similarity; Spectral Angle Mapper (SAM) assesses angular differences between spectral vectors to reflect material consistency; Relative Global Size Error (ERGAS) measures global radiometric error across multiple spectral bands; Correlation Coefficient (CC) determines the spectral correlation coefficient between the reconstructed data cube and the reference data cube; Learned Perceptual Patch Similarity (LPIPS) uses a deep neural network to simulate human visual perception of texture details; and Deep Image Structure and Texture Similarity (DISTS) jointly analyzes structural and texture features to assess overall visual similarity. Higher PSNR, SSIM, and CC values ​​indicate superior performance, while lower FID, SAM, ERGAS, LPIPS, and DISTS scores indicate improved reconstruction quality.

[0183] Each dataset was divided into two subsets: 80% for training and 20% for evaluation. During training, a patch-based technique was used to extract overlapping regions from the hyperspectral images. For a 2x magnification factor, 64×64 pixel blocks were extracted with 16-pixel overlap; for a 4x magnification factor, the overlap was set to 32 pixels. To simulate image degradation, this invention applies a blurring model using a disk kernel to simulate the blurring effects common in remote sensing. Subsequently, bicubic interpolation was applied to create low-resolution images exhibiting different levels of blur.

[0184] This invention's method is implemented in PyTorch. During training, a spectral angle mapper is used, along with the Adam optimizer and the ReduceLROnPlateau learning rate scheduler. The initial learning rate is set to 0.0004, and if no improvement is observed for two consecutive epochs, it is reduced by a factor of 0.24. 48 images are randomly sampled per batch to adapt to the network's size.

[0185] The method of this invention is compared with seven existing techniques at 2x and 4x magnification factors, including bicubic interpolation, five convolutional neural network-based methods, and diffusion models, all trained on the same dataset with the original parameter settings. This standardized protocol ensures a fair comparison between spatial reconstruction accuracy, spectral fidelity, and computational efficiency.

[0186] Quantitative evaluation of the first dataset. Table 1 focuses on a comparative analysis of the quantitative results obtained by different methods on the first dataset:

[0187] Table 1. Quantization performance of different methods on the first dataset.

[0188] ;

[0189] Scale is the magnification factor. Amplification factor of 2 The magnification factor is 4x, metric is the index, Bicubic is bicubic interpolation, MCNet is a hybrid 2D / 3D convolutional network, SSPSR is a spatial spectral prior network, GRLIN is a group-based embedding learning and ensemble network, EDiffSR is an efficient diffusion probability network, SNLSR is a spectral prior super-resolution network, DBSR is a deep blind super-resolution network, and Our is the method of this invention. The method of this invention achieves significant advantages at both 2x and 4x magnification factors. Evaluation results show that the method of this invention is the best in six of the eight key metrics (bold), and ranks second in the other two (underlined). Regarding spectral accuracy, compared to SNLSR, the framework of this invention reduces the SAM value by 0.3741 (2x magnification factor) and 0.2548 (4x magnification factor), demonstrating enhanced spectral integrity through unmixing techniques. In visual quality assessment, compared to the second-best method DBSR, the FID score is reduced by 2.8749 (2x magnification factor) and 6.8721 (4x magnification factor), indicating superior perceptual quality. Although the SSIM scores were slightly lower by 0.0003 (2x magnification factor) and 0.0001 (4x magnification factor), the overall performance remained highly competitive. In radiometric evaluation, the framework of this invention improved the ERGAS metric by 0.2382 at a 2x magnification factor scaling and achieved a peak CC of 0.9974, indicating excellent consistency with the reference data. The perceptual metrics LPIPS and DISTS reached optimal values ​​of 0.0150 / 0.0993 (2x magnification factor) and 0.0596 / 0.1042 (4x magnification factor) respectively, highlighting superior visual quality. The framework of this invention effectively addresses the challenge of subpixel material recovery while preserving spectral consistency and spatial detail. By combining material scale information with spatial super-resolution, the proposed framework generates reconstructions with fine spatial detail and accurate spectral features.

[0190] Figure 11 The quantitative analysis of spectral fidelity using representative reflectance curves magnified four times from the first dataset is further demonstrated. The spectral curves clearly show that the method of the present invention more effectively preserves the key spectral features of selected pixels. Experimental results confirm the effectiveness of the proposed framework in enhancing spectral fidelity. By introducing unmixing techniques, the framework refines the material distribution and is able to accurately reconstruct the material components in the image.

[0191] Quantitative evaluation of the second dataset. Table 2 highlights the performance of various methods on the second dataset:

[0192] Table 2. Quantization performance of different methods on the second dataset.

[0193] ;

[0194] Of the eight evaluation metrics, the proposed framework performed best on five (bold) and second best on the remaining three (underline). In terms of visual quality, compared to the second-best method DBSR, it reduced FID by 2.6836 (2x magnification factor). SSIM improved slightly by 0.0019 (2x magnification factor) and significantly by 0.0400 (4x magnification factor). Regarding spectral fidelity, SAM reduced by 0.3702 (2x magnification factor) and 0.7829 (4x magnification factor), while ERGAS reduced by 0.0722 (2x magnification factor) and 0.1285 (4x magnification factor). The CC metric improved by 0.0047 (2x magnification factor) and 0.0288 (4x magnification factor). Perceptual quality assessments show that LPIPS reduced by 0.0004 (2x magnification factor) and 0.0002 (4x magnification factor), while DISTS reduced by 0.0034 (2x magnification factor) and 0.0151 (4x magnification factor). Experimental results demonstrate that the method of this invention achieves the optimal balance between spatial detail reconstruction and spectral feature preservation in complex geographical scenes. This success stems from the novel framework's ability to combine material distribution constraints from the unmixing network with spatial information from the super-resolution network, thereby achieving accurate sub-pixel-level material composition information recovery.

[0195] exist Figure 12 In the second dataset, the spectral curves of pixels highlighted in red were compared at a 4x magnification scale. The curves of the proposed framework represent the most accurate reality. These results validate the superior performance of the proposed framework in maintaining spectral fidelity and demonstrate the effectiveness of the material composition information derived from the unmixing network for reconstructing spectral features in hyperspectral images.

[0196] To evaluate the contribution of each component, ablation experiments were conducted on the second dataset with a 4x magnification factor. The experiments investigated different strategies for integrating material proportion information. "NUnmixing" refers to a novel framework without any super-resolution unmixing network assistance. "UnmixingLR" refers to a framework containing an unmixing network used only for processing low-resolution images, while "UnmixingSR" involves an unmixing network applied only to the initial super-resolution image. "Inventive" refers to the full version of the proposed novel framework. As shown in Table 3, the novel framework of this invention achieves the best overall performance. The model using NUnmixing performed the worst across all metrics, demonstrating that unmixing enhances super-resolution and highlighting the superior effectiveness of the framework proposed in this invention.

[0197] Table 3. Results of different methods for on-site assisted unmixing networks on the second set of datasets.

[0198] ;

[0199] This invention aims not only to improve spatial resolution but also to accurately recover the material composition of images. Unlike traditional super-resolution methods that primarily focus on visual quality, the proposed framework introduces a priori material composition information to directly estimate the proportions of various land cover materials. The proposed framework comprises two models: a material composition-guided model and a super-resolution model. It transforms the input low-resolution hyperspectral image into material composition information and incorporates this information into the super-resolution process, thereby enhancing the discriminability and interpretability of the image in terms of material composition information. The first model, the material composition-guided model, consists of an unmixing network, a correction network, and a reconstruction network. The second model, the super-resolution model, consists of a super-resolution network. Specifically, an unmixing network is developed to extract material composition information from both the reference and generated images. A correction network is developed to correct errors in the material composition information. A reconstruction network is developed to generate realistic images using the corrected material proportion information. A super-resolution network is developed to reconstruct spatially consistent structures. Finally, by integrating the outputs of the two sub-networks, the proposed framework achieves joint enhancement of spatial resolution and material representation. Qualitative and quantitative experiments demonstrate that the proposed framework significantly improves the quality of spatial detail and the accuracy of material proportion estimation. This invention provides high-quality images, offering clearer and more intuitive information for photogrammetry in remote sensing applications.

[0200] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hyperspectral image super-resolution method based on unmixing, characterized in that, The method comprises the following steps: The hyperspectral image super-resolution framework is trained, a composite loss function is introduced to optimize network parameters, and after the training is completed, the composite loss function is verified. The hyperspectral image super-resolution framework takes a low-resolution hyperspectral image as input and outputs a super-resolution hyperspectral image. The hyperspectral image super-resolution framework comprises a material component guided model and a super-resolution model, and the low-resolution hyperspectral image is input into the material component guided model and the super-resolution model. The super-resolution model is composed of a super-resolution network, and the super-resolution network comprises a spatial information reconstruction module, an edge refinement module and a spectral information reconstruction module. Low-resolution hyperspectral images Input the super-resolution network, then split it into two branches. The first branch... Upsampling, the second branch will The input is processed by the spatial information reconstruction module, and the result is then input into the edge refinement module, and finally into the spectral information reconstruction module. The upsampling result and the result from the spectral information reconstruction module are added element-wise to output the super-resolution hyperspectral features. : ； In the formula, is a low-resolution hyperspectral image, , is an image height, is an image width, is an image waveband, is a super-resolution network; The material component guided model comprises an unmixing network, a correction network and a reconstruction network. Will and Two demixing networks are input separately, the first demixing network is... As input, output a low-resolution abundance map. and low-resolution reconstruction features, for Upsampling is performed, and the upsampling result is input into the correction network; The second unmixing network uses As input, output the abundance map of the super-resolution image. and super-resolution reconstructed features, then Input the super-resolution reconstructed features into the correction network; the correction network outputs the corrected abundance features, which are then input into the reconstruction network; the reconstruction network outputs the reconstructed features.

2. The unmixing-based hyperspectral image super-resolution method of claim 1, wherein, Sub-pixel convolution layers are used for up-sampling. The spatial information reconstruction module first utilizes preliminary feature encoding on the , the result of which is input to a ConvLSTM skip connection layer, then up-sampling is performed, and finally the fusion features are output to obtain a spatial feature cube; The edge refinement module takes the spatial feature cube as input, extracts multi-scale edge information through a Laplacian operator and a Canny operator respectively, adds the results of the Laplacian operator and the Canny operator element by element to obtain edge information, and inputs the edge information into fusion, the fusion result is input into the edge attention layer and the ConvLSTM skip connection layer respectively, the processing results of the edge attention layer and the ConvLSTM skip connection layer are multiplied element by element, the multiplication result is added element by element with the spatial feature cube, and the addition result is subtracted element by element with the edge information to obtain edge features; The spectral information reconstruction module takes edge features as input, first performs spatiotemporal feature learning through a first ConvLSTM skip connection layer, and then performs deep feature modeling through a second ConvLSTM skip connection layer An activation function introduces nonlinearity, and then the modeling result is input into a spectral attention layer, and the result of the spectral attention layer is multiplied element by element with the edge features to obtain spectral features; The ConvLSTM skip-connection layer is a residual unit, which receives the input features and divides the input features into two branches, the first branch passes through the ConLSTM layer to output updated features, and the second branch sequentially passes through and the activation function, and the results of the two branches are added element by element to output the feature fusion result.

3. The unmixing-based hyperspectral image super-resolution method of claim 2, wherein, The unmixing network is a spectral spatial network guided by abundance, and comprises a spectral module, a spatial module and an abundance guided attention unmixing module. The spectral module takes a hyperspectral image as input, sequentially passes through , a batch normalization layer, an activation function, and , and iterates 4 times, and each iteration result is added to the previous iteration result element by element, and finally outputs a spectral feature ; The spatial module takes the hyperspectral image as input, sequentially passes through , a batch normalization layer, an activation function, and , and iterates 4 times, and each iteration result is added to the previous iteration result element by element, and finally outputs the spatial feature .

4. The unmixing-based hyperspectral image super-resolution method of claim 3, wherein, The abundance attention unmixing module unmixes the input image to obtain a plurality of unmixing results and fuse to obtain a fusion feature : ； In the formula, is activation function, is convolution operation; Using constructing a key matrix in an attention mechanism and a value matrix and using a coarse abundance map constructing a query matrix in an attention mechanism let be a dimension value of , is a dimension value of ； ； ； ； will be described below. , from reshaped into a sequence form , will be described below. from reshaped into a sequence form , the number of pixels, ; Weighted features are: ； wherein is a normalized exponential function, is a transpose symbol; Final abundance plot Is: ； If the input of the unmixing network is , is ; If the input of the unmixing network is , is .

5. The unmixing-based hyperspectral image super-resolution method of claim 4, wherein, The correction network constructs a linear weighted fusion model. ； wherein is up-sampling, is element-wise addition, is the corrected abundance information; Will Transposition, and the corresponding hyperspectral features are multiplied by matrix to get end member feature matrix .

6. The unmixing-based hyperspectral image super-resolution method of claim 5, wherein, Reconstruction network utilization Reconstructing hyperspectral image features, including estimating abundance maps, extracting endmember features, and image reconstruction; The estimated abundance profile comprises and constructing a similarity matrix : ； wherein is a transpose symbol, follows and is a constraint and non-negative constraint; The endmember feature extraction and image reconstruction comprise a reconstruction network output expression. ； In the formula, to reconstruct hyperspectral image features; Adopting multi-source information fusion strategy, will and characteristics of the level of integration: ； In the formula, is the super-resolution result after fusion, and is a learnable super-parameter matrix.

7. The unmixing-based hyperspectral image super-resolution method of claim 6, wherein, A super-resolution loss function is established, comprising an L1 loss, a spatial loss and a spectral loss. L1 loss is: ； wherein, is the original hyperspectral image, is the original hyperspectral image of the i-th sample, is the original hyperspectral image of the i-th sample, is the original hyperspectral image of the i-th sample, is the original hyperspectral image of the i-th sample, is the original hyperspectral image of the i-th sample, is the number of training samples, is the index of training samples, is the index of training samples, Loss of space Is: ； ； ； wherein , is a substitution variable, is an image band index, , is a pixel row index, , is a pixel column index, , is the pixel value at the th image band, the th row, the th column. Spectral loss Is: ； Super-resolution loss is: ； establishing an unmixing loss, including a reconstruction loss and an abundance loss, the reconstruction loss is: ； wherein, is a pixel index, , is the true endmember hyperspectral signature of the th pixel, is the reconstructed hyperspectral signature of the th pixel, is the L2 norm.

8. The unmixing-based hyperspectral image super-resolution method of claim 7, wherein, The abundance loss includes a sum of a constraint loss and a non-negative constraint loss, and the constraint loss is is: ； wherein is the number of endmembers, is the endmember index, , is the the first endmember of the first pixel; The abundance matrix is changed to: ； Non-negative constraint loss is: ； The abundance loss is: ； unmixing loss is: ； Composite loss function is: ； In the formula, is a weight hyperparameter of is a weight hyperparameter of

Images