A hyperspectral image unmixing method, system, device and storage medium

Abstract

The application provides a hyperspectral image unmixing method, system, device and storage medium, belonging to the technical field of hyperspectral image processing, comprising: obtaining an original hyperspectral image; processing the original hyperspectral image by using a spectral unmixing network architecture, carrying out mixed pixel decomposition on the hyperspectral image, obtaining an endmember spectrum and an abundance map; in the spectral unmixing network architecture, a dynamic image block number allocation strategy is adopted to dynamically extract patches from the input original hyperspectral image, the patch feature sequence after dynamic allocation is converted into tokens of a unified dimension, and the tokens are input into a stacked Transformer Block module, and enhanced features with global semantic and structural features are output; the enhanced features are input into a decoder, and are restored into the endmember spectrum and the abundance map corresponding to the space of the input hyperspectral image. In a complex mixed pixel scene, the unmixing precision is significantly improved, unnecessary calculation processes are reduced, the inference efficiency of the model is improved, and the balance between precision and efficiency is realized.

Description

Technical Field

[0001] This invention belongs to the field of hyperspectral image processing technology, specifically relating to a hyperspectral image demixing method, system, device, and storage medium. Background Technology

[0002] In the task of hyperspectral remote sensing image unmixing, the core of spectral unmixing lies in separating endmember spectra from the observed spectra (i.e., identifying the pure components contained in the mixed spectra, such as "vegetation spectra" and "soil spectra") and estimating their abundance (i.e., the proportion of each endmember in the mixed spectrum). However, hyperspectral images generally suffer from problems such as complex distribution of ground features, blurred boundary transitions, and diverse spectral mixing forms, making it difficult for traditional methods based on linear or nonlinear models to balance accuracy and robustness.

[0003] Existing linear methods (such as LMM and FCLSU) are based on the assumption of linear spectral mixing. While computationally simple, they cannot characterize nonlinear mixing processes such as multiple scattering that are prevalent in real-world scenes, leading to a significant decrease in accuracy in complex regions. Nonlinear methods (such as polynomial and kernel models) attempt to approximate nonlinear effects by introducing higher-order terms or mappings, but these methods are complex, parameter-sensitive, computationally expensive, and their predefined functions are difficult to adapt to diverse real-world mixing mechanisms, limiting their generalization ability and practicality. Both approaches struggle to balance accuracy, robustness, and efficiency. With the introduction of deep learning methods into spectral-spatial joint modeling, convolutional neural networks and autoencoders have shown outstanding performance in feature extraction. However, convolutional neural networks rely on local convolutional kernels, resulting in a limited receptive field and difficulty in modeling spectral and spatial relationships between distant pixels in an image, leading to insufficient ability to capture global structure and long-range dependencies. Although autoencoders can learn nonlinear mappings, their goal is mostly to minimize the overall reconstruction error. They tend to learn an averaged, undiscriminative latent representation, making it difficult to clearly separate endmembers with similar spectral features. Furthermore, they lack explicit modeling of spatial context relationships, making it difficult to effectively characterize long-range dependencies. Both of these factors limit further improvement in demixing accuracy.

[0004] The Visual Transformer (ViT), leveraging its self-attention mechanism for global modeling, has been increasingly applied to hyperspectral hybrid pixel decomposition. However, its fixed number and shape of patches result in insufficient fine-grained information, making it difficult to capture local details of ground object boundaries and spectral variations. Furthermore, redundant patch partitioning leads to wasted computational resources, limiting the model's efficiency in practical applications. Summary of the Invention

[0005] To address the shortcomings of existing technologies in processing hyperspectral image demixing, this invention provides a hyperspectral image demixing method, system, device, and storage medium.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A hyperspectral image unmixing method includes the following steps: Acquire the raw hyperspectral image; The original hyperspectral image is decomposed into mixed pixels using a spectral unmixing network architecture to obtain endmember spectra and abundance maps. The spectral unmixing network architecture is based on the ViT encoder, which replaces the fixed image blocks of the original ViT encoder with a dynamic image block number allocation strategy, and embeds a channel self-attention module in the Transformer coding block of the original ViT encoder. In the spectral unmixing network architecture, a dynamic image patch allocation strategy is used to dynamically extract patches from the input original hyperspectral image. The dynamically allocated feature sequence is converted into a block sequence representation of a unified dimension. The unified dimension block sequence representation is input into a stacked Transformer encoding block with embedded channel attention, and the output is an enhanced feature with global semantic and structural features. The enhanced feature is input into the decoder, and inverse patch mapping, upsampling and bi-branch linear mapping are performed in sequence to restore the endmember spectrum and abundance map corresponding to the input hyperspectral image space.

[0007] Preferably, the spectral demixing network architecture is composed of Cascaded subnetworks The system is composed of sub-networks, each containing an independent block segmenter and a Transformer encoding block structure. Each sub-network corresponds to a different image block partitioning granularity. Feature reuse and relation reuse mechanisms are introduced between each level of the cascaded sub-networks to dynamically determine whether to continue with a higher resolution patch allocation. Specifically, the sub-network with the fewest patches first performs feature encoding and abundance distribution prediction on the input image. If the prediction result at this stage meets the preset confidence level, the result is output and the inference process is terminated; otherwise, the next level sub-network is activated after calculation to perform the next level of fine-grained allocation calculation.

[0008] Preferably, the relation reuse mechanism is used to guide the calculation of self-attention in the next-level sub-network. Specifically, it calculates the attention weight matrix of the previous level based on the multi-head attention mechanism of the previous level network, performs scale alignment on the attention weight matrix of the previous level, introduces learnable weight coefficients, and combines the scale-aligned previous attention map with the current attention map through residual fusion to obtain the fused attention weight matrix. The fused attention weight matrix is ​​then used to replace the attention weight matrix of the next-level sub-network in the calculation.

[0009] Preferably, the feature reuse mechanism is used for channel splicing and fusion, specifically by performing channel alignment mapping on the final features output by the previous-level network, and then splicing and fusing the mapped previous-level features with the input features of the current stage.

[0010] Preferably, the step of inputting the enhanced features into the decoder and sequentially performing inverse patch mapping, upsampling, and bi-branch linear mapping to restore the endmember spectrum and abundance map corresponding to the input hyperspectral image space specifically involves: The feature sequence after dynamic patch allocation and ViT encoding is as follows: ,in, Indicates the number of patches. This represents the embedding feature dimension of each patch; The feature sequence is converted into a two-dimensional feature map by the inverse patch mapping operation, and then the two-dimensional feature map is upsampled to restore the original spatial size, resulting in a spatially aligned feature map. The spatially aligned feature maps are input into two parallel linear mapping heads to predict the endmember spectra and abundance coefficients. The endmember prediction head maps the feature vector of each pixel position to the spectral space corresponding to the number of endmembers. The abundance prediction head obtains the proportion coefficient of each endmember in the pixel based on the abundance constraints and abundance coefficient mapping.

[0011] Preferably, the method further includes calculating the reconstructed spectrum of the pixel based on the predicted endmember spectral matrix and abundance map; constructing a loss function to minimize the reconstruction error between the reconstructed spectrum and the original observed spectrum; and optimizing the network parameters of the spectral unmixing network architecture.

[0012] The present invention also provides a hyperspectral image demixing system, specifically comprising: The data module is used to acquire raw hyperspectral images.

[0013] The demixing module is used to perform mixed pixel decomposition on the original hyperspectral image using a spectral demixing network architecture to obtain endmember spectra and abundance maps. The spectral demixing network architecture is based on the ViT encoder, which replaces the fixed image blocks of the original ViT encoder with a dynamic image block number allocation strategy, and embeds a channel self-attention module in the Transformer coding block of the original ViT encoder. In the spectral unmixing network architecture, a dynamic image patch allocation strategy is used to dynamically extract patches from the input original hyperspectral image. The dynamically allocated feature sequence is converted into a block sequence representation of a unified dimension. The unified dimension block sequence representation is input into a stacked Transformer encoding block with embedded channel attention, and the output is an enhanced feature with global semantic and structural features. The enhanced feature is input into the decoder, and inverse patch mapping, upsampling and bi-branch linear mapping are performed in sequence to restore the endmember spectrum and abundance map corresponding to the input hyperspectral image space.

[0014] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps described in the hyperspectral image demixing method.

[0015] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, can execute the steps described in the hyperspectral image demixing method.

[0016] The hyperspectral image unmixing method provided by this invention has the following beneficial effects: This invention employs a dynamic image patch allocation strategy, guided by the difficulty of image discrimination. It utilizes a small number of patches to quickly model easily distinguishable regions, gradually introducing finer patches as the model information becomes insufficient, thus enhancing the representation of complex regions. This avoids information fragmentation caused by multi-level computation. Furthermore, a channel self-attention mechanism is introduced into the Transformer coding block to fully exploit the potential correlations between hyperspectral bands, highlighting discriminative spectral features and suppressing noise and redundant dimensions, thereby enhancing the discriminativeness and robustness of endmember extraction and abundance estimation. This improves the accuracy and efficiency of the model in processing hyperspectral image demixing. Attached Figure Description

[0017] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a spectral unmixing network architecture diagram of a hyperspectral image unmixing method according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the cascaded dynamic patch quantity allocation in an embodiment of the present invention.

[0020] Figure 3 This invention provides a comparison of endmember spectral visualization results for the Samson dataset in various embodiments. Figure 3 (a) is the result diagram of the tree endmember; Figure 3 (b) is the result diagram of the water-end element; Figure 3 (c) is the result diagram of soil endmembers.

[0021] Figure 4 This is a comparison chart of the abundance map results of the Samson dataset in an embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0023] Example This invention provides a method for demixing hyperspectral images, specifically including the following steps: Step 1: Obtain the original hyperspectral image from a publicly available hyperspectral remote sensing dataset and preprocess the original hyperspectral image.

[0024] Step 2: Construct a spectral unmixing network architecture, such as... Figure 1 As shown, the model as a whole adopts an end-to-end encoder-decoder framework based on the ViT structure to achieve the task of simultaneously extracting endmember spectra and abundance maps from the input hyperspectral image (HSI).

[0025] Step 3: Process the preprocessed original hyperspectral image using a spectral unmixing network architecture. Perform mixed pixel decomposition on the hyperspectral image to obtain endmember spectra and abundance maps. The overall processing flow of the model can be divided into three main stages: dynamic patch extraction, channel enhancement and deep feature extraction, and high-level semantic unmixing.

[0026] (1) At the input end, the preprocessed hyperspectral image is... The data is fed into an improved Tokens-to-Token Vision Transformer (T2T-ViT) encoder for feature extraction. This module divides the original image into several non-fixed-size image patches using a staged deformable patch partitioning strategy, and unfolds local regions to form initial token representations. The model introduces a dynamic patch allocation mechanism based on adaptive aggregation, which can adaptively adjust the number and range of patches according to local spectral-spatial differences, thereby achieving high-fidelity modeling of spatial structure and spectral details.

[0027] A cascaded dynamic patch quantity allocation strategy was adopted for dynamic patch extraction. By constructing multi-level Transformer sub-networks, progressive image modeling from coarse-grained to fine-grained was achieved. The cascaded dynamic patch quantity allocation is as follows: Figure 2 As shown.

[0028] The entire network is composed of Cascaded subnetworks The network is composed of independent Patch Tokenizer and Transformer encoder structures, each corresponding to a different patch partitioning granularity. ,in, Indicates the first The number of patches in each stage. The network is first divided into coarsest-grained subnetworks (fewest patches). For the input image Perform fast feature encoding and abundance distribution prediction. If the prediction results at this stage have sufficiently high confidence, output the results directly and terminate the inference process early; otherwise, activate the next sub-network. Further supplementary features are extracted at a finer-grained patch level, and updated abundance predictions are output. This process continues until the confidence condition is met or the finest-grained stage is reached. .

[0029] Let the first The output prediction of the stage subnetwork is The corresponding classification probability can be calculated using the softmax function as follows: (1); in, For the number of categories (or endmembers), Indicates the first Phase 1 to the first The predicted probability of the class. The prediction confidence for the current stage is defined as the maximum softmax function:

[0030] (2); If the conditions are met ,in, If a pre-set reliability threshold is set, the output at the current stage is determined to have sufficient reliability, the inference process is terminated, and the output is released. This serves as the final prediction result; otherwise, the process proceeds to the next stage. The sub-networks are used to obtain higher-precision inference results. Through this inference mechanism of "coarse-grained priority, confidence-driven, and fine-grained supplementation", the model can maintain high unmixing accuracy while avoiding performing complete fine-grained inference on all samples, effectively reducing computational overhead.

[0031] This cascaded strategy implements an adaptive patch allocation mechanism based on image discrimination difficulty, enabling the inference process to exhibit the characteristics of "fast convergence on simple samples and thorough modeling on complex samples." Compared with the traditional ViT model with fixed patch partitioning, this strategy significantly improves demixing accuracy in complex mixed pixel scenes, while reducing unnecessary computation on a large number of low-complexity samples, greatly improving the model's inference efficiency.

[0032] In the attention mechanism of ViT and its variants, the model typically constructs a Self-Attention graph independently at each stage to model the relationships between patches. However, this "from scratch" relationship modeling ignores the potential structural correlations between different stages, causing the network to repeatedly learn similar attention distributions in deep structures, increasing computational overhead and limiting the hierarchical integration capability of feature representations. After the (k-1)th level sub-network completes its computation, the dynamic patch allocation framework introduces Relation Reuse and Feature Reuse mechanisms between each cascaded sub-network. The output of Relation Reuse is used to initialize and guide the self-attention computation of the k-th level sub-network, while the output of Feature Reuse is concatenated with the original input features of the k-th level sub-network, serving as the input to the Transformer encoder of the k-th level sub-network. This cross-stage information flow improves the stability of the attention distribution and the discriminativeness of the feature representation, enhancing the temporal consistency and cross-scale correlation of features, reducing redundant computation, and improving representational power.

[0033] Let the first The input Patch feature sequence in the stage is ,in This represents the number of patches in this phase. For the feature dimension, the standard multi-head self-attention (MHSA) process can be represented as:

[0034] (3); (4); in, These are the query, key, and representation obtained through linear mapping, respectively. This is the attention weight matrix. The output features are those generated after attention aggregation.

[0035] In the relationship reuse mechanism, an attention graph from the previous stage is introduced. It is then combined with the current stage attention through residual fusion, as shown in the following formula: (5); in, This indicates the attention map scale alignment operation achieved through bilinear interpolation or sparse mapping. These are learnable weight coefficients used to control the fusion ratio of relational information between preceding and subsequent stages. This mechanism can explicitly inherit the global dependency structure of previous stages in earlier stages, reducing repetitive attention reconstruction processes and improving the convergence efficiency of relational modeling.

[0036] In the feature reuse mechanism, a cross-stage feature fusion module is designed to fuse the features output from the previous stage. After channel alignment mapping, the data is concatenated and fused with the input features of the current stage: (6); in, The channel number mapping operation ensures dimension matching. This fusion strategy injects low-to-mid-frequency context information extracted in previous stages into the current stage, enabling the network to fully reference existing global context when capturing high-frequency details, thereby improving the discriminativeness and stability of the representation.

[0037] Relation reuse and feature reuse mechanisms together constitute the core design of cross-stage information interaction in the dynamic patch framework. On the one hand, relation reuse effectively avoids repeated learning of attention distribution and reduces computational overhead; on the other hand, feature reuse enables progressive integration of multi-stage features, alleviating feature drift in deep Transformers, thereby improving the overall accuracy and convergence efficiency of hybrid pixel decomposition.

[0038] (2) In the ViT structure, the backbone network typically uses Multi-Head Self-Attention (MHSA) to model long-distance dependencies in the spatial dimension. However, the features of hyperspectral images are not only correlated in the spatial dimension, but also have significant physical coupling relationships in the spectral dimension (i.e., the channel dimension). Traditional spatial attention mechanisms, while capturing the global context, often ignore the complementarity and redundancy between spectral channels, causing the model to fail to fully utilize the dependencies between channels for information reweighting, thus limiting the feature representation ability. While keeping the original ViT backbone coding framework unchanged, module enhancement is performed on the Transformer coding blocks, that is, a Sqeuuze-and-Excitation (SE) channel attention module is embedded after the residual link of each Transformer coding block to achieve feature enhancement based on channel dependencies.

[0039] This module adaptively assigns importance weights to each channel, thereby highlighting channel features that contribute more to the unmixing task, suppressing invalid or noisy channels, and improving the discriminative power of spectral features. Specifically, let the input features of a certain layer be:

[0040] (7); in, For batch size, For the number of channels, and The spatial dimension is defined as follows. The SE channel attention mechanism first aggregates the spatial dimensions through Global Average Pooling (GAP) to obtain a global channel description vector:

[0041] (8); Then, Channel attention weights are learned from a bottleneck network consisting of two fully connected (FC) layers and non-linear activations: (9); in, , For learnable weight matrix, It is the ReLU activation function. For the Sigmoid function, This is the channel compression ratio, used to reduce computing costs.

[0042] Finally, the attention weights Multiply the original feature channel by channel to achieve channel weighting: (10); Obtain the enhanced feature map .

[0043] The tokens obtained after PatchEmbedding projection are fed into a series of stacked TransformerBlocks for deep feature modeling. Each Block consists of Multi-Head Self-Attention (MHSA), a channel attention module based on the Squeeze-and-Excitation (SE) mechanism, and a Feed-Forward Network (FFN). The channel attention module is used to explicitly model the channel dependencies in the spectral dimension, improving the discriminative and expressive power of hyperspectral features. Through the stacking of multiple TransformerBlocks, the model can progressively capture long-range spectral-spatial dependencies and global contextual information, obtaining global token representations with rich semantic and structural features.

[0044] (3) After completing the global feature extraction in the encoder stage, the network needs to restore the high-dimensional Patch-level feature sequence to the endmember and abundance distribution map corresponding to the input image space in order to achieve the goal of hybrid pixel decomposition.

[0045] A lightweight decoding module was designed to rearrange, upsample, and reconstruct the spectrum of the encoded features, thus completing the mapping from the high-dimensional embedding space to the physical meaning space.

[0046] Suppose the feature sequence after dynamic patch allocation and ViT encoding is: (11); in, Indicates the number of patches. This represents the embedding feature dimension of each patch. The goal is to restore these features to the same spatial resolution as the original input image. First, the sequence features are converted into a two-dimensional feature map through the inverse patch mapping operation (i.e., rearrangement and concatenation):

[0047] (12); in, The size of the spatial grid after the patch is divided. If or Then, the original space size is restored through upsampling:

[0048] (13); After obtaining the spatially aligned feature maps, the decoder predicts the endmember spectra and abundance coefficients using two parallel linear heads. Specifically, the endmember prediction head predicts the feature vectors at each pixel location. Mapping to endmembers The corresponding spectral space:

[0049] (14); in, , Indicates the number of spectral channels. For pixels The endmember spectral estimation matrix is ​​obtained. Meanwhile, the abundance prediction head obtains the proportion coefficients of each endmember in the pixel through another set of parameter mappings:

[0050] (15); in, , ,and All abundance coefficients are guaranteed to be non-negative and sum to 1 to satisfy the abundance constraint: (16); (17); Finally, the reconstructed spectrum of each pixel can be obtained by a linear combination of the endmember spectrum and the abundance coefficient: (18); in, Represents a pixel The reconstructed spectrum should be as close as possible to the original observed spectrum. During the training phase, model optimization is achieved by minimizing the reconstruction error loss function.

[0051] (19); Through the above decoding process, the network maintains the high discriminative feature representation brought about by dynamic patch allocation and channel attention enhancement, while achieving an effective mapping from the feature domain to the physical domain, enabling endmember and abundance estimation to be recovered with high accuracy in both spatial and spectral dimensions.

[0052] The encoded global tokens are mapped to two branches, endmembers and abundance, through the decoding head: on the one hand, the endmember spectral matrix is ​​predicted through a fully connected layer. On the other hand, tokens are upsampled to the original image resolution and pixel-level abundance maps are predicted. During training, by combining reconstruction loss and abundance constraint loss, the model is guided to learn the optimal endmember and abundance representation end-to-end, thereby achieving mixed pixel decomposition of hyperspectral images to obtain endmember spectra and abundance maps.

[0053] To verify the effectiveness and superiority of the spectral unmixing network architecture, experiments and comparative analyses were conducted on publicly available hyperspectral remote sensing datasets. The comparison methods included five mainstream unmixing methods: traditional linear unmixing methods (Collab, FCLSU, NMF), methods incorporating nonlinear constraints (CyCU), and deep learning-driven methods (DeepTrans). The comparative evaluation highlights the performance advantages of our proposed method in spectral unmixing tasks.

[0054] Finally, ablation experiments were designed to analyze the role of dynamic patch allocation mechanism and channel attention module in improving overall model performance, thereby verifying the necessity and effectiveness of each key design step.

[0055] The root mean square error (RMSE) and spectral angular distance (SAD) were selected as the evaluation indicators to quantitatively measure the results from the aspects of reconstruction accuracy and spectral fidelity, respectively.

[0056] To ensure the validity and reproducibility of the experiment, the hyperparameter settings for the Samson dataset are shown in Table 1. Specifically, these include the number of endmembers. Number of bands Patch size, reconstruction and spectral angular distance loss weighting parameters Number of training epochs, learning rate and weight decay rate .

[0057] Table 1 Hyperparameter Settings Tables 2 and 3 present the experimental results of the experimental method and five comparative methods on the Samson dataset under the RMSE and SAD metrics, respectively. In the tables, the red bold values ​​represent the best results achieved in the overall evaluation of that category, while the black bold values ​​correspond to the second-best results, in order to more intuitively compare the performance differences between different methods. Figure 3 This section presents a comparison of the endmember spectral visualization results for various methods on the Samson dataset. Figure 4 The abundance plot results of each method on the Samson dataset are compared to visually evaluate the performance of the proposed method in terms of endmember extraction accuracy and spectral preservation ability.

[0058] In terms of abundance estimation, the proposed method achieved an average RMSE of 0.0928, outperforming other comparative methods and demonstrating the highest overall accuracy. Specifically, the proposed method achieved optimal results for both water and soil endmembers, effectively reducing abundance estimation errors. For tree endmembers, the method's results were slightly lower than Collab and DeepTrans, but the overall difference was small, still at a superior level. Regarding endmember extraction, the proposed method achieved an average SAD value of 0.0671, outperforming other comparative methods and demonstrating the best performance. Specifically, the proposed method achieved optimal or near-optimal results for tree, water, and soil endmembers, reflecting its advantage in spectral fidelity modeling. In summary, the proposed dynamic patch allocation mechanism not only effectively reduces abundance estimation errors but also improves the accuracy of endmember spectral reconstruction, exhibiting strong adaptability and robustness across different land cover categories.

[0059] Table 2 Comparison of RMSE results in the Samson dataset Table 3 Comparison of SAD results in the Samson dataset Ablation Experiments: To verify the effectiveness of the three modules in the unmixing network—dynamic patch allocation, relation reuse and feature reuse, and channel attention enhancement—in the hybrid pixel decomposition task, systematic ablation experiments were conducted on the Samson dataset. The experiments used the complete model as a baseline, removing different modules sequentially and comparing performance differences under RMSE and SAD metrics. The results are shown in Table 4.

[0060] Table 4. Quantitative experimental results of the module ablation experiment As shown in Table 4, the complete model outperforms other control models in both RMSE and SAD metrics, validating the comprehensive advantages of the proposed method in spectral preservation and abundance estimation. Specifically, removing dynamic patch assignment resulted in the most significant performance degradation, indicating that the mechanism effectively adapts to spatial heterogeneity and improves endmember extraction accuracy. Removing relation reuse and feature reuse also led to performance degradation, demonstrating the important role of this mechanism in reducing feature redundancy and enhancing inter-layer correlations. Removing the channel attention enhancement module reduced the model's sensitivity to key spectral dimensions, resulting in a decrease in overall spectral fidelity. Furthermore, removing all modules simultaneously resulted in the most severe performance degradation, approaching the level of traditional methods.

[0061] In summary, dynamic patch allocation, relation and feature reuse, and channel attention enhancement all play a key role in improving the performance of hybrid pixel decomposition in different dimensions, and their synergistic effect ensures the robustness and accuracy of the method in complex scenarios.

[0062] The present invention also provides a hyperspectral image demixing system, comprising: The data module is used to acquire raw hyperspectral images.

[0063] The demixing module is used to decompose the original hyperspectral image into mixed pixels using a spectral demixing network architecture to obtain endmember spectra and abundance maps. The spectral demixing network architecture is based on the ViT encoder, which replaces the fixed image blocks of the original ViT encoder with a dynamic image block number allocation strategy, and embeds a channel self-attention module in the Transformer coding block of the original ViT encoder. In the spectral unmixing network architecture, a dynamic image patch allocation strategy is adopted to dynamically extract patches from the input original hyperspectral image. The dynamically allocated feature sequence is converted into a block sequence representation of uniform dimension. The uniform-dimensional block sequence representation is input into the stacked Transformer encoding blocks embedded with channel attention, and the output is an enhanced feature with global semantic and structural features. The enhanced feature is input into the decoder, which is then processed by inverse patch mapping, upsampling and bi-branch linear mapping in sequence to restore the endmember spectrum and abundance map corresponding to the input hyperspectral image space.

[0064] The modules in the aforementioned hyperspectral image demixing system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0065] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in an embodiment of a hyperspectral image demixing method. Specific implementation methods can be found in the method embodiments, and will not be repeated here.

[0066] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in an embodiment of a hyperspectral image demixing method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0067] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for demixing hyperspectral images, characterized in that, Includes the following steps: Acquire the raw hyperspectral image; The original hyperspectral image is decomposed into mixed pixels using a spectral unmixing network architecture to obtain endmember spectra and abundance maps. The spectral unmixing network architecture is based on the ViT encoder, which replaces the fixed image blocks of the original ViT encoder with a dynamic image block number allocation strategy, and embeds a channel self-attention module in the Transformer coding block of the original ViT encoder. In the spectral unmixing network architecture, a dynamic image patch allocation strategy is adopted to dynamically extract patches from the input original hyperspectral image. The dynamically allocated feature sequence is converted into a block sequence representation of a unified dimension. The block sequence representation of the unified dimension is input into the stacked Transformer encoding blocks embedded with channel attention, and the output is an enhanced feature with global semantic and structural features. The enhanced features are input into the decoder and sequentially processed by inverse patch mapping, upsampling, and bi-branch linear mapping to restore the endmember spectrum and abundance map corresponding to the input hyperspectral image space.

2. The hyperspectral image demixing method according to claim 1, characterized in that, The spectral demixing network architecture is composed of Cascaded subnetworks The system consists of a sub-network, each containing an independent block segmenter and a Transformer encoding block structure. Each sub-network corresponds to a different image block partitioning granularity. Feature reuse and relation reuse mechanisms are introduced between each level of the cascaded sub-networks to dynamically determine whether to continue allocating more high-resolution patches. Specifically, the sub-network with the fewest patches first performs feature encoding and abundance distribution prediction on the input image. If the prediction result at this stage meets the preset confidence level, the result is output and the inference process is terminated. Otherwise, after calculation, the next level sub-network is activated to perform the next level of fine-grained allocation calculation.

3. The hyperspectral image demixing method according to claim 2, characterized in that, The relation reuse mechanism is used to guide the calculation of self-attention in the next-level sub-network. Specifically, it is based on the multi-head attention mechanism of the previous-level network to calculate the attention weight matrix of the previous level, scale-align the attention weight matrix of the previous level, introduce learnable weight coefficients, and combine the scale-aligned previous attention map with the current attention map through residual fusion to obtain the fused attention weight matrix. The fused attention weight matrix is ​​used to replace the attention weight matrix of the next-level sub-network in the calculation.

4. The hyperspectral image demixing method according to claim 2, characterized in that, The feature reuse mechanism is used for channel splicing and fusion. Specifically, it performs channel alignment mapping on the final features output by the previous level network, and splices and fuses the mapped previous level features with the input features of the current stage.

5. The hyperspectral image demixing method according to claim 1, characterized in that, The step of inputting the enhanced features into the decoder and sequentially performing inverse patch mapping, upsampling, and bi-branch linear mapping to restore the endmember spectrum and abundance map corresponding to the input hyperspectral image space is as follows: The feature sequence after dynamic patch allocation and ViT encoding is as follows: ,in, Indicates the number of patches. This represents the embedding feature dimension of each patch; The feature sequence is converted into a two-dimensional feature map by the inverse patch mapping operation, and then the two-dimensional feature map is upsampled to restore the original spatial size, resulting in a spatially aligned feature map. The spatially aligned feature maps are input into two parallel linear mapping heads to predict the endmember spectra and abundance coefficients. The endmember prediction head maps the feature vector of each pixel position to the spectral space corresponding to the number of endmembers. The abundance prediction head obtains the proportion coefficient of each endmember in the pixel based on the abundance constraints and abundance coefficient mapping.

6. The hyperspectral image demixing method according to claim 1, characterized in that, It also includes calculating the reconstructed spectrum of pixels based on the predicted endmember spectral matrix and abundance map; constructing a loss function to minimize the reconstruction error between the reconstructed spectrum and the original observed spectrum; and optimizing the network parameters of the spectral unmixing network architecture.

7. A hyperspectral image demixing system, characterized in that, include: The data module is used to acquire raw hyperspectral images; The demixing module is used to perform mixed pixel decomposition on the original hyperspectral image using a spectral demixing network architecture to obtain endmember spectra and abundance maps. The spectral demixing network architecture is based on the ViT encoder, which replaces the fixed image blocks of the original ViT encoder with a dynamic image block number allocation strategy, and embeds a channel self-attention module in the Transformer coding block of the original ViT encoder. In the spectral unmixing network architecture, a dynamic image patch allocation strategy is adopted to dynamically extract patches from the input original hyperspectral image. The dynamically allocated feature sequence is converted into a block sequence representation of a unified dimension. The block sequence representation of the unified dimension is input into the stacked Transformer encoding blocks embedded with channel attention, and the output is an enhanced feature with global semantic and structural features. The enhanced features are input into the decoder and sequentially processed by inverse patch mapping, upsampling, and bi-branch linear mapping to restore the endmember spectrum and abundance map corresponding to the input hyperspectral image space.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the method according to any one of claims 1 to 6.

Images