A hyperspectral unmixing method and system based on multi-scale spatial-spectral pyramid Mamba fusion

Abstract

The present application relates to the technical field of hyperspectral remote sensing image processing, and discloses a hyperspectral unmixing method based on multi-scale space-spectrum pyramid Mamba fusion, which comprises the following steps: constructing a double-branch multi-scale feature encoder, extracting multi-scale spatial features and modeling outputting a spatial abundance map through SpaMamba, extracting multi-scale spectral features and modeling outputting a spectral abundance map through SpeMamba, and estimating pixel-level endmember uncertainty through a variational autoencoder; inputting the spatial abundance map, the spectral abundance map and the uncertainty into an uncertainty perception pyramid Mamba fusion module, dynamically and deeply fusing the spatial and spectral features on a multi-scale pyramid structure, and outputting a fused abundance map; inputting the fused abundance map and the spectral abundance map into a decoupled guided progressive abundance refinement module, performing closed-loop correction, and outputting a refined abundance map; and reconstructing the spectrum based on the refined abundance map and an endmember matrix; and the present application effectively solves the problems of insufficient spatial heterogeneity modeling and insufficient endmember variability description.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral remote sensing image processing technology, specifically to a hyperspectral demixing method and system for multi-scale spatial-spectral pyramid Mamba fusion. Background Technology

[0002] Hyperspectral remote sensing imaging can acquire spectral information of ground features across hundreds of consecutive bands, providing a data foundation for subsequent ground feature identification and quantitative analysis. However, due to the limited spatial resolution of sensors and the complex distribution of ground features, a single pixel may consist of the spectra of multiple substances, forming a mixed pixel. Hyperspectral unmixing (HU) is the process of separating endmembers (i.e., the spectral characteristics of pure substances) and their corresponding abundances (the proportion of each endmember in that pixel) from mixed pixels.

[0003] The Linear Mixing Model (LMM), as the most widely used model, assumes that the spectral signal received by the sensor is a linear combination of the spectra of each endmember within a pixel, with the weights representing the abundance of each endmember. Early unmixing methods often used geometric methods to extract endmembers and then estimated the abundance based on the LMM. Although traditional sparse regression, matrix factorization, and statistical modeling methods have improved the unmixing accuracy to some extent, in real-world remote sensing scenarios, endmember spectral variability is prevalent due to natural factors such as illumination and atmospheric effects, making it difficult for traditional geometric methods to accurately describe complex mixing processes.

[0004] With the development of deep learning, autoencoders, due to their natural structural fit with unmixing tasks, have been introduced into the unmixing field in recent years. Encoders compress high-dimensional spectra into a low-dimensional space, and decoders use this low-dimensional representation to reconstruct the original spectrum. Their weight matrix can be viewed as endmembers, enabling simultaneous learning of endmembers and abundance. Researchers have successively proposed a series of deep unmixing models, such as convolutional autoencoders, cascaded autoencoders, and attention-enhanced networks. To address the endmember variability problem, variational autoencoders have been introduced into unmixing tasks, characterizing the spectral variability of endmembers through probabilistic modeling. The Transformer architecture, due to its excellent long sequence modeling capabilities, has been applied to hyperspectral unmixing, but its time and space efficiency are limited by its quadratic computational complexity.

[0005] However, existing methods still face the following key technical challenges: First, the information characteristics of the spatial and spectral domains are fundamentally different. Although most methods adopt spatial-spectral dual-branch processing, they fail to effectively utilize the differences in domain characteristics. Moreover, the two branches in the encoder stage are independent of each other and lack effective interaction, resulting in insufficient handling of spatial heterogeneity and endmember variability. Second, after dual-branch processing, existing methods only perform simple splicing or weighted summation of the results, failing to effectively utilize the complementary features extracted by the two branches for deep fusion. Third, current models lack effective follow-up optimization processing after generating abundance maps, and the abundance maps produced by the models often have problems such as blurred boundaries and missing details.

[0006] Therefore, there is an urgent need for a hyperspectral unmixing method that can fully address spatial heterogeneity and endmember variability, achieve deep fusion of spatial-spectral features, and effectively post-process abundance maps. Summary of the Invention

[0007] The purpose of this invention is to provide a hyperspectral demixing method and system for multi-scale spatial-spectral pyramid Mamba fusion, so as to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a hyperspectral demixing method for multi-scale spatial-spectral pyramid Mamba fusion, comprising the following steps:

[0009] S1: Acquire raw hyperspectral image data, construct a dual-branch multi-scale feature encoder, extract multi-scale spatial features through the spatial encoder, and perform spatial global dependency modeling using the SpaMamba module to output spatial abundance map A. spa Multi-scale spectral features are extracted using a spectral encoder and then modeled for long-range spectral dependence using the SpeMamba module, outputting spectral abundance map A. spe ; Pixel-level endmember uncertainty U is estimated using a variational autoencoder spa and U spe ;

[0010] S2: The spatial abundance map A spa Spectral abundance diagram A spe and uncertainty U spa U spe The uncertainty-aware pyramid Mamba fusion module is used to input spatial and spectral features into a multi-scale pyramid structure. These features are flattened into sequences and input into the Mamba module for sequence modeling. The Mamba-enhanced features are then concatenated with the uncertainty prior along the channel dimension and input into a gated convolutional network to generate fusion weights. Based on these weights, the spatial and spectral features are dynamically and deeply fused, outputting a fusion abundance map A. fused ;

[0011] S3: The fusion abundance map A fused and the spectral abundance diagram A spe The input-decoupled, progressive abundance refinement module uses the spectral abundance map as a conservative prior and performs closed-loop correction on the fused abundance through multi-scale iterative optimization, outputting a refined abundance map A. final ;

[0012] S4: Based on the refined abundance map A final The spectral reconstruction is performed using the endmember matrix E to obtain the reconstructed hyperspectral image.

[0013] Preferably, the spatial encoder adopts a multi-scale resolution pyramid structure, extracts spatial features in three parallel branches at 1 / 4, 1 / 2 and original resolutions, and then splices and fuses them after upsampling and alignment. It then fuses the channels through 1×1 convolution and adds residual connections, and subsequently uses spatial attention to enhance key spatial locations.

[0014] Preferably, the SpaMamba module generates a spatial attention weight map to weight the features, adds learnable positional encoding to the weighted feature map, flattens it into a sequence, inputs it into Mamba for global dependency modeling, and the output is fused with the original features through residual connections.

[0015] Preferably, the variational autoencoder takes the pixel spectrum of the original hyperspectral image as input and outputs the mean μ of the endmember distribution through two independent encoders. s μ d And log-variance logσ² s ,logσ² d After reparameterization, the decoder generates the transformation matrix ψ∈ and perturbation matrix ΔM∈ Endmembers are generated using the formula E=Sigmoid(M0ψ+ΔM), where M0 is the basic endmember matrix initialized by VCA, P is the number of endmembers, and L is the number of bands; the pixel-level endmember uncertainty is determined by the logarithmic variance using the formula Calculated.

[0016] Preferably, the input of the spectral encoder is the original hyperspectral image and the first reconstruction result adaptively weighted and fused by learnable weights. The spectral encoder uses three parallel convolutional kernels to capture the global spectral contour, local spectral changes and fine-grained spectral details respectively. After feature concatenation, it is batch normalized and activated by LeakyReLU. Then, the key bands are enhanced by channel attention and connected with the original feature residuals.

[0017] Preferably, the SpeMamba module explicitly models the local correlation between bands through graph convolution, constructs a learnable band adjacency matrix, normalizes it with Softmax, multiplies it with spectral features, and flattens the features into a spectral sequence to input Mamba to capture long-range dependencies between bands. The graph convolution output and the Mamba output are concatenated in the channel dimension and then fused through a linear layer.

[0018] Preferably, the uncertainty-aware pyramid Mamba fusion module performs dynamic deep fusion at both the original scale and the 1 / 2 downsampling scale. At each scale, the spatial abundance map and spectral abundance map are projected to the same latent space dimension through a 1×1 convolution. After being flattened into sequences, they are input into the Mamba module. The Mamba-enhanced features are concatenated with the uncertainty prior and then input into a gated convolutional network to generate a gated signal G=σ(Conv 1×1 ([M spa M spe U spa U spe The fusion results are categorized by F. fused =M spa ⊙G+M spe ⊙(1-G) combination; the fusion results of the two scales are fused into a pyramid through a cross-scale gating network to output the final fused abundance map.

[0019] Preferably, the decoupled-guided progressive abundance refinement module stitches the fused abundance map and the spectral abundance map along the channel dimension, and generates the refinement uncertainty U through three layers of 3×3 convolution and one layer of 1×1 convolution. ref Based on this uncertainty, adaptive initialization fusion A is performed. init =Softmax(σ(U ref )⊙A spe +(1-σ(U ref ))⊙A fused Subsequently, iterative optimization was performed using a multi-scale refinement network with three different dilation rates (1, 2, and 4). In each iteration, the current abundance and refinement uncertainty were concatenated and input into the refinement network. After fusion by 1×1 convolution, candidate abundances were generated through Softmax. The abundances were then updated by generating fusion weights through an update gate network until an adaptively determined number of iterations was reached.

[0020] Preferred: The total loss function used during training is:

[0021]

[0022] Where L rec To reconstruct the loss, a combination of mean square error and spectral angular distance is used; L kl For KL divergence loss; L volFor endmember volume constraint loss; L sad For endmember spectral angular distance constraint loss; L self For self-supervised constraint loss; L ref To refine the loss; λ kl , λ vol , λ sad , λ self , λ ref These are the corresponding weighting coefficients.

[0023] The hyperspectral unmixing system based on the above-described multi-scale spatial-spectral pyramid Mamba fusion includes: a dual-branch multi-scale feature encoder for extracting spatial abundance maps, spectral abundance maps, and pixel-level endmember uncertainties; an uncertainty-aware pyramid Mamba fusion module for adaptive deep fusion of spatial and spectral features; a decoupled-guided progressive abundance refinement module for iteratively optimizing and correcting the fused abundance; and a spectral reconstruction module for spectral reconstruction based on the refined abundance map and endmember matrix.

[0024] Compared with the prior art, the beneficial effects of this invention are as follows:

[0025] This invention proposes a dual-branch multi-scale feature encoder. The spatial branch utilizes a multi-scale resolution pyramid and the SpaMamba module to suppress the loss of smoothing details and enhance the adaptability to spatial heterogeneity. The spectral branch employs a multi-scale convolution kernel and the SpeMamba module, combined with VAE probabilistic modeling, to achieve explicit characterization of endmember variability and accurate differentiation of similar endmembers. The dual-branch multi-scale collaboration provides a high-quality feature foundation for unmixing, effectively solving the problems of insufficient spatial heterogeneity modeling and inadequate characterization of endmember variability.

[0026] This invention proposes an uncertainty-aware pyramid Mamba fusion module, which innovatively introduces the pixel-level endmember uncertainty estimated by VAE as a learnable gating signal into the Mamba fusion framework. It dynamically modulates the fusion weights of spatial and spectral features on a multi-scale pyramid structure and generates interpretable weights at the endmember level, thereby achieving adaptive deep fusion of spatial and spectral features and overcoming the shortcomings of insufficient fusion by simple splicing or weighted summation in existing technologies.

[0027] This invention designs a progressive abundance refinement module based on uncertainty decoupling. It uses the abundance estimation of spectral branches as a conservative prior and performs closed-loop correction of the fused abundance through iterative optimization and constraints. This effectively recovers the small endmembers that may be lost during the fusion process, while avoiding over-smoothing or the introduction of artifacts. It solves the problem of existing models lacking effective post-processing of abundance maps.

[0028] Experimental results on three public benchmark datasets, Samson, Jasper Ridge, and Houston, demonstrate that the endmember extraction accuracy is superior to existing comparative methods, verifying the superiority of this invention in handling endmember variability and spatial heterogeneity. Attached Figure Description

[0029] Figure 1 (a) is a schematic diagram of the overall architecture of the multi-scale spatial-spectral pyramid Mamba fusion network (MSSPM-Net) of the present invention; (b) is a schematic diagram of the structure of the SpaMamba module of the present invention; (c) is a schematic diagram of the structure of the SpeMamba module of the present invention; and (d) is a schematic diagram of the structure of the uncertainty-aware Mamba fusion module (UMF) of the present invention.

[0030] Figure 2 The above is a visualization comparison of the abundance plots of this invention on the Samson dataset;

[0031] Figure 3 This is a comparison chart of the endmember spectral extraction results of this invention on the Samson dataset;

[0032] Figure 4 This is a visualization comparison of the abundance plots of this invention on the Jasper Ridge dataset;

[0033] Figure 5 This is a comparison chart of the endmember spectral extraction results of this invention on the Jasper Ridge dataset;

[0034] Figure 6 This is a visualization comparison of the abundance plots of this invention on the Houston dataset;

[0035] Figure 7 This is a comparison chart of the endmember spectral extraction results of this invention on the Houston dataset. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Example 1

[0038] like Figure 1As shown, the multi-scale spatial-spectral pyramid Mamba fusion network (MSSPM-Net) proposed in this invention mainly consists of three parts: a dual-branch multi-scale feature encoder (DMFE), an uncertainty-aware pyramid Mamba fusion module (UPMF), and a decoupled guided progressive abundance refinement module (DGPR).

[0039] The dual-branch multi-scale feature encoder (DMFE) consists of three parallel sub-modules: a spatial encoder, a spectral encoder, and a variational autoencoder (VAE).

[0040] The spatial encoder takes the original HSI as input and captures multi-granular spatial structures at multiple scales, which helps to preserve both local details and high-level semantic information. Three parallel branches extract spatial features at 1 / 4, 1 / 2, and original resolutions. After upsampling and alignment, these features are concatenated and fused. A 1×1 convolution is used to fuse channels and residual connections are added. Then, spatial attention is used to enhance key spatial locations. The enhanced features are input to the SpaMamba module, which generates a spatial attention weight map to weight the features. The weighted feature map is then infused with learnable positional encodings, flattened into a sequence, and input into Mamba for global dependency modeling. The output is fused with the original features via residual connections. The enhanced features are then progressively reduced in dimensionality through two layers of spatial attention calibration and three layers of convolution. Finally, a Softmax layer is used to output a spatial abundance map. .

[0041] The variational autoencoder of this invention introduces residual connections and parameter range constraints to improve training stability based on the existing dual-encoder architecture. This part takes the original HSI pixel spectrum as input and outputs the mean of the endmember distribution through two independent encoders. Sum of logarithmic variance ,in The global structure of the control terminal combination, The local perturbations of the control endmembers, after reparameterization, are used by the decoder to generate the transformation matrix. and perturbation matrix Endmembers are generated using the following formula:

[0042]

[0043] in The basic endmember matrix initialized for VCA is designed to allow endmembers to adaptively adjust within the linear combination space of the initial spectrum. Describe the transformation relationship between endmembers. To capture the factors contributing to spectral variation, the pixel-level endmember uncertainty estimated by VAE is calculated from the log-variance, and the spatial uncertainty is obtained using the following formula. and spectral uncertainty :

[0044]

[0045] This uncertainty reflects the model's confidence in the endmember estimation. High uncertainty indicates that the pixel may be in a mixed region, a noisy environment, or a scene with changing lighting, making the endmember estimation unreliable.

[0046] The input to the spectral encoder consists of the original HSI and the first reconstruction result, which are adaptively weighted and fused using learnable weights. Calculations were performed using spatial abundance maps and endmember matrices:

[0047]

[0048] Then, the components are fused using learnable weights W:

[0049]

[0050] in For the original HSI, the weight W is automatically adjusted during training, enabling the model to dynamically balance the contributions of the original spectral information and spatial prior information according to the characteristics of the dataset. The spectral encoder uses three parallel convolutional kernels (7×7, 5×5, 3×3) to capture the global spectral contour, local spectral variations, and fine-grained spectral details, respectively. After feature concatenation, it is activated by BatchNorm and LeakyReLU, and then key bands are enhanced through channel attention and connected to the original feature residuals. Subsequently, the features are enhanced by two SpeMamba modules. This module explicitly models the local correlation between bands through graph convolution and constructs a learnable band adjacency matrix. After being normalized by Softmax, the data is multiplied by the spectral features, enabling the model to focus on groups of similar bands in the spectrum. On the other hand, the features are flattened into a spectral sequence input to Mamba, implicitly capturing long-range dependencies between bands at arbitrary distances. The graph convolutional output and the Mamba output are concatenated along the channel dimension and then fused through a linear layer, achieving complementarity between local structural priors and global sequence modeling. After channel attention calibration and progressive dimensionality reduction, the features are output as a spectral abundance map via Softmax. .

[0051] This invention introduces pixel-level endmember uncertainty into the spatial-spectral fusion process, in and After being input into the UPMF module, the dual-branch features are dynamically and deeply fused at the original scale and the 1 / 2 downsampled scale. The low-scale features can provide a larger receptive field to perceive the global context, while the high-scale features retain the original resolution to maintain boundary accuracy. The complementary fusion of the two can take into account both global and local details. At each scale, a 1×1 convolution is first performed to... and Projected onto the same latent space dimension, the spatial and spectral features are then flattened into sequences and input into the Mamba module for sequence modeling. The spatial sequence captures global dependencies between pixels along the spatial dimension, while the spectral sequence captures spectral correlations between endmembers along the band dimension. Subsequently, the Mamba-enhanced features are concatenated with two uncertain priors along the channel dimension and input into a gated convolutional network.

[0052]

[0053] in (∙) is the Sigmoid function. , These are the spatial and spectral features enhanced by the Mamba module, respectively. Although the uncertainty in UPMF originates from the VAE's estimation of the endmember posterior distribution, while the spatial-spectral encoder outputs an abundance estimate, the two are not strictly corresponding. However, they are both determined by the same observed pixel and are jointly affected by factors such as mixing complexity, noise interference, and model mismatch. Therefore, when a pixel is in a high-mixing region or is strongly affected by noise, the endmember posterior distribution is usually more dispersed, and the abundance estimate is more prone to ambiguity. Based on this, this invention treats the uncertainty of the VAE estimation as a proxy confidence signal characterizing the difficulty of pixel unmixing and introduces it as a learnable gating mechanism into the Mamba fusion framework to dynamically adjust the fusion weights of spatial and spectral features. The gating network automatically learns the optimal fusion strategy through backpropagation, rather than manually pre-setting fixed rules. The fusion results at the current scale are combined according to the weight ratios.

[0054]

[0055] After the fusion is performed independently at both scales, the low-scale features are upsampled to the original resolution using bilinear interpolation. Subsequently, the high- and low-scale features are concatenated again and then processed by a cross-scale gating network to generate pyramid fusion weights. The specific formula is as follows:

[0056]

[0057] The high-scale fusion result is The low-scale fusion result is upsampled to It adaptively integrates low-scale global semantic information with high-scale local detail features, and finally outputs a fused abundance map:

[0058]

[0059] Meanwhile, the output of the gated network is further mapped to the endmember dimension, generating interpretable weights at the endmember level. It is used to analyze the dependence of different endmembers on spatial and spectral information.

[0060] This invention introduces a DGPR module after the fusion module to measure the fusion abundance. and spectral abundance After inputting into the DGPR module, the abundance estimation of spectral branches is used as a conservative prior. Closed-loop correction is performed to address the above issues, thereby achieving higher accuracy. First, and The data is concatenated along the channel dimension, then passed through three 3×3 convolutions, each followed by batch normalization and ReLU, then a 1×1 convolution to compress the channels to 1, and finally Softplus activation to generate finer uncertainty. This uncertainty quantifies the degree of consistency between the fusion result and the spectral prior, and adaptive initialization fusion is performed based on this:

[0061]

[0062] Subsequently, DGPR gradually corrected the abundance through multi-scale iterative optimization. The network was refined using convolutions with three different dilation rates (1, 2, and 4) to extract multi-scale contextual information, enabling the model to simultaneously perceive local details and global structure. The number of iterations was adaptively adjusted based on global uncertainty; that is, high-uncertainty scenarios required more iterations. In each iteration, the current abundance was... (Right now ) and refined uncertainty The input is concatenated along the channel dimension, and features are extracted in parallel by a multi-scale refinement network. The outputs of the three branches are concatenated along the channel dimension and then fused by a 1x1 convolution. Finally, candidate abundances are generated through Softmax. Update the gating network to , and As input, pixel-wise, uncertainty-guided fusion weights are generated through convolutional layers and sigmoid activation. This leads to an update of the abundance:

[0063]

[0064] After each iteration, abundance constraints are maintained using Softmax. To encourage abundance distribution to concentrate on a few dominant endmembers, a scaling module is introduced. This module adjusts the concentration of abundance distribution through convolution and Softmax, ultimately outputting a refined abundance map. And carry out a second reconstruction:

[0065]

[0066] To prevent the refinement process from degenerating into a weak correction or approximate identity mapping, this invention introduces a refinement loss into the DGPR module. The relationship between initial abundance and refined abundance is regularized.

[0067] Example 2

[0068] The reconstruction loss of this invention uses a combination of mean square error (MSE) and spectral angular distance (SAD), as shown in the following formula:

[0069]

[0070] in and These are the original spectrum and the reconstructed spectrum, respectively. The weight of the SAD loss is preset to 0.5, and the total reconstruction loss is the weighted sum of the first and second reconstructions:

[0071]

[0072] in For the losses of the first reconstruction, For the losses of the second reconstruction, To control the weights of the second reconstruction, the model uses KL divergence constraints on the posterior distribution. With standard Gaussian prior The distances between them are calculated independently of the VAE for spatial and spectral branches:

[0073]

[0074] in Total number of pixels To address the hidden variable dimension and stabilize early training, a KL annealing strategy is employed, linearly increasing the number of hidden variables during the first half of training. The specific formula for constraining the spectral angular distance between each endmember and its global mean is as follows:

[0075]

[0076] in Let be the spectral vector of the p-th endmember in the n-th pixel. To find the mean value of the p-th endmember across all pixels, an endmember volume constraint is introduced to ensure that the endmembers form a compact simplex and avoid excessive divergence.

[0077]

[0078] Introducing self-supervised constraints, including mean constraints and spectral regularization terms:

[0079]

[0080] in The endmember mean. The spectral mean of the image. =0.3, =0.15, the first term constrains the endmembers to approximate the overall image distribution, and the second term regularizes the endmember spectra based on the L1 / 2 norm, prompting the endmember spectra to maintain a reasonable energy concentration. The DGPR thinning loss is as follows:

[0081]

[0082] in: This is the refined abundance vector. For initial fusion abundance, To refine the uncertainty of the previous estimate, To complete the detailed re-estimation of uncertainty, , These are the gradient operators for the horizontal and vertical directions, respectively. =0.3, =0.1 is the weight. In summary, the total loss function of MSSPM-Net is defined as the weighted sum of the above terms:

[0083]

[0084] in , , , , Hyperparameters are used to balance the contributions of each loss term.

[0085] Example 3

[0086] This invention was validated on three public datasets: Samson, Jasper Ridge, and Houston, and compared with six different methods: CyCU, PGMSU, DeepTrans, DFFN, SSAF, and UNMamba.

[0087] Dataset Descriptions: 1) Samson Dataset: This dataset contains 95×95 pixels, with a spectral range of 401-889 nm and a total of 156 spectral bands. It includes three endmembers: soil, trees, and water. Due to its relatively low susceptibility to noise and bad bands, it is often used for preliminary validation of unmixing algorithms. 2) Jasper Ridge Dataset: This dataset was acquired by the AVIRIS sensor. The original image size is 512×614 pixels, with a spectral range of 380-2500 nm and 224 bands. To eliminate the effects of water vapor absorption and atmospheric effects, bands 1-3, 108-112, 154-166, and 220-224 were removed, leaving 198 bands for experimentation. This invention uses 100×100 pixel sub-images for analysis, containing four main land features: roads, soil, water, and vegetation. 3) Houston Dataset: This dataset was acquired by ITRES. The CASI-1500 sensor collected data on the University of Houston campus in 2013. It is the official data of the IEEE GRSS data fusion competition. The original image contains 144 bands with a spectral range of 364-1046 nm. This invention selects a 170×170 pixel sub-region as the experimental area, which mainly includes four endmembers: running track, healthy grass, parking lot 1 and parking lot 2.

[0088] Experimental Setup: 1) Parameter Settings: All experiments in this invention were implemented using the PyTorch framework. Python 3.10.12 was used as the programming language, and PyTorch 2.5.1 was used as the programming framework. The hardware platform consisted of an Intel i5-12500H CPU and an NVIDIA GeForce RTX 3050 GPU with 4GB of video memory. The model training used the AdamW optimizer. The initial learning rate was set according to the dataset: Samson was 0.0018, Jasper Ridge was 0.0015, and Houston was 0.001. The weight decay was set to 1×10⁻⁻⁻⁶. 5 To promote model convergence, a step-down learning rate scheduling strategy (StepLR) was adopted, with a decay factor of 0.95 multiplied every 50 epochs. Samson and Jasper Ridge had a total of 1000 training rounds, while Houston had 500 rounds. Pure MSE was used for reconstruction loss. The training was divided into two stages: in the early stage, reconstruction loss, KL divergence and FCLSU soft label supervision were used

[40] ; in the later stage, endmember volume constraints, spectral angle constraints, self-supervised regularization terms and refinement loss were added to further optimize endmember quality. The hyperparameters of each dataset were determined by grid search, and the configuration is shown in Table 1.

[0089] 2) Evaluation metrics: Root mean square error (RMSE) and spectral angular distance (SAD) are commonly used abundance estimation and endmember spectral evaluation metrics in hyperspectral unmixing, and have been widely applied in related research.

[0090] To evaluate unmixing performance, this invention uses RMSE, specifically the overall average RMSE (aRMSE), to assess the accuracy of the abundance map. For the p-th endmember, the calculation formula is as follows:

[0091]

[0092] in, and Let Sp be the estimated abundance and true abundance of the p-th endmember in the n-th pixel, respectively. SA and the overall average SA (aSAD) are used as metrics to evaluate the accuracy of endmember extraction. The calculation formulas are as follows:

[0093]

[0094]

[0095] in and These are the estimated spectrum and the true spectrum of the p-th endmember, respectively.

[0096] Table 1. Hyperparameter settings for different datasets

[0097]

[0098] Experiments were conducted on the Samson dataset. Table 2 shows the quantitative results of various comparative methods on the Samson dataset. The method of this invention achieved the best results in both average RMSE and SAD. UNMamba and DFFN achieved the second-best results in average RMSE and average SAD, respectively. DFFN benefits from its feature fusion module effectively enhancing the original image through spectral and spatial similarity matrices, but its endmember learning process is relatively decoupled from abundance estimation, leading to performance degradation on some land cover types. UNMamba performs well on specific endmembers, which is attributed to its cascaded spatial... The global receptive field introduced by the spectral Mamba structure can effectively capture the continuous distribution characteristics of ground objects, but its ability to distinguish spectrally similar ground objects is limited. CyCU-Net exhibits large performance fluctuations across endmembers. While the cycle consistency constraint is effective for soil types, it lacks explicit modeling of endmember variability. PGMSU has strong probabilistic generation capabilities and certain advantages in endmember distribution modeling, but due to the lack of spatial neighborhood constraints, the resulting abundance map is prone to insufficient spatial continuity. SSAF-Net performs poorly in aquatic environments, but its proportional perturbation model still performs well in soil and tree endmembers. Figure 2 and Figure 3 The visualization results of the abundance plots and endmember spectra of each method on Samson are shown. It can be seen that MSSPM can recover the spectral composition more accurately while maintaining the clarity of spatial boundaries. This result is due to the effective suppression of unreliable features by the fusion module and the refinement module.

[0099] Table 2. Comparison of unmixing performance on the SAMSON dataset

[0100]

[0101] Experiments were conducted on the Jasper Ridge dataset. Table 3 shows the quantitative results of each method on the Jasper Ridge dataset. The data shows that the method proposed in this invention achieves the best results in both average RMSE and average SAD, especially with a significant reduction in RMSE. This indicates that the fusion strategy of this model and subsequent iterative optimization effectively alleviate the misdecomposition problem in complex regions. Figure 4 and Figure 5 From the visualization results, compared with other methods, this method has a clear boundary outline on the abundance map and better endmember differentiation results. Compared with SSAF, PGMSU and other models that also use endmember variability modeling, these models are worse in water body processing, and non-water body areas are incorrectly assigned higher abundance values. However, the method of this invention performs evenly on each endmember and performs well in water body endmembers. This shows that the fusion strategy of this invention does indeed play a role in improving accuracy. The UNMamba model, which is also a Mamba framework, also performs well, which shows the potential of the Mamba framework in unmixing.

[0102] Table 3. Comparison of unmixing performance on the Jasper Ridge dataset

[0103]

[0104] The experiments conducted on the Houston dataset are shown in Table 4, which presents the quantitative results of each comparative method on the Houston dataset. MSSPM-Net still achieves the best overall performance on this dataset, indicating that the method has good generalization ability in complex real-world scenarios. Figure 6 and Figure 7The visualization comparison results of the estimated abundance map and the extracted endmember spectra are shown. The abundance map generated by MSSPM is in high agreement with the reference map, clearly showing the boundaries of material distribution. DeepTrans has difficulty capturing the subtle spectral differences between parking lot 1 and parking lot 2, while UNMamba and CyCU-Net failed to extract the RunningTrack, mixing it into the Grass Healthy endmember. This may be because the model is not obvious about some subtle spectral changes and cannot distinguish similar spectra. The PGMSU and SSAF models still have excellent performance, which highlights the advantages of endmember variability modeling and confirms the rationality of this method. The method of this invention is based on this kind of endmember variability modeling and introduces UPMF and DGPR modules to further enhance the ability to correct local mixed regions.

[0105] Table 4. Comparison of unmixing performance on the Houston dataset

[0106]

[0107] To verify the effectiveness of each core component in MSSPM-Net, this invention utilizes Jasper... Systematic ablation experiments were conducted on the Ridge dataset, with five consecutive independent experiments for each configuration. The baseline model used the same dual-branch spatial-spectral encoder architecture but did not include the Mamba enhancement module, UPMF fusion module, and DGPR refinement module. Subsequently, the Mamba, UPMF, and DGPR modules from the dual branches were gradually introduced, as shown in Table 5. After adding the Mamba module, the RMSE and SAD values ​​decreased, demonstrating the significant effectiveness of the Mamba module in feature processing. However, adding only the DGPR module resulted in a deterioration in performance. But when the Mamba or UPMF module was added to the DGPR module, the model performance improved, verifying the refinement post-processing capability of this module and indicating that the DGPR module requires good feature input as a foundation, which is consistent with the module's positioning. The UPMF module performed well, both when used alone and in combination with other modules, demonstrating the effectiveness of the fusion method. When all modules were used simultaneously, the model achieved optimal performance, which verifies the synergistic effect among the Mamba, UPMF, and DGPR modules.

[0108] Table 5. Ablation experiments on the Jasper Ridge dataset

[0109]

[0110] It should be noted that, in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0111] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A hyperspectral unmixing method for multi-scale spatial-spectral pyramid Mamba fusion, characterized in that, Includes the following steps: S1: Acquire raw hyperspectral image data, construct a dual-branch multi-scale feature encoder, extract multi-scale spatial features through the spatial encoder, and perform spatial global dependency modeling using the SpaMamba module to output the spatial abundance map A. spa Multi-scale spectral features are extracted using a spectral encoder and then modeled for long-range spectral dependence using the SpeMamba module, outputting spectral abundance map A. spe ; Pixel-level endmember uncertainty U is estimated using a variational autoencoder spa and U spe ; S2: The spatial abundance map A spa Spectral abundance diagram A spe and uncertainty U spa U spe The uncertainty-aware pyramid Mamba fusion module is used to input spatial and spectral features into a multi-scale pyramid structure. These features are flattened into sequences and input into the Mamba module for sequence modeling. The Mamba-enhanced features are then concatenated with the uncertainty prior along the channel dimension and input into a gated convolutional network to generate fusion weights. Based on these weights, the spatial and spectral features are dynamically and deeply fused, outputting a fusion abundance map A. fused ; S3: The fusion abundance map A fused and the spectral abundance diagram A spe The input-decoupled, progressive abundance refinement module uses the spectral abundance map as a conservative prior and performs closed-loop correction on the fused abundance through multi-scale iterative optimization, outputting a refined abundance map A. final ; S4: Based on the refined abundance map A final The spectral reconstruction is performed using the endmember matrix E to obtain the reconstructed hyperspectral image.

2. The hyperspectral unmixing method for multi-scale spatial-spectral pyramid Mamba fusion according to claim 1, characterized in that: The spatial encoder adopts a multi-scale resolution pyramid structure, extracting spatial features in three parallel branches at 1 / 4, 1 / 2 and original resolutions. After upsampling and alignment, the features are spliced ​​and fused, and then fused through a 1×1 convolutional channel with residual connections added. Finally, spatial attention is used to enhance key spatial locations.

3. The hyperspectral unmixing method for multi-scale spatial-spectral pyramid Mamba fusion according to claim 2, characterized in that: The SpaMamba module generates a spatial attention weight map to weight the features. The weighted feature map is then encoded with learnable positional codes, flattened into a sequence, and input into Mamba for global dependency modeling. The output is then fused with the original features via residual connections.

4. The hyperspectral demixing method for multi-scale spatial-spectral pyramid Mamba fusion according to claim 3, characterized in that: The variational autoencoder takes the pixel spectrum of the original hyperspectral image as input and outputs the mean μ of the endmember distribution through two independent encoders. s μ d And log-variance logσ² s ,logσ² d After reparameterization, the decoder generates the transformation matrix ψ∈ and perturbation matrix ΔM∈ Endmembers are generated using the formula E=Sigmoid(M0ψ+ΔM), where M0 is the basic endmember matrix initialized by VCA, P is the number of endmembers, and L is the number of bands; the pixel-level endmember uncertainty is determined by the logarithmic variance using the formula Calculated.

5. The hyperspectral unmixing method for multi-scale spatial-spectral pyramid Mamba fusion according to claim 4, characterized in that: The input to the spectral encoder is the original hyperspectral image and the first reconstruction result, which are adaptively weighted and fused using learnable weights. The spectral encoder uses three parallel convolutional kernels to capture the global spectral contour, local spectral variations, and fine-grained spectral details, respectively. After feature concatenation, the features are batch normalized and activated by LeakyReLU. Then, key bands are enhanced by channel attention and connected to the original feature residuals.

6. The hyperspectral demixing method for multi-scale spatial-spectral pyramid Mamba fusion according to claim 5, characterized in that: The SpeMamba module explicitly models the local correlations between bands through graph convolution, constructs a learnable band adjacency matrix, normalizes it with Softmax, and multiplies it with spectral features. At the same time, the features are flattened into spectral sequences and input into Mamba to capture long-range dependencies between bands. The graph convolution output and the Mamba output are concatenated in the channel dimension and then fused through a linear layer.

7. The hyperspectral unmixing method for multi-scale spatial-spectral pyramid Mamba fusion according to claim 6, characterized in that: The uncertainty-aware pyramid Mamba fusion module performs dynamic deep fusion at both the original scale and the 1 / 2 downsampling scale. At each scale, spatial abundance maps and spectral abundance maps are projected to the same latent space dimension through 1×1 convolutions. After being flattened into sequences, these sequences are input into the Mamba module. The Mamba-enhanced features are then concatenated with the uncertainty prior and input into a gated convolutional network to generate a gated signal G=σ(Conv). 1×1 ([M spa M spe U spa U spe The fusion results are categorized by F. fused =M spa ⊙G+M spe ⊙(1-G) combination; the fusion results of the two scales are fused into a pyramid through a cross-scale gating network to output the final fused abundance map.

8. The hyperspectral unmixing method for multi-scale spatial-spectral pyramid Mamba fusion according to claim 7, characterized in that: The decoupled-guided progressive abundance refinement module stitches the fused abundance map and spectral abundance map along the channel dimension, and generates the refinement uncertainty U through three layers of 3×3 convolutions and one layer of 1×1 convolution. ref Based on this uncertainty, adaptive initialization fusion A is performed. init =Softmax(σ(U ref )⊙A spe +(1-σ(U ref ))⊙A fused Subsequently, iterative optimization was performed using a multi-scale thinning network with three different dilation rates (1, 2, and 4). In each iteration, the current abundance and thinning uncertainty were concatenated and input into the thinning network. After fusion by 1×1 convolution, candidate abundances were generated through Softmax. The abundances were then updated by generating fusion weights through an update gate network until an adaptively determined number of iterations was reached.

9. The hyperspectral unmixing method for multi-scale spatial-spectral pyramid Mamba fusion according to claim 8, characterized in that: The total loss function used during training is: Where L rec To reconstruct the loss, a combination of mean square error and spectral angular distance is used; L kl For KL divergence loss; L vol For endmember volume constraint loss; L sad For endmember spectral angular distance constraint loss; L self For self-supervised constraint loss; L ref To refine the loss; λ kl , λ vol , λ sad , λ self , λ ref These are the corresponding weighting coefficients.

10. A hyperspectral demixing system for multi-scale spatial-spectral pyramid Mamba fusion according to any one of claims 1-9, characterized in that, include: A dual-branch multi-scale feature encoder is used to extract spatial abundance maps, spectral abundance maps, and pixel-level endmember uncertainties. The uncertainty-aware pyramid Mamba fusion module is used for adaptive deep fusion of spatial and spectral features; the decoupled guided progressive abundance refinement module is used for iterative optimization and correction of the fused abundance; and the spectral reconstruction module is used for spectral reconstruction based on the refined abundance map and endmember matrix.

Images