Hierarchical superpixel multi-expert mamba network construction method for hyperspectral image classification

Abstract

The application relates to a hierarchical superpixel multi-expert Mamba network construction method for hyperspectral image classification, and particularly relates to a hierarchical superpixel multi-expert Mamba network construction method for hyperspectral image classification. The application is used for the field of hyperspectral image classification, and aims at solving the problems of insufficient modeling of long-distance space spectrum dependence, insufficient description of local fine-grained changes and difficulty in effective cooperation of hierarchical multi-scale features in the prior art. The application aims at the significant differences in expression ability and semantic emphasis of different scale features, and aims at the problems of insufficient modeling of long-distance space spectrum dependence and easy neglect of local boundary and fine-grained change information in the hyperspectral image. The process is as follows: a hierarchical superpixel multi-expert Mamba network model for hyperspectral image classification is constructed; the model sequentially comprises an LHSM, an HSMoE, an FBDMamba and a classification layer; a to-be-measured hyperspectral image is classified, and a classification result is output.

Description

Technical Field

[0001] This invention is applicable to the field of hyperspectral image classification, specifically relating to a method for constructing hierarchical superpixel multi-expert Mamba networks for hyperspectral image classification. Background Technology

[0002] Hyperspectral images (HSI) possess rich spectral resolution and can acquire ground feature information across continuous and dense spectral bands, making them valuable for applications in land cover mapping, precision agriculture, environmental monitoring, and urban analysis. Compared to traditional RGB images, hyperspectral images not only contain more detailed spectral response information but also complex spatial structural features, providing a more robust data foundation for refined ground feature identification. However, hyperspectral image classification still faces numerous challenges. On one hand, different categories often exhibit strong spectral similarity, while within the same category, factors such as illumination variations, shadow interference, and background mixing can lead to significant intra-class differences. On the other hand, complex scenes commonly suffer from blurred boundaries, dense distribution of small targets, and significant regional transitions, further increasing the difficulty of hyperspectral classification tasks.

[0003] In recent years, with the rapid development of deep learning technology, significant progress has been made in hyperspectral image classification. Compared with traditional machine learning methods, deep learning methods can automatically learn more discriminative spatial-spectral features through end-to-end training, and therefore have gradually become the mainstream research direction in this field. Based on different network architectures, existing deep learning methods can be mainly divided into methods based on convolutional neural networks (CNNs), graph convolutional networks (GCNs), and Transformers. These models, from different perspectives such as local spatial structure modeling, graph relation modeling, and global context modeling, continuously improve the performance of hyperspectral image classification and promote the continuous development of related research.

[0004] Building upon this foundation, researchers have proposed a series of representative hyperspectral image classification models. Hong et al. proposed SpectralFormer, which learns richer local spectral representations through a spectral grouping embedding mechanism, thereby enhancing spectral feature modeling capabilities. Li et al. proposed CLONN, a model that constructs a lightweight network structure oriented towards the channel dimension to reduce the parameter scale and computational complexity of CNNs. Roy et al. proposed morphFormer, which constructs a spectral-space-shape attention Transformer by jointly modeling spectral, spatial, and morphological information to improve classification accuracy and robustness. Sun et al. proposed the spectral-space feature tokenization Transformer, or SSFTT, which uses a Gaussian weighted tokenization mechanism to transform shallow spatial-spectral features into sequential semantic representations, thereby enhancing feature extraction capabilities through sequence modeling. Zhao et al. proposed GSC-ViT, a model that combines grouped separable convolutions with the Vision Transformer, effectively achieving joint modeling of local details and global spatial spectral information.

[0005] While the aforementioned CNN, Transformer, and GCN methods have achieved good results in hyperspectral image classification, they still have certain structural limitations. Specifically, CNNs are limited by their local receptive fields, making it difficult to fully model long-distance spatial dependencies, thus limiting their ability to express global semantic information. Although Transformers have advantages in global context modeling, the computational complexity of their self-attention mechanism increases quadratically with sequence length, often resulting in high computational and storage overhead on high-dimensional hyperspectral data, particularly unfavorable for pixel-level fine-grained modeling. Meanwhile, GCNs also face several challenges in hyperspectral image classification: their performance is highly dependent on the quality of the graph structure construction, and manually defined adjacency relationships often fail to adequately characterize complex spatial-spectral correlations; furthermore, GCNs mainly rely on local neighborhood aggregation, making it difficult to effectively capture long-distance dependency information. Therefore, it is urgent to explore a new hyperspectral image classification framework that maintains high computational efficiency while also possessing global dependency modeling capabilities, to better adapt to the representation learning needs of high-dimensional spatial-spectral data.

[0006] In recent years, the Selective State Space Model (SSM), or Mamba, has been introduced into the field of remote sensing, demonstrating its ability to model long-range dependencies with linear complexity. This characteristic significantly reduces the computational burden when extracting global spatial information, making the modeling of complex spatial features more efficient and accurate. Li et al. proposed the MambaHSI model. MambaHSI is the first image-level hyperspectral image classification model based on the Mamba architecture, capable of efficiently modeling long-range dependencies across the entire image with linear complexity and adaptively fusing spatial and spectral features, thereby significantly improving classification performance. Pan et al. proposed the mambaLG model. MambaLG is a two-branch hyperspectral image classification method that combines local and global spatial modeling with spectral awareness, aiming to improve global modeling capabilities while preserving local details, thereby achieving higher classification accuracy and efficiency. Wang et al. proposed DBMGNet, which effectively improves the representation ability of spatial-spectral features in hyperspectral images by constructing a two-branch Mamba-GCN hybrid framework.

[0007] Despite the advantages mentioned above, Mamba still presents some noteworthy issues when applied to HSI classification tasks:

[0008] Hyperspectral images possess complex spatial structures and strong spectral correlations, accompanied by significant intra-class discriminative and class transition phenomena. Existing methods often suffer from an imbalance in their focus during modeling: convolutional models, while possessing good local modeling capabilities, are limited by their finite receptive field, making it difficult to effectively capture long-range spatial-spectral dependencies; while sequence-based modeling strategies, although enhancing global context awareness, typically rely on a single information propagation path, failing to adequately characterize fine-grained changes in boundary regions, small-scale targets, and class transition regions. Therefore, in complex and heterogeneous scenes, how to achieve collaborative modeling of long-range semantic dependencies and local discriminative details within a unified framework remains one of the core problems that urgently need to be solved in hyperspectral image classification.

[0009] Hierarchical superpixels can provide effective multi-scale structural priors for hyperspectral image classification, helping to alleviate pixel-level redundancy and enhance region consistency modeling. However, existing methods typically rely on fixed cross-layer mapping relationships and shared feature transformation methods, making it difficult to adapt to significant differences between features at different scales. Specifically, static mappings cannot adaptively adjust to changes in feature distribution, easily causing cross-layer information transmission biases in complex scenes; simultaneously, features at different scales have significant differences in their modeling focus, and a uniform transformation strategy cannot fully characterize their scale specificity, easily leading to multi-scale information aliasing and weakening discriminative ability. Therefore, how to achieve adaptive optimization of cross-layer mappings while preserving hierarchical structural priors and perform differentiated modeling of multi-scale features remains a key challenge in hyperspectral hierarchical representation learning. Summary of the Invention

[0010] The purpose of this invention is to address the problems faced by existing hyperspectral image classification methods, such as insufficient modeling of long-range spatial spectral dependencies, inadequate characterization of local fine-grained changes, and difficulty in effectively coordinating hierarchical multi-scale features. Existing methods also address the significant differences in expressive power and semantic emphasis of features at different scales, as well as the inadequacy of existing modeling of long-range spatial spectral dependencies in hyperspectral images and the easy neglect of local boundary and fine-grained change information. Therefore, this invention proposes a hierarchical superpixel multi-expert Mamba network construction method for hyperspectral image classification.

[0011] The specific process of constructing a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification is as follows:

[0012] Step 1: Obtain the raw hyperspectral image;

[0013] Step 2: Construct the HSMMamba network model;

[0014] The HSMMamba network model represents a hierarchical superpixel multi-expert Mamba network model used for hyperspectral image classification.

[0015] The HSMMamba network model consists of LHSM, HSMoE, FBDMamba, and a classification layer.

[0016] LHSM is a learnable hierarchical superpixel mapping module;

[0017] HSMoE is a hierarchical superpixel multi-expert hybrid module;

[0018] FBDMamba is a forward-backward-differential Mamba module;

[0019] The working process of the HSMMamba network model is as follows:

[0020] The original hyperspectral image is sequentially input into LHSM, HSMoE, FBDMamba and the classification layer, and the classification layer outputs the classification result.

[0021] Step 3: Classify the original hyperspectral image to be tested based on the trained HSMMamba network model and output the classification results.

[0022] The beneficial effects of this invention are as follows:

[0023] To address the challenges of hyperspectral image classification, such as insufficient long-range spatial-spectral dependency modeling, inadequate characterization of local fine-grained variations, and difficulty in effectively coordinating hierarchical multi-scale features, this invention proposes a hierarchical superpixel multi-expert Mamba network (HSMMamba) for hyperspectral image classification. This network addresses these challenges through three aspects: hierarchical structure prior modeling, multi-scale differential feature enhancement, and global-local joint representation. The aim is to achieve adaptive optimization of cross-layer mapping relationships, selective enhancement of multi-scale features, and collaborative modeling of long-range dependencies and local detail information within a unified framework, thereby improving spatial-spectral representation capabilities in complex scenes. The main contributions of this invention are summarized as follows: To address the limitations of traditional hierarchical superpixel methods, which rely on fixed cross-layer mapping relationships and struggle to adapt to complex terrain structures, this invention proposes a learnable hierarchical superpixel mapping mechanism. While preserving hierarchical structure priors, learnable residuals are introduced to adaptively optimize cross-layer mapping, enabling dynamic adjustment of the multi-scale feature transfer process and improving the consistency and accuracy between regional representation and pixel-level reconstruction.

[0024] To address the significant differences in expressive power and semantic emphasis among features at different scales, this invention designs a hierarchical scale-aware multi-expert hybrid enhancement mechanism. By constructing a multi-branch expert network targeting pixel-level, superpixel-level, and larger-scale regions, it performs targeted modeling of fine-grained boundary information, mesoscale structural transitions, and large-scale regional consistency, thereby enhancing the model's ability to characterize complex multi-scale spatial structures and further improving classification performance and robustness.

[0025] To address the issues of insufficient modeling of long-range spatial-spectral dependencies and the easy neglect of local boundary and fine-grained variation information in hyperspectral images, this invention proposes a forward-backward-difference joint modeling FBD-Mamba module. This module constructs three collaborative branches—forward, backward, and central difference—to enhance global context modeling capabilities while effectively characterizing local variations and boundary structures, thereby significantly improving the discriminativeness and stability of feature representations in complex scenes.

[0026] This invention constructs a unified HSMMamba hyperspectral image classification framework, organically integrating hierarchical superpixel modeling, multi-scale expert enhancement, and forward-backward-differential sequence modeling. This framework can collaboratively mine multi-scale spatial structure information and long-range spectral dependencies under a unified structure, thereby improving feature representation ability and discrimination performance overall. Extensive experimental results on several representative hyperspectral datasets (Pavia University, Salinas, and HongHu) demonstrate that the proposed HSMMamba significantly outperforms existing mainstream methods in terms of classification accuracy, model robustness, and cross-scene generalization ability.

[0027] Hyperspectral image classification aims to fully exploit spatial and spectral information to achieve precise identification of complex ground features. However, existing methods still face several challenges in hyperspectral image classification: on the one hand, traditional convolutional networks struggle to effectively model long-range spatial-spectral dependencies, while self-attention-based models typically involve high computational complexity; on the other hand, while hierarchical superpixels can provide effective multi-scale structural priors, existing methods mostly rely on fixed cross-layer mapping relationships and shared feature transformation strategies, making it difficult to adapt to the significant heterogeneity between features at different scales. To address this, this invention proposes a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification, termed HSMMamba. First, a learnable hierarchical superpixel mapping module is designed. By introducing learnable residuals based on fixed hierarchical superpixel mapping priors, adaptive optimization of cross-layer upsampling and downsampling relationships is achieved, thereby enhancing the consistency of multi-level feature propagation and reconstruction. Second, a hierarchical superpixel multi-expert hybrid module is proposed. This module performs expert-level modeling of pixel-level and superpixel-level features at different scales and achieves adaptive fusion through a gating mechanism to improve the expressive power of multi-level heterogeneous features. Furthermore, to address Mamba's limitations in modeling local details, a forward-backward-difference Mamba module was constructed. By jointly modeling long-range dependencies on local variations, this enhances the ability to characterize boundary details and class transition regions. Experimental results on three representative hyperspectral datasets (Pavia University, Salinas, and HongHu) show that the overall accuracies of the proposed HSMMamba are 98.98%, 99.3%, and 95.65%, respectively. Compared with existing models based on Mamba, CNN, GCN, and Transformer, HSMMamba achieves a maximum accuracy improvement of 7.88%, demonstrating superior generalization and robustness, especially in small-sample scenarios. Attached Figure Description

[0028] Figure 1 This is a block diagram of the overall structure of the hierarchical superpixel multi-expert Mamba network (HSMMamba) for hyperspectral image classification according to the present invention.

[0029] Figure 2 The images in the PU dataset are real images and differential effect maps. (a) Real images, (b) Differential effect maps.

[0030] Figure 3 Here is a block diagram of the FBDMamba structure;

[0031] Figure 4 An adaptive weight graph for different level branches corresponding to the three datasets;

[0032] Figure 5 A visualization of the difference between Mamba with and without differences;

[0033] Figure 6 The classification graphs obtained by all methods on the PU dataset are: (a) ground truth graph, (b) spectralFormer, (c) CLOLN, (d) DBMGNet, (e) morphFormer, (f) SSFTT, (g) GSC-Vit, (h) mambaLG, (i) MambaHSI, and (j) PS2Mamba proposed in this invention.

[0034] Figure 7 The classification graphs obtained by all methods on the SA dataset are: (a) ground truth graph, (b) spectralFormer, (c) CLOLN, (d) DBMGNet, (e) morphFormer, (f) SSFTT, (g) GSC-Vit, (h) mambaLG, (i) MambaHSI, and (j) PS2Mamba proposed in this invention.

[0035] Figure 8 The classification graphs obtained by all methods on the HongHu dataset are: (a) ground truth graph, (b) spectralFormer, (c) CLOLN, (d) DBMGNet, (e) morphFormer, (f) SSFTT, (g) GSC-Vit, (h) mambaLG, (i) MambaHSI, and (j) PS2Mamba proposed in this invention. Detailed Implementation

[0036] Specific Implementation Method 1: The specific process of constructing a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification in this implementation method is as follows:

[0037] Step 1: Obtain the raw hyperspectral image;

[0038] Step 2: Construct the HSMMamba network model;

[0039] The HSMMamba network model represents a hierarchical superpixel multi-expert Mamba network model used for hyperspectral image classification.

[0040] The HSMMamba network model consists of LHSM, HSMoE, FBDMamba, and a classification layer.

[0041] LHSM stands for Learnable Hierarchical Superpixel Mapping.

[0042] HSMoE stands for Hierarchical Superpixel Mixture-of-Experts.

[0043] FBDMamba is a Forward-Backward-Differential Mamba module.

[0044] The working process of the HSMMamba network model is as follows:

[0045] The original hyperspectral image is sequentially input into LHSM, HSMoE, FBDMamba and the classification layer, and the classification layer outputs the classification result.

[0046] Step 3: Classify the original hyperspectral image to be tested based on the trained HSMMamba network model and output the classification results.

[0047] The STEAM module maps the original hyperspectral image into a low-level feature representation, thereby reducing redundant information while unifying feature dimensions. Based on this low-level feature, HSMMamba further enhances the spatial-spectral representation capability through a hierarchical feature modeling process. Specifically, the model first introduces a learnable hierarchical superpixel mapping module, which adaptively corrects the cross-layer upsampling relationship while preserving the prior hierarchical superpixel structure. This achieves accurate alignment and stable transfer between pixel-level features and representations of different hierarchical regions, and improves the consistency of multi-scale information interaction. Subsequently, a hierarchical scale-aware multi-expert hybrid enhancement module performs differentiated modeling of features at different scales. By assigning dedicated expert branches to pixel-level, fine-grained superpixels, and larger-scale regions, it strengthens local details, regional consistency, and contextual structure information, thereby mitigating the discriminative power degradation caused by the aliasing of heterogeneous features at multiple scales. Finally, the FBD-Mamba module collaboratively models through forward, backward, and differential branches, effectively capturing long-range spatial-spectral dependencies while highlighting local boundaries, fine-grained changes, and class transition regions, further improving the discriminative power of feature representations. By organically combining learnable hierarchical mapping, multi-expert enhancement, and FBD-Mamba, HSMMamba can simultaneously model global semantic consistency and local fine structure within a unified framework, providing more robust feature representations for hyperspectral image classification in complex scenes. Finally, the model integrates enhanced features from different levels through an adaptive fusion strategy to complete classification prediction. The overall block diagram is shown below. Figure 1 As shown.

[0048] This invention addresses the challenges of losing local fine-grained variation information, the difficulty in adaptively adapting multi-scale representations, and the ineffective modeling of long-range spatial-spectral dependencies in hyperspectral image classification. It proposes a hyperspectral classification framework, HSMMamba, which utilizes hierarchical superpixels and Mamba for collaborative modeling. This method systematically improves traditional methods in three aspects: global dependency modeling, hierarchical feature propagation, and multi-scale discriminative enhancement. Specifically, the learnable hierarchical superpixel mapping mechanism, while preserving prior hierarchical superpixel structures, introduces learnable residuals to adaptively optimize cross-layer mapping relationships, thereby improving the accuracy and consistency of multi-level feature propagation and effectively enhancing the collaborative ability between pixel-level and region-level representations. Based on this, the hierarchical scale-aware multi-expert hybrid enhancement mechanism assigns dedicated expert branches to pixel-level, fine-grained superpixels, and larger-scale superpixels to address the heterogeneity of features at different scales, achieving targeted enhancement of coarse-grained regional consistency, mesoscale structural transitions, and fine-grained boundary details. Finally, the FBD-Mamba module uses forward, backward, and differential branches for joint modeling. While maintaining global context awareness, it further highlights the discriminative features of local changes, boundary details, and category transition regions. The synergistic effect of these three branches enables HSMMamba to fully extract multi-scale spatial structure and spectral discriminative information from hyperspectral images within a unified framework.

[0049] Extensive experimental results demonstrate that HSMMamba outperforms existing mainstream methods in overall classification accuracy, average accuracy, Kappa coefficient, and visualization classification performance, exhibiting stronger stability and robustness, particularly in complex scenes, boundary regions, and areas with high class similarity. Ablation experiments further validate the effectiveness of each key module, indicating that the learnable hierarchical superpixel mapping, hierarchical scale-aware multi-expert mechanism, and FBD-Mamba are not simply superimposed, but rather effectively complement each other in feature representation, information transmission, and classification, jointly improving model performance. Overall, HSMMamba provides an effective solution for hyperspectral image classification that balances global dependencies, local details, and multi-scale structural modeling, and also offers new insights for refined ground feature identification in complex remote sensing scenarios. Future work will further explore the framework's generalization capabilities on larger datasets and its application potential in real-time inference and lightweight deployment.

[0050] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the learnable hierarchical superpixel mapping module LHSM includes a stem module, a first linear layer, a second Softmax activation function layer, a third linear layer, a fourth Softmax activation function layer, a fifth linear layer, a sixth average pooling layer (AvgPool), a seventh RMSNorm layer, an eighth linear layer, a ninth average pooling layer, a tenth RMSNorm layer, an eleventh linear layer, a twelfth average pooling layer, and a thirteenth RMSNorm layer.

[0051] The RMSNorm layer is a root mean square layer normalization;

[0052] The stem module consists of the fourteenth batch normalized BN layer, the fifteenth 1×1 convolutional layer, and the sixteenth LeakyReLU activation function layer.

[0053] The other steps and parameters are the same as in Specific Implementation Method 1.

[0054] Specific Implementation Method 3: This implementation method differs from Specific Implementation Method 1 or 2 in that the forward-backward-differential Mamba module FBDMamba includes a forward selective state space model, a backward selective state space model, and a differential selective state space model.

[0055] Other steps and parameters are the same as in specific implementation method one or two.

[0056] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that the classification layer includes an average pooling layer, a fully connected layer (FC), and a multilayer perceptron (MLP).

[0057] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0058] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that the working process of the HSMMamba network model is as follows:

[0059] Step 2: Input the original hyperspectral image into the Learnable Hierarchical Superpixel Mapping (LHSM) module. The LHSM module outputs a feature map. Feature map and feature map ;

[0060] Step 22: Output feature maps from the Learnable Hierarchical Superpixel Mapping (LHSM) module. Feature map and feature map Input Hierarchical Superpixel Multi-Expert Hybridization Module (HSMoE), Output Features of Hierarchical Superpixel Multi-Expert Hybridization Module (HSMoE) The loss function of the hierarchical superpixel multi-expert hybrid module HSMoE is constructed. ;

[0061] Steps 2 and 3: Output features from the hierarchical superpixel multi-expert hybrid module HSMoE Input forward-backward-differential Mamba module FBDMamba, output features of the forward-backward-differential Mamba module FBDMamba ;

[0062] Step 24: The features from Step 22... Features of steps two and three Input the classification layer, and the classification layer outputs the classification result.

[0063] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0064] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that, in step Two-One, the original hyperspectral image is input to a Learnable Hierarchical Superpixel Mapping (LHSM) module, and the LHSM module outputs a feature map. Feature map and feature map ;

[0065] The specific process is as follows:

[0066] Step 2: Input the original hyperspectral image into the STEM module; the STEM module outputs a feature map. Based on feature maps Obtain feature map ;

[0067] The specific process is as follows:

[0068] 1) The original hyperspectral image is sequentially input into the fourteenth batch normalized BN layer, the fifteenth 1×1 convolutional layer, and the sixteenth LeakyReLU activation function layer. The sixteenth LeakyReLU activation function layer outputs the feature map. ;

[0069] ,in Represents the set of real numbers. and These represent the spatial dimensions. Indicates feature dimension;

[0070] 2) Feature map Perform hierarchical division and construct Layered superpixel structure;

[0071] For the Layers, pixels The superpixel number to which it belongs is denoted as , One pixel corresponds to multiple superpixels;

[0072] in, Indicates the first Total number of superpixels in the layer; ;

[0073] Different levels correspond to different granularity of region division, thus forming a hierarchical expression from fine to coarse.

[0074] 3) Perform feature aggregation for each superpixel; specifically:

[0075] No. Layer The region feature of a superpixel is defined as:

[0076] (1)

[0077] in, Indicates the first Layer The set of pixels contained in a superpixel ;

[0078] express The number of elements contained;

[0079] Indicates the 0th layer. Features of each pixel; Indicates the first Layer Region features of one superpixel;

[0080] Equation (1) achieves the mapping from pixel-level features to superpixel-level features through regional average aggregation, enabling the model to enhance the consistent representation within the region while preserving local spectral details.

[0081] 4) Unlike methods that directly perform cross-layer transfer using a fixed mapping matrix, LHSM introduces a learnable fully connected transformation and normalization mechanism during the pixel-to-superpixel mapping process to adapt to the constantly changing structural distribution in complex scenarios, such as... Figure 2 As shown;

[0082] First, construct the pixel to the... Initial mapping matrix of layer superpixels ; indicates as:

[0083] (2)

[0084] in, This is a learnable residual term used to adaptively adjust the cross-layer mapping weights; The first term represents the learnable residual. Layer superpixels; The first term represents the learnable residual. Layer superpixels;

[0085] Subsequently, the mapping matrix Applying Softmax normalization row by row yields the Softmax-normalized superpixel matrix, represented as:

[0086] (3)

[0087] in, express The OK;

[0088] Ensure that the weight distribution of each pixel when projecting onto a superpixel satisfies the probability constraint;

[0089] Therefore, pixel-level features can be weighted and aggregated and projected into the superpixel space, resulting in a Softmax-normalized superpixel matrix. With feature map Multiply, then sequentially input into a fully connected layer and a non-linear activation function layer to obtain the first... Feature maps of layer superpixels ; indicates as:

[0090] (4)

[0091] in, and These represent the weight matrix and bias term of the fully connected layer, respectively. Represents a nonlinear activation function;

[0092] This design enables cross-layer mapping to move beyond static aggregation and dynamically adjust the representation of region features according to the task objective during training, thereby enhancing the discriminativeness and adaptability of region representation.

[0093] To achieve the reverse complementation of pixel-level representation by region information, LHSM retransmits the transformed superpixel features back to the pixel location; for any pixel , pixels In the The mapping feature corresponding to the layer is defined as follows:

[0094] (5)

[0095] in, Represents pixels The superpixel number to which it belongs. Indicates the first Layer superpixel Features; Indicates the first Layer pixels Features;

[0096] Based on the Layer pixels Features Get the first Layer feature map ;

[0097] pixels It inherits the enhanced representation of its superpixel region. Through this process, pixel-level features can establish stable connections with the semantics of regions at different levels, thereby preserving both local details and structural context information.

[0098] 5) Feature map With feature maps of each layer The feature maps are then fused to obtain the final feature map. ; indicates as:

[0099] (6)

[0100] in, Indicates to After channel concatenation, the input is given to the linear layer, and the feature map is output by the linear layer.

[0101] As shown in equations (1)–(6), LHSM achieves adaptive optimization of cross-scale feature relationships by adopting a hierarchical modeling approach of “superpixel partitioning—region aggregation—learnable mapping—pixel backpropagation—multi-layer fusion” while preserving hierarchical superpixel priors. This module not only enhances the consistency within regions and the accuracy of cross-layer information transmission, but also provides a more discriminative basic representation for subsequent multi-expert augmentation and sequence modeling.

[0102] Step 2.12: From the feature map Extracting consecutive pixel blocks as feature maps ;

[0103] Steps 2-13: Transfer the feature map The input is sequentially fed into a first linear layer and a second softmax activation function layer. The second softmax activation function layer outputs a feature map. ;

[0104] Step 214: Transfer the feature map and feature map By adding elements one by one, we obtain the feature map. ;

[0105] Step 215: From the feature map Extracting consecutive pixel blocks as feature maps ;

[0106] Step 216: Transfer the feature map The input is sequentially fed into the third linear layer and the fourth softmax activation function layer. The fourth softmax activation function layer outputs the feature map. ;

[0107] Step 217: Transfer the feature map and feature map By adding elements one by one, we obtain the feature map. ;

[0108] Step 218: Transfer the feature map The input layers are sequentially processed by the fifth linear layer, the sixth average pooling layer, and the seventh RMSNorm layer. The seventh RMSNorm layer outputs a feature map. ;

[0109] feature map and feature map Element-wise multiplication yields the feature map. ;

[0110] Step 219: Transfer the feature map The input layers are sequentially the eighth linear layer, the ninth average pooling layer, and the tenth RMS Orm layer. The tenth RMS Orm layer outputs a feature map. ;

[0111] feature map and feature map Element-wise multiplication yields the feature map. ;

[0112] Step 210: Transfer the feature map The input layers are sequentially the eleventh linear layer, the twelfth average pooling layer, and the thirteenth RMS Orm layer. The thirteenth RMS Orm layer outputs a feature map. ;

[0113] feature map and feature map Element-wise multiplication yields the feature map. .

[0114] To alleviate the problems of high redundancy in pixel-level features and difficulty in adaptively transferring regional structural information, this invention proposes a Learnable Hierarchical Superpixel Mapping (LHSM) module. This module aims to leverage the structural priors provided by hierarchical superpixels to establish a learnable cross-layer mapping relationship between pixel-level and region-level representations, thereby enhancing the expressive power of multi-scale spatial information in hyperspectral images.

[0115] The other steps and parameters are the same as those in one of the specific implementation methods one to five.

[0116] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that, in step two, the feature map output by the Learnable Hierarchical Superpixel Mapping Module (LHSM) is... Feature map and feature map The input layer is a hierarchical superpixel multi-expert hybridization module (HSMoE), and the output layer is a feature map. The specific process is as follows:

[0117] After processing by the LHSM module, the input hyperspectral features are organized into multiple hierarchical representations, including pixel-level features and superpixel-level features at different scales. Because these hierarchical features differ significantly in spatial granularity, structural semantics, and discriminative emphasis, a uniform feature transformation approach often fails to fully exploit the complementary information between different hierarchical features. Specifically, pixel-level features focus more on fine-grained local spectral variations, while superpixel-level features emphasize internal consistency and a broader contextual structure. Therefore, adaptive enhancement and effective fusion of features at different hierarchical levels are crucial for improving overall representational capability.

[0118] To address this, this invention designs a Hierarchical Superpixel Mixture-of-Experts (HSMoE) module prior to FBDMamba, used for selective modeling of pixel-level and superpixel-level features output by LHSM. This module learns the transformation patterns of different hierarchical features through multiple expert branches and adaptively assigns the importance of each branch using a gating mechanism, thereby obtaining a more robust hierarchical fusion representation.

[0119] Step Two Two One,

[0120] Feature maps output by the Learnable Hierarchical Superpixel Mapping (LHSM) module Perform routing noise regularization to obtain pixel-level features. ;

[0121] The feature map output by the Learnable Hierarchical Superpixel Mapping (LHSM) module Perform routing noise regularization to obtain superpixel-level features. ;

[0122] The feature map output by the Learnable Hierarchical Superpixel Mapping (LHSM) module Perform routing noise regularization to obtain superpixel-level features. ;

[0123] ,in Represents pixel-level features. and These represent two different scales of superpixel-level features. Indicates the number of feature units, Indicates feature dimension;

[0124] Step 2: To reduce the distribution differences between different levels of representation, pixel-level features are first mapped using a linear mapping. Superpixel-level features Superpixel-level features Projecting onto a unified feature space yields the aligned hierarchical feature representation; represented as:

[0125] (7)

[0126] in, and For learnable projection parameters;

[0127] and This represents the aligned hierarchical features.

[0128] Subsequently, HSMoE viewed the three hierarchical branches as three experts with different functional preferences, and... and Perform independent transformations to obtain the features corresponding to the pixel-level experts. Features corresponding to mesoscale superpixel experts Features corresponding to large-scale superpixel experts ; indicates as:

[0129] (8)

[0130] in, These represent the nonlinear mapping functions corresponding to pixel-level experts, mesoscale superpixel experts, and large-scale superpixel experts, respectively.

[0131] By using independent expert modeling, features at different levels can be enhanced in their respective more suitable representation spaces, thereby avoiding the feature confusion problem caused by uniform transformation.

[0132] Steps 2 and 3: Identify the features corresponding to pixel-level experts. Features corresponding to mesoscale superpixel experts Features corresponding to large-scale superpixel experts By splicing the data, we can obtain the features. ;

[0133] Step 224: Features With pixel-level features Superpixel-level features Superpixel-level features By splicing the data, we can obtain the features. ;

[0134] As the output feature map of the hierarchical superpixel multi-expert hybrid module HSMoE;

[0135] Step 225: Calculate the loss function of the hierarchical superpixel multi-expert hybrid module HSMoE; expressed as:

[0136] (9)

[0137] in, This represents the loss of the hierarchical superpixel multi-expert hybrid module HSMoE; This represents the main classification loss (cross-entropy loss). This represents the expert equilibrium loss; The equilibrium coefficient is used to control the strength of the expert equilibrium constraints in the overall optimization.

[0138] The expert equilibrium loss Represented as:

[0139] (10)

[0140] in, express , or ; This represents the average gating weight corresponding to pixel-level experts; This represents the average gating weight corresponding to the mesoscale superpixel expert; Indicates the average gating weights corresponding to large-scale superpixel experts; index Indicates index ,index or index .

[0141] Expert equilibrium loss By constraining the average response of the three experts to be close to an equilibrium distribution, the expert collapse problem can be effectively alleviated, allowing branches at different scales to be adequately trained.

[0142] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0143] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One to Seven in that the average gating weight corresponding to the pixel-level expert is... Represented as:

[0144]

[0145] The average gating weight corresponding to the mesoscale superpixel expert Represented as:

[0146]

[0147] The average gating weight corresponding to the large-scale superpixel expert Represented as:

[0148]

[0149] in, Indicates the first The gating weights corresponding to pixel-level experts for each feature unit. Features representing pixel-level experts Features corresponding to mesoscale superpixel experts or features corresponding to large-scale superpixel experts Total number of mid-pixels and superpixels;

[0150] Indicates the first The gating weights corresponding to the mesoscale superpixel experts of each feature unit;

[0151] Indicates the first Gating weights corresponding to large-scale superpixel experts for each feature unit;

[0152] The first The gating weights corresponding to the pixel-level experts of each feature unit The acquisition process is as follows:

[0153] The features corresponding to the pixel-level experts in step 222 Input gating network (gating network in the hybrid expert model MOE), gating network output the first... The gating weights corresponding to the pixel-level experts of each feature unit , ;

[0154] The first The gating weights corresponding to the mesoscale superpixel experts for each feature unit The acquisition process is as follows:

[0155] The features corresponding to the mesoscale superpixel expert in step 222 are... Input gating network (gating network in the hybrid expert model MOE), gating network output the first... Gating weights corresponding to mesoscale superpixel experts for each feature unit , ;

[0156] The first The gating weights corresponding to the large-scale superpixel experts for each feature unit The acquisition process is as follows:

[0157] The features corresponding to the large-scale superpixel expert in step 222 Input gating network (gating network in the hybrid expert model MOE), gating network output the first... Gating weights corresponding to large-scale superpixel experts for each feature unit , ;

[0158] To achieve adaptive selection and effective collaboration of features at different levels, this invention further introduces a gating network to dynamically model the contributions of the three expert branches. Specifically, firstly, a global descriptive model is performed on pixel-level features, mesoscale superpixel features, and large-scale superpixel features. These features are then input into the gating network to estimate the importance of each expert branch under the current input sample. Subsequently, the gating network outputs three normalized response weights, corresponding to the pixel-level expert, mesoscale superpixel expert, and large-scale superpixel expert, respectively. Because these weights are normalized, they can characterize the relative contributions of features at different levels to the final representation under unified constraints.

[0159] Based on this, the outputs of the three expert branches are adaptively fused according to their corresponding weights to obtain the final feature representation of the HSMoE module. Unlike fixed fusion methods, this strategy can dynamically adjust the strength of different levels of information based on the feature distribution of the input scene, giving the model stronger scene adaptability. Specifically, when the input samples rely more on local fine-grained spectral differences for discrimination, the gating mechanism tends to increase the response strength of pixel-level experts; while when the classification results rely more on the consistency within the region and the broader contextual structure, superpixel-level experts, especially higher-level superpixel branches, will be given greater weight.

[0160] Therefore, this gating fusion mechanism can not only effectively alleviate the heterogeneity problem between features at different levels, but also enhance the complementarity between multi-level representations, so that the model can always obtain more robust and discriminative feature representations when facing scenarios with different spatial structures and category distributions.

[0161] It should be noted that relying solely on the gating network for dynamic allocation may lead to an imbalance in expert utilization during training, where some experts dominate for extended periods while others are undertrained, thus limiting the model's expressive power. To address this, this invention further introduces an expert balance loss to constrain the utilization of the three experts, encouraging the gating network to allocate activation probabilities more evenly among them. Within a batch, the average gating weights of the three experts are as follows: , or .

[0162] By introducing expert balance loss, HSMoE can not only adaptively select more suitable hierarchical experts based on the input content, but also avoid excessive imbalance in the division of labor among experts during training, thereby improving the stability of model training and the sufficiency of feature learning. Since this module is located before FBDMamba, its role is to provide more robust and balanced hierarchical fusion features for subsequent sequence modeling, laying a better input foundation for subsequent long-distance dependency modeling and local detail characterization.

[0163] In summary, HSMoE achieves adaptive enhancement and robust fusion of LHSM output features through a collaborative design of "multi-level expert modeling - dynamic gating fusion - balanced loss constraints", effectively improving the discriminativeness and generalization ability of hierarchical representation.

[0164] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0165] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One through Eight in that, in steps two and three, the output features of the Hierarchical Superpixel Multi-Expert Hybridization Module (HSMoE) are... Input forward-backward-differential Mamba module FBDMamba, output features of the forward-backward-differential Mamba module FBDMamba ;

[0166] The specific process is as follows:

[0167] Step 231: Output features from the hierarchical superpixel multi-expert hybrid module HSMoE Input a forward selective state-space model, output features from the forward selective state-space model. The specific process is as follows:

[0168] The output features of the hierarchical superpixel multi-expert hybrid module HSMoE Input layer normalized LN, layer normalized LN output features ;

[0169] Features Input to a linear layer, output features from the linear layer ;

[0170] Features Input to convolutional layer, output features from convolutional layer ;

[0171] Features The input is a sigmoid activation function layer, and the output of the sigmoid activation function layer is a feature. ;

[0172] Features Input State-Space Model (SSM), Output Features of State-Space Model (SSM) ;

[0173] Features The input is a sigmoid activation function layer, and the output of the sigmoid activation function layer is a feature. ;

[0174] Features and characteristics Element-wise multiplication yields the features. ;

[0175] Features Input to a linear layer, output features from the linear layer ;

[0176] Step 232: Output features from the hierarchical superpixel multi-expert hybrid module HSMoE Input a backward selective state-space model, output features from a forward selective state-space model. The specific process is as follows:

[0177] The output features of the hierarchical superpixel multi-expert hybrid module HSMoE Input layer normalized LN, layer normalized LN output features ;

[0178] Features Input to a linear layer, output features from the linear layer ;

[0179] Features Input to convolutional layer, output features from convolutional layer ;

[0180] Features The input is a sigmoid activation function layer, and the output of the sigmoid activation function layer is a feature. ;

[0181] Features Input State-Space Model (SSM), Output Features of State-Space Model (SSM) ;

[0182] Features The input is a sigmoid activation function layer, and the output of the sigmoid activation function layer is a feature. ;

[0183] Features and characteristics Element-wise multiplication yields the features. ;

[0184] Features Input to a linear layer, output features from the linear layer ;

[0185] Step 233: Features and characteristics By fusing, features are obtained. ; indicates as:

[0186]

[0187] in, The table shows the parameters that can be learned; The table shows the parameters that can be learned;

[0188] Steps two, three, and four: Output features from the hierarchical superpixel multi-expert hybrid module HSMoE Input a differential selective state-space model, and the differential selective state-space model outputs features; the specific process is as follows:

[0189] The output features of the hierarchical superpixel multi-expert hybrid module HSMoE Input layer normalized LN, layer normalized LN output features ;

[0190] Features Input to a linear layer, output features from the linear layer ;

[0191] Features Input to convolutional layer, output features from convolutional layer ;

[0192] Features The input is a sigmoid activation function layer, and the output of the sigmoid activation function layer is a feature. ;

[0193] Features Input State-Space Model (SSM), Output Features of State-Space Model (SSM) ;

[0194] Features The input is a sigmoid activation function layer, and the output of the sigmoid activation function layer is a feature. ;

[0195] Features and characteristics Element-wise multiplication yields the features. ;

[0196] Features Input to a linear layer, output features from the linear layer ;

[0197] Steps two, three, and five.

[0198] Features Input deep stochastic regularization, output features ;

[0199] Features Input deep stochastic regularization, output features ;

[0200] Steps 2, 3, and 6: Features and characteristics By summing, we obtain the characteristics. ; indicates as:

[0201]

[0202] Steps 2, 3, and 7: Features Input to RMSNorm layer, output features of RMSNorm layer ;

[0203] The RMSNorm layer is a root mean square layer normalization.

[0204] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.

[0205] After processing by the HSMoE module, the input features have undergone adaptive enhancement and fusion at the pixel level and different superpixel scales, resulting in a more robust hierarchical representation. However, although Mamba has significant advantages in long-range dependency modeling and can effectively capture global spatial-spectral context relationships in hyperspectral images, its response to local fine-grained changes, boundary details, and class transition regions remains relatively insufficient. Especially in hyperspectral images, different classes often exhibit continuous transitions or local mixed distributions. Simply relying on global context propagation can easily weaken local difference information, thus affecting the discrimination ability in complex and heterogeneous scenes.

[0206] To further illustrate this point, Figure 2 The comparison results between real images and corresponding differencing images in the PU dataset are presented. It can be observed that, compared to the original images, the differencing representation can better highlight region boundaries, texture changes, and class transition locations, and to some extent suppress redundant responses in large smooth areas. This indicates that differencing information has a natural advantage in characterizing local structural changes, providing more sensitive discrimination cues for boundary regions, small-scale targets, and fine-grained features. Inspired by this, this invention further introduces differencing branches based on Mamba's bidirectional sequence modeling to explicitly enhance the modeling capability for local change patterns.

[0207] Based on the above considerations, this invention proposes a Forward-Backward-Differential Mamba (FBDMamba) module, which aims to enhance the characterization of local details and boundary changes by introducing differential responses while maintaining the advantages of Mamba's global dependency modeling. The specific structure is as follows: Figure 3 As shown.

[0208] The core idea of ​​the FBDMamba module is to achieve a joint representation of global and local information through collaborative modeling of forward, backward, and difference branches. First, the forward branch captures global spatial dependencies from the image's starting point to its ending point, while the backward branch captures supplementary inverse dependencies from the opposite direction, further enhancing the completeness of global information. The difference branch, by emphasizing the differences between the forward and backward branches, highlights local details and boundary variations.

[0209] When fusing these three types of information, this invention employs a gating fusion mechanism to adaptively adjust the contribution ratio of each branch. The gating network dynamically adjusts the activation weights of the forward, backward, and differential branches based on the distribution of input features, thereby achieving weighted fusion of different features. Unlike traditional fixed-weighting methods, the gating mechanism can more flexibly allocate information according to data features, thus enhancing the model's expressive power and adaptability.

[0210] To improve the stability and robustness of model training, this invention introduces residual connections after fusion and combines DropPath and RMSNorm for output normalization. Residual connections can preserve the original input information and avoid feature degradation caused by excessively deep network layers; DropPath enhances the model's generalization ability by randomly discarding paths and prevents overfitting; RMSNorm, as a normalization method, helps stabilize the training process and improve the consistency of feature representations.

[0211] Through the above design, FBDMamba can efficiently integrate information from the forward, backward, and difference branches while preserving the uniqueness of features at different levels. The forward and backward branches focus on global dependency modeling, while the difference branch emphasizes the details of local variations. Finally, the gated fusion mechanism allows the contribution of each branch to be adaptively adjusted according to the input features, thereby enhancing the model's ability to represent complex hyperspectral images.

[0212] Introduced after HSMoE, FBDMamba effectively combines global and local information, providing richer and more discriminative feature representations for subsequent classification tasks. In its multi-level spatial structure, FBDMamba not only improves the model's ability to model long-range dependencies but also enhances its sensitivity to fine-grained targets and class transition regions, making hyperspectral image classification tasks more robust and accurate.

[0213] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One to Nine in that, in step Two Four, the features of step Two Two are... Features of steps two and three Input the classification layer, and the classification layer outputs the classification result;

[0214] The specific process is as follows:

[0215] The features of step two two The input is sequentially average pooling, then a fully connected (FC) layer, and the FC layer outputs features. ;

[0216] Features Input to a multilayer perceptron (MLP), output features of the multilayer perceptron (MLP) ;

[0217] Features and characteristics The classification results are obtained by summing each element.

[0218] The other steps and parameters are the same as those in any of the specific implementation methods one to nine.

[0219] Example:

[0220] To fully validate the performance of the proposed HSMMamba, this invention employs three typical hyperspectral datasets for testing: the PU (Pavia University) dataset, the SA (Salinas) dataset, and the HongHu dataset from Wuhan University. These three datasets differ significantly in scene type and spectral characteristics, allowing for a multi-dimensional examination of the model's generalization ability and classification performance. We will describe each dataset in detail.

[0221] The PU (Pavia University) dataset was acquired over Pavia, Italy, using the ROSIS (Reflective Optics System Imaging Spectrometer) sensor. The images have a spatial resolution of 1.3 meters, a size of 610×340 pixels, and contain 103 spectral bands. Ground truth labels cover nine categories, such as "trees," "bare soil," and "asphalt road."

[0222] The SA (Salinas) dataset was acquired by the AVIRIS (Airborne Visible / Infrared Imaging Spectrometer) sensor in the Salinas Valley region of California, USA. The images are 512×217 pixels in size, with a spatial resolution of 3.7 meters, and contain 204 spectral channels. This dataset defines 16 land cover categories, covering various crop types such as "grapes," "lettuce," and "vineyards."

[0223] The HongHu dataset was collected in Honghu City, Hubei Province, China, from 16:23 to 17:37 on November 20, 2017, using a Headwall Nano-Hyperspec hyperspectral imager (focal length 17 mm) mounted on a DJI Matrice 600 Pro drone platform. This area is a typical agricultural complex, containing not only various crops but also multiple varieties of the same crop. The image size is 940×475 pixels, with a spectral range of 400–1000 nm and a total of 270 spectral channels.

[0224] All experiments in this invention were performed on a platform with an Intel® Core™ i9-14900KF CPU, an NVIDIA GeForce RTX4090D GPU, and 128GB of random access memory, using the PyTorch language framework. Furthermore, this invention employs three common evaluation metrics to assess the model's classification performance: Overall Accuracy (OA), Average Accuracy (AA), and the Kappa coefficient. The OA value represents the ratio of correctly classified samples to the total number of samples; the AA value represents the average classification accuracy for each class; and the Kappa coefficient is a metric used to measure the model's robustness.

[0225] To verify the effectiveness and superiority of the proposed HSMMamba model, this invention selects several representative hyperspectral image classification methods as comparison models. These methods cover advanced networks based on Transformer, CNN, GCN, and Mamba, enabling a comprehensive evaluation of the proposed method's performance from different perspectives. The comparison methods include SpectralFormer (2021), SSFTT (2022), MorphFormer (2023), CLOLN (2024), GSCViT (2024), MambaLG (2025), MambaHSI (2024), and DBMGNet (2025).

[0226] SpectralFormer: Proposes a spectral grouping embedding mechanism. By grouping and modeling the spectral dimensions, it can learn richer local spectral representations, thereby enhancing the ability to extract hyperspectral features.

[0227] SSFTT: By introducing a Gaussian weighted labeling mechanism, shallow spectral-spatial semantic features are extracted using sequence modeling, achieving efficient Transformer representation learning.

[0228] MorphFormer: Proposes a spectral-spatial-morphological joint attention Transformer, which further improves the accuracy and robustness of hyperspectral image classification by fusing spectral features, spatial features and morphological features.

[0229] CLOLN: Designs a lightweight channel-guided network structure that reduces the number of parameters and computational complexity of convolutional neural networks while balancing classification performance and model efficiency.

[0230] GSCViT combines grouped separable convolution with Vision Transformer to model both local details and global contextual information, thereby achieving a unified spectral-spatial feature representation.

[0231] MambaLG: Constructs a local-global dual-branch structure that combines local spatial modeling with long-range spectral sensing, enhancing global modeling capabilities while preserving fine features.

[0232] MambaHSI: As one of the earliest Mamba models applied to hyperspectral image classification, this method can adaptively fuse spatial and spectral information and achieve long-distance dependency modeling with linear complexity.

[0233] DBMGNet: Constructs a dual-branch Mamba-GCN hybrid framework, which improves the model's ability to represent complex scenes by integrating graph convolution modeling capabilities with state space modeling capabilities.

[0234] Ablation experiment:

[0235] To verify the effectiveness of the key modules proposed in this invention, ablation experiments were conducted on the UP, SA, and HH datasets. The experiments started with a baseline model and gradually introduced the LHSM, HSMoE, and FBDMamba modules to construct five comparison schemes. The experimental results for each group are shown in Table 1.

[0236] Specifically, Case 1 is the baseline model; Case 2 introduces the LHSM module on the baseline model; Case 3 further adds the HSMoE module on the basis of Case 2; Case 4 adds the FBDMamba module on the basis of Case 2; and Case 5 is the complete model, which integrates the three modules LHSM, HSMoE and FBDMamba.

[0237] As can be seen from Table 1, with the gradual introduction of each module, the classification performance of the model on the three datasets has been continuously improved, indicating that each module designed in this invention has a positive effect on the hyperspectral image classification task.

[0238] First, comparing Case 1 and Case 2, it can be seen that adding LHSM to the baseline model improves the performance on the UP, SA, and HH datasets by 2.79%, 1.53%, and 2.06%, respectively. This result shows that the LHSM module can effectively optimize the feature transfer relationship between hierarchical superpixels, enhancing the cross-layer information interaction capability while preserving the superpixel structural prior, thereby improving the model's representation of regional structural information.

[0239] Secondly, comparing Case 2 and Case 3 reveals that the model performance was further improved after introducing HSMoE. Specifically, the classification accuracy on UP, SA, and HH improved by 1.01%, 1.69%, and 0.70% respectively compared to Case 2. This indicates that the HSMoE module can perform more flexible expert assignment and feature enhancement for features at different levels and granularities, effectively improving the model's ability to model complex spatial heterogeneous structures.

[0240] Furthermore, comparing Case 2 and Case 4, it can be seen that after adding FBDMamba, the model's classification accuracy on the three datasets reached 96.41%, 98.20%, and 94.27%, respectively, representing improvements of 1.24%, 2.18%, and 1.73% compared to Case 2. Meanwhile, compared to Case 3 which included HSMoE, Case 4 also achieved better results on all three datasets. This indicates that the FBDMamba module has stronger feature modeling capabilities in hyperspectral image classification, effectively supplementing local differences and fine-grained boundary information while maintaining the advantages of Mamba's long-range dependency modeling, thereby further enhancing classification discriminative power.

[0241] Finally, the complete model, Case 5, achieved performance improvements of 6.60%, 4.81%, and 5.17% compared to the baseline model, Case 1, on the three datasets. Furthermore, compared to Cases 2, 3, and 4, which only incorporated partial modules, the complete model still exhibited stable and significant performance gains. This fully demonstrates the strong complementarity among the three modules: LHSM, HSMoE, and FBDMamba. LHSM enhances the adaptive mapping capability between hierarchical superpixels, HSMoE strengthens the selective expression of features at different granularities, and FBDMamba improves the joint modeling capability of global dependencies and local details. The synergistic effect of these three modules collectively improves the model's classification performance in complex hyperspectral scenes.

[0242]

[0243] Parameter sensitivity analysis:

[0244] Looking at the trend of hierarchical levels, all three datasets achieved optimal or near-optimal classification results when the number of hierarchical levels = 3, indicating that a three-layer hierarchical structure is more conducive to modeling hierarchical spatial information in hyperspectral images. When the number of layers is small, the model struggles to fully utilize multi-level superpixel priors, resulting in limited ability to express regional structural relationships. Furthermore, when the number of layers increases to 4, model performance actually declines, indicating that excessively deep hierarchical divisions introduce more redundant information and increase the difficulty of cross-layer mapping and feature transfer, thus affecting the final classification result. Therefore, a three-layer hierarchical structure can achieve a better balance between the richness of hierarchical information and the stability of information transfer.

[0245] From the perspective of learning rate, the model exhibits certain differences under different learning rates. Overall, an excessively large learning rate can easily lead to oscillations during training, affecting the model's stable convergence; while an excessively small learning rate reduces parameter update efficiency, making it difficult for the model to fully learn discriminative features. In contrast, and Nearer learning rates typically yield better and more stable results, indicating that a moderate learning rate is more beneficial for optimizing HSMMamba training.

[0246] The experimental results from the three datasets show that the model performs best with 3 hierarchical levels, and also achieves more stable performance within a moderate learning rate range. Therefore, this invention sets the number of hierarchical levels to 3 in subsequent experiments.

[0247] Different training sample ratios: To evaluate the classification ability of HSMMamba under different training sample conditions, this invention sets different numbers of training samples on the UP, SA, and HH datasets, and compares the results with various comparative methods. The results are as follows: Figure 7 As shown. In the three datasets, the number of training samples for each class is set to... The overall classification accuracy (OA) is the average of 10 different random seed experiments.

[0248] Adaptive weight analysis for branches at different levels:

[0249] Figure 4This paper presents the adaptive weight distribution of HSMMamba across different levels on the PU, SA, and HongHu datasets. It shows significant differences in weight allocation across the three datasets, indicating that the model can adaptively adjust its dependence on features at different levels based on the characteristics of different scenes. Overall, the Pixel-level branch focuses more on fine-grained spectral discrimination, the Intermediate Superpixel branch emphasizes mid-level region structure and local consistency, while the High-level Superpixel branch emphasizes high-level contextual information and overall region organization. Therefore, the weight differences across the datasets essentially reflect variations in scene complexity, spatial structure features, and inter-class separability.

[0250] Overall, the PU dataset tends to rely more on high-level regional contextual information for discrimination, because many categories in its urban scenes depend not only on pixel spectra but also on spatial layout and regional consistency. The SA dataset has a relatively balanced weight distribution, indicating that agricultural scene classification requires both fine spectral information and appropriate mid-level regional structure assistance. The HongHu dataset, on the other hand, relies more on high-level contextual information. This is because high resolution and fine-grained categories make single-pixel features more susceptible to interference from local noise, shadows, and boundary mixing, thus requiring stronger regional contextual information to improve discrimination stability.

[0251] Overall, the weight distributions on different datasets show good consistency with their scene features, indicating that the adaptive fusion mechanism proposed in this invention can effectively perceive the hierarchical requirements under different scenes, thereby improving the model's adaptability and classification performance to complex hyperspectral images. Figure 4 .

[0252] Local visualization analysis of FBDMamba:

[0253] To further verify the effectiveness of the differential Mamba branch (DiffMamba) of FBDMamba in hyperspectral image classification, this invention compares the local prediction performance of the bidirectional Mamba (FBMamba) model with and without the differential branch on the HongHu dataset. Specifically, two representative local regions were selected from the overall classification results for magnified visualization to observe the differences in model performance in fine-grained boundaries, local structural changes, and class transition regions. The visualization results show that after introducing DiffMamba, the prediction of local regions is more continuous and refined, with clear boundary contours, and a significant reduction in misclassification of fine structures and transition regions; while the model using only bidirectional Mamba still exhibits blurred boundaries and local breaks in some complex regions. These experimental results demonstrate that DiffMamba, by introducing a central differential branch, effectively enhances the perception of fine-grained local changes, further improving the model's classification accuracy and robustness in complex scenes. Figure 5 .

[0254] Comparative analysis of HSMoE with different feature fusion strategies:

[0255] To evaluate the effectiveness of HSMoE in multi-scale feature fusion, this invention compares it with common feature fusion strategies such as Gate, Add, Concat, and BSConv. In the experiments, the rest of the network structure remained unchanged, with only the feature fusion module replaced to ensure fairness in the comparison. All methods were tested on the Pavia University, Salinas, and HongHu datasets, and overall accuracy (OA) was used as the evaluation metric.

[0256] The experimental results are shown in Table 2. It can be seen that compared with several traditional fusion strategies, HSMoE achieves superior OA performance on all three datasets, indicating its stronger discriminative ability and adaptability in the feature fusion process. Compared with static fusion methods such as Gate, Add, and Concat, HSMoE can specifically enhance features at different scales through a multi-expert mechanism, thus more effectively preserving fine-grained boundary information and regional structure information. Furthermore, compared with BSConv, HSMoE demonstrates greater flexibility and robustness in handling heterogeneous features across scales. These results show that the proposed HSMoE can more fully leverage the complementary advantages of multi-scale features, thereby improving hyperspectral image classification performance.

[0257]

[0258] Quantitative analysis:

[0259] Tables 3 to 5 show the classification results of each comparative method on the PU, Salinas, and Honghu datasets, including overall accuracy (OA), average accuracy (AA), Kappa coefficient, and classification accuracy for each class. The best results are highlighted in bold. As can be seen from Tables 2 to 5, compared with other comparative methods, the HSMMamba proposed in this invention achieves superior classification performance on all three datasets, especially in OA, AA, and Kappa metrics, demonstrating a strong overall advantage.

[0260] Further analysis reveals that traditional CNN- and Transformer-based methods often struggle to fully extract spatial-spectral discriminative information from hyperspectral images when training samples are limited, resulting in relatively low classification accuracy. In contrast, HSMMamba, by combining hierarchical superpixel representations, MoE feature enhancement, and Mamba's long-range dependency modeling capabilities, can more effectively utilize limited samples to learn more discriminative feature representations, thus maintaining stable and excellent classification performance even in small-sample scenarios. This result fully validates the effectiveness and robustness of HSMMamba in hyperspectral image classification tasks.

[0261] Table 3. Classification results of different methods on the PU dataset

[0262]

[0263] Table 4. Classification results of different methods on the SA dataset

[0264]

[0265] Table 5. Classification results of different methods on the HH dataset

[0266]

[0267] Visual evaluation: To more intuitively analyze the classification performance of the proposed HSMMamba on different remote sensing hyperspectral datasets, this invention performs a visual analysis of the classification results of all comparison methods. Figure 6 , Figure 7 , Figure 8 The classification results of each method on the PU, Salinas, and Honghu datasets are presented respectively. These figures show that, compared with other comparative methods, the classification map obtained by HSMMamba performs better in terms of boundary clarity, class coherence, and overall classification accuracy. Its results are closer to the true distribution of ground features, further validating the effectiveness and superiority of this method in hyperspectral image classification tasks.

[0268] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A method for constructing a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification, characterized in that: The specific process of the method is as follows: Step 1: Obtain the raw hyperspectral image; Step 2: Construct the HSMMamba network model; The HSMMamba network model represents a hierarchical superpixel multi-expert Mamba network model used for hyperspectral image classification. The HSMMamba network model consists of LHSM, HSMoE, FBDMamba, and a classification layer. LHSM is a learnable hierarchical superpixel mapping module; HSMoE is a hierarchical superpixel multi-expert hybrid module; FBDMamba is a forward-backward-differential Mamba module; The working process of the HSMMamba network model is as follows: The original hyperspectral image is sequentially input into LHSM, HSMoE, FBDMamba and the classification layer, and the classification layer outputs the classification result. Step 3: Classify the original hyperspectral image to be tested based on the trained HSMMamba network model and output the classification results.

2. The method for constructing a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification according to claim 1, characterized in that: The learnable hierarchical superpixel mapping module (LHSM) includes a stem module, a first linear layer, a second softmax activation function layer, a third linear layer, a fourth softmax activation function layer, a fifth linear layer, a sixth average pooling layer, a seventh RMSNorm layer, an eighth linear layer, a ninth average pooling layer, a tenth RMSNorm layer, an eleventh linear layer, a twelfth average pooling layer, and a thirteenth RMSNorm layer. The RMSNorm layer is a root mean square layer normalization; The stem module consists of the fourteenth batch normalized BN layer, the fifteenth 1×1 convolutional layer, and the sixteenth LeakyReLU activation function layer.

3. The method for constructing a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification according to claim 2, characterized in that: The forward-backward-differential Mamba module FBDMamba includes a forward selective state space model, a backward selective state space model, and a differential selective state space model.

4. The method for constructing a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification according to claim 3, characterized in that: The classification layer includes an average pooling layer, a fully connected layer (FC), and a multilayer perceptron (MLP).

5. The method for constructing a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification according to claim 4, characterized in that: The working process of the HSMMamba network model is as follows: Step 2: Input the original hyperspectral image into the Learnable Hierarchical Superpixel Mapping (LHSM) module. The LHSM module outputs a feature map. Feature map and feature map ; Step 22: Output feature maps from the Learnable Hierarchical Superpixel Mapping (LHSM) module. Feature map and feature map Input Hierarchical Superpixel Multi-Expert Hybridization Module (HSMoE), Output Features of Hierarchical Superpixel Multi-Expert Hybridization Module (HSMoE) The loss function of the hierarchical superpixel multi-expert hybrid module HSMoE is constructed. ; Steps 2 and 3: Output features from the hierarchical superpixel multi-expert hybrid module HSMoE Input forward-backward-differential Mamba module FBDMamba, output features of the forward-backward-differential Mamba module FBDMamba ; Step 24: The features from Step 22... Features of steps two and three Input the classification layer, and the classification layer outputs the classification result.

6. The method for constructing a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification according to claim 5, characterized in that: In step two, the original hyperspectral image is input to the Learnable Hierarchical Superpixel Mapping (LHSM) module, which outputs a feature map. Feature map and feature map ; The specific process is as follows: Step 2: Input the original hyperspectral image into the STEM module; the STEM module outputs a feature map. Based on feature maps Obtain feature map ; The specific process is as follows: 1) The original hyperspectral image is sequentially input into the fourteenth batch normalized BN layer, the fifteenth 1×1 convolutional layer, and the sixteenth LeakyReLU activation function layer. The sixteenth LeakyReLU activation function layer outputs the feature map. ; 2) Feature map Perform hierarchical division and construct Layered superpixel structure; For the Layers, pixels The superpixel number to which it belongs is denoted as , ; in, Indicates the first Total number of superpixels in the layer; ; 3) Perform feature aggregation for each superpixel; specifically: No. Layer The region feature of a superpixel is defined as: (1) in, Indicates the first Layer The set of pixels contained in a superpixel ; express The number of elements contained; Indicates the 0th layer. Features of each pixel; Indicates the first Layer Region features of one superpixel; 4) First, construct the path from pixel to the... Initial mapping matrix of layer superpixels ; indicates as: (2) in, For learnable residuals; The first term represents the learnable residual. Layer superpixels; The first term represents the learnable residual. Layer superpixels; Subsequently, the mapping matrix Applying Softmax normalization row by row yields the Softmax-normalized superpixel matrix, represented as: (3) in, express The OK; The superpixel matrix after Softmax normalization With feature map Multiply, then sequentially input into a fully connected layer and a non-linear activation function layer to obtain the first... Feature maps of layer superpixels ; indicates as: (4) in, and These represent the weight matrix and bias term of the fully connected layer, respectively. Represents a non-linear activation function; For any pixel Pixels In the The mapping feature corresponding to the layer is defined as follows: (5) in, Represents pixels The superpixel number to which it belongs. Indicates the first Layer superpixel Features; Indicates the first layer pixels Features; Based on the layer pixels Features Get the first Layer feature map ; 5) Feature map With feature maps of each layer The feature maps are then fused to obtain the final feature map. ; indicates as: (6) in, Indicates to After channel concatenation, the input is given to the linear layer, and the feature map is output by the linear layer. Step 2.12: From the feature map Extracting consecutive pixel blocks as feature maps ; Steps 2-13: Transfer the feature map The input is sequentially fed into a first linear layer and a second softmax activation function layer. The second softmax activation function layer outputs a feature map. ; Step 214: Transfer the feature map and feature map By adding elements one by one, we obtain the feature map. ; Step 215: From the feature map Extracting consecutive pixel blocks as feature maps ; Step 216: Transfer the feature map The input is sequentially fed into the third linear layer and the fourth softmax activation function layer. The fourth softmax activation function layer outputs the feature map. ; Step 217: Transfer the feature map and feature map By adding elements one by one, we obtain the feature map. ; Step 218: Transfer the feature map The input layers are sequentially processed by the fifth linear layer, the sixth average pooling layer, and the seventh RMSNorm layer. The seventh RMSNorm layer outputs a feature map. ; feature map and feature map Element-wise multiplication yields the feature map. ; Step 219: Transfer the feature map The input layers are sequentially the eighth linear layer, the ninth average pooling layer, and the tenth RMS Orm layer. The tenth RMS Orm layer outputs a feature map. ; feature map and feature map Element-wise multiplication yields the feature map. ; Step 210: Transfer the feature map The input layers are sequentially the eleventh linear layer, the twelfth average pooling layer, and the thirteenth RMS Orm layer. The thirteenth RMS Orm layer outputs a feature map. ; feature map and feature map Element-wise multiplication yields the feature map. .

7. The method for constructing a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification according to claim 6, characterized in that: In step two, the Learnable Hierarchical Superpixel Mapping (LHSM) module outputs a feature map. Feature map and feature map The input is a hierarchical superpixel multi-expert hybridization module (HSMoE), and the output is a feature map. The specific process is as follows: Step 221: Process the feature map output by the Learnable Hierarchical Superpixel Mapping (LHSM) module. Perform routing noise regularization to obtain pixel-level features. ; The feature map output by the Learnable Hierarchical Superpixel Mapping (LHSM) module Perform routing noise regularization to obtain superpixel-level features. ; The feature map output by the Learnable Hierarchical Superpixel Mapping (LHSM) module Perform routing noise regularization to obtain superpixel-level features. ; Step 2: First, pixel-level features are mapped using linear mapping. Superpixel-level features Superpixel-level features Projecting onto a unified feature space yields the aligned hierarchical feature representation; represented as: (7) in, and For learnable projection parameters; and This represents the aligned hierarchical features. Subsequently, and Perform independent transformations to obtain the features corresponding to the pixel-level experts. Features corresponding to mesoscale superpixel experts Features corresponding to large-scale superpixel experts ; indicates as: (8) in, These represent the nonlinear mapping functions corresponding to pixel-level experts, mesoscale superpixel experts, and large-scale superpixel experts, respectively. Steps 2 and 3: Identify the features corresponding to pixel-level experts. Features corresponding to mesoscale superpixel experts Features corresponding to large-scale superpixel experts By splicing the data, we can obtain the features. ; Step 224: Features With pixel-level features Superpixel-level features Superpixel-level features By splicing the data, we can obtain the features. ; As the output feature map of the hierarchical superpixel multi-expert hybrid module HSMoE; Step 225: Calculate the loss function of the hierarchical superpixel multi-expert hybrid module HSMoE; expressed as: (9) in, This represents the loss of the hierarchical superpixel multi-expert hybrid module HSMoE; This represents the primary classification loss; This represents the expert equilibrium loss; This is the balance coefficient; The expert equilibrium loss Represented as: (10) in, express , or ; This represents the average gating weight corresponding to pixel-level experts; This represents the average gating weight corresponding to the mesoscale superpixel expert; Indicates the average gating weights corresponding to large-scale superpixel experts; index Indicates index ,index or index .

8. The method for constructing a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification according to claim 7, characterized in that: The average gating weight corresponding to the pixel-level expert Represented as: The average gating weight corresponding to the mesoscale superpixel expert Represented as: The average gating weight corresponding to the large-scale superpixel expert Represented as: in, Indicates the first The gating weights corresponding to pixel-level experts for each feature unit. Features representing pixel-level experts Features corresponding to mesoscale superpixel experts or features corresponding to large-scale superpixel experts Total number of mid-pixels and superpixels; Indicates the first The gating weights corresponding to the mesoscale superpixel experts of each feature unit; Indicates the first Gating weights corresponding to large-scale superpixel experts for each feature unit; The first The gating weights corresponding to the pixel-level experts of each feature unit The acquisition process is as follows: The features corresponding to the pixel-level experts in step 222 Input to a gating network, output the first... The gating weights corresponding to the pixel-level experts of each feature unit , ; The first The gating weights corresponding to the mesoscale superpixel experts for each feature unit The acquisition process is as follows: The features corresponding to the mesoscale superpixel expert in step 222 are... Input to a gating network, output the first... Gating weights corresponding to mesoscale superpixel experts for each feature unit , ; The first The gating weights corresponding to the large-scale superpixel experts for each feature unit The acquisition process is as follows: The features corresponding to the large-scale superpixel expert in step 222 Input to a gating network, output the first... Gating weights corresponding to large-scale superpixel experts for each feature unit , .

9. The method for constructing a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification according to claim 8, characterized in that: In steps two and three, the hierarchical superpixel multi-expert hybrid module HSMoE outputs features. Input forward-backward-differential Mamba module FBDMamba, output features of the forward-backward-differential Mamba module FBDMamba The specific process is as follows: Step 231: Output features from the hierarchical superpixel multi-expert hybrid module HSMoE Input a forward selective state-space model, output features from the forward selective state-space model. The specific process is as follows: The output features of the hierarchical superpixel multi-expert hybrid module HSMoE Input layer normalized LN, layer normalized LN output features ; Features Input to a linear layer, output features from the linear layer ; Features Input to convolutional layer, output features from convolutional layer ; Features The input is a sigmoid activation function layer, and the output of the sigmoid activation function layer is a feature. ; Features Input State-Space Model (SSM), Output Features of State-Space Model (SSM) ; Features The input is a sigmoid activation function layer, and the output of the sigmoid activation function layer is a feature. ; Features and characteristics Element-wise multiplication yields the features. ; Features Input to a linear layer, output features from the linear layer ; Step 232: Output features from the hierarchical superpixel multi-expert hybrid module HSMoE Input a backward selective state-space model, output features from a forward selective state-space model. The specific process is as follows: The output features of the hierarchical superpixel multi-expert hybrid module HSMoE Input layer normalized LN, layer normalized LN output features ; Features Input to a linear layer, output features from the linear layer ; Features Input to convolutional layer, output features from convolutional layer ; Features The input is a sigmoid activation function layer, and the output of the sigmoid activation function layer is a feature. ; Features Input State-Space Model (SSM), Output Features of State-Space Model (SSM) ; Features The input is a sigmoid activation function layer, and the output of the sigmoid activation function layer is a feature. ; Features and characteristics Element-wise multiplication yields the features. ; Features Input to a linear layer, output features from the linear layer ; Step 233: Features and characteristics By fusing, features are obtained. ; indicates as: in, The table shows the parameters that can be learned; The table shows the parameters that can be learned; Steps two, three, and four: Output features from the hierarchical superpixel multi-expert hybrid module HSMoE Input a differential selective state-space model, and the differential selective state-space model outputs features; the specific process is as follows: The output features of the hierarchical superpixel multi-expert hybrid module HSMoE Input layer normalized LN, layer normalized LN output features ; Features Input to a linear layer, output features from the linear layer ; Features Input to convolutional layer, output features from convolutional layer ; Features The input is a sigmoid activation function layer, and the output of the sigmoid activation function layer is a feature. ; Features Input State-Space Model (SSM), Output Features of State-Space Model (SSM) ; Features The input is a sigmoid activation function layer, and the output of the sigmoid activation function layer is a feature. ; Features and characteristics Element-wise multiplication yields the features. ; Features Input to a linear layer, output features from the linear layer ; Steps two, three, and five: Features Input deep stochastic regularization, output features ; Features Input deep stochastic regularization, output features ; Steps 2, 3, and 6: Features and characteristics By summing, we obtain the characteristics. ; indicates as: Steps 2, 3, and 7: Features Input to RMSNorm layer, output features of RMSNorm layer ; The RMSNorm layer is a root mean square layer normalization.

10. The method for constructing a hierarchical superpixel multi-expert Mamba network for hyperspectral image classification according to claim 9, characterized in that: In step two-four, the features of step two-two will be... Features of steps two and three Input the classification layer, and the classification layer outputs the classification result; The specific process is as follows: The features of step two two The input is sequentially average pooling, then a fully connected (FC) layer, and the FC layer outputs features. ; Features Input to a multilayer perceptron (MLP), output features of the multilayer perceptron (MLP) ; Features and characteristics The classification results are obtained by summing each element.

Images