A hyperspectral image classification method based on multi-scale spiral scanning empty spectrum double-branch Mamba
By using a multi-scale spiral scanning spatial spectrum dual-branch Mamba model, the problem of insufficient multi-scale modeling capability in hyperspectral image classification is solved, achieving efficient feature extraction and fusion, improving classification accuracy and reducing computational complexity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- QINGDAO UNIV OF TECH
- Filing Date
- 2026-04-17
- Publication Date
- 2026-06-09
Smart Images
Figure CN122176530A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to hyperspectral image classification methods, and in particular to a hyperspectral image classification method based on multi-scale spiral scanning spatial spectrum dual-branch Mamba, belonging to the field of remote sensing information processing technology. Background Technology
[0002] Hyperspectral imaging is an important remote sensing technology that can simultaneously acquire spatial information of ground objects and continuous spectral information from the visible to the near-infrared bands. Benefiting from the rich spectral and spatial information, hyperspectral image (HSI) classification has become an important research direction in the field of remote sensing and has been widely applied in various remote sensing scenarios such as mineral identification, urban planning, military reconnaissance, and precision agriculture. Current deep learning-based hyperspectral image classification methods are mainly represented by convolutional neural networks (CNNs) and Transformer structures.
[0003] Traditional CNN methods with fixed convolutional kernels struggle to adapt to features with significant scale differences in hyperspectral images; while mainstream Transformer models suffer from high secondary computational complexity due to their self-attention mechanism. The Mamba architecture, proposed in recent years, offers a novel approach to lightweight hyperspectral image classification due to its linear complexity advantage. However, this method, based on a state-space model's sequence modeling paradigm, suffers from inherent limitations in preserving the two-dimensional spatial topology of hyperspectral images due to its unidirectional row-first serialization method. This leads to insufficient synergy between spatial and spectral features, limiting further improvements in classification accuracy.
[0004] Against this backdrop, the present invention provides a hyperspectral image classification method based on multi-scale spiral scanning spatial spectrum dual-branch Mamba, aiming to systematically solve the problems of insufficient multi-scale modeling capability, high computational complexity, and poor adaptability to targets of different sizes in the existing technology. It achieves a significant improvement in classification accuracy and effectively reduces the computational burden, possessing high efficiency and good application competitiveness. Summary of the Invention
[0005] This invention proposes a hyperspectral image classification method based on multi-scale spiral scanning spatial spectrum dual-branch Mamba, aiming to systematically solve the problems of insufficient multi-scale modeling capability, high computational complexity, and poor adaptability to targets of different sizes in existing technologies.
[0006] This invention proposes a hyperspectral image classification method based on a multi-scale spiral scanning spatial-spectral dual-branch Mamba model, aiming to systematically address the inherent shortcomings of standard Mamba, such as the susceptibility to damage of local spatial structures, insufficient modeling of long-distance spectral dependencies, and inadequate adaptive capability of multi-scale ground feature features. Specifically, a multi-scale spiral scanning spatial-spectral dual-branch Mamba model is proposed. This model designs a multi-scale spiral state-space model, extracting multi-scale spatial features through lightweight multi-scale convolution, and uses spiral scanning to convert two-dimensional spatial features into a one-dimensional sequence. The combination of these two approaches enables the model to simultaneously capture local features and global context. The center-first spiral scanning strategy effectively preserves the orderliness of the spatial structure, strengthens the dominance of the central pixel, and suppresses background noise and mixed pixel interference, laying a sequence feature foundation for efficient modeling of long-distance spatial associations. Considering the rich information in continuous spectral bands, an additional scanning module is designed in the spectral dimension to model semantic attributes in hyperspectral images through bidirectional interaction along the spectral dimension. To alleviate the feature redundancy problem in the fusion of spatial and spectral features, a gating mechanism is used to adaptively assign different weights to the two features, introducing a feature competition mechanism to promote efficient complementarity and fusion of spatial and spectral features.
[0007] The hyperspectral image classification method proposed in this invention, based on multi-scale spiral scanning spatial spectrum dual-branch Mamba, achieves a significant improvement in classification accuracy and effectively reduces the computational burden, possessing high efficiency and good application competitiveness.
[0008] The present invention proposes a hyperspectral image classification method based on multi-scale spiral scanning spatial Mamba, which mainly consists of four modules: a hyperspectral preprocessing module, a multi-scale spiral scanning spatial Mamba module, a bidirectional scanning spectral Mamba module, and an adaptive enhancement fusion module.
[0009] Hyperspectral preprocessing module: The input is a hyperspectral image cube. ,in These represent the height, width, and number of spectral bands of the input data, respectively. Using each pixel in the input hyperspectral image as the center, based on the given patch size... The area is divided into HSI patches, and mirror symmetry is used to fill the edge pixels. HSI patches ,in and This represents the patch size and the number of spectral bands. It is mapped using a 1×1 convolution (Conv1×1) to... ,in This represents the feature dimension. It is then normalized through a layer normalization layer before being input into the two-branch feature extraction structure.
[0010] Multi-scale spiral scan spatial Mamba module: This module aims to extract spatial features of HSI in the spatial dimension. After the features are input, they are first divided into feature tensors through linear projection mapping in a linear layer. With gate tensor . The feature tensor The output features are sequentially processed using depthwise convolution (DWConv) and SiLU activation functions, a multi-scale spiral state-space model (MS-SSM), and Layer Norm. Then, the output features are compared with a gated tensor activated by the GELU activation function. Element-wise multiplication is performed to achieve the interaction and aggregation of spatial features. MS-SSM is a key component of the multi-scale spiral scan spatial Mamba module, employing a dual-branch parallel deep convolutional structure and utilizing 3×3 and 5×5 convolutional kernels to extract multi-scale local spatial features from the input features. Each deep convolutional branch is followed by a spiral scan operation, unfolding the two-dimensional spatial feature map into a one-dimensional sequence. Unlike conventional flat and cross-scanning, spiral scanning traverses neighboring pixels from the center pixel along a spiral trajectory, maximizing the preservation of relative positional information between pixels during sequence transformation. Simultaneously, the use of both forward and backward scan directions effectively mitigates the potential loss of spatial structural information in sequence modeling due to the lack of built-in positional encoding in SSM. The sequences generated by the two multi-scale branches are fed into SSM to model global dependencies, further exploring long-range correlations in the spatial dimension. Finally, a gating mechanism automatically assigns weights to branches of different scales and scan directions based on the input feature content, suppressing redundant information.
[0011] Bidirectional scanning spectroscopy Mamba module: This module aims to extract rich spectral information. Features are mapped to an intermediate dimension via linear projection and then divided into feature tensors. With gate tensor Feature tensor Local feature enhancement is achieved through activation using the DWConv and SiLU activation functions. Since SSM is unidirectional, a single-direction spectral scan is insufficient to capture the contextual information between spectral bands, resulting in low spectral information utilization. Therefore, a bidirectional spectral scan approach is used. The two-dimensional feature map is flattened into a one-dimensional sequence along the spectral dimension, and the original sequence is concatenated with the inverse sequence after spectral dimension flipping to construct a bidirectional spectral input sequence, thereby improving the completeness of spectral context modeling. Subsequently, the sequence is used to establish long-range spectral dependency modeling through SSM, and after normalization, it is combined with a gated tensor activated by the GELU activation function. Perform element-wise multiplication, and then map back to the original feature dimension through linear projection.
[0012] Adaptive Enhanced Fusion Module: This module aims to aggregate spectral and spatial features. Its structure comprises two parallel feature encoding paths. For spectral features, a channel attention mechanism is used for further feature extraction. For spatial features, a depthwise separable convolution is employed to perform lightweight enhancement on the spatial feature map. Finally, the spatial features and the enhanced spectral features are concatenated along the channel dimension, and the gate weights are generated through a two-layer feedforward network (FFN) and a softmax layer. And split into spatial branch gating With spectral branching gate To enhance the dynamism and robustness of gating, the global mean of the gating weights is first calculated and multiplied by a coefficient of 0.5 to obtain the dynamic threshold. Then, hard threshold filtering is applied to the gating weights: this achieves adaptive competitive selection of spatial and spectral features, effectively suppressing feature redundancy and improving the robustness of HSI classification with low computational complexity. Finally, based on the filtered gating weights... and Element-wise weighted fusion of spatial and spectral features is performed.
[0013] The fused spectral and spatial features are residually connected with the normalized embedded features to complete the feature update of the Mamba layer. After the forward propagation of all Mamba layers is completed, the features are first aggregated globally in terms of spatial dimension through adaptive average pooling, and then normalized and flattened to obtain the global aggregated features. The global aggregated features are then fed into the linear classification head to output the category prediction results.
[0014] Through the above technical solutions, the present invention has the following advantages: (1) A spatial-spectral dual-branch Mamba model based on multi-scale spiral scanning is proposed. The spatial-spectral dual-branch structure is adopted to make full use of spatial-spectral feature information, and the lightweight design of the model is achieved through a single Mamba layer, so that the model can balance classification performance and running efficiency. (2) A multi-scale spiral scanning spatial Mamba module is proposed. By using lightweight multi-scale convolution and center-first spiral scanning, global long-distance dependencies are established while multi-scale spatial features are extracted and dynamically fused, which significantly improves classification accuracy. (3) Design a bidirectional scanning spectral Mamba module to overcome the unidirectionality of the state space model through the bidirectional scanning mechanism, extract spectral features, and make full use of spectral information to ensure the classification performance of the model. (4) Design an adaptive enhancement fusion module to enhance spectral features through channel attention and use a gating mechanism to dynamically allocate the weights of spectral and spatial features to achieve complementary enhancement and fusion of the two, effectively suppressing feature redundancy and improving the robustness of hyperspectral image classification. Attached Figure Description
[0015] Figure 1 The present invention provides a framework diagram for a hyperspectral image classification method based on multi-scale spiral scanning spatial spectrum dual-branch Mamba. Figure 2 The structural diagram of the multi-scale spiral scanning spatial Mamba module of this invention. Figure 3 The structural diagram of the bidirectional scanning spectroscopy Mamba module of this invention. Figure 4 The structure diagram of the adaptive enhancement fusion module of this invention. Figure 5 The present invention provides a flowchart of a hyperspectral image classification method based on multi-scale spiral scanning spatial spectrum dual-branch Mamba. Detailed Implementation
[0017] The invention will now be described in more detail with reference to the accompanying drawings.
[0018] This invention proposes a hyperspectral image classification method based on multi-scale spiral scanning spatial spectrum dual-branch Mamba, aiming to systematically solve the problems of insufficient multi-scale modeling capability, high computational complexity, and poor adaptability to targets of different sizes in existing technologies.
[0019] The hyperspectral image classification method proposed in this invention, based on multi-scale spiral scanning spatial spectrum dual-branch Mamba, achieves an excellent balance between classification accuracy and computational efficiency, and has significant practical application value and prospects for promotion.
[0020] Please refer to Figure 1 This is a framework diagram of a hyperspectral image classification method based on multi-scale spiral scanning spatial-spectral dual-branch Mamba, as proposed in this invention. The proposed hyperspectral image classification method based on multi-scale spiral scanning spatial-spectral dual-branch Mamba mainly consists of four modules: a hyperspectral preprocessing module, a multi-scale spiral scanning spatial Mamba module, a bidirectional scanning spectral Mamba module, and an adaptive enhancement fusion module. Please refer to [link / reference]. Figure 5 This is a flowchart of a hyperspectral image classification method based on multi-scale spiral scanning spatial spectrum dual-branch Mamba according to the present invention. The specific analysis steps are as follows: S1: Input data and preprocess it, processing the hyperspectral image cube. ,in The height, width, and number of spectral bands of the input data are respectively used. Each pixel in the input hyperspectral image is taken as the center, and the image patch size is adjusted accordingly. Extracting spatial neighborhoods yields hyperspectral image patches. ,in and Indicates the image patch size and the number of spectral bands; S2: Feature embedding, using 1×1 convolution to... Mapping to embedded features ,in Indicates the feature dimension; S3: After normalizing the embedded features, they are fed into the Mamba layer. This layer consists of two parallel Mamba blocks, corresponding to the spatial branch and the spectral branch, respectively. The two branches independently complete feature modeling and extraction. Please refer to Figure 2 This is a structural diagram of the multi-scale spiral scanning spatial Mamba (MS-SpaM) module of this invention. Spatial feature extraction... After normalization, it becomes , The MS-SpaM module first divides the data into feature tensors through linear layer linear projection mapping. With gate tensor : . in, For linear projection layers, This is a splitting operation.
[0021] feature tensor Activation functions are applied sequentially using DWConv and SiLU:
[0022] After passing through MS-SSM, two parallel depthwise convolution branches are used, employing 3×3 and 5×5 convolution kernels respectively. Figure 2 Taking the first branch in (c) as an example, after a 3×3 convolution, a spiral scanning module is connected to expand the spatial feature map into a one-dimensional sequence according to the spatial dimension. Please refer to [link / reference]. Figure 2 (b) illustrates a spiral scan. The spiral scan starts from the center pixel and visits neighboring pixels sequentially along a spiral trajectory, maximizing the preservation of relative positional information between pixels during sequence conversion. Simultaneously, a forward scan direction is used to alleviate the lack of built-in positional encoding in the SSM. The generated sequence is then fed into the SSM for global dependency modeling to obtain branch sequence features. :
[0023] The corresponding adaptive weights are generated using Softmax. :
[0024] Multiplying the two elements one by one yields the weighted feature. :
[0025] Similarly, another forward scan branch and two other reverse scan branches are obtained, and the weighted features are obtained accordingly. , and The weighted features of the four branches , , and Summing element by element yields the fusion characteristics. .
[0026] Fusion features The spatial features of the output are obtained after normalization. ,Will With gated tensors activated by GELU Element-wise multiplication is performed to achieve the interaction and aggregation of spatial features:
[0027] in This indicates element-wise multiplication.
[0028] For spectral feature extraction, please refer to [link / reference]. Figure 3 This is a structural diagram of the bidirectional scanning spectroscopy Mamba module of the present invention.
[0029] The extraction of spectral features involves mapping the features to an intermediate dimension via linear projection, and then dividing them into feature tensors. With gate tensor Feature tensor Local feature enhancement is achieved through the DWConv and SiLU activation functions.
[0030] The enhanced features are reshaped into a spectral dimension sequence, and a bidirectional spectral sequence with forward and reverse directions is constructed. This sequence is then input into a selective state-space model to model global spectral dependencies. Normalized and gated tensors activated by GELU Perform element-wise multiplication, and then map back to the original feature dimension through linear projection.
[0031] S4: Feature fusion, please refer to Figure 4 This is a structural diagram of the adaptive enhancement fusion module of the present invention.
[0032] For spectral features, a channel attention mechanism is used for further feature extraction. First, global average pooling is performed on the input spectral features to compress the spatial dimension and highlight global spectral information. Then, nonlinear mapping is performed through consecutive fully connected layers to generate feature vectors with global spectral awareness, thereby capturing long-range dependencies across spectral channels.
[0033] For spatial features, depthwise separable convolution is used to perform lightweight enhancement on the spatial feature map to adapt to the weight allocation of the subsequent gating mechanism.
[0034] Spatial features and enhanced spectral features are concatenated along the channel dimension, and gating weights are generated using a two-layer feedforward network FFN and Softmax. And split into spatial branch gating With spectral branching gate First, calculate the global mean of the gating weights and multiply it by a coefficient of 0.5 to obtain the dynamic threshold. Then perform hard threshold filtering on the gating weights:
[0035]
[0036] Based on filtered gate weights and Element-wise weighted fusion of spatial and spectral features is performed.
[0037] S5: Perform residual connections between the fused spectral and spatial features and the normalized embedded features to complete the feature update of the Mamba layer; S6: After all Mamba layers have completed forward propagation, the features are first aggregated in global spatial dimensions through adaptive average pooling, and then normalized and flattened to obtain global aggregated features. The global aggregated features are then fed into the linear classification head to output the category prediction results. To illustrate the technical superiority of the method of this invention, we compare it with other methods. The comparison methods include: 2D-CNN, SpectralFormer, Morphformer, SSFTT, CTMixer, MiM, and SS Mamba. This invention uses five commonly used evaluation metrics: Overall Accuracy (OA), Average Accuracy (AA), kappa coefficient (K), number of parameters, and floating-point operations (FLOPs) for evaluation. OA represents the proportion of correctly classified samples out of the total number of test samples, reflecting the overall classification performance of the model. AA represents the average accuracy of all individual categories. Kappa reflects the consistency between the predicted label and the true label of the land cover, comprehensively considering the impact of correctly and incorrectly classified land covers. Parameters represent the total number of learnable parameters of the model, and FLOPs count the number of floating-point operations during inference. The performance data of the method of this invention and other methods on the Indian Pines dataset are shown in Table 1. It can be seen that the method of this invention is significantly superior to other existing methods, significantly improving classification accuracy while greatly reducing computational requirements, making it an efficient and competitive solution.
[0038] Table 1. Performance data of the method of this invention and other methods on the Indian Pines dataset. Method Parameters(K) FLOPs(M) OA (%) AA (%) K(×100) 2D-CNN 537.87 68.53 75.89 86.64 72.81 SpectralFormer 407.96 20.77 85.69 93.18 83.79 morphformer 278 41.78 94.46 96.80 93.65 SSFTT 875.22 59.25 96.11 97.92 95.55 CTMixer 617 30 94.88 98.00 94.15 MiM 217.36 139.4 87.20 86.83 85.41 SS Mamba 360.62 30.89 92.94 96.37 92.00 Ours 96.35 13.7 98.38 99.23 98.14
Claims
1. A hyperspectral image classification method based on multi-scale spiral scanning spatial-spectral dual-branch Mamba, characterized in that, Includes the following steps: S1: Data input and preprocessing: Taking each pixel in the input hyperspectral image as the center, the spatial neighborhood is cropped according to different image patch sizes P to obtain hyperspectral image patches; S2: Feature embedding, using 1×1 convolution to map hyperspectral image patches to embedded features; S3: Spatial-spectral feature extraction. The embedded features are normalized and then fed into the Mamba layer. This layer consists of two parallel Mamba blocks, corresponding to the spatial branch and the spectral branch, respectively. The two branches independently complete feature modeling and extraction. The spatial branch divides the features into feature tensors and gate tensors. After local modeling is completed by depth convolution, the feature tensors are input into the multi-scale spiral state space model MS-SSM. After normalization, they are multiplied element-wise by the GELU-activated gate tensor and linearly projected back to the original dimension to obtain the spatial features. The MS-SSM module extracts multi-scale features through 3×3 and 5×5 parallel deep convolutional branches, generates sequential features through spiral scanning and models them with a state space model, and then generates adaptive weights through Softmax to complete gating and multi-scale aggregation. In particular, the combination of multi-scale deep convolutional layers and spiral scanning extracts multi-scale features while using spiral scanning to ensure spatial context information and reduce the impact of noise and mixed pixels. Spiral scanning traverses the neighborhood from the center pixel to the outside, which can preserve the relative position information of pixels to the greatest extent compared with conventional scanning. The spectral branch linearly projects the features to the intermediate dimension and segments them. The feature tensor is reshaped into a spectral dimension sequence after being enhanced by depth convolution and SiLU activation. A bidirectional input state space model with forward and reverse sequences is constructed to model global dependencies. After normalization, the gating tensor activated by GELU is multiplied element-wise and linearly projected back to the original dimension to obtain the spectral features. S4: Feature Fusion. The generated spatial and spectral features are input into the Adaptive Enhancement Fusion Module (AEFM). The spatial features are further extracted using depthwise separable convolution, and the spectral features are enhanced using a channel attention mechanism. The two types of processed features are concatenated and input into a feedforward neural network. After Softmax activation, two fusion weights with a sum of 1 are generated. The global mean of the fusion weights is calculated, and a dynamic threshold is constructed using a preset ratio of this mean. Weights below the threshold are reset to zero, while weights above the threshold are retained. The weights are multiplied by their corresponding features and then added together to achieve adaptive weighted fusion of spatial and spectral features. S5: Feature update, which performs residual connection between the fused spectral and spatial features and the normalized embedded features to complete the feature update of the Mamba layer; S6: Classification prediction. After all Mamba layers have completed forward propagation, the features are first aggregated globally in terms of spatial dimensions using adaptive average pooling, and then normalized and flattened to obtain the global aggregated features. The global aggregated features are then fed into the linear classification head to output the class prediction results.