Hyperspectral image classification method based on state space model

Abstract

This paper proposes a hyperspectral image classification method based on a state-space model, which primarily addresses the poor classification accuracy of existing methods. The solution includes: 1) inputting a hyperspectral image and preprocessing it; 2) using a neighborhood spectral fusion method to eliminate spectral variability, and then using a subspectral scanning method to perceive differences in spectral details and obtain image spectral classification information; 3) constructing a spatial feature extraction network using an SS2D layer, an improved spatial attention mechanism, and multiple residual blocks, and using this network to obtain spatial classification information for the hyperspectral image; 4) obtaining the final classification result by fusing spatial and spectral classification information at the decision layer. This method can reduce the impact of spectral variability within a relatively small range, not neglecting local spectral information while being able to perceive comprehensive spectral information, and simultaneously combining spatial information to effectively improve the classification of hyperspectral images.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and machine learning technology, and further relates to hyperspectral image classification, specifically a hyperspectral image classification method based on a state-space model, which can be applied to fields such as environmental monitoring and civil affairs. Background Art

[0002] Hyperspectral image classification is a key task in hyperspectral image processing. Its goal is to accurately classify different ground cover types at the pixel level to guide production practices. This technology is widely used in civil engineering, environmental monitoring, and agriculture. Among the currently popular hyperspectral image classification methods, those based on deep learning have attracted considerable attention and achieved impressive performance. These methods all learn and classify the features of hyperspectral image pixels from a small, carefully selected training set.

[0003] Existing techniques have introduced the traditional KNN algorithm into the field of hyperspectral image classification, classifying images at the spectral level, and using random forests to extract spectral information for classification. While these approaches have improved classification performance to some extent, they focus solely on spectral information and ignore spatial information, limiting their classification performance due to spectral variability. With the further development of related technologies, 2DCNN [X. Zhu, D. Cheng, Z. Zhang, S. Lin, and J. Dai, “An Empirical Study of Spatial Attention Mechanisms in Deep Networks,” in 2019 IEEE / CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), October 2019] has been proposed, combining spatial and spectral information for hyperspectral classification, achieving promising results. Furthermore, 3DCNN has been proposed, utilizing a combination of spatial and spectral information for hyperspectral image classification, achieving state-of-the-art results in this field. However, 3DCNN suffers from a limited receptive field and cannot focus on global spectral details, leaving room for improvement in its classification performance.

[0004] The paper CasRNN [R. Hang, Q. Liu, D. Hong, and P. Ghamisi, “Cascaded Recurrent Neural Networks for Hyperspectral Image Classification,” IEEE Transactions on Geoscience and Remote Sensing, pp. 5384–5394, Aug. 2019] innovatively used RNN for spectral classification and achieved good results. SpectralFormer[D.Hong et al.,“SpectralFormer:Rethinking Hyperspectral Image Classification with transformers,”IEEE Transactions on Geoscience and Remote Sensing,pp.1–15,Jan.2022] and GraphGST[M.Jiang et al.,"GraphGST:Graph Generative Structure-Aware transformer forHyperspectral Image Classification,"in IEEE Transactions on Geoscience andRemote Sensing,vol.62,pp.1-16,2024,Art no.5504016] use Transformer to perform hyperspectral image classification. Although the effect has been improved to a certain extent, the problem of high computational complexity exists, which makes its computational efficiency low and limits its application scenarios.

[0005] Therefore, in the hyperspectral image classification task, how to improve computational efficiency while ensuring classification accuracy is a question worth exploring. Summary of the Invention

[0006] In view of the problems existing in the above-mentioned hyperspectral image classification methods, the present invention proposes a hyperspectral image classification method based on a state space model, which introduces the state space model (SSM) into the hyperspectral image classification task to reduce the computational complexity and improve the classification performance; a hyperspectral image classification model (Spectral-Spatial Unified Mamba, SSUM) is constructed by two branches: a network for perceiving spatial information (Spatial Mamba branch) and a network for perceiving spectral information (Spectral Mamba branch), aiming to extract hyperspectral image features from both spectral and spatial perspectives; the network for perceiving spectral information consists of two parts: a neighbor spectrum fusion strategy (Nearest-neighbor spectrum fusion, NSF) and a sub-spectrum scanning strategy (Sub-spectrum scanning, SS), which can reduce the impact of spectral variability in a smaller range, and perceive local spectral information by dividing the spectrum into small segments while being able to perceive comprehensive spectral information; in the network branch for perceiving spatial information, a two-dimensional selective scanning module (2D SelectiveScan Module, SS2D) and a spatial attention module (Spatial Attention Module, SS2D) are combined to form a spectral classification model. Attention (SA) is combined to fully extract the spatial features of HSI; finally, the output features of the two branches are combined to obtain the classification result. This invention can effectively improve the classification performance of hyperspectral images.

[0007] The present invention achieves the above-mentioned purpose by the following specific steps:

[0008] (1) Given a hyperspectral image X HSI ∈R W×H×B , where W, H and B represent the width, height and number of spectral bands of the image respectively; and HSI Perform dimensionality reduction processing to obtain the reduced-dimensional image X PCA ; Assume that the pixel to be classified is located at point (i, j), take the spectral value x at point (i, j) center ∈R 1×1×B , a small-scale full-band image block X around point (i, j) with a length and width of P patch1 ∈R P×P×B , and X PCA A large-scale image block X with a length and width of N at the midpoint (i, j) patch2 ∈R N×N×n ;

[0009] (2) Use the neighborhood spectral fusion method to eliminate the interference of spectral variability on the classification of hyperspectral image pixels. The implementation steps are as follows:

[0010] (2.1) Calculate the vector x at point (i, j) center and the average value x of its surrounding pixels avg ;

[0011] (2.2) Calculate x using Einstein summation center and x avg Fused spectral vector x fus ;

[0012] (2.3) x fus Input into the first state space model S6 for feature extraction, and connect through the residual layer to obtain the first spectral feature F that integrates the small-scale spatial features nsf ;

[0013] (3) Use the sub-spectral scanning method to perceive the spectral detail differences of hyperspectral image pixels and obtain spectral classification information according to the following steps:

[0014] (3.1) for vector x center Split and reorganize to obtain the reorganized two-dimensional tensor x' center , and then sent it to the second state space model S6 for feature extraction to obtain the second spectral feature F ss ;

[0015] (3.2) The feature F ss With F nsf After addition, the data is sent to the linear layer for processing spectral information to obtain spectral classification information;

[0016] (4) Construct a spatial feature extraction network and use it to obtain spatial classification information of hyperspectral images:

[0017] (4.1) The first part of the network is composed of the SS2D layer and the improved spatial attention mechanism SA. The improved spatial attention mechanism SA adds residual links to the original spatial attention mechanism to make the input and output image blocks of the same size. The second part of the network is then composed of the SS2D layer of the first part and a layer normalization LayerNorm operation. Finally, the third part of the network is composed of a Linear layer for processing the original spatial information and the activation function SiLU. These three parts together constitute the spatial feature extraction network.

[0018] (4.2) X patch2 As input to the spatial feature extraction network, the spatial features of the hyperspectral image are obtained; and the spatial features are fed into a linear layer for processing the fusion of the three spatial information to obtain spatial classification information;

[0019] (5) The spatial classification information and the spectral classification information are spliced ​​and sent to the multi-layer perceptron (MLP) for decision-layer feature fusion to obtain the final classification result of the combined spatial information and spectral information.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] First, since the method of hyperspectral image classification based on the state space model (SSM) of the present invention has a faster detection speed compared with the method based on Transformer; compared with the method based on CNN, it can focus on more comprehensive spectral information.

[0022] Secondly, the present invention proposes a neighborhood spectrum fusion method (Nearest-neighbor spectrum fusion, NSF), which can suppress the spatial variability problem of hyperspectral images in a small range while retaining all spectral information; at the same time, a sub-spectrum scanning method (SS) is proposed, which can divide the hyperspectral image pixels into several sub-spectra, use the state space model to focus on their details, and enhance the classification performance from the spectral dimension.

[0023] Third, the present invention constructs a spatial information extraction network (Spatial Mamba branch, SM), introduces the SS2D model into the hyperspectral image classification task, and uses the long-distance modeling ability of SS2D in images to extract a wide range of spatial features of hyperspectral images; at the same time, because the present invention connects an improved spatial attention mechanism to the SS2D when constructing the model, it can significantly improve the spatial feature extraction ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the implementation process of the method of the present invention; wherein a is the overall flow chart of the present invention, b is the spatial information extraction network (Spatial Mamba branch, SM) architecture diagram of the present invention, c is a schematic diagram of the neighbor spectrum fusion method (Nearest-neighbor spectrum fusion, NSF), d is a schematic diagram of the sub-spectrum scanning method (SS), and e is a schematic diagram of the 2D selective scanning method (2D Selective Scan Module, SS2D);

[0025] Figure 2 This is a flowchart of the improved spatial attention mechanism SA (Self-Attention) in the present invention;

[0026] Figure 3 The figure compares the classification effects of the present invention and the existing methods on the Pavia University dataset; (a)-(l) are respectively the classification result diagrams of the existing methods ground truth, KNN, RF, 1DCNN, 2DCNN, HybridSN, IRTS-3DCNN, CasRNN, ViT, SpectralFormer, GraphGST and the method SSUM of the present invention on the Pavia University dataset.

[0027] Figure 4 Comparison of the classification effects of the present invention and existing methods on the Salinas Valley dataset; (a)-(l) are respectively the classification result diagrams of the existing methods ground truth, KNN, RF, 1DCNN, 2DCNN, HybridSN, IRTS-3DCNN, CasRNN, ViT, SpectralFormer, GraphGST and the present invention method SSUM on the Salinas Valley dataset. DETAILED DESCRIPTION

[0028] The present invention will be further described below with reference to the accompanying drawings.

[0029] Example 1: Refer to the attached Figure 1 The present invention proposes a hyperspectral image classification method based on a state space model, which specifically includes the following steps:

[0030] Step 1. Given a hyperspectral image X HSI ∈R W×H×B , where W, H and B represent the width, height and number of spectral bands of the image respectively; and HSI Perform dimensionality reduction processing to obtain the reduced-dimensional image X PCA ; Assume that the pixel to be classified is located at point (i, j), take the spectral value x at point (i, j) center ∈R 1×1×B , a small-scale full-band image block X around point (i, j) with a length and width of P patch1 ∈R P×P×B , and X PCA A large-scale image block X with a length and width of N at the midpoint (i, j) patch2 ∈R N×N×n ;

[0031] Step 2. Use the neighborhood spectral fusion method to eliminate the interference of spectral variability on the classification of hyperspectral image pixels. The implementation steps are as follows:

[0032] (2.1) Calculate the vector x at point (i, j) centerand the average value x of its surrounding pixels avg In this embodiment, the average value x is calculated according to the following formula: avg :

[0033] X patch1 =[x0,x1,...,x i ,...,x n ],

[0034]

[0035] Where n = P × P represents X patch1 The total number of pixels in .

[0036] (2.2) Calculate x using Einstein summation center and x avg Fused spectral vector x fus ;

[0037] (2.3) x fus Input into the first state space model S6 for feature extraction, and connect through the residual layer to obtain the first spectral feature F that integrates the small-scale spatial features nsf :

[0038] F nsf =S6(x fus )+x fus .

[0039] Step 3. Use the subspectral scanning method to perceive the spectral detail differences of the hyperspectral image pixels and obtain spectral classification information according to the following steps:

[0040] (3.1) for vector x center Split and reorganize to obtain the reorganized two-dimensional tensor x' center , and then sent it to the second state space model S6 for feature extraction to obtain the second spectral feature F ss ; In this embodiment, vector x center Split and reorganize, specifically by taking the input vector x center The method is to divide it into m sub-spectra of length 10 and then combine them.

[0041] (3.2) The feature F ss With F nsf After addition, the data is sent to the linear layer for processing spectral information to obtain spectral classification information;

[0042] Step 4. Build a spatial feature extraction network and use it to obtain spatial classification information of hyperspectral images:

[0043] (4.1) The first part of the network is composed of the SS2D layer and the improved spatial attention mechanism SA. The SS2D layer is a selective scanning network built on the state-space model. The improved spatial attention mechanism SA adds residual links to the original spatial attention mechanism to make the input and output image blocks of the same size. The second part of the network is then composed of the SS2D layer of the first part and a layer normalization LayerNorm operation. Finally, a linear layer Linear for processing the original spatial information and the activation function SiLU are used to form the third part of the network. These three parts together constitute the spatial feature extraction network.

[0044] (4.2) X patch2 As the input of the spatial feature extraction network, the spatial features of the hyperspectral image are obtained; and the spatial features are sent to the linear layer for processing the fusion of the three spatial information to obtain spatial classification information. In this embodiment, the spatial features of the hyperspectral image are obtained according to the following steps:

[0045] (4.2.1) X patch2 As the input of the first part, after the SS2D layer completes the operation, its output is sent to the two branches respectively. The first branch obtains the first output feature F through the improved spatial attention mechanism in the first part. mamba The second branch obtains the second output feature F through the layer normalization LayerNorm operation of the second part ln :

[0046] F mamba =SA[SS2D(X patch2 )],

[0047] F ln =LayerNorm[SS2D(X patch2 )];

[0048] (4.2.2) X patch2 As the input of the third part, the third feature F is obtained through the linear layer Linear and the activation function SiLU. PCA :

[0049] F PCA =SiLU[Linear(X patch2 )],

[0050] (4.2.3) The features F extracted from the three parts of the network mamba 、F ln and F PCA The spatial characteristics of the hyperspectral image are obtained by adding them together.

[0051] Step 5. After splicing the spatial classification information and the spectral classification information, they are sent to the multi-layer perceptron (MLP) for decision-layer feature fusion to obtain the final classification result of the combined spatial information and spectral information.

[0052] Example 2: The overall implementation steps of the image classification method proposed in the present invention are the same as those of Example 1. Figure 1 and 2 , the neighborhood spectrum fusion, sub-spectrum scanning method, and the construction and training process of the spatial information extraction network model in the present invention are further explained, and the implementation process of the method of the present invention is described in detail:

[0053] Step 1: Data input preparation

[0054] Given a hyperspectral image X HSI ∈R W×H×B , where W, H, and B represent the width, height, and number of spectral bands of the image respectively. First, HSI Perform dimensionality reduction processing to obtain the reduced-dimensional image X PCA .

[0055] X PCA =PCA(X HSI ) (1)

[0056] For each pixel of the hyperspectral image, we need to classify it. Assume that the pixel to be classified is located at point (i, j). We need to obtain the spectral value x at point (i, j) center ∈R 1×1×B , a small-scale full-band image block X around point (i, j) patch1 ∈R P×P×B , and X PCA The large-scale image block X at point (i, j) patch2 ∈R N×N×n .

[0057] Step 2: Nearest-neighbor spectrum fusion (NSF)

[0058] Reference Figure 1 In c, the neighborhood spectral fusion method can overcome the interference of spectral variability on the classification of hyperspectral image pixels, as follows:

[0059] First, the vector x at point (i, j) center and the average value x of the surrounding pixels avg Obtained by the following formula:

[0060] X patch =[x0,x1,x2.....x n ] (2)

[0061]

[0062] Where n = P × P.

[0063] Second, the use of x center and x avg Calculate the fused spectral vector x fus , using Einstein summation, the formula is as follows:

[0064] x fus =einsum(x center ,x avg ) (4)

[0065] Third, x fus Input into the S6 model for feature extraction, connected through the residual layer, and the obtained feature F nsf Expressed as:

[0066] F nsf =S6(x fus )+x fus (5)

[0067] Step 3: Sub-spectrum scanning (SS)

[0068] Reference Figure 1 In d, a sub-spectral scanning method is used to perceive the spectral detail differences of hyperspectral image pixels, as follows:

[0069] First, for the input x center , this embodiment divides it into sub-spectra of several lengths, and then reassembles them into a new two-dimensional tensor, where m is the number of divisions. It is then sent to S6, another size different from that of step 2, for feature extraction. The formula is as follows:

[0070] x' center =reshape(x center ) (6)

[0071] F ss =reshape[S6(x' center )] (7)

[0072] Second, F ss With F nsf After addition, they are fed into a linear layer to obtain spectral classification information.

[0073] Step 4: Spatial Information Extraction Network (Spatial Mamba branch, SM)

[0074] Reference Figure 1b in X PCA Get X patch2 ∈R N×N×n Used to extract large-scale spatial features. The details are as follows:

[0075] First, the spatial information extraction network mainly consists of SS2D layers, an improved spatial attention mechanism and several residual blocks. The formula is as follows:

[0076] F PCA =SiLU[Linear(X patch )] (8)

[0077] F mamba =SA[SS2D(X patch )] (9)

[0078] F ln =LayerNorm[SS2D(X patch )] (10)

[0079] Where SiLU represents the activation function and SA represents the improved spatial self-attention mechanism, such as Figure 2 As shown in the figure; LayerNrom represents layer normalization. Second, the three features are summed and passed through a linear layer to obtain spatial classification information. To ensure uniform data size during network transmission, a layer is added after the standard spatial attention mechanism. The spatial feature map is expanded to the original image size and then multiplied with the original image.

[0080] Finally, the obtained spatial classification information is concatenated with the spectral classification information obtained in step three and sent to an MLP layer for decision layer feature fusion to obtain the final classification result of the combined spatial information and spectral information.

[0081] Step 5: Model training

[0082] The training details of the spatial-spectral combined with hyperspectral image classification method based on the state-space model are as follows:

[0083] First, scan each point of the hyperspectral image and execute step 1 to obtain the x value of each pixel. center 、X patch1 and X patch2 Second, execute step 2 to obtain F that integrates the neighborhood spatial spectrum information. nsf Third, execute step 3 to obtain spectral classification information. Fourth, execute step 4 to obtain the classification result of the joint spatial information and spectral information. Fifth, calculate the loss function by combining the classification result and the label:

[0084]

[0085] Where, y=[y0,y1,y2.....y n ] is the true label represented as a one-hot vector, z=[z0,z1,z2.....z n ] is the predicted probability after the softmax operation. The sum is taken over all classes, and the logarithm is applied element-wise to the predicted probability. This loss function quantifies the gap between the estimated distribution and the actual distribution, penalizing the model for incorrect predictions. Finally, the model is backpropagated based on the loss function results to adjust the parameters.

[0086] Step 6: Image Classification of the Model

[0087] First, scan each point of the hyperspectral image and execute step 1 to obtain the x value of each pixel. center 、X patch1 and X patch2 . Second, x center Send SS, X patch1 The features extracted by the two are added and fused to obtain the spectral classification features. patch2 The spatial classification features are sent to SM to obtain the spatial classification features, which are finally fused with the spectral classification features obtained in the second step to obtain the final classification results.

[0088] The effect of the present invention will be further described below in conjunction with experiments:

[0089] 1. Experimental conditions:

[0090] The experiments of the present invention were carried out in a hardware environment equipped with an NVIDIA GeForce RTX 2080 graphics card with 12GB of video memory, a Windows 10 operating system, and a software environment of Python 3.7.

[0091] 2. Experimental content:

[0092] The proposed method is qualitatively and quantitatively compared with ten popular algorithms on two public datasets for hyperspectral image classification. The public hyperspectral anomaly detection datasets used in the experiment include Pavia University and Salinas Valley; the ten popular algorithms compared are: KNN[W.Li,Q.Du,F.Zhang,and W.Hu,“Collaborative representation based k-nearest neighbor classifier for hyperspectral image-agery,”in 2014 6th Workshop on Hyperspectral Image and Signal Processing:Evolution in Remote Sensing(WHISPERS),Lausanne,Switzerland,Jun.2014.], RF[S.Amini,S.Homayouni,A.Safari,and AADarvishsefat,“Object-based classification of hyperspectral data using Random Forest algorithm,”Geo-spatial Information Science,pp.127–138,Apr.2018], 1DCNN, 2DCNN[X.Zhu,D.Cheng,Z.Zhang,S.Lin,and J.Dai,“An Empirical Study of Spatial Attention Mechanisms in Deep Networks," in 2019IEEE / CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), Oct.2019], HybridSN [SKRoy, G.Krishna, SRDubey, and B.B. Chaudhuri, "HybridSN: Exploring 3-D–2-D CNN Feature Hierarchy for Hyper-spectral Image Classification,” IEEE Geoscience and Remote Sensing Letters, pp.277–281, Feb. 2020], IRTS-3DCNN [C.-I.Chang, KYMa, C.-C., Liang, Y.-M., Kuo, S., Chen, S., & Zhong, S. "Iterative Random Training Sampling Spectral Spatial Classification for Hyperspectral Images," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 13, pp. 3986–4007, Jan. 2020], CasRNN [R. Hang, Q. Liu, D. Hong, & P. Ghamisi, "Cascaded Recurrent Neural Networks for Hyperspectral Image Classification," IEEE Transactions on Geoscience and Remote Sensing, pp. 5384–5394, Aug. 2019], VIT [J. Li, K. Zheng, W. Liu, Z. Li, H. Yu and L. Ni, "Model-Guided Coarse-to-Fine Fusion Network for Unsupervised Hyperspectral Image Super-Resolution," in IEEE Geoscience and Remote Sensing Letters, vol. 20, pp. 1-5, 2023, Art no. 5508605], SpectralFormer [D. Hong et al., "SpectralFormer: Rethinking Hyperspectral Image Classification with transformers," IEEE Transactions on Geoscience and Remote Sensing, pp. 1–15, Jan. 2022], and GraphGST [M. Jiang et al.,"GraphGST: Graph Generative Structure-Awaretransformer for Hyperspectral Image Classification," in IEEE Transactions onGeoscience and Remote Sensing, vol.62, pp.1-16, 2024, Art no.5504016]. .

[0093] Table 1. Comparison of classification results on the Pavia University dataset.

[0094]

[0095] Table 2. Comparison of classification results on the Salinas dataset.

[0096]

[0097]

[0098] This quantitative comparison of the present invention with ten popular comparison algorithms on two datasets shows the pixel category in the first column, and the remaining columns indicate the classification accuracy of the corresponding method for a particular category. The evaluation metrics used are Overall Accuracy (OA), Average Accuracy (AA), and Kappa Coefficient (κ). These three metrics assess the accuracy of the algorithm from different perspectives, with larger values ​​indicating higher detection accuracy. As shown in Tables 1 and 2, the present invention achieves the best detection accuracy.

[0099] The above simulation analysis proves the correctness and effectiveness of the method proposed in the present invention.

[0100] Parts of the present invention that are not described in detail belong to common knowledge among those skilled in the art.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, for professionals in this field, after understanding the content and principles of the present invention, they may make various modifications and changes in form and details without departing from the principles and structure of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A hyperspectral image classification method based on a state space model, characterized in that: The steps include: (1) Given a hyperspectral image X HSI ∈R W×H×B , where W, H and B represent the width, height and number of spectral bands of the image respectively; and HSI Perform dimensionality reduction processing to obtain the reduced-dimensional image X PCA ; Assume that the pixel to be classified is located at point (i, j), take the spectral value x at point (i, j) center ∈R 1×1×B , a small-scale full-band image block X around point (i, j) with a length and width of P patch1 ∈R P×P×B , and X PCA A large-scale image block X with a length and width of N at the midpoint (i, j) patch2 ∈R N×N×n ; (2) Use the neighborhood spectral fusion method to eliminate the interference of spectral variability on the classification of hyperspectral image pixels. The implementation steps are as follows: (2.1) Calculate the vector x at point (i, j) center and the average value x of its surrounding pixels avg ; (2.2) Calculate x using Einstein summation center and x avg Fused spectral vector x fus ; (2.3) x fus Input into the first state space model S6 for feature extraction, and connect through the residual layer to obtain the first spectral feature F that integrates the small-scale spatial features nsf ; (3) Use the sub-spectral scanning method to perceive the spectral detail differences of hyperspectral image pixels and obtain spectral classification information according to the following steps: (3.1) for vector x center Split and reorganize to obtain the reorganized two-dimensional tensor x' center , and then sent it to the second state space model S6 for feature extraction to obtain the second spectral feature F ss ; (3.2) The feature F ss With F nsf After addition, the data is sent to the linear layer for processing spectral information to obtain spectral classification information; (4) Construct a spatial feature extraction network and use it to obtain spatial classification information of hyperspectral images: (4.1) The first part of the network is composed of the SS2D layer and the improved spatial attention mechanism SA. The improved spatial attention mechanism SA adds residual links to the original spatial attention mechanism to make the input and output image blocks of the same size. The second part of the network is then composed of the SS2D layer of the first part and a layer normalization LayerNorm operation. Finally, the third part of the network is composed of a Linear layer for processing the original spatial information and the activation function SiLU. These three parts together constitute the spatial feature extraction network. (4.2) X patch2 As input to the spatial feature extraction network, the spatial features of the hyperspectral image are obtained; and the spatial features are fed into a linear layer for processing the fusion of the three spatial information to obtain spatial classification information; (5) The spatial classification information and the spectral classification information are spliced ​​and sent to the multi-layer perceptron (MLP) for decision-layer feature fusion to obtain the final classification result of the combined spatial information and spectral information.

2. The method according to claim 1, wherein: The vector x at the point (i, j) in step (2.1) center and the average value x of its surrounding pixels avg , calculated according to the following formula: X patch1 =[x0,x1,...,x i ,...,x n ], Where n = P × P represents X patch1 The total number of pixels in .

3. The method according to claim 1, wherein: Step (2.3) the first spectral feature F nsf Specifically, it is obtained by combining the state space model S6 with the residual network according to the following formula: F nsf =S6(x fus )+x fus 。 4. The method according to claim 1, wherein: Step (3.1) of the vector x center Split and reorganize, specifically the input vector x center Split into m sub-spectra of length 10, and then recombine the split sub-spectra to obtain a new two-dimensional tensor x' center .

5. The method according to claim 1, wherein: The SS2D layer described in step (4.1) is a selective scanning network built on the basis of the state space model.

6. The method according to claim 1, wherein: The spatial features of the hyperspectral image in step (4.2) are obtained specifically according to the following steps: (4.2.1) X patch2 As the input of the first part, after the SS2D layer completes the operation, its output is sent to the two branches respectively. The first branch obtains the first output feature F through the improved spatial attention mechanism in the first part. mamba The second branch obtains the second output feature F through the layer normalization LayerNorm operation of the second part ln : F mamba =SA[SS2D(X patch2 )], F ln =LayerNorm[SS2D(X patch2 )]; (4.2.2) X patch2 As the input of the third part, the third feature F is obtained through the linear layer Linear and the activation function SiLU. PCA : F PCA =SiLU[Linear(X patch2 )], (4.2.3) The features F extracted from the three parts of the network mamba 、F ln and F PCA The spatial characteristics of the hyperspectral image are obtained by adding them together.

Images