Hyperspectral image classification method based on state space model

Abstract

The invention provides a hyperspectral image classification method based on a state space model. The problem that an existing method is poor in classification precision is mainly solved. According to the scheme, the method comprises the following steps: 1) inputting a hyperspectral image, and preprocessing the hyperspectral image; 2) eliminating spectrum variability by using a neighborhood spectrum fusion method, sensing spectrum detail differences by using a sub-spectrum scanning method, and obtaining image spectrum classification information; 3) constructing a spatial feature extraction network by using an SS2D layer, an improved spatial attention mechanism and a plurality of residual blocks, and acquiring spatial classification information of the hyperspectral image by using the network; and 4) fusing space and spectrum classification information at a decision layer to obtain a final classification result. According to the method, the influence caused by spectrum variability can be reduced in a relatively small range, local spectrum information is not ignored on the premise that comprehensive spectrum information can be perceived, and spatial information is combined, so that the classification effect of the hyperspectral image is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and machine learning, and further relates to hyperspectral image classification, specifically a hyperspectral image classification method based on a state space model, which can be applied to environmental monitoring, civil and other fields. Background Art

[0002] Hyperspectral image classification is one of the important tasks in hyperspectral image processing. Its purpose is to accurately divide different surface covers at the pixel level to further guide production practice. This technology is widely used in civil, environmental monitoring and agricultural fields. In recent years, among the existing popular hyperspectral image classification methods, hyperspectral image classification methods based on deep learning have attracted much attention from researchers and achieved satisfactory performance. These methods all learn the features of hyperspectral image pixels from a small number of selected training sets and classify them.

[0003] In the existing technology, the traditional KNN algorithm is introduced into the field of hyperspectral image classification, and it is classified from the spectral information level, and the spectral information is extracted by random forest for classification. Although the classification effect is improved to a certain extent, it only focuses on the spectral information and ignores the spatial information. Due to the influence of spectral variability, its classification performance is limited. With the further development of related technologies, 2DCNN [X.Zhu,D.Cheng,Z.Zhang,S.Lin,and J.Dai, "An Empirical Study of Spatial Attention Mechanisms in Deep Networks," in 2019IEEE / CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), Oct. 2019] uses 2DCNN to combine spatial information and spectral information to perform hyperspectral classification tasks, and has achieved good results; and a method of using 3DCNN to classify hyperspectral images using spatial and spectral combination has been proposed, which has achieved the best results in this field. However, 3DCNN has a limited receptive field and cannot pay attention to global spectral detail information, and its classification performance still has room for improvement.

[0004] The literature CasRNN [R. Hang, Q. Liu, D. Hong, and P. Ghamisi, "Cascaded Recurrent Neural Networks for Hyperspectral Image Classification," IEEE Transactions on Geoscience and Remote Sensing, pp. 5384–5394, Aug. 2019] innovatively used RNN for spectral classification and achieved good results. SpectralFormer[D.Hong et al.,“SpectralFormer:Rethinking Hyperspectral Image Classification with transformers,”IEEETransactions on Geoscience and Remote Sensing,pp.1–15,Jan.2022] and GraphGST[M.Jiang et al.,"GraphGST:Graph Generative Structure-Aware transformer forHyperspectral Image Classification,"in IEEE Transactions on Geoscience andRemote Sensing,vol.62,pp.1-16,2024,Art no.5504016] use Transformer to perform hyperspectral image classification. Although the effect has been improved to a certain extent, there is a problem of high computational complexity, which makes its computational efficiency low and limits its application scenarios.

[0005] Therefore, in the hyperspectral image classification task, how to improve computational efficiency while ensuring classification accuracy is a question worth exploring. Summary of the invention

[0006] In view of the problems existing in the above-mentioned hyperspectral image classification method, the present invention proposes a hyperspectral image classification method based on a state space model, and introduces a state space model (SSM) into the hyperspectral image classification task to reduce the computational complexity and improve the classification performance; a hyperspectral image classification model (Spectral-Spatial Unified Mamba, SSUM) is constructed by two branches, namely a network for perceiving spatial information (Spatial Mamba branch) and a network for perceiving spectral information (Spectral Mamba branch), aiming to extract hyperspectral image features from both spectral and spatial perspectives; the network for perceiving spectral information consists of a neighbor spectrum fusion strategy (Nearest-neighbor spectrum fusion, NSF) and a sub-spectrum scanning strategy (Sub-spectrum scanning, SS), which can reduce the impact of spectral variability in a smaller range, and perceive local spectral information by dividing the spectrum into small segments under the premise of being able to perceive comprehensive spectral information; in the network branch for perceiving spatial information, a two-dimensional selective scanning module (2D SelectiveScan Module, SS2D) and a spatial attention module (Spatial Attention, SA) is combined to fully extract the spatial features of HSI; finally, the output features of the two branches are combined to obtain the classification result. The present invention can effectively improve the classification performance of hyperspectral images.

[0007] The present invention achieves the above-mentioned purpose by the following specific steps:

[0008] (1) Given a hyperspectral image X HSI ∈R W×H×B , where W, H and B represent the width, height and number of spectral bands of the image respectively; and for X HSI Perform dimensionality reduction processing to obtain the reduced-dimensional image X PCA ; Assume that the pixel to be classified is located at point (i, j), take the spectral value x at point (i, j) center ∈R 1×1×B , a small-scale full-band image block X with a length and width of P around point (i, j) patch1 ∈R P×P×B , and X PCA A large-scale image block X with a length and width of N at the midpoint (i, j) patch2 ∈R N×N×n ;

[0009] (2) The neighborhood spectral fusion method is used to eliminate the interference of spectral variability on the classification of hyperspectral image pixels. The implementation steps are as follows:

[0010] (2.1) Calculate the vector x at point (i, j) center and the average value x of its surrounding pixels avg ;

[0011] (2.2) Using Einstein summation to calculate x center and x avg The fused spectral vector x fus ;

[0012] (2.3) x fus Input into the first state space model S6 for feature extraction, and connect through the residual layer to obtain the first spectral feature F that integrates the small-scale spatial features nsf ;

[0013] (3) Use the sub-spectrum scanning method to perceive the spectral detail differences of the hyperspectral image pixels and obtain the spectral classification information according to the following steps:

[0014] (3.1) For vector x center Split and reorganize to obtain the reorganized two-dimensional tensor x' center , and then sent it to the second state space model S6 for feature extraction to obtain the second spectral feature F ss ;

[0015] (3.2) The feature F ss With F nsf After addition, the data are sent to the linear layer for processing spectral information to obtain spectral classification information;

[0016] (4) Construct a spatial feature extraction network and use it to obtain the spatial classification information of hyperspectral images:

[0017] (4.1) The first part of the network is composed of the SS2D layer and the improved spatial attention mechanism SA, wherein the improved spatial attention mechanism SA adds a residual link on the basis of the original spatial attention mechanism to make the size of the input and output image blocks consistent; then the SS2D layer of the first part and a layer normalization LayerNorm operation are used to form the second part of the network; finally, a linear layer Linear for processing the original spatial information and an activation function SiLU are used to form the third part of the network; these three parts together constitute the spatial feature extraction network;

[0018] (4.2) X patch2 As the input of the spatial feature extraction network, the spatial features of the hyperspectral image are obtained; and the spatial features are sent to a linear layer for processing the fusion of three types of spatial information to obtain spatial classification information;

[0019] (5) The spatial classification information and the spectral classification information are concatenated and sent to the multi-layer perceptron (MLP) for decision-layer feature fusion to obtain the final classification result of the combined spatial information and spectral information.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] First, since the method of hyperspectral image classification based on the state space model (SSM) of the present invention has a faster detection speed compared with the method based on Transformer; compared with the method based on CNN, it can focus on more comprehensive spectral information.

[0022] Second, the present invention proposes a neighborhood spectrum fusion method (Nearest-neighbor spectrum fusion, NSF), which can suppress the spatial variability problem of hyperspectral images in a small range while retaining all spectral information; at the same time, a sub-spectrum scanning method (Sub-spectrum scanning, SS) is proposed, which can divide the pixels of the hyperspectral image into several sub-spectra, use the state space model to focus on its details, and enhance the classification performance from the spectral dimension.

[0023] Third, the present invention constructs a spatial information extraction network (Spatial Mamba branch, SM), introduces the SS2D model into the hyperspectral image classification task, and uses the long-distance modeling ability of SS2D in images to extract a wide range of spatial features of hyperspectral images; at the same time, because the present invention connects an improved spatial attention mechanism behind SS2D when constructing the model, it can significantly improve the spatial feature extraction ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the implementation process of the method of the present invention; wherein a is the overall flow chart of the present invention, b is the spatial information extraction network (Spatial Mamba branch, SM) architecture diagram in the present invention, c is a schematic diagram of the neighborhood spectrum fusion method (Nearest-neighbor spectrum fusion, NSF), d is a schematic diagram of the sub-spectrum scanning method (Sub-spectrum scanning, SS), and e is a schematic diagram of the 2D selective scanning method (2D Selective Scan Module, SS2D);

[0025] Figure 2 This is a flowchart of the improved spatial attention mechanism SA (Self-Attention) in the present invention;

[0026] Figure 3 The figure compares the classification effects of the present invention and the existing methods on the Pavia University dataset; (a)-(l) are respectively the classification result diagrams of the existing methods ground truth, KNN, RF, 1DCNN, 2DCNN, HybridSN, IRTS-3DCNN, CasRNN, ViT, SpectralFormer, GraphGST and the method SSUM of the present invention on the Pavia University dataset.

[0027] Figure 4 : This is a comparison chart of the classification effects of the present invention and the existing methods on the Salinas Valley dataset; wherein (a)-(l) are respectively the classification result charts of the existing methods ground truth, KNN, RF, 1DCNN, 2DCNN, HybridSN, IRTS-3DCNN, CasRNN, ViT, SpectralFormer, GraphGST and the method SSUM of the present invention on the Salinas Valley dataset. DETAILED DESCRIPTION

[0028] The present invention will be further described below in conjunction with the accompanying drawings.

[0029] Embodiment 1: Refer to the attached Figure 1 The present invention proposes a hyperspectral image classification method based on a state space model, which specifically includes the following steps:

[0030] Step 1. Given a hyperspectral image X HSI ∈R W×H×B , where W, H and B represent the width, height and number of spectral bands of the image respectively; and for X HSI Perform dimensionality reduction processing to obtain the reduced-dimensional image X PCA ; Assume that the pixel to be classified is located at point (i, j), take the spectral value x at point (i, j) center ∈R 1×1×B , a small-scale full-band image block X with a length and width of P around point (i, j) patch1 ∈R P×P×B , and X PCA A large-scale image block X with a length and width of N at the midpoint (i, j) patch2 ∈R N×N×n ;

[0031] Step 2. Use the neighborhood spectral fusion method to eliminate the interference of spectral variability on the classification of hyperspectral image pixels. The implementation steps are as follows:

[0032] (2.1) Calculate the vector x at point (i, j) centerand the average value x of its surrounding pixels avg ; In this embodiment, the average value x is calculated according to the following formula avg :

[0033] X patch1 =[x0,x1,...,x i ,...,x n ],

[0034]

[0035] Where n = P × P represents X patch1 The total number of pixels in .

[0036] (2.2) Using Einstein summation to calculate x center and x avg The fused spectral vector x fus ;

[0037] (2.3) x fus Input into the first state space model S6 for feature extraction, and connect through the residual layer to obtain the first spectral feature F that integrates the small-scale spatial features nsf :

[0038] F nsf =S6(x fus )+x fus .

[0039] Step 3. Use the sub-spectrum scanning method to perceive the spectral detail differences of the hyperspectral image pixels and obtain the spectral classification information according to the following steps:

[0040] (3.1) For vector x center Split and reorganize to obtain the reorganized two-dimensional tensor x' center , and then sent it to the second state space model S6 for feature extraction to obtain the second spectral feature F ss ; In this embodiment, vector x center Split and reorganize the input vector x center The method is to divide it into m sub-spectra of length 10 and then combine them.

[0041] (3.2) The feature F ss With F nsf After addition, the data are sent to the linear layer for processing spectral information to obtain spectral classification information;

[0042] Step 4. Build a spatial feature extraction network and use it to obtain the spatial classification information of the hyperspectral image:

[0043] (4.1) The first part of the network is composed of the SS2D layer and the improved spatial attention mechanism SA. The SS2D layer is a selective scanning network built on the basis of the state space model. The improved spatial attention mechanism SA adds residual links on the basis of the original spatial attention mechanism to make the size of the input and output image blocks consistent. Then, the SS2D layer of the first part and a layer normalization LayerNorm operation are used to form the second part of the network. Finally, a linear layer Linear for processing the original spatial information and an activation function SiLU are used to form the third part of the network. These three parts together constitute the spatial feature extraction network.

[0044] (4.2) X patch2 As the input of the spatial feature extraction network, the spatial features of the hyperspectral image are obtained; and the spatial features are sent to the linear layer for processing the fusion of the three spatial information to obtain spatial classification information. In this embodiment, the spatial features of the hyperspectral image are obtained according to the following steps:

[0045] (4.2.1) X patch2 As the input of the first part, after the SS2D layer completes the operation, its output is sent to the two branches respectively. The first branch obtains the first output feature F through the improved spatial attention mechanism in the first part. mamba The second branch is subjected to the layer normalization LayerNorm operation of the second part to obtain the second output feature F ln :

[0046] F mamba =SA[SS2D(X patch2 )],

[0047] F ln =LayerNorm[SS2D(X patch2 )];

[0048] (4.2.2) X patch2 As the input of the third part, the third feature F is obtained through the linear layer Linear and the activation function SiLU. PCA :

[0049] F PCA =SiLU[Linear(X patch2 )],

[0050] (4.2.3) The features F extracted from the three parts of the network mamba 、F ln and F PCA The spatial characteristics of the hyperspectral image are obtained by adding them together.

[0051] Step 5. After splicing the spatial classification information and the spectral classification information, send them to the multi-layer perceptron MLP for decision-layer feature fusion to obtain the final classification result of the combined spatial information and spectral information.

[0052] Embodiment 2: The overall implementation steps of the image classification method proposed in the present invention are the same as those of Embodiment 1. Figure 1 and 2 , the neighborhood spectrum fusion, sub-spectrum scanning method, and the construction and training process of the spatial information extraction network model in the present invention are further explained, and the implementation process of the method of the present invention is described in detail:

[0053] Step 1: Data input preparation

[0054] Given a hyperspectral image X HSI ∈R W×H×B , where W, H, and B represent the width, height, and number of spectral bands of the image, respectively. First, HSI Perform dimensionality reduction processing to obtain the reduced-dimensional image X PCA .

[0055] X PCA = PCA(X HSI ) (1)

[0056] For each pixel of the hyperspectral image, we need to classify it. Assume that the pixel to be classified is located at point (i, j). We need to obtain the spectral value x at point (i, j) center ∈R 1×1×B , a small-scale full-band image patch X around point (i, j) patch1 ∈R P×P×B , and X PCA The large-scale image block X at point (i, j) patch2 ∈R N×N×n .

[0057] Step 2: Nearest-neighbor spectrum fusion (NSF)

[0058] Reference Figure 1 In c, the use of neighborhood spectral fusion method can overcome the interference of spectral variability on the classification of hyperspectral image pixels, as follows:

[0059] First, the vector x at point (i, j) center and the average value x of the surrounding pixels avg Obtained by the following formula:

[0060] X patch =[x0,x1,x2.....x n ] (2)

[0061]

[0062] Where n = P×P.

[0063] Second, using x center and x avg Calculate the fused spectral vector x fus , using Einstein summation, the formula is as follows:

[0064] x fus =einsum(x center ,x avg ) (4)

[0065] Third, x fus Input into the S6 model for feature extraction, connected through the residual layer, and the obtained feature F nsf It is expressed as:

[0066] F nsf =S6(x fus )+x fus (5)

[0067] Step 3: Sub-spectrum scanning (SS)

[0068] Reference Figure 1 In d, a sub-spectrum scanning method is used to perceive the spectral detail differences of the pixels in the hyperspectral image, as follows:

[0069] First, for the input x center , this embodiment divides it into sub-spectra of several lengths, and then recombines them into a new two-dimensional tensor, where m is the number of divisions. It is then sent to another S6 with a different size from step 2 for feature extraction. The formula is as follows:

[0070] x' center = reshape(x center ) (6)

[0071] F ss = reshape[S6(x' center )] (7)

[0072] Second, F ss With F nsf After addition, they are fed into a linear layer to obtain spectral classification information.

[0073] Step 4: Spatial Information Extraction Network (Spatial Mamba branch, SM)

[0074] Reference Figure 1b in X PCA Get X patch2 ∈R N×N×n Used to extract large-scale spatial features. The details are as follows:

[0075] First, the spatial information extraction network is mainly composed of SS2D layers, an improved spatial attention mechanism and several residual blocks. The formula is as follows:

[0076] F PCA =SiLU[Linear(X patch )] (8)

[0077] F mamba =SA[SS2D(X patch )] (9)

[0078] F ln =LayerNorm[SS2D(X patch )] (10)

[0079] Where SiLU represents the activation function and SA represents the improved spatial self-attention mechanism, such as Figure 2 As shown in the figure; LayerNrom stands for layer normalization. Second, the three features are added together and passed through a linear layer to obtain spatial classification information. In order to ensure that the data is of uniform size when transmitted in the network, an additional layer is added after the general spatial attention mechanism. The spatial feature map is expanded to the original image size and then multiplied with the original image.

[0080] Finally, the obtained spatial classification information is concatenated with the spectral classification information obtained in step three and sent to an MLP layer for decision layer feature fusion to obtain the final classification result of the combined spatial information and spectral information.

[0081] Step 5: Model training

[0082] The training details of the spatial-spectral combined with hyperspectral image classification method based on the state-space model are as follows:

[0083] First, scan each point of the hyperspectral image and execute step 1 to obtain the x value of each pixel. center , X patch1 and X patch2 Second, execute step 2 to obtain F, which integrates the neighborhood spatial spectral information. nsf . Third, execute step 3 to obtain the spectral classification information. Fourth, execute step 4 to obtain the classification result of the joint spatial information and spectral information. Fifth, calculate the classification result and the label to obtain the loss function:

[0084]

[0085] Where y=[y0,y1,y2.....y n ] is the true label represented as a one-hot vector, z = [z0,z1,z2.....z n ] is the predicted probability after the softmax operation. Sum all classes and apply the log operation to the predicted probability element by element. This loss function quantifies the gap between the estimated distribution and the actual distribution, and imposes a penalty on the model's incorrect predictions. Finally, the model is back-propagated according to the loss function result to adjust the parameters.

[0086] Step 6: Image Classification of the Model

[0087] First, scan each point of the hyperspectral image and execute step 1 to obtain the x value of each pixel. center , X patch1 and X patch2 Second, x center Send to SS, X patch1 The features extracted by the two are added and fused to obtain the spectral classification features. patch2 The spatial classification features are sent to SM to obtain the spatial classification features, which are finally fused with the spectral classification features obtained in the second step to obtain the final classification result.

[0088] The effect of the present invention is further described below in conjunction with experiments:

[0089] 1. Experimental conditions:

[0090] The experiments of the present invention were carried out in a hardware environment equipped with an NVIDIA GeForce RTX 2080 graphics card with 12GB of video memory and a software environment of Windows 10 operating system and Python 3.7.

[0091] 2. Experimental content:

[0092] The proposed method is qualitatively and quantitatively compared with ten popular algorithms on two public datasets of hyperspectral image classification. The public hyperspectral anomaly detection datasets used in the experiment include Pavia University and Salinas Valley; the ten popular algorithms compared are: KNN [W.Li, Q.Du, F.Zhang, and W.Hu, "Collaborative representation based k-nearest neighbor classifier for hyperspectral imagery," in 2014 6th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), Lausanne, Switzerland, Jun. 2014.], RF [S.Amini, S.Homayouni, A.Safari, and AADarvishsefat, "Object-based classification of hyperspectral data using Random Forest algorithm," Geo-spatial Information Science, pp. 127–138, April 2018], 1DCNN, 2DCNN [X.Zhu, D.Cheng, Z.Zhang, S.Lin, and J.Dai, "An Empirical Study of Spatial Attention Mechanisms in Deep Networks," in 2019IEEE / CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), Oct. 2019], HybridSN [SKRoy, G.Krishna, SRDubey, and B.B. Chaudhuri, "HybridSN: Exploring 3-D–2-D CNN Feature Hierarchy for Hyper-spectral Image Classification,” IEEE Geoscience and Remote Sensing Letters, pp.277–281, Feb. 2020], IRTS-3DCNN [C.-I.Chang, KYMa, C.-C., Liang, Y.-M., Kuo, S., Chen, S., & Zhong, S. (2020, January). Iterative Random Training Sampling Spectral Spatial Classification for Hyperspectral Images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13, 3986–4007. doi:10.1109 / JSTARS.2019.2948070, CasRNN (Hang, R., Liu, Q., Hong, D., & Ghamisi, P. (2019, August). Cascaded Recurrent Neural Networks for Hyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing, 57(8), 5384–5394. doi:10.1109 / TGRS.2019.2916680), VIT (Li, J., Zheng, K., Liu, W., Li, Z., Yu, H., & Ni, L. (2023). Model-Guided Coarse-to-Fine Fusion Network for Unsupervised Hyperspectral Image Super-Resolution. IEEE Geoscience and Remote Sensing Letters, 20, 1–5. doi:10.1109 / LGRS.2022.3230007), SpectralFormer (Hong, D., et al. (2022, January). SpectralFormer: Rethinking Hyperspectral Image Classification with transformers. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–15. doi:10.1109 / TGRS.2021.3137370), and GraphGST (Jiang, M., et al.,"GraphGST: Graph Generative Structure-Awaretransformer for Hyperspectral Image Classification," in IEEE Transactions onGeoscience and Remote Sensing, vol.62, pp.1-16, 2024, Art no.5504016]. .

[0093] Table 1. Comparison of classification results on the Pavia University dataset.

[0094]

[0095] Table 2. Comparison of classification results on the Salinas dataset.

[0096]

[0097]

[0098] The quantitative comparison between the present invention and ten popular comparison algorithms in two data sets, the first column of the table represents the category of pixels, and the remaining columns represent the classification accuracy of the corresponding method for a certain category. The evaluation indicators used are Overall Accuracy (OA), Average Accuracy (AA) and Kappa Coefficient (κ). These three evaluate the accuracy of the algorithm from different angles, and the larger the value, the higher the detection accuracy. As shown in Table 1 and Table 2, the present invention has the best detection accuracy.

[0099] The above simulation analysis proves the correctness and effectiveness of the method proposed in the present invention.

[0100] Parts of the present invention that are not described in detail belong to common knowledge among those skilled in the art.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. It is obvious that for professionals in this field, after understanding the content and principles of the present invention, they may make various modifications and changes in form and details without departing from the principles and structures of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A hyperspectral image classification method based on a state space model, characterized in that: The steps include: (1) Given a hyperspectral image X HSI ∈R W×H×B , where W, H and B represent the width, height and number of spectral bands of the image respectively; and for X HSI Perform dimensionality reduction processing to obtain the reduced-dimensional image X PCA ; Assume that the pixel to be classified is located at point (i, j), take the spectral value x at point (i, j) center ∈R 1×1×B , a small-scale full-band image block X with a length and width of P around point (i, j) patch1 ∈R P×P×B , and X PCA A large-scale image block X with a length and width of N at the midpoint (i, j) patch2 ∈R N×N×n ; (2) The neighborhood spectral fusion method is used to eliminate the interference of spectral variability on the classification of hyperspectral image pixels. The implementation steps are as follows: (2.1) Calculate the vector x at point (i, j) center and the average value x of its surrounding pixels avg ; (2.2) Using Einstein summation to calculate x center and x avg The fused spectral vector x fus ; (2.3) x fus Input into the first state space model S6 for feature extraction, and connect through the residual layer to obtain the first spectral feature F that integrates the small-scale spatial features nsf ; (3) Use the sub-spectrum scanning method to perceive the spectral detail differences of the hyperspectral image pixels and obtain the spectral classification information according to the following steps: (3.1) for vector x center Split and reorganize to obtain the reorganized two-dimensional tensor x' center , and then sent it to the second state space model S6 for feature extraction to obtain the second spectral feature F ss ; (3.2) The feature F ss With F nsf After addition, the data are sent to the linear layer for processing spectral information to obtain spectral classification information; (4) Construct a spatial feature extraction network and use it to obtain the spatial classification information of hyperspectral images: (4.1) The first part of the network is composed of the SS2D layer and the improved spatial attention mechanism SA, wherein the improved spatial attention mechanism SA adds a residual link on the basis of the original spatial attention mechanism to make the size of the input and output image blocks consistent; then the SS2D layer of the first part and a layer normalization LayerNorm operation are used to form the second part of the network; finally, a linear layer Linear for processing the original spatial information and an activation function SiLU are used to form the third part of the network; these three parts together constitute the spatial feature extraction network; (4.2) X patch2 As the input of the spatial feature extraction network, the spatial features of the hyperspectral image are obtained; and the spatial features are sent to a linear layer for processing the fusion of three types of spatial information to obtain spatial classification information; (5) The spatial classification information and the spectral classification information are concatenated and sent to the multi-layer perceptron (MLP) for decision-layer feature fusion to obtain the final classification result of the combined spatial information and spectral information.

2. The method according to claim 1, characterized in that: The vector x at the point (i, j) in step (2.1) center and the average value x of its surrounding pixels avg , calculated according to the following formula: X patch1 =[x0,x1,...,x i ,...,x n ], Where n = P × P represents X patch1 The total number of pixels in .

3. The method according to claim 1, characterized in that: Step (2.3) The first spectral feature F nsf , specifically, it is obtained by combining the state space model S6 with the residual network according to the following formula: F nsf =S6(x fus )+x fus 。 4. The method according to claim 1, characterized in that: Step (3.1) describes the vector x center Split and reorganize, specifically the input vector x center Split into m sub-spectra of length 10, and then recombine the split sub-spectra to obtain a new two-dimensional tensor x' center .

5. The method according to claim 1, characterized in that: The SS2D layer described in step (4.1) is a selective scanning network built on the basis of the state space model.

6. The method according to claim 1, characterized in that: The spatial features of the hyperspectral image in step (4.2) are obtained specifically according to the following steps: (4.2.1) X patch2 As the input of the first part, after the SS2D layer completes the operation, its output is sent to the two branches respectively. The first branch obtains the first output feature F through the improved spatial attention mechanism in the first part. mamba The second branch is subjected to the layer normalization LayerNorm operation of the second part to obtain the second output feature F ln : F mamba =SA[SS2D(X patch2 )], F ln =LayerNorm[SS2D(X patch2 )]; (4.2.2) X patch2 As the input of the third part, the third feature F is obtained through the linear layer Linear and the activation function SiLU. PCA : F PCA =SiLU[Linear(X patch2 )], (4.2.3) The features F extracted from the three parts of the network mamba 、F ln and F PCA The spatial characteristics of the hyperspectral image are obtained by adding them together.

Images