A hyperspectral classification method based on locally enhanced selective state space model

Abstract

The present invention proposes a hyperspectral classification method based on a local enhanced selective state space model, comprising the following steps: reading a three-dimensional hyperspectral image and a category label corresponding to each pixel; normalizing the numerical values ​​in the hyperspectral image; cropping and dividing the three-dimensional hyperspectral image into image blocks with each pixel as the center point to construct an image block data set; randomly selecting data from the image block data set as a training set; training a neural network model on the training set, inputting the image blocks of the training set into a convolutional dimensionality reduction backbone constructed with point-by-point convolution as the basic unit, enhancing the selective state space model using a multi-directional local scanning mechanism, dividing multiple local window images and integrating the information; fusing spatial features and spectral features; constructing a classification head to predict category information; and training an optimization model on the training set.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a hyperspectral classification method based on a local enhanced selective state space model. Background Art

[0002] Hyperspectral images contain rich spectral information ranging from visible light to infrared wavelengths, and they fuse spatial and spectral information. They have broad applications in agricultural exploration, military reconnaissance, environmental monitoring, and geological exploration. As a foundational technology for hyperspectral applications, hyperspectral image classification, which assigns a class label to each pixel, has garnered significant attention. However, hyperspectral image classification still faces challenges, such as high hyperspectral dimensionality, an imbalanced number of labeled samples, and spectral variability.

[0003] In recent years, the development of deep learning has driven progress in hyperspectral classification technology. Deep learning models, such as convolutional neural networks and self-attention networks, have garnered significant attention and achieved remarkable success in hyperspectral image classification, and have long dominated the field. Convolutional neural networks, known for leveraging local contextual information through shift-invariant convolutions, have long held a dominant position.

[0004] However, the local receptive field and shared static weights of convolutional neural networks limit their ability to model long-range dependencies and their ability to effectively capture global context. Self-attention-based networks have dynamic weights and the ability to capture global information, but are limited by quadratic computational complexity, resulting in significant computational overhead. Recently, selective state-space models have rapidly emerged in long sequence modeling, with their linear complexity, as a strong competitor to traditional self-attention networks. However, their application in hyperspectral image classification is still in its infancy, and the selective state-space model has not been improved to be suitable for hyperspectral classification tasks.

[0005] In view of this, it is necessary to provide a hyperspectral classification method based on the local enhanced selective state space model to overcome the above defects. Summary of the Invention

[0006] The purpose of the present invention is to provide a hyperspectral classification method based on a locally enhanced selective state space model. A multi-directional local scanning mechanism is used to improve the existing selective state space model. According to the characteristics of hyperspectral images, a bidirectional spectral scanning technology is used to extract the spectral information of the central area of ​​the hyperspectral image. The spectral and spatial information are fused to improve the hyperspectral classification performance. The hyperspectral ground object classification task is better achieved, and the classification performance in the case of small samples is improved. The method is applied to the field of intelligent processing of aerial hyperspectral images.

[0007] In order to achieve the above object, the technical solution provided by the present invention is implemented as follows: a hyperspectral classification method based on a locally enhanced selective state space model comprises the following steps:

[0008] S1, reads the three-dimensional hyperspectral image and the category label corresponding to each pixel;

[0009] S2, normalize the values ​​in the hyperspectral image and map the data to the interval [0, 1];

[0010] S3, taking each pixel of the 3D hyperspectral image as the center point, divides the neighborhood of the center point into image blocks of fixed size and constructs an image block dataset, where the label of the center point pixel is set as the category label;

[0011] S4, randomly select data from the image block dataset as the training set, and the rest of the data as the test set;

[0012] S5, train the neural network model on the training set, input the image blocks of the training set into the convolution dimensionality reduction backbone built with point-by-point convolution as the basic unit, and obtain the feature map y∈R after point-by-point convolution dimensionality reduction H×W×D , using the multi-directional local scanning mechanism to enhance the selective state space model, the feature map y∈R H×W×D Divide the image y into multiple local windows p Then, the information between different spectral bands is integrated to obtain the hyperspectral feature map y stack Input selective state space model;

[0013] S6, extract the feature map y∈R obtained after dimensionality reduction H×W×D The central area of ​​the image block y is obtained center ∈R s ×s×D , and combine the selective state space model with bidirectional scanning in the spectral dimension to obtain the regional image block y center ∈R s×s×D The spatial and spectral characteristics of

[0014] S7, uses a multi-scale convolutional neural network to fuse spatial and spectral features, eliminating redundant information and retaining valuable spatial structure information;

[0015] S8, uses a fully connected feedforward neural network to build a classification head and predict category information; trains the proposed model on the training set, uses an adaptive moment estimation optimizer to iteratively update the model parameters, and tests the model performance on the test set to verify the effectiveness of the proposed model.

[0016] Preferably, step S5 further includes step S51, for an input training set The image block is calculated as follows:

[0017]

[0018] Among them, H, W and C represent the length, width and number of bands of the image respectively. Indicates that the output feature map is at the cth position (h, w) out The value of each channel, Indicates that the input feature map is at position (h, w) c in The value of each channel, Indicates that the convolution kernel is in the input channel c in and output channel c out The weight between out Less than c in .

[0019] Preferably, step S5 further includes step S52, wherein the feature map obtained after point-by-point convolution dimensionality reduction is y∈R H×W×D , using the multi-directional local scanning mechanism to enhance the selective state space model, the feature map y∈R after dimensionality reduction H×W×D Divide into multiple local window images y p , the calculation process is:

[0020]

[0021] Among them, Partition means dividing the feature map into windows of size h×w in the dimensions of height and width, n h and n w Represents the number of windows in the height and width directions respectively;

[0022] S53, the local window image y p Rearrange:

[0023]

[0024] Among them, Rearrange means that different local window images y p rearrange;

[0025] The rearranged local window image y R Transpose the internal height and width dimensions to obtain the transposed local window image Y T :

[0026]

[0027] Among them, Transpose means transposing in the height and width dimensions;

[0028] S54, the transposed local window image y T Flip in the spatial dimension to get the flipped local window image [y RF,y TF ]:

[0029] [y RF ,y TF ]=Flip([y R ,y T ]),

[0030] Among them, Flip means flipping in the spatial dimension.

[0031] S56, flatten and stack all the above local window images, integrate the information between different spectral bands of the local window images, and obtain the hyperspectral feature map y stack :

[0032] y stack ∈R 4×L×C =Stack(Flatten([y R ,y T ,y RF ,y TF ])),

[0033] Among them, Flatten means flattening and arranging in sequence in the spatial dimension, and Stack means stacking these sequences together to integrate the obtained hyperspectral feature map y stack It has four sequences of local window: horizontal, vertical, forward and reverse;

[0034] S57, hyperspectral feature map y stack The four sequences of input selective state space model are processed as shown in the formula:

[0035] y spatial ∈R L×C =Sum(SSM(y stack )),

[0036] Among them, Sum means adding the four output results, SSM means selective state space model,

[0037] h′ t =Ah t-1 +By t

[0038] The calculation formula is: t =Ch′ t ,

[0039] Among them, y t Indicates input y stack The tth value in h t It is the state representation from input sequence to output sequence, z represents the output, A is the high-order polynomial mapping operator matrix, B and C are transformed by input y stackThe linear mapping is obtained, and the mapping formula is:

[0040] B=w b y stack +b b

[0041] C=w c y stack +b c ,

[0042] Among them, w and b are trainable parameters.

[0043] Preferably, step S6 further includes step S61, where the calculation formula for extracting spectral features is as follows:

[0044] y flatten ∈R S×D =Flatten(y center ∈R w×w×D ),

[0045] Among them, Flatten means flattening in the spatial dimension, S = w × w;

[0046] y ssm ∈R S×D =SSM(y flatten ∈R S×D ),

[0047] Among them, SSM represents the selective spatial state model, which scans bidirectionally in the spectral dimension;

[0048] y spectral ∈R 1×C =Sigmoid(Average(y ssm )),

[0049] Among them, Average means taking the average value in the spatial dimension, and the sigmoid function is used to generate the control signal in the spectral dimension. Its calculation formula is:

[0050]

[0051] Multiply the attention signal of the spectral channel with the spatial feature to obtain the fusion feature of the spectral feature and the spatial feature. The calculation formula is: feature =y spectral ⊙y spatial

[0052] Here, ⊙ represents the Hadamard product.

[0053] Preferably, step S7 further includes step S71, and the calculation process is as follows:

[0054] y s1 =DWConv3×3 (PWConv(y feature ))

[0055] y s2 =PWConv(DWConv 5×5 (y s1 ))

[0056] y signal =Gelu(PWConv(y feature ))

[0057] z out =y signal ☉y s2 ,

[0058] Among them, y feature Represents input, PWConv represents point-by-point convolution, DWConv 3×3 Denotes a depth-separable convolution with a convolution kernel of 3×3, DWConv 5×5 represents a depth-wise separable convolution with a convolution kernel of 5×5, Gelu represents a Gaussian error linear function, and ⊙ represents a Hadamard product.

[0059] Compared with the existing technology, the beneficial effect is that it comprehensively considers the characteristics of aerial hyperspectral images, combines the local state space multi-directional scanning mechanism and the bidirectional spectral scanning mechanism, can effectively extract high-level semantic information in hyperspectral images, better realizes the hyperspectral ground object classification task, and improves the classification performance in small sample conditions.

[0060] Other features and advantages of the present invention will be set forth in the following description, and some will be apparent from the description, or will be understood through practice of the present invention. Features and advantages of the present invention can be realized and obtained through the elements and combinations specifically indicated in the appended claims. These and other features of the present invention will become more apparent from the following description and the appended claims, or will be understood through practice of the embodiments described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0062] Figure 1 This is a flow chart of the hyperspectral classification method based on the local enhanced selective state space model provided by the present invention.

[0063] Figure 2 Schematic diagram of a multi-scale convolutional neural network in an embodiment of the present invention.

[0064] Figure 3 This is a comparison chart of the classification results of the embodiments of the present invention on the Indian Pines dataset.

[0065] Figure 4 3 is a comparison chart of the classification results of the embodiments of the present invention on the WHU-Hi-LongKou dataset. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and beneficial technical effects of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described in this specification are only for the purpose of explaining the present invention and are not intended to limit the present invention.

[0067] See also Figures 1 to 2 The present invention provides a hyperspectral classification method based on a local enhanced selective state space model, comprising the following steps:

[0068] S1, reads the three-dimensional hyperspectral image and the category label corresponding to each pixel;

[0069] S2, normalize the values ​​in the hyperspectral image and map the data to the interval [0, 1];

[0070] Specifically, step S2 further includes step S21. In this embodiment, two public datasets, Indian Pines (IP) and WHU-Hi-LongKou (LK), are used as examples to normalize the values ​​in the hyperspectral image and map the values ​​to the [0, 1] interval.

[0071] The processing process is expressed as follows:

[0072]

[0073] Among them, x max and x min They represent the maximum and minimum values ​​in the hyperspectral image respectively; the hyperspectral image is divided into multiple three-dimensional image blocks to construct a data set, and the background image blocks that do not contain category information are removed.

[0074] S3, taking each pixel of the 3D hyperspectral image as the center point, divides the neighborhood of the center point into image blocks of fixed size and constructs an image block dataset, where the label of the center point pixel is set as the category label;

[0075] S4, randomly select a certain proportion (1%) of the data in the image block dataset as the training set, and the rest of the data as the test set;

[0076] S5, train the neural network model on the training set, input the image blocks of the training set into the convolution dimensionality reduction backbone built with point-by-point convolution as the basic unit, and obtain the feature map y∈R after point-by-point convolution dimensionality reduction without changing the spatial resolution through the 1×1 convolution kernel. H×W×D , using the multi-directional local scanning mechanism to enhance the selective state space model, the feature map y∈R H×W×D Divide the image y into multiple local windows p Then, the information between different spectral bands is integrated to obtain the hyperspectral feature map y stack Input selective state space model can reduce computational burden;

[0077] Specifically, step S5 further includes step S51, for an input training set y∈R H×W×D The image block is calculated as follows:

[0078]

[0079] Among them, H, W and C represent the length, width and number of bands of the image respectively. Indicates that the output feature map is at the cth position (h, w) out The value of each channel, Indicates that the input feature map is at position (h, w) c in The value of each channel, Indicates that the convolution kernel is in the input channel c in and output channel c out The weight between out Less than c in .

[0080] Specifically, step S5 also includes step S52, the feature map obtained after point-by-point convolution dimensionality reduction is y∈R H×W×D , using the multi-directional local scanning mechanism to enhance the selective state space model, the feature map y∈R after dimensionality reduction H×W×D Divide into multiple local window images y p , the calculation process is:

[0081]

[0082] Among them, Partition means dividing the feature map into windows of size h×w in the dimensions of height and width, n h and n w Represents the number of windows in the height and width directions respectively;

[0083] S53, the local window image yp Rearrange:

[0084]

[0085] Among them, Rearrange means that different local window images y p rearrange;

[0086] The rearranged local window image y R Transpose the internal height and width dimensions to obtain the transposed local window image y T :

[0087]

[0088] Among them, Transpose means transposing in the height and width dimensions;

[0089] S54, the transposed local window image y T Flip in the spatial dimension to get the flipped local window image [y RF ,y TF ]:

[0090] [y RF ,y TF ]=Flip([y R ,y T ),

[0091] Among them, Flip means flipping in the spatial dimension.

[0092] S56, the above-mentioned local window image (y p 、y R 、y T 、[y RF ,y TF ]) are flattened and stacked together to integrate the information between different spectral bands of the local window image to obtain the hyperspectral feature map y stack :

[0093] y stack ∈R 4×L×C =Stack(Flatten([y R ,y T ,y RF ,y TF ])),

[0094] Among them, Flatten means flattening and arranging in sequence in the spatial dimension, and Stack means stacking these sequences together to integrate the obtained hyperspectral feature map y stack It has four sequences of local window: horizontal, vertical, forward and reverse;

[0095] S57, hyperspectral feature map y stack The four sequences of input selective state space model are processed as shown in the formula:

[0096] y spatial ∈R L×C =Sum(SSM(y stack )),

[0097] Among them, Sum means adding the four output results, SSM means selective state space model,

[0098] h′ t =Ah t-1 +By t

[0099] The calculation formula is: t =Ch′ t ,

[0100] Among them, y t Indicates input y stack The tth value in h t It is the state representation from input sequence to output sequence, z represents the output, A is the high-order polynomial mapping operator matrix, B and C are transformed by input y stack The linear mapping is obtained, and the mapping formula is:

[0101] B=w b y stack +b b C=w c y stack +b c ,

[0102] Among them, w and b are trainable parameters

[0103] S6, extract the feature map y∈R obtained after dimensionality reduction H×W×D The central area of ​​the image block y is obtained center ∈R s ×s×D , and combine the selective state space model with bidirectional scanning in the spectral dimension to obtain the regional image block y center ∈R s×s×D The spatial and spectral characteristics of

[0104] Specifically, step S6 further includes step S61, where the calculation formula for extracting spectral features is as follows:

[0105] y flatten ∈R S×D =Flatten(y center ∈R w×w×D ),

[0106] Among them, Flatten means flattening in the spatial dimension, S = w × w;

[0107] y ssm ∈R S×D =SSM(y flatten ∈R S×D ,

[0108] Among them, SSM represents the selective spatial state model, which obtains spatial features by bidirectional scanning in the spectral dimension;

[0109] y specral ∈R 1×C =Sigmoid(Average(y ssm )),

[0110] Among them, Average means taking the average value in the spatial dimension, and the sigmoid function is used to generate the control signal in the spectral dimension. Its calculation formula is:

[0111]

[0112] Multiply the attention signal of the spectral channel with the spatial feature to obtain the fusion feature of the spectral feature and the spatial feature. The calculation formula is: feature =y spectral ⊙y spatial

[0113] Where ⊙ represents the Hadamard product, y feature Obtained by multiplying spatial features and spectral features.

[0114] S7, uses a multi-scale convolutional neural network to fuse spatial and spectral features, eliminating redundant information and retaining valuable spatial structure information;

[0115] Specifically, step S7 further includes step S71, and the calculation process is as follows:

[0116] y s1 =DWCov 3×3 (PWConv(y featne ))

[0117] y s2 =PWConv(DWConv 5×5 (y s1 ))

[0118] y signal =Gelu(PWConv(y feature ))

[0119] z out =y signal ☉y s2 ,

[0120] Among them, y feature Represents input, PWConv represents point-by-point convolution, DWConv 3×3 Denotes a depth-separable convolution with a convolution kernel of 3×3, DWConv 5×5 represents a depth-wise separable convolution with a convolution kernel of 5×5, Gelu represents a Gaussian error linear function, and ⊙ represents a Hadamard product.

[0121] S8, uses a fully connected feedforward neural network to build a classification head to predict category information; uses an adaptive moment estimation optimizer to iteratively update model parameters, trains and optimizes the selective state space model on the training set, and obtains an improved selective space state model.

[0122] The performance of the improved selective spatial state model was tested on the test set to verify its effectiveness. In this example, the overall accuracy (OA), average accuracy (AA), and Kappa coefficient (K) were used to measure classification accuracy. The model was compared with the mainstream benchmark models SSRN, DBDA, SSFTT, and SSTN. The IP dataset used 10% as the training set and 90% as the test set. The LK dataset used 1% as the training set and the remaining 99% as the test set.

[0123] The SSRN method comes from Zhong, Z., Li, J., Luo, Z., Chapman, M., 2018b. Spectral–Spatial Residual Network for Hyperspectral Image Classification: A3-DDeepLearning Framework. IEEE Transactions on Geoscience and Remote Sensing56, 847-858.

[0124] The SSTN method comes from Zhong, Z., Li, Y., Ma, L., Li, J., Zheng, W.-S., 2022. Spectral–Spatial Transformer Network for Hyperspectral Image Classification: AFactorized Architecture Search Framework. IEEE Transactions on Geoscience and Remote Sensing 60, 1-15.

[0125] The SSFTT method comes from Sun, L., Zhao, G., Zheng, Y., Wu, Z., 2022. Spectral–SpatialFeature Tokenization Transformer for Hyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing 60, 1-14.

[0126] The DBDA method comes from Li, R., Zheng, S., Duan, C., Yang, Y., Wang, X., 2020. Classification of Hyperspectral Image Based on Double-Branch Dual-Attention Mechanism Network, Remote Sensing. MDPI AG, p.582.

[0127] The classification results on the IP and LK datasets are shown in Tables 1 and 2. It can be seen that on the two datasets, compared with different mainstream classification models, the improved selective spatial state model proposed in this invention achieves the best performance in the three indicators of classification accuracy OA, AA and κ.

[0128] Table 1 IP dataset

[0129]

[0130] Table 2 LK dataset

[0131]

[0132]

[0133] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the hyperspectral classification method based on the locally enhanced selective state space model are implemented.

[0134] In yet another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the hyperspectral classification method based on the locally enhanced selective state space model.

[0135] The present invention also provides a computer program product, comprising a computer program, which implements the steps of the hyperspectral classification method based on the local enhanced selective state space model when executed by a processor.

[0136] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0137] In particular, according to some embodiments of the present disclosure, the process described above can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, the above-mentioned functions defined in the method of some embodiments of the present disclosure are performed.

[0138] It should be noted that the computer-readable medium described in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. Furthermore, in some embodiments of the present disclosure, a computer-readable signal medium may include a mission data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated mission data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0139] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital task data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), internets (e.g., the Internet), and peer-to-peer networks (e.g., adhoc peer-to-peer networks), as well as any currently known or future developed networks.

[0140] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to detecting a query operation on a production collaboration document in the switch production line management application, determines the network connection status of the switch production line management application; in response to determining that the network connection status of the switch production line management application represents an offline state, replaces the web page entry information corresponding to the production collaboration document with target entry file information, and loads target web page resource information to display the web page of the production collaboration document offline in the switch production line management application, wherein the target entry file information is file information of a pre-downloaded entry file corresponding to the web page of the production collaboration document, and the target web page resource information is locally stored resource information corresponding to the web page; in response to determining that the network connection status of the switch production line management application represents an online state and the web page resource information corresponding to the production collaboration document is not stored locally, downloads the web page resource information of the web page from the production line document server, wherein the web page resource information includes the entry file and resource information; displays the web page of the production collaboration document in the switch production line management application according to the web page resource information, and stores the web page resource information in a local database.

[0141] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including product-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0142] The present invention is not limited to what is described in the specification and embodiments, and additional advantages and modifications will be readily apparent to those skilled in the art. Therefore, the present invention is not limited to the specific details, representative devices, and illustrative examples shown and described herein without departing from the spirit and scope of the general concept defined by the claims and their equivalents.

Claims

1. A hyperspectral classification method based on a locally enhanced selective state space model, characterized in that: The following steps are involved: S1, reads the three-dimensional hyperspectral image and the category label corresponding to each pixel; S2, normalize the values ​​in the hyperspectral image and map the data to the interval [0, 1]; S3, taking each pixel of the 3D hyperspectral image as the center point, divides the neighborhood of the center point into image blocks of fixed size and constructs an image block dataset, where the label of the center point pixel is set as the category label; S4, randomly select data from the image block dataset as the training set, and the rest of the data as the test set; S5, train the neural network model on the training set, input the image blocks of the training set into the convolution dimensionality reduction backbone built with point-by-point convolution as the basic unit, and obtain the feature map y∈R after point-by-point convolution dimensionality reduction H×W×D , using the multi-directional local scanning mechanism to enhance the selective state space model, the feature map y∈R H×W×D Divide the image y into multiple local windows p Then, the information between different spectral bands is integrated to obtain the hyperspectral feature map y stack Input selective state space model; S6, use the spectral feature extraction module to extract the feature map y∈R obtained after dimensionality reduction H×W×D The central area of ​​the image block y is obtained center ∈R s×s×D , and combine the selective state space model with bidirectional scanning in the spectral dimension to obtain the regional image block y center ∈R s×s×D The spatial and spectral characteristics of S7, uses a multi-scale convolutional neural network to fuse spatial and spectral features, eliminating redundant information and retaining valuable spatial structure information; S8 uses a fully connected feedforward neural network to build a classification head to predict category information; train and optimize the selective state space model on the training set.

2. The hyperspectral classification method based on the locally enhanced selective state space model according to claim 1, characterized in that: Step S5 also includes step S51, for an input training set The image block is calculated as follows: Among them, H, W and C represent the length, width and number of bands of the image respectively. Indicates that the output feature map is at the cth position (h, w) out The value of each channel, Indicates that the input feature map is at position (h, w) c in The value of each channel, Indicates that the convolution kernel is in the input channel c in and output channel c out The weight between out Less than c in .

3. The hyperspectral classification method based on the locally enhanced selective state space model according to claim 2, characterized in that: Step S5 also includes step S52, where the feature map obtained after point-by-point convolution dimensionality reduction is y∈R H×W×D , using the multi-directional local scanning mechanism to enhance the selective state space model, the feature map y∈R after dimensionality reduction H×W×D Divide into multiple local window images y p , the calculation process is: Among them, Partition means dividing the feature map into windows of size h×w in the dimensions of height and width, n h and n w Represents the number of windows in the height and width directions respectively; S53, the local window image y p Rearrange: Among them, Rearrange represents the different local window images y p rearrange; The rearranged local window image y R Transpose the internal height and width dimensions to obtain the transposed local window image Y T : Among them, Transpose means transposing in the height and width dimensions; S54, the transposed local window image y T Flip in the spatial dimension to get the flipped local window image [y RF ,y TF ]: [and RF .and TF ]=Flip([and R ,and T ]), Among them, Flip means flipping in the spatial dimension; S56, flatten and stack all the above local window images, integrate the information between different spectral bands of the local window images, and obtain the hyperspectral feature map y stack : and stack ∈R 4×L×C =Stack(Flatten([and R ,and T ,and RF ,and TF ]), Among them, Flatten means flattening and arranging in sequence in the spatial dimension, and Stack means stacking these sequences together to integrate the obtained hyperspectral feature map y stack It has four sequences of local window: horizontal, vertical, forward and reverse; S57, hyperspectral feature map y stack The four sequences of input selective state space model are processed as shown in the formula: y spatial ∈R L×C =sum(SSM(y stack )), Among them, Sum means adding the four output results, SSM means selective state space model, h′ t =Ah t-1 +By t The calculation formula is: t =Ch′ t , Among them, y t Indicates input y stack The tth value in h t It is the state representation from input sequence to output sequence, z represents output, A is the high-order polynomial mapping operator matrix, B and C are transformed by input y stack The linear mapping is obtained, and the mapping formula is: B=w b y stack +b b C=w c y stack +b c , Among them, w and b are trainable parameters.

4. The hyperspectral classification method based on the locally enhanced selective state space model according to claim 1, characterized in that: Step S6 also includes step S61, where the calculation formula for extracting spectral features is as follows: and flatten ∈R S×D =Flatten(and center ∈R w×w×D ), Among them, Flatten means flattening in the spatial dimension, S = w × w; y ssm ∈R S×D =SSM(y flatten ∈R S×D , Among them, SSM represents the selective spatial state model, which scans bidirectionally in the spectral dimension; y spatial ∈R 1×C =Sigmoid(Average(y ssm )), Among them, Average means taking the average value in the spatial dimension, and the sigmoid function is used to generate the control signal in the spectral dimension. Its calculation formula is: Multiply the attention signal of the spectral channel with the spatial feature to obtain the fusion feature of the spectral feature and the spatial feature. The calculation formula is: feature =y spectral ⊙y spatial Here, ⊙ represents the Hadamard product.

5. The hyperspectral classification method based on the locally enhanced selective state space model according to claim 1, characterized in that: Step S7 also includes step S71, and the calculation process is as follows: y s1 =DWConv 3×3 (PWConv(y feature )) y s2 =PWConv(DWConv 5×5 (y s1 )) the signal =Gelu(PWConv(y feature )) z out =y signal ⊙y s2 , Among them, y feature Represents input, PWConv represents point-by-point convolution, DWConv 3×3 Denotes a depth-separable convolution with a convolution kernel of 3×3, DWConv 5×5 represents a depth-wise separable convolution with a convolution kernel of 5×5, Gelu represents a Gaussian error linear function, and ⊙ represents a Hadamard product.

Images