Hyperspectral classification method based on low-frequency characteristics and Mama

Abstract

The embodiment of the invention discloses a hyperspectral classification method based on low-frequency features and Mama, and relates to the technical field of hyperspectral remote sensing image processing, and the method comprises the steps: obtaining a to-be-classified hyperspectral image, carrying out the extraction of a plurality of image blocks, inputting the image blocks into a trained hyperspectral classification model, and carrying out the preprocessing of the image blocks to obtain a preprocessed feature map; performing feature decoupling extraction based on the feature map to obtain low-frequency features, and inputting the low-frequency features into a space Mama module to extract long-distance space-spectrum dependence features; obtaining a fusion feature based on the low-frequency feature and the long-distance space-spectrum dependence feature; and determining a ground object category corresponding to each image block based on the fusion features, and generating a ground object distribution diagram corresponding to the hyperspectral image. According to the invention, by separating the high and low frequency features and retaining the low frequency features, key spectral and spatial features can be screened and focused, irrelevant information interference is reduced, and the ground feature category corresponding to each image block can be judged more quickly and accurately according to the hyperspectral image.

Description

Technical Field

[0001] The present application relates to the technical field of hyperspectral remote sensing image processing, and in particular to a hyperspectral classification method based on low-frequency features and Mamba. Background Art

[0002] Hyperspectral images can obtain the reflection or radiation information of objects from many continuous and narrow spectral bands, usually including dozens or even hundreds of bands, with extremely high spectral resolution. This allows hyperspectral images to carefully depict the spectral characteristics of objects. Different objects will present different spectral curves in different bands, so that objects that are similar in appearance but have different spectral characteristics can be effectively distinguished. Hyperspectral has a wide range of applications in crop monitoring, forest identification, mineral exploration, environmental monitoring, urban planning, ocean exploration and other fields.

[0003] Traditional hyperspectral image classification methods are mainly divided into two types: pure pixel-based and mixed pixel-based. Pure pixel-based methods mainly use spectral features, such as spectral matching and statistical model classification, but are easily interfered by various factors, resulting in reduced classification accuracy. Mixed pixel-based methods take into account spatial resolution and complexity of objects, and regard pixels as a mixture of objects, but there are problems such as large data volume, strong correlation, high redundancy, and difficulty in obtaining training samples.

[0004] Deep learning models can automatically extract important features from data without manually designing feature engineering, and have powerful feature extraction capabilities. Among them, convolutional neural networks (CNNs) can be used for hyperspectral image classification. Specifically, they can automatically extract features such as texture and shape in images through convolution kernels and pooling operations, and combine these features for classification. However, convolutional neural network models require a large amount of data and computing resources, and are prone to overfitting. Recurrent neural networks (RNNs) can process sequence data, but their applications in hyperspectral image classification are relatively rare. Although RNN models can capture temporal dependencies in data, they may face problems such as computational complexity and gradient vanishing when processing high-dimensional data. Graph convolutional neural networks (GCNs) have advantages in processing graph-structured data. In hyperspectral image classification, GCN models can use the spatial relationship between pixels for classification. However, GCN models need to build graph structures, and have high requirements for the quality and complexity of graph construction. Generative adversarial networks (GANs) can generate realistic data samples, but their application in hyperspectral image classification is still in the exploratory stage. The GAN model can improve the generalization ability of the classifier by generating adversarial results, but the training process is complex and difficult to control.

[0005] The Transformer model can also be applied in the field of hyperspectral image classification. This model is based on the self-attention mechanism and can automatically focus on different parts of the data to capture global feature information. Compared with CNN, RNN and GCN, Transformer has excellent global feature capture capabilities. It does not require convolution kernel sliding or sequence processing, and directly focuses on the relationship between pixels in hyperspectral images, making full use of spectral and spatial information. In addition, Transformer is more suitable for long sequence data processing, avoiding the problem of gradient disappearance or explosion, and stably processing multi-band long sequence information of hyperspectral images. However, the Transformer-based model has the disadvantage of high computational complexity. Especially when processing large-scale hyperspectral image data, the self-attention mechanism has a large amount of computation, resulting in slow training and inference speed.

[0006] Therefore, there is currently a lack of a method that can accurately and efficiently separate high- and low-frequency features from original hyperspectral images. Summary of the invention

[0007] The embodiment of the present application provides a hyperspectral classification method based on low-frequency features and Mamba to solve the defects of the above-mentioned related technologies. The technical solution is as follows:

[0008] In a first aspect, an embodiment of the present application provides a hyperspectral classification method based on low-frequency features and Mamba, comprising:

[0009] Acquire a hyperspectral image to be classified, and extract a plurality of image blocks of the same size based on the hyperspectral image;

[0010] Inputting each of the image blocks into a trained hyperspectral classification model, preprocessing each of the image blocks through the hyperspectral classification model to obtain a preprocessed feature map;

[0011] Perform feature decoupling based on the feature graphs to extract low-frequency features corresponding to each of the feature graphs;

[0012] Inputting the extracted low-frequency features into the spatial Mamba module in the hyperspectral classification model to extract long-range spatial-spectral dependency features;

[0013] The fusion features are obtained based on the low-frequency features and the corresponding long-range spatial-spectral dependency features;

[0014] The ground object category corresponding to each image block is determined based on the mapping relationship between the fusion features and the ground object category, and a ground object distribution map corresponding to the hyperspectral image is generated.

[0015] In an optional solution of the first aspect, extracting a plurality of image blocks of the same size based on the hyperspectral image by the hyperspectral classification model includes:

[0016] Extracting each pixel point in the hyperspectral image by using the hyperspectral classification model;

[0017] Taking each pixel as the center, extract an image block with a preset width and a preset height;

[0018] The size of each image block is w×h×p;

[0019] Wherein, w is the preset width, h is the preset height, and p is the number of bands.

[0020] In an optional solution of the first aspect, preprocessing each of the image blocks by using the hyperspectral classification model to obtain a preprocessed feature map includes:

[0021] Input the image blocks into the two-dimensional convolution layer of the trained hyperspectral classification model, map each image block into a feature space of the same dimension, and output a feature map of uniform dimension;

[0022] The dimension of the output feature map is

[0023] Where b is the batch size of all image patches and c is the dimension of the feature space.

[0024] In an optional solution of the first aspect, the method further includes:

[0025] Each of the feature maps is divided into a high-frequency component and a low-frequency component by the hyperspectral classification model;

[0026] Based on the two-dimensional convolution layer of the hyperspectral classification model, the high-frequency component and the low-frequency component are respectively reduced in dimension, and the high-frequency component and the low-frequency component after the dimension reduction are output;

[0027] The performing feature decoupling based on the feature graph to extract low-frequency features corresponding to each feature graph includes:

[0028] Inputting the reduced-dimensional high-frequency component and the reduced-dimensional low-frequency component into two consecutive high- and low-frequency separation modules in the hyperspectral classification model for feature decoupling, and processing the reduced-dimensional high-frequency component and the reduced-dimensional low-frequency component by discrete wavelet transform to separate high-frequency features and low-frequency features;

[0029] The low frequency features are preserved.

[0030] In an optional solution of the first aspect, the step of inputting the extracted low-frequency features into a spatial Mamba module in the hyperspectral classification model to extract long-range spatial-spectral dependent features comprises:

[0031] Flattening the low-frequency features into corresponding feature units through the unit module of the spatial Mamba module;

[0032] The feature unit is processed by the Mamba module of the spatial Mamba module, so that the hyperspectral classification model extracts long-distance spatial-spectral dependent features based on the feature unit according to the pre-learned long-distance spatial-spectral functional relationship between features.

[0033] In an optional solution of the first aspect, obtaining the fused feature based on the low-frequency feature and the corresponding long-range spatial-spectral dependent feature includes:

[0034] Preprocessing the long-distance space-spectrum dependence feature to change the size of the long-distance space-spectrum dependence feature to the same size as the low-frequency feature;

[0035] The preprocessed long-distance spatial-spectral dependent feature and the low-frequency feature are added to obtain the fused feature.

[0036] In an optional solution of the first aspect, the step of training the hyperspectral classification model includes:

[0037] Acquire a sample hyperspectral image, extract a plurality of image blocks based on the sample hyperspectral image, extract a preset number of image blocks and a ground object type label corresponding to each image block to generate a training set;

[0038] Inputting the training set into the spectral classification model to train the spectral classification model, so that the spectral classification model learns the long-range spatial-spectral function relationship between features based on the training set;

[0039] The output result of the spectral classification model is obtained, and a loss function is constructed based on the output result and the ground object type label corresponding to each image block in the training set. Then, when the spectral classification model converges according to the loss function, the model parameters of the converged spectral classification model are output to obtain the trained hyperspectral classification model.

[0040] In a second aspect, the embodiment of the present application further provides a hyperspectral classification device based on low-frequency features and Mamba, comprising:

[0041] An image processing module, used for acquiring a hyperspectral image to be classified, and extracting a plurality of image blocks of the same size based on the hyperspectral image;

[0042] A hyperspectral classification module, used for inputting each of the image blocks into a trained hyperspectral classification model, preprocessing each of the image blocks through the hyperspectral classification model, and obtaining a preprocessed feature map;

[0043] The hyperspectral classification module is also used to perform feature decoupling based on the feature graphs to extract low-frequency features corresponding to each of the feature graphs;

[0044] The hyperspectral classification module is also used to input the extracted low-frequency features into the spatial Mamba module in the hyperspectral classification model to extract long-distance spatial-spectral dependency features;

[0045] The hyperspectral classification module is also used to obtain fusion features based on low-frequency features and corresponding long-range spatial-spectral dependency features;

[0046] The hyperspectral classification module is also used to determine the ground object category corresponding to each image block based on the mapping relationship between the fusion features and the ground object category, and generate a ground object distribution map corresponding to the hyperspectral image.

[0047] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method provided in the first aspect of the embodiment of the present application or any one of the implementation methods of the first aspect is implemented.

[0048] In a fourth aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided by the first aspect of the embodiment of the present application or any one of the implementations of the first aspect.

[0049] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0050] The embodiment of the present application provides a hyperspectral classification method based on low-frequency features and Mamba. By separating high- and low-frequency features, discarding high-frequency features, retaining low-frequency features, and inputting low-frequency features into a spatial Mamba model, it is possible to screen and focus on key spectral and spatial features, reduce interference from irrelevant information, and thus more accurately extract long-distance spatial-spectral dependent features, so that the category of land objects corresponding to each image block can be judged more quickly and accurately according to the hyperspectral image, and reliable data support can be provided for the classification of land object types in the fields of crop monitoring, forest identification, mineral exploration, environmental monitoring, urban planning, and ocean exploration; in addition, by screening and focusing on key spectral and spatial features and reducing interference from irrelevant information, the feature learning process of the model during training can be accelerated, and the dependence on a large amount of training data can be reduced, so that the feature learning efficiency of the model can be improved, so that the model can extract key features more quickly and accurately when processing hyperspectral images. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 It is a flowchart of a hyperspectral classification method based on low-frequency features and Mamba provided in an embodiment of the present application;

[0053] Figure 2 This is one of the ground object classification schematic diagrams of a hyperspectral classification method based on low-frequency features and Mamba provided in an embodiment of the present application;

[0054] Figure 3 This is the second schematic diagram of a ground feature classification method based on low-frequency features and Mamba's hyperspectral classification method provided in an embodiment of the present application;

[0055] Figure 4 This is the third schematic diagram of a ground feature classification method based on low-frequency features and Mamba's hyperspectral classification method provided in an embodiment of the present application;

[0056] Figure 5 This is the fourth schematic diagram of a ground object classification method based on low-frequency features and Mamba's hyperspectral classification method provided in an embodiment of the present application;

[0057] Figure 6 It is a structural schematic diagram of a hyperspectral classification device based on low-frequency features and Mamba provided in an embodiment of the present application;

[0058] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] The terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or modules is not limited to the listed steps or modules, but may optionally include steps or modules that are not listed, or may optionally include other steps or modules that are inherent to these processes, methods, products or devices.

[0061] It should be noted that the terms "first\second" involved in the present application are only used to distinguish similar objects, and do not represent a specific order for the objects. It is understandable that "first\second" can be interchanged with a specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those described or illustrated herein.

[0062] The present application is described in detail below with reference to specific embodiments.

[0063] Next, combine Figure 1 , introduces a hyperspectral classification method based on low-frequency features and Mamba provided in the embodiment of the present application. For details, please refer to Figure 1 , Figure 1 FIG. 1 is a flow chart of a hyperspectral classification method based on low-frequency features and Mamba provided in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0064] S101, obtaining a hyperspectral image to be classified, and extracting a plurality of image blocks of the same size based on the hyperspectral image;

[0065] S102, inputting each of the image blocks into a trained hyperspectral classification model, and preprocessing each of the image blocks using the hyperspectral classification model to obtain a preprocessed feature map;

[0066] S103, performing feature decoupling based on the feature graphs to extract low-frequency features corresponding to each of the feature graphs;

[0067] S104, inputting the extracted low-frequency features into the spatial Mamba module in the hyperspectral classification model to extract long-distance spatial-spectral dependency features;

[0068] S105, obtaining fusion features based on low-frequency features and corresponding long-range spatial-spectral dependency features;

[0069] S106, determining the ground object category corresponding to each image block based on the mapping relationship between the fusion feature and the ground object category, and generating a ground object distribution map corresponding to the hyperspectral image.

[0070] Specifically, in S101, each pixel point in the hyperspectral image may be extracted by the hyperspectral classification model;

[0071] Taking each pixel as the center, extract an image block with a preset width and a preset height;

[0072] The size of each image block is w×h×p;

[0073] Wherein, w is the preset width, h is the preset height, and p is the number of bands.

[0074] Specifically, each pixel point in the hyperspectral image to be classified can be selected, and an image block centered on the pixel and retaining all band information of the original hyperspectral image can be extracted in the height and width dimensions respectively.

[0075] Specifically, in S102, the image block may be input into a two-dimensional convolutional layer of a trained hyperspectral classification model, each image block may be mapped into a feature space of the same dimension, and a feature map of a unified dimension may be output;

[0076] The dimension of the output feature map is

[0077] Where b is the batch size of all image patches and c is the dimension of the feature space.

[0078] Specifically, the feature map is recorded as Parameters w and h can both be set to 11, parameter b to 128, and parameter c to 128.

[0079] Specifically, in S103, before performing feature decoupling based on the feature graph, each feature graph F(x) can be divided into a high-frequency component F by using the hyperspectral classification model. H (x) and low-frequency component F L (x);

[0080] The two-dimensional convolutional layer based on the hyperspectral classification model is used to classify the high-frequency components F H (x) and low-frequency component F L (x) Perform dimensionality reduction and output the high-frequency component and the low-frequency component after dimensionality reduction.

[0081] Specifically, the complexity of subsequent high- and low-frequency classification can be reduced by reducing the dimension. The high-frequency component y after dimension reduction is H (x) and the low-frequency component y after dimensionality reduction L The size of (x) is Wherein a is the scaling factor of dimensionality reduction, and in this embodiment, a is set to 4.

[0082] Specifically, in S103, the high-frequency component y after dimension reduction can be H (x) and the low-frequency component y after dimension reduction L (x) inputting the two consecutive high- and low-frequency separation modules in the hyperspectral classification model to perform feature decoupling, processing the reduced-dimensional high-frequency component and the reduced-dimensional low-frequency component by discrete wavelet transform, and separating high-frequency features and low-frequency features;

[0083] The low frequency features are preserved.

[0084] In some embodiments, the high-low frequency separation module is implemented based on discrete wavelet transform. H (x) and y L (x) The feature is decomposed into four key components: LL, LH, HL and HH. Among them, LL, as a low-frequency component, contains the main information and structure of the input feature, which is highly similar to the original image and retains its core content and structure. LH, HL and HH represent the details and edge information of the high-frequency component in the horizontal, vertical and diagonal directions respectively.

[0085] The calculation process can be expressed by the following formula, where DWT stands for discrete wavelet transform:

[0086]

[0087] Furthermore, the acquired high-frequency features and low-frequency features can be fused separately to obtain the high-frequency features Y finally extracted by discrete wavelet transform. H and low frequency feature Y L , apply the formula:

[0088]

[0089] in,

[0090] After that, two two-dimensional convolution operations with two convolution kernels of 1 are used to transform Y H and YL The number of channels is mapped back to the same dimension as the input data, that is, the same dimension as the feature map, that is, channel c.

[0091] Specifically, the spatial Mamba module is composed of a unit module and a Mamba module. In S104, the low-frequency features can be flattened into corresponding feature units by the unit module of the spatial Mamba module, and the formula is applied:

[0092] Token=Flatten(Y L (x));

[0093] in, Where b represents the batch size, w and h represent the width and height respectively, and c represents the dimension of the feature space.

[0094] Among them, Token can be understood as the smallest unit corresponding to each part or each feature in the image. Through the flattening operation, the multi-dimensional image features can be converted into a low-dimensional vector, that is, flattened to obtain the feature unit token.

[0095] Further, the feature unit is processed by the Mamba module of the spatial Mamba module, so that the hyperspectral classification model extracts the long-distance spatial-spectral dependent feature M(x) based on the feature unit according to the pre-learned long-distance spatial-spectral functional relationship between the features.

[0096] In some embodiments, the long-distance space-spectrum dependence feature M(x) may be preprocessed to process the size of the long-distance space-spectrum dependence feature to the same size as the low-frequency feature;

[0097] The preprocessed long-distance spatial-spectral dependent feature and the low-frequency feature are added to obtain the fused feature.

[0098] Specifically, it can be expressed as follows:

[0099] M(x)=Reshape(SiLU(GN(Mamba(Token))));

[0100] That is, the Token is input into the Mamba architecture to extract the long-range spatial-spectral dependency feature M(x), and finally, the extracted M(x) is reshaped.

[0101] in, GN represents group normalization, SiLU represents the activation function, and Reshape represents the reshaping operation.

[0102] Specifically, in S105, the low-frequency feature Y obtained in S103 can be L (x) is added to the long-distance spatial-spectral dependent feature M(x) obtained in S104 to generate a fused feature to achieve a more comprehensive feature representation.

[0103] In some embodiments, the step of training the hyperspectral classification model includes:

[0104] A sample hyperspectral image is obtained, a plurality of image blocks are extracted based on the sample hyperspectral image, and a preset number of image blocks and a ground object type label corresponding to each image block are extracted to generate a training set.

[0105] The training set is input into the spectral classification model to train the spectral classification model, so that the spectral classification model learns the long-range spatial-spectral function relationship between features based on the training set.

[0106] The output result of the spectral classification model is obtained, and a loss function is constructed based on the output result and the ground object type label corresponding to each image block in the training set. Then, when the spectral classification model converges according to the loss function, the model parameters of the converged spectral classification model are output to obtain the trained hyperspectral classification model.

[0107] In some embodiments, 100 pixels may be selected as a training set and a validation set, and all remaining labeled pixels may be a test set. The training set, validation set, and test set are used for training, validation, and testing of the model, respectively.

[0108] In some embodiments, considering that a pre-trained hyperspectral classification model is directly applied to the process of land object classification, there may be some data that has not been processed by the model during the training process, resulting in poor land object classification results of the model for new hyperspectral images. It can be considered to collect some sample data to test the pre-trained hyperspectral classification model. If the model output result shows that the land object classification result is highly accurate, it indicates that the model can be directly used for land object classification, and S102 and subsequent steps are executed. Otherwise, the pre-trained hyperspectral classification model needs to be retrained to adjust the weight parameters of the hyperspectral classification model. The embodiments of the present application are not limited to this.

[0109] The specific steps may include:

[0110] Selecting a preset number of image blocks from the hyperspectral image to be classified as sample image blocks, and obtaining a ground object category label corresponding to each of the sample image blocks;

[0111] Inputting each of the sample image blocks into the trained hyperspectral classification model to obtain the ground object category output by the trained hyperspectral classification model;

[0112] According to the comparison result of the ground object category label of each sample image block and the ground object category output by the trained hyperspectral classification model, the accuracy of ground object classification of the trained hyperspectral classification model is determined; specifically, the accuracy of ground object classification of the trained hyperspectral classification model can be judged by the ratio of the number of correctly classified sample image blocks to the total number of sample image blocks;

[0113] If the accuracy value is greater than the preset threshold, it is determined that the trained hyperspectral classification model can be directly applied to ground object classification, and S102 and subsequent steps are executed;

[0114] Otherwise, the sample set, validation set and test set for model training are re-collected from the hyperspectral image to be classified to retrain the trained hyperspectral classification model.

[0115] Specifically, in S106, the fused feature δ(x) can be mapped to the corresponding ground object category label through the global average pooling layer and the fully connected layer of the hyperspectral classification model to generate a ground object distribution map corresponding to the hyperspectral image.

[0116] For example, the corresponding feature type can be determined according to the mapping relationship between the fusion feature and the feature type, and the color corresponding to each feature type can be determined, and the color is filled into the map area corresponding to the fusion feature, so as to generate a feature distribution map, as shown in the following example. Figure 2-5 shown.

[0117] In some specific embodiments, the publicly available hyperspectral datasets Matiwan Village dataset and WHU-LongKou dataset may be selected to perform the above steps S101-S106, specifically including:

[0118] Matiwan Village dataset The original hyperspectral image of the dataset is collected by the Gaofen Special Aerial System Full Spectrum Multimodal Imaging Spectrometer above City A. It has 250 bands, a spectral range of 400-1000nm, an image size of 3750×1580 pixels, and a spatial resolution of 0.5m. There are 18 types of ground objects, mainly cash crops. More details of the Matiwan Village dataset are as follows Figure 2 as shown in .

[0119] The WHU-LongKou dataset was collected by a DJI M600 Pro drone platform equipped with a Headwall Nano-Hyperspec imaging sensor over City B. HSI contains a total of 270 bands, with a spectral range from 400nm to 1000nm and a spatial resolution of approximately 0.463m. A total of 9 types of land features are included in the study area, 6 of which are cash crops. More information is available at Figure 3 as shown in .

[0120] In some embodiments, the overall accuracy (OA), average accuracy (AA) and Kappa coefficient (K) can be used to evaluate the classification accuracy and compare with 7 mainstream benchmark models, including ML-based RF model, RNN-based Bi-LSTM model, CNN-based3D-CNN model, CNN-based CLOLN model, Transformer-based Spectral Former model, Transformer-based morphFormer model, SSM-based Mamba HSI model, SSM-based LF-MambaNet model. For the Matiwan Village dataset, 100 pixels are selected as training sets and validation sets respectively, and the remaining samples are used as test sets; for the WHU-LongKou dataset, 30 pixels can be selected as training sets and validation sets, and the rest are used for testing, aiming to verify the classification performance of the classification method provided in the embodiment of the present application under small sample conditions, and the results are shown in Table 1.

[0121] Table 1

[0122]

[0123] Specifically, Figure 4 middle Figure 4 a is an accurate land feature classification map, Figure 4 b corresponds to the object classification map output by the ML-based RF model, Figure 4 c corresponds to the RNN-based Bi-LSTM model, Figure 4 d corresponds to the object classification map output by the CNN-based 3D-CNN model. Figure 4 e corresponds to the object classification map output by the CNN-based CLOLN model, Figure 4 f corresponds to the object classification map output by the Transformer-based Spectral Former model, Figure 4 g corresponds to the object classification map output by the Transformer-based morphFormer model, Figure 4h corresponds to the object classification map output by the SSM-based Mamba HSI model, Figure 4 i corresponds to the object classification map output by the SSM-based LF-MambaNet model. From Table 1 and Figure 4 It can be found that the SSM-based method achieved the best classification effect on the Matiwan Village dataset. The LF-MambaNet model provided in the embodiment of the present application obtained the highest OA (85.89%), AA (92.52%) and Kappa coefficient (0.8375), and achieved the best classification performance in 18 categories. Most of the objects in the MV dataset are crops or cash crops, and their spectral information is very similar. Therefore, the methods based on ML, RNN and CNN are difficult to learn the deep features in the training samples, and it is impossible to effectively distinguish these objects. The OA of the LF-MambaNet model provided in the embodiment of the present application is 6.87% higher than that of MambaHSI, and the training time for each round is shortened by 0.45 seconds. This is mainly because LF-MambaNet abandons high-frequency features and retains low-frequency features and key information. In this way, while removing noise and interference, it reduces the complexity of training, thereby improving accuracy and improving training efficiency. Judging from the classification graph results, except for the SSM-based method, the results obtained by the other classifiers all have salt and pepper noise. The classification map of the LF-MambaNet model provided in the embodiment of the application has the least noise, the lightest degree of ground object confusion, and the highest degree of closeness to the real ground image.

[0124] The classification accuracy and classification diagram of the WHU-LongKou dataset are shown in Table 2 and Figure 5 As shown, Figure 5 middle Figure 5 a is an accurate land feature classification map, Figure 5 b corresponds to the object classification map output by the ML-based RF model, Figure 5 c corresponds to the RNN-basedBi-LSTM model, Figure 5 d corresponds to the object classification map output by the CNN-based 3D-CNN model. Figure 5 e corresponds to the object classification map output by the CNN-based CLOLN model, Figure 5 f corresponds to the object classification map output by the Transformer-based Spectral Former model, Figure 5 g corresponds to the object classification map output by the Transformer-based morphFormer model, Figure 5 h corresponds to the object classification map output by the SSM-based Mamba HSI model, Figure 5i corresponds to the object classification map output by the SSM-based LF-MambaNet model. The performance of the LF-MambaNet model provided in the embodiment of the present application in this data set is slightly inferior to that of the Matiwan Villages data set. It achieved the highest classification accuracy in 4 of the 9 categories. The Transformer-based method that emphasizes local feature extraction achieved the best results in the classification of five types of objects, which also highlights the influence of the establishment of local context features on the classification effect of hyperspectral images. From the classification map, the LF-MambaNet model provided in the embodiment of the present application can better distinguish objects with extremely similar spectral and spatial characteristics, such as broad-leaved soybeans and narrow-leaved soybeans. This distinguishing ability enables the LF-MambaNet model provided in the embodiment of the present application to have an advantage in complex scenes, and can dynamically adjust weights to enhance the ability to distinguish different features. Even if there is noise interference in the data or the features of the objects are unclear, the model can still stably output accurate classification results.

[0125] The following are device embodiments of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0126] See next Figure 6 , is a schematic diagram of the structure of a small sample hyperspectral classification device based on low-frequency features and Mamba provided by an exemplary embodiment of the present application. The device can be implemented as all or part of a terminal through software, hardware, or a combination of both, and can also be integrated on a server as an independent module. A small sample hyperspectral classification device 60 based on low-frequency features and Mamba in an embodiment of the present application can be applied to a terminal or a cloud. The device 60 includes an image processing module 601 and a hyperspectral classification module 602, wherein:

[0127] The image processing module 601 is used to obtain a hyperspectral image to be classified, and extract a plurality of image blocks of the same size based on the hyperspectral image;

[0128] The hyperspectral classification module 602 is used to input each of the image blocks into a trained hyperspectral classification model, and preprocess each of the image blocks through the hyperspectral classification model to obtain a preprocessed feature map;

[0129] The hyperspectral classification module 602 is further used to perform feature decoupling based on the feature graphs, and extract low-frequency features corresponding to each of the feature graphs;

[0130] The hyperspectral classification module 602 is also used to input the extracted low-frequency features into the spatial Mamba module in the hyperspectral classification model to extract long-distance spatial-spectral dependent features;

[0131] The hyperspectral classification module 602 is also used to obtain fusion features based on the low-frequency features and the corresponding long-range spatial-spectral dependency features;

[0132] The hyperspectral classification module 602 is further used to determine the ground object category corresponding to each image block based on the mapping relationship between the fusion feature and the ground object category, and generate a ground object distribution map corresponding to the hyperspectral image.

[0133] It should be noted that the device 60 provided in the above embodiment only uses the division of the above functional modules as an example when executing the hyperspectral classification method based on low-frequency features and Mamba. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above embodiment and the hyperspectral classification method embodiment based on low-frequency features and Mamba belong to the same concept. The implementation process is detailed in the method embodiment and will not be repeated here.

[0134] An embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the program.

[0135] See also Figure 7 , which is a structural block diagram of an electronic device provided in an embodiment of the present application.

[0136] like Figure 7 As shown, the electronic device 700 includes: a processor 701 and a memory 702 .

[0137] In the embodiment of the present application, the processor 701 is the control center of the computer system, which can be a processor of a physical machine or a processor of a virtual machine. The processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 701 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array).

[0138] The processor 701 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also called a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0139] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments of the present application, the non-transitory computer-readable storage medium in the memory 702 is used to store at least one instruction, which is used to be executed by the processor 701 to implement the method in the embodiment of the present application.

[0140] In some embodiments, the electronic device 700 further includes: a peripheral device interface 703 and at least one peripheral device 704. The processor 701, the memory 702 and the peripheral device interface 703 can be connected via a bus or a signal line. Each peripheral device 704 can be connected to the peripheral device interface 703 via a bus, a signal line or a circuit board. Specifically, the peripheral device 704 includes: a display screen, a camera and an audio circuit. The peripheral device interface 703 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 701 and the memory 702.

[0141] In some embodiments of the present application, the processor 701, the memory 702, and the peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments of the present application, any one or two of the processor 701, the memory 702, and the peripheral device interface 703 can be implemented on a separate chip or circuit board. This embodiment of the present application does not specifically limit this.

[0142] The electronic device structure block diagram shown in the embodiment of the present application does not constitute a limitation on the electronic device 700. The electronic device 700 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0143] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method of any of the above embodiments are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0144] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution can be essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A hyperspectral classification method based on low-frequency features and Mamba, characterized in that: include: Acquire a hyperspectral image to be classified, and extract a plurality of image blocks of the same size based on the hyperspectral image; Inputting each of the image blocks into a trained hyperspectral classification model, preprocessing each of the image blocks through the hyperspectral classification model to obtain a preprocessed feature map; Perform feature decoupling based on the feature graphs to extract low-frequency features corresponding to each of the feature graphs; Inputting the extracted low-frequency features into the spatial Mamba module in the hyperspectral classification model to extract long-range spatial-spectral dependency features; The fusion features are obtained based on the low-frequency features and the corresponding long-range spatial-spectral dependency features; The ground object category corresponding to each image block is determined based on the mapping relationship between the fusion features and the ground object category, and a ground object distribution map corresponding to the hyperspectral image is generated.

2. The method according to claim 1, characterized in that The step of extracting a plurality of image blocks of the same size based on the hyperspectral image by using the hyperspectral classification model includes: Extracting each pixel point in the hyperspectral image by using the hyperspectral classification model; Taking each pixel as the center, extract an image block with a preset width and a preset height; The size of each image block is w×h×p; Wherein, w is the preset width, h is the preset height, and p is the number of bands.

3. The method according to claim 2, characterized in that The step of inputting each of the image blocks into a trained hyperspectral classification model and preprocessing each of the image blocks by the hyperspectral classification model to obtain a preprocessed feature map includes: Input the image blocks into the two-dimensional convolution layer of the trained hyperspectral classification model, map each image block into a feature space of the same dimension, and output a feature map of uniform dimension; The dimension of the output feature map is Where b is the batch size of all image patches and c is the dimension of the feature space.

4. The method according to claim 3, characterized in that Before performing feature decoupling based on the feature graph, the method further includes: Each of the feature maps is divided into a high-frequency component and a low-frequency component by the hyperspectral classification model; Based on the two-dimensional convolution layer of the hyperspectral classification model, the high-frequency component and the low-frequency component are respectively reduced in dimension, and the high-frequency component and the low-frequency component after the dimension reduction are output; The performing feature decoupling based on the feature graph to extract low-frequency features corresponding to each feature graph includes: Inputting the reduced-dimensional high-frequency component and the reduced-dimensional low-frequency component into two consecutive high- and low-frequency separation modules in the hyperspectral classification model for feature decoupling, and processing the reduced-dimensional high-frequency component and the reduced-dimensional low-frequency component by discrete wavelet transform to separate high-frequency features and low-frequency features; The low frequency features are preserved.

5. The method according to claim 1, characterized in that The extracted low-frequency features are input into the spatial Mamba module in the hyperspectral classification model to extract long-distance spatial-spectral dependent features, including: Flattening the low-frequency features into corresponding feature units through the unit module of the spatial Mamba module; The feature unit is processed by the Mamba module of the spatial Mamba module, so that the hyperspectral classification model extracts long-distance spatial-spectral dependent features based on the feature unit according to the pre-learned long-distance spatial-spectral functional relationship between features.

6. The method according to claim 5, characterized in that The fusion feature is obtained based on the low-frequency feature and the corresponding long-distance spatial-spectral dependency feature, including: Preprocessing the long-distance space-spectrum dependence feature to change the size of the long-distance space-spectrum dependence feature to the same size as the low-frequency feature; The preprocessed long-distance spatial-spectral dependent feature and the low-frequency feature are added to obtain the fused feature.

7. The method according to claim 1, characterized in that The training steps of the hyperspectral classification model include: Acquire a sample hyperspectral image, extract a plurality of image blocks based on the sample hyperspectral image, extract a preset number of image blocks and a ground object type label corresponding to each image block to generate a training set; Inputting the training set into the spectral classification model to train the spectral classification model, so that the spectral classification model learns the long-range spatial-spectral function relationship between features based on the training set; The output result of the spectral classification model is obtained, and a loss function is constructed based on the output result and the ground object type label corresponding to each image block in the training set. Then, when the spectral classification model converges according to the loss function, the model parameters of the converged spectral classification model are output to obtain the trained hyperspectral classification model.

8. A hyperspectral classification device based on low-frequency features and Mamba, characterized in that: include: An image processing module, used for acquiring a hyperspectral image to be classified, and extracting a plurality of image blocks of the same size based on the hyperspectral image; A hyperspectral classification module, used for inputting each of the image blocks into a trained hyperspectral classification model, preprocessing each of the image blocks through the hyperspectral classification model, and obtaining a preprocessed feature map; The hyperspectral classification module is also used to perform feature decoupling based on the feature graphs to extract low-frequency features corresponding to each of the feature graphs; The hyperspectral classification module is also used to input the extracted low-frequency features into the spatial Mamba module in the hyperspectral classification model to extract long-distance spatial-spectral dependency features; The hyperspectral classification module is also used to obtain fusion features based on low-frequency features and corresponding long-range spatial-spectral dependency features; The hyperspectral classification module is also used to determine the ground object category corresponding to each image block based on the mapping relationship between the fusion features and the ground object category, and generate a ground object distribution map corresponding to the hyperspectral image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images