A hyperspectral classification method based on low-frequency features and mamba

Abstract

The embodiment of the application discloses a hyperspectral classification method based on low-frequency features and Mamba, relates to the technical field of hyperspectral remote sensing image processing, and comprises the following steps: acquiring a hyperspectral image to be classified, extracting a plurality of image blocks, inputting the hyperspectral classification model trained into a preprocessed feature map obtained through preprocessing; based on the feature map, low-frequency features are extracted through feature decoupling, and long-distance spatial-spectral dependence features are extracted by inputting a spatial Mamba module; fusion features are obtained based on the low-frequency features and the long-distance spatial-spectral dependence features; the ground object category corresponding to each image block is determined based on the fusion features, and a ground object distribution map corresponding to the hyperspectral image is generated. By separating high-frequency features and low-frequency features and retaining the low-frequency features, the application can filter and focus on key spectral and spatial features, reduce irrelevant information interference, and enable the ground object category corresponding to each image block to be determined more quickly and accurately according to the hyperspectral image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hyperspectral remote sensing image processing, and particularly relates to a hyperspectral classification method based on low-frequency features and Mamba. BACKGROUND

[0002] Hyperspectral images can obtain the reflection or radiation information of ground objects from a large number of continuous and narrow spectral bands, usually containing dozens or even hundreds of bands, with extremely high spectral resolution. This allows hyperspectral images to finely depict the spectral characteristics of ground objects, and different ground objects will exhibit different spectral curves at different bands, thereby effectively distinguishing ground objects that are similar in appearance but different in spectral characteristics. Hyperspectral images have a wide range of applications in crop monitoring, forest identification, mineral exploration, environmental monitoring, urban planning, and ocean exploration.

[0003] Traditional hyperspectral image classification methods are mainly divided into two categories: pure pixel-based methods and mixed pixel-based methods. Pure pixel-based methods mainly use spectral features, such as spectral matching and statistical model classification, but are easily affected by various factors, leading to a decrease in classification accuracy. Mixed pixel-based methods consider spatial resolution and ground object complexity, treating pixels as a mixture of ground objects, but have problems such as large data volume, strong correlation, high redundancy, and difficulty in obtaining training samples.

[0004] Deep learning models can automatically extract important features from data without the need for manual feature engineering, and have strong feature extraction capabilities. Among them, convolutional neural networks (CNN) can be used for hyperspectral image classification, specifically by using convolution kernels and pooling operations to automatically extract texture, shape, and other features from images, and combining these features for classification. However, convolutional neural network models require a large amount of data and computing resources, and are prone to overfitting. Recurrent neural networks (RNN) can handle sequential data, but their application in hyperspectral image classification is relatively limited. RNN models can capture temporal dependencies in data, but may face computational complexity and gradient vanishing problems when dealing with high-dimensional data. Graph convolutional neural networks (GCN) have advantages in handling graph-structured data, and in hyperspectral image classification, GCN models can utilize spatial relationships between pixels for classification. However, GCN models require the construction of a graph structure, and have high requirements for the quality and complexity of the constructed graph. Generative adversarial networks (GAN) can generate realistic data samples, but their application in hyperspectral image classification is still in the exploratory stage. GAN models can improve the generalization ability of classifiers through adversarial generation, but the training process is complex and difficult to control.

[0005] The Transformer model can also be applied in the field of hyperspectral image classification. The model takes the self-attention mechanism as the core and can automatically focus on different parts of the data to capture global feature information. Compared with CNN, RNN and GCN, the Transformer has excellent global feature capturing capability, does not need to slide the convolution kernel or process the sequence, directly focuses on the relationship between pixels in the hyperspectral image, and fully utilizes the spectral and spatial information. In addition, the Transformer is more suitable for long sequence data processing, avoiding the problem of gradient disappearance or explosion, and stably processing the multi-band long sequence information of the hyperspectral image. However, the model based on the Transformer has the disadvantage of high computational complexity, especially when processing large-scale hyperspectral image data, the amount of calculation of the self-attention mechanism is large, resulting in slow training and inference speed.

[0006] Therefore, there is currently a lack of a method that can accurately and efficiently separate high and low frequency features from original hyperspectral images. SUMMARY

[0007] The hyperspectral classification method based on low frequency features and Mamba provided by the embodiments of the present application is used to solve the defects of the related art, and the technical solution is as follows:

[0008] In a first aspect, the hyperspectral classification method based on low frequency features and Mamba provided by the embodiments of the present application comprises:

[0009] Obtaining a hyperspectral image to be classified, and extracting a plurality of image blocks of the same size based on the hyperspectral image;

[0010] Inputting each of the image blocks into a trained hyperspectral classification model, pre-processing each of the image blocks by the hyperspectral classification model, and obtaining a feature map after pre-processing;

[0011] Decoupling features based on the feature map, and extracting low frequency features corresponding to each of the feature maps;

[0012] Inputting the extracted low frequency features into a spatial Mamba module in the hyperspectral classification model, and extracting long-distance spatial-spectral dependence features;

[0013] Obtaining fusion features based on the low frequency features and the corresponding long-distance spatial-spectral dependence features;

[0014] Determining the ground object category corresponding to each of the image blocks based on the mapping relationship between the fusion features and the ground object category, and generating a ground object distribution map corresponding to the hyperspectral image.

[0015] In an optional implementation of the first aspect, the extracting, by the hyperspectral classification model, a plurality of image blocks of the same size based on the hyperspectral image comprises:

[0016] extracting each pixel point in the hyperspectral image by the hyperspectral classification model;

[0017] extracting an image block of a preset width and a preset height with each pixel point as the center;

[0018] each image block has a size of w×h×p;

[0019] wherein w is the preset width, h is the preset height, and p is the number of bands.

[0020] In an optional implementation of the first aspect, the method further comprises:

[0021] inputting the image blocks into a two-dimensional convolution layer of the trained hyperspectral classification model, mapping each image block to a feature space of the same dimension, and outputting a feature map of a uniform dimension;

[0022] the dimension of the output feature map is

[0023] wherein b is the batch size of all image blocks, and c is the dimension of the feature space.

[0024] In an optional implementation of the first aspect, the method further comprises:

[0025] dividing each feature map into a high-frequency component and a low-frequency component by the hyperspectral classification model;

[0026] performing dimension reduction on the high-frequency component and the low-frequency component based on a two-dimensional convolution layer of the hyperspectral classification model, and outputting a reduced high-frequency component and a reduced low-frequency component;

[0027] the feature decoupling based on the feature map comprises:

[0028] inputting the reduced high-frequency component and the reduced low-frequency component into two consecutive high-low frequency separation modules in the hyperspectral classification model to perform feature decoupling, processing the reduced high-frequency component and the reduced low-frequency component by discrete wavelet transform, and separating a high-frequency feature and a low-frequency feature;

[0029] retaining the low-frequency feature.

[0030] In an optional implementation of the first aspect, the low-frequency feature is input into a spatial Mamba module in the hyperspectral classification model, and a long-distance spatial-spectral dependence feature is extracted, including:

[0031] The low-frequency feature is flattened into corresponding feature units by a unit module of the spatial Mamba module;

[0032] The feature units are processed by a Mamba module of the spatial Mamba module, so that the hyperspectral classification model extracts a long-distance spatial-spectral dependence feature based on the feature units according to a long-distance spatial-spectral function relationship between features learned in advance.

[0033] In an optional implementation of the first aspect, the fusion feature is obtained based on the low-frequency feature and the long-distance spatial-spectral dependence feature, including:

[0034] The long-distance spatial-spectral dependence feature is preprocessed to have the same size as the low-frequency feature;

[0035] The preprocessed long-distance spatial-spectral dependence feature and the low-frequency feature are added to obtain the fusion feature.

[0036] In an optional implementation of the first aspect, the training step of the hyperspectral classification model includes:

[0037] A sample hyperspectral image is obtained, a plurality of image blocks are extracted based on the sample hyperspectral image, and a training set is generated by extracting a preset number of image blocks and a ground object type label corresponding to each image block;

[0038] The training set is input into the spectral classification model to train the spectral classification model, so that the spectral classification model learns a long-distance spatial-spectral function relationship between features based on the training set;

[0039] An output result of the spectral classification model is obtained, a loss function is constructed based on the output result and the ground object type label corresponding to each image block in the training set, and when the spectral classification model converges according to the loss function, a model parameter of the converged spectral classification model is output to obtain the trained hyperspectral classification model.

[0040] In a second aspect, the embodiments of the present application further provide a hyperspectral classification device based on low-frequency features and Mamba, including:

[0041] An image processing module is configured to obtain a hyperspectral image to be classified, and extract a plurality of image blocks of the same size based on the hyperspectral image;

[0042] a hyperspectral classification module, configured to input each of the image blocks into a trained hyperspectral classification model, and to perform preprocessing on each of the image blocks by using the hyperspectral classification model to obtain a feature map after preprocessing;

[0043] The hyperspectral classification module is further configured to perform feature decoupling based on the feature map, and to extract low-frequency features corresponding to each of the feature maps;

[0044] The hyperspectral classification module is further configured to input the extracted low-frequency features into a spatial Mamba module in the hyperspectral classification model, and to extract long-distance spatial-spectral dependence features;

[0045] The hyperspectral classification module is further configured to obtain fusion features based on the low-frequency features and the corresponding long-distance spatial-spectral dependence features;

[0046] The hyperspectral classification module is further configured to determine a ground object category corresponding to each of the image blocks based on a mapping relationship between the fusion features and the ground object categories, and to generate a ground object distribution map corresponding to the hyperspectral image.

[0047] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method provided in the first aspect or any of the implementation manners of the first aspect when executing the program.

[0048] In a fourth aspect, a non-transitory computer-readable storage medium is provided, which stores a computer program executable by a processor to implement the method provided in the first aspect or any of the implementation manners of the first aspect.

[0049] The technical scheme provided by some embodiments of the present application has at least the following beneficial effects:

[0050] The high-spectrum classification method based on low-frequency features and Mamba provided by the embodiment of the application can filter and focus on key spectral and spatial features, reduce irrelevant information interference, and thus more accurately extract long-distance spatial-spectrum dependent features, so that the corresponding ground object category of each image block can be more quickly and accurately determined according to the high-spectrum image, and reliable data support can be provided for ground object type division in fields such as crop monitoring, forest identification, mineral exploration, environmental monitoring, city planning, and ocean exploration. In addition, by filtering and focusing on key spectral and spatial features and reducing irrelevant information interference, the feature learning process of the model in the training process can be accelerated, and the dependence on a large amount of training data can be reduced, the feature learning efficiency of the model can be improved, and the model can more quickly and accurately extract key features when processing high-spectrum images. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the application or related art, the drawings needed to be used in the embodiments or related art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 is a flowchart of a high-spectrum classification method based on low-frequency features and Mamba provided by the embodiment of the application;

[0053] Figure 2 is one of ground object classification diagrams of a high-spectrum classification method based on low-frequency features and Mamba provided by the embodiment of the application;

[0054] Figure 3 is another of ground object classification diagrams of a high-spectrum classification method based on low-frequency features and Mamba provided by the embodiment of the application;

[0055] Figure 4 is a third of ground object classification diagrams of a high-spectrum classification method based on low-frequency features and Mamba provided by the embodiment of the application;

[0056] Figure 5 is a fourth of ground object classification diagrams of a high-spectrum classification method based on low-frequency features and Mamba provided by the embodiment of the application;

[0057] Figure 6 is a structural diagram of a high-spectrum classification device based on low-frequency features and Mamba provided by the embodiment of the application;

[0058] Figure 7 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0059] For the purposes of the present application, the technical solutions and advantages thereof are more clearly apparent, the technical solutions of the present application will be described clearly and completely below in conjunction with the drawings of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0060] The terms “include” and “have” and any variations thereof in the specification and claims of the present application and the above-described drawings are intended to cover the non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but can optionally include steps or modules that are not listed, or can optionally include other steps or modules inherent to the process, method, product or device.

[0061] It should be noted that the terms “first” and “second” involved in the present application are only to distinguish similar objects, and do not represent a specific order of the objects. Understandably, “first” and “second” can be interchanged in a specific order or sequence as appropriate. It should be understood that the objects distinguished by “first” and “second” can be interchanged as appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those described or illustrated herein.

[0062] The present application will be described in detail below in conjunction with specific embodiments.

[0063] Next, a hyperspectral classification method based on low-frequency features and Mamba provided by an embodiment of the present application will be described. Figure 1 , Figure 1 , Figure 1 A flowchart of a hyperspectral classification method based on low-frequency features and Mamba provided by an embodiment of the present application is shown. As shown in Figure 1 , the method comprises the following steps:

[0064] S101, obtaining a hyperspectral image to be classified, and extracting a plurality of image blocks of the same size based on the hyperspectral image;

[0065] S102, inputting each of the image blocks into a trained hyperspectral classification model, and pre-processing each of the image blocks by the hyperspectral classification model to obtain a pre-processed feature map;

[0066] S103, decoupling features based on the feature map, and extracting low-frequency features corresponding to each feature map;

[0067] S104, inputting the extracted low-frequency features into a spatial Mamba module in the hyperspectral classification model to extract long-distance spatial-spectral dependence features;

[0068] S105, obtaining fusion features based on low-frequency features and corresponding long-distance spatial-spectral dependence features;

[0069] S106, determining the ground object category corresponding to each image block based on the mapping relationship between the fusion features and the ground object category, and generating the ground object distribution map corresponding to the hyperspectral image.

[0070] Specifically, in S101, each pixel point in the hyperspectral image can be extracted by the hyperspectral classification model;

[0071] Each image block is extracted with a preset width and a preset height as the center;

[0072] The size of each image block is w x h x p;

[0073] Wherein, w is the preset width, h is the preset height, and p is the number of bands.

[0074] Specifically, each pixel point in the hyperspectral image to be classified can be selected, and an image block centered on the pixel and retaining all band information of the original hyperspectral image can be extracted in the height and width dimensions.

[0075] Specifically, in S102, the image block can be input into the two-dimensional convolution layer of the trained hyperspectral classification model, and each image block can be mapped to a feature space with the same dimension, and a feature map with uniform dimension can be output;

[0076] The dimension of the output feature map is

[0077] Wherein, b is the batch size of all image blocks, and c is the dimension of the feature space.

[0078] Specifically, the feature map is denoted as The parameters w and h can be set to 11, the parameter b can be set to 128, and the parameter c can be set to 128.

[0079] Specifically, in S103, before decoupling features based on the feature map, each feature map F(x) can be divided into high-frequency component F H (x) and low-frequency component F L (x) by the hyperspectral classification model.

[0080] The two-dimensional convolution layer based on the hyperspectral classification model respectively processes the high-frequency component F H (x) and the low-frequency component F L (x) to reduce the dimension, and outputs the reduced high-frequency component and the reduced low-frequency component.

[0081] Specifically, the complexity of subsequent high-low frequency classification can be reduced by dimension reduction, and the size of the reduced high-frequency component y H (x) and the reduced low-frequency component y L (x) is where a is a dimension reduction factor, and a is set to 4 in this embodiment.

[0082] Specifically, in S103, the reduced high-frequency component y H (x) and the reduced low-frequency component y L (x) can be input into the two continuous high-low frequency separation modules in the hyperspectral classification model for feature decoupling, and the reduced high-frequency component and the reduced low-frequency component are processed by discrete wavelet transform to separate high-frequency features and low-frequency features.

[0083] The low-frequency features are retained.

[0084] In some embodiments, the high-low frequency separation module is implemented based on discrete wavelet transform, which decomposes y H (x) and y L (x) into four key components: LL, LH, HL and HH. Among them, LL is the low-frequency component, which contains the main information and structure of the input feature, and is highly similar to the original image, retaining its core content and structure. While LH, HL and HH represent the details and edge information of the high-frequency component in the horizontal, vertical and diagonal directions, respectively.

[0085] The calculation process can be represented by the following formula, where DWT represents discrete wavelet transform:

[0086]

[0087] Further, the obtained high-frequency features and low-frequency features can be fused respectively to obtain the discrete wavelet transform final extracted high-frequency features Y H and low-frequency features Y L , and the formula is applied:

[0088]

[0089] wherein,

[0090] Then, two-dimensional convolution operation with two convolution kernels of 1 is used to obtain Y H and YL The number of channels is mapped back to the same dimension as the input data, that is, the same dimension as the feature map, that is, channel c.

[0091] Specifically, the spatial Mamba module is composed of a unit module and a Mamba module. In S104, the low-frequency features can be flattened into corresponding feature units by the unit module of the spatial Mamba module, and the formula is applied:

[0092] Token=Flatten(Y L (x));

[0093] Wherein, Where b represents the batch size, w and h represent the width and height respectively, and c represents the dimension of the feature space.

[0094] Wherein, Token can be understood as the minimum unit corresponding to each part or each feature in the image. Through the flattening (Flatten) operation, the multi-dimensional image features can be converted into low-dimensional vectors, that is, the feature units token are obtained by flattening.

[0095] Further, the Mamba module of the spatial Mamba module processes the feature units, so that the hyperspectral classification model extracts long-distance spatial-spectral dependent features M(x) based on the feature units according to the long-distance spatial-spectral function relationship between the features learned in advance.

[0096] In some embodiments, the long-distance spatial-spectral dependent features M(x) can also be preprocessed to process the size of the long-distance spatial-spectral dependent features to the same size as the low-frequency features.

[0097] The preprocessed long-distance spatial-spectral dependent features and the low-frequency features are added to obtain the fusion features.

[0098] Specifically, it can be represented as shown in the following formula:

[0099] M(x)=Reshape(SiLU(GN(Mamba(Token))));

[0100] That is, Token is input into the Mamba architecture to extract long-distance spatial-spectral dependent features M(x), and finally, the extracted M(x) is reshaped.

[0101] Wherein, GN represents group normalization, SiLU is a sine linear unit representing an activation function, and Reshape represents a reshaping operation.

[0102] Specifically, in S105, the low-frequency feature Y obtained in S103 can be added to the long-distance spatial-spectrum dependent feature M(x) obtained in S104 to generate a fusion feature, so as to achieve a more comprehensive feature representation. L (x) adding the long-distance spatial-spectrum dependent feature M(x) obtained in S104 to generate a fusion feature, so as to achieve a more comprehensive feature representation.

[0103] In some embodiments, the training step of the hyperspectral classification model comprises:

[0104] A sample hyperspectral image is obtained, and a plurality of image blocks are extracted based on the sample hyperspectral image. A training set is generated by extracting a preset number of image blocks and a ground object type label corresponding to each image block.

[0105] The training set is input into the spectral classification model to train the spectral classification model, so that the spectral classification model learns the long-distance spatial-spectrum function relationship between features based on the training set.

[0106] An output result of the spectral classification model is obtained, a loss function is constructed based on the output result and the ground object type label corresponding to each image block in the training set, and the model parameters of the converged spectral classification model are output when the spectral classification model converges according to the loss function, so as to obtain the trained hyperspectral classification model.

[0107] In some embodiments, 100 pixels can be selected as the training set and the validation set, and all the remaining labeled pixels can be used as the test set. The training set, the validation set, and the test set are respectively used for training, validation, and testing of the model.

[0108] In some embodiments, considering that the pre-trained hyperspectral classification model is directly applied to the process of ground object classification, there may be some data that the model has not processed during the training process, resulting in poor ground object classification results of the model on new hyperspectral images. Therefore, part of the sample data can be collected to test the pre-trained hyperspectral classification model. If the model output result shows that the ground object classification result is high in accuracy, it indicates that the model can be directly used for ground object classification, and S102 and the subsequent steps are executed. Otherwise, the pre-trained hyperspectral classification model needs to be retrained to adjust the weight parameters of the hyperspectral classification model, which is not limited in the embodiments of the present application.

[0109] Specifically, the method can comprise the following steps:

[0110] A preset number of image blocks are selected from the hyperspectral image to be classified as sample image blocks, and a ground object category label corresponding to each sample image block is obtained.

[0111] Each sample image block is input into the trained hyperspectral classification model, and a ground object category output by the trained hyperspectral classification model is obtained.

[0112] According to the comparison result of the ground object category label of each sample image block and the ground object category output by the trained hyperspectral classification model, the accuracy of ground object classification of the trained hyperspectral classification model is determined; specifically, the accuracy of ground object classification of the trained hyperspectral classification model can be determined by the proportion of the number of correctly classified sample image blocks to the total number of sample image blocks.

[0113] If the value of the accuracy is greater than the preset threshold value, it is determined that the trained hyperspectral classification model can be directly applied to ground object classification, and S102 and the subsequent steps are executed.

[0114] Otherwise, the sample set, the verification set and the test set used for model training are re-collected from the hyperspectral image to be classified, and the trained hyperspectral classification model is retrained.

[0115] Specifically, in S106, the fusion feature δ(x) can be mapped to the corresponding ground object category label through the global average pooling layer and the full connection layer of the hyperspectral classification model, and the ground object distribution map corresponding to the hyperspectral image is generated.

[0116] Exemplarily, the corresponding ground object type can be determined according to the mapping relationship between the fusion feature and the ground object type, the color corresponding to each ground object type can be determined, and the color is filled into the map area corresponding to the fusion feature, so as to generate the ground object distribution map, as shown in Figures 2-5 .

[0117] In some specific embodiments, the public hyperspectral data sets Matiwan Village data set and WHU-LongKou data set can be selected to execute the steps of S101-S106, and the specific steps include:

[0118] The original hyperspectral image of the Matiwan Village data set is collected by the high-resolution special aerial system full-spectrum multi-modal imaging spectrometer above the city A, has 250 bands, the spectral range is 400-1000nm, the image size is composed of 3750x1580 pixels, and the spatial resolution is 0.5m. The ground object category is 18 categories, mainly economic crops. More details of the Matiwan Village data set are shown in Figure 2 .

[0119] The WHU-LongKou dataset was collected by DJI M600 Pro unmanned aerial vehicle platform equipped with Headwall Nano-Hyperspec imaging sensor over City B. HSI contains 270 bands in total, with spectral range from 400nm to 1000nm, and spatial resolution of about 0.463m. The study area contains 9 ground objects in total, of which 6 are economic crops. More information is shown in Figure 3

[0120] In some embodiments, the classification accuracy can be evaluated using overall accuracy (OA), mean accuracy (AA) and kappa coefficient (K), and compared with 7 mainstream benchmark models, including ML-based RF model, RNN-based Bi-LSTM model, CNN-based 3D-CNN model, CNN-based CLOLN model, Transformer-based Spectral Former model, Transformer-based morphFormer model, SSM-based Mamba HSI model, and SSM-based LF-MambaNet model. For the Matiwan Village dataset, 100 pixels are selected from it as the training set and the validation set respectively, and the remaining samples are used as the test set; for the WHU-LongKou dataset, 30 pixels can be selected as the training set and the validation set, and the rest are used for testing, aiming to verify the classification performance of the classification method provided in the embodiments of the present application under the condition of small sample, and the results are shown in Table 1.

[0121] Table 1

[0122]

[0123] Specifically, Figure 4 in Figure 4 a is an accurate ground object classification map, Figure 4 b corresponds to the ground object classification map output by the ML-based RF model, Figure 4 c corresponds to the RNN-based Bi-LSTM model, Figure 4 d corresponds to the ground object classification map output by the CNN-based 3D-CNN model, Figure 4 e corresponds to the ground object classification map output by the CNN-based CLOLN model, Figure 4 f corresponds to the ground object classification map output by the Transformer-based Spectral Former model, Figure 4 g corresponds to the ground object classification map output by the Transformer-based morphFormer model, Figure 4 ​h corresponds to the ground object classification map output by the SSM-based Mamba HSI model, Figure 5 i corresponds to the ground object classification map output by the SSM-based LF-MambaNet model, from Table 1 and Figure 5 It can be found from Table 1 and Table 2 that the SSM-based method achieves the best classification effect on the Matiwan Village dataset. The LF-MambaNet model provided in the embodiments of the present application obtains the highest OA (85.89%), AA (92.52%) and Kappa coefficient (0.8375), and achieves the best classification performance in 18 categories. Most of the ground objects in the MV dataset are crops or economic crops, and their spectral information is extremely similar, so it is difficult for the methods based on ML, RNN and CNN to learn the deep features in the training samples, and it is also difficult to effectively distinguish these ground objects. The OA of the LF-MambaNet model provided in the embodiments of the present application is improved by 6.87% compared with the Mamba HSI, and the training time of each round is shortened by 0.45 seconds. This is mainly because the LF-MambaNet discards high-frequency features and retains low-frequency features and key information, so that the noise and interference are removed while the complexity of training is reduced, thereby improving the precision and improving the training efficiency. From the classification map results, it can be seen that the results of the remaining classifiers all have salt and pepper noise phenomenon except the SSM-based method. The LF-MambaNet model provided in the embodiments of the present application has the least noise in the classification map, the least confusion of ground objects, and the highest closeness to the real ground image.

[0124] The classification accuracy and classification map of the WHU-LongKou dataset are shown in Table 2 and Figure 5 , Figure 5 Table 2 Figure 5 a is an accurate ground object classification map, Figure 5 b corresponds to the ground object classification map output by the ML-based RF model, Figure 5 c corresponds to the RNN-based Bi-LSTM model, Figure 5 d corresponds to the ground object classification map output by the CNN-based 3D-CNN model, Figure 5 e corresponds to the ground object classification map output by the CNN-based CLOLN model, Figure 5 f corresponds to the ground object classification map output by the Transformer-based Spectral Former model, Figure 6 g corresponds to the ground object classification map output by the Transformer-based morphFormer model, Figure 7 h corresponds to the ground object classification map output by the SSM-based Mamba HSI model, Figure 7i corresponds to the ground object classification map output by the SSM-based LF-MambaNet model. The LF-MambaNet model provided in the embodiments of the present application performs slightly worse in this dataset than in the Matiwan Villages dataset, and it achieves the highest classification accuracy in 4 of the 9 classes. The Transformer-based method that emphasizes local feature extraction achieves the best results in the classification of five types of ground objects, which highlights the influence of the establishment of local context features on the classification effect of hyperspectral images. From the classification map, it can be seen that the LF-MambaNet model provided in the embodiments of the present application can better distinguish ground objects with very similar spectral and spatial features, such as broadleaf soybeans and narrowleaf soybeans. This distinguishing ability makes the LF-MambaNet model provided in the embodiments of the present application advantageous in complex scenes, and can dynamically adjust the weight to enhance the discrimination ability of different features. Even if the data is disturbed by noise or the features of the ground objects are unclear, the model can still stably output accurate classification results.

[0125] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the present application.

[0126] Next, please refer to ​ , a structural schematic diagram of a small sample hyperspectral classification device based on low-frequency features and Mamba provided by an exemplary embodiment of the present application. The device can be realized by software, hardware or a combination of the two to become all or part of the terminal, and can also be integrated as an independent module on the server. The small sample hyperspectral classification device based on low-frequency features and Mamba 60 in the embodiments of the present application can be applied to a terminal or cloud, and the device 60 includes an image processing module 601 and a hyperspectral classification module 602, wherein:

[0127] The image processing module 601 is configured to obtain a hyperspectral image to be classified, and extract a plurality of image blocks of the same size based on the hyperspectral image;

[0128] The hyperspectral classification module 602 is configured to input each of the image blocks into a trained hyperspectral classification model, and pre-process each of the image blocks through the hyperspectral classification model to obtain a pre-processed feature map;

[0129] The hyperspectral classification module 602 is further configured to decouple features based on the feature map, and extract low-frequency features corresponding to each of the feature maps;

[0130] The hyperspectral classification module 602 is further configured to input the extracted low-frequency features into a spatial Mamba module in the hyperspectral classification model, and extract long-distance spatial-spectral dependence features.

[0131] The hyperspectral classification module 602 is also used to obtain fused features based on low-frequency features and corresponding long-range spatial-spectral dependent features;

[0132] The hyperspectral classification module 602 is also used to determine the land cover category corresponding to each image patch based on the mapping relationship between fusion features and land cover categories, and generate a land cover distribution map corresponding to the hyperspectral image.

[0133] It should be noted that the device 60 provided in the above embodiments, when executing the hyperspectral classification method based on low-frequency features and Mamba, is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above embodiments and the hyperspectral classification method embodiments based on low-frequency features and Mamba belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0134] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0135] Please see ​ This is a structural block diagram of an electronic device provided in an embodiment of this application.

[0136] like ​ As shown, the electronic device 700 includes a processor 701 and a memory 702.

[0137] In this embodiment, the processor 701 is the control center of the computer system, and can be a processor of a physical machine or a processor of a virtual machine. The processor 701 may include one or more processing cores, such as a 4-core processor or an 8-core processor. The processor 701 can be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array).

[0138] Processor 701 may also include a main processor and a coprocessor. The main processor is a processor used to process data in the wake-up state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor used to process data in the standby state.

[0139] The memory 702 can include one or more computer-readable storage media. The computer-readable storage media can be non-transitory. The memory 702 can also include high-speed random access memory and can include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. In some embodiments of the present application, the non-transitory computer-readable storage medium in the memory 702 is used to store at least one instruction for being executed by the processor 701 to implement the method in the embodiments of the present application.

[0140] In some embodiments, the electronic device 700 further includes a peripheral device interface 703 and at least one peripheral device 704. The processor 701, the memory 702 and the peripheral device interface 703 can be connected through a bus or a signal line. Each peripheral device 704 can be connected to the peripheral device interface 703 through a bus, a signal line or a circuit board. Specifically, the peripheral device interface 703 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 701 and the memory 702.

[0141] In some embodiments of the present application, the processor 701, the memory 702 and the peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments of the present application, any one or two of the processor 701, the memory 702 and the peripheral device interface 703 can be implemented on a separate chip or circuit board. The embodiments of the present application do not make specific limitations in this regard.

[0142] The structure block diagram of the electronic device shown in the embodiments of the present application does not constitute a limitation on the electronic device 700, and the electronic device 700 can include more or fewer components than shown, or combine certain components, or adopt a different arrangement of components.

[0143] The embodiments of the present application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the preceding embodiments. The computer-readable storage medium can include, but is not limited to, any type of disk including a floppy disk, an optical disk, a DVD, a CD-ROM, a microdrive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic or optical card, a nanosystem (including molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0144] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and the necessary general hardware platform from the above description of the embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions or the part that contributes to the related art can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in the various embodiments or some parts of the embodiments.

[0145] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A hyperspectral classification method based on low frequency features and Mamba, characterized in that, The method comprises the following steps: acquiring a hyperspectral image to be classified, and extracting a plurality of image blocks of the same size based on the hyperspectral image; inputting each of the image blocks into a trained hyperspectral classification model, and pre-processing each of the image blocks by the hyperspectral classification model to obtain a feature map after pre-processing; decoupling features based on the feature map, and extracting low-frequency features corresponding to each of the feature maps; inputting the extracted low-frequency features into a spatial Mamba module in the hyperspectral classification model to extract long-distance spatial-spectral dependence features; obtaining fusion features based on the low-frequency features and the corresponding long-distance spatial-spectral dependence features; determining a ground object category corresponding to each of the image blocks based on a mapping relationship between the fusion features and the ground object category, and generating a ground object distribution map corresponding to the hyperspectral image.

2. The method of claim 1, wherein, The method comprises the following steps: extracting each pixel point in the hyperspectral image by the hyperspectral classification model; extracting an image block of a preset width and a preset height with each pixel point as the center; The size of each image block is ; wherein w is the preset width, h is the preset height, and p is the number of bands.

3. The method of claim 2, wherein, The method comprises the following steps: inputting the image blocks into a two-dimensional convolution layer of the trained hyperspectral classification model, mapping each of the image blocks to a feature space of the same dimension, and outputting a feature map of a uniform dimension; The dimension of the output feature map is ; wherein b is the batch size of all image blocks, and c is the dimension of the feature space.

4. The method of claim 3, wherein, Before the decoupling of features based on the feature map, the method further comprises the following steps: dividing each of the feature maps into a high-frequency component and a low-frequency component by the hyperspectral classification model; performing dimension reduction on the high-frequency component and the low-frequency component by a two-dimensional convolution layer of the hyperspectral classification model, and outputting a reduced high-frequency component and a reduced low-frequency component; The method comprises the following steps: inputting the reduced high-frequency component and the reduced low-frequency component into two consecutive high-low frequency separation modules in the hyperspectral classification model to decouple features, processing the reduced high-frequency component and the reduced low-frequency component by discrete wavelet transform, and separating high-frequency features and low-frequency features; retaining the low-frequency features.

5. The method of claim 1, wherein, The method comprises the following steps: flattening the low-frequency features into corresponding feature units by a unit module of the spatial Mamba module; processing the feature units by a Mamba module of the spatial Mamba module, so that the hyperspectral classification model extracts long-distance spatial-spectral dependence features based on the feature units according to a long-distance spatial-spectral function relationship between features learned in advance.

6. The method of claim 5, wherein, The fusion feature based on the low-frequency feature and the corresponding long-distance spatial-spectral dependence feature comprises: The long-distance spatial-spectral dependence feature is preprocessed, and the size of the long-distance spatial-spectral dependence feature is processed to be the same as the size of the low-frequency feature; The preprocessed long-distance spatial-spectral dependence feature and the low-frequency feature are added to obtain the fusion feature.

7. The method of claim 1, wherein, The training step of the hyperspectral classification model comprises: A sample hyperspectral image is obtained, and a plurality of image blocks are extracted based on the sample hyperspectral image; a training set is generated by extracting a preset number of image blocks and a ground object type label corresponding to each image block; The training set is input into the hyperspectral classification model to train the hyperspectral classification model, so that the hyperspectral classification model learns the long-distance spatial-spectral function relationship between features based on the training set; An output result of the hyperspectral classification model is obtained, a loss function is constructed based on the output result and the ground object type label corresponding to each image block in the training set, and when the hyperspectral classification model converges according to the loss function, the model parameters of the converged hyperspectral classification model are output to obtain the trained hyperspectral classification model.

8. A hyperspectral classification device based on low frequency features and Mamba, characterized in that, Comprise: An image processing module is configured to obtain a hyperspectral image to be classified, and extract a plurality of image blocks of the same size based on the hyperspectral image; A hyperspectral classification module is configured to input each of the image blocks into a trained hyperspectral classification model, and perform preprocessing on each of the image blocks by the hyperspectral classification model to obtain a preprocessed feature map; The hyperspectral classification module is further configured to perform feature decoupling based on the feature map to extract a low-frequency feature corresponding to each of the feature maps; The hyperspectral classification module is further configured to input the extracted low-frequency feature into a spatial Mamba module in the hyperspectral classification model to extract a long-distance spatial-spectral dependence feature; The hyperspectral classification module is further configured to obtain a fusion feature based on the low-frequency feature and the corresponding long-distance spatial-spectral dependence feature; The hyperspectral classification module is further configured to determine a ground object category corresponding to each of the image blocks based on a mapping relationship between the fusion feature and the ground object category, and generate a ground object distribution map corresponding to the hyperspectral image.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the method of any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images