A hyperspectral image classification method based on a small amount of training samples

Abstract

The application is a hyperspectral image classification method based on a small amount of training samples. The application relates to the technical field of image processing and classification, and the application obtains an original hyperspectral data set, labels class information of interest, uses principal component analysis on the original data set for dimension reduction, reduces the amount of data to be processed in a non-overlapping information reservation manner, selects a suitable spatial neighborhood size, divides the data into a training set, a validation set and a test set according to a certain proportion, establishes a dual attention model, feeds the training data into the dual attention mechanism model for training, adjusts the parameters of the model training process in a timely manner, selects the best model through cross-validation, and tests the best model on the test set to verify the classification effect.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of image processing and classification thereof, and is a hyperspectral image classification method based on a small amount of training samples. BACKGROUND

[0002] In actual classification applications, the number of labeled samples that can be obtained and utilized in a hyperspectral dataset is small, and the information of the labeled samples obtained artificially additionally needs to consume manpower and material resources, which also causes great difficulty to most current supervised classification methods, so how to improve the classification precision under a small amount of labeled samples and reduce the cost of the artificial labeling of ground information is a direction and hotspot that needs to be focused on. In recent years, the hyperspectral image classification method based on a deep learning model and an attention mechanism is widely researched because of its superior performance, however, the spatial attention mechanism and the channel attention mechanism applied at present are separated from each other, that is, the spatial attention only extracts the correlation information in the spatial dimension, and the channel attention mechanism only focuses on the correlation between different channel dimensions. SUMMARY

[0003] In order to solve the precision problem of the classification of a hyperspectral dataset image under a small amount of training samples, the application provides a hyperspectral image classification method based on a small amount of training samples, establishes a network structure based on a new double-attention model, realizes the fine classification of a hyperspectral image under a limited sample on the basis of capturing cross-latitude interaction information, and provides the following technical scheme.

[0004] A hyperspectral image classification method based on a small amount of training samples comprises the following steps:

[0005] Step 1: obtaining an original hyperspectral dataset, and labeling the class information of interest;

[0006] Step 2: performing dimension reduction on the original dataset by using a principal component analysis method, and reducing the amount of data to be processed by retaining non-overlapping information;

[0007] Step 3: selecting a suitable spatial neighborhood size, dividing the data into a training set, a validation set and a test set according to a certain proportion, and establishing a double-attention model;

[0008] Step 4: sending the training data into the double-attention mechanism model for training;

[0009] Step 5: timely adjusting the parameters of the model training process, and selecting the best model through cross-validation;

[0010] Step 6: testing the best model trained on the test set, and verifying the classification effect.

[0011] Preferably, step 1 is specifically:

[0012] The collected hyperspectral data is preprocessed, and the category information is labeled to provide key prior information for the category information extraction of the classification model.

[0013] Preferably, step 3 is specifically:

[0014] In order to extract the spatial-spectral joint information of the hyperspectral image, the data is divided into three-dimensional data blocks, and 9x9xS is selected, S being the spectral dimension after dimension reduction using principal component analysis. In the division of the training set, the lowest proportion is selected, with 3% of the training samples in the IP data set, 1% of the training samples in the UP data set, and 0.5% of the training samples in the SV data set.

[0015] Preferably, the dual attention model is divided into two branches, the upper branch being used to extract the spectral information of the hyperspectral image, referred to as the spectral branch, and the lower branch being used to extract the spatial information of the hyperspectral image, referred to as the spatial branch. The input hyperspectral data block is sent into both branches, then the extracted information is fused and sent to the classifier.

[0016] Preferably, for the spectral branch, the ECA module is applied to the spectral dimension of the hyperspectral image to further strengthen the weight of important spectral information. The backbone structure of feature extraction is a four-layer DenseNet network, and in the middle of each layer, the ECA module is also used to help the DenseNet network capture the correlation information between different channels. After feature extraction, the ChannelTriple Attention module is sent, and the final spectral feature map is gradually obtained.

[0017] Preferably, the three-dimensional data input into the spatial branch is directly extracted by a three-dimensional convolution and a DenseNet network to obtain spatial features. The extracted feature map is sent to the Spatital Triple Attention module, and the final spatial feature map is gradually obtained. The spatial feature map and the spectral feature map are connected and used together with the Softmax classifier for classification.

[0018] Preferably, step 4 is specifically:

[0019] The dual attention model captures cross-latitude interaction information and realizes fine classification of hyperspectral images under the limitation of a small number of training samples. In order to improve the feature reuse capability of the DenseNet network and the quality of the extracted features, a lightweight and effective channel attention module ECA is introduced into the DenseNet. The local cross-channel interaction strategy is realized through one-dimensional convolution without introducing too many parameters.

[0020] Preferably, step 5 is specifically:

[0021] The comparison experiment is carried out with the current most advanced DBDA model and deep learning methods CDCNN, SSRN and DBMA, the batch size is set to 16, 200 epochs are trained each time, and the corresponding early stopping strategy is set to prevent overfitting, that is, the loss value does not decrease for 20 batches, and the model training is terminated in time, the learning rate is 0.0001, and the cosine annealing method is used to dynamically reduce the learning rate during training.

[0022] Preferably, the step 6 is specifically: using the trained model to measure the precision index, stability, model classification time and the like on the test set, and drawing a complete classification graph.

[0023] The present application has the following beneficial effects:

[0024] The data show that the method has obvious advantages under the condition of less training samples, and has higher stability than other methods. The classification model based on the double attention mechanism in the application is realized based on image processing, deep learning and the like, and compared with other technologies, the classification precision under a small amount of training samples can be obviously improved, so that the demand for initial labeled samples of the model is not large, the time cost caused by manual labeling of class information can be greatly saved, and the present application has certain practical significance. In addition, the robustness of the model can be guaranteed, and a set of classification system with considerable effect can be obtained through a small amount of samples, and the time cost is within an acceptable range. BRIEF DESCRIPTION OF DRAWINGS

[0025] Fig. 1 It is a flow block diagram of hyperspectral image classification based on a double attention mechanism model;

[0026] Fig. 2 It is a structure diagram of the classification model. DETAILED DESCRIPTION

[0027] The present application is described in detail below in combination with specific embodiments. Embodiment one:

[0029] According to Figs. 1-2 To solve the above technical problems, the specific optimization technical scheme adopted is: the present application relates to a hyperspectral image classification method based on a small amount of training samples:

[0030] A hyperspectral image classification method based on a small amount of training samples, comprising the following steps:

[0031] Step 1: obtain the original hyperspectral data set, and label the class information of interest;

[0032] The step 1 is specifically:

[0033] The collected hyperspectral data is preprocessed, and the category information is labeled to provide key prior information for the category information extraction of the classification model.

[0034] Step 2: Dimensionality reduction is performed on the original data set using principal component analysis, and the amount of data to be processed is reduced by retaining non-overlapping information.

[0035] Step 3: Select an appropriate spatial neighborhood size, and divide the data into training set, validation set and test set according to a certain proportion, and establish a dual attention model.

[0036] The step 3 is specifically:

[0037] In order to extract the space-spectrum joint information of the hyperspectral image, the data is divided into three-dimensional data blocks, and 9*9*S is selected, S is the spectral dimension after dimensionality reduction using principal component analysis. In the division of the training set, the lowest proportion is selected, and the IP data set has only 3% of the training samples, the UP data set has only 1% of the training samples, and the SV data set has only 0.5% of the training samples.

[0038] The dual attention model is divided into two branches, the upper branch is used to extract the spectral information of the hyperspectral, which is called the spectral branch, and the lower branch is used to extract the spatial information of the hyperspectral, which is called the spatial branch. The input hyperspectral data block is sent into the two branches at the same time, then the extracted information is fused and sent to the classifier.

[0039] For the spectral branch, the ECA module is applied to the spectral dimension of the hyperspectral to further strengthen the weight of important spectral information. The backbone structure of feature extraction is a four-layer DenseNet network, and ECA module is also used in the middle of each layer to help DenseNet network capture the associated information between different channels. After feature extraction, the ChannelTripleAttention module is sent, and the final spectral feature map is gradually obtained. It is worth noting that the channel attention here only acts on the channel dimension in the three-dimensional convolution process, not the spectral dimension. Taking the first three-dimensional convolution of the spectral branch as an example, the feature map size obtained is (9*9*47*24), which represents the length, width, spectral dimension and channel dimension (determined by the number of three-dimensional convolution kernels in the previous layer) of the hyperspectral data block in turn. After feature extraction, the ChannelTripleAttention module is sent, and the final spectral feature map is gradually obtained.

[0040] The 3D data input to the spatial branch is directly processed by a 3D convolution and DenseNet network to extract spatial features. The extracted feature map is then fed into the Spatital TripleAttention module to gradually obtain the final spatial feature map. This map is then connected with the spectral feature map and classified together using a Softmax classifier.

[0041] Step 4: Feed the training data into the dual attention mechanism model for training;

[0042] Step 4 specifically involves:

[0043] The dual attention model achieves fine classification of hyperspectral images by capturing cross-dimensional interaction information and under the constraint of a small number of training samples. In order to improve the feature reuse capability of the DenseNet network and the quality of the extracted features, a lightweight and effective channel attention module ECA is introduced into DenseNet. It implements a local cross-channel interaction strategy through one-dimensional convolution without introducing too many parameters.

[0044] In addition, we attempted to apply the channel attention model to the spectral dimension of the hyperspectral domain, rather than the channel dimension during the convolution operation. Without a significant increase in computational cost, the model achieved impressive classification results with a very limited number of training samples.

[0045] Comparative experiments were conducted using the model of this invention and other advanced models such as SSRN, DBMA, and DBDA on three common hyperspectral datasets: Indian Pines (IP), Salinas Valley (SV), and Pavia University (UP). The accuracy of each method was measured using the three most widely used quantitative metrics: overall accuracy (OA), average accuracy (AA), and Kappa coefficient (K), with the average of 10 experiments taken. For the Indian Pines dataset, only 3% of the samples were used as the training set, 3% as the test set, and the remaining samples were used for testing. It should be noted that the number of samples in classes 1, 7, 9, and 16 of this dataset is very small, with less than one sample for 3% of the data; therefore, the training and validation samples for these classes were set to 2. For the Salinas Valley (SV) dataset, only 0.5% of the training and validation sets were selected; for the Pavia University (UP) dataset, 1% of the training and validation sets were selected. The experimental results are shown in Table 1.

[0046] Table 1 Comparison of Classification Accuracy Results

[0047]

[0048] Data shows that the method of the present invention has significant advantages when there are few training samples, and the model of the present invention has higher stability than other methods.

[0049] The training and testing times for different models are shown in Table 2. Compared with other methods, although this method slightly increases the training cost, its improvement in accuracy is very significant.

[0050] Table 2 Model training and testing time

[0051]

[0052] Step 5: Adjust the parameters during the model training process in a timely manner, and select the best model through cross-validation;

[0053] Step 5 specifically involves:

[0054] Comparative experiments were conducted with the state-of-the-art DBDA model and deep learning methods CDCNN, SSRN, and DBMA. The batch size was set to 16, and each training session consisted of 200 epochs. An early stopping strategy was implemented to prevent overfitting, meaning that the training was terminated when the loss value stopped decreasing after 20 batches. The learning rate was 0.0001, and cosine annealing was used to dynamically reduce the learning rate during training.

[0055] Step 6: Test the best-trained model on the test set to verify its classification performance.

[0056] Preferably, step 6 specifically involves: using the trained model to measure indicators such as accuracy, stability, and model classification time on the test set, and drawing a complete classification map.

[0057] The above description is merely a preferred embodiment of a hyperspectral image classification method based on a small number of training samples. The scope of protection for such a method is not limited to the above embodiments; all technical solutions falling within this conceptual framework are within the scope of protection of this invention. It should be noted that for those skilled in the art, any improvements and variations made without departing from the principles of this invention should also be considered within the scope of protection of this invention.

Claims

1. A hyperspectral image classification method based on a small number of training samples, characterized by: Includes the following steps: Step 1: Obtain the original hyperspectral dataset and label the categories of interest; Step 1 specifically involves: The collected hyperspectral data is preprocessed and the category information is labeled to provide key prior information for the classification model to extract category information. Step 2: Use principal component analysis to reduce the dimensionality of the original dataset. This reduces the amount of data to be processed by preserving non-overlapping information. Step 3: Select an appropriate spatial neighborhood size and divide the data into training, validation, and test sets according to a certain ratio to establish a dual attention model; Step 3 specifically involves: To extract the spatial-spectral joint information of hyperspectral images, the data was divided into three-dimensional data blocks, with a 9×9×S block size selected. S is the spectral dimension after dimensionality reduction using principal component analysis. The lowest proportion was selected for the training set: only 3% of the training samples were in the IP dataset, only 1% in the UP dataset, and only 0.5% in the SV dataset. The dual attention model is divided into two branches. The upper branch is used to extract spectral information from the hyperspectral spectrum and is called the spectral branch. The lower branch is used to extract the spatial information of the hyperspectral data and is called the spatial branch. The input hyperspectral data block is fed into both branches at the same time, and then the extracted information is fused and fed into the classifier. For the spectral branch, the ECA module is applied to the spectral dimension of the hyperspectral region to further strengthen the weight of important spectral information. The backbone structure of feature extraction is a four-layer DenseNet network. In the middle of each layer, the ECA module is also used to help the DenseNet network capture the correlation information between different channels. After feature extraction, it is fed into the Channel TripleAttention module and gradually obtains the final spectral feature map. The 3D data input to the spatial branch is directly processed by a 3D convolution and DenseNet network to extract spatial features. The extracted feature map is then fed into the Spatital Triple Attention module to gradually obtain the final spatial feature map. This map is then connected with the spectral feature map and classified together using a Softmax classifier. Step 4: Feed the training data into the dual attention mechanism model for training; Step 4 specifically involves: The dual attention model achieves fine classification of hyperspectral images by capturing cross-dimensional interaction information and under the constraint of a small number of training samples. In order to improve the feature reuse capability of DenseNet network and the quality of the extracted features, a lightweight and effective channel attention module ECA is introduced into DenseNet. It implements a local cross-channel interaction strategy through one-dimensional convolution without introducing too many parameters. Step 5: Adjust the parameters during the model training process in a timely manner, and select the best model through cross-validation; Step 5 specifically involves: Comparative experiments were conducted with the state-of-the-art DBDA model and deep learning methods CDCNN, SSRN, and DBMA. The batch size was set to 16, and each training session consisted of 200 epochs. An early stopping strategy was implemented to prevent overfitting, i.e., the model training was terminated when the loss value stopped decreasing after 20 batches. The learning rate was 0.0001, and cosine annealing was used to dynamically reduce the learning rate during the training process. Step 6: Test the best-trained model on the test set to verify its classification performance; Step 6 specifically involves: using the trained model to measure accuracy, stability, and model classification time on the test set, and drawing a complete classification map.

Images