Cross-domain small sample learning method for hyperspectral image classification based on machine learning method

Abstract

This invention discloses a cross-domain few-shot learning hyperspectral image classification method based on machine learning, relating to the field of hyperspectral remote sensing image classification technology. The invention includes the following steps: S1: Obtaining the source domain labeled sample dataset S L Target domain labeled sample dataset T L S1: Test dataset Te; S2: Construct a deep neural network; S3: Label the sample dataset S using the source domain. L The deep neural network is trained to obtain an initial model, and then the target domain is used to label the sample dataset T. L The initial deep neural network model is retrained to obtain the final deep neural network model; S4: A test network is constructed, and the test network is used to classify the test dataset Te to obtain the final test results. This application better realizes cross-domain few-shot learning, and can obtain more sufficient semantic information with spectral detail features and local detail spectral features, effectively improving the image classification effect.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral remote sensing image classification technology, specifically to a cross-domain few-shot learning hyperspectral image classification method based on machine learning. Background Technology

[0002] Hyperspectral images have the advantage of resolving fine spectral differences and can be widely used in fields such as mineral exploration, coastline mapping, and environmental monitoring. Hyperspectral image classification assigns a category to each pixel in an image based on its brightness and proximity, aiming to roughly distinguish various land features in remote sensing images. Currently, the most commonly used hyperspectral image classification method is based on label information. Over the years, deep learning methods have achieved significant results in hyperspectral image classification tasks due to their powerful feature extraction capabilities. Examples include image classification methods based on 2D-Convolutional Neural Networks (2D-CNN), 3D-CNN, recurrent neural networks (RNN), and recurrent neural networks (GCN). These existing image classification methods have significantly improved feature extraction capabilities, but they often lack the ability to represent detailed sequence features in spectral representation. In recent years, deep learning methods based on Transformer networks have begun to be applied to hyperspectral classification tasks due to their powerful spectral feature extraction capabilities, demonstrating their strong spectral representation abilities.

[0003] While existing deep learning methods can achieve good classification performance, they typically require sufficient labeled data. However, obtaining labeled samples for hyperspectral images is time-consuming and labor-intensive, making it difficult to acquire enough labeled samples for training in practice. Therefore, this limits the applicability of these existing image classification methods in challenging scenarios.

[0004] Moreover, in practical remote sensing applications, the distribution of labeled data is often uneven. For example, hyperspectral images with less labeled data are called the target domain, while those with more labeled data are called the source domain. To better address hyperspectral image classification tasks in small-shot cross-domain tasks, researchers have proposed Few-shot learning (FSL), which uses the idea of ​​meta-learning. When there is only one or a few target classes with labeled data, general information learned from the source domain (i.e., meta-knowledge) can help predict the target class data. However, previous few-shot learning methods, in distinguishing subtle spectral features between similar categories, usually only focus on the domain shift problem, while neglecting the mining of semantic information. Therefore, how to fully extract the semantic information of hyperspectral images to reduce the domain shift problem and make good use of the spectral features of hyperspectral images to achieve high-precision classification of hyperspectral images has become a new research hotspot. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides a cross-domain few-shot learning hyperspectral image classification method based on machine learning.

[0006] The technical solution of this invention is as follows:

[0007] This application provides a cross-domain few-shot learning hyperspectral image classification method based on machine learning, comprising the following steps:

[0008] S1: Obtain the source domain labeled sample dataset S L Target domain labeled sample dataset T L and the test dataset Te;

[0009] S2: Constructing a deep neural network: The deep neural network consists of an input layer, a mapping layer, a Transformer network, and an output layer;

[0010] The mapping layer is used to label the source domain sample dataset S. L and the target domain labeled sample dataset T L A mapping operation is performed to obtain the feature space mapping features; the Transformer network is used to obtain the semantic information of the mapped sample data and the spectral features of the mapped sample data with local interest, and to obtain the semantic information of the spectral detail features, the local detail spectral features, and the relation score.

[0011] S3: Label the sample dataset S using the source domain L The deep neural network is trained to obtain an initial model, and then the target domain is used to label the sample dataset T. L The initial deep neural network model is trained again to obtain the final deep neural network model;

[0012] S4: Construct a test network, use the test network to classify the test dataset Te, and compare it with the labeled samples of the test dataset Te to obtain the final test results.

[0013] Preferably, in step S1, the source domain labeled sample dataset S is obtained. L Target domain labeled sample dataset T L The method for using the test dataset Te is as follows: the Chikusei hyperspectral dataset is selected as the source domain dataset, and the Indian Pines hyperspectral dataset is selected as the target domain dataset; the source domain labeled sample dataset S is obtained from the source domain dataset. L Obtain the target domain labeled sample dataset T from the target domain dataset. L And the test dataset Te.

[0014] Preferably, in step S2, the mapping layer consists of a two-dimensional convolutional neural network layer and a BN layer.

[0015] Preferably, in step S2, the Transformer network consists of a multi-head attention mechanism module, a semantic enhancement module, and a relation network; the multi-head self-attention module consists of at least two stacked attention mechanism modules; the semantic enhancement module consists of a two-dimensional convolutional neural network, an activation function layer, a dropout layer, and a normalization layer; the relation network includes two convolutional blocks and two fully connected layers, wherein the two convolutional blocks and two fully connected layers are set sequentially in the manner of the first convolutional block, the second convolutional block, the first fully connected layer, and the second fully connected layer.

[0016] Preferably, step S3 specifically includes the following steps:

[0017] S3-1: Label the source domain sample dataset S L and the target domain labeled sample dataset T L Mapping operations are performed through a mapping layer;

[0018] S3-2: Label the source domain sample dataset S L The mapped sample data is input into the multi-head attention mechanism module of the Transformer network. The two attention mechanism modules stacked in the multi-head attention mechanism module are used to learn and obtain the semantic information of the mapped sample data and the spectral features of the mapped sample data with local attention.

[0019] S3-3: The semantic enhancement module is used to enhance the semantic information of the mapped sample data obtained in step S3-2 and extract the local spectral features with local interest, thereby obtaining the semantic information of spectral detail features and the local detail spectral features.

[0020] S3-4: Each convolutional block in the relation network performs convolution processing on the semantic information of the spectral detail features and the local detail spectral features output in step S3-3 to obtain the convolutional category association features. Every two convolutional blocks in the relation network perform convolution processing on the convolutional category association features to obtain the final category association features. The first fully connected layer flattens the final category association features to obtain the preliminary correlation probability features. The second fully connected layer flattens the preliminary correlation probability features again to obtain the relation score.

[0021] S3-5: Construct a loss function, then use the loss function to calculate the total loss of the deep neural network, optimize the gradient, and backpropagate to update the model parameters of the Transformer network, thereby pre-training the deep neural network and obtaining the initial model of the deep neural network.

[0022] S3-6: Label the sample dataset T using the target domain L The initial deep neural network model is retrained, and the parameters of the initial deep neural network model are fine-tuned to obtain the final deep neural network model.

[0023] Preferably, in step S3-1, the source domain labeled sample dataset S is... L and the target domain labeled sample dataset T L When performing mapping operations using a mapping layer, the output parameters of the two-dimensional convolutional neural network layer of the mapping layer are consistent with the smaller spectral dimension of the source domain dataset and the target domain dataset.

[0024] Preferably, step S3-3 specifically includes the following steps: the two-dimensional convolutional neural network in the semantic enhancement module performs convolution processing on the semantic information obtained in step S3-2 and the spectral features with local interest to obtain enhanced semantic information and enhanced local spectral features; then, the GELU activation function layer performs nonlinear transformation on the enhanced semantic information and enhanced local spectral features respectively to obtain semantic information with nonlinear factors and local spectral features with nonlinear factors; the dropout layer performs redundancy removal operation on the semantic information with nonlinear factors and local spectral features with nonlinear factors to obtain compressed semantic information and compressed local spectral features; then, the normalization layer performs normalization operation on the compressed semantic information and compressed local spectral features to obtain semantic information of spectral detail features and local detail spectral features.

[0025] Preferably, in steps S3-5, the total loss of the deep neural network is actually the total loss of the Transformer network; the total loss function of the Transformer network in this application... The calculation formula is shown in equation (5):

[0026] (5)

[0027] In equation (5), and These represent the loss of the source domain dataset and the loss of the target domain dataset, respectively.

[0028] The loss of the source domain dataset Calculated using equation (6):

[0029] (6)

[0030] In Equation (6), W, ξ, and H represent the loss in the multi-head attention mechanism module, the semantic enhancement module, and the relational network, respectively.

[0031] The loss of the target domain dataset Calculate using equation (7):

[0032] (7)

[0033] In Equation (7), W, ξ, and H represent the loss representations in the multi-head attention mechanism module, the semantic enhancement module, and the relational network, respectively.

[0034] Preferably, in step S4, the test network consists of an input layer, a mapping layer, a multi-head attention mechanism module, a semantic enhancement module, a KNN classifier, and an output layer.

[0035] Preferably, in step S4, the multi-head attention mechanism module of the test network has the same structure, function, and parameter settings as the multi-head attention mechanism module in the Transformer network of the final deep neural network model; the semantic enhancement module of the test network has the same structure, function, and parameter settings as the semantic enhancement module in the Transformer network of the final deep neural network model.

[0036] Preferably, in step S4, the process of classifying the test dataset Te using the test network mainly includes the following steps:

[0037] S4-1: Perform a dimension space mapping operation on the test dataset Te in the target domain dataset using the mapping layer to obtain the feature space mapped features;

[0038] S4-2: Input the mapped sample data of the test dataset Te into the multi-head attention mechanism module of the test network. Use the two attention mechanism modules stacked in the multi-head attention mechanism module to learn and obtain the semantic information of the mapped sample data and the spectral features of the mapped sample data with local attention.

[0039] S4-3: The semantic enhancement module is used to enhance the semantic information of the mapped sample data obtained in step S4-2 and extract the local spectral features to obtain the semantic information and local spectral features of the spectral detail features.

[0040] S4-4: The KNN classifier classifies the semantic information of the spectral detail features and the local detail spectral features obtained in step S4-3, respectively, to obtain the classification results;

[0041] S4-5: The loss function compares the classification results with the ground map (i.e., the labeled samples of the test dataset Te) to obtain the overall accuracy (OA), average accuracy (AA), and Kappa coefficient, which are then output through the output layer of the test network.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] This application presents a cross-domain few-shot learning hyperspectral image classification method based on machine learning. It can fully extract semantic information and local detail spectral features in cross-domain few-shot hyperspectral image classification tasks. The extraction of local detail spectral features makes it easier for this application to make better use of the spectral features of hyperspectral images, and can achieve good land cover classification and differentiation effects even when the number of samples is very small.

[0044] In this application, the deep neural network is first trained using the source domain labeled sample dataset SL to obtain an initial model. Then, the initial model is trained again using the target domain labeled sample dataset TL to fine-tune the parameters of the initial model, thus obtaining the final model. This setup allows for better cross-domain few-shot learning, obtaining more comprehensive semantic information and local detail spectral features with spectral details. This effectively improves the Transformer network's ability to distinguish subtle spectral features, thereby enhancing the deep neural network's performance in classifying cross-domain, few-shot hyperspectral images. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating the overall process of a cross-domain few-shot learning hyperspectral image classification method based on machine learning, as described in this invention.

[0046] Figure 2 This is a schematic diagram of the deep neural network structure in this application;

[0047] Figure 3 for Figure 2 A schematic diagram of the semantic enhancement module in Chinese;

[0048] Figure 4 for Figure 2 A schematic diagram of the structure of the relationship network in China. Detailed Implementation

[0049] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0050] Explanation of terms in this invention:

[0051] KNN: Nearest Neighbor, is a simple, efficient, and commonly used classification method that mainly determines the category of a sample to be classified based on the category of one or more nearest neighbor samples.

[0052] Transformer is a model that uses attention mechanisms to improve training speed. It can be considered a deep learning model entirely based on self-attention, as it is suitable for parallel computation.

[0053] This embodiment provides a cross-domain few-shot learning hyperspectral image classification method based on machine learning, and its overall flowchart is as follows: Figure 1 As shown, the specific steps include:

[0054] S1: The Chikusei hyperspectral dataset is selected as the source domain dataset, and the Indian Pines hyperspectral dataset is selected as the target domain dataset; the source domain labeled sample dataset S is obtained from the source domain dataset. L Obtain the target domain labeled sample dataset T from the target domain dataset. L and the test dataset Te;

[0055] Both the source domain dataset and the target domain dataset selected in this application consist of labeled samples and unlabeled samples; wherein, the source domain labeled sample dataset S L The target domain labeled sample dataset T consists of 200 randomly selected samples from each land cover category in the source domain dataset; L The test dataset Te is composed of 5 randomly selected samples from each land cover category of the labeled samples. The test dataset Te is derived from the target domain dataset after removing the target domain labeled sample dataset T. L All samples (including labeled and unlabeled samples) after the labeled samples in the dataset.

[0056] S2: Constructing a deep neural network:

[0057] The deep neural network consists of an input layer, a mapping layer, a Transformer network, and an output layer;

[0058] The mapping layer consists of a two-dimensional convolutional neural network layer and a BN layer;

[0059] The Transformer network consists of a multi-head attention mechanism module, a semantic enhancement module, and a relation network.

[0060] The multi-head self-attention module consists of at least two stacked attention mechanism modules;

[0061] The semantic enhancement module consists of a two-dimensional convolutional neural network, an activation function layer, a dropout layer, and a normalization layer.

[0062] The relation network described in this application includes two convolutional blocks and two fully connected layers. The two convolutional blocks and two fully connected layers are configured sequentially as a first convolutional block, a second convolutional block, a first fully connected layer, and a second fully connected layer. Each convolutional block consists of a two-dimensional convolutional layer, a batch normalization layer, a ReLU activation function layer, and a max pooling layer, configured sequentially. The structure of the relation network described in this application and the functions of each module contained therein are the same as those in existing technologies (such as "Learning to Compare: Relation Network for Few-Shot Learning").

[0063] In this application, the first and second convolutional blocks operate on the same steps. Taking the first convolutional block as an example, when an image is input to it, the two-dimensional convolutional layer of the first convolutional block first performs a convolution operation to extract feature information. Then, the extracted feature information is batch normalized through a BN layer to enhance the training speed and stability of the model. After that, the batch normalized feature information is non-linearly transformed through a ReLU activation function layer to improve the expressive power of the features. Finally, the non-linearly transformed image information is downsampled through a max pooling layer to reduce the size of the feature map. The feature map size output by the max pooling layer of the second convolutional block is set to 64. The two fully connected layers are used to flatten the features output by the convolutional blocks connected to them. The feature dimension of the feature map output by the first fully connected layer is set to 8, and the feature dimension of the feature map output by the second fully connected layer is set to 1. In this application, the first fully connected layer uses the ReLU activation function, and the second fully connected layer uses the Sigmoid function. The use of ReLU and Sigmoid activation functions in the first and second fully connected layers, respectively, generates a reasonable range of relation scores. This is because the Sigmoid function restricts the feature output to between 0 and 1, thus yielding a relation score. The relation score can be interpreted as a probability or confidence level. By setting the probability or confidence level, the deep neural network can better classify and predict different categories of relations.

[0064] S3: Label the sample dataset S using the source domain L The deep neural network is trained to obtain an initial model, and then the target domain is used to label the sample dataset T. L The initial deep neural network model is retrained to fine-tune its parameters, resulting in the final deep neural network model. This process includes the following steps:

[0065] S3-1: Label the source domain sample dataset S L and the target domain labeled sample dataset T L Perform mapping operations; specifically:

[0066] Source domain labeled sample dataset S from the source domain dataset L And the target domain labeled sample dataset T in the target domain dataset L A dimensional space mapping operation is performed using a mapping layer to obtain feature space mapped features; the mapping layer used in this application is a mapping layer in the prior art; the specific steps of step S3 are as follows:

[0067] Source domain labeled sample dataset S of the source domain dataset.L And the target domain labeled sample dataset T L The sample data in the input layer is fed into the two-dimensional convolutional neural network layer of the mapping layer for convolution operation to obtain preliminary convolutional mapping features; then the obtained preliminary convolutional mapping features are fed into the BN layer of the mapping layer for normalization operation to obtain the mapped sample data.

[0068] In this application, the spatial spectral dimension of the mapped sample data is 9×9×100. The convolutional kernel of the two-dimensional convolutional neural network in the mapping layer of this application is 3×3, with a stride of 1; the specific parameter settings of the two-dimensional convolutional neural network layer in the mapping layer are as follows:

[0069] In this application, when the mapping layer labels the source domain sample dataset S... L and the target domain labeled sample dataset T L When performing mapping operations, the input dimension settings of the two-dimensional convolutional neural network layer in the mapping layer are different. Specifically: when the mapping layer is applied to the source domain labeled sample dataset S... L When performing the mapping operation, the input dimension of the two-dimensional convolutional neural network layer of the mapping layer is set to the spectral dimension of the source domain dataset. This is done when the mapping layer is processing the labeled sample dataset T in the target domain. L When performing the mapping operation, the input dimension of the two-dimensional convolutional neural network layer of the mapping layer is set to the spectral dimension of the target domain dataset. In addition, in order to maintain the uniformity of the spectral dimension of the output result of the two-dimensional convolutional neural network layer of the mapping layer, this application selects the one with the smaller spectral dimension between the source domain dataset and the target domain dataset as the output parameter of the two-dimensional convolutional neural network layer of the mapping layer. In this embodiment, the spectral dimension of the source domain dataset is 100 and the spectral dimension of the target domain dataset is 200. Therefore, in this embodiment, 100 is selected as the output parameter of the two-dimensional convolutional neural network layer of the mapping layer.

[0070] S3-2: Label the source domain sample dataset S L The mapped sample data is input into the multi-head attention mechanism module of the Transformer network. The two attention mechanism modules stacked in the multi-head attention mechanism module are used to learn and obtain the semantic information of the mapped sample data and the spectral features of the mapped sample data with local attention. In this way, the internal correlation between feature categories can be effectively captured, thereby effectively reducing the domain offset problem between the source domain dataset and the target domain dataset.

[0071] In this embodiment, the calculation process of the two stacked attention mechanism modules in the multi-head attention mechanism module is consistent with the calculation process of multiple attention mechanism modules in a multi-head attention mechanism module composed of multiple stacked attention mechanism modules in the prior art (e.g., the paper "attention is all you need"), and will not be repeated here. The multi-head attention mechanism module in this application consists of two attention mechanism modules, which can realize the function of dividing the Transformer network into two heads, forming two subspaces, allowing the Transformer network to focus on two aspects of information;

[0072] The calculation process of the attention mechanism module in this application is the same as that in the prior art, and the calculation formula of the attention mechanism module in this application is also the same as that in the prior art, as shown in equation (1):

[0073] (1)

[0074] In equation (1), Q represents the query matrix, K represents the key matrix, and V represents the value matrix; d is the query dimension or the key dimension, and K... T This represents the transpose of the key matrix K;

[0075] Q in equation (1) is calculated using equation (2):

[0076] (2)

[0077] In equation (2), to These represent the feature vectors of all query matrices;

[0078] K in equation (1) is calculated using equation (3):

[0079] (3)

[0080] In equation (3), to These represent the eigenvectors of all the key matrices;

[0081] V in equation (1) is calculated using equation (4):

[0082] (4)

[0083] In equation (4), to These represent the eigenvectors of all value matrices;

[0084] S3-3: The semantic enhancement module is used to enhance the semantic information of the mapped sample data obtained in step S3-2 and extract local spectral features with local interest, thereby obtaining semantic information of spectral detail features and local detail spectral features. Specifically, it includes the following steps:

[0085] Specifically, the two-dimensional convolutional neural network in the semantic enhancement module of this application convolves the semantic information obtained in step S3-2 and the spectral features with local interest to obtain enhanced semantic information and enhanced local spectral features. Then, the GELU activation function layer performs nonlinear transformation on the enhanced semantic information and enhanced local spectral features to obtain semantic information with nonlinear factors and local spectral features with nonlinear factors. The dropout layer removes redundancy from the semantic information with nonlinear factors and the local spectral features with nonlinear factors to obtain compressed semantic information and compressed local spectral features. Then, the normalization layer normalizes the compressed semantic information and compressed local spectral features to obtain semantic information with spectral detail features and local detail spectral features. This effectively improves the Transformer network's ability to distinguish subtle spectral features, thereby improving the image classification performance of the Transformer network and even the entire deep neural network.

[0086] In this embodiment, the input and output parameters of the two-dimensional convolutional neural network of the semantic enhancement module are consistent with the output parameters of the multi-head attention mechanism module; in this embodiment, the convolutional kernel of the two-dimensional convolutional neural network of the semantic enhancement module is 3×3, the stride is 1; the dropout layer parameter is set to 0.1;

[0087] S3-4: Each convolutional block in the relation network performs convolution processing on the semantic information and local detail spectral features of the spectral detail features output in step S3-3 to obtain the convolutional category association features. Every two convolutional blocks in the relation network perform convolution processing on the convolutional category association features to obtain the final category association features. The first fully connected layer flattens the final category association features to obtain the preliminary relevance probability features with a feature dimension of 8. The second fully connected layer flattens the preliminary relevance probability features with a feature dimension of 8 again to obtain the relation score with a dimension of 1.

[0088] S3-5: Construct a loss function, then use the loss function to calculate the total loss of the deep neural network, optimize the gradient, and backpropagate to update the model parameters of the Transformer network, thus pre-training the deep neural network and obtaining the initial model of the deep neural network. All parameters of the Transformer network in the initial model of the deep neural network are defined as basic parameters. In this application, the total loss of the deep neural network is actually the total loss of the Transformer network. The total loss function of the Transformer network in this application... The calculation formula is shown in equation (5):

[0089] (5)

[0090] In equation (5), and These represent the loss of the source domain dataset and the loss of the target domain dataset, respectively.

[0091] The loss of the source domain dataset Calculated using equation (6):

[0092] (6)

[0093] In Equation (6), W, ξ, and H represent the loss in the multi-head attention mechanism module, the semantic enhancement module, and the relational network, respectively.

[0094] The loss of the target domain dataset Calculate using equation (7):

[0095] (7)

[0096] In Equation (7), W, ξ and H represent the loss representation in the multi-head attention mechanism module, the semantic enhancement module and the relational network, respectively.

[0097] In the pre-training process of the deep neural network, the learning rate and weight decay coefficient of the Transformer network are set to 0.001. In addition, the number of training epochs is set to 130 during the pre-training process of the deep neural network. The results show that the learning rate decreases to 0.0001 and 0.00001 after 70 and 110 training epochs, respectively.

[0098] S3-6: Label the sample dataset T using the target domain LThe initial deep neural network model is retrained, and its parameters are fine-tuned to obtain the final deep neural network model. All parameters of the Transformer network in the final deep neural network model are defined as the fine-tuned parameters. In steps S4-5, the learning rate and weight decay coefficient of the Transformer network are set to 0.001. Furthermore, in steps S3-6, during the retraining of the initial deep neural network model, the number of training epochs is set to 50. The results show that the learning rate decreases to 0.0001 and 0.00001 after 30 and 40 epochs, respectively. In this application, the target domain labeled sample dataset T is used. L By retraining the initial deep neural network model and fine-tuning its parameters, the final deep neural network model is obtained. This allows for better cross-domain few-shot learning, obtaining more comprehensive semantic information and local spectral details with spectral details. This effectively improves the Transformer network's ability to distinguish subtle spectral features, thereby enhancing the deep neural network's performance in classifying cross-domain, few-shot hyperspectral images.

[0099] S4: Construct a test network, classify the test dataset Te, and compare it with the labeled samples of the test dataset Te to obtain the final test results.

[0100] The test network in this application consists of an input layer, a mapping layer, a multi-head attention mechanism module, a semantic enhancement module, a KNN classifier, and an output layer. The multi-head attention mechanism module of the test network has the same structure, function, and parameter settings as the multi-head attention mechanism module in the Transformer network of the final deep neural network model. The semantic enhancement module of the test network also has the same structure, function, and parameter settings as the semantic enhancement module in the Transformer network of the final deep neural network model.

[0101] The process of classifying the test dataset Te using a test network in this invention mainly includes the following steps:

[0102] S4-1: Perform dimensional space mapping operation on the test dataset Te in the target domain dataset using the mapping layer to obtain the feature space mapping features; the specific steps of the mapping operation of the mapping layer on the test dataset Te are the same as those in step S3-1; since the spectral dimension of the target domain dataset is 200, the input and output parameters of the two-dimensional convolutional neural network layer of the mapping layer in this step are both 200.

[0103] S4-2: Input the mapped sample data of the test dataset Te into the multi-head attention mechanism module of the test network. Use the two stacked attention mechanism modules in the multi-head attention mechanism module to learn and obtain the semantic information of the mapped sample data and the spectral features of the mapped sample data with local attention, which makes it easier for the KNN classifier to perform better classification.

[0104] S4-3: The semantic enhancement module is used to enhance the semantic information of the mapped sample data obtained in step S4-2 and extract the local spectral features with local interest, thereby obtaining the semantic information of spectral detail features and local detail spectral features. The specific steps of step S4-3 are the same as those of step S3-3.

[0105] S4-4: The KNN classifier classifies the semantic information of the spectral detail features and the local detail spectral features obtained in step S4-3, respectively, to obtain the classification results;

[0106] S4-5: The loss function compares the classification results with the ground map (i.e., the labeled samples of the test dataset Te) to obtain the final test results of overall accuracy (OA), average accuracy (AA), and Kappa coefficient, which are then output through the output layer of the test network.

[0107] To verify the effectiveness of this invention, this application also compares the cross-domain few-shot learning hyperspectral image classification method based on machine learning described in this application with two deep learning-based image classification methods: 3 Dimension Convolutional Neural Network (3DCNN, from "Exploring 3-d–2-dcnn feature hierarchy for hyperspectral image classification"), spectral-spatial residual network (SSRN, from "Spectral-spatial residual network for hyperspectral image classification: A 3-D deep learning framework"), and deep few-shot learning+NN (DFSL+NN, from "Deep few-shot learning for hyperspectral image classification"), deep few-shot learning+SVM (DFSL+SVM, from "Deep few-shot learning for hyperspectral image classification"), relation network few-shot classification (RN-FSC, from "Deep relation network for hyperspectral image few-shot classification"), and deep cross-domain few-shot learning. This application compares the classification performance of four advanced FSL methods, including Deep Cross-Domain Few-Shot Learning (DCFSL, from "Deep Cross-Domain Few-Shot Learning for Hyperspectral Image Classification"). Specifically, this application uses the aforementioned six existing spectral image classification methods to classify the test dataset Te in the target domain dataset described in this application to obtain classification accuracy. At the same time, the application also compares the classification accuracy of the test dataset Te using the deep neural network final model obtained by the cross-domain few-shot learning hyperspectral image classification method (SSTFSL) based on machine learning described in this application.

[0108] To ensure a fair comparison, this application first uses the source domain labeled sample dataset S to compare the network models used in the six existing spectral image classification methods and the image classification method described in this application. L Train the dataset T, then label the sample dataset T using the target domain. L The training was performed again, and then tested using the test dataset Te to obtain the overall accuracy (OA), average accuracy (AA), and Kappa coefficient test results. The test results of the six existing spectral image classification methods and the image classification method described in this application are shown in Table 1:

[0109]

[0110] Table 1

[0111] As can be seen from Table 1, the last three rows represent the overall accuracy (OA), average accuracy (AA), and Kappa coefficient (KAPPA). Generally speaking, the higher the overall accuracy (OA), average accuracy (AA), and Kappa coefficient (KAPPA), the higher the classification accuracy of the model.

[0112] Overall accuracy (OA) is the ratio of the number of correctly classified samples across all categories output by the model to the total number of samples in the test set. Compared to existing image classification methods (i.e., 3D-CNN, SSRN, DFSL+NN, DFSL+SVM, RN-FSC, and DCFSL), the image classification method described in this application achieves the highest overall accuracy of 76.17%. The OA value obtained by the image classification method described in this application is 14.00% higher than the highest overall accuracy achieved by the existing image classification methods (i.e., 3D-CNN, SSRN, DFSL+NN, DFSL+SVM, RN-FSC, and DCFSL). This indicates that the method described in this application excels in the ability to correctly classify samples.

[0113] Average accuracy (AA) is the average of the accuracy for each category. Compared to the existing image classification methods (i.e., 3D-CNN, SSRN, DFSL+NN, DFSL+SVM, RN-FSC, and DCFSL), the image classification method described in this application achieves the highest average accuracy, reaching 80.93%. Compared to the existing image classification methods (i.e., 3D-CNN, SSRN, DFSL+NN, DFSL+SVM, RN-FSC, and DCFSL), the image classification method described in this application improves the average accuracy (AA) by 3.90% ((0.8093-0.7789) / 0.7789)×100%), compared to the highest AA value of 77.89% for the existing image classification methods. This indicates that regardless of the number of land cover types in the dataset, the image classification method described in this application can achieve good classification results.

[0114] Compared to existing image classification methods (i.e., 3D-CNN, SSRN, DFSL+NN, DFSL+SVM, RN-FSC, and DCFSL), the image classification method described in this application also performs best in terms of KAPPA accuracy. Compared to the existing image classification methods (i.e., 3D-CNN, SSRN, DFSL+NN, DFSL+SVM, RN-FSC, and DCFSL), the image classification method described in this application achieves a KAPPA accuracy of 0.7322. This represents a 16.89% improvement in KAPPA value compared to the highest KAPPA value achievable by the aforementioned existing image classification methods ((0.7322-0.6264) / 0.6264)×100%). This indicates that the image classification method described in this application obtains highly consistent classification categories when performing image classification.

[0115] In summary, based on the data interpretation in Table 1, the image classification method described in this application performs excellently in terms of overall accuracy (OA), average accuracy (AA), and Kappa coefficient (KAPPA). In other words, the image classification method described in this application has high classification accuracy and consistency.

[0116] Since the six spectral image classification methods in this application—3 Dimension Convolutional Neural Network (3DCNN), spectral-spatial-residual network (SSRN), few-shot learning+NN (DFSL+NN), deep few-shot learning+SVM (DFSL+SVM), relation network few-shot classification (RN-FSC), and deep cross-domain few-shot learning (DCFSL)—are all existing technologies, this application utilizes these existing technologies to classify the source domain labeled sample dataset S described in this application. L Training and testing on the test dataset Te described in this application are both existing technologies, and therefore will not be described in detail.

Claims

1. A hyperspectral image classification method based on cross-domain small sample learning of a machine learning method, characterized by: The method comprises the following steps: S1: Obtain a source domain labeled sample dataset S L , a target domain labeled sample dataset T L , and a test dataset T e ; S2: constructing a deep neural network: the deep neural network is composed of an input layer, a mapping layer, a Transformer network and an output layer; wherein the mapping layer is used for performing mapping operation on the source domain labeled sample dataset S L and the target domain labeled sample dataset T L to obtain a feature space mapping feature; The Transformer network is used to obtain semantic information of the mapped sample data and spectral features with local attention of the mapped sample data, and obtain semantic information of spectral detail features and local detail spectral features and relationship scores; S3: using the source domain labeled sample dataset S L training the deep neural network to obtain a deep neural network initial model, and then using the target domain labeled sample dataset T L re-training the deep neural network initial model to obtain a deep neural network final model; Step S3 specifically comprises the following steps: S3-1: a source domain labeled sample dataset S L and a target domain labeled sample dataset T L mapping operation is performed by the mapping layer; S3-2: input the mapped sample data of the source domain labeled sample data set S L The multi-head attention mechanism module of the Transformer network inputs the mapped sample data to learn the semantic information of the mapped sample data and the spectral features with local attention of the mapped sample data by using two attention mechanism modules stacked in the multi-head attention mechanism module. S3-3: The semantic enhancement module is used to enhance the semantic information and the local spectral features with the semantic information and the spectral features with local attention of the mapped sample data obtained in step S3-2, so as to obtain the semantic information of the spectral detail features and the local detail spectral features; S3-4: Each convolution block in the relationship network is used to perform convolution processing on the semantic information of the spectral detail features and the local detail spectral features output in step S3-3, to obtain convolutional category correlation features, each two convolution blocks in the relationship network are used to perform convolution processing on the convolutional category correlation features, to obtain final category correlation features, a first full connection layer is used to flatten the final category correlation features, to obtain preliminary correlation probability features, and a second full connection layer is used to flatten the preliminary correlation probability features again, to obtain relationship scores; S3-5: A loss function is constructed, then the total loss of the deep neural network is calculated by using the loss function, the gradient is optimized, and the model parameters of the Transformer network are updated by back propagation, so that the pre-training of the deep neural network is realized, and an initial model of the deep neural network is obtained; S3-6: Label the sample dataset T with target domain L re-training the initial deep neural network model, fine-tuning the initial deep neural network model parameters, and obtaining the final deep neural network model. S4: Construct a test network, and use the test network to test the test data set T e Classify and compare with the labeled samples of the test data set T e , and obtain the final test result.

2. The machine learning method based cross-domain few-shot learning hyperspectral image classification method according to claim 1, characterized in that: The Transformer network is composed of a multi-head attention mechanism module, a semantic enhancement module and a relationship network; the multi-head attention mechanism module is composed of at least two stacked attention mechanism modules; the semantic enhancement module is composed of a two-dimensional convolutional neural network, an activation function layer, a dropout layer and a normalization layer; and the relationship network comprises two convolution blocks and two full connection layers, and the two convolution blocks and the two full connection layers are sequentially arranged in the form of a first convolution block, a second convolution block, a first full connection layer and a second full connection layer.

3. The method of claim 2, wherein the method is a machine learning method based cross-domain few-shot learning hyperspectral image classification method. In step S3-1, a source domain labeled sample dataset S L and a target domain labeled sample dataset T L When the mapping operation is performed by using the mapping layer, the output parameters of the two-dimensional convolutional neural network layer of the mapping layer are consistent with the smaller spectral dimension of the source domain dataset and the target domain dataset.

4. The method of claim 2, wherein the method is a machine learning method based cross-domain few-shot learning hyperspectral image classification method. Step S3-3 specifically comprises the following steps: The two-dimensional convolutional neural network in the semantic enhancement module performs convolution processing on the semantic information and the spectral features with local attention obtained in step S3-2, to obtain enhanced semantic information and enhanced local spectral features; then, the GELU activation function layer performs nonlinear changes on the enhanced semantic information and the enhanced local spectral features respectively, to obtain semantic information with a nonlinear factor and local spectral features with a nonlinear factor; the dropout layer performs a redundancy removal operation on the semantic information with a nonlinear factor and the local spectral features with a nonlinear factor, to obtain compressed semantic information and compressed local spectral features; and then, the normalization layer performs a normalization operation on the compressed semantic information and the compressed local spectral features, to obtain the semantic information of the spectral detail features and the local detail spectral features.

5. The machine learning method based cross-domain few-shot learning hyperspectral image classification method according to claim 2, characterized in that: In step S3-5, the total loss of the deep neural network is actually the total loss of the Transformer network; wherein the total loss function of the Transformer network is L total The calculation formula is shown as formula (5): L total =L source +L target （5） In formula (5), L source and L target respectively represent the loss of the source domain dataset and the loss of the target domain dataset; And the loss of the source domain dataset L source Calculated by formula (6): L source =L source W+L source ξ+L source H （6） In formula (6), W, ξ and H respectively represent losses in the multi-head attention mechanism module, the semantic enhancement module and the relationship network; And the loss of the target domain dataset L target Calculated by equation (7): L target =L target W+L target ξ+L In formula (6), W, ξ and H respectively represent losses in the multi-head attention mechanism module, the semantic enhancement module and the relationship network; target H （7） In formula (7), W, ξ and H are respectively a multi-head attention mechanism module, a semantic enhancement module and a loss representation in a relationship network.

6. The machine learning method based cross-domain few-shot learning hyperspectral image classification method according to claim 1, characterized in that: In step S4, the test network is composed of an input layer, a mapping layer, a multi-head attention mechanism module, a semantic enhancement module, a KNN classifier and an output layer.

7. The machine learning method based cross-domain few-shot learning hyperspectral image classification method according to claim 6, characterized in that: In step S4, the multi-head attention mechanism module of the test network is the same as the multi-head attention mechanism module in the Transformer network of the deep neural network final model in terms of structure, function and parameter setting; the semantic enhancement module of the test network is the same as the semantic enhancement module in the Transformer network of the deep neural network final model in terms of structure, function and parameter setting.

8. The machine learning method based cross-domain few-shot learning hyperspectral image classification method according to claim 6, characterized in that: In step S4, the process of classifying the test data set Te by using the test network mainly includes the following steps: S4-1: performing a dimension space mapping operation on the test data set Te in the target domain data set by using the mapping layer to obtain mapped feature space features; S4-2: inputting the mapped sample data of the test data set Te into the multi-head attention mechanism module of the test network to learn by using the two attention mechanism modules stacked in the multi-head attention mechanism module, so as to obtain semantic information of the mapped sample data and spectral features with local attention of the mapped sample data; S4-3: the semantic enhancement module is used to enhance the semantic information and extract local spectral features of the mapped sample data obtained in step S4-2, so as to obtain semantic information of spectral detail features and local detail spectral features; S4-4: the KNN classifier classifies the semantic information of the spectral detail features and the local detail spectral features obtained in step S4-3 respectively, so as to obtain classification results; S4-5: a loss function compares the classification results with a ground object map to obtain overall accuracy, average accuracy and a Kappa coefficient, and the output layer of the test network is used for outputting.

Images