Small sample hyperspectral image classification method and system, storage medium and equipment

Abstract

The embodiment of the invention discloses a small sample hyperspectral image classification method, which comprises the following steps: acquiring a source domain hyperspectral image and a target domain hyperspectral image, defining unmarked data in the target domain hyperspectral image as XU, and defining marked data of the target domain hyperspectral image and the source domain hyperspectral image as XL; enhancing the unmarked data XU of the hyperspectral image of the target domain according to a spectral feature offset strategy, and determining sample pair data; determining a first feature vector of the sample pair data, further determining an unsupervised contrast loss function Luc, determining a supervised contrast loss function Lsc and a classification loss function Lcls according to the marked data XL, and constructing a final loss function according to the unsupervised contrast loss function Luc, the supervised contrast loss function Lsc and the classification loss function Lcls; training an image classification model according to the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached; and inputting a test sample into the trained image classification model to obtain a classification result.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a small sample hyperspectral image classification method, system, storage medium and device. Background Art

[0002] In recent years, with the rapid progress of sensor technology and image processing technology, hyperspectral images have shown great application potential in many fields with their rich spectral information and fine spatial resolution, including but not limited to urban environmental monitoring, ecosystem analysis and deep space exploration. As a key technology for parsing these complex information, hyperspectral image classification is of self-evident importance; however, the classification task of hyperspectral images faces a major challenge: the scarcity of high-quality labeled data. Since the labeling process is time-consuming and expensive, the number of labeled samples that can be used to train classification models is limited, which greatly limits the performance of traditional classification algorithms.

[0003] In order to overcome this problem, small sample learning methods have begun to be used in existing technologies. These methods aim to utilize a small number of labeled samples and a large amount of unlabeled data or auxiliary information to effectively improve the generalization ability of classification models. Currently, small sample learning methods in the field of hyperspectral image classification are mainly divided into three schools: fine-tuning-based methods, data enhancement-based methods, and transfer learning-based methods. Although these methods have made certain progress in their respective fields, they still have many limitations.

[0004] Methods based on fine-tuning are often limited by the complexity of the network structure and are difficult to optimize effectively when samples are scarce; although methods based on data enhancement can expand the sample size, the quality and diversity of the newly generated samples are difficult to guarantee; and methods based on transfer learning, especially prototype networks and metric-based methods, although they can learn cross-domain discriminative features, are still insufficient when dealing with the complex spectral and spatial relationships unique to hyperspectral images. Summary of the invention

[0005] Based on this, it is necessary to propose a small sample hyperspectral image classification method to address the above problems.

[0006] A small sample hyperspectral image classification method, the method comprising the following steps:

[0007] Get the source domain hyperspectral image and the target domain hyperspectral image, and define the unlabeled data in the target domain hyperspectral image as X U , define the label data of target domain hyperspectral image and source domain hyperspectral image as X L ;

[0008] According to the spectral feature migration strategy, the unlabeled data X of the target domain hyperspectral image is UPerform enhancement and determine the sample pair data;

[0009] Determine a first feature vector of the sample pair data by a feature extraction network in an image classification model;

[0010] According to the unlabeled sample X U Determine the unsupervised contrast loss function L uc , according to the labeled data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , according to the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Construct the final loss function;

[0011] Training an image classification model according to the source domain hyperspectral image and the target domain hyperspectral image until a maximum number of iterations is reached;

[0012] The test samples are input into the trained image classification model to obtain the classification results.

[0013] In the above scheme, the unlabeled data X of the target domain hyperspectral image is shifted according to the spectral feature shift strategy. U Enhance and determine the sample pair data, including:

[0014] Randomly select a spectrum of dimensions and transform the target domain hyperspectral image unlabeled data X U Horizontal bidirectional shift and vertical bidirectional shift are performed respectively to obtain sample pair data corresponding to the two views.

[0015] In the above scheme, the feature extraction network includes a lightweight spatial-spectral convolution module and a multi-scale converter consisting of a group convolution layer and a cross-channel fusion module:

[0016] The size of the input matrix in the group convolution layer is (9, 9, 256), the size of each convolution kernel is (1, 1), the step size is (1, 1), and the output matrix size is (9, 9, 128);

[0017] The cross-channel fusion module divides the matrix of size (9, 9, 128) into two equal parts along the spectral dimension;

[0018] Use the same depth convolution on the two parts to get two output matrices and Among them, the size of the input matrix is ​​(9, 9, 64), the size of each convolution kernel is (3, 3), and the size of the output matrix is ​​(9, 9, 64);

[0019] For the two matrices and Interact to obtain the cross-channel fused feature X′:

[0020]

[0021] X′=Concat(X′ 1 ,X′ 2 )

[0022] Among them, α is the fusion weight coefficient.

[0023] In the above scheme, the marking data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , including:

[0024] Determine the label data X L The average value of the feature vector is used as the class prototype of this category;

[0025] For each query sample, its feature vector is obtained through the feature extraction network and the feature vector z is determined L The Euclidean distance between the prototype of the mth class;

[0026] Determine the predicted probability that the query sample belongs to the mth class based on the Euclidean distance

[0027] According to the feature vector z L The true probability y of belonging to the mth class im and feature z L Predictions belonging to class m

[0028] Probability Construct a classification loss function based on cross entropy:

[0029]

[0030] Among them, y im is the true probability, is the predicted probability.

[0031] In the above scheme, the marking data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , including:

[0032]

[0033] Among them, z L is the feature vector of the labeled sample obtained by the feature extraction network, is the feature vector of the same type as the labeled sample, is the feature vector of a different class from the labeled sample, τ is the temperature factor, and sim(.) is the cosine similarity.

[0034] In the above scheme, the unlabeled sample X U Determine the unsupervised contrast loss function L uc , specifically including:

[0035] Determine the mean of the feature vector of the unlabeled sample under the view p of the lth class

[0036]

[0037] Where p∈{h,v}, h is the horizontal bidirectional offset, v is the vertical bidirectional offset, T is the number of samples of the l-th view p, and B is the total number of unlabeled samples;

[0038] Constructing the unsupervised contrast loss function L uc :

[0039]

[0040] in, ∈ is the temperature factor, and B is the total number of unlabeled samples.

[0041] In the above scheme, the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Construct the final loss function, including:

[0042] L Total =(1-λ)L cls +λ(L sc +L uc )

[0043] Among them, λ is the weight parameter, L uc is the unsupervised contrast loss function, L sc is the supervised contrast loss function, L cls is the classification loss function.

[0044] The present application also discloses a small sample hyperspectral image classification system, characterized in that the system comprises: a data acquisition unit, a loss function construction unit, a classification model training unit and a classification model testing unit;

[0045] The data acquisition unit is used to acquire a source domain hyperspectral image and a target domain hyperspectral image, and define the unlabeled data in the target domain hyperspectral image as X U, define the label data of target domain hyperspectral image and source domain hyperspectral image as X L ; According to the spectral feature shift strategy, the target domain hyperspectral image unlabeled data X U Perform enhancement to determine sample pair data; determine a first feature vector of the sample pair data through a feature extraction network in an image classification model;

[0046] The loss function construction unit is used to construct a loss function according to the unlabeled sample X U Determine the unsupervised contrast loss function L uc , according to the labeled data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , according to the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Constructing a final loss function; the classification model training unit is used to train an image classification model according to the source domain hyperspectral image and the target domain hyperspectral image until a maximum number of iterations is reached;

[0047] The classification model testing unit is used to input the test sample into the trained image classification model to obtain the classification result.

[0048] The present application also discloses a readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the following steps:

[0049] Get the source domain hyperspectral image and the target domain hyperspectral image, and define the unlabeled data in the target domain hyperspectral image as X U , define the label data of target domain hyperspectral image and source domain hyperspectral image as X L ;

[0050] According to the spectral feature migration strategy, the unlabeled data X of the target domain hyperspectral image is U Perform enhancement and determine the sample pair data;

[0051] Determine a first feature vector of the sample pair data by a feature extraction network in an image classification model;

[0052] According to the unlabeled sample X U Determine the unsupervised contrast loss function L uc , according to the labeled data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , according to the unsupervised contrast loss function L uc , supervised contrast loss function Lsc And the classification loss function L cls Constructing a final loss function; the classification model training unit is used to train an image classification model according to the source domain hyperspectral image and the target domain hyperspectral image until a maximum number of iterations is reached;

[0053] The test samples are input into the trained image classification model to obtain the classification results.

[0054] The present application also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor in the following steps:

[0055] Get the source domain hyperspectral image and the target domain hyperspectral image, and define the unlabeled data in the target domain hyperspectral image as X U , define the label data of target domain hyperspectral image and source domain hyperspectral image as X L ;

[0056] According to the spectral feature migration strategy, the unlabeled data X of the target domain hyperspectral image is U Perform enhancement and determine the sample pair data;

[0057] Determine a first feature vector of the sample pair data by a feature extraction network in an image classification model;

[0058] According to the unlabeled sample X U Determine the unsupervised contrast loss function L uc , according to the labeled data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , according to the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Constructing a final loss function; the classification model training unit is used to train an image classification model according to the source domain hyperspectral image and the target domain hyperspectral image until a maximum number of iterations is reached;

[0059] The test samples are input into the trained image classification model to obtain the classification results.

[0060] The embodiment of the present invention has the following beneficial effects: firstly, a source domain hyperspectral image and a target domain hyperspectral image are obtained, and the unlabeled data in the target domain hyperspectral image is defined as X U , define the label data of target domain hyperspectral image and source domain hyperspectral image as X L ; According to the spectral feature shift strategy, the target domain hyperspectral image unlabeled data X UEnhance and determine the sample pair data; determine the first feature vector of the sample pair data through the feature extraction network in the image classification model; according to the unlabeled sample X U Determine the unsupervised contrast loss function L uc , according to the labeled data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , according to the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Constructing a final loss function; the classification model training unit is used to train an image classification model according to the source domain hyperspectral image and the target domain hyperspectral image until a maximum number of iterations is reached; inputting a test sample into the trained image classification model to obtain a classification result; the present invention improves the accuracy of small sample hyperspectral image classification, effectively enhances robustness, and can be used for mineral exploration, ecological monitoring, smart agriculture and medical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0062] in:

[0063] Figure 1 A schematic diagram of a small sample hyperspectral image classification method in an embodiment;

[0064] Figure 2a This is a pseudo-color composite image of the Chikusei dataset;

[0065] Figure 2b It is the true value map of the Chikusei dataset;

[0066] Figure 2c This is a pseudo-color composite image of the Indian Pines dataset;

[0067] Figure 2d is the true value map of the Indian Pines dataset;

[0068] Figure 3 A schematic diagram of the classification results of Indian Pines data using the small sample hyperspectral image classification method DCFSL;

[0069] Figure 4 This is a schematic diagram of the results of classifying Indian Pines data using the method of the present invention. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0071] In the following description, a large number of specific details are given in order to provide a more thorough understanding of the present invention; however, it is obvious to those skilled in the art that the present invention can be implemented without one or more of these details; in other examples, in order to avoid confusion with the present invention, some technical features known in the art are not described, and it should be understood that the present invention can be implemented in different forms and should not be interpreted as limited to the embodiments presented here; on the contrary, providing these embodiments will make the disclosure thorough and complete and fully convey the scope of the present invention to those skilled in the art.

[0072] The purpose of the terms used herein is only to describe specific embodiments and is not intended to be limiting of the present invention. When used herein, the singular forms "a", "an" and "said / the" are also intended to include the plural forms, unless the context clearly indicates otherwise, and it should also be understood that the terms "consisting of" and / or "comprising", when used in this specification, determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0073] In order to thoroughly understand the present invention, a detailed structure will be presented in the following description to illustrate the technical solution proposed by the present invention; optional embodiments of the present invention are described in detail as follows, but in addition to these detailed descriptions, the present invention may also have other implementation methods.

[0074] like Figure 1 As shown, in one embodiment, a small sample hyperspectral image classification method is provided, and the small sample hyperspectral image classification method includes steps S101 to S106, which are described in detail as follows:

[0075] S101, obtain source domain hyperspectral image and target domain hyperspectral image, define the unlabeled data in the target domain hyperspectral image as X U , define the label data of target domain hyperspectral image and source domain hyperspectral image as XL ;

[0076] Clearly distinguishing the data in the source domain and the target domain, and clearly defining labeled data (for supervised learning) and unlabeled data (for unsupervised learning or data enhancement), is an important step in transfer learning and small sample learning, which helps the model better adapt to the target domain.

[0077] S102: Spectral feature shift strategy is used to shift the unlabeled data X of the target domain hyperspectral image. U Perform enhancement and determine the sample pair data;

[0078] By enhancing unlabeled data through the spectral feature shift strategy, more samples related to the target domain can be generated, which helps to alleviate the small sample problem. This strategy can increase the generalization ability of the model, especially when the spectral features change in the target domain.

[0079] In some embodiments, the target domain hyperspectral image unlabeled data X is shifted according to the spectral feature shift strategy. U Enhance and determine the sample pair data, including:

[0080] Randomly select a dimension of spectrum and transform the target domain hyperspectral image unlabeled data X U Horizontal bidirectional shift and vertical bidirectional shift are performed respectively to obtain sample pair data corresponding to the two views.

[0081] Preferably, the Chikusei dataset is selected as the source domain, and its pseudo-color composite image is shown in Figure 2(a), and its true value image is shown in Figure 2(b). The Indian Pines dataset is selected as the target domain, and its pseudo-color composite image is shown in Figure 2(c), and its true value image is shown in Figure 2(d). The Indian Pines dataset contains 16 types of land objects and a total of 42,776 labeled samples.

[0082] The Chikusei dataset is defined as the source domain, and the IndianPines dataset is defined as the target domain. Since the data of the source domain and the target domain are trained alternately in meta-learning, the present invention constructs a meta-data set based on the source domain and the target domain. In the set of training source domain data, 180 samples are randomly selected as training samples. In the set of training target domain data, refer to Figure 1 , randomly select a-dimensional spectra, perform horizontal and vertical bidirectional shifts on the unlabeled samples in the target domain, and obtain two views of the unlabeled samples. The generated unlabeled samples containing two views and 180 samples randomly selected from the data after data augmentation are used as training samples, where a is 50 in this example.

[0083] S103, determining a first feature vector of the sample pair data through a feature extraction network in an image classification model;

[0084] A feature extraction network is used to extract the first feature vector from the sample pair, which provides the basis for subsequent classification and contrastive learning.

[0085] In some embodiments, the feature extraction network includes a lightweight spatial-spectral convolution module and a multi-scale transformer consisting of a group convolution layer and a cross-channel fusion module:

[0086] The size of the input matrix in the group convolution layer is (9, 9, 256), the size of each convolution kernel is (1, 1), the stride is (1, 1), and the output matrix size is (9, 9, 128);

[0087] The cross-channel fusion module divides the matrix of size (9, 9, 128) into two equal parts along the spectral dimension;

[0088] Use the same depth convolution for both parts to get two output matrices and Among them, the size of the input matrix is ​​(9, 9, 64), the size of each convolution kernel is (3, 3), and the size of the output matrix is ​​(9, 9, 64);

[0089] For two matrices and Interact to obtain the cross-channel fused feature X ′ :

[0090]

[0091] X ′ =Concat(X′ 1 ,X′ 2 )

[0092] Among them, α is the fusion weight coefficient.

[0093] The above creates a shortcut between the input and output of the cross-channel fusion module, which can effectively reduce overfitting.

[0094] The multi-scale converter is composed of a normalization layer, a multi-scale self-attention network, a normalization layer, a point convolution, a batch normalization layer, an activation function, and a multi-layer perceptron connected in sequence. A shortcut is created between the input of the normalization layer and the output of the point convolution. The multi-scale self-attention network contains two parts: local-global self-attention (LGSA) and cross-group self-attention (CGSA). Its specific structure is as follows:

[0095] (1) Local-Global Self-Attention (LGSA):

[0096] Local-global self-attention first groups the input data of size (9,9,128) along the spatial dimension to obtain multiple local sub-data, each of which has a size of (3,3,128); uses depthwise convolution to process the input data to obtain global data, with a convolution kernel of (7,7), and the size of the output data is (3,3,128);

[0097] The local sub-data and global data are concatenated into a matrix X and input into the multi-head attention network to obtain the local token L and the global token Λ:

[0098] Q=XW Q ,K=XW K ,V=XW V

[0099] Q={Q 1 ,Q 2 ,...,Q i ,...,Q n}

[0100] K={K 1 ,K 2 ,...,K i ,...,K n}

[0101] V={V 1 ,V 2 ,...,V i ,...,V n}

[0102]

[0103] O = Concat(Z 1 ,Z 2 ,...,Z i ,...,Z n )W O

[0104] Where d is the number of channels of the attention head, W Q , W K , W V , W O is the projection matrix;

[0105] (2) Cross-Group Self-Attention (CGSA)

[0106] The global token Λ output by the local-global self-attention is projected after the layer normalization (LN) layer and the GELU activation function to obtain the query matrix Q Λ and the key matrix K Λ :

[0107]

[0108] The query matrix Q Λ , key matrix K Λ And the local token L are input into the multi-head attention network and undergo the following calculations:

[0109] Q Λ = {Q Λ1 ,Q Λ2 ,...,Q Λi ,...,Q Λn}

[0110] K Λ ={K Λ1 ,K Λ2 ,...,K Λi ,...,K Λn}

[0111] L={L 1 ,L 2 ,...,L i ,...,L n}

[0112]

[0113] O Λ =Concat(Z Λ1 ,Z Λ2 ,...,Z Λi ,...,Z Λn )W ΛO

[0114] Where d is the number of channels of the attention head, W ΛO is the projection matrix.

[0115] O Λ After two-dimensional convolution, the output of the cross-group self-attention size is (9, 9, 128).

[0116] S104: According to the unlabeled sample X U Determine the unsupervised contrast loss function L uc , according to the labeled data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , according to the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Construct the final loss function;

[0117] Combining the classification loss function (for supervised learning), the supervised contrast loss function (for feature comparison of labeled data) and the unsupervised contrast loss function (for feature comparison of unlabeled data), this multi-loss function design helps to make full use of limited data resources and improve the performance of the model.

[0118] In some embodiments, based on the label data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , including:

[0119] Determine the labeled data X L The average value of the feature vector is used as the class prototype of this category;

[0120] For each query sample, its feature vector is obtained through the feature extraction network and the feature vector z is determined L The Euclidean distance between the prototype of the mth class;

[0121] Determine the predicted probability that the query sample belongs to the mth class based on the Euclidean distance

[0122] According to the feature vector z L The true probability y of belonging to the mth class im and feature z L Predicted probability of belonging to the mth class Construct a classification loss function based on cross entropy:

[0123]

[0124] Among them, y im is the true probability, is the predicted probability.

[0125] In some embodiments, based on the label data X L Determine the supervised contrast loss function L sc , including:

[0126]

[0127] Among them, z L is the feature vector of the labeled sample obtained by the feature extraction network, is the feature vector of the same type as the labeled sample, is the feature vector of a different class from the labeled sample, τ is the temperature factor, and sim(.) is the cosine similarity.

[0128] In some embodiments, based on the label data X L Determine the supervised contrast loss function L sc And the classification loss function Lcls , including:

[0129] Determine the mean of the feature vector of the unlabeled sample under the view p of the lth class

[0130]

[0131] Where p∈{h,v}, h is the horizontal bidirectional offset, v is the vertical bidirectional offset, T is the number of samples of the l-th view p, and B is the total number of unlabeled samples;

[0132] Constructing the unsupervised contrast loss function L uc :

[0133]

[0134] in, ∈ is the temperature factor, B is the total number of unlabeled samples, and the unsupervised contrast loss L uc Encourage two groups to align across views.

[0135] In some embodiments, according to the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Construct the final loss function, including:

[0136] L Total =(1-λ)L cls +λ(L sc +L uc )

[0137] Among them, λ is the weight parameter, which is 1e-5 in this example, L uc is the unsupervised contrast loss function, L sc is the supervised contrast loss function, L cls is the classification loss function.

[0138] The present invention designs a semi-supervised loss function based on the contrastive learning principle, and uses a group rather than a traditional method based on a single sample to calculate the unsupervised contrast loss, so as to obtain more discriminative features.

[0139] S105, training an image classification model according to the source domain hyperspectral image and the target domain hyperspectral image until a maximum number of iterations is reached;

[0140] Using data from the source domain and the target domain to jointly train the model helps the model better adapt to the target domain. The training process can be controlled by monitoring whether the number of training sets reaches the maximum number of iterations, which helps avoid overfitting or underfitting problems.

[0141] Specifically, the gradient descent method is used to update the parameters of the cross-scene hyperspectral image classification network: first set the initial value of the learning rate α, the number of meta-learning sets T, and the number of batches B; initialize the parameters of the network model; then construct the training data for each set according to the batch size, calculate the loss function, update the gradient of the loss function through back propagation, and update the network training parameters through forward propagation according to the direction of the loss function descent, and alternately train the source domain and target domain data until the maximum number of sets T is reached.

[0142] S106: Input the test sample into the trained image classification model to obtain a classification result.

[0143] Furthermore, the test data is input into the trained feature extraction network based on the convolutional converter, and the samples are classified using the KNN algorithm to obtain the classification results and simulation result diagrams. The effect of the present invention can be further illustrated by the following simulation results.

[0144] 1. Simulation conditions

[0145] Test data: Chikusei is used as the source domain data and IndianPines dataset is used as the target domain. Five labeled samples are randomly selected from each category in the target domain.

[0146] Test environment: Ubuntu 18.04LTS system, NVIDIA GeForce GTX 4090GPU, Pytorch deep learning framework.

[0147] Evaluation index: The index calculation formula is as follows:

[0148]

[0149]

[0150] Among them, TP is the number of samples correctly classified into this category, FN is the number of samples of this category that are incorrectly classified as other categories, TN is the number of non-class samples classified as other categories, and FP is the number of other samples that are incorrectly classified as samples of this category. x i+ represents the sum of all elements in the i-th row of the confusion matrix, that is, the sum of the actual number of this category, x +i represents the sum of all elements in the i-th column of the confusion matrix, that is, the sum of the number of predicted cost categories, N represents the sum of all elements, c is the number of categories, x ii Represents the elements on the diagonal of the confusion matrix, that is, the number of correct predictions for each category.

[0151] 2. Simulation content

[0152] Simulation 1: Under the above conditions, the existing small sample hyperspectral image classification method DCFSL is used to classify the Indian Pines data. The results are as follows: Figure 3 shown.

[0153] Simulation 2: Under the above conditions, the Indian Pines data is classified using the method of the present invention. The results are as follows: Figure 4 As shown, Figure 3 , 4 The color marking module is the same as that in Figure 2(d).

[0154] pass Figure 3 and Figure 4 By comparison, it can be found that the classification result map obtained by the method proposed in the present invention is closer to the distribution of objects in the true value map, the classification effect obtained in most categories is better, and better classification performance is achieved.

[0155] 3. Evaluation indicators

[0156] The above two methods were tested 10 times using the same 10 random numbers, and the confusion matrix, classification accuracy PA of each category, average classification accuracy AA, overall classification accuracy OA and KAPPA coefficient were calculated. The test results are shown in the following table:

[0157]

[0158]

[0159] It can be seen that compared with the current more advanced DCFSL method, the classification accuracy of the categories numbered 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, and 15 in the present invention is improved, and the average classification accuracy, overall classification accuracy, and KAPPA coefficient are also significantly improved.

[0160] In summary, the present invention proposes a small sample hyperspectral image classification method based on semi-supervised contrastive learning. The data from the source domain and the target domain are alternately trained to learn a collaborative discriminative embedding space. First, a feature extraction network combining convolution and transformer is designed, which takes into account the extraction of local information and remote information. Secondly, a spectral feature offset strategy is proposed to generate enhanced samples for calculating unsupervised contrast loss. Then, a semi-supervised loss function based on contrastive learning is used to obtain more discriminative features. Then, the network training parameters are initialized, and the classification network is updated by the gradient descent method until the maximum number of iterations is reached. Finally, the test sample is input into the trained network to obtain feature embedding, and the KNN algorithm is used for classification.

[0161] The present application also discloses a small sample hyperspectral image classification system, the system comprising: a data acquisition unit, a loss function construction unit, a classification model training unit and a classification model testing unit;

[0162] The data acquisition unit is used to acquire the source domain hyperspectral image and the target domain hyperspectral image, and define the unlabeled data in the target domain hyperspectral image as X U , define the label data of target domain hyperspectral image and source domain hyperspectral image as X L ; According to the spectral feature shift strategy, the unlabeled data X of the target domain hyperspectral image U Perform enhancement to determine sample pair data; determine the first feature vector of the sample pair data through a feature extraction network in an image classification model;

[0163] The loss function construction unit is used to construct a loss function according to the unlabeled sample X U Determine the unsupervised contrast loss function L uc , according to the labeled data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , according to the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Construct the final loss function;

[0164] A classification model training unit, used for training an image classification model according to the source domain hyperspectral image and the target domain hyperspectral image until a maximum number of iterations is reached;

[0165] The classification model testing unit is used to input the test samples into the trained image classification model to obtain the classification results.

[0166] The present application also discloses a readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the following steps:

[0167] Get the source domain hyperspectral image and the target domain hyperspectral image, and define the unlabeled data in the target domain hyperspectral image as X U , define the label data of target domain hyperspectral image and source domain hyperspectral image as X L ;

[0168] According to the spectral feature migration strategy, the unlabeled data X of the target domain hyperspectral image is U Perform enhancement and determine the sample pair data;

[0169] Determine a first feature vector of the sample pair data by a feature extraction network in an image classification model;

[0170] According to the unlabeled sample X U Determine the unsupervised contrast loss function L uc , according to the labeled data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , according to the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Construct the final loss function;

[0171] Training an image classification model according to the source domain hyperspectral image and the target domain hyperspectral image until a maximum number of iterations is reached;

[0172] The test samples are input into the trained image classification model to obtain the classification results.

[0173] The present application also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor in the following steps:

[0174] Get the source domain hyperspectral image and the target domain hyperspectral image, and define the unlabeled data in the target domain hyperspectral image as X U , define the label data of target domain hyperspectral image and source domain hyperspectral image as X L ;

[0175] According to the spectral feature migration strategy, the unlabeled data X of the target domain hyperspectral image is U Perform enhancement and determine the sample pair data;

[0176] Determine a first feature vector of the sample pair data by using a feature extraction network in an image classification model;

[0177] According to the unlabeled sample X U Determine the unsupervised contrast loss function L uc , according to the labeled data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , according to the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Construct the final loss function; train the image classification model according to the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached;

[0178] The test samples are input into the trained image classification model to obtain the classification results.

[0179] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0180] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0181] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present application. It should be pointed out that, for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. The above disclosure is only a preferred embodiment of the present invention, and of course it cannot be used to limit the scope of the rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope covered by the present invention.

Claims

1. A small sample hyperspectral image classification method, characterized in that: The method comprises: Get the source domain hyperspectral image and the target domain hyperspectral image, and define the unlabeled data in the target domain hyperspectral image as X U , define the label data of target domain hyperspectral image and source domain hyperspectral image as X L ; According to the spectral feature migration strategy, the unlabeled data X of the target domain hyperspectral image is U Perform enhancement and determine the sample pair data; Determine a first feature vector of the sample pair data by a feature extraction network in an image classification model; According to the unlabeled sample X U Determine the unsupervised contrast loss function L uc , according to the labeled data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , according to the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Construct the final loss function; Training an image classification model according to the source domain hyperspectral image and the target domain hyperspectral image until a maximum number of iterations is reached; The test samples are input into the trained image classification model to obtain the classification results.

2. The small sample hyperspectral image classification method according to claim 1, characterized in that: The target domain hyperspectral image unlabeled data X is shifted according to the spectral feature shift strategy U Enhance and determine the sample pair data, including: Randomly select a spectrum of dimensions and transform the target domain hyperspectral image unlabeled data X U Horizontal bidirectional shift and vertical bidirectional shift are performed respectively to obtain sample pair data corresponding to the two views.

3. The small sample hyperspectral image classification method according to claim 2 is characterized in that: The feature extraction network includes a lightweight spatial-spectral convolution module and a multi-scale transformer consisting of a group convolution layer and a cross-channel fusion module: The size of the input matrix in the group convolution layer is (9, 9, 256), the size of each convolution kernel is (1, 1), the step size is (1, 1), and the output matrix size is (9, 9, 128); The cross-channel fusion module divides the matrix of size (9, 9, 128) into two equal parts along the spectral dimension; Use the same depth convolution on the two parts to get the two output matrices and Among them, the size of the input matrix is ​​(9, 9, 64), the size of each convolution kernel is (3, 3), and the size of the output matrix is ​​(9, 9, 64); For the two matrices and Interact to obtain the cross-channel fused feature X′: X′=Concat(X′1,X′2) Among them, α is the fusion weight coefficient.

4. The small sample hyperspectral image classification method according to claim 3 is characterized in that: According to the label data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , including: Determine the label data X L The average value of the feature vector is taken as the class prototype of this category; For each query sample, its feature vector is obtained through the feature extraction network and the feature vector z is determined L The Euclidean distance between the prototype of the mth class; Determine the predicted probability that the query sample belongs to the mth class based on the Euclidean distance According to the feature vector z L The true probability y of belonging to the mth class im and feature z L Predicted probability of belonging to the mth class Construct a classification loss function based on cross entropy: Among them, y im is the true probability, is the predicted probability.

5. The small sample hyperspectral image classification method according to claim 4 is characterized in that: According to the label data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , including: Among them, z L is the feature vector of the labeled sample obtained by the feature extraction network, is the feature vector of the same type as the labeled sample, is the feature vector of a different class from the labeled sample, τ is the temperature factor, and sim(.) is the cosine similarity.

6. The small sample hyperspectral image classification method according to claim 5 is characterized in that: According to the unlabeled sample X U Determine the unsupervised contrast loss function L uc , including: Determine the mean of the feature vector of the unlabeled sample under the view p of the lth class Where p∈{h,v}, h is the horizontal bidirectional offset, v is the vertical bidirectional offset, T is the number of samples of the l-th view p, and B is the total number of unlabeled samples; Constructing the unsupervised contrast loss function L uc : in, ∈ is the temperature factor, and B is the total number of unlabeled samples.

7. The small sample hyperspectral image classification method according to claim 6, characterized in that: According to the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Construct the final loss function, including: L Total =(1-λ)L cls +λ(L sc +L uc ) Among them, λ is the weight parameter, L uc is the unsupervised contrast loss function, L sc is the supervised contrast loss function, L cls is the classification loss function.

8. A small sample hyperspectral image classification system, characterized in that: The system comprises: a data acquisition unit, a loss function construction unit, a classification model training unit and a classification model testing unit; The data acquisition unit is used to acquire a source domain hyperspectral image and a target domain hyperspectral image, and define the unlabeled data in the target domain hyperspectral image as X U , define the label data of target domain hyperspectral image and source domain hyperspectral image as X L ; According to the spectral feature shift strategy, the target domain hyperspectral image unlabeled data X U Perform enhancement to determine sample pair data; determine a first feature vector of the sample pair data through a feature extraction network in an image classification model; According to the unlabeled sample X U Determine the unsupervised contrast loss function L uc , according to the labeled data X L Determine the supervised contrast loss function L sc And the classification loss function L cls , according to the unsupervised contrast loss function L uc , supervised contrast loss function L sc And the classification loss function L cls Construct the final loss function; The classification model training unit is used to train the image classification model according to the source domain hyperspectral image and the target domain hyperspectral image until a maximum number of iterations is reached; The classification model testing unit is used to input the test sample into the trained image classification model to obtain the classification result.

9. A readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to execute the steps of the method according to any one of claims 1 to 7.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

Images