A small sample hyperspectral image classification method, system, storage medium and device

Abstract

The embodiment of the application discloses a small sample hyperspectral image classification method, comprising the following steps: acquiring source domain hyperspectral images and target domain hyperspectral images, defining unmarked data in the target domain hyperspectral images as X U , defining marked data of the target domain hyperspectral images and the source domain hyperspectral images as X L ; enhancing the unmarked data X U of the target domain hyperspectral images according to a spectral feature offset strategy, determining sample pair data; determining a first feature vector of the sample pair data, and further determining an unsupervised contrast loss function L uc ; determining a supervised contrast loss function L L and a classification loss function L sc according to the marked data X cls ; constructing a final loss function according to the unsupervised contrast loss function L uc , the supervised contrast loss function L sc and the classification loss function L cls ; training an image classification model according to the source domain hyperspectral images and the target domain hyperspectral images until a maximum iteration number is reached; and inputting a test sample into the trained image classification model to obtain a classification result.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, system, storage medium, and device for classifying small-sample hyperspectral images. Background Technology

[0002] In recent years, with the rapid advancement of sensor technology and image processing technology, hyperspectral images, with their rich spectral information and fine spatial resolution, have shown great application potential in many fields, including but not limited to urban environmental monitoring, ecosystem analysis, and deep space exploration. Hyperspectral image classification, as a key technology for interpreting this complex information, is of paramount importance. However, the task of hyperspectral image classification faces a major challenge: the scarcity of high-quality labeled data. Because the labeling process is both time-consuming and expensive, the number of labeled samples that can be used to train classification models is limited, which greatly restricts the performance of traditional classification algorithms.

[0003] To overcome this challenge, few-shot learning methods have begun to be used in existing technologies. These methods aim to effectively improve the generalization ability of classification models by utilizing a small number of labeled samples and a large amount of unlabeled data or auxiliary information. Currently, few-shot learning methods in the field of hyperspectral image classification are mainly divided into three major schools: fine-tuning-based methods, data augmentation-based methods, and transfer learning-based methods. Although these methods have made some progress in their respective fields, they still have many limitations.

[0004] Fine-tuning-based methods are often limited by the complexity of the network structure and are difficult to optimize effectively when samples are scarce. Data augmentation-based methods can expand the sample size, but the quality and diversity of the newly generated samples are difficult to guarantee. Transfer learning-based methods, especially prototype networks and metric-based methods, can learn discriminative features across domains, but they are still insufficient when dealing with the complex spectral and spatial relationships unique to hyperspectral images. Summary of the Invention

[0005] Based on this, it is necessary to propose a small-sample hyperspectral image classification method to address the above problems.

[0006] A method for classifying small-sample hyperspectral images, the method comprising the following steps:

[0007] Acquire source domain hyperspectral images and target domain hyperspectral images, and define unlabeled data in the target domain hyperspectral image as X. U The labeled data for the target domain hyperspectral image and the source domain hyperspectral image are defined as X. L ;

[0008] Based on the spectral feature shifting strategy, the unlabeled data X of the target domain hyperspectral image is processed. UEnhancement is performed to identify sample pairs of data;

[0009] The first feature vector of the sample pair data is determined by the feature extraction network in the image classification model;

[0010] According to the unlabeled sample X U Determine the unsupervised contrastive loss function L uc According to the marked data X L Determine the supervised contrast loss function L sc and classification loss function L cls According to the unsupervised contrastive loss function L uc Supervised contrastive loss function L sc and classification loss function L cls Construct the final loss function;

[0011] The image classification model is trained based on the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached.

[0012] The test sample is input into the trained image classification model to obtain the classification result.

[0013] In the above scheme, the unlabeled data X of the target domain hyperspectral image is processed according to the spectral feature shift strategy. U Enhancement is performed to identify sample pairs of data, specifically including:

[0014] Randomly select 'a' dimensions of the spectrum, and convert the unlabeled data X of the target domain hyperspectral image into... U By performing horizontal and vertical bidirectional shifts respectively, sample pairs of data corresponding to the two views are obtained.

[0015] In the above scheme, the feature extraction network includes a lightweight spatial spectral convolution module consisting of a group convolutional layer and a cross-channel fusion module, and a multi-scale converter.

[0016] The input matrix in the group convolutional layer has a size of (9, 9, 256), each convolutional kernel has a size of (1, 1), a stride of (1, 1), and the output matrix has a size of (9, 9, 128).

[0017] The cross-channel fusion module divides a matrix of size (9, 9, 128) into two equal parts along the spectral dimension;

[0018] Applying the same depthwise convolution to the two parts yields two output matrices. and The input matrix has a size of (9, 9, 64), each convolutional kernel has a size of (3, 3), and the output matrix has a size of (9, 9, 64).

[0019] For the two matrices and Interact to obtain the cross-channel fused feature X′:

[0020]

[0021] X′=Concat(X′1,X′2)

[0022] Where α is the fusion weight coefficient.

[0023] In the above scheme, the step of using the marked data X L Determine the supervised contrast loss function L sc and classification loss function L cls Specifically, it includes:

[0024] Determine the tag data X L The average value of the feature vectors is used as the class prototype for that category;

[0025] For each query sample, its feature vector is obtained through a feature extraction network, and the feature vector z is determined. L The Euclidean distance between the class and the class prototype of the m-th class;

[0026] The predicted probability of the query sample belonging to class m is determined based on the Euclidean distance.

[0027] According to the feature vector z L The true probability y of belonging to class m im and feature z L Prediction belonging to class m

[0028] probability Construct a classification loss function based on cross-entropy:

[0029]

[0030] Among them, y im For true probability, To predict probabilities.

[0031] In the above scheme, the step of using the marked data X L Determine the supervised contrast loss function L sc and classification loss function L cls Specifically, it includes:

[0032]

[0033] Among them, z L The feature vector of the labeled sample obtained by the feature extraction network. For feature vectors of the same type as the labeled samples, Let τ be the feature vector that is different from the labeled sample, τ be the temperature factor, and sim(.) be the cosine similarity.

[0034] In the above scheme, the step of using the unlabeled sample X U Determine the unsupervised contrastive loss function L uc Specifically, it includes:

[0035] Determine the mean of the feature vectors of the unlabeled samples under the l-th class view p.

[0036]

[0037] Where p∈{h,v}, h is the horizontal bidirectional offset, v is the vertical bidirectional offset, T is the number of samples of the l-th view p, and B is the total number of unlabeled samples;

[0038] Construct an unsupervised contrastive loss function L uc :

[0039]

[0040] in, ∈ represents the temperature factor, and B represents the total number of unlabeled samples.

[0041] In the above scheme, the step of using the unsupervised contrastive loss function L... uc Supervised contrastive loss function L sc and classification loss function L cls Constructing the final loss function specifically includes:

[0042] L Total =(1-λ)L cls +λ(L sc +L uc )

[0043] Where λ is the weight parameter, L uc For the unsupervised contrastive loss function, L sc To supervise the contrastive loss function, L cls This is the classification loss function.

[0044] This application also discloses a small-sample hyperspectral image classification system, characterized in that the system includes: a data acquisition unit, a loss function construction unit, a classification model training unit, and a classification model testing unit;

[0045] The data acquisition unit is used to acquire source domain hyperspectral images and target domain hyperspectral images, and defines unlabeled data in the target domain hyperspectral image as X. U The labeled data for the target domain hyperspectral image and the source domain hyperspectral image are defined as X. LAccording to the spectral feature shifting strategy, the unlabeled data X of the target domain hyperspectral image is processed. U Enhancement is performed to determine sample pair data; the first feature vector of the sample pair data is determined through the feature extraction network in the image classification model.

[0046] The loss function construction unit is used to construct the loss function based on the unlabeled sample X. U Determine the unsupervised contrastive loss function L uc According to the marked data X L Determine the supervised contrast loss function L sc and classification loss function L cls According to the unsupervised contrastive loss function L uc Supervised contrastive loss function L sc and classification loss function L cls Construct the final loss function; the classification model training unit is used to train the image classification model based on the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached.

[0047] The classification model testing unit is used to input test samples into the trained image classification model to obtain classification results.

[0048] This application also discloses a readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the following steps:

[0049] Acquire source domain hyperspectral images and target domain hyperspectral images, and define unlabeled data in the target domain hyperspectral image as X. U The labeled data for the target domain hyperspectral image and the source domain hyperspectral image are defined as X. L ;

[0050] Based on the spectral feature shifting strategy, the unlabeled data X of the target domain hyperspectral image is processed. U Enhancement is performed to identify sample pairs of data;

[0051] The first feature vector of the sample pair data is determined by the feature extraction network in the image classification model;

[0052] According to the unlabeled sample X U Determine the unsupervised contrastive loss function L uc According to the marked data X L Determine the supervised contrast loss function L sc and classification loss function L cls According to the unsupervised contrastive loss function L uc Supervised contrastive loss function L sc and classification loss function L clsConstruct the final loss function; the classification model training unit is used to train the image classification model based on the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached.

[0053] The test sample is input into the trained image classification model to obtain the classification result.

[0054] This application also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program is processed by the processor in the following steps:

[0055] Acquire source domain hyperspectral images and target domain hyperspectral images, and define unlabeled data in the target domain hyperspectral image as X. U The labeled data for the target domain hyperspectral image and the source domain hyperspectral image are defined as X. L ;

[0056] Based on the spectral feature shifting strategy, the unlabeled data X of the target domain hyperspectral image is processed. U Enhancement is performed to identify sample pairs of data;

[0057] The first feature vector of the sample pair data is determined by the feature extraction network in the image classification model;

[0058] According to the unlabeled sample X U Determine the unsupervised contrastive loss function L uc According to the marked data X L Determine the supervised contrast loss function L sc and classification loss function L cls According to the unsupervised contrastive loss function L uc Supervised contrastive loss function L sc and classification loss function L cls Construct the final loss function; the classification model training unit is used to train the image classification model based on the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached.

[0059] The test sample is input into the trained image classification model to obtain the classification result.

[0060] The embodiments of the present invention have the following beneficial effects: first, source domain hyperspectral images and target domain hyperspectral images are acquired, and unlabeled data in the target domain hyperspectral image is defined as X. U The labeled data for the target domain hyperspectral image and the source domain hyperspectral image are defined as X. L According to the spectral feature shifting strategy, the unlabeled data X of the target domain hyperspectral image is processed. UEnhancement is performed to determine sample pairs of data; the first feature vector of the sample pairs of data is determined through the feature extraction network in the image classification model; based on the unlabeled sample X... U Determine the unsupervised contrastive loss function L uc According to the marked data X L Determine the supervised contrast loss function L sc and classification loss function L cls According to the unsupervised contrastive loss function L uc Supervised contrastive loss function L sc and classification loss function L cls The final loss function is constructed; the classification model training unit is used to train the image classification model based on the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached; the test sample is input into the trained image classification model to obtain the classification result; the present invention improves the accuracy of small sample hyperspectral image classification and effectively enhances robustness, and can be used for mineral exploration, ecological monitoring, smart agriculture and medical diagnosis. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] in:

[0063] Figure 1 This is a schematic diagram of a small-sample hyperspectral image classification method in one embodiment;

[0064] Figure 2a A pseudo-color composite image of the Chikusei dataset;

[0065] Figure 2b The ground truth plot for the Chikusei dataset;

[0066] Figure 2c A pseudo-color composite image from the Indian Pines dataset;

[0067] Figure 2d This is a ground truth plot for the Indian Pines dataset;

[0068] Figure 3 This is a schematic diagram showing the results of classifying Indian Pines data using the DCFSL (Digital Direct Image Classification) method, a small-sample hyperspectral image classification method.

[0069] Figure 4 This is a schematic diagram illustrating the results of classifying Indian Pines data using the method of the present invention. Detailed Implementation

[0070] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention; however, it will be apparent to those skilled in the art that the invention may be practiced without one or more of these details; in other instances, certain technical features well-known in the art have not been described in order to avoid confusion with the invention. It should be understood that the invention can be practiced in different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided to make the disclosure thorough and complete and to fully convey the scope of the invention to those skilled in the art.

[0072] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. When used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms, unless the context clearly indicates otherwise. The terms “comprising” and / or “including,” when used in this specification, identify the presence of said features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups. When used herein, the term “and / or” includes any and all combinations of the associated listed items.

[0073] To fully understand the present invention, a detailed structure will be presented in the following description in order to illustrate the technical solution proposed by the present invention; optional embodiments of the present invention are described in detail below, however, in addition to these detailed descriptions, the present invention may have other embodiments.

[0074] like Figure 1 As shown, in one embodiment, a small-sample hyperspectral image classification method is provided, which includes steps S101 to S106, detailed below:

[0075] S101. Obtain the source domain hyperspectral image and the target domain hyperspectral image, and define the unlabeled data in the target domain hyperspectral image as X. U The labeled data for the target domain hyperspectral image and the source domain hyperspectral image are defined as X.L ;

[0076] Clearly distinguishing between source and target domain data, and clearly defining labeled data (for supervised learning) and unlabeled data (for unsupervised learning or data augmentation), is an important step in transfer learning and few-shot learning, which helps the model better adapt to the target domain.

[0077] S102. Based on the spectral feature shifting strategy, process the unlabeled data X of the target domain hyperspectral image. U Enhancement is performed to identify sample pairs of data;

[0078] Augmenting unlabeled data using spectral feature shifting strategies can generate more samples relevant to the target domain, helping to alleviate the small sample size problem. This strategy can increase the model's generalization ability, especially when spectral features change in the target domain.

[0079] In some embodiments, the unlabeled data X of the target domain hyperspectral image is processed according to a spectral feature shift strategy. U Enhancement is performed to identify sample pairs of data, specifically including:

[0080] Randomly select 'a' dimensions of the spectrum, and convert the unlabeled data X from the hyperspectral image of the target domain... U By performing horizontal and vertical bidirectional shifts respectively, sample pairs of data corresponding to the two views are obtained.

[0081] Preferably, the Chikusei dataset is selected as the source domain, and its pseudo-color composite image is shown in Figure 2(a), and its ground truth image is shown in Figure 2(b). The Indian Pines dataset is selected as the target domain, and its pseudo-color composite image is shown in Figure 2(c), and its ground truth image is shown in Figure 2(d). The Indian Pines dataset contains 16 land cover categories and a total of 42,776 labeled samples.

[0082] The Chikusei dataset is defined as the source domain, and the Indian Pines dataset as the target domain. Since the source and target domain data are trained alternately in meta-learning, this invention constructs a meta-dataset based on the source and target domains. In the set of training source domain data, 180 samples are randomly selected as training samples. In the set of training target domain data, [the following is referenced]... Figure 1 Randomly select 'a' dimensions of the spectrum, and perform horizontal and vertical bidirectional shifts on the unlabeled samples in the target domain to obtain two views of the unlabeled samples. Use the generated unlabeled samples containing the two views and 180 samples randomly selected from the data augmented data as training samples, where 'a' is 50 in this example.

[0083] S103. Determine the first feature vector of the sample pair data through the feature extraction network in the image classification model;

[0084] The first feature vector is extracted from the sample pair using a feature extraction network, which provides the foundation for subsequent classification and contrastive learning.

[0085] In some embodiments, the feature extraction network includes a lightweight spatial spectral convolutional module consisting of a group convolutional layer and a cross-channel fusion module, and a multi-scale transformer.

[0086] The input matrix in the grouped convolutional layer has a size of (9, 9, 256), each convolutional kernel has a size of (1, 1), a stride of (1, 1), and the output matrix has a size of (9, 9, 128).

[0087] The cross-channel fusion module divides a matrix of size (9, 9, 128) into two equal parts along the spectral dimension;

[0088] Applying the same depthwise convolution to both parts yields two output matrices. and The input matrix has a size of (9, 9, 64), each convolutional kernel has a size of (3, 3), and the output matrix has a size of (9, 9, 64).

[0089] For two matrices and Interact to obtain the cross-channel fused feature X ′ :

[0090]

[0091] X ′ =Concat(X′1,X′2)

[0092] Where α is the fusion weight coefficient.

[0093] The above-mentioned shortcut created between the input and output of the cross-channel fusion module can effectively reduce overfitting.

[0094] The multi-scale transformer consists of a normalization layer, a multi-scale self-attention network, another normalization layer, point convolutions, batch normalization layers, activation functions, and a multilayer perceptron connected sequentially. A shortcut is created between the input of the normalization layer and the output of the point convolutions. The multi-scale self-attention network comprises two parts: Local-Global Self-Attention (LGSA) and Cross-Group Self-Attention (CGSA). Its specific structure is as follows:

[0095] (1) Local-Global Self-Attention (LGSA):

[0096] Local-global self-attention first groups the input data of size (9,9,128) along the spatial dimension to obtain multiple local sub-data, each of size (3,3,128); then, it processes the input data with depthwise convolution to obtain global data, with a convolution kernel of (7,7), and the output data of size (3,3,128).

[0097] The local sub-data and global data are concatenated into a matrix X, which is then input into a multi-head attention network to obtain the local token L and the global token Λ.

[0098] Q = XW Q K = XW K V = XW V

[0099] Q = {Q1, Q2, ..., Q} i ,...,Q n}

[0100] K = {K1,K2,...,K} i ,...,K n}

[0101] V = {V1,V2,...,V} i ,...,V n}

[0102]

[0103] O = Concat(Z1, Z2, ..., Z) i ,...,Z n W O

[0104] Where d is the number of channels in the attention head, and W Q W K W V W O The projection matrix;

[0105] (2) Crossgroup Self-Attention (CGSA)

[0106] The global token Λ output by local-global self-attention is projected after passing through a layer normalization (LN) layer and a GELU activation function to obtain the query matrix Q. Λ and key matrix K Λ :

[0107]

[0108] Query matrix Q Λ Key matrix K Λ The local token L is input into the multi-head attention network and undergoes the following calculations:

[0109] Q Λ ={Q Λ1 Q Λ2 ,...,Q Λi ,...,Q Λn}

[0110] K Λ ={K Λ1 ,K Λ2 ,...,K Λi ,...,K Λn}

[0111] L = {L1, L2, ..., L} i ,...,L n}

[0112]

[0113] O Λ =Concat(Z) Λ1 Z Λ2 ,...,Z Λi ,...,Z Λn W ΛO

[0114] Where d is the number of channels in the attention head, and W ΛO This is the projection matrix.

[0115] O Λ After two-dimensional convolution, the output with cross-group self-attention size (9,9,128) is obtained.

[0116] S104, Based on the unlabeled sample X U Determine the unsupervised contrastive loss function L uc According to the marked data X L Determine the supervised contrast loss function L sc and classification loss function L cls According to the unsupervised contrastive loss function L uc Supervised contrastive loss function L sc and classification loss function L cls Construct the final loss function;

[0117] By combining classification loss function (for supervised learning), supervised contrastive loss function (for feature comparison of labeled data), and unsupervised contrastive loss function (for feature comparison of unlabeled data), this multi-loss function design helps to make full use of limited data resources and improve model performance.

[0118] In some embodiments, based on the tag data X LDetermine the supervised contrast loss function L sc and classification loss function L cls Specifically, it includes:

[0119] Determine the labeled data X L The average value of the feature vectors is used as the class prototype for that category;

[0120] For each query sample, its feature vector is obtained through a feature extraction network, and the feature vector z is determined. L The Euclidean distance between the class and the class prototype of the m-th class;

[0121] The predicted probability of a query sample belonging to class m is determined based on Euclidean distance.

[0122] Based on the eigenvector z L The true probability y of belonging to class m im and feature z L Predicted probability of belonging to class m Construct a classification loss function based on cross-entropy:

[0123]

[0124] Among them, y im For true probability, To predict probabilities.

[0125] In some embodiments, based on the tag data X L Determine the supervised contrast loss function L sc Specifically, it includes:

[0126]

[0127] Among them, z L The feature vector of the labeled sample obtained by the feature extraction network. For feature vectors of the same type as the labeled samples, Let τ be the feature vector that is different from the labeled sample, τ be the temperature factor, and sim(.) be the cosine similarity.

[0128] In some embodiments, based on the tag data X L Determine the supervised contrast loss function L sc and classification loss function L cls Specifically, it includes:

[0129] Determine the mean of the feature vectors of the unlabeled samples under the l-th class view p.

[0130]

[0131] Where p∈{h,v}, h is the horizontal bidirectional offset, v is the vertical bidirectional offset, T is the number of samples of the l-th view p, and B is the total number of unlabeled samples;

[0132] Construct an unsupervised contrastive loss function L uc :

[0133]

[0134] in, ∈ represents the temperature factor, B represents the total number of unlabeled samples, and L represents the unsupervised contrastive loss. uc This encourages the two groups to maintain consistency across different perspectives.

[0135] In some embodiments, based on the unsupervised contrastive loss function L uc Supervised contrastive loss function L sc and classification loss function L cls Constructing the final loss function specifically includes:

[0136] L Total =(1-λ)L cls +λ(L sc +L uc )

[0137] Where λ is the weight parameter, which takes the value 1e-5 in this example, L uc For the unsupervised contrastive loss function, L sc To supervise the contrastive loss function, L cls This is the classification loss function.

[0138] This invention designs a semi-supervised loss function based on the principle of contrastive learning. It uses groups instead of the traditional method based on individual samples to calculate the unsupervised contrastive loss, which can obtain more discriminative features.

[0139] S105. Train the image classification model based on the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached;

[0140] Using data from both the source and target domains to train the model helps the model better adapt to the target domain. By monitoring whether the number of training sets has reached the maximum number of iterations, the training process can be controlled, which helps to avoid overfitting or underfitting.

[0141] Specifically, the gradient descent method is used to update the parameters of the cross-scene hyperspectral image classification network: first, set the initial learning rate α, the number of meta-learning sets T, and the batch size B; initialize the parameters of the network model; then, construct the training data for each set according to the batch size, calculate the loss function, update the gradient of the loss function through backpropagation, and update the network training parameters through forward propagation according to the direction of loss function descent. The source domain and target domain data are trained alternately until the maximum number of sets T is reached.

[0142] S106. Input the test sample into the trained image classification model to obtain the classification result.

[0143] Furthermore, the test data is input into the trained feature extraction network based on convolutional converters, and the KNN algorithm is used to classify the samples to obtain classification results and simulation results. The effectiveness of this invention can be further illustrated by the following simulation results.

[0144] I. Simulation Conditions

[0145] Test data: Chikusei dataset was used as the source domain, and the IndianPines dataset was used as the target domain. Five labeled samples were randomly selected for each category in the target domain.

[0146] Test environment: Ubuntu 18.04LTS system, NVIDIA GeForce GTX 4090 GPU, PyTorch deep learning framework.

[0147] Evaluation indicators: The calculation formulas for the indicators are as follows:

[0148]

[0149]

[0150] Where TP represents the number of samples correctly classified into the current class, FN represents the number of samples in the current class that were incorrectly classified into other classes, TN represents the number of samples not in the current class that were classified into other classes, and FP represents the number of other samples that were incorrectly classified into the current class. x i+ x represents the sum of all elements in the i-th row of the confusion matrix, i.e., the sum of the actual quantities of this category. +i Let N represent the sum of all elements in the i-th column of the confusion matrix, which is the sum of the number of predicted cost categories. Here, N represents the sum of all elements, and c is the number of categories. x ii This represents the elements on the diagonal of the confusion matrix, i.e., the number of correctly predicted categories.

[0151] II. Simulation Content

[0152] Simulation 1: Under the above conditions, the existing small-sample hyperspectral image classification method DCFSL is used to classify the IndianPines data. The results are as follows: Figure 3 As shown.

[0153] Simulation 2: Under the above conditions, the method of this invention was used to classify Indian Pines data, and the results are as follows. Figure 4 As shown, where, Figure 3 , 4 The color marking module is the same as that in Figure 2(d).

[0154] pass Figure 3 and Figure 4 The comparison reveals that the classification result map obtained by the method proposed in this invention is closer to the ground feature distribution in the ground value map, and the classification effect is better in most categories, achieving superior classification performance.

[0155] III. Evaluation Indicators

[0156] The two methods described above were tested 10 times each using the same 10 random numbers. The confusion matrix, classification accuracy (PA) for each category, average classification accuracy (AA), overall classification accuracy (OA), and KAPPA coefficient were calculated. The test results are shown in the table below:

[0157]

[0158]

[0159] As can be seen, compared with the current more advanced DCFSL method, the present invention improves the classification accuracy of categories numbered 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, and 15. At the same time, the average classification accuracy, the overall classification accuracy, and the KAPPA coefficient are also significantly improved.

[0160] In summary, this invention proposes a few-sample hyperspectral image classification method based on semi-supervised contrastive learning. By alternating training with data from the source and target domains, a collaborative discriminative embedding space is learned. First, a feature extraction network combining convolution and transformers is designed to extract both local and long-range information. Second, a spectral feature shift strategy is proposed to generate augmented samples for calculating unsupervised contrastive loss. Next, a semi-supervised loss function based on contrastive learning is used to obtain more discriminative features. Then, the network training parameters are initialized, and the classification network is updated using gradient descent until the maximum number of iterations is reached. Finally, test samples are input into the trained network to obtain feature embeddings, which are then classified using the KNN algorithm.

[0161] This application also discloses a small-sample hyperspectral image classification system, which includes: a data acquisition unit, a loss function construction unit, a classification model training unit, and a classification model testing unit;

[0162] The data acquisition unit is used to acquire source domain hyperspectral images and target domain hyperspectral images, defining unlabeled data in the target domain hyperspectral image as X. U The labeled data for the target domain hyperspectral image and the source domain hyperspectral image are defined as X. L Based on the spectral feature shift strategy, the unlabeled data X of the target domain hyperspectral image are processed. U Enhancement is performed to determine the sample pair data; the first feature vector of the sample pair data is determined through the feature extraction network in the image classification model.

[0163] Loss function construction unit, used to construct loss function based on the unlabeled sample X U Determine the unsupervised contrastive loss function L uc According to the marked data X L Determine the supervised contrast loss function L sc and classification loss function L cls According to the unsupervised contrastive loss function L uc Supervised contrastive loss function L sc and classification loss function L cls Construct the final loss function;

[0164] The classification model training unit is used to train an image classification model based on the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached.

[0165] The classification model test unit is used to input test samples into the trained image classification model to obtain classification results.

[0166] This application also discloses a readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the following steps:

[0167] Acquire source domain hyperspectral images and target domain hyperspectral images, and define unlabeled data in the target domain hyperspectral image as X. U The labeled data for the target domain hyperspectral image and the source domain hyperspectral image are defined as X. L ;

[0168] Based on the spectral feature shifting strategy, the unlabeled data X of the target domain hyperspectral image is processed. U Enhancement is performed to identify sample pairs of data;

[0169] The first feature vector of the sample pair data is determined by the feature extraction network in the image classification model;

[0170] According to the unlabeled sample X U Determine the unsupervised contrastive loss function L uc According to the marked data X L Determine the supervised contrast loss function L sc and classification loss function L cls According to the unsupervised contrastive loss function L uc Supervised contrastive loss function L sc and classification loss function L cls Construct the final loss function;

[0171] The image classification model is trained based on the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached.

[0172] The test sample is input into the trained image classification model to obtain the classification result.

[0173] This application also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program is processed by the processor in the following steps:

[0174] Acquire source domain hyperspectral images and target domain hyperspectral images, and define unlabeled data in the target domain hyperspectral image as X. U The labeled data for the target domain hyperspectral image and the source domain hyperspectral image are defined as X. L ;

[0175] Based on the spectral feature shift strategy, the unlabeled data X of the target domain hyperspectral image are processed. U Enhancement is performed to identify sample pairs of data;

[0176] The first feature vector of the sample pair data is determined by the feature extraction network in the image classification model;

[0177] According to the unlabeled sample X U Determine the unsupervised contrastive loss function L uc According to the marked data X L Determine the supervised contrast loss function L sc and classification loss function L cls According to the unsupervised contrastive loss function L uc Supervised contrastive loss function L sc and classification loss function L cls Construct the final loss function; train the image classification model based on the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached;

[0178] The test sample is input into the trained image classification model to obtain the classification result.

[0179] Those skilled in the art will understand that implementing all or part of the processes in the above embodiments can be accomplished by instructing related hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0180] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0181] The embodiments described above are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application's patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. The embodiments disclosed above are merely preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made according to the claims of this invention are still within the scope of this invention.

Claims

1. A method for classifying small-sample hyperspectral images, characterized in that, The method includes: Obtain source domain hyperspectral images and target domain hyperspectral images, and define unlabeled data in the target domain hyperspectral image as... The labeled data for the target domain hyperspectral image and the source domain hyperspectral image are defined as follows: ; Unlabeled data of the target domain hyperspectral image are processed according to the spectral feature shifting strategy. Enhancement is performed to identify sample pairs of data; The first feature vector of the sample pair data is determined by the feature extraction network in the image classification model; According to the unlabeled data Determine the unsupervised contrastive loss function According to the tag data Determine the supervised comparison loss function and classification loss function According to the unsupervised contrastive loss function Supervised contrastive loss function and classification loss function Construct the final loss function; The image classification model is trained based on the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached. The test sample is input into the trained image classification model to obtain the classification result.

2. The small-sample hyperspectral image classification method according to claim 1, characterized in that, The unlabeled data of the target domain hyperspectral image is processed according to the spectral feature shifting strategy. Enhancement is performed to identify sample pairs of data, specifically including: Randomly select 'a' dimensions of the spectrum, and then use the unlabeled data from the hyperspectral image of the target domain. By performing horizontal and vertical bidirectional shifts respectively, sample pairs of data corresponding to the two views are obtained.

3. The small-sample hyperspectral image classification method according to claim 2, characterized in that, The feature extraction network includes a lightweight spatial spectral convolution module consisting of a group convolutional layer and a cross-channel fusion module, and a multi-scale converter. The input matrix in the group convolutional layer has a size of (9, 9, 256), each convolutional kernel has a size of (1, 1), a stride of (1, 1), and the output matrix has a size of (9, 9, 128). The cross-channel fusion module divides a matrix of size (9, 9, 128) into two equal parts along the spectral dimension; Applying the same depthwise convolution to the two parts yields two output matrices. and The input matrix has a size of (9, 9, 64), each convolutional kernel has a size of (3, 3), and the output matrix has a size of (9, 9, 64). For the two matrices and Interact to obtain cross-channel fused features : in, These are the weighting coefficients for fusion.

4. The small-sample hyperspectral image classification method according to claim 3, characterized in that, According to the tag data Determine the supervised comparison loss function and classification loss function Specifically, it includes: Determine the tag data The average value of the feature vectors is used as the class prototype of the category to which it belongs; For each query sample, its feature vector is obtained through a feature extraction network, and the feature vector is determined. The Euclidean distance between the class and the class prototype of the m-th class; The predicted probability of the query sample belonging to class m is determined based on the Euclidean distance. : According to the feature vector The true probability of belonging to class m and characteristics Predicted probability of belonging to class m Construct a classification loss function based on cross-entropy: in, For true probability, To predict probabilities.

5. The small-sample hyperspectral image classification method according to claim 4, characterized in that, According to the tag data Determine the supervised comparison loss function and classification loss function Specifically, it includes: in, The feature vector of the labeled sample obtained by the feature extraction network. For feature vectors of the same type as the labeled samples, For feature vectors that are different from the labeled samples, is the temperature factor, B is the total number of unlabeled data, and sim(.) is the cosine similarity.

6. The small-sample hyperspectral image classification method according to claim 5, characterized in that, The based on the unlabeled data Determine the unsupervised contrastive loss function Specifically, it includes: Determine the unlabeled data Mean of eigenvectors under class view p : in, h represents the horizontal bidirectional offset, v represents the vertical bidirectional offset, and T represents the first... The number of samples in class view p, where B is the total number of unlabeled data; Constructing an unsupervised contrastive loss function : in, , is the temperature factor, and B is the total number of unlabeled data.

7. The small-sample hyperspectral image classification method according to claim 6, characterized in that, The unsupervised contrastive loss function Supervised contrastive loss function and classification loss function Constructing the final loss function specifically includes: in, For weight parameters, For unsupervised contrastive loss function, To supervise the contrastive loss function, This is the classification loss function.

8. A small-sample hyperspectral image classification system, characterized in that, The system includes: a data acquisition unit, a loss function construction unit, a classification model training unit, and a classification model testing unit; The data acquisition unit is used to acquire source domain hyperspectral images and target domain hyperspectral images, defining unlabeled data in the target domain hyperspectral image as... The labeled data for the target domain hyperspectral image and the source domain hyperspectral image are defined as follows: ;Unlabeled data of the target domain hyperspectral image according to the spectral feature shift strategy Enhancement is performed to determine sample pair data; the first feature vector of the sample pair data is determined through the feature extraction network in the image classification model. According to the unlabeled data Determine the unsupervised contrastive loss function According to the tag data Determine the supervised comparison loss function and classification loss function According to the unsupervised contrastive loss function Supervised contrastive loss function and classification loss function Construct the final loss function; The classification model training unit is used to train an image classification model based on the source domain hyperspectral image and the target domain hyperspectral image until the maximum number of iterations is reached. The classification model testing unit is used to input test samples into the trained image classification model to obtain classification results.

9. A readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method as claimed in any one of claims 1 to 7.

10. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 7.

Images