Contrast learning classification method and system for hyperspectral remote sensing images

Abstract

The invention discloses a hyperspectral remote sensing image comparative learning classification method and system, and the method employs the prior information to carry out the effective screening and mining of negative samples, enables the negative samples to fully cover the whole sample space, enables the difficulty of the negative samples to be distributed in a balanced manner, reduces the difficulty of comparative learning in the discrimination of positive and negative samples, and improves the classification efficiency. Therefore, the final classification effect is improved. According to the method, a false negative example elimination strategy guided by a false label is adopted, false negative examples in a negative sample set can be effectively eliminated by introducing false label information, and the purity of negative samples is improved. The strategy not only ensures the accuracy of the negative sample in the contrast learning process, but also improves the consistency of the optimization directions of the contrast learning task and the classification task, and finally improves the classification precision.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing images, and in particular relates to a comparative learning classification method and system for hyperspectral remote sensing images. Background Art

[0002] Hyperspectral remote sensing images (HSI) contain rich spectral information and offer unique advantages in object recognition and image classification. This makes HSI classification tasks of great research and practical significance in environmental monitoring, agricultural resource management, mineral exploration, urban planning, and military reconnaissance. However, the high dimensionality and complexity of hyperspectral data, coupled with the limited availability of labeled samples, pose significant challenges to classification.

[0003] In recent years, contrastive learning, a self-supervised learning method, has made significant progress in hyperspectral image classification. Contrastive learning trains models by constructing pairs of similar and dissimilar samples, enabling them to learn meaningful feature representations without extensive labeled data. Despite this, current research focuses primarily on data augmentation methods and network architecture improvements, while systematic approaches to negative sample mining remain limited. In particular, methods for effectively screening and mining negative samples using prior information are still underdeveloped. Existing contrastive learning frameworks typically employ random sampling to construct batch data. For example, the prior art document CN118570631A provides a small-sample hyperspectral image classification method based on contrastive learning and Mahalanobis distance domain adaptation. This method utilizes source domain data to supplement target domain data. However, this method suffers from uneven category distribution, resulting in negative samples being concentrated in specific categories and failing to fully cover the entire sample space. Furthermore, the difficulty of negative samples is unevenly distributed, increasing the difficulty of contrastive learning in distinguishing between positive and negative samples, thus affecting the final classification performance. In addition, current contrastive learning methods lack a simple and effective mechanism to introduce hard and easy negative samples, which limits the application effect of contrastive learning in hyperspectral classification tasks. Summary of the Invention

[0004] In order to solve the problems existing in the prior art, the present invention proposes a comparative learning classification method and system for hyperspectral remote sensing images.

[0005] The technical solutions of the present invention are as follows:

[0006] A comparative learning classification method for hyperspectral remote sensing images, comprising:

[0007] Constructing a contrastive learning classification model for hyperspectral remote sensing images, wherein the contrastive learning classification model includes a classification task model of a shared encoder and a contrastive learning task model;

[0008] The classification task model and the contrastive learning task model are collaboratively trained based on the raw data of the hyperspectral remote sensing image until a loss function obtained by weighted fusion of the classification loss of the classification task model and the contrastive loss of the contrastive learning task model converges; during the collaborative training process, the classification task model generates pseudo labels by using an encoder and a classification tower structure as a classifier to assist the contrastive learning task model in mining negative samples; the contrastive learning task model mines negative samples based on the pseudo labels, taking into account false negative example elimination and mixed mining of difficult and easy negative samples, and optimizes classifier parameters based on the mined negative samples;

[0009] Hyperspectral remote sensing image contrast learning classification is performed based on the trained contrast learning classification model.

[0010] Furthermore, the specific method of the collaborative training includes:

[0011] A) Data preparation:

[0012] Get N labeled pixels X based on land cover category N =x1,x2,...,x N ∈R 1×1×B and M unlabeled pixels X M =x1,x2,...,x M ∈R 1×1×B HSI data composed of H=R H×W×B , where each labeled pixel in the labeled pixels corresponds to a class label, forming a class label Y N =y1,y2,...,y N ∈R c , where H and W are the height and width of the hyperspectral remote sensing image, B is the number of spectral channels, and C is the number of land cover classes; x1,x2,...,x N are the 1st, 2nd, ..., Nth pixels in the marked pixels respectively; x1, x2, ..., x M are the 1st, 2nd, ..., Mth pixels in the unlabeled pixels respectively; y1, y2, ..., y N are the class labels corresponding to the 1st, 2nd, ..., Nth pixels in the labeled pixels respectively; R is a real number;

[0013] Mark pixel X N and unlabeled pixels X M The adjacent cuboids centered at each pixel in form HSI patches, which are recorded as label data Z N =z1,z2,...,z N ∈R h×h×B and unlabeled data Z M =z1,z2,...,zM ∈R h×h×B ;z1,z2,...,z N are the 1st, 2nd, ..., Nth samples in the labeled data, z1, z2, ..., z M are the 1st, 2nd, …, Mth samples in the unlabeled data respectively; h is the patch size;

[0014] B) Classification Task:

[0015] Initially train the classification model and label the data Z N The input encoder generates a feature embedding vector, which is then passed through a classification tower structure consisting of an MLP layer and a Softmax layer to generate the probability P and pseudo label of each sample belonging to a given land cover category, and the classification model is optimized through the cross-entropy classification loss.

[0016] Unlabeled data prediction, use the optimized classification model to predict unlabeled data Z M Pseudo labels of

[0017] C) Contrastive learning task:

[0018] Data enhancement, the labeled data Z N Divide into several batches of samples, and use spectral parity mask enhancement for each batch of samples to generate two different views of the same sample, forming a network composed of anchor samples. and positive samples Positive sample pairs composed of;

[0019] Negative sample mining, using false negative elimination strategy, based on the unlabeled data Z in the classification task M Predicted pseudo labels, excluding anchor samples The samples with the same pseudo-labels are selected to reduce false negative samples; based on the pseudo-labels, the introduction of difficult and easy negative samples is balanced by calculating the spatial-spectral joint similarity between the anchor samples and the negative samples, and mixed mining of difficult and easy negative samples is performed to generate a negative sample queue.

[0020] Feature embedding generation: the positive sample pairs enter the encoder with the same parameters as the encoder in the classification task for encoding, and the negative sample queue enters the momentum encoder to learn the negative sample features. After generating feature embedding vectors respectively, they are optimized through the projection head to obtain the final feature embedding vector.

[0021] Loss calculation,InfoNCE loss is used as the contrast loss function to calculate the contrast loss.

[0022] Furthermore, the expression of the cross entropy classification loss is:

[0023]

[0024] Where yij is the true label, p ij ∈P is the predicted probability of sample i corresponding to category j; N and C represent the number of samples and the number of categories respectively.

[0025] Furthermore, the specific method of the false negative example elimination strategy includes:

[0026] Based on the labeled data Z N , preliminarily train the classification tower structure composed of MLP and Softmax layers to give the classification tower structure preliminary classification capabilities;

[0027] Use the classification tower structure to assign pseudo labels to the unlabeled data Z M ;

[0028] For anchor samples From the unlabeled data Z M Take out a set of pseudo-labeled samples F from i , identify F i The potential false negative examples with the same category as the anchor samples are removed to obtain a set of pseudo-labeled samples with different labels from the anchor samples, and this set is used as the negative sample set N for the contrast task training. i .

[0029] Furthermore, the specific method of mixed mining of difficult and easy negative samples includes:

[0030] Calculate anchor point samples Its negative sample set N i The spatial-spectral joint similarity of each negative sample in , the similarity score is defined as

[0031]

[0032] sim=e w×spe+(1-w)×spa

[0033]

[0034]

[0035] Where, spa is the spatial similarity; spe is the spectral similarity; and are the x-coordinates of the anchor samples and negative samples, respectively. and are the y coordinates of the anchor sample and the negative sample respectively, and the parameter w is the weight of the spectral similarity;

[0036] Finally, the number of negative samples K is introduced as a hyperparameter, and N is quantified according to the spatial-spectral joint similarity. iThe hyperparameter K controls the hardness of negative samples, that is, the degree of difficulty in mixing negative samples, by specifying the number of negative samples to be extracted according to the similarity score.

[0037] Furthermore, the InfoNCE loss is defined as follows:

[0038]

[0039] Where K is the number of negative samples in the queue, 1 [k≠i] ∈{0,1} is the indicator function, which is equal to 1 when k≠i. represents the cosine similarity between two feature embedding vectors, and the parameter τ represents the temperature parameter.

[0040] Furthermore, the loss function obtained by weighted fusion of the classification loss of the classification task model and the contrastive loss of the contrastive learning task model is expressed as:

[0041] Loss=CE loss +λ·InfoNCE loss

[0042] Where λ is the weight of InfoNCE loss.

[0043] A contrastive learning classification system for hyperspectral remote sensing images, comprising: a model building module, a collaborative training module and a contrastive learning classification module;

[0044] The model building module is used to build a contrastive learning classification model for hyperspectral remote sensing images, wherein the contrastive learning classification model includes a classification task model of a shared encoder and a contrastive learning task model;

[0045] The collaborative training module is used to collaboratively train the classification task model and the contrastive learning task model based on the original data of the hyperspectral remote sensing image until the loss function obtained by weighted fusion of the classification loss of the classification task model and the contrastive loss of the contrastive learning task model converges. During the collaborative training process, the classification task model generates pseudo labels by acting as an encoder of the classifier to assist the contrastive learning task model in mining negative samples. The contrastive learning task model mines negative samples based on the pseudo labels, taking into account false negative example elimination and mixed mining of difficult and easy negative samples, and optimizes the classifier parameters based on the mined negative samples.

[0046] The contrastive learning classification module is used to perform contrastive learning classification of hyperspectral remote sensing images based on the trained contrastive learning classification model.

[0047] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to call and run the computer program stored in the memory to execute any of the above methods.

[0048] A computer-readable storage medium stores a computer program, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention proposes a comparative learning classification method and system for hyperspectral remote sensing images. The method uses prior information to effectively screen and mine negative samples, so that negative samples fully cover the entire sample space. At the same time, the difficulty of negative samples is evenly distributed, reducing the difficulty of comparative learning in distinguishing positive and negative samples, thereby improving the final classification effect.

[0051] The method proposed in this paper adopts a pseudo-label-guided false negative elimination strategy. By introducing pseudo-label information, it can effectively eliminate false negatives from the negative sample set and improve the purity of negative samples. This strategy not only ensures the accuracy of negative samples during contrastive learning, but also improves the consistency of the optimization direction of contrastive learning and classification tasks, ultimately improving classification accuracy.

[0052] The method also employs a mixed mining strategy for difficult and easy negative samples based on similarity calculation. By using a joint spatial-spectral similarity calculation, negative samples are subdivided into difficult and easy negative samples, making negative sample mining more flexible and effective. This strategy not only better reflects the true distribution of negative samples but also improves the robustness of contrastive learning and enhances classification performance by mixing difficult and easy negative samples.

[0053] The proposed system model framework offers excellent flexibility and adaptability. The proposed hyperspectral remote sensing image contrast learning classification framework can flexibly support the adaptive selection and mining of negative samples, adapting to diverse datasets and application scenarios. This flexibility enables the proposed method to achieve good results in various hyperspectral remote sensing image classification tasks, demonstrating its broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Flowchart of the comparative learning classification method for hyperspectral remote sensing images in the embodiment;

[0055] Figure 2 This is an architecture diagram of a comparative learning classification method for hyperspectral remote sensing images in an embodiment;

[0056] Figure 3 Schematic diagram of the encoder module in the embodiment;

[0057] Figure 4 This is a diagram of the negative sample mining strategy in the embodiment;

[0058] Figure 5 The false color composite image of the Indian Pines dataset, the corresponding real map, and the example image in the embodiment;

[0059] Figure 6 False color composite images of the WHU-Hi-HongHu dataset, the corresponding real maps, and example images in the embodiment;

[0060] Figure 7 False color composite images of the Salinas dataset, corresponding real maps, and example images in the embodiments;

[0061] Figure 8 Schematic diagram of the classification performance of IP, WHHH and SA datasets under different weights in the embodiment;

[0062] Figure 9 Schematic diagram of the classification performance of IP, WHHH and SA datasets under different numbers of negative samples in the embodiment;

[0063] Figure 10(a) is a schematic diagram of the overall classification accuracy of different algorithms under the IP dataset;

[0064] Figure 10(b) is a schematic diagram of the overall classification accuracy of different algorithms under the WHHH dataset;

[0065] Figure 10(c) is a schematic diagram of the overall classification accuracy of different algorithms under the SA dataset. DETAILED DESCRIPTION

[0066] The present invention is further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent modifications of the present invention by those skilled in the art fall within the scope defined by the claims attached to this application.

[0067] Example 1:

[0068] The present invention provides a comparative learning classification method for hyperspectral remote sensing images, such as Figure 1 and Figure 2 Shown, including:

[0069] S1. Construct a contrastive learning classification model for hyperspectral remote sensing images. The contrastive learning classification model includes a classification task model of a shared encoder and a contrastive learning task model.

[0070] S2. Co-train the classification task model and the contrastive learning task model based on the raw data of the hyperspectral remote sensing imagery until the loss function obtained by weighted fusion of the classification loss of the classification task model and the contrastive loss of the contrastive learning task model converges. During the co-training process, the classification task model generates pseudo-labels by acting as a classifier encoder and a classification tower structure to assist the contrastive learning task model in mining negative samples. The contrastive learning task model mines negative samples based on the pseudo-labels, taking into account false negative example elimination and mixed mining of difficult and easy negative samples, and optimizes the classifier parameters based on the mined negative samples.

[0071] S3. Perform comparative learning and classification of hyperspectral remote sensing images based on the trained comparative learning classification model.

[0072] Example 2:

[0073] This embodiment is further designed based on the first embodiment in that the specific method of collaborative training in this embodiment includes:

[0074] A) Data preparation:

[0075] Get N labeled pixels X based on land cover category N =x1,x2,...,x N ∈R 1×1×B and M unlabeled pixels X M =x1,x2,...,x M ∈R 1×1×B HSI data composed of H=R H×W×B , where each labeled pixel in the labeled pixels corresponds to a class label, forming a class label Y N =y1,y2,...,y N ∈R c , where H and W are the height and width of the hyperspectral remote sensing image, B is the number of spectral channels, and C is the number of land cover classes; x1,x2,...,x N are the 1st, 2nd, ..., Nth pixels in the marked pixels respectively; x1, x2, ..., x M are the 1st, 2nd, ..., Mth pixels in the unlabeled pixels respectively; y1, y2, ..., y N are the class labels corresponding to the 1st, 2nd, ..., Nth pixels in the labeled pixels respectively; R is a real number;

[0076] Mark pixel X N and unlabeled pixels X M The adjacent cuboids centered at each pixel in form HSI patches, which are recorded as label data Z N =z1,z2,...,z N ∈R h×h×B and unlabeled data ZM =z1,z2,...,z M ∈R h×h×B ;z1,z2,...,z N are the 1st, 2nd, ..., Nth samples in the labeled data, z1, z2, ..., z M are the 1st, 2nd, …, Mth samples in the unlabeled data respectively; h is the patch size;

[0077] B) Classification Task:

[0078] like Figure 2 As shown, the classification model is initially trained and the labeled data set Z N Each sample is input into the encoder to generate a feature embedding vector, and then the classification tower structure composed of the MLP layer and the Softmax layer generates the probability P and pseudo label of each sample belonging to the established land cover category, and the classification model is optimized by the cross entropy CE classification loss.

[0079] Unlabeled data prediction, using the optimized classification model to predict the unlabeled data set Z M The pseudo label of each sample in ;

[0080] C) Contrastive learning task:

[0081] Data enhancement, the labeled data Z N Divide into several batches of samples, and use spectral parity mask enhancement for each batch of samples to generate two different views of the same sample, forming a network composed of anchor samples. and positive samples Positive sample pairs composed of;

[0082] Negative sample mining, such as Figure 4 As shown, we first adopt the false negative elimination strategy based on the unlabeled data set Z in the classification task. M The pseudo labels predicted for each sample in , excluding the anchor samples The method selects samples with the same pseudo-label as the anchor sample to reduce false negative samples. Then, based on the pseudo-label, the spatial-spectral joint similarity between the anchor sample and the negative sample is calculated to adjust the "hardness" of the negative sample, balance the introduction of difficult and easy negative samples, perform mixed mining of difficult and easy negative samples, and generate a negative sample queue.

[0083] Feature embedding generation: the positive sample pairs enter the encoder with the same parameters as the encoder in the classification task for encoding, and the negative sample queue enters the momentum encoder to learn the negative sample features. It maintains the moving average of the encoder parameters and provides a more stable and consistent negative sample representation. After generating the feature embedding vectors respectively, they are optimized through the projection head to obtain the final feature embedding vector.

[0084] Loss calculation,InfoNCE loss is used as the contrast loss function to calculate the contrast loss.

[0085] Example 3:

[0086] This embodiment is further designed on the basis of the second embodiment in that the expression of the cross entropy CE classification loss in this embodiment is:

[0087]

[0088] Where y ij is the true label, p ij ∈P is the predicted probability of sample i corresponding to category j; N and C represent the number of samples and the number of categories respectively.

[0089] Example 4:

[0090] This embodiment further enhances the second embodiment in that the false negative elimination strategy is primarily to train a basic classifier, assign pseudo labels to all samples, and after selecting anchor samples, divide negative samples into samples with the same label as the anchor sample and samples with different labels. The former are more likely to contain a large number of pseudo negative samples. Based on this classification, the goal is to eliminate false negative samples from the negative sample set as effectively as possible. The specific methods include:

[0091] Based on the labeled data Z N , preliminarily train the classification tower structure composed of MLP (Multi-layer Perceptron) and Softmax layer to give the classification tower structure preliminary classification capabilities;

[0092] Use the classification tower structure to assign pseudo labels to the unlabeled data set Z M Each sample;

[0093] For anchor samples From the unlabeled data set Z M Take out a set of pseudo-labeled samples F from i , identify F i The potential false negative examples with the same category as the anchor samples are removed to obtain a set of pseudo-labeled samples with different labels from the anchor samples, and this set is used as the negative sample set N for the contrast task training. i .

[0094] The false negative elimination strategy aims to eliminate samples whose pseudo labels belong to the same category as the positive samples, that is, false negative examples, thereby avoiding the sample noise caused by false negative examples in traditional contrastive learning tasks due to random negative sample sampling, and strengthening the consistency of the optimization direction of contrastive learning tasks and classification tasks.

[0095] Embodiment 5:

[0096] This embodiment is further designed based on the fourth embodiment in that the specific method of mixed mining of difficult and easy negative samples in this embodiment includes:

[0097] First, we introduce hard negative samples and easy negative samples.

[0098] Hard negative samples: is a true negative example, and the anchor have different labels (i.e., they belong to different classes) but have similar characteristics and are therefore difficult to distinguish from anchors.

[0099] Easy negative samples: is a true negative example, and the anchor Having different labels and at the same time different characteristics makes it easy to distinguish from anchors.

[0100] Reference Figure 4 , where (a) (f) describes the traditional random negative sampling process in contrastive learning; (b) and (c) show the false negative elimination strategy guided by pseudo labels; (d) and (e) represent the mixed mining strategy of hard and easy negative samples based on pseudo label guidance and similarity calculation. Figure 3 (a) and (f), due to the lack of supervision information, the existing contrastive learning framework usually constructs a batch of negative sample pairs by random sampling. Randomly sampled negative samples may introduce false negative samples, resulting in performance degradation. In order to mitigate the impact of false negative examples, this application proposes a false negative example elimination strategy based on pseudo labels, referring to Figure 3 (b) and (c), specifically, we first train a base classifier to assign pseudo labels to all samples. Once the anchor is selected, the negative samples can be divided into samples with the same label as the anchor and samples with different labels. The former is more likely to contain a large number of false negatives. Based on this classification, our goal is to eliminate false negatives from the negative sample set as effectively as possible. Based on the labeled data Z N , perform preliminary training on the classification tower in the network to enable the classification structure to have preliminary classification capabilities; use the classification network to classify the unlabeled data Z M Assign pseudo labels; finally, for the anchor point, a set of pseudo-labeled samples F with different labels from its own are obtained i , and use this group of samples as the negative sample set for contrast task training.

[0101] Reference Figure 3 (d) and (e), we propose an efficient and easy-to-implement strategy for mining a mixture of hard and easy negative samples, guided by pseudo-label information. Guided by pseudo-labels, we can effectively identify negative samples with different labels from the anchors. Furthermore, our method allows us to adjust the "hardness" level of negative samples based on their spatial-spectral similarity. This flexibility enables users to balance the selection of the best negative samples from both hard and easy negative samples.

[0102] Calculate anchor point samples Its negative sample set N i The spatial-spectral joint similarity of each negative sample in , the similarity score is defined as

[0103]

[0104] sim=e w×spe+(1-w)×spa

[0105]

[0106]

[0107] Where, spa is the spatial similarity; spe is the spectral similarity; and are the x-coordinates of the anchor samples and negative samples, respectively. and are the y coordinates of the anchor sample and the negative sample respectively, and the parameter w is the weight of the spectral similarity;

[0108] Finally, the number of negative samples K is introduced as a hyperparameter, and N is quantified according to the spatial-spectral joint similarity. i The hyperparameter K controls the hardness of negative samples, that is, the degree of difficulty in mixing negative samples, by specifying the number of negative samples to be extracted according to the similarity score.

[0109] Based on similarity, negative samples are classified into two categories: hard negative samples and easy negative samples. Negative samples with high similarity to the anchor are considered hard negative samples, meaning they are difficult to distinguish from the anchor. Negative samples with low similarity to the anchor are considered easy negative samples, meaning they are easy to distinguish from the anchor. This classification of hard and easy negative samples allows for more detailed and effective mining of the information contained in negative samples. The mixed mining strategy for hard and easy negative samples aims to eliminate the imbalance in the distribution of hard and easy negative samples, reduce the difficulty of distinguishing positive and negative samples, and thus better align the optimization direction of contrastive learning with that of the classification task.

[0110] Example 6:

[0111] This embodiment is further designed based on the fifth embodiment in that the InfoNCE loss is defined as follows:

[0112]

[0113] Where K is the number of negative samples in the queue, 1 [k≠i] ∈{0,1} is the indicator function, which is equal to 1 when k≠i. Represents the cosine similarity between two feature embedding vectors, and the parameter τ represents the temperature parameter, with a default value of 0.5. The goal of the contrastive training process is to minimize the InfoNCE loss, which ensures that positive samples are closer in the embedding space, while negative samples are pushed further away.

[0114] Embodiment seven:

[0115] This embodiment is further designed on the basis of the first embodiment in that the loss function obtained by weighted fusion of the classification loss of the classification task model and the contrastive loss of the contrastive learning task model is expressed as follows:

[0116] Loss=CE loss +λ·InfoNCE loss

[0117] Where λ is the weight of InfoNCE loss.

[0118] Embodiment 8:

[0119] A comparative learning classification system for hyperspectral remote sensing images of the present invention comprises: a model building module, a collaborative training module and a comparative learning classification module;

[0120] A model building module is used to build a contrastive learning classification model for hyperspectral remote sensing images. The contrastive learning classification model includes a classification task model of a shared encoder and a contrastive learning task model.

[0121] A collaborative training module is used to collaboratively train the classification task model and the contrastive learning task model based on the raw data of hyperspectral remote sensing images until the loss function obtained by weighted fusion of the classification loss of the classification task model and the contrastive loss of the contrastive learning task model converges. During the collaborative training process, the classification task model generates pseudo labels by acting as an encoder of the classifier to assist the contrastive learning task model in mining negative samples. The contrastive learning task model mines negative samples based on the pseudo labels, taking into account false negative example elimination and mixed mining of difficult and easy negative samples, and optimizes the classifier parameters based on the mined negative samples.

[0122] The contrastive learning classification module is used to perform contrastive learning classification of hyperspectral remote sensing images based on the trained contrastive learning classification model.

[0123] Embodiment 9:

[0124] This example uses the method of the present invention to perform comparative learning classification of hyperspectral remote sensing images, which includes the following steps:

[0125] A novel hyperspectral remote sensing image classification framework based on contrastive learning is constructed; this framework consists of two parts: the first is the classification task architecture, and the second is the contrastive learning task architecture. These two architectures complement each other and achieve complementary advantages in the final classification optimization through the classification and contrastive learning tasks. The contrastive learning task helps the classification task model converge better, and the classification model assists the contrastive learning task in mining negative samples by generating pseudo-labels, thus ensuring consistency in the optimization direction of the contrastive learning and classification tasks.

[0126] Use the classifier to generate pseudo labels and filter out false negatives through pseudo labels;

[0127] Under the guidance of pseudo labels, mixed mining of difficult and easy negative samples is achieved by calculating sample similarity;

[0128] The mined negative samples are applied to the contrastive learning classification task to optimize the classifier; by adjusting the ratio of difficult and easy negative samples, the classification performance of hyperspectral remote sensing images is improved.

[0129] The novel hyperspectral remote sensing image classification framework includes both classification and contrastive learning tasks based on the Transformer architecture. This approach utilizes the Transformer architecture for classification, leveraging its powerful feature extraction capabilities to process the high-dimensional nature of hyperspectral data. In contrastive learning, negative sample mining adaptively constructs sample pairs, generating more discriminative feature representations. This architecture leverages the effectiveness of contrastive learning to improve classifier performance.

[0130] Pseudo-label generation is achieved by training a supervised classifier to classify unlabeled samples. Using limited available labeled data, a supervised classifier model is trained. After training, the classifier classifies unlabeled hyperspectral image data and generates pseudo-labels. These pseudo-labels are used to guide the selection and mining of negative samples, ensuring that the selected negative samples are highly informative and representative.

[0131] The mixed mining strategy for difficult and easy negative samples involves calculating the joint spatial-spectral similarity between negative samples and anchor samples, and sorting and filtering the negative samples based on this similarity. Spatial similarity measures the similarity between samples in their spatial distribution, while spectral similarity measures the similarity in their spectral features. Combining these two similarity scores, negative samples are sorted and a subset of representative negative samples with strong information representation capabilities is selected. These negative samples include samples with varying degrees of difficulty compared to the anchor samples, which can improve the robustness and classification performance of contrastive learning.

[0132] Mixed mining of difficult and easy negative samples is achieved by adjusting the number of negative samples using hyperparameters to control the difficulty of distinguishing anchors from negative samples. By setting these hyperparameters, the model can adjust the number of negative samples and their difficulty level to find the optimal negative sample combination. Hyperparameter setting, including adjusting the number of negative samples, determines the difficulty level of the negative samples. Through experimental verification and adjustment, the optimal negative sample combination for the specific task and dataset of hyperspectral remote sensing image classification is found, thereby optimizing the performance of contrastive learning and classification tasks.

[0133] Spectral remote sensing image classification tasks encompass a wide range of application scenarios. The method presented in this paper is applicable to a variety of hyperspectral remote sensing image classification tasks, including but not limited to crop classification, land use classification, mineral resource exploration, and environmental monitoring. Experimental validation using datasets from various fields demonstrates that this method exhibits high classification accuracy and robustness across a wide range of application scenarios.

[0134] Embodiment 10:

[0135] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method of any of the above embodiments.

[0136] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above embodiments are implemented.

[0137] Application examples:

[0138] This example simulates the implementation of the comparative learning classification method for hyperspectral remote sensing images of the present invention. This example uses three public datasets, namely the Indian Pines dataset (IP), the WHU-Hi-HongHu dataset (WHHH), and the Salinas (SA) dataset. The Indian Pines dataset was acquired in 1992 using the Airborne Visible / Infrared Imaging Spectrometer (AVIRIS) sensor, covering the northwestern region of Indiana, USA. The dataset consists of 224 spectral bands with a wavelength range from 0.4μm to 2.5μm, a spatial resolution of 20 meters, and a wavelength range of 145×145 pixels. It contains 16 different land cover categories. In the analysis of this example, 24 frequency bands related to water absorption and noise were deleted, narrowing the selection range to 200 frequency bands. The false color image and the classified ground truth are as follows. Figure 5 shown.

[0139] The WHU-Hi-HongHu dataset was acquired on November 20, 2017 in Honghu City, Hubei Province, China. The dataset was captured using a Headwall Nano-Hyperspec imaging sensor with a focal length of 17 mm, mounted on a DJIMatrice 600Pro unmanned aerial vehicle (UAV) operating at an altitude of 100 meters. The resulting images have a resolution of 940×475 pixels and contain 270 spectral bands ranging from 400 to 1000 nm, providing a spatial resolution of approximately 0.043 meters for hyperspectral UAV imagery. The captured scenes feature a variety of crop types, including different varieties of the same crop, such as Chinese cabbage, cabbage, brassica rapa, and small brassica rapa. In total, the dataset covers 22 different land cover types. The false color images and classified ground truth are shown in Figure 2. Figure 6 shown.

[0140] The Salinas dataset was collected by an airborne hyperspectral sensor and covers an agricultural area in the Salinas Valley, California, USA. The dataset consists of 512×217 pixels with 224 spectral bands ranging from 400nm to 2500nm and a spatial resolution of approximately 3.7 meters. It includes 16 different crop types, such as lettuce, broccoli, onion, and cauliflower. The false color images and classification ground truth are as follows: Figure 7 shown.

[0141] Training details: PyTorch training framework is used for model implementation and training. The basic hardware environment includes Platinum 8336C CPU @ 2.30GHz, 64 cores, 256GB memory and two NVIDIA GeForce RTX4090 GPUs, each with 24GB memory. The partitioning of the training and test datasets used in this study is shown in Tables I-III.

[0142] Table I Number of training and testing samples in the INDIAN PINES dataset

[0143]

[0144] Table II Number of training and testing samples in the WHU-HI-HONGHU dataset

[0145]

[0146]

[0147] Table III Number of training and testing samples in the SALINAS dataset

[0148]

[0149] Evaluation metrics: This example uses overall accuracy (OA), average accuracy (AA), kappa coefficient (κ), and classification accuracy of each land cover class to measure the classification effect of the algorithm.

[0150] Model training: This example uses the PyTorch framework to train the model in a specific hardware environment. Each dataset is divided into training and test sets according to a specific ratio. For example, 10 samples from each category are selected as the training set, and the remaining samples are used as the test set.

[0151] Parameter Analysis: This example analyzes two key parameters: weights and the number of negative samples. The weights control the spatial similarity in the joint spatial-spectral similarity calculation, while the number of negative samples influences the proportion of hard negative samples in the contrastive learning cohort. These two parameters are tested on different datasets to determine their impact on model performance and to find the optimal parameter values ​​for each dataset.

[0152] Comparative experiment: This example compares the proposed method with several representative algorithms, including HiT, SSFTT, and CASST. Under the same training and testing conditions, the classification performance of each algorithm on different datasets is compared to verify the effectiveness and superiority of the proposed method.

[0153] Parameter analysis results: This example tests the weight w and the number of negative samples K on the IP, WHHH and SA datasets. Figure 8 As shown in Figure 2, the optimal control weights for achieving the highest OA on different datasets are different. The IP and SA datasets perform best at a lower weight value of 0.2, while the WHHH dataset is suitable for a medium weight value of 0.4. In addition, Figure 9 The effect of the number of selected negative samples on OA in the same dataset was evaluated. It is not difficult to see that the optimal number of negative samples varies on different datasets. The IP and WHHH datasets achieved the best results with a moderate number of negative samples (256).

[0154] Comparative test results:

[0155] To evaluate the effectiveness of the proposed method, this example conducts comparative experiments using several representative algorithms, including HiT, SSFTT, CASST, SC-EADNet, S3Net, and SS-MTr, to compare the proposed method (Ours) with existing algorithms.

[0156] 1) Quantitative results:

[0157] To quantitatively analyze the effectiveness of the proposed method, we summarize the classification performance on the IP, WHHH, and SA datasets, as shown in Tables IV-VI, respectively. Each table includes the classification accuracy of each method in different categories, as well as the OA, AA, and Kappa coefficient (κ). The best performance for each metric is highlighted in bold.

[0158] Table IV Classification results of experiments on the Indian pine dataset (10 labels per class)

[0159]

[0160]

[0161] Table V Classification results of experiments on the WHU-Hi-HongHu dataset (10 labels per class)

[0162]

[0163] Table VI Classification results of experiments on the Salinas dataset (10 labels per class)

[0164]

[0165]

[0166] As shown in Tables IV-VI, the proposed method further improves classification performance compared to other algorithms, with OA, AA, and κ achieving the best results. Specifically, compared to the second-best method, the proposed method improves OA by 4.37%, 0.73%, and 0.78% on the IP, PU, ​​and WHHH datasets, respectively. Furthermore, AA and κ also improve on these three datasets by 1.03%, 0.49%, and 0.35%, and by 4.86%, 0.88%, and 0.86%, respectively.

[0167] Looking at the accuracy of individual subclasses in each dataset, the proposed method still outperforms the comparison model. As shown in Table IV, the classification accuracy for 10 of the 16 subclasses in the IP dataset is significantly higher than that of the other methods. This is particularly true for the second and tenth subclasses, where the proposed method achieves improvements of 13.41% and 6.76%, respectively, compared to the second-ranked methods, CASST and SSFTT. For subclasses where the algorithm performs relatively poorly in the IP dataset, the performance difference is relatively small compared to the highest-performing classification metric in that subclass. As can be seen in Table V, the proposed method also performs well in the subclass classification of the WHHH dataset. For example, significant results of 4.74%, 7.03%, 4.86%, and 6.43% were achieved for classes 11, 12, 20, and 21, respectively. These improvements represent significant improvements, while the gap between the best method and most other poorly performing subclasses is relatively small. For the SA dataset, as shown in Table VI, the classification accuracy of most subclasses using these methods is already very high, leaving limited room for improvement. However, the method in this example still performs best, with the classification accuracy of most subclasses close to 100%, especially the eighth subclass, which has a significant improvement of 4.51%, once again verifying the effectiveness of the method proposed in this invention.

[0168] This example also verifies the performance of each method using different training sample sizes. As shown in Figure 10, the experiment demonstrates that the classification accuracy of each model improves to varying degrees with increasing sample size. Notably, the proposed model performs competitively across different training sample sizes on the IP, WHHH, and SA datasets. This experiment further confirms the effectiveness of the proposed method.

[0169] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A comparative learning classification method for hyperspectral remote sensing images, characterized in that: include: Constructing a contrastive learning classification model for hyperspectral remote sensing images, wherein the contrastive learning classification model includes a classification task model of a shared encoder and a contrastive learning task model; The classification task model and the contrastive learning task model are collaboratively trained based on the raw data of the hyperspectral remote sensing image until a loss function obtained by weighted fusion of the classification loss of the classification task model and the contrastive loss of the contrastive learning task model converges; during the collaborative training process, the classification task model generates pseudo labels by using an encoder and a classification tower structure as a classifier to assist the contrastive learning task model in mining negative samples; the contrastive learning task model mines negative samples based on the pseudo labels, taking into account false negative example elimination and mixed mining of difficult and easy negative samples, and optimizes classifier parameters based on the mined negative samples; Hyperspectral remote sensing image contrast learning classification is performed based on the trained contrast learning classification model.

2. The comparative learning classification method for hyperspectral remote sensing images according to claim 1, characterized in that: The specific method of the collaborative training includes: A) Data preparation: Get N labeled pixels X based on land cover category N =x1,x2,...,x N ∈R 1×1×B and M unlabeled pixels X M =x1,x2,...,x M ∈R 1×1×B HSI data composed of H=R H×W×B , where each labeled pixel in the labeled pixels corresponds to a class label, forming a class label Y N =y1,y2,...,y N ∈R c , where H and W are the height and width of the hyperspectral remote sensing image, B is the number of spectral channels, and C is the number of land cover classes; x1,x2,...,x N are the 1st, 2nd, ..., Nth pixels in the marked pixels respectively; x1, x2, ..., x M are the 1st, 2nd, ..., Mth pixels in the unlabeled pixels respectively; y1, y2, ..., y N are the class labels corresponding to the 1st, 2nd, ..., Nth pixels in the labeled pixels respectively; R is a real number; Mark pixel X N and unlabeled pixels X M The adjacent cuboids centered at each pixel in form HSI patches, which are recorded as label data Z N =z1,z2,...,z N ∈R h×h×B and unlabeled data Z M =z1,z2,...,z M ∈R h×h×B ;z1,z2,...,z N are the 1st, 2nd, ..., Nth samples in the labeled data, z1, z2, ..., z M are the 1st, 2nd, …, Mth samples in the unlabeled data respectively; h is the patch size; B) Classification Task: Initially train the classification model and label the data Z N The input encoder generates a feature embedding vector, which is then passed through a classification tower structure consisting of an MLP layer and a Softmax layer to generate the probability P and pseudo label of each sample belonging to a given land cover category, and the classification model is optimized through the cross-entropy classification loss. Unlabeled data prediction, use the optimized classification model to predict unlabeled data Z M Pseudo labels of C) Contrastive learning task: Data enhancement, the labeled data Z N Divide into several batches of samples, and use spectral parity mask enhancement for each batch of samples to generate two different views of the same sample, forming a network composed of anchor samples. and positive samples Positive sample pairs composed of; Negative sample mining, using false negative elimination strategy, based on the unlabeled data Z in the classification task M Predicted pseudo labels, excluding anchor samples The samples with the same pseudo-labels are selected to reduce false negative samples; based on the pseudo-labels, the introduction of difficult and easy negative samples is balanced by calculating the spatial-spectral joint similarity between the anchor samples and the negative samples, and mixed mining of difficult and easy negative samples is performed to generate a negative sample queue. Feature embedding generation: the positive sample pairs enter the encoder with the same parameters as the encoder in the classification task for encoding, and the negative sample queue enters the momentum encoder to learn the negative sample features. After generating feature embedding vectors respectively, they are optimized through the projection head to obtain the final feature embedding vector. Loss calculation,InfoNCE loss is used as the contrast loss function to calculate the contrast loss.

3. The comparative learning classification method for hyperspectral remote sensing images according to claim 2, characterized in that: The expression of the cross entropy classification loss is: Where y ij is the true label, p ij ∈P is the predicted probability of sample i corresponding to category j; N and C represent the number of samples and the number of categories respectively.

4. The comparative learning classification method for hyperspectral remote sensing images according to claim 2, characterized in that: The specific method of the false negative example elimination strategy includes: Based on the labeled data Z N , preliminarily train the classification tower structure composed of MLP and Softmax layers to give the classification tower structure preliminary classification capabilities; Use the classification tower structure to assign pseudo labels to the unlabeled data Z M ; For anchor samples From the unlabeled data Z M Take out a set of pseudo-labeled samples F from i , identify F i The potential false negative examples with the same category as the anchor samples are removed to obtain a set of pseudo-labeled samples with different labels from the anchor samples, and this set is used as the negative sample set N for the contrast task training. i .

5. The comparative learning classification method for hyperspectral remote sensing images according to claim 4, characterized in that: The specific method of mixed mining of difficult and easy negative samples includes: Calculate anchor point samples Its negative sample set N i The spatial-spectral joint similarity of each negative sample in , the similarity score is defined as yes=and w×spe+(1-w)×spa Where, spa is the spatial similarity; spe is the spectral similarity; and are the x-coordinates of the anchor samples and negative samples, respectively. and are the y coordinates of the anchor sample and the negative sample respectively, and the parameter w is the weight of the spectral similarity; Finally, the number of negative samples K is introduced as a hyperparameter, and N is quantified according to the spatial-spectral joint similarity. i The hyperparameter K controls the hardness of negative samples, that is, the degree of difficulty in mixing negative samples, by specifying the number of negative samples to be extracted according to the similarity score.

6. The comparative learning classification method for hyperspectral remote sensing images according to claim 5, characterized in that: The InfoNCE loss is defined as follows: Where K is the number of negative samples in the queue, 1 [k≠i] ∈{0,1} is the indicator function, which is equal to 1 when k≠i. represents the cosine similarity between two feature embedding vectors, and the parameter τ represents the temperature parameter.

7. The comparative learning classification method for hyperspectral remote sensing images according to claim 1, characterized in that: The loss function obtained by weighted fusion of the classification loss of the classification task model and the contrastive loss of the contrastive learning task model is expressed as: Loss=CE loss +λ·InfoNCE loss Where λ is the weight of InfoNCE loss.

8. A comparative learning classification system for hyperspectral remote sensing images, characterized in that: include: Model building module, collaborative training module and contrastive learning classification module; The model building module is used to build a contrastive learning classification model for hyperspectral remote sensing images, wherein the contrastive learning classification model includes a classification task model of a shared encoder and a contrastive learning task model; The collaborative training module is used to collaboratively train the classification task model and the contrastive learning task model based on the original data of the hyperspectral remote sensing image until the loss function obtained by weighted fusion of the classification loss of the classification task model and the contrastive loss of the contrastive learning task model converges. During the collaborative training process, the classification task model generates pseudo labels by acting as a classifier encoder and a classification tower structure to assist the contrastive learning task model in mining negative samples. The contrastive learning task model mines negative samples based on the pseudo labels, taking into account false negative example elimination and mixed mining of difficult and easy negative samples, and optimizes the classifier parameters based on the mined negative samples. The contrastive learning classification module is used to perform contrastive learning classification of hyperspectral remote sensing images based on the trained contrastive learning classification model.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor is configured to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images