A method for remote sensing image classification based on multi-modal prompt learning and revised knowledge distillation

Abstract

This invention discloses a remote sensing image classification method based on multimodal cue learning and corrective knowledge distillation, relating to the field of multimodal remote sensing image processing. The method uses a large pre-trained base model as the teacher model, fine-tuned with a small number of labeled samples to generate pseudo-labels for unlabeled samples. Simultaneously, a multimodal cue learning module is designed to extract spectral cue vectors and sparsely mix multiple LiDAR cue vectors using a gated network. These vectors are then input into a small base model serving as the student model to fuse multimodal features. To alleviate the problem of inaccurate pseudo-labels in the teacher model, a corrective knowledge distillation strategy is proposed. Based on category similarity, a reliable subset is selected from the labeled samples, and a corrected probability distribution is reconstructed for each unlabeled sample. Finally, the student model is optimized by jointly applying the supervised loss, distillation loss, and corrective distillation loss.

Description

Technical Field

[0001] This invention relates to the field of multimodal remote sensing image processing, specifically a method for remote sensing image classification based on multimodal cue learning and corrective knowledge distillation. Background Technology

[0002] Multimodal remote sensing images are taken from ground targets using different sensors, capturing their diverse characteristics. For example, hyperspectral images contain multiple spectral bands, reflecting subtle differences between different categories of ground targets; lidar data can reflect the elevation information of ground targets. Therefore, how to extract discriminative fusion features from multimodal remote sensing images and achieve more accurate pixel-level classification has become one of the current research hotspots in the field of remote sensing.

[0003] Representative methods for multimodal remote sensing image fusion and classification include those based on handcrafted features and deep learning. Handcrafted feature-based methods typically involve two separate steps: first, multimodal feature extraction and fusion, followed by classification. However, these methods not only rely on expert knowledge but also often extract limited features focusing on only one or a few aspects, making it difficult to fully capture the inherent characteristics of ground targets. In contrast, deep learning-based methods can automatically extract high-level, abstract, and compact features from raw data with less reliance on human intervention.

[0004] The most commonly used methods in deep learning are primarily based on convolutional neural networks (CNNs) and visual transformers. Due to their inherent mechanisms of local receptive fields and weight sharing, CNN-based methods typically focus on extracting local features. For example, existing techniques have proposed a patch-to-patch CNN that utilizes multi-branch CNNs to extract multi-scale features from multi-source remote sensing images. Another existing technique utilizes a randomized multilayer network to achieve joint classification of hyperspectral images and LiDAR data; this network combines a dual-channel CNN architecture to capture spectral and spatial features from different data sources.

[0005] Compared to convolutional neural networks, visual transformer-based methods typically focus more on extracting global features. In multimodal remote sensing data processing, different types of visual lexicalization operations are first performed to obtain a series of visual lexical units. Typically, each lexical unit contains local features of the sample, including spectral features, spatial features, or spectral-spatial features. Subsequently, self-attention or cross-attention mechanisms calculate attention scores based on the similarity between different labels, thereby promoting the weighted aggregation of global features. Existing techniques also employ spectral sequence transformers to capture long-range dependencies in the spectral dimension of hyperspectral images and utilize multi-layer transformers to extract spatial features from hyperspectral images and LiDAR data. Furthermore, cross-attention mechanisms are used to fuse multimodal features. Existing techniques also propose a learnable spectral and spatial morphology network that first combines spectral and spatial morphology convolutional operations and then introduces an attention mechanism to enhance the interaction of structural and shape information between hyperspectral image labels and class image labels. Furthermore, existing techniques introduce self-guided and cross-guided attention mechanisms in each layer, allowing the model to focus on regions and channels of interest. A spectral supplementation module is also designed to obtain a more discriminative center pixel representation. In addition, existing technologies show that by leveraging the local feature extraction capabilities of convolution operators and the long-range dependency modeling advantages of transformers, local and global spectral-spatial features can be effectively extracted.

[0006] Currently, most deep learning models need to be trained from scratch for specific downstream tasks, requiring not only carefully designed network architectures but also a large number of labeled samples for support. Fortunately, in recent years, remote sensing image processing methods based on foundational models have been proposed. These foundational models are typically based on transformer architectures, employ self-supervised learning algorithms, and are trained on large-scale remote sensing datasets. Such foundational models can extract generalized features extensively and exhibit good performance in various downstream tasks, thus attracting increasing attention. For example, existing techniques address the common problem of small object information loss in traditional mask-based self-supervised learning methods, proposing an incomplete patch mask mechanism that randomly retains certain pixels during the masking process. EarthGPT is a multimodal large language model designed for comprehensive image interpretation across multiple remote sensing modalities. It effectively integrates visual and linguistic data by utilizing visual enhancement perception mechanisms, cross-modal mutual understanding methods, and unified instruction tuning methods, thereby improving performance. However, most of the aforementioned factorization machines are primarily designed for RGB images and struggle to model the relationships between different spectral bands in multispectral images. Therefore, SpectralGPT includes a 3D masking mechanism for processing spectral data, an encoder for extracting image features, and a decoder for multi-objective reconstruction. Furthermore, to enhance the model's generalization ability across different scales of image processing, a progressive pre-training strategy is introduced. Subsequently, some researchers designed a base model based on a visual transformer and applied it to hyperspectral image interpretation. This work first collected a large-scale hyperspectral dataset, then designed a novel sparse sampling attention mechanism and a spectral enhancement model to further improve the factorization machine's capabilities, thus achieving excellent performance in various hyperspectral interpretation tasks.

[0007] However, training the base model is not only time-consuming but also computationally expensive. Furthermore, repeated training is unavoidable to ensure relatively better hyperparameters. Therefore, how to fully explore the potential of the base model and achieve better performance by tuning only a small number of parameters has gradually become a popular research topic. Visual cue learning, originating from natural language processing, is called visual cue learning. Visual cue learning enables the model to generalize to specific downstream tasks by adding additional learnable visual lexical units to the basic modules of the visual transformer. Visual cue learning is an efficient parameter tuning method that keeps the parameters of the pre-trained model unchanged, with only the newly added visual lexical units having learnable parameters. This not only significantly reduces the time and computational resources required for fine-tuning but also avoids the risk of network performance degradation. Therefore, inspired by visual cue learning, a multimodal cue learning method is proposed for the joint classification of hyperspectral images and LiDAR data. First, complementary information is encoded into hyperspectral image cues and LiDAR image cues. Then, the self-attention mechanism in the visual transformer is used to achieve information interaction between multimodal remote sensing images. Furthermore, the added set of prompts places an additional burden on the base model. To address this, a selection and blending mechanism is designed based on prompts from a hybrid expert model. First, a subset of prompts is chosen from the set that is more advantageous for the downstream classification task. Then, the selected prompts are blended into a single prompt, thereby reducing computational cost and the impact of noisy pixels.

[0008] Visual cue learning typically involves designing efficient supervised learning algorithms on a limited number of labeled samples, which can risk overfitting. To address this, a knowledge distillation framework is introduced, transferring knowledge from a larger teacher-based model to a smaller student-based model by aligning the predicted probability distributions across a large number of unlabeled samples. However, the probability distribution predicted by the teacher model may be inaccurate, potentially misleading the student model's learning. Summary of the Invention

[0009] This invention provides a remote sensing image classification method based on multimodal cue learning and corrective knowledge distillation, which solves the problems mentioned in the background art.

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] A remote sensing image classification method based on multimodal cue learning and corrective knowledge distillation includes the following steps:

[0012] Step 1: Construct local regions from hyperspectral images and LiDAR data, and train the teacher base model using pre-acquired labeled samples to obtain the predicted probability distributions of labeled and unlabeled samples; during this process, the parameters of the base module remain unchanged, while the classifier parameters are learnable;

[0013] Step 2: Obtain spectral cues through the spectral cue learning module; segment the local region of the hyperspectral image into patches and map each patch to the embedding space; calculate the score of each patch using a gating network, and perform a weighted summation of the patch scores mapped to the embedding space to obtain the hybrid cue;

[0014] Step 3: Input the spectral cue and mixed cue into the basic module of the student basic model for feature extraction to obtain the prediction results of labeled and unlabeled samples; for each unlabeled sample prediction result, count the c categories with the highest frequency among the top M most similar unlabeled samples; perform a weighted summation of the prediction scores of the top c categories of the top M labeled samples to obtain a correction vector for each unlabeled sample.

[0015] Step 4: Construct the final learning objective using knowledge distillation loss, corrected knowledge distillation loss, and cross-entropy loss.

[0016] The present invention has the following advantages:

[0017] 1. Efficient parameter tuning: Through multimodal cue learning, only a small number of parameters need to be updated to transfer the knowledge of a large pre-trained base model to a specific classification task, which greatly reduces the computational cost.

[0018] 2. Unlabeled Data Utilization and Risk Mitigation: By constructing a knowledge distillation framework, pseudo-supervisory information is generated for unlabeled samples using the teacher model, thus expanding the training data. Simultaneously, a unique corrective knowledge distillation strategy effectively corrects erroneous predictions from the teacher model, preventing incorrect knowledge from misleading the student model and improving the model's robustness and accuracy.

[0019] 3. Sparse Hybrid Cueing Mechanism: A gating network is used to dynamically select and mix a few key spatial cue vectors, which not only enhances the discriminative power of feature extraction, but also avoids the computational burden and noise interference caused by introducing too many cue vectors. Attached Figure Description

[0020] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a remote sensing image classification method based on multimodal cue learning and corrective knowledge distillation. Detailed Implementation

[0022] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0023] It should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0024] For examples, please refer to Figure 1 A remote sensing image classification method based on multimodal cue learning and corrective knowledge distillation includes the following steps:

[0025] Step 1: Construct local regions from hyperspectral images and LiDAR data, and train the teacher base model using pre-acquired labeled samples to obtain the predicted probability distributions of labeled and unlabeled samples; during this process, the parameters of the base module remain unchanged, while the classifier parameters are learnable;

[0026] Given an initial hyperspectral image and lidar data Among them, determining the nearest neighbor representation of the hyperspectral image. Nearest neighbor representation of LiDAR data Local regions can be constructed by selecting multiple pixels surrounding a given pixel; generally, larger base models can learn more knowledge, thus possessing stronger feature representation capabilities; therefore, we first consider the larger base model as the teacher model and the smaller base model as the student model, determining the teacher and student models based on the size of the base models; then, we add a learnable parameter to the output layer of the teacher model. Linear classifier To obtain the predicted probability distribution, the process is as follows:

[0027] ,

[0028] In the formula, These are the parameters of the teacher model. To utilize the predicted probability distribution obtained from the teacher model, The classifier parameters corresponding to the teacher model remain unchanged during subsequent training. Represent the features corresponding to the hyperspectral image; minimize the cross-entropy. To optimize :

[0029] ,

[0030] In the formula, Represents the number of labeled samples. The true labels representing the training samples, Represent the optimal classifier parameters; calculate the probability distribution predicted by the teacher model for unlabeled samples:

[0031] ,

[0032] In the formula, Test samples representing hyperspectral images.

[0033] Step 2: Obtain spectral cues through the spectral cue learning module; segment the local region of the hyperspectral image into patches and map each patch to the embedding space; calculate the score of each patch using a gating network, and perform a weighted summation of the patch scores mapped to the embedding space to obtain the hybrid cue;

[0034] The probability distribution predicted by the unlabeled samples in the teacher model is obtained; the model's performance on specific downstream tasks is improved by adding a small number of learnable parameters (cue vectors) to the input space; for hyperspectral images, cue vectors are obtained through a cue learning module, which can be a multilayer perceptron or a convolutional neural network; to facilitate the acquisition of multiple cue vectors, a spectral cue learner is constructed using a one-dimensional convolutional network, and the spectral cue vector corresponding to the spectral cue learner is calculated. :

[0035] ,

[0036] In the formula, Represents the spectral features corresponding to each pixel. These are the learnable parameters of the spectral cue learner; for LiDAR data, the cue vector should focus on extracting spatial features; therefore, a LiDAR cue learner is constructed using a two-dimensional convolutional neural network, and the cue vector corresponding to the LiDAR cue learner is calculated. :

[0037] ,

[0038] In the formula, These are the parameters of the LiDAR cue learner. Specifically, a 2D convolution with equal width is chosen to achieve dimensionality consistency in the spatial domain; the cue vectors corresponding to multiple LiDAR cue learners are weighted and aggregated to form a single cue vector, which... The cue vector corresponding to each LiDAR cue learner is considered as... Each expert carries a spatial feature corresponding to one pixel; a gating mechanism is used to obtain the activation value corresponding to each expert.

[0039] ,

[0040] In the formula, , For gating networks; for Sort the values ​​and take the first few from largest to smallest. indivual The value is used as the activation value, and the rest... Value set to :

[0041] ,

[0042] In the formula, For the first The labeled sample pair of the th _th The expert's activation value This indicates taking the first option from largest to smallest. indivual value;

[0043] Calculate the probability of each pixel being selected:

[0044] ,

[0045] In the formula, This represents the softmax function;

[0046] Choose according to their respective probabilities of being selected. Blending of pixels:

[0047] ,

[0048] In the formula, Represents a mixed cue vector. For the first The sample corresponds to the first The probability of the expert's choice, the... The sample at the th The prompt vector is generated by an expert.

[0049] Step 3: Input the spectral cue and mixed cue into the basic module of the student basic model for feature extraction to obtain the prediction results of labeled and unlabeled samples; for each unlabeled sample prediction result, count the c categories with the highest frequency among the top M most similar unlabeled samples; perform a weighted summation of the prediction scores of the top c categories of the top M labeled samples to obtain a correction vector for each unlabeled sample.

[0050] The cue vectors from the hyperspectral image HSI and the LiDAR data are obtained; these cue vectors are then input into the student's basic model for feature extraction, yielding multimodal features:

[0051] ,

[0052] In the formula, These represent the parameters of the basic modules of the student model. and These are the multimodal features corresponding to labeled and unlabeled samples, respectively;

[0053] ,

[0054] In the formula, Indicates the first Each transformer module extracts features from labeled samples. Representative level unification; when hour, ; Representing multi-head self-attention, calculate the output of each self-attention head:

[0055] ,

[0056] In the formula, , and These represent the query, key, and value in the basic module, respectively, and can be viewed through three matrices. Obtained by performing a linear mapping; This represents the total number of labeled and unlabeled samples. Indicates the number of labeled samples. Indicates the number of unlabeled samples; This represents the dimension of each head embedding feature; Indicates the number of attention heads. Indicates the initial embedding dimension;

[0057] The outputs of each self-attention head are combined to obtain the output of multi-head self-attention:

[0058] ,

[0059] In the formula, Used to restore the combined features to the original embedding dimension;

[0060] Add a linear classifier to the output of the base model:

[0061] ,

[0062] In the formula, This represents the predicted probability of all training samples on the student model. These represent the corresponding parameters of the student model classifier;

[0063] For the prediction results of labeled samples, the traditional supervised classification loss is used to measure the empirical loss of the student model:

[0064] ,

[0065] In the formula, This represents the loss for labeled samples; the probability distribution predicted by the teacher model is treated as pseudo-supervision information for unlabeled samples, and Kullback-Leibler divergence is used. Loss measures the probability of the student model predicting unlabeled samples. The predicted probability of the teacher model for unlabeled samples Differences between them:

[0066] ,

[0067] Since the teacher model may still make incorrect predictions, the predicted probability distribution corresponding to the labeled samples is reconstructed into an additional approximation target, and only a subset is selected to achieve the reconstruction; first, the predicted probability of the i-th unlabeled sample is calculated. and the predicted probability of labeled samples Similarities between them:

[0068] ,

[0069] In the formula, Represents the similarity between the i-th unlabeled sample and all training samples; from Select A candidate set is formed by selecting the labeled samples that are most similar to the test sample. :

[0070] ,

[0071] In the formula, choose Find the maximum value and pass the candidate set. Form a subset of candidate scores In the candidate set In this model, all training scores and test scores have high similarity, but they may come from multiple categories; therefore, the predicted probability of the student model for the i-th unlabeled sample... The number of occurrences of each category is counted. ,in , For indicator functions, Let j be the predicted probability of the j-th labeled sample; in The category that appears most frequently is determined in the data. Select the corresponding training scores to form the final candidate set. :

[0072] ,

[0073] Reconstruction Goals Obtained by weighted summation:

[0074] ,

[0075] In the formula, Representative candidate set Correlation with the i-th prediction result;

[0076] Step 4: Construct the final learning objective using knowledge distillation loss, corrected knowledge distillation loss, and cross-entropy loss.

[0077] The total loss includes cross-entropy loss. Distillation loss and corrected distillation loss The process can be represented as:

[0078] ,

[0079] In the formula, It is the balance coefficient.

[0080] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for remote sensing image classification based on multimodal cue learning and corrective knowledge distillation, characterized in that, Includes the following steps: Step 1: Construct local regions from hyperspectral images and LiDAR data, and train the teacher base model using pre-acquired labeled samples to obtain the predicted probability distributions of labeled and unlabeled samples; during this process, the parameters of the base module remain unchanged, while the classifier parameters are learnable; Step 2: Obtain spectral cues through the spectral cue learning module; segment the local region of the hyperspectral image into patches and map each patch to the embedding space; calculate the score of each patch using a gating network, and perform a weighted summation of the patch scores mapped to the embedding space to obtain the hybrid cue; Step 3: Input the spectral cue and mixed cue into the basic module of the student basic model for feature extraction to obtain the prediction results of labeled and unlabeled samples; for each unlabeled sample prediction result, count the c categories with the highest frequency among the top M most similar unlabeled samples; perform a weighted summation of the prediction scores of the top c categories of the top M labeled samples to obtain a correction vector for each unlabeled sample. Step 4: Construct the final learning objective using knowledge distillation loss, corrected knowledge distillation loss, and cross-entropy loss.

2. The remote sensing image classification method based on multimodal cue learning and corrective knowledge distillation according to claim 1, characterized in that, Step 1 specifically includes: Given an initial hyperspectral image and lidar data Among them, determining the nearest neighbor representation of the hyperspectral image. Nearest neighbor representation of LiDAR data Local regions can be constructed by selecting multiple pixels surrounding a given pixel; generally, larger base models can learn more knowledge, thus possessing stronger feature representation capabilities; therefore, we first consider the larger base model as the teacher model and the smaller base model as the student model, determining the teacher and student models based on the size of the base models; then, we add a learnable parameter to the output layer of the teacher model. Linear classifier To obtain the predicted probability distribution, the process is as follows: ， In the formula, These are the parameters of the teacher model. To utilize the predicted probability distribution obtained from the teacher model, The classifier parameters corresponding to the teacher model remain unchanged during subsequent training. Represent the features corresponding to the hyperspectral image; minimize the cross-entropy. To optimize : ， In the formula, Represents the number of labeled samples. This represents the true label of the i-th training sample. Represent the optimal classifier parameters; calculate the probability distribution predicted by the teacher model for unlabeled samples: ， In the formula, Test samples representing hyperspectral images.

3. The remote sensing image classification method based on multimodal cue learning and corrective knowledge distillation according to claim 2, characterized in that, Step 2 specifically includes: Spectral cues are obtained through a spectral cue learning module; local regions are segmented into patches, and each patch is mapped to an embedding space; a gating network is used to calculate the score of each patch, and the scores of the patches mapped to the embedding space are weighted and summed to obtain hybrid cues, including: The probability distribution predicted by the unlabeled samples in the teacher model is obtained; the model's performance on specific downstream tasks is improved by adding a small number of learnable parameters (cue vectors) to the input space; for hyperspectral images, cue vectors are obtained through a cue learning module, which can be a multilayer perceptron or a convolutional neural network; to facilitate the acquisition of multiple cue vectors, a spectral cue learner is constructed using a one-dimensional convolutional network, and the spectral cue vector corresponding to the spectral cue learner is calculated. : ， In the formula, Represents the spectral features corresponding to each pixel. These are the learnable parameters of the spectral cue learner; for LiDAR data, the cue vector should focus on extracting spatial features; therefore, a LiDAR cue learner is constructed using a two-dimensional convolutional neural network, and the cue vector corresponding to the LiDAR cue learner is calculated. : ， In the formula, These are the parameters of the LiDAR cue learner. Specifically, a 2D convolution with equal width is chosen to achieve dimensionality consistency in the spatial domain; the cue vectors corresponding to multiple LiDAR cue learners are weighted and aggregated to form a single cue vector, which... The cue vector corresponding to each LiDAR cue learner is considered as... Each expert carries a spatial feature corresponding to one pixel; a gating mechanism is used to obtain the activation value corresponding to each expert. ， In the formula, , For gating networks; for Sort the values ​​and take the first few from largest to smallest. indivual The value is used as the activation value, and the rest... Value set to : ， In the formula, For the first The labeled sample pair of the th _th The expert's activation value This indicates taking the first option from largest to smallest. indivual value; Calculate the probability of each pixel being selected: ， In the formula, This represents the softmax function; Choose according to their respective probabilities of being selected. Blending of pixels: ， In the formula, Represents a mixed cue vector. For the first The sample corresponds to the first The probability of the expert's choice, the... The sample at the th The prompt vector is generated by an expert.

4. The remote sensing image classification method based on multimodal cue learning and corrective knowledge distillation according to claim 3, characterized in that, Step 3 specifically includes: Spectral cues and mixed cues are input into the base module of the student's basic model for feature extraction, obtaining the predicted probability distributions of labeled and unlabeled samples. For each unlabeled sample's output probability distribution, the top M most similar labeled samples are selected, and the c most frequent categories among these top M most similar unlabeled samples are statistically analyzed. The predicted scores of the top c categories of the top M labeled samples are weighted and summed to obtain a correction vector for each unlabeled sample, including: The cue vectors from the hyperspectral image HSI and the LiDAR data are obtained; these cue vectors are then input into the student's basic model for feature extraction, yielding multimodal features: ， In the formula, These represent the parameters of the basic modules of the student model. and These are the multimodal features corresponding to labeled and unlabeled samples, respectively; ， In the formula, Indicates the first Each transformer module extracts features from labeled samples. Representative level unification; when hour, ; Representing multi-head self-attention, calculate the output of each self-attention head: ， In the formula, , and These represent the query, key, and value in the basic module, respectively, and can be viewed through three matrices. Obtained by performing a linear mapping; This represents the total number of labeled and unlabeled samples. Indicates the number of labeled samples. Indicates the number of unlabeled samples; This represents the dimension of each head embedding feature; Indicates the number of attention heads. Indicates the initial embedding dimension; The outputs of each self-attention head are combined to obtain the output of multi-head self-attention: ， In the formula, Used to restore the combined features to the original embedding dimension; Add a linear classifier to the output of the base model: ， In the formula, This represents the predicted probability of all training samples on the student model. These represent the corresponding parameters of the student model classifier; For the prediction results of labeled samples, the traditional supervised classification loss is used to measure the empirical loss of the student model: ， In the formula, This represents the loss for labeled samples; the probability distribution predicted by the teacher model is treated as pseudo-supervision information for unlabeled samples, and Kullback-Leibler divergence is used. Loss measures the probability of the student model predicting unlabeled samples. The predicted probability of the teacher model for unlabeled samples Differences between them: ， Since the teacher model may still make incorrect predictions, the predicted probability distribution corresponding to the labeled samples is reconstructed into an additional approximation target, and only a subset is selected to achieve the reconstruction; first, the predicted probability of the i-th unlabeled sample is calculated. and the predicted probability of labeled samples Similarities between them: ， In the formula, Represents the similarity between the i-th unlabeled sample and all training samples; from Select A candidate set is formed by selecting the labeled samples that are most similar to the test sample. : ， In the formula, choose Find the maximum value and pass the candidate set. Form a subset of candidate scores In the candidate set In this model, all training scores and test scores have high similarity, but they may come from multiple categories; therefore, the predicted probability of the student model for the i-th unlabeled sample... The number of occurrences of each category is counted. ,in , For indicator functions, Let be the predicted probability of the j-th labeled sample; in The category that appears most frequently is determined in the data. Select the corresponding training scores to form the final candidate set. : ， Reconstruction Goals Obtained by weighted summation: ， In the formula, Representative candidate set The correlation with the i-th prediction result.

5. The remote sensing image classification method based on multimodal cue learning and corrective knowledge distillation according to claim 4, characterized in that, Step 4 specifically includes: The Kullback-Leibler divergence between the output scores of the teacher and student models, and the Kullback-Leibler divergence between the student model output and the modified vector, were obtained; the total loss includes cross-entropy loss. Distillation loss and corrected distillation loss : ， In the formula, It is the balance coefficient.

Images