A cross-domain small sample classification method for remote sensing images based on confidence dynamic learning

Abstract

The present invention discloses a cross-domain small-sample classification method for remote sensing images based on confidence dynamic learning, comprising the following steps: obtaining remote sensing image data of a source domain and a target domain; pre-training a deep learning network model using the source domain remote sensing image data to obtain pre-training initialization parameters; constructing a semi-supervised cross-domain collaborative training model and loading the initialization parameters; performing supervised learning based on the source domain remote sensing image data; processing the semi-supervised cross-domain collaborative training model; processing a confidence dynamic learning strategy; updating the initialization parameters based on a total loss function to obtain a deep learning network model for small-sample classification; inputting the remote sensing image data to be tested into the deep learning network model for small-sample classification to obtain corresponding classification results. The present invention overcomes the current shortcomings of insufficient utilization of target domain data information and uncertainty in deep learning models that impair cross-domain learning, improves the generalization ability of deep learning models, and enhances the performance of cross-domain small-sample classification of remote sensing images.

Description

Technical Field

[0001] The present invention relates to the technical field of cross-domain small-sample classification of remote sensing images, and in particular to a cross-domain small-sample classification method for remote sensing images with dynamic confidence learning. Background Art

[0002] Remote sensing image classification is a key issue in remote sensing technology. It aims to leverage computer vision algorithms to extract rich semantic information from remote sensing images and accurately assign semantic labels to them. This approach supports intelligent remote sensing data interpretation tasks such as urban planning, land statistics analysis, and precise target identification. The rise of deep learning models has brought significant breakthroughs in remote sensing image classification. However, most current deep learning-based methods assume that the training and test sets are sampled from the same data distribution, an assumption that is difficult to achieve in practical applications. In real-world applications, the data distribution of remote sensing images is influenced by numerous factors, such as sensor type, illumination angle, and shooting position, resulting in significant differences in imaging, resolution, and illumination. Furthermore, different remote sensing datasets have varying applications and contain varying categories, further exacerbating data domain shift. Current supervised learning deep models have limited generalization capabilities when faced with domain shift and limited labeled samples in real-world open environments, resulting in low classification accuracy and making remote sensing image data classification difficult to meet practical application requirements. Therefore, how to enhance the generalization ability of the model by using existing labeled datasets for training and effectively transferring the learned knowledge so that it can adapt to classification tasks with different data distributions and categories in open environments with few annotations and across domains has become a major challenge in the field of remote sensing image classification.

[0003] Given these issues, it is necessary to consider the domain shift caused by distributional and category differences in remote sensing image data in order to perform cross-domain small-shot classification. Specifically, the model learns knowledge from the source domain, then transfers and generalizes this knowledge to the target domain for learning. Adaptation is then performed using a small number of labeled training samples from the target domain, enabling the model to achieve good small-shot classification performance in the target domain. Existing cross-domain small-shot classification methods are primarily based on meta-learning or transfer learning paradigms, with the latter attracting attention for its superior performance. Current state-of-the-art transfer learning methods utilize unlabeled partial target domain data as additional data resources, performing self-supervised learning through pseudo-labeling and consistency regularization, combined with supervised learning from labeled source domain data, achieving good cross-domain small-shot classification performance. However, due to data domain shift, the output of deep learning models for target domain data samples is uncertain, resulting in unreliable cross-domain knowledge transfer and severely impairing the model's cross-domain small-shot classification capabilities. Summary of the Invention

[0004] In order to solve the above problems, the present invention provides a cross-domain small sample classification method for remote sensing scenes based on dynamic confidence learning, which can alleviate the negative impact of high uncertainty in the existing technology on cross-domain learning of models and improve the performance of deep learning methods for cross-domain small sample classification of remote sensing images.

[0005] In order to achieve the purpose of the present invention, the following technical solutions are adopted:

[0006] The present invention provides a cross-domain small sample classification method for remote sensing images based on confidence dynamic learning, comprising the following steps: S101: acquiring source domain remote sensing image data and target domain remote sensing image data; wherein the source domain remote sensing image data carries a label, and the target domain remote sensing image data does not carry a label; the label indicates the category of the remote sensing image data; S102: pre-training a deep learning network model using the source domain remote sensing image data to obtain pre-training initialization parameters of the deep learning network model; S103: constructing a semi-supervised cross-domain collaborative training model for cross-domain feature representation learning and loading the pre-training initialization parameters; wherein the semi-supervised cross-domain collaborative training model includes a first model and a second model; S104: inputting the source domain remote sensing image data into the first model Perform supervised learning to enable the first model to have the ability to extract general feature information; S105: perform semi-supervised cross-domain collaborative training model processing on the target domain remote sensing image data to obtain a first prediction probability and a second prediction probability; S106: perform confidence dynamic learning strategy processing on the first prediction probability and the second prediction probability; S107: construct a total loss function and update the pre-training initialization parameters based on the total loss function to obtain a deep learning network model for small sample classification; S108: the remote sensing image data to be tested is input into the deep learning network model for small sample classification to obtain a classification result of the remote sensing image data to be tested, and the classification result indicates the category of the remote sensing image data to be tested; wherein the remote sensing image data to be tested partially carries a label.

[0007] Furthermore, constructing a semi-supervised cross-domain collaborative training model for cross-domain feature representation learning and loading the pre-training initialization parameters includes: loading the pre-training initialization parameters into the first model and the second model at the same time, wherein the first model includes a first feature extractor and a first classifier, and the second model includes a second feature extractor and a second classifier.

[0008] Furthermore, the step of inputting the source domain remote sensing image data into the first model for supervised learning so that the first model has a general feature information extraction capability includes: S104.1: extracting features from the source domain remote sensing image data to obtain source domain features. in, is the source domain remote sensing image data, f1 is the first feature extractor, s represents the source domain, and i is the source domain sample index; S104.2: classify and predict the source domain features to obtain the source domain classification prediction probability Wherein, c1 is the first classifier, softmax(·) is the normalized exponential function; S104.3: Based on the source domain classification prediction probability and the label of the source domain remote sensing image data, the first model is supervised learning using the source domain supervised loss, and the source domain supervised loss L S The expression is as follows:

[0009]

[0010] in, is the cross entropy loss function, is the source domain remote sensing image data label, N S is the number of source domain remote sensing image data; the general feature information at least includes mid- and low-level feature information.

[0011] Furthermore, the processing of the target domain remote sensing image data by the semi-supervised cross-domain collaborative training model includes: S105.1: performing enhancement processing on the target domain remote sensing image data to obtain a strongly enhanced data sample and a weakly enhanced data sample; S105.2: processing the strongly enhanced data sample by a first feature extractor to obtain a first feature map in, is the strongly enhanced data sample, f1 is the first feature extractor, ts is the target domain strong enhancement, and j is the target domain sample index; S105.3: Process the first feature map through the first classifier to obtain a first prediction probability Where c1 is the first classifier; S105.4: the weakly enhanced data sample is processed by the second feature extractor to obtain a second feature map in, is the weakly enhanced data sample, f2 is the second feature extractor, and tw is the target domain weak enhancement; S105.5: the second feature map is processed by the second classifier to obtain a second prediction probability Where c2 is the second classifier.

[0012] Furthermore, the target domain remote sensing image data is enhanced, including weak enhancement processing and strong enhancement processing: wherein, the weak enhancement processing includes: S105.1.11: randomly cropping the target domain remote sensing image data to obtain a first cropped image; S105.1.12: horizontally flipping the first cropped image to obtain a first flipped image; S105.1.13: normalizing the first flipped image to obtain a weakly enhanced image data sample. The strong enhancement processing includes: S105.1.21: randomly cropping the target domain remote sensing image data to obtain a second cropped image; S105.1.22: horizontally flipping the second cropped image to obtain a second flipped image; S105.1.23: color dithering the second flipped image to obtain a color-changed image; S105.1.24: Gaussian blurring the color-changed image to obtain a blurred image; S105.1.25: random grayscale transformation processing the blurred image to obtain a random grayscale image; S105.1.26: normalizing the random grayscale image to obtain a strongly enhanced image data sample.

[0013] Furthermore, the confidence dynamic learning strategy processing of the first prediction probability and the second prediction probability includes: S106.1: performing sample division processing on the target domain remote sensing image data based on the second prediction probability to obtain a positive sample set and a negative sample set; the positive sample set includes positive samples, positive pseudo labels and positive sample probabilities, and the negative sample set includes negative samples, negative pseudo labels and negative sample probabilities; S106.2: performing positive sample learning processing and negative sample learning processing on the positive sample set and the negative sample set respectively based on the first prediction probability; S106.3: performing entropy minimization learning processing based on the first prediction probability.

[0014] Furthermore, the sample division processing of the target domain remote sensing image data based on the second prediction probability includes: S106.1.1: calculating a confidence dynamic threshold according to the second prediction probability, and the confidence dynamic threshold t is calculated as follows: t k =(1-α)t k-1 +αρ k , where t k is the dynamic confidence threshold under k rounds, ρ k is the average confidence level of the second prediction probability of all samples in the current round. α is a hyperparameter with an initial value of 1 and decreases with the increase of rounds. The update formula of α is: α = α - (1 / K)k, K is the total number of training rounds, and k is the current round number. S106.1.2: Sharpen the second prediction probability to obtain the pseudo label p j ′: in, is the second feature map, τ is the temperature parameter used to sharpen the second prediction probability, and c2 is the second classifier; S106.1.3: Based on the confidence dynamic threshold, use a positive and negative sample selection function to perform positive and negative sample division to obtain a positive sample set and a negative sample set, wherein the positive and negative sample selection function is expressed as follows: Among them, P is the positive sample, N is the negative sample, j is the sample index, Select a function for positive samples, Select a function for negative samples, select a function is a binary vector, when g j =1 means the sample is selected. j =0 means that the sample is not selected, ρ j is the second predicted probability of sample j The confidence level,

[0015] Furthermore, the performing positive sample learning processing and negative sample learning processing on the positive sample set and the negative sample set respectively includes: wherein the positive sample learning processing includes: performing positive sample learning processing based on the positive pseudo labels and the positive sample probability through a positive sample loss function, and the positive sample loss function expression is as follows: in, Select function for positive samples, p j ′ is a pseudo label, is the first predicted probability, is the number of positive samples of the target domain remote sensing image data, and j is the index of the target domain remote sensing image data. The negative sample learning process includes: performing negative sample learning process based on the negative pseudo label and the negative sample probability through a negative sample loss function, and the negative sample loss function expression is as follows: in, Select a function for negative samples, is the number of negative samples of remote sensing image data in the target domain.

[0016] Furthermore, performing entropy minimization learning processing based on the first predicted probability includes: performing entropy minimization learning processing on the first predicted probability through an entropy minimization loss function, wherein the entropy minimization loss function is expressed as follows: in, is the first predicted probability, N TU is the number of remote sensing image data in the target domain, and j is the sample index of the target domain.

[0017] Furthermore, the total loss function is obtained by combining supervised learning of labeled remote sensing image data in the source domain with the confidence dynamic learning strategy, and is expressed as follows:

[0018] L=L S +λ(L P +L N )+L SE ,

[0019] in,

[0020]

[0021]

[0022]

[0023] L S is the source domain cross entropy loss function, L P is the target domain positive sample consistency regularization loss function, L N is the target domain negative sample consistency regularization loss function, L SE is the entropy minimization loss function, and λ is the hyperparameter used to adjust the weight of the loss function is the label corresponding to the image sample, Output prediction probability for the classifier, N S is the number of source domain samples, Select function for positive samples, p j ′ is a pseudo label, is the first predicted probability, is the number of positive samples of remote sensing image training data in the target domain, Select function for negative samples, p j ′ is a pseudo label, is the number of negative samples of the target domain remote sensing image training data, N TU is the number of target domain remote sensing image training data samples, and j is the target domain sample index.

[0024] This paper addresses the existing problem of uncertainty in cross-domain learning, which leads to poor reliability in cross-domain knowledge transfer. We propose a cross-domain small-sample classification method for remote sensing images using dynamic confidence learning. First, a deep learning network model is pre-trained using source domain samples. Then, using semi-supervised cross-domain collaborative training, the model is trained simultaneously using source and target domain remote sensing image data to achieve knowledge transfer from the source domain to the target domain. Cross-domain learning of the deep learning network model is then combined with a dynamic confidence learning strategy. Finally, by fine-tuning the classifier, high-performance small-sample classification is achieved in the target domain.

[0025] It should be understood that the contents described in the summary of the present invention are not intended to limit the key features or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0027] Figure 1 Schematic diagram of the method flow of the present invention;

[0028] Figure 2 Schematic diagram of the method framework of the present invention;

[0029] Figure 3 Schematic diagram of the confidence dynamic strategy processing flow of the present invention. DETAILED DESCRIPTION

[0030] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0031] In the description of the embodiments of the present invention, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to." The term "based on" should be understood as "based, at least in part, on." The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment." The term "some embodiments" should be understood as "at least some embodiments." Other explicit and implicit definitions may be included below.

[0032] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and description are to be regarded as illustrative in nature and not restrictive. There is no requirement for the order in which the method steps are described; any order that can be implemented is within the scope of the present invention.

[0033] like Figure 1 and Figure 2 As shown, the present invention provides a cross-domain small sample classification method for remote sensing scenes based on dynamic confidence learning, comprising:

[0034] S101: Acquire source domain remote sensing image data and target domain remote sensing image data.

[0035] The source domain remote sensing image data carries labels, and the target domain remote sensing image data does not carry labels. The labels indicate the categories of the remote sensing image data, such as scene semantic categories, target model categories, etc. The content indicated by the labels can be limited according to actual needs and is not limited here.

[0036] There are domain differences between source and target domain remote sensing image data, including differences in data distribution and category differences. Data distribution differences primarily manifest in differences in color, resolution, and imaging characteristics of acquired images due to differences in remote sensing platforms. Category differences manifest in the presence of remote sensing image data with at least some or all categories differing between the two datasets. This invention addresses the difficulties in classifying and predicting images with domain differences and effectively improves the accuracy of classification results for these images through its solution.

[0037] S102: Pre-training a deep learning network model using the source domain remote sensing image data to obtain pre-training initialization parameters of the deep learning network model.

[0038] The deep learning network model includes a feature extractor and a classifier.

[0039] In some embodiments, a deep learning network model is pre-trained using labeled source domain remote sensing image data under a supervised learning paradigm to obtain pre-trained initialization parameters for the deep learning network model. Supervised learning refers to training the model using a large amount of known labeled remote sensing image data, and then performing classification prediction on test image data with unknown labels. Pre-training the deep learning network model primarily includes the following steps:

[0040] S102.1: Input the source domain remote sensing image data into the deep learning backbone network model, perform feature extraction on the source domain remote sensing image data, and obtain feature results in, is the source domain remote sensing image data, i is the source domain remote sensing image data index, and f is the feature extractor;

[0041] S102.2: Perform classification prediction on the feature results to obtain classification prediction probability:

[0042]

[0043] in, is the source domain remote sensing image sample, i is the source domain sample index, c is the classifier, and softmax(·) is the normalized exponential function;

[0044] S102.3: Under a supervised learning paradigm, based on the classification prediction probability and the label of the source domain remote sensing image data, a cross-entropy loss function is used to optimize the deep learning backbone network model. The cross-entropy loss function is as follows:

[0045]

[0046] in, is the label of the source domain remote sensing image data, N S is the number of source domain remote sensing image data, and s represents the source domain.

[0047] S102.4: Parameter optimization is performed using the cross entropy loss function to obtain pre-training initialization parameters for the deep learning network model.

[0048] S103: Construct a semi-supervised cross-domain collaborative training model for cross-domain feature representation learning and load pre-training initialization parameters.

[0049] The semi-supervised cross-domain collaborative training model includes a first model and a second model. The first model and the second model use the deep learning network model structure. The first model includes a first feature extractor and a first classifier, and the second model includes a second feature extractor and a second classifier.

[0050] In some embodiments, under a semi-supervised learning paradigm, a semi-supervised cross-domain collaborative training model is constructed using both labeled source domain remote sensing image data and unlabeled target domain remote sensing image data to train a deep learning network model. The initialization parameters are loaded into the first model and the second model. The first model updates its model parameters using gradient descent, while the second model updates its parameters using exponential moving average.

[0051] The exponential sliding average parameter update expression is as follows:

[0052] θ2=mθ1+(1-m)θ2

[0053] Among them, θ2 is the second model parameter, θ1 is the first model parameter, and m is the momentum hyperparameter. The empirical value of the hyperparameter can be m=0.99, and other appropriate parameters can also be selected according to actual conditions, which is not limited here.

[0054] Gradient descent updates the model parameters by calculating the loss function, determining the gradient of the loss function, and backpropagating to update the model parameters. The loss function here can be implemented using any calculation method suitable for this situation in the existing technology and is not limited here.

[0055] S104: Inputting the source domain remote sensing image data into the first model for supervised learning, so that the first model has the ability to extract general feature information.

[0056] The general feature information at least includes mid- and low-level feature information, such as mid- and low-level texture, color, and / or edge information.

[0057] For the labeled source domain remote sensing image data, it is input into the first model. Through supervised learning, the model synchronously learns the source domain knowledge, establishes knowledge transfer between the source domain and the target domain, and ensures the convergence of model training. The specific steps are as follows:

[0058] S104.1: Extract features from the source domain remote sensing image data to obtain source domain features

[0059]

[0060] in, is the source domain remote sensing image sample, f1 is the first feature extractor, s represents the source domain, and i is the source domain sample index;

[0061] S104.2: Perform classification prediction on the source domain features to obtain source domain classification prediction probability

[0062]

[0063] Where c1 is the first classifier and softmax(·) is the normalized exponential function;

[0064] S104.3: Based on the source domain classification prediction probability and the label of the source domain remote sensing image data, the first model is supervised learning using the source domain supervised loss, wherein the source domain supervised loss L S The expression is as follows:

[0065]

[0066] in, is the cross entropy loss function, is the corresponding label of the source domain image data sample, N S S105: Process the target domain remote sensing image data using the semi-supervised cross-domain collaborative training model to obtain a first prediction probability and a second prediction probability.

[0067] Figure 2 As shown in the (a) semi-supervised cross-domain collaborative training model section, this step can be implemented as follows:

[0068] S105.1: Perform enhancement processing on the target domain remote sensing image data to obtain strongly enhanced data samples and weakly enhanced data samples.

[0069] S105.2: Process the strongly enhanced data through a first feature extractor to obtain a first feature map.

[0070] S105.3: Process the first feature map through a first classifier to obtain a first prediction probability.

[0071] S105.4: The weakly enhanced data is processed by a second feature extractor to obtain a second feature map.

[0072] S105.5: Process the second feature map through a second classifier to obtain a second prediction probability.

[0073] The enhancement processing mentioned in S105.1 is described in detail below. The enhancement processing refers to performing weak data enhancement processing and strong data enhancement processing on the first target domain remote sensing image data to obtain weakly enhanced data and strongly enhanced data, respectively. Weak data enhancement only performs a simple transformation on the image without changing the image properties, while strong data enhancement is based on the simple transformation and also performs changes to the image properties such as color and clarity.

[0074] The weak enhancement processing includes: randomly cropping the first target domain remote sensing image data to obtain a first cropped image; horizontally flipping the cropped image to obtain a first flipped image; and normalizing the flipped image to obtain first normalized image data.

[0075] The strong enhancement processing includes: randomly cropping the target domain remote sensing image data to obtain a second cropped image; horizontally flipping the second cropped image to obtain a second flipped image; color dithering the second flipped image to obtain a color-changed image; Gaussian blurring the color-changed image to obtain a blurred image; random grayscale transformation processing the blurred image to obtain a random grayscale image; and normalizing the random grayscale image to obtain a strongly enhanced image data sample.

[0076] S105.2 and S105.3 mainly process the strongly enhanced data to obtain a first prediction probability. This step can be implemented in the first model, specifically by the following method: the strongly enhanced data is processed by the first feature extractor to obtain a first feature map The first feature map is processed by a first classifier to obtain a first prediction probability in, is the strong enhancement data sample, f1 is the first feature extractor, ts represents the target domain strong enhancement, and j is the target domain sample index; is the first feature map, c1 is the first classifier, and softmax(·) is the normalized exponential function.

[0077] S105.3 and S105.4 mainly process the weakly enhanced data to obtain a second prediction probability. This step can be implemented in the second model, specifically by the following method: the weakly enhanced data is processed by the second feature extractor to obtain a second feature map. The second feature map is processed by the second classifier to obtain a second prediction probability in, is the weakly enhanced data sample, f2 is the second feature extractor, tw represents the weak enhancement of the target domain, j is the index of the target domain remote sensing image data, c2 is the second classifier, and softmax(·) is the normalized exponential function.

[0078] S106: Perform confidence dynamic learning strategy processing on the first prediction probability and the second prediction probability. Figure 3 .

[0079] Figure 2 As shown in the (b) Confidence Dynamic Learning Strategy section, this step can be implemented as follows:

[0080] S106.1: Based on the second predicted probability, sample partitioning processing is performed on the target domain remote sensing image data to obtain a positive sample set and a negative sample set; the positive sample set includes positive samples, positive pseudo labels and positive sample probabilities, and the negative sample set includes negative samples, negative pseudo labels and negative sample probabilities.

[0081] The sample partitioning process can be implemented in the following ways:

[0082] S106.1.1: Calculate a dynamic confidence threshold based on the second predicted probability. The calculation process of the dynamic confidence threshold is as follows:

[0083] a) Divide the target domain remote sensing image data into different training batches and calculate the confidence of the second prediction probability of each sample in these training batches at a time:

[0084] ρ j =max(p j ′),

[0085] Among them, p j ′ is a pseudo label, max(·) is the maximum value operation; small batch.

[0086] b) Calculate the average confidence of the second prediction probability of samples in the current training mini-batch:

[0087]

[0088] Among them, B is the total number of samples in the training mini-batch, and b is the current training mini-batch.

[0089] c) Calculate the overall confidence of the second prediction probability of samples in the current training round:

[0090]

[0091] Among them, k is the current training round, is the number of training mini-batches in the current training round, N TU is the total number of remote sensing image data in the target domain.

[0092] d) Calculating a dynamic confidence threshold based on the overall confidence of the second prediction probability of the samples in the current training round, wherein the dynamic confidence threshold calculation formula is as follows:

[0093] t k =(1-α)t k-1 +αρ k

[0094] Among them, t k is the dynamic confidence threshold ρ under k rounds k is the average confidence level of the second prediction probability of all samples in the current round. α is a hyperparameter with an initial value of 1 and decreases with the increase of rounds. The α update formula is: α = α - (1 / K)k, where K is the total number of training rounds and k is the value of the current round.

[0095] S106.1.2: Sharpen the second predicted probability to obtain a pseudo label p j ′:

[0096]

[0097] in, is the second feature map, τ is the temperature parameter used to sharpen the second prediction probability, and c2 is the second classifier;

[0098] S106.1.3: Based on the confidence dynamic threshold, use a positive-negative sample selection function to perform positive-negative division to obtain a positive sample set and a negative sample set, wherein the positive-negative sample selection function is expressed as follows:

[0099]

[0100] Among them, P is the positive sample, N is the negative sample, j is the sample index, Select a function for positive samples, Select a function for negative samples, select a function is a binary vector, when g j =1 means the sample is selected. j =0 means that the sample is not selected, ρ j is the confidence level of the second predicted probability of the sample, ρ j =max(p j ′).

[0101] S106.2: Perform positive sample learning processing and negative sample learning processing on the positive sample and the negative sample respectively based on the first prediction probability.

[0102] The following introduces the positive sample learning process and the negative sample learning process respectively.

[0103] The positive sample learning process can be implemented in the following manner: based on the positive pseudo labels and the positive sample probability, the positive sample learning process is performed using a positive sample loss function, and the positive sample loss function expression is as follows:

[0104]

[0105] in, Select function for positive samples, p j ′ is a pseudo label, is the first predicted probability, is the number of positive samples of the target domain remote sensing image data, TU represents the unlabeled target domain, and j is the index of the target domain remote sensing image data.

[0106] The negative sample learning process can be implemented in the following manner: based on the negative pseudo label and the negative sample probability, the negative sample learning process is performed using a negative sample loss function, and the negative sample loss function is expressed as follows:

[0107]

[0108] in, Select a function for negative samples, is the number of negative samples of the target domain remote sensing image training data.

[0109] S106.3: Perform entropy minimization learning on the first prediction probability based on the first prediction probability.

[0110] This step can be implemented in the following manner: performing entropy minimization processing on the first predicted probability by using an entropy minimization loss function, wherein the entropy minimization loss function is expressed as follows:

[0111]

[0112] in, is the first predicted probability, N TUis the number of remote sensing image data in the target domain, and j is the sample index of the target domain.

[0113] S107: Construct a total loss function and update the pre-training initialization parameters based on the total loss function to obtain a deep learning network model for small sample classification.

[0114] The source domain labeled remote sensing images are subjected to supervised learning and combined with the confidence dynamic learning strategy to obtain a total loss function. The pre-training initialization parameters are updated based on the total loss function to obtain a deep learning network model for small sample classification.

[0115] The total loss function expression is as follows:

[0116] L=L S +λ(L P +L N )+L SE

[0117] in,

[0118]

[0119]

[0120]

[0121] Among them, L S is the source domain cross entropy loss function, L P is the target domain positive sample consistency regularization loss function, L N is the target domain negative sample consistency regularization loss function, L SE is the entropy minimization loss function, and λ is a hyperparameter used to adjust the weight of the loss function. Its initial value is 0 and gradually increases to 1 as the number of training rounds increases. Its calculation formula is as follows: K is the total number of training rounds, k is the current training round; is the label corresponding to the image sample, Output prediction probability for the classifier, N S is the number of source domain samples, Select function for positive samples, p j ′ is a pseudo label, is the first predicted probability, is the number of positive samples of remote sensing image training data in the target domain, Select function for negative samples, p j ′ is a pseudo label, is the number of negative samples of the target domain remote sensing image training data, N TU is the number of target domain remote sensing image training data samples, and j is the target domain sample index.

[0122] S108: The remote sensing image data to be tested is input into the deep learning network model for small sample classification to obtain a classification result of the remote sensing image data to be tested.

[0123] The classification result indicates the category of the remote sensing image data to be tested. The remote sensing image data to be tested carries a second number of labels, which is no more than 20% of the number of remote sensing image data to be tested. This is only an exemplary description and not a limitation of the present invention.

[0124] In some embodiments, the remote sensing scene image data to be tested may originate from the target domain.

[0125] This step can be implemented in the following ways:

[0126] S108.1: Divide the remote sensing scene image data to be tested into a support set and a query set, wherein the support set data carries a second number of labels, and the query set does not carry any labels;

[0127] S108.2: Inputting the support set data into the deep learning network model for small sample classification, wherein the feature extractor weight parameters are frozen and not updated, and the classifier parameters are fine-tuned to obtain a deep learning network model for small sample classification of target domain remote sensing scene images;

[0128] S108.3: Input the query set data into the deep learning network model for small sample classification of target domain remote sensing scene images to obtain a classification result of the query set data, where the classification result indicates the category of the query set data.

[0129] In summary, the remote sensing cross-domain small-sample classification method based on dynamic confidence learning can perform small-sample classification on remote sensing scene images under cross-domain conditions, and is suitable for open environments, small sample sizes, and sparsely labeled conditions in real-world remote sensing classification applications. Based on the semi-supervised cross-domain collaborative training model, the deep learning network model can simultaneously learn from both the source and target domains to achieve cross-domain knowledge transfer from the source to the target domain. Using a confidence threshold, the target domain training data samples can be divided into positive and negative samples for positive and negative learning, respectively. This maximizes the utilization of pseudo-label information while avoiding the negative interference of low-quality pseudo-labels on cross-domain learning, achieving highly reliable cross-domain knowledge transfer.

[0130] Compared with the prior art, the present invention has the following beneficial effects:

[0131] 1) Compared with existing technologies, the method of the present invention can better accurately classify open cross-domain scenes in actual remote sensing scene applications and remote sensing scene images under small sample and few annotation constraints.

[0132] 2) By dividing positive and negative samples by confidence thresholds for dynamic learning, it can handle a variety of cross-domain remote sensing image small sample classification tasks, including cross-domain scenarios between natural, visible light, and SAR image data.

[0133] 3) The labor and time costs of remote sensing data annotation can be reduced by utilizing unlabeled data and small sample learning methods.

[0134] 4) This method uses a smaller backbone network, which has a fast training speed while ensuring high classification performance of small samples across domains, and has good practical application value.

[0135] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or perform equivalent replacements on some of the technical features thereof. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A cross-domain small sample classification method for remote sensing images based on confidence dynamic learning, characterized by: The following steps are involved: S101: Acquire source domain remote sensing image data and target domain remote sensing image data; The source domain remote sensing image data carries a label, and the target domain remote sensing image data does not carry a label; the label indicates the category of the remote sensing image data; S102: Pre-training a deep learning network model using the source domain remote sensing image data to obtain pre-training initialization parameters of the deep learning network model; S103: Constructing a semi-supervised cross-domain collaborative training model for cross-domain feature representation learning and loading the pre-training initialization parameters; The semi-supervised cross-domain collaborative training model includes a first model and a second model; The constructing of a semi-supervised cross-domain collaborative training model for cross-domain feature representation learning and loading the pre-training initialization parameters includes: Loading the pre-trained initialization parameters into the first model and the second model simultaneously, The first model includes a first feature extractor and a first classifier, and the second model includes a second feature extractor and a second classifier; S104: Inputting the source domain remote sensing image data into the first model for supervised learning, so that the first model has a general feature information extraction capability; Inputting the source domain remote sensing image data into the first model for supervised learning so that the first model has the ability to extract general feature information includes: S104.1: Extract features from the source domain remote sensing image data to obtain source domain features : ,in, is the source domain remote sensing image data, is the first feature extractor, represents the source domain, is the source domain sample index; S104.2: Perform classification prediction on the source domain features to obtain source domain classification prediction probability : ,in, is the first classifier, is the normalized exponential function; S104.3: Based on the source domain classification prediction probability and the label of the source domain remote sensing image data, the first model is supervised learning using the source domain supervised loss. The expression is as follows: in, is the cross entropy loss function, is the source domain remote sensing image data label, is the number of remote sensing image data in the source domain; The general feature information includes at least mid- and low-level feature information; S105: Performing semi-supervised cross-domain collaborative training model processing on the target domain remote sensing image data to obtain a first prediction probability and a second prediction probability; The performing the semi-supervised cross-domain collaborative training model processing on the target domain remote sensing image data includes: S105.1: Performing enhancement processing on the target domain remote sensing image data to obtain strongly enhanced data samples and weakly enhanced data samples; S105.2: Process the strongly enhanced data sample through a first feature extractor to obtain a first feature map : ,in, is the strongly enhanced data sample, is the first feature extractor, Strong enhancement for the target domain, is the target domain sample index; S105.3: Process the first feature map through a first classifier to obtain a first prediction probability : ,in is the first classifier; S105.4: The weakly enhanced data sample is processed by a second feature extractor to obtain a second feature map : ,in, is the weakly enhanced data sample, is the second feature extractor, Weakly enhance the target domain; S105.5: Process the second feature map through a second classifier to obtain a second prediction probability : ,in is the second classifier; S106: Perform confidence dynamic learning strategy processing on the first predicted probability and the second predicted probability; S107: constructing a total loss function and updating the pre-training initialization parameters based on the total loss function to obtain a deep learning network model for small sample classification; S108: The remote sensing image data to be tested is input into the deep learning network model for small sample classification to obtain a classification result of the remote sensing image data to be tested, where the classification result indicates the category of the remote sensing image data to be tested; The remote sensing image data to be tested partially carries labels.

2. The method according to claim 1, characterized in that Performing enhancement processing on the target domain remote sensing image data, including weak enhancement processing and strong enhancement processing: The weak enhancement process includes: S105.1.11: Randomly crop the target domain remote sensing image data to obtain a first cropped image; S105.1.12: Horizontally flip the first cropped image to obtain a first flipped image; S105.1.13: Normalize the first flipped image to obtain a weakly enhanced image data sample; The strong enhancement process includes: S105.1.21: Randomly crop the target domain remote sensing image data to obtain a second cropped image; S105.1.22: Horizontally flip the second cropped image to obtain a second flipped image; S105.1.23: Perform color dithering on the second flipped image to obtain a color-changed image; S105.1.24: Performing Gaussian blurring on the color-changed image to obtain a blurred image; S105.1.25: Subject the blurred image to a random grayscale transformation to obtain a random grayscale image; S105.1.26: Normalize the random grayscale image to obtain a strongly enhanced image data sample.

3. The method according to claim 1, wherein The performing confidence dynamic learning strategy processing on the first predicted probability and the second predicted probability includes: S106.1: Perform sample partitioning processing on the target domain remote sensing image data based on the second predicted probability to obtain a positive sample set and a negative sample set; the positive sample set includes positive samples, positive pseudo labels, and positive sample probabilities; and the negative sample set includes negative samples, negative pseudo labels, and negative sample probabilities; S106.2: Perform positive sample learning processing and negative sample learning processing on the positive sample set and the negative sample set respectively based on the first predicted probability; S106.3: Perform entropy minimization learning processing based on the first prediction probability.

4. The method according to claim 3, wherein The performing sample division processing on the target domain remote sensing image data based on the second prediction probability includes: S106.1.1: Calculate a confidence dynamic threshold based on the second predicted probability, the confidence dynamic threshold The calculation formula is as follows: in, is the dynamic confidence threshold under k rounds, is the average confidence value of the second prediction probability of all samples in the current round, is a hyperparameter, its initial value is 1, and it decreases as the number of rounds increases. The update formula is: , is the total number of training rounds, is the current round number; S106.1.2: Sharpen the second predicted probability to obtain a pseudo label : , in, is the second feature map, is the temperature parameter used to sharpen the second predicted probability, is the second classifier; S106.1.3: Based on the confidence dynamic threshold, use a positive-negative sample selection function to perform positive-negative sample division to obtain a positive sample set and a negative sample set, wherein the positive-negative sample selection function is expressed as follows: in, is a positive sample, is a negative sample, is the sample index, Select a function for positive samples, Select a function for negative samples, select a function is a binary vector, when Indicates that the sample is selected when Indicates that the sample is not selected. For samples The second predicted probability The confidence level, .

5. The method according to claim 3, characterized in that The positive sample set and the negative sample set are respectively subjected to positive sample learning processing and negative sample learning processing. include: Among them, the positive sample learning process includes: Positive sample learning is performed based on the positive pseudo labels and the positive sample probability through a positive sample loss function, and the positive sample loss function expression is as follows: in, Select a function for positive samples, is a pseudo label, is the first predicted probability, is the number of positive samples of remote sensing image data in the target domain, Indexing remote sensing image data for the target domain; Among them, negative sample learning processing includes: Negative sample learning is performed based on the negative pseudo label and the negative sample probability through a negative sample loss function, and the negative sample loss function expression is as follows: in, Select a function for negative samples, is the number of negative samples of remote sensing image data in the target domain.

6. The method according to claim 3, wherein The entropy minimization learning process based on the first prediction probability includes: The first prediction probability is subjected to entropy minimization learning processing by using an entropy minimization loss function, wherein the entropy minimization loss function is expressed as follows: in, is the first predicted probability, is the number of remote sensing image data in the target domain, is the target domain sample index.

7. The method according to claim 1, characterized in that The total loss function is obtained by combining supervised learning of labeled remote sensing image data in the source domain with the confidence dynamic learning strategy, and is expressed as follows: in, , ， ， ， in, is the source domain cross entropy loss function, is the target domain positive sample consistency regularization loss function, is the target domain negative sample consistency regularization loss function, is the entropy minimization loss function, is the hyperparameter used to adjust the weight of the loss function is the label corresponding to the image sample, Output the predicted probability for the classifier, is the number of source domain samples, Select a function for positive samples, is a pseudo label, is the first predicted probability, is the number of positive samples of remote sensing image training data in the target domain, Select a function for negative samples, is a pseudo label, is the number of negative samples of the target domain remote sensing image training data, is the number of training data samples of remote sensing images in the target domain, is the target domain sample index.

Images