Cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning

Abstract

The present invention relates to a cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning, comprising the following steps: A) acquiring remote sensing images and constructing input data; B) constructing a loss function based on self-supervised contrastive learning and in combination with known class samples and unknown class samples of target domain images, and constructing a deep domain adaptation learning network, and training it using the input data and the loss function; C) classifying the target domain images using the deep domain adaptation learning network, extracting target image feature vectors of the target domain images to construct a feature database, extracting query image feature vectors of the target domain query images, calculating the Euclidean distance between the query image feature vector and the target image feature vector in the feature database, and selecting the desired retrieval target based on the Euclidean distance. The cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning of the present invention can still have good cross-domain classification and retrieval accuracy when there are unknown categories in the target domain.

Description

Technical Field

[0001] The present invention relates to the technical field of optical remote sensing image retrieval, and in particular to a cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning. Background Art

[0002] In recent years, advances in Earth observation technology have provided humanity with an increasing amount of high-resolution remote sensing imagery, bringing tremendous opportunities to the remote sensing field and significantly promoting its application in various fields. Remote sensing image scene classification and retrieval are fundamental tasks in remote sensing image interpretation. They enable the rapid and accurate understanding and management of remote sensing image data, playing a vital role in environmental monitoring, land use, and visual navigation.

[0003] Deep convolutional neural networks (CNNs), developed in recent years, possess powerful feature-fitting capabilities and have demonstrated significant superiority in remote sensing scene classification and retrieval tasks. The general process involves first fine-tuning a backbone network pre-trained on a common large-scale image dataset (such as ImageNet) on a remote sensing image dataset. The network's activation output is then extracted as an image feature representation for retrieval or classification.

[0004] However, most existing CNN-based methods are supervised and usually assume that the training set and test set share the same data distribution. In addition, in practical applications, due to differences in imaging conditions such as sensors, shooting angles, and shooting weather, the same type of ground objects may have huge differences in different data distributions. This phenomenon is called data offset. When there is data offset between the training set and the test set, the model's generalization effect on the test set is very poor, and re-labeling the test set is time-consuming, labor-intensive, and impractical. Moreover, most existing domain adaptation remote sensing scene classification or retrieval methods are proposed for closed set scenes, that is, they assume that the target domain and the source domain share the same label space. In complex practical scenarios, this assumption is easily violated because the categories of the source domain are often incomplete and the source domain cannot cover all classes. The target domain can contain class samples that are not shared by the source domain. In this case, traditional domain adaptation methods generally lead to negative transfer because unknown class samples are easily misfitted to shared categories, resulting in poor model generalization.

[0005] In view of this, it is necessary to design a cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning, which can still have good classification and retrieval accuracy of the target domain when there are unknown categories in the target domain.

[0007] In order to solve the above technical problems, the present invention provides a cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning, comprising the following steps:

[0008] A) acquiring a remote sensing image and dividing the remote sensing image into a source domain image and a target domain image to construct input data;

[0009] B) constructing a loss function based on self-supervised contrastive learning and combining known classes and unknown classes of the target domain image, constructing a deep domain adaptation learning network, and training the deep domain adaptation learning network using the input data and the loss function;

[0010] C) classifying the target domain image using the trained deep domain adaptation learning network, extracting the target image feature vector of the target domain image to construct a feature database, extracting the query image feature vector of the target domain query image, calculating the Euclidean distance between the query image feature vector and all the target image feature vectors in the feature database, arranging them according to the Euclidean distance, and obtaining the required retrieval target according to a set Euclidean distance range.

[0011] Furthermore, the step of constructing the input data includes: extracting a number of images {1, 2, ..., N} from the remote sensing image dataset to construct the source domain image The source domain image contains n s Zhang has annotated source domain images Represents annotated source domain image The corresponding tags, where represents the label space of the labeled source domain image, C represents the total number of categories of the labeled source domain image; the target domain image is The target domain image contains n t An unlabeled target domain image Among them, the target domain image The label space is: {1,2,…,C,C+1}, where C+1 represents the unknown category of the unlabeled target domain image.

[0012] Furthermore, the deep domain adaptation learning network includes multiple feature encoding networks f(·), multiple contrastive learning networks g(·) and multiple classifiers c(·).

[0013] Furthermore, the feature encoding network f(·) is a deep residual network with the fully connected layer removed, and the average pooling layer of the deep residual network is replaced by a bottleneck layer.

[0014] Furthermore, the contrastive learning network g(·) is a perceptron with a ReLU (rectified linear unit) activation function.

[0015] Furthermore, the classifier c(·) is a fully connected network, and the output dimension of the classifier c(·) is consistent with the number of categories of the target domain image.

[0016] Furthermore, the step of constructing the loss function includes:

[0017] B11) Constructing source domain classification loss: Perform supervised learning on the source domain image and use cross entropy loss to calculate the classification accuracy:

[0018]

[0019] Among them, L softmax is the classification loss function, The source domain image that represents the source domain image has annotated images The true category distribution of represents the probability distribution of the source domain weakly enhanced sample categories output by the classifier, Represents the collection of labeled samples with annotated images in the source domain;

[0020] B12) Constructing a self-supervised contrast loss: Constructing a target domain strong enhancement sample of the target domain image and weakly enhanced samples in the target domain To calculate the contrast loss L ssl :

[0021]

[0022] Where sim(·) is the similarity metric function, θ is the scaling factor, A∈{0,1} is an indicator function used to evaluate whether k is equal to j, and B represents the number of samples selected for one training;

[0023] B13) Construct the known class recognition loss as:

[0024]

[0025] Among them, μ represents the proportion of samples that meet the known class threshold selection requirements in one training, H(·) represents the cross entropy loss, is a weakly enhanced sample from the target domain The collection of pseudo labels of known classes in the target domain obtained by screening, ind represents the weakly enhanced samples in the target domain The category to which ind∈{1,2,…,C} belongs after filtering by the known class pseudo-label. Represents the target domain strong enhancement sample The predicted category probability distribution of Represents the collection of labeled samples of strongly enhanced samples in the target domain;

[0026] B14) Construct unknown class recognition loss: The unknown class recognition loss is the consistency classification loss L of high confidence unknown class samples unknown :

[0027]

[0028] in, is a weakly enhanced sample from the target domain The collection of pseudo labels of unknown classes in the target domain obtained by screening, Represents the target domain strong enhancement sample The predicted category probability distribution of ;

[0029] B15) The total loss function L constructed is:

[0030] L=L softmax +αL ssl +βL known +γL unknown

[0031] Among them, α, β and γ are parameters that weigh the optimization objectives of the model.

[0032] Furthermore, the target domain weakly enhanced sample By comparing the unlabeled target domain image Obtained by random cropping and flipping; the target domain strongly enhanced sample By comparing the unlabeled target domain image The random enhancement method is used to obtain the weakly enhanced samples in the source domain; the weakly enhanced samples in the source domain are obtained by Obtained using random cropping and flipping.

[0033] Furthermore, the training steps of the deep domain adaptation learning network include:

[0034] B21) The source domain weakly enhanced samples and the target domain weakly enhanced samples are and target domain strong enhancement samples Input into the feature encoding network f(·) to obtain the source domain features Weakly enhanced image features in the target domain and target domain strong enhanced image features

[0035] B22) Weakly enhancing the image features of the target domain and the target domain strongly enhanced image features Input the contrastive learning network g(·) to obtain the projected target domain weakly enhanced image embedding feature and target domain strongly enhanced image embedding features To calculate the contrast loss L ssl ;

[0036] B23) the source domain features The target domain weakly enhances image features and the target domain strongly enhanced image features Input the classifier c(·) to obtain the probability distribution of the source domain weak enhancement sample category predicted by the classifier The target domain weakly enhanced sample category probability distribution And the target domain strongly enhanced sample category probability distribution

[0037] B24) Probability distribution of the weakly enhanced sample categories in the source domain Based on the classification loss function L softmax , calculate the classification loss of the source domain;

[0038] B25) Weakly enhance the probability distribution of sample categories in the target domain First, find the category with the maximum predicted probability, compare the probability value of the category with the predefined threshold τ, discard the samples with a probability less than τ, retain the samples with a probability greater than τ as pseudo-label samples, and use the category with the maximum predicted probability as the known hard label. The screening formula is:

[0039]

[0040] in, express The category where the maximum predicted probability that meets the threshold condition is located;

[0041] B26) Using the target domain weak enhancement sample Pseudo labels of known classes As the corresponding target domain strong enhancement sample The supervisory signal is used to calculate the strong enhancement sample of the target domain The known class recognition loss L known ;

[0042] B27) Select the probability distribution of the target domain weakly enhanced sample category The samples with lower confidence are selected as candidate unknown class samples. The specific selection formula is:

[0043]

[0044] in is the candidate unknown class sample after preliminary screening, t l Select a threshold for the candidate samples, and then select the samples whose predicted probability of being an unknown class is higher than the set unknown class sample selection threshold as unknown class samples.

[0045]

[0046] in For the candidate sample The probability of predicting the unknown category, t uk Select a threshold for the unknown class sample, Pseudo labels for unknown classes in the target domain;

[0047] B28) Use the target domain unknown class pseudo label As the target domain strong enhancement sample The supervision signal is used to calculate the consistency classification loss L of the unknown category samples. unknown , derive the total loss function L, and update the parameters of the deep domain adaptation learning network.

[0048] Furthermore, the steps of obtaining the search target are:

[0049] C21) extracting the query image feature vector based on the trained feature encoding network;

[0050] C22) calculating the Euclidean distance between the query image feature vector and each of the target image feature vectors in the feature database one by one;

[0051] C23) Sort the target image feature vectors according to the Euclidean distance between the target image feature vector and the query image feature vector to obtain the retrieval target corresponding to the target image feature vector.

[0052] Through the above technical solution, in the cross-domain remote sensing scene classification and retrieval method of self-supervised contrastive learning provided by the present invention, it is first necessary to construct input data, including data of source domain images and target domain images, wherein the data of source domain images are labeled data and the data of target domain images are unlabeled, and the constructed input data are correspondingly enhanced. Subsequently, the enhanced source domain image data and target domain image data are input into the corresponding feature encoding network, the output result is compared with the input data, and a loss function is constructed based on self-supervised contrastive learning in combination with the known classes and unknown classes of the target domain image, so that the network parameters of the feature encoding network can be adjusted based on the loss function, thereby reducing the influence of the feature encoding network on the existence of unknown class samples in the target domain, so that the trained deep domain adaptation learning network has better effect in classifying or retrieving data containing unknown class samples.

[0053] Other advantages of the present invention and the technical effects of the preferred embodiments will be further described in the following specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flow chart of the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning of the present invention;

[0055] Figure 2 Schematic diagram of the principle of the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning of the present invention;

[0056] Figure 3 Schematic diagram of the training process of the deep domain adaptation learning network in the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning of the present invention;

[0057] Figure 4 Schematic diagram of the retrieval process in the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning of the present invention;

[0058] Figure 5 It is the classification confusion matrix of adversarial discriminant domain adaptation in the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning of the present invention;

[0059] Figure 6 It is a classification confusion matrix with batch singular value constraints in the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning of the present invention;

[0060] Figure 7 It is the classification confusion matrix of the deep domain adaptation network in the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning of the present invention;

[0061] Figure 8It is the classification confusion matrix of back-propagation open set domain adaptation in the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning of the present invention;

[0062] Figure 9 It is the classification confusion matrix of the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning of the present invention. DETAILED DESCRIPTION

[0063] The following describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not intended to limit the present invention.

[0064] like Figure 1 and Figure 2 As shown in FIG, as an embodiment of the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning provided by the present invention, the method includes the following steps:

[0065] A) obtaining a remote sensing image and dividing the remote sensing image into a source domain image and a target domain image to construct input data;

[0066] B) Based on self-supervised contrastive learning and combining known and unknown class samples of the target domain image to construct a loss function, construct a deep domain adaptation learning network, and train the deep domain adaptation learning network using the input data and loss function;

[0067] C) Using the trained deep domain adaptation learning network to classify the target domain image, extract the target image feature vector of the target domain image to build a feature database, extract the query image feature vector of the target domain query image, calculate the Euclidean distance between the query image feature vector and all target image feature vectors in the feature database, arrange them according to the Euclidean distance, and obtain the required retrieval target according to the set Euclidean distance range.

[0068] Specifically, in one embodiment of the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning provided by the present invention, the step of constructing the input data includes: extracting a number of images {1, 2, ..., N} from the remote sensing image dataset, constructing the source domain image The source domain image contains n s Zhang has annotated source domain images Represents annotated source domain image The corresponding tags, where represents the label space of the labeled source domain image, C represents the total number of categories of the labeled source domain image; the target domain image is The target domain image contains n t An unlabeled target domain image Among them, the target domain image The label space is: {1,2,…,C,C+1}, where C+1 represents the unknown category of the unlabeled target domain image.

[0069] Furthermore, in an embodiment of the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning provided by the present invention, Figure 3 and Figure 4 As shown in the figure, the deep domain adaptation learning network includes multiple feature encoding networks f(·), multiple contrastive learning networks g(·) and multiple classifiers c(·); among them, the feature encoding network is a deep residual network with the fully connected layer removed, and its final average pooling layer is replaced by a bottleneck layer, and the output is a 256-dimensional feature vector; the contrastive learning network g(·) is a perceptron with a ReLU (rectified linear unit) activation function; the classifier c(·) is a fully connected network, the input of the classifier c(·) is a 256-dimensional feature vector, and its output dimension is consistent with the number of categories of the target domain image (that is, the probability distribution of {1,2,…,C,C+1} classes).

[0070] Furthermore, in one embodiment of the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning provided by the present invention, the step of constructing the loss function includes:

[0071] B11) Constructing source domain classification loss: Since the source domain images are semantically annotated (i.e., they are all labeled source domain images), supervised learning can be performed on them, and the cross entropy loss is used to calculate the classification accuracy:

[0072]

[0073] Among them, L softmax is the classification loss function, Source domain annotated images representing source domain images The true category distribution of Represents the probability distribution of the source domain weakly enhanced sample category output by the classifier, The collection of labels representing the annotated images in the source domain;

[0074] B12) Constructing self-supervised contrastive loss: Self-supervised contrastive learning learns representations by maximizing the information between different views of the data. Specifically, it learns more discriminative image features by encouraging two views of the same image from the target domain (i.e., strongly enhanced view and weakly enhanced view) to be similar, and two views from different images to be dissimilar. Therefore, target domain strongly enhanced samples of target domain images can be constructed. and weakly enhanced samples in the target domain To calculate the contrast loss L ssl :

[0075]

[0076] Among them, sim(·) is the similarity metric function, θ is the scaling factor, A∈{0,1} is an indicator function used to evaluate whether k is equal to j, and B represents the number of samples selected for one training; specifically, the target domain weakly enhanced samples It is composed of unlabeled target domain images Obtained by random cropping and flipping; target domain strongly enhanced samples It is composed of unlabeled target domain images It is obtained using random enhancement method;

[0077] B13) Construct the known class recognition loss as:

[0078]

[0079] Among them, μ represents the proportion of samples that meet the known class threshold selection requirements in one training, H(·) represents the cross entropy loss, is a weakly enhanced sample from the target domain The collection of pseudo labels of known classes in the target domain obtained by screening, ind represents the weakly enhanced samples in the target domain The category to which ind∈{1,2,…,C} belongs after being filtered by the known class pseudo-label. Represents a strongly enhanced sample in the target domain The predicted category probability distribution of Represents the collection of labeled samples of strongly enhanced samples in the target domain;

[0080] B14) Construct unknown class recognition loss: The unknown class recognition loss is the consistency classification loss L of high confidence unknown class samples unknown :

[0081]

[0082] in, is a weakly enhanced sample from the target domain The collection of pseudo labels of unknown classes in the target domain obtained by screening, Represents a strongly enhanced sample in the target domain The predicted category probability distribution of ;

[0083] B15) The total loss function L constructed is:

[0084] L=L softmax +αL ssl +βL known +γL unknown

[0085] Among them, α, β and γ are parameters that weigh the optimization objectives of the model.

[0086] Furthermore, in one embodiment of the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning provided by the present invention, the training steps of the deep domain adaptation learning network include:

[0087] B21) The feature encoding network f(·) can be set to three, and the source domain weakly enhanced samples and the target domain weakly enhanced samples and target domain strong enhancement samples are input into the corresponding feature encoding network f(·) to obtain the source domain features f i s , weakly enhanced image features in the target domain and target domain strong enhanced image features Among them, the source domain weak enhancement samples are composed of labeled source domain images Obtained using random cropping and flipping;

[0088] B22) The contrastive learning network g(·) can be set to two, and the target domain weakly enhances the image features. and target domain strong enhanced image features They are input into the corresponding contrastive learning network g(·) respectively, and the projected target domain weakly enhanced image embedding features are obtained in decibels. and target domain strongly enhanced image embedding features To calculate the contrast loss L ssl ;

[0089] B23) The classifier c(·) can be set to three, and the source domain features fi i s , weakly enhanced image features in the target domain and target domain strong enhanced image features Input them into the corresponding classifier c(·) respectively to obtain the probability distribution of the source domain weak enhancement sample category predicted by the classifier Probability distribution of weakly enhanced sample categories in the target domain And the target domain strongly enhances the probability distribution of sample categories

[0090] B24) Probability distribution of weakly enhanced sample categories in the source domain Based on the classification loss function L softmax , calculate the classification loss of the source domain;

[0091] B25) Weakly enhance the probability distribution of sample categories in the target domain First find the probability distribution of weakly enhanced sample categories in the target domain The category with the maximum predicted probability is then compared with the predefined threshold τ, and samples with values ​​less than the predefined threshold τ are discarded to retain samples with values ​​greater than the predefined threshold τ as pseudo-label samples, and the category with the maximum predicted probability is used as the known hard label. The sample screening formula is:

[0092]

[0093] in, Represents the probability distribution of weakly enhanced sample categories in the target domain The category with the maximum predicted probability that meets the threshold condition;

[0094] B26) Use weakly enhanced samples from the target domain Pseudo labels of known classes As the corresponding target domain strong enhancement sample The supervisory signal is used to calculate the strong enhancement samples in the target domain The known class recognition loss L known ;

[0095] B27) Select the probability distribution of weakly enhanced sample categories in the target domain The samples with lower confidence are selected as candidate unknown class samples. The specific selection formula is:

[0096]

[0097] in is the candidate unknown class sample after preliminary screening, t l Select a threshold for the candidate samples, and then select the samples whose predicted probability of being an unknown class is higher than the set unknown class sample selection threshold as unknown class samples.

[0098]

[0099] in Candidate samples The probability of predicting the unknown category, t uk Select a threshold for unknown class samples, Pseudo labels for unknown classes in the target domain;

[0100] B28) Use pseudo labels of unknown classes in the target domain As a strong enhancement sample in the target domain The supervision signal is used to calculate the consistency classification loss L of the unknown category samples. unknown , derive the total loss function L and update the parameters of the deep domain adaptation learning network.

[0101] Furthermore, in one embodiment of the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning provided by the present invention, the step of acquiring the target domain image is:

[0102] C21) extracting a query image feature vector based on the trained feature encoding network;

[0103] C22) calculating the Euclidean distance between the query image feature vector and each target image feature vector in the feature database one by one;

[0104] C23) Sort the target image feature vectors according to the Euclidean distance between the target image feature vector and the query image feature vector to obtain the retrieval target corresponding to the target image feature vector.

[0105] The above construction of input data and training of deep domain adaptation learning network are implemented based on the PyTorch library of Python language. In addition, simulation experiments of domain adaptation methods such as ADDA (Adversarial Discriminative Domain Adaptation), BSP (Batch Spectral Penalization), DAN (Deep Adaptation Network) and OSBP (Open Set Domain Adaptation by Backpropagation) are also carried out to compare with the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning of the present invention. The overall classification accuracy and classification confusion matrix are used to evaluate the classification effect, and the average normalized modified retrieval rank (ANMRR), average retrieval precision (mAP) and PK (retrieval precision of the top K images) are used to evaluate the retrieval effect. Among them, the higher the average retrieval precision (mAP) and the retrieval precision PK of the top K images, the better the retrieval performance, and the smaller the average normalized modified retrieval rank ANMRR value, the better the retrieval performance. The comparison results are shown in Table 1:

[0106] method Classification accuracy ANMRR mAP P5 P10 P20 P50 P100 ADDA 0.602 0.2872 0.5845 0.7770 0.7540 0.7215 0.6546 0.5280 BSP 0.616 0.2800 0.5928 0.8070 0.7675 0.7238 0.6498 0.5324 DAN 0.6 0.2622 0.5997 0.7930 0.7695 0.7375 0.6658 0.5503 OSBP 0.6563 0.2725 0.5921 0.7260 0.7000 0.6880 0.6365 0.5403 The present invention 0.8063 0.2222 0.6777 0.8800 0.8635 0.8318 0.7630 0.6103

[0107] Table 1

[0108] The results in Table 1 show that the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning of the present invention has achieved the highest retrieval accuracy. Compared with the contrasting methods, the classification accuracy of the present invention method has increased by 15% to 20.63%. At the same time, the retrieval accuracy has also surpassed the contrasting methods in all aspects. Specifically, the average retrieval accuracy of the present invention method has been improved by at least 7.8% compared with the contrasting methods, and the P5-P100 and ANMRR of the present invention method are better than those of the contrasting methods. In addition, Figures 5 to 9 The classification confusion matrices of different methods and the present invention are also presented. The values ​​on the diagonal of the classification confusion matrix represent the probability of correctly classifying a particular class, while the values ​​outside the diagonal represent the probability of incorrectly classifying another class. The results demonstrate that the present method effectively improves the classification accuracy of the target domain, particularly significantly improving the classification accuracy of unknown classes in the target domain while simultaneously reducing confusion between unknown and known classes. In summary, the cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning proposed in this invention can effectively improve cross-domain classification and retrieval performance under conditions of data distribution differences and inconsistent category spaces.

[0109] Through the above technical solution, in the cross-domain remote sensing scene classification and retrieval method of self-supervised contrastive learning provided by the present invention, it is first necessary to construct input data, including data of source domain images and target domain images, wherein the data of source domain images are labeled data and the data of target domain images are unlabeled, and the constructed input data are correspondingly enhanced. Subsequently, the enhanced source domain image data and target domain image data are input into the corresponding feature encoding network, the output result is compared with the input data, and a loss function is constructed based on self-supervised contrastive learning in combination with the known classes and unknown classes of the target domain image, so that the network parameters of the feature encoding network can be adjusted based on the loss function, thereby reducing the influence of the feature encoding network on the existence of unknown class samples in the target domain, so that the trained deep domain adaptation learning network has better effect in classifying or retrieving data containing unknown class samples.

[0110] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Within the technical concept of the present invention, the technical solution of the present invention may be subjected to various simple modifications, including combining the specific technical features in any suitable manner. To avoid unnecessary repetition, the present invention will not further describe various possible combinations. However, these simple modifications and combinations should also be regarded as disclosed in the present invention and fall within the scope of protection of the present invention.

Claims

1. A cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning, characterized by: The steps include: A) acquiring a remote sensing image and dividing the remote sensing image into a source domain image and a target domain image to construct input data; B) constructing a loss function based on self-supervised contrastive learning and combining known class samples and unknown class samples of the target domain image, constructing a deep domain adaptation learning network, and training the deep domain adaptation learning network using the input data and the loss function; C) classifying the target domain image using the trained deep domain adaptation learning network, extracting target image feature vectors of the target domain image to construct a feature database, extracting query image feature vectors of the target domain query image, calculating the Euclidean distance between the query image feature vector and all target image feature vectors in the feature database, arranging them according to the Euclidean distance, and obtaining the desired retrieval target according to a set Euclidean distance range; The deep domain adaptation learning network includes multiple feature encoding networks , multiple contrastive learning networks and multiple classifiers ; The steps of constructing the loss function include: B11) Constructing source domain classification loss: Perform supervised learning on the source domain image and use cross entropy loss to calculate the classification accuracy: in, is the classification loss function, The source domain image that represents the source domain image has annotated images The true category distribution of represents the probability distribution of the source domain weakly enhanced sample categories output by the classifier, Represents the collection of labeled samples with annotated images in the source domain; B12) Constructing a self-supervised contrast loss: Constructing a target domain strong enhancement sample of the target domain image and weakly enhanced samples in the target domain , to calculate the contrast loss : Among them, sim( ) is the similarity measurement function, is a scaling factor, and A∈{0, 1} is an indicator function used to evaluate whether k is equal to , B represents the number of samples selected for one training; B13) Construct the known class recognition loss as: in, Indicates the proportion of samples that meet the known class threshold selection requirements within one training session, represents the cross entropy loss, is a weakly enhanced sample from the target domain The collection of pseudo labels of known classes in the target domain obtained by screening, Represents the weakly enhanced sample in the target domain The category to which it belongs after being filtered by the known class pseudo-labels, and , Represents the target domain strong enhancement sample The predicted category probability distribution of Represents the collection of labeled samples of strongly enhanced samples in the target domain; B14) Construct unknown class recognition loss: unknown class recognition loss is the consistency classification loss of high confidence unknown class samples : in, is a weakly enhanced sample from the target domain The collection of pseudo labels of unknown classes in the target domain obtained by screening, Represents the target domain strong enhancement sample The predicted category probability distribution of ; B15) Constructing the total loss function for: in, and Parameters of the optimization objective for the trade-off model.

2. The cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning according to claim 1 is characterized in that: The step of constructing the input data includes: extracting a number of images from the remote sensing image dataset , construct the source domain image , the source domain image contains Zhang has annotated source domain images , Represents annotated source domain image The corresponding tags, where represents the label space of the annotated source domain image, represents the total number of categories of the labeled source domain image; the target domain image is , the target domain image contains An unlabeled target domain image , where the target domain image The label space is: , represents the unknown category of the unlabeled target domain image.

3. The cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning according to claim 2 is characterized in that: The feature encoding network A deep residual network is constructed by removing the fully connected layer, and the average pooling layer of the deep residual network is replaced by a bottleneck layer.

4. The cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning according to claim 3 is characterized in that: The contrastive learning network is a perceptron with ReLU activation function.

5. The cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning according to claim 4 is characterized in that: The classifier is a fully connected network, the classifier The output dimension of is consistent with the number of categories of the target domain image.

6. The cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning according to claim 5 is characterized in that: The target domain weakly enhanced sample By comparing the unlabeled target domain image , obtained by random cropping and flipping; the target domain strongly enhanced sample By comparing the unlabeled target domain image The random enhancement method is used to obtain the weakly enhanced samples in the source domain; the weakly enhanced samples in the source domain are obtained by Obtained using random cropping and flipping.

7. The cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning according to claim 6 is characterized in that: The training steps of the deep domain adaptation learning network include: B21) The source domain weakly enhanced samples and the target domain weakly enhanced samples and target domain strong enhancement samples Input to the feature encoding network In order to obtain the source domain features , weakly enhanced image features in the target domain and target domain strong enhanced image features ; B22) Weakly enhance the image features of the target domain and the target domain strongly enhanced image features Enter the contrastive learning network , get the target domain weakly enhanced image embedding features after projection and target domain strongly enhanced image embedding features , to calculate the contrast loss ; B23) The source domain features , the target domain weakly enhanced image features and the target domain strongly enhanced image features Input the classifier , to obtain the probability distribution of the source domain weak enhancement sample category predicted by the classifier , the probability distribution of weakly enhanced sample categories in the target domain And the target domain strongly enhanced sample category probability distribution ; B24) Probability distribution of weakly enhanced sample categories in the source domain , based on the classification loss function , calculate the classification loss of the source domain; B25) Weakly enhance the probability distribution of sample categories in the target domain First, find the category with the maximum predicted probability, and compare the probability value of the category with the predefined threshold Compare and discard the value less than For samples larger than The sample is taken as a pseudo-label sample, and the category with the maximum predicted probability is taken as the known class hard label. The screening formula is: in, ,express The category where the maximum predicted probability that meets the threshold condition is located; B26) Use the target domain weak enhancement sample Pseudo labels of known classes As the corresponding target domain strong enhancement sample The supervisory signal is used to calculate the strong enhancement sample of the target domain The known class recognition loss ; B27) Select the probability distribution of the weakly enhanced sample category in the target domain The samples with lower confidence are selected as candidate unknown class samples. The specific selection formula is: in is the candidate unknown class sample after preliminary screening, Select a threshold for the candidate samples, and then select the samples whose predicted probability of being an unknown class is higher than the set unknown class sample selection threshold as unknown class samples. in For the candidate sample The probability of predicting the unknown category, t uk Select a threshold for the unknown class sample, Pseudo labels for unknown classes in the target domain; B28) Use the target domain unknown class pseudo label As the target domain strong enhancement sample The supervision signal is used to calculate the consistency classification loss of the unknown category samples , the total loss function is obtained and use the gradient descent algorithm to update the parameters of the deep domain adaptation learning network.

8. The cross-domain remote sensing scene classification and retrieval method based on self-supervised contrastive learning according to claim 7 is characterized in that: The steps of obtaining the retrieval target are: C21) extracting the query image feature vector based on the trained feature encoding network; C22) calculating the Euclidean distance between the query image feature vector and each target image feature vector in the feature database one by one; C23) Sort the target image feature vectors according to the Euclidean distance between the target image feature vector and the query image feature vector to obtain the retrieval target corresponding to the target image feature vector.

Images