Remote sensing sample labeling method based on combined transfer learning

Abstract

The application provides a remote sensing sample labeling method based on combined transfer learning, which comprises the following steps: obtaining a source domain sample set and a target domain sample set; constructing a transfer learning model by using a combined transfer method comprising a first transfer method and a second transfer method; inputting the source domain sample set and the target domain sample set into the transfer learning model, achieving class balance of the source domain sample set and the target domain sample set by using the first transfer method, and saving the internal relationship between the features and the labels in the source domain sample set; adapting the features of the source domain sample set and the features of the target domain sample set by using the second transfer method to obtain new feature representations of the source domain sample set and the target domain sample set; and obtaining the labels of the target domain sample set according to the new feature representations of the target domain sample set and the internal relationship between the features and the labels in the source domain sample set. The application can reduce the distribution difference between the source domain and the target domain, alleviate the problem of too large data distortion caused by transfer, and obtain better labeling results.

Description

TECHNICAL FIELD

[0001] The present application relates to remote sensing image classification technology, and in particular to a remote sensing sample labeling method based on combined transfer learning. BACKGROUND

[0002] Remote sensing technology is currently widely used in geoscience applications, such as forest resource planning, crop yield estimation, environmental assessment, disaster monitoring, etc. Remote sensing image classification technology is a key step to convert remote sensing images from data to information.

[0003] The most commonly used method in remote sensing image classification is supervised classification method. Supervised classification method requires manual determination of classification system, training samples of each class (i.e. label information of training samples), etc. Supervised classification method can ensure good classification accuracy, but obtaining training samples is a time-consuming and laborious process.

[0004] In the conventional mode, a remote sensing satellite will image a certain area according to a certain revisit period, forming a series of time-series remote sensing images. If sample collection and labeling are performed on each newly acquired remote sensing image, the operation is tedious and the labor cost is too high. Therefore, it has certain practical value to study how to use the remote sensing images and sample labeling information of the previous phase to classify the newly acquired remote sensing images.

[0005] Due to the characteristics of remote sensing images, the feature-based transfer method (one of the transfer learning methods) applied to remote sensing samples often cannot achieve good results, and some important properties of the source domain and the target domain may be lost or weakened, such as the difference between the source domain and the target domain, and the data distortion affecting the labeling results of the target domain samples.

[0006] Therefore, a new remote sensing sample labeling method is needed to solve the above problems. SUMMARY

[0007] Based on the above technical problems, the present application provides a remote sensing sample labeling method based on combined transfer learning, which controls the class balance of the source domain and the target domain, reduces the distribution difference between the source domain and the target domain, and alleviates the problem of excessive data distortion during transfer, to obtain better labeling results.

[0008] To achieve the above technical purpose, the present application provides a remote sensing sample labeling method based on combined transfer learning, comprising:

[0009] S1 obtaining a source domain sample set and a target domain sample set;

[0010] S2 constructing a transfer learning model, the transfer learning model is constructed by a combined transfer method, the combined transfer method includes a first transfer method and a second transfer method;

[0011] S3 inputs the source domain sample set and the target domain sample set into the transfer learning model to utilize the first transfer method to make the source domain sample set and the target domain sample set reach class balance, and save the internal relation between the features and the labels in the source domain sample set;

[0012] S4 utilizes the second transfer method to adapt the features of the source domain sample set and the features of the target domain sample set, to obtain new feature representations of the source domain sample set and the target domain sample set;

[0013] S5 obtains the labels of the target domain sample set according to the new feature representations of the target domain sample set and the internal relation between the features and the labels in the source domain sample set.

[0014] In an embodiment of the present application, step S4 comprises:

[0015] S41 obtains a feature tensor of the source domain sample set and a feature tensor of the target domain sample set according to the features of the source domain sample set and the features of the target domain sample set respectively, and learns through the feature tensor of the source domain sample set and the feature tensor of the target domain sample set to obtain a shared feature space;

[0016] S42 aligns the source domain sample set and the target domain sample set to the shared feature space;

[0017] S43 obtains the new feature representations of the source domain sample set and the target domain sample set in the shared feature space according to an objective function of the transfer learning model.

[0018] In an embodiment of the present application, the transfer learning model comprises an objective function, and the objective function comprises a first optimization item, a second optimization item and a third optimization item:

[0019]

[0020] wherein, X is a feature space, Y is a label space, s refers to the source domain sample set, t refers to the target domain sample set, s→t refers to balancing the source domain according to the class distribution, P is a data distribution of the sample set, H is a shared feature space of the source domain sample set and the target domain sample set, f is a mapping function of the features of the source domain sample set mapped into the shared feature space, f -1 is an approximate inverse function of f, k is a mapping function of the features of the target domain sample set mapped into the shared feature space, and k -1 is an approximate inverse function of k.

[0021] In an embodiment of the present application, the first optimization item is used to optimize the class balance of the source domain sample set and the target domain sample set and the internal relation between the features and the labels in the source domain sample set;

[0022] The second optimization item to optimize the new feature representation of the source domain sample set in the shared feature space;

[0023] a third optimization term to optimize the new feature representation of the target domain sample set in the shared feature space.

[0024] In an embodiment of the present application, step S42 comprises:

[0025] A likelihood function is constructed based on the process of respectively aligning the shared feature space of the source domain sample set and the target domain sample set:

[0026]

[0027] wherein f is a mapping function of the feature of the source domain sample set mapped into the shared feature space, f -1 is an approximate inverse function of f, k is a mapping function of the feature of the target domain sample set mapped into the shared feature space, k -1 is an approximate inverse function of k, F is a Frobenius norm, χ s is a feature tensor of the source domain sample set, χ t is a feature tensor of the target domain sample set;

[0028] The first alignment matrix set and the second alignment matrix set are constructed, and the feature core tensors of the source domain sample set and the target domain sample set under the shared feature space are constructed.

[0029] The source domain sample set and the target domain sample set are aligned to the shared feature space by using the first alignment matrix set, the second alignment matrix set, the feature core tensors and the likelihood function.

[0030] In an embodiment of the present application, step S43 comprises:

[0031] According to the maximum likelihood estimation algorithm, the objective function of the transfer learning model is optimized by minimizing the likelihood function.

[0032] Based on the objective function of the optimized transfer learning model, the new feature representation of the source domain sample set and the new feature representation of the target domain sample set under the shared feature space are obtained.

[0033] In an embodiment of the present application, the objective function of the transfer learning model further comprises a fourth optimization term:

[0034]

[0035] wherein the fourth optimization term is to optimize the label of the target domain sample set under the shared feature space.

[0036] In an embodiment of the present application, step S3 comprises:

[0037] inputting the source domain sample set and the target domain sample set into the transfer learning model;

[0038] the source domain sample set and the target domain sample set are randomly iteratively extracted by using a random oversampling algorithm through the transfer learning model, so that the source domain sample set and the target domain sample set reach class balance;

[0039] the correlation between the features and the labels in the source domain sample set is measured based on a Hilbert-Schmidt independence criterion;

[0040] the internal relationship between the features and the labels of the source domain sample set under the class balance is obtained and saved according to the measurement result.

[0041] In an embodiment of the present application, the random oversampling algorithm comprises:

[0042] the source domain sample set and the target domain sample set are combined to form a data set;

[0043] the features of the data set are mapped in a reconstructed kernel Hilbert space, and the labels of the data set are mapped into a kernel Gram matrix;

[0044] the source domain and the target domain are kept in proportion balance in terms of the number of samples of different classes according to the randomly iteratively oversampled source domain samples and target domain samples by using the random oversampling method;

[0045] wherein the random oversampling method is represented as:

[0046]

[0047] l(·) is a loss function, represents that the feature of the ith sample in the c class of the source domain sample set and the target domain sample set is mapped into a kernel Gram matrix, represents that the label of the ith sample in the c class of the source domain sample set and the target domain sample set is mapped into a kernel Gram matrix, the label of the target domain sample set is represented by a pseudo label, and ns and nt represent the number of samples of the source domain sample set and the target domain sample set respectively, is the number of samples belonging to the c class in the source domain sample set, is the number of samples belonging to the c class in the target domain sample set, and θ is a weighting parameter.

[0048] In an embodiment of the present application, the Hilbert-Schmidt independence criterion is:

[0049]

[0050] Wherein, Xs is the feature of the source domain sample set, Ys is the label of the source domain sample set, λ is the weight parameter of HSIC, Ns is the sample number of the source domain sample set, Ks is the kernel Gram matrix of the feature mapping of the source domain sample set in the reconstructed kernel Hilbert space, Bs is the kernel Gram matrix of the label mapping of the source domain sample set in the reconstructed kernel Hilbert space, and E is a central matrix with a mean of 0.

[0051] The beneficial effects of the present application are: a remote sensing sample labeling method based on combined transfer learning is provided, the first transfer method is used to make the source domain sample set and the target domain sample set reach class balance, while the internal relationship between the features and the labels in the source domain sample set is saved, the second transfer method is used to obtain a shared feature space based on a feature tensor, the source domain sample set and the target domain sample set are simultaneously aligned to the shared feature space, new feature representations of the source domain sample set and the target domain sample set are obtained, and the labels of the target domain sample set are obtained according to the new feature representations of the target domain sample set and the saved internal relationship between the features and the labels in the source domain sample set. The labeling method controls the class balance of the source domain sample set and the target domain sample set, reduces the distribution difference of the classes of the source domain and the target domain, saves the internal relationship between the features and the labels in the source domain sample set, and simultaneously aligns the source domain sample set and the target domain sample set to the shared feature space, thereby relieving the problem of too large data distortion during transfer. The method can guarantee the effectiveness of transfer learning and obtain better labeling results when the source domain sample set containing rich prior knowledge is fully utilized. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings described in the following are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0053] Figure 1 The method flowchart of the remote sensing sample labeling based on combined transfer learning of the present application. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, but not all the embodiments. It should be noted that the various embodiments in the present application and the various features of each embodiment can be combined with each other, and the formed technical solutions are all within the protection scope of the present application.

[0055] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0056] Referring to Figure 1 The present application provides a remote sensing sample labeling method based on combined transfer learning, which comprises the following steps:

[0057] S1, obtaining a source domain sample set and a target domain sample set;

[0058] S2, constructing a transfer learning model, the transfer learning model is constructed by a combined transfer method, the combined transfer method comprises a first transfer method and a second transfer method;

[0059] S3, inputting the source domain sample set and the target domain sample set into the transfer learning model, so as to make the source domain sample set and the target domain sample set reach class balance by the first transfer method, and saving the internal relationship between the features and the labels in the source domain sample set;

[0060] S4, adapting the features of the source domain sample set and the features of the target domain sample set by the second transfer method, to obtain new feature representations of the source domain sample set and the target domain sample set;

[0061] S5, obtaining the labels of the target domain sample set according to the new feature representations of the target domain sample set and the internal relationship between the features and the labels in the source domain sample set.

[0062] It should be noted that there are two important concepts in transfer learning, domain and task. The domain is divided into source domain and target domain, and each specific domain is composed of two parts: feature space and marginal probability distribution. In a specific domain, the task is composed of label space and prediction function. Transfer learning is a framework for learning a new field (i.e. remote sensing images and their sample labeling information in the target domain) by transferring information from an old field with similarity (i.e. remote sensing images and their sample labeling information in the source domain). The transfer learning method mainly includes instance-based transfer learning, feature-based transfer learning, model-based transfer learning and relationship-based transfer learning. Feature-based transfer learning mainly combines feature extraction methods to map the source domain and the target domain to a new common feature space at the same time, so as to reduce the distribution difference between their corresponding new feature representations and improve the similarity between the source domain and the target domain.

[0063] It should be understood that in the transfer learning, the source domain is a field with a large amount of data annotation and complete experience, and is the object of knowledge transfer; the target domain is the object to obtain knowledge for annotation.

[0064] In the present application, the source domain sample set is a remote sensing image sample set containing label information, and the target domain sample set is a remote sensing image sample set without label.

[0065] The main idea of the present application is to construct a transfer learning model, which is constructed by a combined transfer learning method.

[0066] In the present application, the first transfer learning method is used to balance the source domain sample set and the target domain sample set, and to save the internal relationship between the features and the labels in the source domain sample set; the second transfer learning method is used to perform feature adaptation-based transfer learning on the source domain sample set and the target domain sample set, to obtain new feature representations of the source domain sample set and the target domain sample set. Then, the knowledge of the source domain sample set is transferred to the target domain sample set by training, to obtain the label information of the target domain sample set, and then the current remote sensing image is classified.

[0067] The transfer learning method of the present application is based on isomorphic feature space and label space, i.e. the feature space and the label space of the source domain sample set and the target domain sample set are the same, but the data distribution of the source domain sample set and the target domain sample set is different. The transfer learning model of the present application includes a target function, and the target function includes a first optimization term, a second optimization term and a third optimization term:

[0068]

[0069] Wherein, X is the feature space, Y is the label space, s refers to the source domain sample set, t refers to the target domain sample set, s→t refers to the source domain being balanced according to the class distribution, P is the data distribution of the sample set, H is the shared feature space of the source domain sample set and the target domain sample set, f is the mapping function of the feature of the source domain sample set to the shared feature space, f -1 is the approximate inverse function of f, k is the mapping function of the feature of the target domain sample set to the shared feature space, k -1 is the approximate inverse function of k.

[0070] From the above formula, it can be seen that the target function of the combined transfer learning is a reversible function, i.e. the transfer process is reversible.

[0071] The above target function includes three terms, the first optimization term is used to optimize the class balance of the source domain sample set and the target domain sample set, and the internal relationship between the features and the labels in the source domain sample set;

[0072] the second optimization term to optimize the new feature representation of the source domain sample set in the shared feature space;

[0073] the third optimization term to optimize the new feature representation of the target domain sample set in the shared feature space.

[0074] It should be noted that when the transfer learning is applied to remote sensing images, the class imbalance problem of the source domain sample set and the target domain sample set may occur. Moreover, due to the characteristics of remote sensing samples, the relationship between the features and the labels of the source domain sample set is important for the transfer learning.

[0075] Therefore, the first transfer learning method of the present application, i.e., the transfer learning method based on class balance.

[0076] The step S3 is specifically as follows:

[0077] The source domain sample set and the target domain sample set are input into the transfer learning model.

[0078] The labeled source domain sample set is represented as D s ={(x i ,y i )|i=1,...,N s},x i ∈R d×1 , and y i is the label of the corresponding source domain sample. The unlabeled target domain sample set is represented as D t ={(x i |x=1,...,N t}。

[0079] The random oversampling algorithm is used to randomly and iteratively extract the source domain sample set and the target domain sample set through the transfer learning model, so that the source domain sample set and the target domain sample set are balanced in classes.

[0080] The steps of the random oversampling algorithm include:

[0081] The source domain sample set and the target domain sample set are combined to form a data set;

[0082] The features of the data set are mapped in the reconstructed kernel Hilbert space, and the labels of the data set are mapped into a kernel Gram matrix K, as shown in the following formula:

[0083]

[0084] wherein K∈R N×N , N is the total number of samples of the source domain sample set and the target domain sample set, and U represents the reconstructed kernel Hilbert space. is a nonlinear function mapping the original source domain sample set data and target domain sample set data into the reconstructed kernel Hilbert space, xi and yj are the features of the i th and j th sample in the data set, respectively, and k(·) is a kernel function.

[0085] The labels of the data set are mapped in the reconstructed kernel Hilbert space, and the labels of the data set are mapped into a kernel Gram matrix B as shown in the following formula:

[0086]

[0087] where B ∈ R N×N , b(·) is a kernel function, yi and yj are the labels of the i th and j th sample in the data set, respectively, and the labels of the target domain sample set are represented by pseudo labels.

[0088] According to the random oversampling method, the oversampled source domain samples and target domain samples are iterated randomly, and the proportions of the sample numbers of different classes in the source domain and the target domain are kept balanced, and the random oversampling method is as shown in the following formula:

[0089]

[0090] where l(·) is a loss function, represents that the feature mapping of the i th sample in the c th class of the source domain sample set and the target domain sample set is a kernel Gram matrix, represents that the label mapping of the i th sample in the c th class of the source domain sample set and the target domain sample set is a kernel Gram matrix, the labels of the target domain sample set are represented by pseudo labels, ns and nt represent the sample numbers of the source domain sample set and the target domain sample set, respectively, is the sample number of the c th class in the source domain sample set, is the sample number of the c th class in the target domain sample set, and θ is a weighting parameter.

[0091] For the above loss function l(·), the invention adopts a square loss, and let n s +n t =N, Then the process of random oversampling is: the oversampled source domain samples and target domain samples are iterated randomly, and the proportions of the sample numbers of different classes in the source domain and the target domain are kept balanced. For example, let j be the j th iteration, and T be the total number of iterations, then the sample numbers of the source domain and the target domain obtained by oversampling in the j th iteration are n s ×j / T and n t ×j / T, respectively, and each class in the source domain and the target domain respectively keeps a balanced n s ×j / (T×m) and n t ×j / (T×m) sample numbers.

[0092] While maintaining class balance between the source and target domain sample sets, the correlation between features and labels of the source domain sample set is measured based on the Hilbert-Schmidt Independence Criterion (HSIC), as shown in the following formula:

[0093]

[0094] Where Xs are the features of the source domain sample set; Ys are the labels of the source domain sample set; λ is the trade-off parameter of HSIC; Ns is the number of samples in the source domain sample set; Ks is the kernel Gram matrix of the source domain sample set features mapped onto the reconstructed kernel Hilbert space; Bs is the kernel Gram matrix of the source domain sample set labels mapped onto the reconstructed kernel Hilbert space; and E is a central matrix with a mean of 0.

[0095] The smaller the HSIC value, the greater the dependency between the features and labels of the source domain sample set. In this invention, the aforementioned intrinsic relationship can be characterized by the magnitude of the dependency between the features and labels of the source domain sample set.

[0096] Based on the measurement results (i.e., the correlation between features and labels of the source domain sample set), the intrinsic relationship between features and labels of the source domain sample set under class balance is obtained and saved.

[0097] Based on the class balance of the source domain sample set and the target domain sample set, this invention performs feature adaptation on the source domain sample set and the target domain sample set through a second transfer learning method. Step S4 specifically includes:

[0098] S41 obtains the feature tensor of the source domain sample set and the feature tensor of the target domain sample set based on the features of the source domain sample set and the feature tensor of the target domain sample set, respectively. A shared feature space is obtained by learning from the feature tensors of the source domain sample set and the feature tensor of the target domain sample set.

[0099] S42 aligns the source domain sample set and the target domain sample set into a shared feature space.

[0100] S43 obtains new feature representations of the source domain sample set and the target domain sample set in the shared feature space based on the objective function of the transfer learning model.

[0101] The specific steps are as follows:

[0102] Let each sample in the source domain It is a K-order tensor, with a total of Ns source domain samples stacked to form a (K+1)-order tensor. And the source domain sample set has labels. The label is recorded as Similarly, Nt The target domain samples are stacked into a (K+1) order tensor. Where n i =mi (i = 1, 2,..., K).

[0103] The source domain sample set and the target domain sample set are respectively aligned to learn the shared feature space. A likelihood function is constructed based on the process of respectively aligning the shared feature space of the source domain sample set and the target domain sample set, and the formula is as shown in the following formula:

[0104]

[0105] Wherein, f is a mapping function of the feature of the source domain sample set mapped into the shared feature space, f 1 is the approximate inverse function of f, k is a mapping function of the feature of the target domain sample set mapped into the shared feature space, k -1 is the approximate inverse function of k, F is the Frobenius norm, χ s is the feature tensor of the source domain sample set, χ t is the feature tensor of the target domain sample set.

[0106] A first alignment matrix set and a second alignment matrix set are constructed. A shared feature space H = {H (i)} i＝1，...，K+1 , and the feature core tensor of the source domain sample set and the target domain sample set under the shared feature space are introduced.

[0107]

[0108]

[0109] Wherein,

[0110] G s = H T M1χ s = f(χ s ) (8)

[0111] G t = H T M2χ t = k(χ t ) (9)

[0112] According to the above formula, the data of the source domain sample set and the target domain sample set are aligned to the shared feature space by using the first alignment matrix set, the second alignment matrix set, the feature core tensor and the likelihood function.

[0113] After aligning the source domain sample set and the target domain sample set to the shared feature space simultaneously, new feature representations of the source domain sample set and the target domain sample set can be obtained, as shown in the following formula:

[0114]

[0115] Assuming that the probability distribution of mapping from the source domain to the shared feature space is P(h|x s )=δ(f(x s )-h), and assuming that the inverse distribution is P(x|h)=N((f -1 (x s ));h,1), the following formula can be obtained:

[0116]

[0117] Then:

[0118]

[0119] From the derivation process of the above probability distribution, by optimizing formula (1) through formula (6), according to the maximum likelihood estimation algorithm, the objective function of the transfer learning model is optimized by minimizing the likelihood function, and when the likelihood function of formula (6) decreases, the in formula (1) will be raised.

[0120] Based on the optimized objective function of the transfer learning model, new feature representations X′ s and X′ t of the source domain sample set and the target domain sample set in the shared feature space are obtained. According to the internal relationship between the features and the labels in the source domain sample set, the labels of the target domain sample set are obtained, that is, the annotation information of the target domain sample set.

[0121] The objective function of the combined transfer learning method can also include a fourth optimization term, as shown in the following formula:

[0122]

[0123] The fourth optimization term is to optimize the labels of the target domain sample set in the shared feature space. This term is optimized based on the internal relationship between the features and the labels of the source domain sample set, and can improve the annotation accuracy of the target domain sample set.

[0124] The application has the beneficial effects that a remote sensing sample labeling method based on combined transfer learning is provided, the first transfer method is used to make the source domain sample set and the target domain sample set reach class balance, while the internal relationship between the features and the labels in the source domain sample set is saved, the second transfer method obtains a shared feature space based on a feature tensor, makes the source domain sample set and the target domain sample set align the shared feature space at the same time, obtains new feature representations of the source domain sample set and the target domain sample set, and obtains the labels of the target domain sample set according to the new feature representations of the target domain sample set and the saved internal relationship between the features and the labels in the source domain sample set. The labeling method controls the class balance of the source domain sample set and the target domain sample set, reduces the distribution difference of the classes of the source domain and the target domain, saves the internal relationship between the features and the labels in the source domain sample set, and makes the source domain sample set and the target domain sample set align the shared feature space at the same time, so that the problem of too large data distortion in transfer is alleviated. When the source domain sample set containing rich prior knowledge is fully utilized, the effectiveness of transfer learning can be ensured, and better labeling results are obtained.

[0125] The above is only a specific embodiment of the application, but the protection scope of the application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the application, which should be covered within the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. A remote sensing sample annotation method based on ensemble transfer learning, characterized in that, include: S1 obtains the source domain sample set and the target domain sample set; S2 constructs a transfer learning model, which is built using a combined transfer method, including a first transfer method and a second transfer method. S3 inputs the source domain sample set and the target domain sample set into the transfer learning model to achieve class balance between the source domain sample set and the target domain sample set using the first transfer method, and preserves the intrinsic relationship between features and labels in the source domain sample set. S4 uses the second transfer method to adapt the features of the source domain sample set and the target domain sample set to obtain new feature representations of the source domain sample set and the target domain sample set; S5 obtains the labels of the target domain sample set based on the new feature representation of the target domain sample set and the inherent relationship between the features and labels in the source domain sample set; Step S3 includes: inputting the source domain sample set and the target domain sample set into the transfer learning model; using the transfer learning model, employing a random oversampling algorithm to randomly and iteratively extract the source domain sample set and the target domain sample set to achieve class balance; measuring the correlation between features and labels in the source domain sample set based on the Hilbert-Schmidt independence criterion; obtaining and saving the intrinsic relationship between features and labels in the source domain sample set under class balance based on the measurement results; wherein, the random oversampling algorithm includes: forming a dataset from the source domain sample set and the target domain sample set; mapping the features of the dataset to the reconstructed kernel Hilbert space, and mapping the labels of the dataset to the kernel Gram matrix; randomly and iteratively oversampling source domain samples and target domain samples according to the random oversampling method to maintain a balanced ratio of samples of different categories in the source domain and the target domain; Step S4 includes: S41 obtaining the feature tensors of the source domain sample set and the target domain sample set based on the features of the source domain sample set and the target domain sample set respectively, and learning a shared feature space through the feature tensors of the source domain sample set and the target domain sample set; S42 aligning the source domain sample set and the target domain sample set to the shared feature space; S43 obtaining new feature representations of the source domain sample set and the target domain sample set in the shared feature space according to the objective function of the transfer learning model; The transfer learning model includes an objective function, which comprises a first optimization term, a second optimization term, and a third optimization term: Wherein, X is a feature space, Y is a label space, s refers to a source domain sample set, t refers to a target domain sample set, s→t refers to that the source domain is balanced according to the class distribution, P is the data distribution of the sample set, H is a shared feature space of the source domain sample set and the target domain sample set, f is a mapping function of the feature mapping of the source domain sample set into the shared feature space, f -1 is an approximate inverse function of f, k is a mapping function of the feature mapping of the target domain sample set into the shared feature space, k -1 is an approximate inverse function of k; Among them, the first optimization term To optimize the class balance between the source and target domain sample sets and the inherent relationship between features and labels in the source domain sample set; the second optimization term... To optimize the new feature representation of the source domain sample set in the shared feature space; the third optimization term To optimize the new feature representation of the target domain sample set in the shared feature space.

2. The remote sensing sample annotation method based on ensemble transfer learning according to claim 1, characterized in that, Step S42 includes: The likelihood function is constructed based on the process of aligning the source domain sample set and the target domain sample set to the shared feature space. where f is a mapping function of the feature of the source domain sample set mapped into the shared feature space, f -1 is an approximate inverse function of f, k is a mapping function of the feature of the target domain sample set mapped into the shared feature space, k -1 is an approximate inverse function of k, F is a Frobenius norm, χ s is a feature tensor of the source domain sample set, χ t is a feature tensor of the target domain sample set; Construct the first alignment matrix set and the second alignment matrix set, as well as the feature core tensors of the source domain sample set and the target domain sample set under the shared feature space; By using the first alignment matrix set, the second alignment matrix set, the feature core tensor, and the likelihood function, the source domain sample set and the target domain sample set are aligned to a shared feature space.

3. The remote sensing sample annotation method based on ensemble transfer learning according to claim 1, characterized in that, Step S43 includes: Based on the maximum likelihood estimation method, the objective function of the transfer learning model is optimized by minimizing the likelihood function; Based on the objective function of the optimized transfer learning model, new feature representations of the source domain sample set and the target domain sample set are obtained under the shared feature space.

4. The remote sensing sample annotation method based on ensemble transfer learning according to claim 1, characterized in that, The objective function of the transfer learning model also includes a fourth optimization term: Among them, the fourth optimization term To optimize the labels of the target domain sample set under a shared feature space.

5. The remote sensing sample annotation method based on ensemble transfer learning according to claim 1, characterized in that, in, The random oversampling method is expressed as: l(·) is the loss function. This indicates that the feature mapping of the i-th sample in the source domain sample set and the target domain sample set of category c is a kernel-gram matrix. Let the label mapping of the i-th sample in each category of the source domain sample set and the target domain sample set be represented by a kernel-gram matrix, and let the labels of the target domain sample set be represented by pseudo-labels, n s and n t Let represent the number of samples in the source domain sample set and the target domain sample set, respectively. It is the number of samples in the source domain sample set that belong to category c. θ is the number of samples belonging to category c in the target domain sample set, and θ is the trade-off parameter.

6. The remote sensing sample annotation method based on ensemble transfer learning according to claim 1, characterized in that, The Hilbert-Schmidt independence criterion is as follows: where X s is the feature of the source domain sample set, Y s is the label of the source domain sample set, λ is the trade-off parameter of HSIC, N s is the number of samples of the source domain sample set, K s is the kernel Gram matrix of the feature mapping of the source domain sample set in the reproducing kernel Hilbert space, B s is the kernel Gram matrix of the label mapping of the source domain sample set in the reproducing kernel Hilbert space, and E is a centering matrix with mean 0.

Images