Common-knowledge-constrained remote sensing sample migration method

Abstract

The application discloses a common knowledge constraint remote sensing sample migration method and relates to the field of remote sensing image processing. The method comprises the following steps: using a feature extraction model to obtain the features and the public feature space of source domain samples and target domain samples; using a feature clusterer to determine common features and non-common features; inputting the common features and the source domain non-common features with random noise into a generator to generate pseudo samples, inputting the pseudo samples into a discriminator, discriminating the pseudo samples according to the target domain sample data, iteratively training and optimizing the generator, and obtaining the trained generator; and inputting the source domain sample data into the trained generator to generate migration samples. The method realizes the sample migration from the source domain to the target domain under the constraint of common features and avoids the occurrence of negative migration.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of remote sensing image processing, and particularly relates to a remote sensing sample migration method based on common knowledge constraints. BACKGROUND

[0002] In recent years, the rapid development of remote sensing technology has promoted the wide application of remote sensing technology in various fields. Among them, the real-time monitoring of multiple satellites provides massive multi-element remote sensing image data support for the development of the entire remote sensing field, laying the foundation for the rapid development of remote sensing technology. Effective use of massive remote sensing image data is one of the important directions of the development of the remote sensing field.

[0003] Artificial intelligence technology, as one of the hottest high-techs today, has a huge development prospect. In fact, the feasibility of artificial intelligence technology is largely dependent on the support of massive data and big data, so artificial intelligence technology can also effectively use massive data to achieve various functions.

[0004] Using artificial intelligence technology in the field of remote sensing can greatly improve the utilization rate of massive remote sensing image data. However, most of the current artificial intelligence applications in the field of remote sensing use supervised learning or semi-supervised learning methods, which cannot directly use massive remote sensing image data and need to rely on labeled remote sensing samples. It is difficult to obtain labeled remote sensing samples, and currently, in addition to the method of manually labeling samples which has high precision but high labor cost and low efficiency, there are also researches on using machine learning methods to label samples, but they still do not reach the standard of large-scale engineering implementation. Therefore, how to maximize the effective use of existing remote sensing labeled samples is one of the current research directions.

[0005] The transfer learning method in remote sensing samples can effectively improve the utilization rate of existing samples. The purpose of transfer learning is to use sufficient labeled samples in the source domain for a small number of samples or no samples in the target domain, and there is partial correlation or no correlation between the sample features of the source domain and the target domain. An existing transfer method, such as TrAdaBoost, optimizes the source domain sample data with high correlation for transfer learning, which improves the performance of transfer learning. However, for this method of optimizing source domain samples, there are mainly the following problems: even for the optimized samples, not all features in the samples are beneficial to transfer learning, and some features with low correlation will have a negative impact on transfer learning, and even cause negative transfer, which needs to ensure that the source domain and the target domain are sufficiently correlated. SUMMARY

[0006] The application provides a common knowledge constraint remote sensing sample migration method, which can solve the problems of the prior art, train a generator based on a target function by directly inputting common features of source domain sample data and target domain sample data into the constructed generator without affecting the distribution of the source domain sample data, and obtain the generator under common knowledge constraint. The source domain sample data is input into the generator under common knowledge constraint to generate sample data fitting the target domain, sample migration under common knowledge constraint is realized, and negative migration is effectively avoided. Meanwhile, the whole sample migration process is end-to-end, and automatic adjustment of the model can be realized.

[0007] To achieve the above technical purposes, the technical scheme of the application is as follows:

[0008] A common knowledge constraint remote sensing sample migration method, which comprises the following steps:

[0009] S1 inputting source domain sample data and target domain sample data into a feature extraction model to obtain feature data and a common feature space of the source domain sample data and the target domain sample data;

[0010] S2 inputting the feature data of the source domain sample data and the target domain sample data into a feature cluster to determine a common feature space and a non-common feature space, extract common features and source domain non-common features, wherein the common feature space and the non-common feature space are subspaces of the common feature space;

[0011] S3 inputting the common features and source domain non-common features with random noise into a generator to generate pseudo samples;

[0012] S4 inputting the pseudo samples and target domain sample data into the discriminator to discriminate the pseudo samples according to the target domain sample data, and optimizing the generator according to a discrimination result and a target function;

[0013] S5 iterating the training process of S3 to S4 until the target function converges;

[0014] S6 inputting the source domain sample data into the trained generator to generate migration samples.

[0015] Optionally, in step S2, inputting the feature data of the source domain sample data and the target domain sample data into the feature cluster to determine the common feature space and the non-common feature space comprises:

[0016] clustering the source domain sample data and the target domain sample data on the common feature space to obtain k groups of mixed sample data x i ,i=1,...,k, wherein each group of mixed sample data contains feature correlation;

[0017] The mixed sample data x i is mapped to a plurality of feature subspaces F j of the common feature space, obtaining a sample-feature set (x i , F j );

[0018] The distribution of (x i , F j ) is analyzed.

[0019] If the distribution has correlation, the feature subspace F j is divided into the common feature space; if the distribution has no correlation, the feature subspace F j is divided into the non-common feature space.

[0020] Optionally, the determination of whether the distribution has correlation is as follows:

[0021] The fitting degree of the feature distribution of each group of mixed samples in the same feature subspace F j to the feature distribution of other groups of mixed samples is calculated using a probability distribution distance measurement algorithm.

[0022] According to the fitting degree, the overall fitting degree of the feature (x i , F j ) of each group of mixed samples in the same feature subspace F j is obtained.

[0023] The number M of mixed samples with an overall fitting degree greater than a first preset threshold is counted, and when M is greater than or equal to a second preset threshold, it is determined that the feature subspace has correlation.

[0024] Optionally, the fitting degree is calculated according to a probability distribution distance measurement algorithm.

[0025] Optionally, the feature extraction model is as follows:

[0026] a machine learning model using a feature extraction operator;

[0027] or a model constructed by a convolutional neural network;

[0028] or a combined model of the machine learning model and the model constructed by the convolutional neural network.

[0029] Optionally, the feature extraction model is an encoder part of a convolutional autoencoder constructed by a convolutional neural network.

[0030] Correspondingly, the generator is a decoder part of the convolutional autoencoder.

[0031] Optionally, the structure of the feature extraction model and the generator is symmetrical.

[0032] Optionally, the generator and the discriminator constitute a generative adversarial network, and the target function is a target function of the generative adversarial network, and the target function is:

[0033]

[0034] Wherein G is a generator, D is a discriminator, E is an expectation function, x is a pseudo sample data generated by the generator, p data is a probability that x comes from a real data distribution, p g is a probability that x comes from a generator output sample.

[0035] The present application proposes a common knowledge constraint remote sensing sample migration method, through feature extraction on source domain and target domain sample data, corresponding feature data of the sample data is obtained, through clustering analysis on the feature data, common features and non-common features of the source domain and target domain samples are obtained, by inputting the common features and the source domain noisy non-common features into the generator, pseudo samples under the constraint of common features are generated, by inputting the pseudo samples and the target domain samples into the discriminator, the generator is iteratively trained and optimized according to the discrimination result and the target function, the generator capable of migrating the source domain features to the target domain is obtained, finally, the source domain samples are input into the generator, and the migration samples fitting the target domain can be directly obtained. The beneficial effects of the present application are:

[0036] (1) Under the technical support of the present application, the source domain sample data can be input into the generator under the constraint of common knowledge, and sample data fitting the target domain can be generated without affecting the distribution of the source domain sample data, so that sample migration under the constraint of common knowledge is realized, and negative migration is effectively avoided.

[0037] (2) The sample migration framework constructed by the present application supports full-automatic training and adjustment of the model, and realizes an end-to-end sample migration process. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0039] Figure 1 It is a flowchart of an embodiment of the common knowledge constraint remote sensing sample migration method of the present application.

[0040] Figure 2 It is a schematic diagram of sample migration model data transmission in an embodiment of the common knowledge constraint remote sensing sample migration method of the present application.

[0041] Figure 3 FIG. 1 is a schematic diagram of a remote sensing sample migration method according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0043] Please refer to Figure 1 , Figure 1 FIG. 1 is a schematic diagram of a remote sensing sample migration method according to an embodiment of the present application. Compared with the traditional transfer learning mode, the method realizes the generation of samples fitting the target domain by inputting source domain samples, realizes the sample migration under the constraint of common knowledge, and effectively avoids negative transfer. The method comprises the following steps:

[0044] S1 inputting source domain sample data and target domain sample data into a feature extraction model to obtain feature data of the source domain sample data and the target domain sample data and a public feature space.

[0045] It should be noted that the embodiments of the present application are aimed at the migration task of remote sensing samples, belong to the category of homogeneous transfer learning, and the source domain and target domain sample data are in the same feature space, i.e. the public feature space.

[0046] Optionally, the feature extraction model is a machine learning model using a feature extraction operator, or a convolutional neural network model, or a combination model of the above models.

[0047] In this embodiment, the encoder part of the convolutional autoencoder constructed using the convolutional neural network is used as the feature extraction model. The autoencoder is a powerful feature detector, which can efficiently represent the learned input data as features through unsupervised learning. The encoder of the convolutional autoencoder adopts a three-layer convolutional neural network structure. The first layer has 16 convolutional kernels with a size of 3x3 and a step of 1. The second layer has 8 convolutional kernels with a size of 3x3 and a step of 1. The third layer has 8 convolutional kernels with a size of 3x3 and a step of 1. A 2x2 max-pooling layer is connected after each convolutional layer for dimension reduction and compression of the features.

[0048] S2 inputs feature data of the source domain sample data and the target domain sample data into a feature clusterer, determines a common feature space and a non-common feature space, extracts common features and source domain non-common features, wherein the common feature space and the non-common feature space are subspaces of the common feature space.

[0049] It should be noted that the source domain sample data and the target domain sample data are both in the common feature space, but the specific distribution is different. The feature data can be divided into common features and non-common features according to the data distribution. The subspace where the common features are located is the common feature space, and the subspace where the non-common features are located is the non-common feature space.

[0050] In this embodiment, the source domain sample data and the target domain sample data are divided into k groups on the common feature space through co-clustering processing, and k groups of mixed sample data x i ,i=1,...,k are obtained. Each group of mixed sample data has feature correlation. The sample feature values are normalized for subsequent analysis and input into the generative adversarial network. The mixed sample data x i is mapped to a plurality of feature subspaces F j , and sample-feature sets (x i ,F j ) are obtained. The distributions of (x1,F j ), (x2,F j ),..., (x k ,F j ) are analyzed. If the distribution has correlation, the feature subspace Fj is divided into the common feature space; if the distribution does not have correlation, the feature subspace Fj is divided into the non-common feature space.

[0051] The determination method of the distribution having correlation is as follows:

[0052] The fitting degree of the feature distribution of each group of mixed samples in the same feature subspace F j and other groups of mixed samples is calculated.

[0053] According to the fitting degree, the overall fitting degree of each group of mixed sample features (x i ,F j ) in the same feature subspace F j is obtained.

[0054] The number M of mixed samples with an overall fitting degree greater than a first preset threshold is counted. When M is greater than or equal to a second preset threshold, it is determined that the feature subspace has correlation.

[0055] Specifically, the steps of performing correlation analysis on the feature distribution of the mixed samples in F1 are as follows: using a probability distribution distance measurement algorithm to calculate the fitting degree of the feature distribution of each group of mixed samples in F j to the feature distribution of other groups of mixed samples, and obtaining the same feature subspace F j according to the fitting degree. i The fitting degree of each group of mixed sample features (x j ,F i ) in F j is calculated, the number M of mixed samples with a total fitting degree greater than a first preset threshold is counted, and when M is greater than or equal to a second preset threshold, it is determined that the feature subspace has correlation. The fitting degree can be represented by KL divergence, and the smaller the KL divergence, the higher the fitting degree. The total fitting degree is a set of KL divergences. The second preset threshold can be 80% of the total number, or other parameters, which are not limited in the embodiment. For example, the KL divergence of each two groups of mixed sample data in (x , F1) is calculated to obtain KL divergence values, and the KL divergence values are compared with the preset threshold. If more than 80% of the KL divergence values correspond to a total fitting degree greater than the first preset threshold, it is determined that the feature subspace F1 has correlation. If the distribution has correlation, the feature subspace F1 is divided into the common feature space, and if the distribution does not have correlation, the feature subspace F1 is divided into the non-common feature space. According to the method, all feature subspaces F j are divided into the common feature space or the non-common feature space.

[0056] It can be understood that the source domain and the target domain sample data have relevant features and irrelevant features in the same feature space. The use of the co-clustering method realizes the clustering of the sample data and the feature data in the same feature space at the same time, and can intuitively reflect the relationship between the source domain and the target domain sample data and the features. The present application aims to distinguish the relevant features and the irrelevant features and divide them into the common feature space and the non-common feature space, respectively. The distribution of the normalized feature values extracted from the sample in the same feature subspace reflects the characteristics of the sample in the feature subspace. In the same common feature subspace, the source domain sample data and the target domain sample data should have consistent distribution, and different groups of data should also have similar distribution. Therefore, by analyzing the correlation between the feature distributions of different groups of data in the same feature subspace, the feature subspace can be divided into the common feature space and the non-common feature space.

[0057] With the technical support of the present application, sample migration under common knowledge constraints can be realized without affecting the distribution of the source domain sample data, effectively avoiding negative transfer.

[0058] S3 inputs the common feature and the source domain non-common feature with random noise into a generator to generate a pseudo sample.

[0059] S4 inputs the pseudo sample and target domain sample data into the discriminator to discriminate the pseudo sample according to the target domain sample data, and optimizes the generator according to a discrimination result and a target function.

[0060] S5 repeats the training process of S3 to S4 until the target function converges.

[0061] It should be noted that the generator used in the present application is symmetrical to the feature extraction model structure for feature extraction, and the purpose is to be able to directly input the extracted common feature into the generator to generate a sample with minimal loss. The entire transfer learning model and data transmission of an embodiment of the present application are as shown in Figure 2 The common feature is directly input into the generator to generate a sample, which can effectively realize the constraint of the common feature and ensure that the generated sample fits the target domain sample as much as possible under the premise that the common feature is unchanged.

[0062] In the present embodiment, the decoder part of the convolutional autoencoder is used as the generator, and a decoder structure symmetrical to the encoder structure is adopted. The decoder adopts a three-layer transposed convolutional neural network structure. The first layer has 8 convolutional kernels with a kernel size of 3x3 and a step size of 1. The second layer has 8 convolutional kernels with a kernel size of 3x3 and a step size of 1. The third layer has 16 convolutional kernels with a kernel size of 3x3 and a step size of 1. A 2x2 up-sampling layer is connected after each transposed convolutional layer to restore the image size.

[0063] The same autoencoder is used in the present embodiment, which improves the reusability of the model and reduces the cost of model construction. The generator and the discriminator constitute a non-traditional generative adversarial network with features as input, and the target function of the generative adversarial network is where G is the generator, D is the discriminator, E is the expectation function, x is the pseudo sample data generated by the generator, p data is the probability that x comes from the real data distribution, and p g is the probability that x comes from the generator output sample. The common feature and the source domain non-common feature with random noise are normalized and input into the decoder part of the convolutional autoencoder to generate a pseudo sample. The pseudo sample and the target domain sample data are input into the discriminator to discriminate the pseudo sample according to the target domain sample data, and the entire generative adversarial network is optimized according to the discrimination result and the target function. The above training process is repeated until the target function converges.

[0064] S6 inputs the source domain sample data into the trained generator to generate a transfer sample.

[0065] It can be understood that the trained generator can generate sample data fitting the target domain by receiving source domain sample data, realize sample migration, and the sample migration process in the embodiment is as shown in the following figure. Figure 3 As shown in the figure, the source domain sample can generate a migration sample fitting the target domain after the self-encoder.

[0066] The sample migration framework constructed by the application supports full-automatic training and adjustment of the model, realizes an end-to-end sample migration process, and automatically generates a sample fitting the target domain after inputting the source domain sample.

[0067] It should be noted that in this paper, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including one" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0068] The above only describes the preferred embodiments of the application and is not intended to limit the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A method for remote sensing sample transfer constrained by common knowledge, characterized in that, The method comprises the following steps: S1 inputting source domain sample data and target domain sample data into a feature extraction model to obtain feature data of the source domain sample data and the target domain sample data and a common feature space; S2 inputting the feature data of the source domain sample data and the target domain sample data into a feature clusterer to determine a common feature space and a non-common feature space, extract common features and source domain non-common features, wherein the common feature space and the non-common feature space are subspaces of the common feature space; S3 inputting the common features and source domain non-common features with random noise into a generator to generate pseudo samples; S4 inputting the pseudo samples and target domain sample data into a discriminator to discriminate the pseudo samples according to the target domain sample data, and optimizing the generator according to a discrimination result and a target function; S5 iterating the training process of S3 to S4 until the target function converges; S6 inputting the source domain sample data into the trained generator to generate a migration sample.

2. The method of claim 1, wherein the common knowledge constraint is a constraint on a number of times a remote sensing sample is migrated. In step S2, inputting the feature data of the source domain sample data and the target domain sample data into a feature clusterer to determine a common feature space and a non-common feature space comprises: Clustering the source domain sample data and the target domain sample data on the public feature space obtains k groups of mixed sample data x i i = 1, …, k, wherein each group of mixed sample data contains feature correlation. mapping the mixed sample data x i to a plurality of feature subspaces F j of the common feature space, obtaining a sample-feature set (x i , F j ); Analysis of the distribution of (x i , F j ) If the distribution has correlation, then the feature subspace F j is partitioned into a common feature space; if the distribution has no correlation, then the feature subspace F j is partitioned into a non-common feature space.

3. The method of claim 2, wherein the common knowledge constraint is a constraint on a number of times a remote sensing sample is migrated. The discrimination mode of the distribution with correlation is: Computing the same feature subspace F j The degree of fitting of each group of mixed samples with the feature distribution of other groups of mixed samples; According to the fitting degree, the same feature subspace F j The overall fitting degree of each group of mixed sample features (x i ,F j ) The number M of mixed samples with a fitting degree greater than a first preset threshold is counted, and when M is greater than or equal to a second preset threshold, it is determined that the feature subspace has a distribution with correlation.

4. The method of claim 3, wherein the common knowledge constraint is a constraint on a number of times a remote sensing sample is migrated. The fitting degree is calculated according to a probability distribution distance measurement algorithm.

5. The method of claim 1, wherein the common knowledge constraint is a constraint that is common to all of the remote sensing samples. The feature extraction model is: a machine learning model using a feature extraction operator; or a model constructed by a convolutional neural network; or a combined model of the machine learning model and the model constructed by the convolutional neural network.

6. The method of claim 5, wherein the common knowledge constraint is a constraint that is common to all of the remote sensing samples. The feature extraction model is an encoder part of a convolutional autoencoder constructed by a convolutional neural network. Correspondingly, the generator is a decoder part of the convolutional autoencoder.

7. The method of claim 6, wherein the common knowledge constraint is a constraint that is common to all of the remote sensing samples. The structure of the feature extraction model and the generator is symmetrical.

8. The method of claim 1, wherein the common knowledge constraint is a constraint that is common to all of the remote sensing samples. The generator and the discriminator constitute a generative adversarial network, and the target function is a target function of the generative adversarial network, which is: where G is the generator, D is the discriminator, E is the expectation function, x is the pseudo sample data generated by the generator, p data is the probability that x comes from the real data distribution, p g is the probability that x comes from the generator output sample.

Images