A spatial target sample augmentation method based on generative adversarial network

Abstract

The present invention discloses a spatial target sample augmentation method based on a generative adversarial network, and the steps are as follows: analyzing the correlation of the invariant characteristics of the spatial target, classifying its invariant features and differential features, and providing optional samples for the generative adversarial network; analyzing the reasons why the original loss function is prone to mode collapse, using a fidelity function to replace the original loss function of the generative adversarial network, and generating samples for the target; based on the natural evolution method, converting the generation process of the generator into a mutation process, and modifying the original single generator into three heterogeneous generators to be evolved; setting up a fitness function to self-optimize the spatial target sample, and inputting the optimized target sample into a discriminator. The present invention improves the original loss function of the generative adversarial network into a fidelity-based loss function, which suppresses the non-gradient guidance of the discriminator to the generator.

Description

Technical Field

[0001] The present invention belongs to the technical field of space intelligent manipulation, and specifically relates to a space target sample augmentation method based on a generative adversarial network. Background Art

[0002] The high recognition rate based on deep neural networks requires a huge and rich dataset as support. The recognition model gradually learns the feature information required for image recognition by training on large datasets, and finally completes the target recognition task by combining high-level features. However, in aerospace missions such as space debris removal, rendezvous and docking, and on-orbit assembly, the application of deep learning methods to identify space targets will face the problem of a sharp drop in recognition accuracy. The main reason is the lack of data for training space target recognition models. Unlike the collection of datasets for conventional ground targets, the state of space targets is unstable, and some may be spacecraft from other parties, so it is impossible to obtain a wide variety of targets as training samples. Even if a high price is paid to obtain the training dataset, with the increasing number of new targets such as failed satellites, space debris, and rocket last stage debris, the dataset cannot be effectively updated.

[0003] To expand the sample set of space targets, traditional methods generally use methods such as image rotation, shearing, blurring, and scaling, but the training of the augmented data set can easily lead to model overfitting. Some traditional methods also use simulated data to train models, but the simulated training data samples generated under ground test conditions are relatively single and limited, and it is difficult to fully simulate the actual space situation, resulting in the trained model being difficult to adapt to the real space environment. Sample generation aims to generate more images of the same type based on a small number of images of the same type, which can be used to expand existing training data and thus enhance the task of space target recognition. With the development of generative adversarial networks, generative models have begun to be used for data augmentation, but the model is prone to collapse during the training phase, and the samples in the generation phase are not realistic enough and have a single pattern.

[0004] Therefore, in order to solve problems such as space target recognition under scarce sample conditions, it is necessary to study the generation technology for scarce cognitive samples in order to expand the space target dataset. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a spatial target sample augmentation method based on a generative adversarial network to solve the problems of single pattern and low sample quality in the existing sample augmentation methods. The present invention can effectively improve the quality of generated samples, reduce pattern collapse and inhibit gradient dissipation.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A spatial target sample augmentation method based on a generative adversarial network of the present invention comprises the following steps:

[0008] 1) Analyze the correlation of the invariant characteristics of spatial targets, classify their invariant features and differential features, and provide optional samples for the generative adversarial network;

[0009] 2) Analyze the reasons why the original loss function is prone to mode collapse, use the fidelity function to replace the original loss function of the generative adversarial network, and generate samples for the target;

[0010] 3) Based on the natural evolution method, the generation process of the generator is transformed into a mutation process, and the original single generator is modified into three heterogeneous generators to be evolved; a fitness function self-optimization space target sample is established, and the optimized target sample is input into the discriminator.

[0011] Furthermore, the invariant characteristic correlation analysis of the space target in step 1) specifically includes:

[0012] 11) Sample similarity analysis: The samples of spatial targets are augmented, requiring the data distribution of the target domain and the source domain to be as similar as possible, so as to ensure that the performance of the recognition task is as similar as possible when the classifier remains unchanged;

[0013] 12) Feature similarity analysis: Before sample augmentation, extract original or additional features from the original sample set, remove features that are greatly affected by migration conversion factors, and select a set of unaffected features as input features for sample augmentation;

[0014] 13) Model similarity analysis: Considering the situation where there are insufficient training samples in the target domain, the classification model is trained in the source domain and the data distribution is kept unchanged, so that the classifier can adapt to the data distribution of the target domain.

[0015] Furthermore, the step 2) is specifically as follows:

[0016] 21) Mode collapse analysis: Through the definition of KL divergence and JS divergence, the defects of the original loss function of the generative adversarial network are analyzed, and it is proved that the original loss function is easy to cause the training process of the generative adversarial network to fall into mode collapse, which is not conducive to the generation of spatial target samples;

[0017] 22) Define a fidelity function and replace the original single variable output of the generative adversarial network discriminator with the output about the fidelity distribution.

[0018] Furthermore, the step 21) is specifically as follows:

[0019] The generative adversarial network consists of two parts: the generator G and the discriminator D. The generator takes random noise z~p(z) as input and outputs the generated sample distribution pg (x); The discriminator uses the real sample distribution p of the sample set data (x) is the input, and the output is the generated sample distribution p g (x) and the true sample distribution p data (x) The difference between them; the original formal adversarial network satisfies the following:

[0020]

[0021] Among them, D(x) represents the output of the discriminator D for the true sample, D(G(z)) represents the output of the discriminator D for the fake sample, and G(z) represents the sample generated by the generator G. Indicates that sample x belongs to the true sample distribution p data (x) expectations, represents the mathematical expectation of the false samples generated by the noise data z~p(z), Indicates that x belongs to the generated sample distribution p g (x); the loss function of the discriminator D is defined as:

[0022]

[0023] The loss function of the generator G is:

[0024]

[0025] The condition for the entire adversarial network to reach Nash equilibrium is:

[0026]

[0027] When p data (x) = p g (x), D * (x) = 0.5 represents the optimal output of the discriminator, p data (x) represents the true sample distribution, p g (x) represents the generated sample distribution;

[0028] Define p data (x) and p g The KL divergence of (x) is:

[0029]

[0030] Where KL(·) represents the KL divergence; define p data (x) and p g JS divergence formula for (x):

[0031]

[0032] where JS(·) represents the JS divergence; when p data (x) = p g (x), then the original loss function Transformed into:

[0033]

[0034] in, Indicates that sample x belongs to the true sample distribution p data (x) expectations, Represents the mathematical expectation of the false samples generated by the noise data z~p(z); Bring the KL divergence and JS divergence into Then we get:

[0035]

[0036] Therefore, when the generative adversarial network is trained using the original loss function, when p data (x) = p g (x),

[0037] At this time, the output of the loss function is a constant value -2log2, that is, the gradient is 0, which ultimately results in the discriminator D having no gradient guidance for the generator G, causing the generative adversarial network model to collapse.

[0038] Furthermore, the step 22) specifically includes:

[0039] In the original loss function, the single scalar 1 or 0 output by the discriminator D easily leads to the loss function output -2log2. This problem can be suppressed by converting the single scalar output of the discriminator D into a multivariate output.

[0040] Using realistic distribution p r As the output of the discriminator D, for any given sample x, the discriminator D satisfies Ω is p r The set of u is the fidelity evaluation standard function; define the true distribution as R1 and the false distribution as R0 to replace the original 1 and 0; then the generative adversarial network becomes:

[0041]

[0042] Among them, D KL (·) represents the KL divergence between the two distributions, D(x) represents the output of the discriminator D for the true sample, Indicates that sample x belongs to the true sample distribution p data (x) expectations, Indicates that x belongs to the generated sample distribution p g(x), R0 represents the distribution of fake samples output by realistic distribution, and R1 represents the distribution of true samples output by realistic distribution;

[0043] The conditions for the adversarial network to reach Nash equilibrium based on fidelity are:

[0044]

[0045] Among them, D * (x,u) represents the distribution based on the realistic distribution p r The optimal output of the discriminator; when R1(u) = 1, R0(u) = 0, that is, the sample distribution p generated by the discriminator g (x) outputs 0. At this time, D * (x,u)=D * (x) = 0.5, that is, the loss function based on distribution degree will not change the original Nash equilibrium.

[0046] Furthermore, the step 3) specifically includes:

[0047] 31) Convert the generation process of the generator into a mutation process, and modify the original single generator into three heterogeneous generators to be evolved;

[0048] 32) Define the fitness function;

[0049] 33) Filter out the child targets that do not meet the requirements through the score of the fitness function; input the qualified samples into the discriminator D and use them as the initialization samples for the next network iteration.

[0050] Furthermore, the step 31) specifically includes:

[0051] Based on the natural evolution method, the generation process of generator G is transformed into a mutation process, and the original single generator G is modified into three heterogeneous generators to be evolved. This improves the diversity and quality of generated samples; Heterogeneous Generator Minimize the KL divergence between D(x) and R0:

[0052]

[0053] Among them, D KL (·) represents the KL divergence between the two distributions, G(z) represents the generated samples, represents the mathematical expectation of the fake samples generated by the noise data z~p(z), D(G(z)) represents the discriminator score of the fake samples, and R0 represents the distribution of fake samples output by the realistic distribution; heterogeneous generator Defined as:

[0054]

[0055] Among them, D KL (·) represents the KL divergence between two distributions, Indicates that x belongs to the true sample distribution p data The joint mathematical expectation of the fake samples generated by (x) and the noise data z~p(z), D(G(z)) represents the discriminator score of the fake samples, D(x) represents the discriminator score of the real samples, and R0 represents the distribution of fake samples output by the realistic distribution; heterogeneous generator They are defined as:

[0056]

[0057] Among them, D KL (·) represents the KL divergence between two distributions, represents the mathematical expectation of the fake samples generated by the noise data z~p(z), D(G(z)) represents the discriminator score of the fake samples, R0 represents the distribution of fake samples output by the realistic distribution, and R1 represents the distribution of true samples output by the realistic distribution; therefore, There are three heterogeneous generators to be evolved.

[0058] Furthermore, the step 32) specifically includes:

[0059] The purpose of the fitness function is to transform the three heterogeneous generators in step 31) The variant samples Perform natural selection, where the subscripts i, j, and k represent the corresponding samples generated by the generator after mutation. The least squares fitness function is used to measure the quality and diversity of the generated fake samples. The fitness function F q Defined as:

[0060]

[0061] in, represents the mathematical expectation of the fake samples generated by the noise data z~p(z), G(z) represents the sample output by the generator, and D(G(z)) represents the discriminator score of the fake sample; the diversity fitness function F d Defined as:

[0062]

[0063] in, and Represent the gradients of the discriminator D and the generator G respectively;

[0064] The overall fitness function F is defined as:

[0065] F=F q +λF d

[0066] Among them, λ≥0 is the adaptive parameter, F q is the fitness function, F d is the diversity fitness function.

[0067] Furthermore, the step 33) specifically includes:

[0068] According to the scores of the fitness function, sort in descending order; find the middle value σ of the sequence, and define the samples with a fitness function score greater than σ as qualified samples; input the qualified samples into the discriminator D;

[0069] All generated samples are selected again according to their performance scores, and the sample sequence after the threshold σ is sorted in descending order. The first sample of the sequence is selected as the initialization sample for the next iterative training. After each evolutionary step of the evolutionary selection, the heterogeneous generator Complete mutation; the discriminator starts to update and further marks the real sample x and the fake sample y as 1 or 0. The discrimination process is as follows:

[0070]

[0071] in, Indicates that x belongs to the generated sample distribution p g (x), D(y) represents the output of the discriminator for the fake sample y, p data (x) represents the data distribution of the real sample, Indicates that y belongs to the generated sample distribution p y (y) expectation, p y (y) represents the data distribution of generated samples.

[0072] Compared with the prior art, the present invention has the following beneficial effects:

[0073] 1) The present invention establishes a sample similarity model, a feature similarity model and a classification model by analyzing the correlation of the invariant characteristics of the spatial targets, provides optional samples for the generative adversarial network, and reduces the training time of the generative adversarial network.

[0074] 2) The present invention improves the original loss function of the generative adversarial network into a loss function based on fidelity, which suppresses the non-gradient guidance of the discriminator to the generator.

[0075] 3) The present invention adopts a natural evolution generative adversarial learning method to transform the sample generation problem into a sample mutation problem, effectively reducing the gradient dissipation and mode collapse problems of the generative adversarial network without changing the original Nash equilibrium condition.

[0076] 4) The present invention uses the loss function gradient of the generator and the discriminator as the basis for discrimination, selects qualified samples based on the sample screening method in natural evolution selection, and prompts the spatial target image generated by the generator to perform self-optimization to achieve the purpose of generating realistic samples; at the same time, the natural evolution selection method prompts the samples generated by the generator to have good effects, fast network training, and can be directly used for related deep learning training. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 is a flow chart of the method of the present invention;

[0078] Figure 2 Schematic diagram of a naturally evolving generative adversarial network. DETAILED DESCRIPTION

[0079] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0080] Reference Figure 1 , Figure 2 As shown, the present invention provides a spatial target sample augmentation method based on a generative adversarial network, the steps are as follows:

[0081] 1) Analyze the correlation of the invariant characteristics of spatial targets, classify their invariant features and differential features, and provide optional samples for the generative adversarial network;

[0082] In the example, in the step 1), the existing target image is set as the source domain, and the generated target image is set as the target domain; the knowledge learned by the deep learning algorithm in the source domain is transferred to the target domain to complete the expansion task of the spatial target. There are some common cross-features between the source domain and the target domain. Through feature transformation, the features of the source domain and the spatial target domain are transformed to the same space, so that the source domain data and the spatial target domain data in the space have the same data distribution, and then adversarial network learning is performed. Considering that the migration method based on the common feature space mainly relies on the same invariant features, it is necessary to analyze the correlation of the invariant features of the cooperative spatial targets and classify their invariant features and difference features.

[0083] The details are as follows:

[0084] 11) Sample similarity analysis;

[0085] In the example, a three-layer convex network is used for spatial target characteristic analysis, and each layer has a relatively independent closed operation mechanism; the first layer is a special single-layer feedforward network composed of an extreme learning machine, which is mainly used to map the channel information of natural images in the source domain to the target data set in a complex environment of the target domain; the second layer is also composed of an extreme learning machine, and its main function is to recognize and expand targets in complex environments; the third layer takes into account spatial information and uses a random walk algorithm for spatial regularization to convert the target image into a grid image, which is then regularized using an energy function, thereby improving the stability and generation ability of the model.

[0086] 12) Feature similarity analysis;

[0087] The nonlinear mapping between low-resolution natural images and high-resolution natural images is modeled through a neural network, and then the learned mapping relationship is transferred to the target domain through a separate processing channel; when performing target expansion tasks under complex backgrounds, if there is a lack of training samples, the relationship previously learned on natural images can be used to extract features of objects of the same category in ordinary natural images as in images under complex backgrounds, to assist in target recognition tasks under complex backgrounds.

[0088] 13) Model similarity analysis;

[0089] The model-based feature analysis mainly considers the situation of insufficient training samples in the target domain. The classification model based on Resnet is trained in the source domain, and the classifier is more appropriately adapted to the data distribution of the target domain using a semi-supervised learning strategy while keeping the data distribution unchanged. First, the Resnet network is used for feature extraction, and it is trained in natural image data by combining the differences between the surrounding pixels and the current pixel. Then, a 5-layer linear principal component analysis network is designed to synthesize the feature information of the target image output by Resnet, so that the difference between the source domain and the target domain is significantly reduced when pre-training the convolutional neural network. Then, after using the deep residual neural network GoogleNet to mine the deep features of the sample neighborhood space, the migrated Resnet network is used for feature extraction again, which improves the classification effect of the residual network in the case of a small number of labels. At the same time, a two-branch 3-layer convolutional neural network is used to extract the channel information and spatial information of the target image in a complex environment, and then the features are fused and classified with the output of the Resnet network. During the classification process, considering that the first three convolutional layers of Resnet capture some low-level features such as edges and corners, the functions of these layers are universal, so the network layers trained in the source domain can be migrated to the target domain. The last three convolutional layers are generally used to extract high-level features of specific target scenes, and it is not suitable to migrate these layers. Therefore, when classifying the target domain, the last three layers are randomly initialized and then trained with a small number of samples from the target domain to complete the model migration for target expansion tasks in complex environments.

[0090] After completing step 1), the samples to be expanded in the sample set have completed sample similarity analysis, feature similarity analysis, and model similarity analysis, providing the premise and conditions for the generative adversarial network to generate samples.

[0091] 2) Analyze the reasons why the original loss function easily leads to mode collapse, use the fidelity function to replace the original loss function of the generative adversarial network, and generate samples for the target;

[0092] Specifically, the step 2) is as follows:

[0093] 21) Mode collapse analysis: Through the definition of KL divergence and JS divergence, the defects of the original loss function of the generative adversarial network are analyzed, and it is proved that the original loss function easily causes the training process of the generative adversarial network to fall into mode collapse, which is not conducive to the generation of spatial target samples;

[0094] 22) Define a fidelity function and replace the original single variable output of the generative adversarial network discriminator with the output about the fidelity distribution.

[0095] Wherein, the step 21) is specifically as follows:

[0096] The generative adversarial network consists of two parts: the generator G and the discriminator D. The generator takes random noise z~p(z) as input and outputs the generated sample distribution p g (x); The discriminator uses the real sample distribution p of the sample set data (x) is the input, and the output is the generated sample distribution p g (x) and the true sample distribution p data (x) The difference between them; the original formal adversarial network satisfies the following:

[0097]

[0098] Among them, D(x) represents the output of the discriminator D for the true sample, D(G(z)) represents the output of the discriminator D for the fake sample, and G(z) represents the sample generated by the generator G. Indicates that sample x belongs to the true sample distribution p data (x) expectations, represents the mathematical expectation of the false samples generated by the noise data z~p(z), Indicates that x belongs to the generated sample distribution p g (x); the loss function of the discriminator D is defined as:

[0099]

[0100] Among them, D(x) represents the output of the discriminator D for the true sample, D(G(z)) represents the output of the discriminator D for the fake sample, and G(z) represents the sample generated by the generator G. Indicates that sample x belongs to the true sample distribution p data (x) expectations, Represents the mathematical expectation of the false samples generated by the noise data z~p(z).

[0101] The loss function of the generator G is:

[0102]

[0103] in, represents the mathematical expectation of the false samples generated by the noise data z~p(z), D(G(z)) represents the output of the discriminator D for the false samples, and G(z) represents the samples generated by the generator G.

[0104] According to the above definition, the condition for the entire adversarial network to reach Nash equilibrium is:

[0105]

[0106] When p data (x) = p g (x), D *(x) = 0.5 represents the optimal output of the discriminator, p data (x) represents the true sample distribution, p g (x) represents the generated sample distribution;

[0107] Define p data (x) and p g The KL divergence of (x) is:

[0108]

[0109] Where KL(·) represents the KL divergence, p g (x) represents the generated sample distribution, p data (x) represents the true sample distribution; define p data (x) and p g JS divergence formula for (x):

[0110]

[0111] Where JS(·) represents the JS divergence, p g (x) represents the generated sample distribution, p data (x) represents the true sample distribution; when p data (x) = p g (x), then the original loss function Transformed into:

[0112]

[0113] Among them, p g (x) represents the generated sample distribution, p data (x) represents the true sample distribution, Indicates that sample x belongs to the true sample distribution p data (x) expectations, Represents the mathematical expectation of the false samples generated by the noise data z~p(z); Bring the KL divergence and JS divergence into Then we get:

[0114]

[0115] Therefore, when the generative adversarial network is trained using the original loss function, when p data (x) = p g (x),

[0116] At this time, the output of the loss function is a constant value -2log2, that is, the gradient is 0, which ultimately results in the discriminator D having no gradient guidance for the generator G, causing the generative adversarial network model to collapse.

[0117] The step 22) specifically includes:

[0118] In the original loss function, the single scalar 1 or 0 output by the discriminator D easily leads to the loss function output -2log2. This problem can be suppressed by converting the single scalar output of the discriminator D into a multivariate output.

[0119] Using realistic distribution p r As the output of the discriminator D, for any given sample x, the discriminator D satisfies Ω is p r A set of, u is the fidelity evaluation standard function; define true as R1, false as R0, to replace the original 1 and 0; then the generative adversarial network becomes:

[0120]

[0121] Among them, D KL (·) represents the KL divergence between the two distributions, D(x) represents the output of the discriminator D for the true sample, Indicates that sample x belongs to the true sample distribution p data (x) expectations, Indicates that x belongs to the generated sample distribution p g (x), R0 represents the distribution of fake samples output by realistic distribution, and R1 represents the distribution of true samples output by realistic distribution;

[0122] The conditions for the adversarial network to reach Nash equilibrium based on fidelity are:

[0123]

[0124] Among them, D * (x,u) represents the distribution based on the realistic distribution p r The optimal output of the discriminator; when R1(u) = 1, R0(u) = 0, that is, the sample distribution p generated by the discriminator g (x) outputs 0. At this time, D * (x,u)=D * (x) = 0.5, that is, the loss function based on distribution degree will not change the original Nash equilibrium.

[0125] 3) Based on the natural evolution method, the generation process of the generator is transformed into a mutation process, and the original single generator is modified into three heterogeneous generators to be evolved; a fitness function self-optimization space target sample is established, and the optimized target sample is input into the discriminator;

[0126] Specifically, the step 3) includes:

[0127] 31) Convert the generation process of the generator into a mutation process, and modify the original single generator into three heterogeneous generators to be evolved;

[0128] 32) Define the fitness function;

[0129] 33) Through the score of the fitness function, filter out the sub-generation targets that do not meet the requirements (sub-generation targets with a single pattern and insufficient samples); input the qualified samples into the discriminator D and use them as the initialization samples for the next network iteration.

[0130] In the example, the step 31) specifically includes:

[0131] Based on the natural evolution method, the generation process of generator G is transformed into a mutation process, and the original single generator G is modified into three heterogeneous generators to be evolved. This improves the diversity and quality of generated samples; Heterogeneous Generator Minimize the KL divergence between D(x) and R0:

[0132]

[0133] Among them, D KL (·) represents the KL divergence between the two distributions, G(z) represents the generated samples, represents the mathematical expectation of the fake samples generated by the noise data z~p(z), D(G(z)) represents the discriminator score of the fake samples, and R0 represents the distribution of fake samples output by the realistic distribution; heterogeneous generator Defined as:

[0134]

[0135] Among them, D KL (·) represents the KL divergence between two distributions, Indicates that x belongs to the true sample distribution p data The joint mathematical expectation of the fake samples generated by (x) and the noise data z~p(z), D(G(z)) represents the discriminator score of the fake samples, D(x) represents the discriminator score of the real samples, and R0 represents the distribution of fake samples output by the realistic distribution; heterogeneous generator They are defined as:

[0136]

[0137] Among them, D KL (·) represents the KL divergence between two distributions, represents the mathematical expectation of the fake samples generated by the noise data z~p(z), D(G(z)) represents the discriminator score of the fake samples, R0 represents the distribution of fake samples output by the realistic distribution, and R1 represents the distribution of true samples output by the realistic distribution; therefore, There are three heterogeneous generators to be evolved.

[0138] The step 32) specifically includes:

[0139] The purpose of the fitness function is to transform the three heterogeneous generators in step 31) The variant samples Perform natural selection, where the subscripts i, j, and k represent the corresponding samples generated by the generator after mutation. The least squares fitness function is used to measure the quality and diversity of the generated fake samples. The fitness function F q Defined as:

[0140]

[0141] in, represents the mathematical expectation of the fake samples generated by the noise data z~p(z), G(z) represents the sample output by the generator, and D(G(z)) represents the discriminator score of the fake sample; the diversity fitness function F d Defined as:

[0142]

[0143] in, and Represent the gradients of the discriminator D and the generator G respectively;

[0144] The overall fitness function F is defined as:

[0145] F=F q +λF d

[0146] Among them, λ≥0 is the adaptive parameter, F q is the fitness function, F d is the diversity fitness function. Further, the step 33) specifically includes:

[0147] According to the scores of the fitness function, sort in descending order; find the middle value σ of the sequence, and define the samples with a fitness function score greater than σ as qualified samples; input the qualified samples into the discriminator D;

[0148] All generated samples are selected again according to their performance scores, and the sample sequence after the threshold σ is sorted in descending order. The first sample of the sequence is selected as the initialization sample for the next iterative training. After each evolutionary step of the evolutionary selection, the heterogeneous generator Complete mutation; the discriminator starts to update and further marks the real sample x and the fake sample y as 1 or 0. The discrimination process is as follows:

[0149]

[0150] in, Indicates that x belongs to the generated sample distribution p g (x), D(y) represents the output of the discriminator for the fake sample y, p data (x) represents the data distribution of the real sample, Indicates that y belongs to the generated sample distribution p y (y) expectation, p y (y) represents the data distribution of generated samples.

[0151] Compared with the traditional space target sample set expansion method, the present invention can increase the space target samples by 2 orders of magnitude and improve the sample diversity by 30 times.

[0152] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0153] The present invention has many specific application paths. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principle of the present invention. These improvements should also be regarded as the protection scope of the present invention.

Claims

1. A spatial target sample augmentation method based on a generative adversarial network, characterized in that: Here are the steps: 1) Analyze the correlation of the invariant characteristics of spatial targets, classify their invariant features and differential features, and provide optional samples for the generative adversarial network; 2) Analyze the reasons why the original loss function is prone to mode collapse, use the fidelity function to replace the original loss function of the generative adversarial network, and generate samples for the target; 3) Based on the natural evolution method, the generation process of the generator is transformed into a mutation process, and the original single generator is modified into three heterogeneous generators to be evolved; a fitness function self-optimization space target sample is established, and the optimized target sample is input into the discriminator; In the step 1), the existing target image is set as the source domain, and the generated target image is set as the target domain; the knowledge learned by the deep learning algorithm in the source domain is transferred to the target domain to complete the expansion task of the spatial target; the source domain and the target domain have common cross-features, and the features of the source domain and the spatial target domain are transformed to the same space through feature transformation, so that the source domain data and the spatial target domain data in the space have the same data distribution, and then adversarial network learning is performed; considering that the migration method based on the common feature space relies on the same invariant features, it is necessary to analyze the correlation of the invariant features of the cooperative spatial target, and classify its invariant features and difference features; The invariant characteristic correlation analysis of the space target in step 1) specifically includes: 11) Sample similarity analysis: The samples of spatial targets are augmented, requiring the data distribution of the target domain and the source domain to be as similar as possible, so as to ensure that the performance of the recognition task is as similar as possible when the classifier remains unchanged; 12) Feature similarity analysis: Before sample augmentation, extract original or additional features from the original sample set, remove features that are greatly affected by migration conversion factors, and select a set of unaffected features as input features for sample augmentation; 13) Model similarity analysis: Considering the situation where there are insufficient training samples in the target domain, the classification model is trained in the source domain and the data distribution is kept unchanged, so that the classifier can adapt to the data distribution of the target domain.

2. The spatial target sample augmentation method based on generative adversarial network according to claim 1, characterized in that: The step 2) is specifically as follows: 21) Mode collapse analysis: Analyze the defects of the original loss function of the generative adversarial network through the definition of KL divergence and JS divergence; 22) Define a fidelity function and replace the original single variable output of the generative adversarial network discriminator with the output about the fidelity distribution.

3. The spatial target sample augmentation method based on generative adversarial network according to claim 2, characterized in that: The step 21) is specifically as follows: The generative adversarial network consists of two parts: the generator G and the discriminator D. The generator takes random noise z~p(z) as input and outputs the generated sample distribution p g (x); The discriminator uses the real sample distribution p of the sample set data (x) is the input, and the output is the generated sample distribution p g (x) and the true sample distribution p data (x) The difference between them; the original formal adversarial network satisfies the following: Among them, D(x) represents the output of the discriminator D for the true sample, D(G(z)) represents the output of the discriminator D for the fake sample, and G(z) represents the sample generated by the generator G. Indicates that sample x belongs to the true sample distribution p data (x) expectations, represents the mathematical expectation of the false samples generated by the noise data z~p(z), Indicates that x belongs to the generated sample distribution p g (x); the loss function of the discriminator D is defined as: The loss function of the generator G is: The condition for the entire adversarial network to reach Nash equilibrium is: When p data (x) = p g (x), D * (x) = 0.5 represents the optimal output of the discriminator; Define p data (x) and p g The KL divergence of (x) is: Where KL(·) represents the KL divergence; define p data (x) and p g JS divergence formula for (x): Among them, JS() represents JS divergence; when p data (x) = p g (x), then the original loss function Transformed into: Bringing KL divergence and JS divergence into Then we get: Therefore, when the generative adversarial network is trained using the original loss function, when p data (x) = p g (x), At this time, the output of the loss function is a constant value -2log2, that is, the gradient is 0, which ultimately results in the discriminator D having no gradient guidance for the generator G, causing the generative adversarial network model to collapse.

4. The spatial target sample augmentation method based on generative adversarial network according to claim 2, characterized in that: The step 22) specifically includes: In the original loss function, the single scalar 1 or 0 output by the discriminator D easily leads to the loss function output -2log2. This problem can be suppressed by converting the single scalar output of the discriminator D into a multivariate output. Using realistic distribution p r As the output of the discriminator D, for any given sample x, the discriminator D satisfies Ω is p r The set of u is the fidelity evaluation standard function; define the true distribution as R1 and the false distribution as R0 to replace the original 1 and 0; then the generative adversarial network becomes: The conditions for the adversarial network to reach Nash equilibrium based on fidelity are: Among them, D * (x,u) represents the distribution based on the realistic distribution p r The optimal output of the discriminator; when R1(u) = 1, R0(u) = 0, that is, the sample distribution p generated by the discriminator g (x) outputs 0. At this time, D * (x,u)=D * (x) = 0.5, that is, the loss function based on distribution degree will not change the original Nash equilibrium.

5. The spatial target sample augmentation method based on generative adversarial network according to claim 1, characterized in that: The step 3) specifically includes: 31) Convert the generation process of the generator into a mutation process, and modify the original single generator into three heterogeneous generators to be evolved; 32) Define the fitness function; 33) Filter out the child targets that do not meet the requirements through the score of the fitness function; input the qualified samples into the discriminator D and use them as the initialization samples for the next network iteration.

6. The spatial target sample augmentation method based on generative adversarial network according to claim 5, characterized in that: The step 31) specifically includes: Based on the natural evolution method, the generation process of generator G is transformed into a mutation process, and the original single generator G is modified into three heterogeneous generators to be evolved. This improves the diversity and quality of generated samples; Heterogeneous Generator Minimize the KL divergence between D(x) and R0: Among them, D KL (·) represents the KL divergence between the two distributions, G(z) represents the generated samples, represents the mathematical expectation of the fake samples generated by the noise data z~p(z), D(G(z)) represents the discriminator score of the fake samples, and R0 represents the distribution of fake samples output by the realistic distribution; heterogeneous generator Defined as: in, Indicates that x belongs to the true sample distribution p data (x) and the joint mathematical expectation of the fake samples generated by the noise data z~p(z), D(x) represents the discriminator score of the real sample; heterogeneous generator They are defined as: Among them, R1 represents the true sample distribution of realistic distribution output; therefore, There are three heterogeneous generators to be evolved.

7. The spatial target sample augmentation method based on generative adversarial network according to claim 6, characterized in that: The step 32) specifically includes: The purpose of the fitness function is to transform the three heterogeneous generators in step 31) The variant samples Perform natural selection, where the subscripts i, j, and k represent the corresponding samples generated by the generator after mutation. The least squares fitness function is used to measure the quality and diversity of the generated fake samples. The fitness function F q Defined as: in, represents the mathematical expectation of the fake samples generated by the noise data z~p(z), G(z) represents the sample output by the generator, and D(G(z)) represents the discriminator score of the fake sample; the diversity fitness function F d Defined as: in, and Represent the gradients of the discriminator D and the generator G respectively; The overall fitness function F is defined as: F=F q +λF d Among them, λ≥0 is the adaptive parameter, F q is the fitness function, F d is the diversity fitness function.

8. The spatial target sample augmentation method based on generative adversarial network according to claim 7, characterized in that: The step 33) specifically includes: According to the scores of the fitness function, sort in descending order; find the middle value σ of the sequence, and define the samples with a fitness function score greater than σ as qualified samples; input the qualified samples into the discriminator D; All generated samples are selected again according to their performance scores, and the sample sequence after the threshold σ is sorted in descending order. The first sample of the sequence is selected as the initialization sample for the next iterative training. After each evolutionary step of the evolutionary selection, the heterogeneous generator Complete mutation; the discriminator starts to update and further marks the real sample x and the fake sample y as 1 or 0. The discrimination process is as follows: in, Indicates that x belongs to the generated sample distribution p g (x), D(y) represents the output of the discriminator for the fake sample y, p data (x) represents the data distribution of the real sample, Indicates that y belongs to the generated sample distribution p y (y) expectation, p y (y) represents the data distribution of generated samples.

Images