Hyperspectral remote sensing data enhancement method based on band preservation and generative adversarial network

Abstract

The present invention relates to a hyperspectral remote sensing data enhancement method based on band preservation and generative adversarial networks. The method combines a traditional feature-preserving data enhancement strategy with a data enhancement method based on a GAN network and creatively applies it to hyperspectral remote sensing data. Based on the large number of bands in hyperspectral images and the quality of data generated by the GAN network, a new band selection fusion strategy is designed. The band selection fusion strategy is integrated into the GAN training process, and a new GAN network is constructed. In order to increase the diversity of samples generated by the GAN network, a new GAN network training process is designed. The hyperspectral remote sensing data enhancement method of the present invention effectively alleviates the small sample size problem and data category imbalance problem encountered in classification problems in the field of hyperspectral remote sensing, thereby improving classification accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of data enhancement, and in particular relates to a hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative networks. Background Art

[0002] After the deep learning method represented by neural networks was proposed, it has been widely used in the field of computer vision. At the same time, researchers have proposed a variety of classification strategies for hyperspectral remote sensing images based on neural network variants, among which the main representatives are neural network architectures such as DBN (deep belief network), RNN (recurrent neural network), GCN (graph convolutional neural network), and CNN (convolutional neural network). The above technical solutions use neurons or convolution layers to adaptively extract typical features of the target data set for classification and recognition, and can achieve ideal accuracy when there is sufficient data. However, in the application scenarios of hyperspectral data, they often face the problems of small samples and data imbalance. In order to alleviate this problem, researchers have used different data enhancement strategies. The data enhancement strategy starts from the root, fully explores the information expression capabilities of existing data, and amplifies and generates data.

[0003] Currently, the following data enhancement methods are widely used for hyperspectral remote sensing data. One is the traditional data enhancement method, which retains some features of the original data or performs a simple transformation on the original data. Although these methods are sometimes simple and efficient, they cannot introduce some nonlinear features that are conducive to improving the robustness of the classification network, and sometimes they are ineffective. One is to use the GAN network to directly generate new data and integrate the GAN with the classifier to improve the classification accuracy. However, the GAN network is difficult to train and converge, and often encounters phenomena such as gradient explosion and gradient disappearance. GAN generates reproduced data based on the original data, and sometimes it cannot reproduce the important features of the original data well. Sometimes, the new data introduces too many nonlinear features, which misleads the classifier, causing the classification accuracy to decrease instead of increase. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] The present invention provides a hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network, comprising:

[0006] Step 1: Obtain the categories and corresponding data samples that need to be amplified in the hyperspectral remote sensing dataset to form a small sample training set;

[0007] Step 2: For each category that needs to be augmented, build a corresponding GAN network and train the corresponding GAN network separately using the noise data and the sample data set of each category in the small sample training set;

[0008] Step 3: Input random noise and category labels into the corresponding trained GAN network to generate a false data set for each category. Perform band fusion on the false data set and the data in the sample data set of the corresponding category to obtain a mixed data set corresponding to each category. The mixed data set serves as the amplified data for the category that needs to be amplified and enhanced.

[0009] In one embodiment of the present invention, in step 2, the corresponding GAN network is trained using the noise data and a sample data set of a category in the small sample training set, including the following steps:

[0010] Step a: Input the noise data into the generator of the GAN network to generate a fake training dataset;

[0011] Step b: dividing the sample data set into a first real data set and a second real data set with the same number of samples by rounding down;

[0012] Step c: performing band fusion on the data in the first real data set and the false training data set to form a fused data set;

[0013] Step d: The fake training dataset and the fused dataset are used as fake data, and the first real dataset and the second real dataset are used as real data, and are input into the discriminator of the GAN network to perform adversarial training. The loss value is calculated and the network parameters of the discriminator and generator of the GAN network are updated according to the loss value until a preset number of training times is reached to obtain a trained GAN network.

[0014] In one embodiment of the present invention, in step a, the size of the generated false training data set is half the size of the sample data set of the corresponding category.

[0015] In one embodiment of the present invention, step c includes: performing band fusion on the data in the first real data set and the false training data set in a one-to-one correspondence to generate corresponding fused data, and all fused data constitute the fused data set, wherein the band fusion includes the following steps:

[0016] Step ①: splitting a data in the first real data set and a corresponding data in the false training data set into spectral dimensions respectively to obtain corresponding multiple band data;

[0017] Step 2: Calculate the band score of each band according to the following formula:

[0018] Score i =KL[P(x i )||( i )]+(( i ));

[0019]

[0020] In the formula, Score i represents the band score of the i-th band, KL represents the KL divergence, σ represents the variance, ( i ) represents the distribution of the i-th band of the real data, G(z i ) represents the distribution of the i-th band of false data;

[0021] Step 3: Arrange the band scores of each band from low to high to obtain a band score sequence;

[0022] Step 4: Construct a mixed sequence M(S). The length of the mixed sequence M(S) is the same as the number of multiple bands. The elements in the mixed sequence M(S) correspond one-to-one to the multiple bands. Initialize the mixed sequence M(S) and set the value of each element in the mixed sequence M(S) to 0.

[0023] Step 5: Update the mixed sequence M(S) according to the band score sequence, and change the values of the corresponding elements in the mixed sequence M(S) of the bands in the first 60% of the band score list to 1;

[0024] Step ⑥: Calculate the fusion data according to the following formula:

[0025] x_mixture=(1-M(S))⊙G(Z)+M(S)⊙P(X);

[0026] Where x_mixture represents the fused data, (1-M(S)) represents the inversion of each element in the mixed sequence M(S), G(Z) represents the distribution of false data, G(Z) = (z1, z2, ... z i …, z n ), P(X) represents the distribution of real data, P(X)=(x1,x2,…x i …, x n ), n is the number of bands, ⊙ represents the multiplication of corresponding positions, where the element 1 in (1-M(S)) and M(S) means that the band data multiplied with it is retained, and the element 0 means that the band data multiplied with it is discarded.

[0027] In one embodiment of the present invention, in step d, during adversarial training, the loss function of the discriminator of the GAN network is:

[0028]

[0029] Where, represents the value output by the discriminator for the first real data set, P1 represents the distribution of the first real data set, represents the value output by the discriminator for the second real data set, P2 represents the distribution of the second real data set, represents the value output by the discriminator for the fusion dataset, P mixture represents the distribution of the fused dataset, represents the value output by the discriminator for the fake training data set, P g represents the distribution of fake training data sets, λ represents the hyperparameter in the gradient penalty, represents the gradient of the discriminator for the distribution input of the fake training data set, || ||2 represents the second-order norm;

[0030] The loss function of the generator of the GAN network is:

[0031]

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The hyperspectral remote sensing data enhancement method based on band preservation and generative adversarial networks of the present invention combines the traditional feature-preserving data enhancement strategy with the data enhancement method based on the GAN network, and creatively applies it to hyperspectral remote sensing data. According to the characteristics of the large number of hyperspectral image bands and the quality of data generated by the GAN network, a new band selection fusion strategy is designed, which is integrated into the GAN training process and a new GAN network is constructed. In order to increase the diversity of samples generated by the GAN network, a new GAN network training process is designed. The hyperspectral remote sensing data enhancement method of the present invention effectively alleviates the small sample problem and data category imbalance problem encountered in classification problems in the field of hyperspectral remote sensing, and improves the accuracy of classification.

[0034] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Schematic diagram of a hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network provided by an embodiment of the present invention;

[0036] Figure 21 is a schematic diagram of the overall framework of a hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network provided by an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram of a band selection fusion strategy provided by an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of a GAN network training process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following is a detailed description of a hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network proposed in accordance with the present invention, in combination with the accompanying drawings and specific implementation methods.

[0040] The aforementioned and other technical contents, features, and effects of the present invention are clearly presented in the following detailed description of the specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a deeper and more specific understanding of the technical means and effects adopted by the present invention to achieve the intended purpose can be obtained. However, the accompanying drawings are provided for reference and illustration purposes only and are not intended to limit the technical solutions of the present invention.

[0041] Example 1

[0042] Please refer to Figure 1 and Figure 2 , Figure 1 Schematic diagram of a hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall framework of a hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network provided by an embodiment of the present invention. As shown in the figure, the hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network of this embodiment includes:

[0043] Step 1: Obtain the categories and corresponding data samples that need to be amplified in the hyperspectral remote sensing dataset to form a small sample training set;

[0044] In an optional embodiment, a classification network can be used to pre-train a specified hyperspectral remote sensing dataset to obtain pre-training results. The sample categories with poor classification results (i.e., classification accuracy below the average) are recorded, for example, label_i, label_j, ..., label_k. These sample categories are recorded as categories that need to be augmented.

[0045] According to the category to be augmented, the data samples corresponding to the category are selected from the hyperspectral remote sensing dataset to form a small sample training set: Dataset_i, Dataset_j...Dataset_k. These small sample training sets are used to train the GAN network (Generative Adversarial Network) of this embodiment.

[0046] Step 2: For each category that needs to be augmented, build a corresponding GAN network and train the corresponding GAN network separately using the noise data and the sample data set of each category in the small sample training set;

[0047] In this embodiment, in order to fully learn the sample features and facilitate subsequent calls, a GAN network is designed to train the same number of categories as the small sample training set.

[0048] See Figure 4 , Figure 4 This is a schematic diagram of a GAN network training process provided by an embodiment of the present invention. As shown in the figure, in this embodiment, the corresponding GAN network is trained using noise data and a sample data set of a category in the small sample training set, including the following steps:

[0049] Step a: Input the noise data into the generator of the GAN network to generate a fake training dataset;

[0050] In an optional embodiment, the size of the generated false training data set is half the size of the sample data set of the corresponding category, which is rounded down.

[0051] Step b: Divide the sample data set into a first real data set and a second real data set with the same number of samples by rounding down;

[0052] Step c: performing band fusion on the data in the first real data set and the false training data set to form a fused data set;

[0053] In an optional implementation, step c includes: performing band fusion on the data in the first real data set and the false training data set in a one-to-one correspondence to generate corresponding fused data, and all fused data constitute a fused data set.

[0054] In this embodiment, a band selection and fusion module is constructed to perform band fusion using the band selection fusion strategy. Figure 3 The following is a schematic diagram of a band selection fusion strategy provided by an embodiment of the present invention. Specifically, band fusion includes the following steps:

[0055] Step ①: Split a data point in the first real data set and a corresponding data point in the fake training data set into spectral dimensions to obtain corresponding multiple band data points;

[0056] Step 2: Calculate the band score of each band according to the following formula:

[0057] Score i =KL[P(x i )||( i )]+(( i ))(1);

[0058]

[0059] In the formula, Score i represents the band score of the i-th band, KL represents the KL divergence, σ represents the variance, ( i ) represents the distribution of the i-th band of the real data, G(z i ) represents the distribution of the i-th band of false data;

[0060] Then, the band scores of multiple bands finally calculated are:

[0061] Score=(Score1, Score2,…Score i …, Score n )(3);

[0062] Where n is the number of bands.

[0063] In this embodiment, the KL divergence is used to measure the similarity between the real band and the generated false band. A larger KL divergence indicates a lower quality false band, resulting in a larger score value, indicating a tendency to retain the real band, and vice versa. A larger variance indicates that the band contains richer information, resulting in a larger score value, indicating a greater tendency to retain the band.

[0064] Step 3: Arrange the band scores of each band from low to high to obtain a band score sequence;

[0065] For example, the band score list can be expressed as:

[0066] Scorelist=[Score i ,core j ,core k …Score n ](4).

[0067] Step 4: Construct a mixed sequence M(S). The length of the mixed sequence M(S) is the same as the number of multiple bands. The elements in the mixed sequence M(S) correspond one-to-one to the multiple bands. Initialize the mixed sequence M(S) and set the value of each element in the mixed sequence M(S) to 0.

[0068] For example, assuming that the spectral dimension is split in step ①, the n band data obtained are represented in the initialized mixed sequence M(S) as: M(S) = [0, 0, 0…, 0], where M(S) contains n elements, where the first element in M(S) corresponds to the first band, the second element corresponds to the fifth band, and so on.

[0069] Optionally, M(S) is a mask function.

[0070] Step 5: Update the mixed sequence M(S) according to the band score sequence, and change the values of the corresponding elements in the mixed sequence M(S) of the bands in the first 60% of the band score list to 1;

[0071] For example, M(S) is updated to the following form:

[0072] M(S)=[1,0,1…0](5).

[0073] Step ⑥: Calculate the fusion data according to the following formula:

[0074] x_mixture=(1-M(S))⊙G(Z)+M(S)⊙P(X)(6);

[0075] Where x_ represents the fused data, (1-()) represents the inversion of each element in the mixed sequence M(S), G(Z) represents the distribution of false data, G(Z) = z1, z2, ... z i …, z n ), P(X) represents the distribution of real data, P(X) = x1, x2, ... x i …, x n ), n is the number of bands,

[0076] ⊙ represents multiplication of corresponding positions, where element 1 in (1-()) and M(S) indicates that the band data multiplied with it is retained, and element 0 indicates that the band data multiplied with it is discarded.

[0077] For example, when n=4, assuming M(S)=(1,0,0,1), then 1-M(S))=(0,1,1,0), then,

[0078] x_mixture=(0,1,1,0)⊙(z1,z2,z3,z4)+(1,0,0,1)⊙(x1,x2,x3,x4)

[0079] =(0,z2,z3,0)+(x1,0,0,x4)

[0080] =(x1, z2, z3, x4) (7);

[0082] Step d: The fake training dataset and the fused dataset are used as fake data, and the first real dataset and the second real dataset are used as real data. They are input into the discriminator of the GAN network for adversarial training. The loss value is calculated and the network parameters of the discriminator and generator of the GAN network are updated according to the loss value until the preset number of training times is reached to obtain a trained GAN network.

[0083] In this embodiment, the true data and the false data have the same size.

[0084] In this embodiment, during adversarial training, the loss function of the discriminator of the GAN network is:

[0085]

[0086] Where, represents the value output by the discriminator for the first real data set, P1 represents the distribution of the first real data set, represents the value output by the discriminator for the second real data set, P2 represents the distribution of the second real data set, represents the value output by the discriminator for the fusion dataset, P mixture represents the distribution of the fused dataset, represents the value output by the discriminator for the fake training data set, P g represents the distribution of fake training data sets, λ represents the hyperparameter in the gradient penalty, represents the gradient of the discriminator for the distribution input of the fake training dataset, and ||||2 represents the second-order norm.

[0087] In this embodiment, the loss function of the discriminator is mainly aimed at the real data set and the fake data set. The last item in the loss function is the gradient penalty item, which is aimed at the fake data generated by the generator.

[0088] In this embodiment, during adversarial training, the loss function of the generator of the GAN network is:

[0089]

[0090] In this embodiment, the loss function of the generator is mainly composed of two fake data sets: the fake data generated by the generator and the fused data generated by the fusion.

[0091] Step 3: Input random noise and category labels into the corresponding trained GAN network to generate a false data set for each category. Perform band fusion on the false data set and the data in the sample data set of the corresponding category to obtain a mixed data set corresponding to each category. The mixed data set serves as the amplified data for the category that needs to be amplified and enhanced.

[0092] In this embodiment, the band fusion step is similar to the band fusion step in the GAN network training process and will not be repeated here.

[0093] In this example, the band selection fusion strategy was used twice for band fusion, but for different purposes. During GAN training, the band selection fusion strategy was added to increase training stability and speed. Furthermore, only half of the real data was mixed, not all features were introduced, which also gave the generator some room to perform. The band selection fusion strategy was added during the generation of augmented data to ultimately achieve the desired mixed data.

[0094] In an optional embodiment, the amplified data obtained according to the above steps can be added to the hyperspectral remote sensing dataset as training data to jointly train the classification network.

[0095] The hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network in the embodiment of the present invention combines the traditional feature-preserving data enhancement strategy with the data enhancement method based on GAN network, and creatively applies it to hyperspectral remote sensing data. According to the characteristics of the large number of hyperspectral image bands and the quality of data generated by GAN network, a new band selection fusion strategy is designed, the band selection fusion strategy is integrated into the GAN training process, and a new GAN network is constructed. In order to increase the diversity of samples generated by GAN network, a new GAN network training process is designed. The hyperspectral remote sensing data enhancement method in the embodiment of the present invention effectively alleviates the small sample problem and data category imbalance problem encountered in classification problems in the field of hyperspectral remote sensing, and improves the classification accuracy.

[0096] Example 2

[0097] In order to verify the effectiveness of the hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network of the present invention, this example conducts a comparative experiment between the method of the present invention and the more popular remote sensing image classification enhancement method in the current research field.

[0098] Comparative experiment:

[0099] Two traditional data augmentation methods were selected: Flip and Radiation strategies, two traditional GAN generation network data augmentation methods were selected: WGAN and DCGAN, and finally three GAN generation network data augmentation methods that have been processed for hyperspectral data were selected: HSGAN, 3DGAMO and AC-WGAN-GP as comparison algorithms. The codes of the comparison algorithms are all open source on relevant websites.

[0100] At the same time, in order to verify the universality of the strategy of this invention, two classification networks were selected as benchmarks, namely the Hamida network and the Li network. Four commonly used hyperspectral remote sensing image datasets, namely Indian Pines, Salinas, Pavia University, and Houston University, were selected for testing. In order to highlight the data category imbalance and small sample problems, 10% of the data in the Indian Pines dataset was selected as the training set; in the Salinas dataset, 1% of the data was selected as the training set; the PaviaU dataset and Houston dataset were artificially created to be more unbalanced: in the PaviaU dataset, the ratio of samples 3, 5, 6, and 7 was fixed to 0.005, and the ratio of the remaining samples was 0.04; in the Houston dataset, the ratio of samples 6, 7, 8, and 13 was fixed to 0.01, and the ratio of the remaining samples was 0.05, highlighting the data category imbalance and small sample problems in remote sensing hyperspectral data. During the experiment, except for the selected data augmentation strategy, the other experimental conditions remained the same.

[0101] OA (overall classification accuracy), AA (average classification accuracy), and the Kappa coefficient were selected as evaluation indicators. Overall classification accuracy refers to the percentage of samples correctly classified by the classifier over the total number of samples in a given test dataset. Class classification accuracy refers to the percentage of correctly classified samples in each class over the total number of samples in that class. The Kappa coefficient is calculated by multiplying the total number of pixels correctly classified by the algorithm over the sum of the diagonal of the confusion matrix, minus the product of the total number of true pixels in a certain class of objects and the total number of classified pixels in that class, and then dividing the result by the square of the total number of pixels minus the product of the total number of true pixels in a certain class of objects and the total number of classified pixels in that class, and then summing the result over all possible classes.

[0102] The training and classification results are listed in Tables 1 to 4. It can be seen that the data augmentation method proposed in this paper achieves better classification results on all four datasets than the control algorithm. This means that the proposed method can effectively improve the accuracy of ground object classification for remote sensing image classification problems.

[0103] Table 1 Classification results of Indian Pines dataset training

[0104]

[0105] Table 2 Classification results of Pavia University dataset training

[0106]

[0107] Table 3 Salinas dataset training classification results

[0108]

[0109] Table 4 Houston University dataset training classification results

[0110]

[0111] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations are intended to cover non-exclusive inclusion, such that an article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the article or device comprising the element.

[0112] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network, characterized in that: include: Step 1: Obtain the categories and corresponding data samples that need to be amplified in the hyperspectral remote sensing dataset to form a small sample training set; Step 2: For each category that needs to be augmented, build a corresponding GAN network and train the corresponding GAN network separately using the noise data and the sample data set of each category in the small sample training set; In step 2, the corresponding GAN network is trained using the noise data and a sample data set of one category in the small sample training set, including the following steps: Step a: Input the noise data into the generator of the GAN network to generate a fake training dataset; Step b: dividing the sample data set into a first real data set and a second real data set with the same number of samples by rounding down; Step c: performing band fusion on the data in the first real data set and the false training data set to form a fused data set; Step d: Input the fake training dataset and the fused dataset as fake data, and the first real dataset and the second real dataset as real data into the discriminator of the GAN network, perform adversarial training, calculate the loss value, and update the network parameters of the discriminator and generator of the GAN network according to the loss value until a preset number of training times is reached, thereby obtaining a trained GAN network; Step 3: Input random noise and category labels into the corresponding trained GAN network to generate a false data set for each category. Perform band fusion on the false data set and the data in the sample data set of the corresponding category to obtain a mixed data set corresponding to each category. The mixed data set serves as the amplified data for the category that needs to be amplified and enhanced.

2. The hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network according to claim 1 is characterized in that: In step a, the size of the generated false training data set is half the size of the sample data set of the corresponding category.

3. The hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network according to claim 1 is characterized in that: The step c includes: performing band fusion on the data in the first real data set and the data in the false training data set in a one-to-one correspondence to generate corresponding fused data, and all fused data constitute the fused data set, wherein the band fusion includes the following steps: Step ①: splitting a data in the first real data set and a corresponding data in the false training data set into spectral dimensions respectively to obtain corresponding multiple band data; Step 2: Calculate the band score of each band according to the following formula: ； ； Where, Indicates the The band score of each band, express KL Divergence, represents the variance, The real data The distribution of bands, The first one represents false data The distribution of bands; Step 3: Arrange the band scores of each band from low to high to obtain a band score sequence; Step 4: Construct a mixed sequence , mixed sequence The length is the same as the number of multiple bands, mixed sequence The elements in correspond to multiple bands one by one, initializing the mixed sequence , let the mixed sequence The value of each element in is 0; Step 5: Update the mixed sequence according to the band score sequence , the first 60% of the bands in the band score list are added to the mixed sequence The value of the corresponding element in becomes 1; Step ⑥: Calculate the fusion data according to the following formula: ； Where, represents fused data, Represents a mixed sequence Invert each element in Table 1 shows the distribution of false data. , represents the distribution of real data, , is the number of bands, Represents the multiplication of corresponding positions, where and Element 1 in the _ means that the band data multiplied by it is retained, and element 0 means that the band data multiplied by it is discarded.

4. The hyperspectral remote sensing data enhancement method based on band preservation and adversarial generative network according to claim 1 is characterized in that: In step d, during adversarial training, the loss function of the discriminator of the GAN network is: ； Where, represents the value output by the discriminator for the first real data set, represents the distribution of the first real dataset, represents the value output by the discriminator for the second real data set, represents the distribution of the second real data set, Represents the value output by the discriminator for the fusion dataset, represents the distribution of the fused dataset, represents the value output by the discriminator for the fake training dataset, represents the distribution of fake training datasets, represents the hyperparameter in the gradient penalty, represents the gradient of the discriminator for the distribution input of the fake training dataset, represents the second-order norm; The loss function of the generator of the GAN network is: 。

Images