Data augmentation method based on generative adversarial model in differential privacy scene

Abstract

The invention discloses a data augmentation method based on a generative adversarial model in a differential privacy scene, and relates to the technical field of Internet of Things. The method comprises the following steps: S1, obtaining a real data set without noise addition, and generating data into a discriminator in a confrontation model; s2, the discriminator learns distribution of real data without noise addition and receives synthetic data generated by the generator; and S3, the generator receives the random noise as input and tries to generate a vivid data sample. According to the method, the differential privacy and the generative adversarial model are combined, the data set of the data collection party can be expanded while the privacy protection requirement of the data holding party is met, and the problem that the data collection party does not have enough data volume is avoided; according to the method, Renyi differential privacy is adopted in the aspect of privacy protection of data, Gaussian noise adding is used in the training process in the generative adversarial model, and the privacy budget consumed in the process is tracked.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Things, and specifically relates to a data augmentation method based on a generative adversarial model in a differential privacy scenario. Background Art

[0002] With the development of the Internet, people have found that data analysis helps to better deal with complex problems; it has achieved excellent results in image recognition, intelligent driving, intelligent medical diagnosis, intelligent translation, data mining, etc.; these achievements have affected all aspects of our lives; for example, in the business field, big data can help companies make better business decisions; by analyzing big data, companies can understand consumer needs and preferences, so as to formulate more accurate marketing strategies and increase sales and profits; at the same time, big data can predict market trends and help companies plan business development; in social services, big data can provide better services to society; for example, in the medical field, big data can be used for disease prediction and prevention; in the education field, big data can help personalized teaching; in the transportation field, big data can be used for traffic management and optimization; in government governance, big data can help the government better understand society and provide better public services; for example, the government can use big data to analyze traffic congestion and formulate better traffic management strategies; use big data to analyze environmental pollution and formulate better environmental protection measures; at present, the application fields of big data cover almost all industries, providing a core means for all walks of life to optimize business processes, improve decision-making efficiency, and create new growth points;

[0003] The privacy protection issues brought about by the rapid development of big data have attracted widespread attention; there is a risk of leakage of personal information during the processing, storage and analysis of personal information; unauthorized data access, system vulnerabilities, hacker attacks, etc. may all lead to the leakage of personal privacy; at the same time, the application of artificial intelligence algorithms to big data involves personal privacy, such as accurately pushing personal behavior trajectories, which can easily make people feel that their privacy is violated; privacy refers to private space, activities and information that citizens do not want others to know; the right to privacy, as a basic personal right, is protected by law; no organization or individual may illegally invade, understand, collect, use and disclose the personal information of others; on the Internet, the risk of personal information leakage is increasing, which may lead to adverse consequences such as identity theft and online fraud, and seriously affect personal image and peace of life; therefore, protecting personal privacy is not only a matter of human freedom and dignity, but also a matter of personal privacy. Respect is also an important means to avoid online fraud, maintain personal image and social order; therefore, the clear protection of personal information by national government laws is of great practical significance for protecting the personal rights and property rights of natural persons and maintaining normal social order; at the same time, it is also crucial to improve relevant laws and regulations and enhance the public's awareness of data privacy protection; every country has laws to protect privacy; the EU takes the General Data Protection Regulation as the core, stipulates the basic requirements, responsibilities and obligations for processing personal data, and emphasizes data security and privacy protection; the United States has passed many laws, such as the California Consumer Act and the Privacy Act, which focus on consumer privacy and data security; the Chinese government has formulated the Cybersecurity Law of the People's Republic of China and the Data Security Law of the People's Republic of China to regulate data processing activities and protect data security and personal privacy;

[0004] However, laws alone are not enough to address the challenges of privacy security. In order to protect data privacy, it is necessary to strengthen the application and research of privacy protection technologies such as data encryption, anonymization, and access control. These technologies are designed to protect personal data from unauthorized access and use. Among them, the issue of differential privacy has attracted widespread attention.

[0005] Differential privacy technology has become the focus of attention in the field of privacy protection in recent years with its rigorous mathematical proof and strong privacy protection. Previous studies have shown that adding a specific noise distribution to a data set can protect privacy leaks caused by slight changes in the data source. Even if the attacker has a lot of background knowledge, it can ensure that the attacker cannot analyze whether the data set contains specific data. On this basis, differential privacy is applicable to various types of data analysis tasks such as statistical queries and machine learning. By adjusting the privacy budget and noise level, it can flexibly balance privacy protection and data availability. Although differential privacy has obvious advantages in theory, it faces challenges such as real-time requirements and user acceptance in practical applications. In order to meet these challenges, it is necessary to strengthen technological innovation so that differential privacy technology can cope with various complex scenarios. In these complex scenarios, differential privacy not only needs to protect the privacy of the data holder, but also needs to ensure the needs of the data collector.

[0006] Differential privacy is a method of processing data before it is made public in order to protect personal privacy. Noise is added to the query results to make personal data obscure. Even if the attacker has a lot of background knowledge, he cannot obtain the private information of a specific individual, which provides a strong guarantee for data security and personal privacy. The advantage of this method is that it concentrates the data on one data collector, thereby reducing the risk of data leakage. However, there is a situation where the data owner may refuse to provide data due to distrust of the data collector. In fact, due to frequent data security incidents, many data holders are unwilling to provide data to data collectors. Therefore, data collectors cannot collect enough data to meet the needs of data users. The main task of data collectors is to collect data through differential privacy protection. However, data collectors also need to provide data to other social organizations. The financial industry is composed of associations or alliances of financial institutions such as banks and securities that use big data analysis technology for risk assessment, precision marketing, anti-fraud and decision support.

[0007] Therefore, in actual data interaction, the party using the data also has a demand for data; at least, the data user needs a large amount of data to support decision-making; if there is not enough data to support decision-making, it will affect the data user's trust in the data collector; therefore, the data collector needs a way to solve this problem. Summary of the invention

[0008] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a data augmentation method based on a generative adversarial model in a differential privacy scenario, thereby solving the problems raised in the above-mentioned background technology.

[0009] In order to solve the above technical problems, the basic concept of the technical solution adopted by the present invention is:

[0010] A data augmentation method based on a generative adversarial model in a differential privacy scenario includes the following steps:

[0011] S1: Obtain a real data set without noise, and generate the data into the discriminator in the adversarial model; specifically, use the one-hot algorithm to process the real data, then normalize the data, divide it into training data and test data, splice the features and labels together for subsequent data generation, and then input the training data into the discriminator; the data collector uses the one-hot algorithm to process the real data it holds, and then inputs the processed data into the discriminant network in the model. The discriminant network will learn the distribution of the real data. In order to generate data that meets the conditions of the data collector, it is necessary to train the discriminant network so that the discriminant network can correctly distinguish between true and false data;

[0012] S2: The discriminator learns the distribution of real data without noise and receives the synthetic data generated by the generator. Specifically, the generator network adjusts its own parameters to synthesize data according to the results returned by the discriminator network. The data collector performs gradient clipping on the discriminator network in each round of iterative training and adds Gaussian noise after gradient clipping. In each iteration of the training process, the parameters of the generator are first fixed, and the gradient g of each sample is calculated from the real data. Then, all the parameters of the discriminator that need to calculate the gradient are traversed, and the number of iterations of the discriminator is set according to the number of iterations of the generator. The gradient of the discriminator is clipped to ensure that the gradient of the discriminator is within the preset range; the upper and lower limits of clipping use fixed thresholds; then a series of operations are performed to protect the privacy data. After completing these operations, the generator will generate a batch of synthetic data, concatenate the noise and category labels as the input of the generator, and then the generator outputs pseudo data; the discriminator receives this batch of pseudo data, outputs the discrimination result of the pseudo data, and calculates the mean; then the method is called and back-propagated, and finally the discriminator parameters are updated; the privacy data includes the addition of Gaussian noise after gradient clipping and the accumulation and update of gradients, which can make the noise added to the data meet the definition of differential privacy;

[0013] S3: The generator receives random noise as input and tries to generate realistic data samples. Specifically, the generator first generates a batch of noise data, and concatenates the noise and category labels into pseudo data as the input of the discriminator. The discriminator outputs the discrimination result of the pseudo data and calculates the mean as the loss of the generator. Finally, the back propagation method is called to calculate the gradient of the generator and update the parameters.

[0014] S31: The generator and the discriminator will conduct adversarial training, and the generator and the discriminator will compete with each other; the generator will continuously synthesize synthetic data close to the real data, and the discriminator will try its best to identify which data are synthetic data. The specific formula is:

[0015] V(D,G)=E X～Pdata(X) [logD(X)]+E Z～Pz(Z) [log(1-D(G(Z)))]

[0016] Among them, X~Pdata(X) represents the distribution of real data, Z~PZ(z) is the distribution of noise data, and the ultimate goal of the generative adversarial network is to maximize V; that is, maximize the first and second terms at the same time; maximize the first term to make D(X) close to 1, so as to obtain the discriminator D; because x is real data, maximizing the first term strengthens the discrimination effect D(X); maximize the second term to make D(G(Z)) close to 0, so as to deceive the discriminator, because G(Z) is generated data rather than real data; thus, a generator for generating data close to real data is obtained.

[0017] S32: If any mechanism M satisfies differential privacy, then for two adjacent data sets D and D' that have at most one different data point, and any output result S of M, the specific formula is as follows:

[0018] Pr[M(D)∈S]≤e ε Pr[M(D')∈S]+δ

[0019] Among them, ε is the privacy budget. The smaller the privacy budget, the stronger the privacy protection ability, but the lower the availability of the original data. The relaxation term δ represents the possibility of differential privacy being violated.

[0020] S33: For any δ (0<δ<1), if ó satisfies the following formula, then the noise YN(0, ó2) satisfies (ε, δ)-differential privacy; the parameter ó determines the scale of the noise, and the specific formula is:

[0021]

[0022] S34: In order to track the privacy budget spent during training, Rényi differential privacy is used to add noise; suppose a random mechanism F satisfies (α, ε)-Rényi differential privacy, that is, for all adjacent data sets x and x', the following formula is satisfied, which means that the Rényi differential privacy divergence of the order α between F(x) and F(x') is restricted by ε. The specific formula is:

[0023] D α (F(x)||F(x'))≤ε;

[0024] S35: The basic mechanism of Rényi differential privacy is Gaussian mechanism, so for the sensitivity S f Function f, if it satisfies the following formula, then the mechanism satisfies (α, ε)-Rényi differential privacy;

[0025]

[0026] Optionally, the Gaussian mechanism provides relaxed differential privacy, where the sensitivity is S f The specific formula is:

[0027] S f =max||f(D)-f(D')||2.

[0028] Optionally, Gaussian distribution is a type of normal distribution, denoted as N(u, ó2); where u is the mean and ó is the variance, reflecting the numerical characteristics of the data such as expectation, variance, skewness and peak. The Gaussian distribution follows the formula as follows:

[0029]

[0030] After adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art. Of course, any product implementing the present invention does not necessarily need to achieve all the advantages described below at the same time:

[0031] 1. The present invention combines differential privacy with the generative adversarial model, which not only meets the privacy protection needs of the data holder, but also expands the data set of the data collector, thus avoiding the problem of insufficient data volume on the data collector side;

[0032] 2. The present invention adopts Rényi differential privacy to protect the privacy of data, uses Gaussian noise to add noise during the training process of the generative adversarial model, and tracks the privacy budget spent in this process;

[0033] 3. The present invention uses a series of optimization strategies to help the generative adversarial model converge. In actual adversarial training, due to the addition of Gaussian noise, the training of the generative adversarial model may be difficult to converge, so the use of optimization strategies can speed up the convergence of training.

[0034] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described below are only some embodiments. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0036] In the figure:

[0037] Figure 1 It is a method model diagram of the present invention;

[0038] Figure 2 It is a data conversion schematic diagram of the present invention;

[0039] Figure 3 It is a schematic diagram of adversarial training of the present invention;

[0040] Figure 4 Schematic diagram of Wasserstein distance of the present invention;

[0041] Figure 5 A schematic diagram of adding noise to the present invention;

[0042] Figure 6 The F1 score of the real data under different classifiers in the present invention;

[0043] Figure 7 It is the F1 score of the synthetic data in the logistic regression classifier in the present invention;

[0044] Figure 8 is the F1 score of the synthetic data in the present invention under the random forest classifier;

[0045] Fig. 9 is the F1 score of the synthetic data in the present invention under the multi-layer perception classifier;

[0046] Fig.10 It is the F1 score of the synthetic data under the gradient boosting classifier in the present invention.

[0047] It should be noted that these drawings and textual descriptions are not intended to limit the conceptual scope of the present invention in any way, but are intended to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0048] The present invention will now be described in further detail with reference to the accompanying drawings.

[0049] We summarize the existing differential privacy schemes; the concept of differential privacy was first proposed by Dwork. Data collectors can divide data into several partitions and add Laplace noise to the count of each partition; in order to adjust the level of noise, differential privacy adjusts the aggregate noise by adjusting the amount of additional noise; differential privacy can reduce the error rate of range queries by generating private histograms in terms of querying data; however, not all data holders are willing to trust data collectors and provide data to them; this will result in the amount of data owned by data collectors not necessarily meeting the needs of data users, so data collectors can use data augmentation technology to expand the data.

[0050] Gaussian noise does not meet the definition of ε-differential privacy (i.e., strict differential privacy definition), but Gaussian noise meets (ε,δ)-differential privacy (i.e., relaxed differential privacy); the Gaussian mechanism achieves privacy protection by adding Gaussian noise to the data. The Gaussian mechanism is suitable for scenarios where personal privacy needs to be protected, such as in data analysis and machine learning applications, by adding Gaussian noise to protect users' sensitive information; compared with the Laplace mechanism, the Gaussian mechanism is more effective in processing continuous data because it allows greater flexibility in noise distribution.

[0051] Generative adversarial models can synthesize generated data close to real data by learning the distribution of real data, in this way, the effect of data augmentation can be achieved; data augmentation can solve the problem caused by the limited amount of data collected by the data collector; therefore, how to expand the dataset in the differential privacy scenario is a problem that needs to be solved; the data collector needs to provide data users with data that enhances differential privacy; therefore, the expanded data should meet the definition of differential privacy; we can increase the amount of data collected by the data collector by generating synthetic data; synthetic data is an effective method for dataset expansion, which can directly generate new samples that are similar to but not exactly the same as real data; generative adversarial models have certain advantages in synthetic data generation; the generative adversarial model consists of two neural networks, namely the generator and the discriminator; the generator is responsible for generating new data samples, while the discriminator is responsible for determining whether the generated samples are real; through continuous adversarial training between the generator and the discriminator, the generator is able to generate data that is closer and closer to the real samples, thereby expanding the dataset; however, the data generated by the generative adversarial model is completely random and has not been added with differential privacy noise, which is different from the data collected in the differential privacy environment; therefore, we design a model that can generate synthetic data that meets the definition of differential privacy.

[0052] A data augmentation method based on a generative adversarial model in a differential privacy scenario includes the following steps:

[0053] Obtain a real data set without noise and generate the data into the discriminator in the adversarial model; the discriminator will learn the distribution of the real data without noise and receive the synthetic data generated by the generator; the generator will receive random noise as input and try to generate realistic data samples;

[0054] The generator and the discriminator will conduct adversarial training; adversarial training is essentially a zero-sum game process, in which the generator and the discriminator compete with each other; the generator continuously synthesizes synthetic data that is close to the real data, and the discriminator identifies which data is synthetic as much as possible; as shown in formula (1),

[0055]

[0056] Among them, X~Pdata(X) represents the distribution of real data, Z~PZ(z) is the distribution of noise data, and the ultimate goal of the generative adversarial network is to maximize V; that is, maximize the first and second terms at the same time; maximize the first term to make D(X) as close to 1 as possible, so as to obtain a better discriminator D; because x is real data, maximizing the first term can enhance the discrimination effect D(X); maximize the second term to make D(G(Z)) as close to 0 as possible, so as to better deceive the discriminator, because G(Z) is generated data rather than real data; through this training method, we can get a generator that can generate data close to real data;

[0057] If any mechanism M satisfies differential privacy, then for two adjacent data sets D and D' that have at most one different data point, and any output result S of M, we have the following formula (2):

[0058] Pr[M(D)∈S]≤e ε Pr[M(D')∈S]+δ (2)

[0059] Among them, ε is the privacy budget. The smaller the privacy budget, the stronger the privacy protection ability, but the lower the availability of the original data. The relaxation term δ represents the possibility of differential privacy being violated.

[0060] The Gaussian mechanism provides relaxed differential privacy, where the sensitivity is S f The following formula 3 shows:

[0061] S f =max||f(D)-f(D')||2 (3)

[0062] Gaussian distribution is a type of normal distribution, often written as N(u, ó2). Where u is the mean and ó is the variance, which to a certain extent can reflect the numerical characteristics of the data such as expectation, variance, skewness and peak. Gaussian distribution complies with the following formula (4);

[0063]

[0064] For any δ (0<δ<1), if ó satisfies the following formula 5, then the noisy YN(0, ó2) satisfies (ε, δ)-differential privacy. The parameter ó determines the scale of the noise:

[0065]

[0066] In order to track the privacy budget spent during training, this scheme uses Rényi differential privacy (RDP) to add noise; suppose a random mechanism F satisfies (α, ε)-RDP, that is, for all adjacent data sets x and x', it satisfies the following formula (6), which means that the Rényi divergence of the order α between F(x) and F(x') is limited by ε:

[0067] D α (F(x)||F(x'))≤ε (6)

[0068] The basic mechanism of Rényi differential privacy is the Gaussian mechanism. Therefore, for a function f with sensitivity Sf, if the following formula (7) is satisfied, the mechanism satisfies (α, ε)-RDP;

[0069]

[0070] In differential privacy scenarios, data collectors do not have enough data because they are not trusted by data holders. To solve this problem, we propose a data augmentation method based on generative adversarial models in differential privacy scenarios. This method can expand the data collector's dataset by synthesizing data.

[0071] The present invention adds differential privacy noise to the generative adversarial model, which is capable of synthesizing data that meets the definition of differential privacy, and in this way expands the data set of the data collector;

[0072] The present invention combines differential privacy with the generative adversarial model, which not only meets the privacy protection needs of the data holder, but also expands the data set of the data collector, thus avoiding the problem of insufficient data volume on the data collector side.

[0073] The present invention adopts Rényi differential privacy in terms of data privacy protection, uses Gaussian noise to add noise during the training process of the generative adversarial model, and tracks the privacy budget spent in this process;

[0074] The present invention uses a series of optimization strategies to help the generative adversarial model converge. In actual adversarial training, due to the addition of Gaussian noise, the training of the generative adversarial model may be difficult to converge, so the use of optimization strategies can speed up the convergence of training.

[0075] Specific:

[0076] The mechanism of the present invention includes the steps of data processing, noise data input and adversarial training, wherein the data processing is to facilitate the discriminator model to understand the characteristics of real data; the noise data input step is to enable the generator to generate false samples from random noise; and the adversarial training is that the generator and the discriminator repeatedly play games until the total number of generated data is completed.

[0077] like Figure 1 As shown in the figure, the framework of a data augmentation scheme based on a generative adversarial model in a differential privacy scenario is presented; first, the real data is processed using a one-hot algorithm, then the data is normalized, and the training data and test data are divided, and the features and labels are concatenated together for subsequent data generation; the training data is then input into the discriminator, which estimates the probability distribution from the training data and determines whether the generator’s data is real or synthetic based on the probability distribution; the generator will continue to adjust until it can synthesize generated data that the discriminator cannot distinguish between true and false.

[0078] Specific:

[0079] A method for data augmentation in a differential privacy scenario, the implementation steps comprising:

[0080] The data collector processes the real data it holds using the one-hot algorithm and then inputs the processed data into the discriminant network in the model. The discriminant network will learn the distribution of the real data. Figure 2 As shown;

[0081] Among them, the one-hot algorithm is a common method for processing categorical features. It can help the model understand and utilize these features, thereby improving the performance of the model. And the one-hot encoding plays a role in expanding features to a certain extent, because it converts each category value into a new feature, increasing the number of features.

[0082] In order to generate data that meets the conditions of the data collector as much as possible, we need to train the discriminant network so that it can correctly distinguish true and false data; that is, when the input data is true, the higher the probability of the discriminator output, the better; when the input data is false, the lower the probability of the discriminator output, the better; and the generator network generates enough data to make the discriminator judge it as true as much as possible; the specific situation is as follows Figure 3 shown.

[0083] The discriminant network samples and discriminates the training data. If the real data is distinguished as true as possible, that is, E x～pdata The value of [logD(x)] is as large as possible; and the discriminant network judges the data generated by the generator as false, that is, E x～pg The value of [log(1-D(x))] should be as large as possible; while the generated network should minimize E x～pg [log(1-D(x))].

[0084] In order to solve the problem of model instability during training, we do not need to carefully balance the training degree of the generator and the discriminator, nor do we need to carefully design the network architecture. Therefore, we use Wasserstein GAN (WGAN) to complete the training process. WGAN mainly measures the minimum gap between two different distributions through the Wasserstein distance. The specific situation is as follows Figure 4 shown.

[0085] Through WGAN, the generative network model can not only quickly grasp the distribution of real data, but also ensure that there are accurate values ​​to indicate the progress of training during a stable training process. The smaller the value, the better the training effect, which means that the synthetic data of the generative model is closer to the real data.

[0086] Specific:

[0087] A method for adding differential privacy noise during training of a generative adversarial model, the implementation steps of which include: utilizing the characteristic that the generator network will adjust its own parameters to synthesize data according to the results returned by the discriminator network; the data collector performs gradient clipping on the discriminator network during each round of iterative training, and adds Gaussian noise after gradient clipping; the specific training process is that in each iteration of the training process, we first fix the parameters of the generator and calculate the gradient g of each sample from the real data; then traverse all the parameters of the discriminator that need to calculate the gradient, and set the number of iterations of the discriminator according to the number of iterations of the generator, and at the same time perform gradient clipping on the gradient of the discriminator to ensure that the gradient of the discriminator is within a preset range; the upper and lower limits of clipping use fixed thresholds; and then perform a series of operations to protect the privacy data, including adding Gaussian noise after gradient clipping and accumulating the gradient And update, this operation can make the noise added to the data meet the definition of differential privacy; when these operations are completed, the generator will generate a batch of synthetic data, concatenate the noise and category labels as the input of the generator, and then the generator outputs pseudo data; the discriminator receives this batch of pseudo data, outputs the judgment result of the pseudo data, and calculates the mean; then calls the method and back propagates, and finally updates the discriminator parameters; after training the discriminator, the parameters of the discriminator are fixed, and the generator is trained to minimize the judgment error of the discriminator on the generated data, thereby generating more realistic synthetic data; first, the generator generates a batch of noise data, and concatenates the noise and category labels into pseudo data as the input of the discriminator. The discriminator outputs the judgment result of the pseudo data and calculates the mean as the loss of the generator; finally, the method back propagates to calculate the gradient of the generator and update the parameters, as shown in the following figure. Figure 5 shown.

[0088] Specifically, the present invention uses the F1 score as an indicator to evaluate the generated data. The F1 score is shown in formula (8). F1 can comprehensively consider the accuracy and recall rate and can be used to evaluate the quality of the generated data.

[0089]

[0090] Figure 6 The F1 value of real data under different classifiers. The X-axis is the model, and the Y-axis is the F1 score. As can be seen from the figure, although the F1 values ​​of real data under logistic regression classifier, random forest classifier, multi-layer perceptron classifier, and gradient boosting classifier are different, they are all roughly distributed around 0.9. This shows that the closer the F1 value of the generated data is to 0.9, the better its accuracy and recall rate are;

[0091] Figure 7 is the F1 value of the generated data under the logistic regression classifier. The X-axis is the privacy budget, and the Y-axis is the value of the F1 score. We conducted three sets of experiments and observed the F1 values ​​of the generated data when the privacy budget was 2, 4, 6, 8, and 10. It can be seen that the experimental results roughly meet the characteristics of differential privacy, that is, the larger the privacy budget, the higher the F1 value, which means that the data availability is better;

[0092] Figure 8 is the F1 score of the generated data under the random forest classifier. The X-axis is the privacy budget, and the Y-axis is the value of the F1 score. We conducted three sets of experiments with privacy budgets of 2, 4, 6, 8, and 10. It can be seen that the results are scattered, but most of the F1 is distributed above 0.6. The characteristics of the random forest classifier are random sampling of data and random selection of features. Since differential privacy noise is added to the synthetic data, the random forest classifier cannot accurately identify some feature values. From this experiment, it can be seen that as the privacy budget increases, the F1 score of the synthetic data under the random forest classifier also increases, which indicates that the distortion of the data is caused by Gaussian noise, rather than insufficient convergence of the model;

[0093] Fig. 9 is the F1 score of the generated data under the multi-layer perceptron classifier. The X-axis is the number of bits of the privacy budget, and the Y-axis is the value of the F1 score. As can be seen from the figure, most of the F1 scores of the three groups of experiments are relatively concentrated, and conform to the rule that the larger the privacy budget, the higher the F1 score value, which indicates that the data distortion is caused by Gaussian noise. The multi-layer perceptron classifier is a general classifier that can estimate any nonlinear function. Therefore, this experiment can prove that it is feasible to add Gaussian noise to synthesize data in generative adversarial training;

[0094] Fig.10is the F1 value of the generated data under the gradient boosting classifier. The X-axis is the privacy budget and the Y-axis is the F1 score. The gradient boosting classifier can combine multiple weak classifiers or regressors into a strong classifier or regressor to improve the prediction performance. As can be seen from the figure, all three groups of experiments meet the characteristic that the F1 score of the data increases with the increase of the privacy budget, which shows that it is feasible to synthesize data that meets the definition of differential privacy through the generative adversarial model.

[0095] The technical effect of the present invention is to solve the problem that the data collector in the differential privacy scenario has insufficient data in actual data interaction in order to cope with the data holder's distrust of the data collector and the data holder's unwillingness to provide data. The present invention designs a generative data model at a limited privacy cost by adding and continuously tracking differential privacy noise during the training process of the generative adversarial network, which can synthesize data to meet the needs of the data holder.

[0096] The technical secret of the present invention lies in that by understanding that when the data collector has insufficient data, the purpose of data augmentation can be achieved by combining generative adversarial networks and differential privacy noise.

[0097] The limitation of the present invention is that the scope of data augmentation is currently limited to text-type data. Due to the limitation of data types, the main application scope of the present invention is Internet security research, product optimization, and social science research.

[0098] The present invention is not limited to the above-mentioned embodiments. Anyone should be aware that any structural changes made under the enlightenment of the present invention, and any technical solutions that are the same or similar to the present invention, fall within the protection scope of the present invention. The technology, shape, and structural parts not described in detail in the present invention are all well-known technologies.

Claims

1. A data augmentation method based on a generative adversarial model in a differential privacy scenario, characterized in that: The steps include: S1: Obtain a real data set without noise, and generate the data into the discriminator in the adversarial model; S2: The discriminator learns the distribution of real data without noise and receives synthetic data generated by the generator; S3: The generator receives random noise as input and tries to generate realistic data samples; S31: The generator and the discriminator will conduct adversarial training, and the generator and the discriminator will compete with each other; the generator will continuously synthesize synthetic data close to the real data, and the discriminator will try its best to identify which data are synthetic data. The specific formula is: Among them, X~Pdata(X) represents the distribution of real data, Z~PZ(z) is the distribution of noise data, and the ultimate goal of the generative adversarial network is to maximize V; that is, maximize the first term and the second term at the same time; maximize the first term so that D(X) is close to 1, thereby obtaining the discriminator D; because x is real data, maximizing the first term strengthens the discrimination effect D(X); maximize the second term so that D(G(Z)) is close to 0, which can deceive the discriminator, because G(Z) is generated data rather than real data; thus, a generator for generating data close to real data is obtained; S32: If any mechanism M satisfies differential privacy, then for two adjacent data sets D and D' that have at most one different data point, and any output result S of M, the specific formula is as follows: Pr[M(D)∈S]≤e ε Pr[M(D')∈S]+δ Among them, ε is the privacy budget. The smaller the privacy budget, the stronger the privacy protection ability, but the lower the availability of the original data. The relaxation term δ represents the possibility of differential privacy being violated. S33: For any δ (0<δ<1), if ó satisfies the following formula, then the noise YN(0, ó2) satisfies (ε, δ)-differential privacy; the parameter ó determines the scale of the noise, and the specific formula is: S34: In order to track the privacy budget spent during training, Rényi differential privacy is used to add noise; suppose a random mechanism F satisfies (α, ε)-Rényi differential privacy, that is, for all adjacent data sets x and x', the following formula is satisfied, which means that the Rényi differential privacy divergence of the order α between F(x) and F(x') is restricted by ε. The specific formula is: D α (F(x)||F(x'))≤ε; S35: The basic mechanism of Rényi differential privacy is Gaussian mechanism, so for the sensitivity S f Function f, if it satisfies the following formula, then the mechanism satisfies (α, ε)-Rényi differential privacy; 2. According to claim 1, the data augmentation method based on the generative adversarial model in the differential privacy scenario is characterized in that: The Gaussian mechanism provides relaxed differential privacy, where the sensitivity is S f The specific formula is: S f =max||f(D)-f(D')||2。 3. According to claim 2, the data augmentation method based on the generative adversarial model in the differential privacy scenario is characterized in that: Gaussian distribution is a type of normal distribution, denoted as N(u, ó2). u is the mean and ó is the variance, which reflects the numerical characteristics of the data such as expectation, variance, skewness and peak. The Gaussian distribution follows the formula as follows:

4. The data augmentation method based on a generative adversarial model in a differential privacy scenario according to claim 1, characterized in that: Specifically in S1, the one-hot algorithm is used to process the real data, and then the data is normalized and divided into training data and test data. The features and labels are concatenated together for subsequent data generation, and then the training data is input into the discriminator.

5. According to claim 4, the data augmentation method based on the generative adversarial model in the differential privacy scenario is characterized in that: The data collector processes the real data it holds using the one-hot algorithm and then inputs the processed data into the discriminant network in the model. The discriminant network will learn the distribution of the real data. In order to generate data that meets the conditions of the data collector, it is necessary to train the discriminant network so that it can correctly distinguish between true and false data.

6. The data augmentation method based on a generative adversarial model in a differential privacy scenario according to claim 1, characterized in that: Specifically in S2, the generative network adjusts its own parameters to synthesize data according to the results returned by the discriminative network. The data collector performs gradient clipping on the discriminative network during each round of iterative training and adds Gaussian noise after gradient clipping.

7. The data augmentation method based on a generative adversarial model in a differential privacy scenario according to claim 6, characterized in that: In each iteration of the training process, the parameters of the generator are first fixed, and the gradient g of each sample is calculated from the real data; then all the parameters of the discriminator that need to calculate the gradient are traversed, and the number of discriminator iterations is set according to the number of generator iterations. At the same time, the gradient of the discriminator is clipped to ensure that the gradient of the discriminator is within the preset range.

8. The data augmentation method based on a generative adversarial model in a differential privacy scenario according to claim 7, characterized in that: The upper and lower limits of cropping use fixed thresholds. Then a series of operations are performed to protect the privacy data. After completing these operations, the generator will generate a batch of synthetic data, concatenating the noise and category labels as the input of the generator, and then the generator will output pseudo data. The discriminator receives this batch of pseudo data, outputs the discrimination results of the pseudo data, and calculates the mean; then calls the method and backpropagates, and finally updates the discriminator parameters.

9. The data augmentation method based on a generative adversarial model in a differential privacy scenario according to claim 7, characterized in that: Privacy data includes Gaussian noise added after gradient clipping, as well as the accumulation and update of gradients, which can make the noise added to the data conform to the definition of differential privacy.

10. The data augmentation method based on a generative adversarial model in a differential privacy scenario according to claim 1, characterized in that: In S3, specifically, the generator first generates a batch of noise data, and concatenates the noise and category labels into pseudo data as the input of the discriminator. The discriminator outputs the judgment result of the pseudo data and calculates the mean as the loss of the generator. Finally, the back propagation method is called to calculate the gradient of the generator and update the parameters.

Images