Multi-party data sharing method and related equipment based on generative adversarial network
Through a multi-party data sharing method based on a generative adversarial network, combined with differential privacy noise perturbation, the acceptance probability of synthetic data is calculated, and the problem of poor generalization ability of generative models in the prior art is solved, and shared data with high efficiency and privacy protection is achieved.
Patent Information
- Application Number
- CN202111522627.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-13
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-12-13
AI Technical Summary
Existing multi-party data sharing technology is difficult to maintain the generalization ability of the generative model, resulting in poor utility of the ultimate shared data.
Using a method based on a generative adversarial network, the average acceptance probability of synthetic data is calculated through alternating training of generator and discriminator, combined with differential privacy noise perturbation, and the synthetic data that meets the preset conditions are used as shared data.
This improves the effectiveness of the ultimate shared data, ensures privacy protection, while the generated data can better meet the needs of the predictive model.
Smart Images

Figure CN114443593B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to data sharing technology, and in particular to a multi-party data sharing method and related equipment based on a generative adversarial network. Background Art
[0002] Privacy-preserving multi-party data sharing technology provides a feasible solution to the privacy leakage problem caused by multi-party data sharing. Differential Privacy (DP) technology proposed in recent years provides a feasible solution to the privacy leakage problem caused by data sharing. Different from the traditional anonymity-based privacy model, differential privacy provides a strict and quantifiable privacy protection method for sensitive data. By adding an appropriate amount of noise to the statistical results to ensure that modifying a record in the data set will not have a significant impact on the statistical results, the requirements of privacy protection are achieved.
[0003] Multi-party data sharing can be achieved by each party publishing its own local data set under privacy protection constraints. The specific method can be: each data owner uses its own local data set to build a generative model, and then uses the learned generative model to generate data, and finally integrates the data generated by each party to form a shared data set. However, due to the limited number of data records in each local data set, it is difficult for the learned generative model to maintain good generalization ability, resulting in poor utility of the shared data obtained in the end. Summary of the invention
[0004] In view of this, the purpose of this application is to propose a multi-party data sharing method and related equipment based on a generative adversarial network.
[0005] Based on the above purpose, the present application provides a multi-party data sharing method based on a generative adversarial network, wherein the generative adversarial network includes a generator network and at least one discriminator network, including:
[0006] Sending random vectors to the generator network one by one for processing to obtain synthetic data corresponding to each of the random vectors;
[0007] Sending the synthetic data corresponding to all the random vectors to each of the discriminator networks for discrimination, obtaining a discrimination result of each of the discriminator networks, and calculating an average acceptance probability of the synthetic data corresponding to each of the random vectors according to the discrimination result of each of the discriminator networks;
[0008] The preset probability is compared with the average acceptance probability of the synthetic data corresponding to each of the random vectors. When the average acceptance probability of the synthetic data corresponding to a certain random vector is greater than or equal to the preset probability, the synthetic data corresponding to the random vector is used as shared data.
[0009] In some embodiments, the multi-party data sharing method based on a generative adversarial network further includes:
[0010] The amount of the shared data obtained by counting is counted, and when the amount of the shared data reaches a preset threshold, the generation of the synthetic data is stopped.
[0011] In some embodiments, the training step of generating an adversarial network includes:
[0012] Sending synthetic data synthesized by the generator network in the initial generative adversarial network to each discriminator network in each of the initial generative adversarial networks respectively;
[0013] Respectively extracting local data corresponding to each discriminator network in the initial generative adversarial network and recording them into each discriminator network in the initial generative adversarial network;
[0014] Discriminate the data generation record and the local data record to obtain a discrimination result output by each discriminator network in the initial generative adversarial network;
[0015] Calculate the loss function of each discriminator network in the initial generative adversarial network according to the discrimination results;
[0016] Calculate the gradient information of each discriminator network in the initial generative adversarial network according to the loss function of each discriminator network in the initial generative adversarial network;
[0017] Using differential privacy noise to perturb the gradient information of each discriminator network in the initial generative adversarial network respectively;
[0018] Update the parameters of each discriminator network in the initial generative adversarial network according to the perturbed gradient information to obtain the discriminator network in the next round of iteration;
[0019] The generator network in the initial generative adversarial network is updated according to the discriminator network with updated parameters, and the updated generator network is used as the generator network in the next round of iteration, until the iteration is completed to obtain the trained generative adversarial network.
[0020] In some implementations, iteratively training the generator network in the initial generative adversarial network and the discriminator network in each of the initial generative adversarial networks to obtain the trained generative adversarial network further includes:
[0021] Presetting an iteration round number threshold, when the iteration round number reaches the iteration round number threshold, stopping the iterative training, and outputting the generative adversarial network trained in this round of iterative training as the trained generative adversarial network;
[0022] or,
[0023] At the end of each round of iteration, the total privacy budget consumed by the iterative training is calculated. When the total privacy budget is greater than the preset privacy budget threshold, the iterative training is stopped, and the generative adversarial network trained in this round of iteration is output as the trained generative adversarial network.
[0024] In some embodiments, the differential privacy noise is Gaussian noise sampled from a Gaussian distribution.
[0025] In some embodiments, perturbing the gradient information using differential privacy noise includes:
[0026] Pruning the L2-norm of each discriminator network in the initial generative adversarial network respectively;
[0027] Reducing the scale of the differential privacy noise according to a preset noise attenuation function;
[0028] The reduced-scale differential privacy noise is used to perturb the pruned gradient information to obtain the perturbed gradient information.
[0029] In some implementations, calculating the average acceptance probability of the synthetic data corresponding to each of the random vectors according to the discrimination results of each of the discriminator networks includes:
[0030] The Sigmoid function is used to calculate the acceptance probability of each discriminator network for the synthetic data corresponding to each random vector;
[0031] The average acceptance probability of the synthetic data is calculated according to the calculated acceptance probability of each discriminator network for the single synthetic data.
[0032] Based on the same concept, the present application also provides a multi-party data sharing device based on a generative adversarial network, comprising:
[0033] The data generation module is configured to send random vectors one by one to the generator network in the trained generative adversarial network for processing to obtain corresponding synthetic data;
[0034] A data discrimination module is configured to send the synthetic data corresponding to each of the random vectors to each discriminator network in the trained generative adversarial network for discrimination, obtain the discrimination result of each discriminator network, and calculate the average acceptance probability of the synthetic data corresponding to each of the random vectors according to the discrimination result of each discriminator network;
[0035] A data sharing module is configured to compare the average acceptance probability of the synthetic data corresponding to each of the random vectors with a preset probability, and obtain shared data when the average acceptance probability of the synthetic data corresponding to a certain random vector is greater than or equal to the preset probability;
[0036] The judgment module is configured to stop the trained generative adversarial network from running when the amount of the shared data reaches a preset threshold.
[0037] In some embodiments, the multi-party data sharing device based on a generative adversarial network also includes a judgment module, which is configured to count the number of the shared data obtained, and stop generating the synthetic data when the number of the shared data reaches a preset threshold.
[0038] Based on the same concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described above when executing the program.
[0039] Based on the same concept, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to implement any of the methods described above.
[0040] From the above, it can be seen that the multi-party data sharing method based on the generative adversarial network provided in the present application is based on the generative adversarial network, comprehensively considers the discrimination results of multiple synthetic data, and uses the synthetic data that meets the preset conditions as shared data, so that the final shared data has higher utility. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0042] Figure 1 This is a flow chart of a multi-party data sharing method based on a generative adversarial network according to an embodiment of the present application;
[0043] Figure 2 A schematic diagram of a scenario in which data is shared by multiple parties;
[0044] Figure 3 This is a flow chart of the generative adversarial network training method according to an embodiment of the present application;
[0045] Figure 4This is a flow chart of the differential privacy noise perturbation gradient information method of an embodiment of the present application;
[0046] Figure 5 This is a schematic diagram of performance comparison of the embodiments of the present application;
[0047] Figure 6a This is a schematic diagram showing a comparison of the prediction performance of various data supported by the Adult dataset of an embodiment of the present application;
[0048] Figure 6b This is a schematic diagram showing a comparison of the prediction performance of various data supported by the Bank data set of an embodiment of the present application;
[0049] Figure 7a This is a schematic diagram of the accuracy performance comparison of the Adult data set of an embodiment of the present application;
[0050] Figure 7b This is a schematic diagram of the accuracy performance comparison of the Bank data set of an embodiment of the present application;
[0051] Figure 8 This is a schematic diagram of a multi-party data sharing device based on a generative adversarial network according to an embodiment of the present application;
[0052] Fig. 9 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0053] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.
[0054] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0055] As described in the background technology section, existing multi-party data sharing solutions are still difficult to meet the needs of high-efficiency multi-party data sharing. In the process of implementing this application, the applicant found that privacy-preserving multi-party data sharing technology provides a feasible solution to the privacy leakage problem caused by multi-party data sharing. Different from traditional anonymity-based privacy models (for example, k-anonymity [1] and l-diversity, etc.), differential privacy provides a strict and quantifiable privacy protection method for sensitive data. By adding an appropriate amount of noise to the statistical results to ensure that modifying a record in the data set will not have a significant impact on the statistical results, the requirements of privacy protection are achieved.
[0056] In view of this, one or more embodiments of this specification provide a multi-party data sharing method (ATLAS) based on a generative adversarial network. Based on the generative adversarial network, the discrimination results of multiple synthetic data are comprehensively considered, and the synthetic data that meets the preset conditions is used as shared data. The technical solutions of each specific embodiment of this application are described below.
[0057] refer to Figure 1 In one embodiment of the present specification, a multi-party data sharing method based on a generative adversarial network is provided, wherein the generative adversarial network includes a generator network and at least one discriminator network, and the method includes the following steps:
[0058] Step S101, sending random vectors to the generator network one by one for processing, to obtain synthetic data corresponding to each random vector;
[0059] In this embodiment, the generative adversarial network (GAN) includes two networks, namely a generator network and a discriminator network. During training, the role of the discriminator network is to distinguish between samples generated by the generator and real samples under privacy protection constraints, while the role of the generator network is to generate generated samples that are as close to real samples as possible to effectively capture the distribution characteristics of real data. After training, the generator can be used to generate new shared data to achieve user privacy protection. In real scenarios, real samples are often distributed among multiple data owners (for example, hospitals have residents' medical records, and banks have customers' account records). Reference Figure 2 , each data owner k holds a discriminator network D k , and the third-party platform holds a generator network G. In the figure, the mall P1 holds the discriminator network D1, the bank P2 holds the discriminator network D2, and the hospital P3 holds the discriminator network D3. These discriminator networks are connected to the third-party platform through encryption. The random vector is sampled from a prior distribution.
[0060] Step S102: sending the synthetic data corresponding to all the random vectors to each of the discriminator networks for discrimination, obtaining the discrimination result of each of the discriminator networks, and calculating the average acceptance probability of the synthetic data corresponding to each of the random vectors according to the discrimination result of each of the discriminator networks;
[0061] In this step, the discriminator network discriminates the synthetic data against its own corresponding local data, and the discrimination result can indicate the authenticity of the synthetic data relative to the local data.
[0062] Step S103: compare the preset probability with the average acceptance probability of the synthetic data corresponding to each of the random vectors; when the average acceptance probability of the synthetic data corresponding to a certain random vector is greater than or equal to the preset probability, use the synthetic data corresponding to the random vector as shared data.
[0063] In this embodiment, the generative adversarial network is obtained through training, which involves the training of two neural networks, namely the discriminator network held by each data owner and the generator network held by the third-party platform. The training of the generative adversarial network is carried out by alternating the training of the generator network and the discriminator network, and the two alternately optimize the following objective function in the form of a game:
[0064]
[0065] Among them, p data Represents the distribution of a data set. It is a commonly used representation in probability theory; is the operator for finding the gradient; E(a) is the mean of a; λ is the penalty coefficient, x k is the sample (i.e. synthetic data), is the interpolated sample, ∈ is the interpolation coefficient sampled from the uniform distribution [0,1]. The latent variable z (usually random noise that follows a Gaussian distribution) generates samples through the generator network G. For the discriminator D, this is a binary classification problem. V(D,G) is the cross entropy loss commonly seen in binary classification problems. In order to ensure that V(D,G) reaches the maximum value, we usually train the discriminator k times and then iterate the generator once (k is usually 1). The training steps of the generative adversarial network can be expressed as:
[0066] Initialize the parameters of the generator network G and the discriminator network D.
[0067] Extract n samples from the training set, and the generator network generates n samples using the defined noise distribution. The generator network G is fixed, and the discriminator network D is trained to distinguish between true and false as much as possible.
[0068] After cyclically updating the discriminator network D k times, update the generator network G once to make the discriminator network unable to distinguish between true and false as much as possible.
[0069] After multiple updates and iterations, in an ideal state, the final discriminator network D cannot distinguish whether a sample comes from the real training sample set or from the sample generated by the generator network G. At this time, the probability of identification is 0.5, and the training is completed.
[0070] In this embodiment, the number of samples n and the number of cycles k are selected according to actual conditions.
[0071] In particular, the discriminator network is used to distinguish between local sensitive data records and records generated by the generator network, while the generator network is committed to generating synthetic records that are considered "real" by these K discriminator networks. Given the above different goals, the objective functions of the generator network and the discriminator network can be written as follows:
[0072]
[0073]
[0074] The gradient update of the generator network requires performing gradient descent on its objective function, which, however, contains sensitive information, namely the feedback of the discriminator network. In order to update the generator gradient without leaking the privacy of each data owner, the generator network gradient calculation is first decomposed, and then the sensitive information related to the discriminator network is determined, which is then desensitized.
[0075] Based on the chain rule of calculus, the gradient calculation of the generator network can be decomposed as follows:
[0076]
[0077] Here m is the batch size, is the gradient of the generator network, θ is the weight parameter, z i is the latent vector sampled from the prior distribution. The above formula indicates that the calculation of the generator gradient involves two factors, namely and Among them, the first factor is the feedback of the discriminator, which contains sensitive information. The second factor It is non-sensitive and can be calculated directly by a third-party platform.
[0078] After the above decomposition of gradient calculation, refer to Figure 3 , the parameters of the generator network can be desensitized by the iterative process of the following steps, and the discriminator network feedback with privacy protection can be generated. That is, the generative adversarial network can include the following training steps:
[0079] Step S301, respectively sending the synthesized data synthesized by the generator network in the initial generative adversarial network to each discriminator network in each of the initial generative adversarial networks;
[0080] Before this step, the parameters of the discriminator network need to be initialized, including the first-order momentum estimation of the Adam algorithm and the second-order momentum estimate and the noise scale σ0 for differential privacy.
[0081] In this step, each data owner receives a batch of synthetic data from the generator network
[0082] Step S302, respectively extracting local data corresponding to each discriminator network in the initial generative adversarial network and recording them into each discriminator network in the initial generative adversarial network;
[0083] In this step, each data owner extracts a batch of sensitive data records from the local area (i.e., local data records).
[0084] Step S303, discriminating the data generation record and the local data record, and obtaining a discrimination result output by each discriminator network in the initial generative adversarial network;
[0085] Step S304, respectively calculating the loss function of each discriminator network in the initial generative adversarial network according to the discrimination results;
[0086] In this step, the loss function can be expressed as: Among them, λ is the penalty coefficient, is the interpolated sample, ∈ is the interpolation coefficient sampled from a uniform distribution [0,1].
[0087] Step S305, respectively calculating the gradient information of each discriminator network in the initial generative adversarial network according to the loss function of each discriminator network in the initial generative adversarial network;
[0088] In this step, the gradient information can be expressed as:
[0089] Step S306: using differential privacy noise to perturb the gradient information of each discriminator network in the initial generative adversarial network;
[0090] In this step, the perturbation result can be expressed as: Among them, σ0 is the initial noise scale, t is the iteration round, m is the batch size, and C is the clipping threshold.
[0091] Step S307: updating the parameters of each discriminator network in the initial generative adversarial network according to the perturbed gradient information to obtain the discriminator network in the next round of iteration;
[0092] In this step, the first-order momentum estimate of the parameters of the discriminator network is and the second-order momentum estimate Can be updated to: Here ⊙ represents the multiplication of corresponding elements, and β is the momentum coefficient. Afterwards, the weights of the discriminator network are updated: Among them, η and ξ are weight coefficients, which can be selected according to actual conditions.
[0093] Step S308, updating the generator network in the initial generative adversarial network according to the discriminator network with updated parameters, and obtaining the updated generator network as the generator network in the next round of iteration, until the iteration is completed to obtain the trained generative adversarial network.
[0094] It can be seen that in the multi-party data sharing method based on the generative adversarial network of the embodiment of the present application, each data owner holds a discriminator network, which is trained using a local sensitive data set. Subsequently, the feedback information of the discriminator network is desensitized and uploaded to a third-party platform. The third-party platform uses the collected feedback information of the discriminator network to update the parameters of the generator network. After the training is completed, the generator network generates shared data with the assistance of the discriminator network. Since the feedback information from each data owner is used in the updating process of the generator network, the final shared data can have a higher utility.
[0095] In some other embodiments, the multi-party data sharing method based on the generative adversarial network in the aforementioned embodiment may further include:
[0096] The amount of the shared data obtained by counting is counted, and when the amount of the shared data reaches a preset threshold, the generation of the synthetic data is stopped.
[0097] In this embodiment, when the amount of the shared data reaches a preset threshold, the generated shared data can already meet the data volume requirement, and the generation of the synthesized data can be stopped.
[0098] In some other embodiments, the iterative training of the generator network in the initial generative adversarial network and the discriminator network in each of the initial generative adversarial networks described in the aforementioned embodiments to obtain the trained generative adversarial network may further include:
[0099] Presetting an iteration round number threshold, when the iteration round number reaches the iteration round number threshold, stopping the iterative training, and outputting the generative adversarial network trained in this round of iterative training as the trained generative adversarial network;
[0100] or,
[0101] At the end of each round of iteration, the total privacy budget consumed by the iterative training is calculated. When the total privacy budget is greater than the preset privacy budget threshold, the iterative training is stopped, and the generative adversarial network trained in this round of iteration is output as the trained generative adversarial network.
[0102] In this embodiment, the iterative operation can be terminated when a preset number of iterations is met, or iterative training can be stopped when the total privacy budget is greater than a preset privacy budget threshold. The iterative training can be completed when either of the two is met.
[0103] In some other embodiments, the differential privacy noise described in the aforementioned embodiments is Gaussian noise sampled from a Gaussian distribution.
[0104] In some other embodiments, reference Figure 4 The above-mentioned embodiment of using differential privacy noise to perturb the gradient information may further include the following steps:
[0105] Step S401, pruning the L2-norm of each discriminator network in the initial generative adversarial network respectively;
[0106] In this step, pruning can constrain the sensitivity of the discriminator network
[0107] Step S402: reducing the scale of the differential privacy noise according to a preset noise attenuation function;
[0108] Step S403: Use the reduced-scale differential privacy noise to perturb the pruned gradient information to obtain perturbed gradient information.
[0109] In this step, the gradient information after perturbation can be expressed as Among them, I is the identity matrix, γ is the decay rate, σ0 is the initial noise scale, and t is the iteration round.
[0110] In this embodiment, the noise adaptive perturbation method is based on the following ideas: On the one hand, since the goal of data sharing is to generate a shared data set that satisfies differential privacy protection, what needs to be considered is the privacy guarantee of the final shared data, rather than the privacy loss of a single training iteration. On the other hand, as the training progresses, the l2-norm of the gradient of the generator network will become smaller and smaller, and therefore more sensitive to perturbations. Therefore, by adaptively adjusting the noise level of the Gaussian mechanism during the training process, the utility of the data can be better guaranteed under the same privacy budget.
[0111] In some other embodiments, the method of calculating the average acceptance probability of the synthetic data corresponding to each of the random vectors according to the discrimination results of each of the discriminator networks described in the above embodiments may further include the following steps:
[0112] The Sigmoid function is used to calculate the acceptance probability of each discriminator network for the synthetic data corresponding to each random vector;
[0113] The average acceptance probability of the synthetic data is calculated according to the calculated acceptance probability of each discriminator network for the single synthetic data.
[0114] In this embodiment, ideally, all the information contained in the weights of the discriminator network can be fed into the generator network through a minimax training procedure, and the trained generator network can reproduce the real data distribution. The synthesis of shared records is done by following the standard practice in GAN, that is, providing the trained generator network with a latent vector z sampled from the prior distribution. i , and then use the output of the generator network as the shared data. However, due to limited privacy budgets, the generator network and the discriminator network may not have been trained for a long enough time. Therefore, using only the trained generator network to generate shared records will ignore the valuable information learned by the discriminator. The output of the discriminator can indicate how true its input records are. Therefore, when using the trained generator network to synthesize shared records, the trained discriminator network can still provide guidance. In this embodiment, a trained discriminator network is used to improve the utility of the final shared data. The synthesized records are input into the trained discriminator network to obtain an acceptance probability proportional to the discriminator output. Specifically, for the synthesized records The discriminator The acceptance probability is:
[0115] As a specific example, the synthetic data and local data are compared. When making the comparison, the local data adopts the widely used real datasets Adult and Bank. The Adult dataset contains 48,842 records of US census data; the Bank dataset contains 45,211 account records of Portuguese banking institutions. The statistics of the two datasets are shown in Table 1 below.
[0116] Table 1 Statistical analysis of Adult and Bank datasets
[0117]
[0118] For IndDPGAN and FedDPGAN, we track their RDP guarantees and convert them to δ = 10 -5 The learning rate of the generator network and the discriminator network is α g and α d All are set to 1×10 -4 , the gradient penalty coefficient λ is set to 10. The clipping threshold C is set to 1. In the degraded version of ATLAS, the noise scale is set to σ = 1.1. In the standard version of ATLAS, we set the noise scale to σ = 2.1 and the decay rate γ = 1.05×10 -4 Table 2 lists the hyperparameter values used in the experiments.
[0119] Table 2 Hyperparameter settings
[0120]
[0121]
[0122] The method of the embodiment of the present application is evaluated by following the standard practice of data sharing tasks, that is, measuring the utility of shared data by the effectiveness of machine learning. In particular, we first train the prediction model using the shared data, and then test the trained prediction model on the actual test set. The higher the accuracy of the prediction model, the better the data practicality.
[0123] The following is a comparison of the accuracy test results of the present invention and the existing methods to illustrate the effect and performance of the present invention. Figure 5 , Figure 6a , Figure 6b , Figure 7a and Figure 7b As shown. Figure 5 , Figure 6a , Figure 6b , Figure 7a and Figure 7bIt can be seen that the method of the embodiment of the present application is superior to the existing methods under different prediction models, different privacy budgets and different numbers of participants, and the model accuracy is close to that of directly training with sensitive data sets, which shows that the method of the embodiment of the present application can better preserve the original distribution characteristics of the original data under a given privacy budget.
[0124] from Figure 6a and Figure 6b It can be seen that the shared data generated by the method of the embodiment of the present application can better support a variety of data prediction or machine learning tasks. Good performance is maintained under different privacy budgets. This is because the collaborative discriminator filtering and adaptive noise perturbation methods can ensure a better balance between privacy and data utility. In addition, the learning paradigm of the method of the embodiment of the present application can also use the data of all parties to guide model updates. In addition, the 1-versus-K design of this method makes it more likely that the generator network will receive information signals from the discriminator network, thereby better guiding the generator to approach the real data distribution.
[0125] from Figure 7a and Figure 7b It can be seen that the accuracy of the method of the embodiment of the present application decreases less than that of the baseline. The reason is that increasing the number of discriminator networks will reduce the variance of the minimum maximum objective function during the entire operation and accelerate the convergence speed.
[0126] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.
[0127] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0128] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a multi-party data sharing device based on a generative adversarial network.
[0129] refer to Figure 8, the multi-party data sharing device based on the generative adversarial network comprises:
[0130] The data generation module 801 is configured to send random vectors to the generator network one by one for processing to obtain synthetic data corresponding to each random vector;
[0131] The data discrimination module 802 is configured to send the synthetic data corresponding to all the random vectors to each of the discriminator networks for discrimination, obtain the discrimination result of each of the discriminator networks, and calculate the average acceptance probability of the synthetic data corresponding to each of the random vectors according to the discrimination result of each of the discriminator networks;
[0132] The data sharing module 803 is configured to compare the preset probability with the average acceptance probability of the synthetic data corresponding to each of the random vectors, and when the average acceptance probability of the synthetic data corresponding to a certain random vector is greater than or equal to the preset probability, the synthetic data corresponding to the random vector is used as the shared data;
[0133] In some other embodiments, the multi-party data sharing device based on a generative adversarial network also includes a judgment module 804, which is configured to count the number of the shared data obtained, and stop generating the synthetic data when the number of the shared data reaches a preset threshold.
[0134] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.
[0135] The device of the above embodiment is used to implement the corresponding multi-party data sharing method based on a generative adversarial network in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.
[0136] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the multi-party data sharing method based on a generative adversarial network described in any of the above embodiments is implemented.
[0137] Fig. 9 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.
[0138] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.
[0139] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.
[0140] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.
[0141] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).
[0142] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).
[0143] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.
[0144] The electronic device of the above embodiment is used to implement the corresponding multi-party data sharing method based on a generative adversarial network in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.
[0145] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the multi-party data sharing method based on a generative adversarial network as described in any of the above embodiments.
[0146] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0147] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the multi-party data sharing method based on the generative adversarial network as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0148] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.
[0149] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.
[0150] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.
[0151] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.
Claims
1. A multi-party data sharing method based on generative adversarial network, in, The generative adversarial network comprises a generator network and at least one discriminator network, and is characterized in that it comprises: Sending random vectors to the generator network one by one for processing to obtain synthetic data corresponding to each of the random vectors; Sending the synthetic data corresponding to all the random vectors to each of the discriminator networks for discrimination, obtaining a discrimination result of each of the discriminator networks, and calculating an average acceptance probability of the synthetic data corresponding to each of the random vectors according to the discrimination result of each of the discriminator networks; Compare the preset probability with the average acceptance probability of the synthetic data corresponding to each of the random vectors, and when the average acceptance probability of the synthetic data corresponding to a certain random vector is greater than or equal to the preset probability, use the synthetic data corresponding to the random vector as shared data; The training step of generating an adversarial network includes: Sending synthetic data synthesized by the generator network in the initial generative adversarial network to each discriminator network in each of the initial generative adversarial networks respectively; Respectively extracting local data corresponding to each discriminator network in the initial generative adversarial network and recording them into each discriminator network in the initial generative adversarial network; Discriminate the data generation record and the local data record to obtain a discrimination result output by each discriminator network in the initial generative adversarial network; Calculate the loss function of each discriminator network in the initial generative adversarial network according to the discrimination results; Calculate the gradient information of each discriminator network in the initial generative adversarial network according to the loss function of each discriminator network in the initial generative adversarial network; Using differential privacy noise to perturb the gradient information of each discriminator network in the initial generative adversarial network respectively; Update the parameters of each discriminator network in the initial generative adversarial network according to the perturbed gradient information to obtain the discriminator network in the next round of iteration; The generator network in the initial generative adversarial network is updated according to the discriminator network with updated parameters, and the updated generator network is used as the generator network in the next round of iteration, until the iteration is completed to obtain the trained generative adversarial network.
2. The method according to claim 1, It is characterized in that The multi-party data sharing method based on the generative adversarial network also includes: The amount of the shared data obtained by counting is counted, and when the amount of the shared data reaches a preset threshold, the generation of the synthetic data is stopped.
3. The method according to claim 1, It is characterized in that Iteratively training the generator network in the initial generative adversarial network and the discriminator network in each of the initial generative adversarial networks to obtain the trained generative adversarial network, further comprising: Presetting an iteration round number threshold, when the iteration round number reaches the iteration round number threshold, stopping the iterative training, and outputting the generative adversarial network trained in this round of iterative training as the trained generative adversarial network; or, At the end of each round of iteration, the total privacy budget consumed by the iterative training is calculated. When the total privacy budget is greater than the preset privacy budget threshold, the iterative training is stopped, and the generative adversarial network trained in this round of iteration is output as the trained generative adversarial network.
4. The method according to claim 1, It is characterized in that The differential privacy noise is Gaussian noise sampled from a Gaussian distribution.
5. The method according to claim 1, It is characterized in that The adopting differential privacy noise to perturb the gradient information includes: Pruning the L2-norm of each discriminator network in the initial generative adversarial network respectively; Reducing the scale of the differential privacy noise according to a preset noise attenuation function; The reduced-scale differential privacy noise is used to perturb the pruned gradient information to obtain the perturbed gradient information.
6. The method according to claim 1, wherein the average acceptance probability of the synthetic data corresponding to each of the random vectors is calculated according to the discrimination results of each of the discriminator networks. include: The Sigmoid function is used to calculate the acceptance probability of each discriminator network for the synthetic data corresponding to each random vector; The average acceptance probability of the synthetic data is calculated according to the calculated acceptance probability of each discriminator network for the single synthetic data.
7. A multi-party data sharing device based on generative adversarial network, It is characterized in that include: The data generation module is configured to send random vectors one by one to the generator network in the trained generative adversarial network for processing to obtain corresponding synthetic data; A data discrimination module is configured to send the synthetic data corresponding to each of the random vectors to each discriminator network in the trained generative adversarial network for discrimination, obtain the discrimination result of each discriminator network, and calculate the average acceptance probability of the synthetic data corresponding to each of the random vectors according to the discrimination result of each discriminator network; A data sharing module is configured to compare the average acceptance probability of the synthetic data corresponding to each of the random vectors with a preset probability, and obtain shared data when the average acceptance probability of the synthetic data corresponding to a certain random vector is greater than or equal to the preset probability; A judgment module is configured to stop the trained generative adversarial network from running when the amount of the shared data reaches a preset threshold; The data generation module is further configured to respectively send the synthesized data synthesized by the generator network in the initial generative adversarial network to each discriminator network in each of the initial generative adversarial networks; respectively extract the local data records corresponding to each discriminator network in the initial generative adversarial network to each discriminator network in the initial generative adversarial network; discriminate the data generation records and the local data records to obtain the discrimination results output by each discriminator network in the initial generative adversarial network; According to the discrimination results, the loss function of each discriminator network in the initial generative adversarial network is calculated respectively; according to the loss function of each discriminator network in the initial generative adversarial network, the gradient information of each discriminator network in the initial generative adversarial network is calculated respectively; Using differential privacy noise to perturb the gradient information of each discriminator network in the initial generative adversarial network respectively; The parameters of each discriminator network in the initial generative adversarial network are updated according to the perturbed gradient information to obtain the discriminator network in the next round of iteration; the generator network in the initial generative adversarial network is updated according to the discriminator network with updated parameters to obtain the updated generator network as the generator network in the next round of iteration, until the iteration is completed to obtain the trained generative adversarial network.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium storing computer instructions, It is characterized in that The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Generative adversarial network organization structure based on discriminator sharing and training method thereof
CN110598843A
SAR image data enhancement method based on generative adversarial network information maximization
CN112529806A
Adversarial generative network model training method based on federated learning and related equipment thereof
CN113435583A
Power system probabilistic optimal power flow calculation method and system, equipment and storage medium
CN113708376A
Training a generative adversarial network
EP3745309A1
