Federal learning method and system based on differential privacy

Abstract

The invention discloses a federated learning method and system based on differential privacy, belongs to the technical field of federated learning, and aims to solve the technical problem of how to realize federated learning under the condition of ensuring local data privacy and defending FL inference attack at the same time. Comprising the following steps that a private generation model is constructed based on an auxiliary classifier GAN, and the private generation model comprises a generator and two discriminators; the classification loss of the real sample and the classification sample is introduced to carry out model training on the private generation model, and a trained generator is obtained; each client carries out local training on the local model based on the synthetic sample to generate local model parameters of the local model, and uploads the local model parameters to a server, and the server carries out aggregation calculation based on the local model parameters and weights corresponding to the clients to obtain global model parameters, and sends the global model parameters to the server; and returning the global model parameters to each client for iterative model training.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning, and more particularly to a federated learning method and system based on differential privacy. Background Art

[0002] Internet of Things (IoT) devices have become the main data generation sources. Servers in the cloud and at the edge use the data collected from the devices to train deep learning models, which have higher accuracy than individual devices. To avoid issues such as privacy, security, regulatory, or economic constraints in the cloud and at the edge, and to directly and fully utilize the potential of the devices, the federated learning (FL) framework has been proposed as a mobile edge computing framework for deep learning.

[0003] In the FedAvg algorithm, data providers (i.e., parties) perform local training on a shared model, and then the server aggregates the local models to estimate the global model, rather than directly collecting private data. FedAvg has been shown to be robust for training convolutional neural networks (CNNs) on image classification datasets (such as MNIST and CIFAR-10) and long short-term memory (LSTMs) on language datasets.

[0004] However, FL is vulnerable to inference attacks (such as reconstruction attacks and membership attacks) by malicious servers / parties. The main reason is that the shared model is updated based on the original data, where patterns are encoded into the model parameters and the original data can be reverse-engineered. To mitigate inference attacks in FL, current methods can be broadly classified into homomorphic encryption, secure multi-party computation (SMC), and differential privacy. However, homomorphic encryption and SMC methods are not suitable for large-scale FL scenarios because these methods require a large amount of additional communication and computational costs. In addition to inference attacks, another challenge faced by FL is data heterogeneity. As is well known, the essence of federated learning is to use local stochastic gradient descent (SGD) to optimize local models, and it is necessary to ensure that the stochastic gradient is an unbiased estimate of the integral. Therefore, the training data should satisfy IID sampling. However, it is unrealistic to ensure that the data on each mobile device is always IID. Therefore, devices face data heterogeneity in practical applications, resulting in non-independent and identically distributed data on mobile devices. It is worth noting that the NonIID data setting reduces the effectiveness of the federated learning model.

[0005] In addition to inference attacks, another challenge faced by FL is data heterogeneity. As is well known, the essence of federated learning is to use local stochastic gradient descent (SGD) to optimize local models, and it is necessary to ensure that the stochastic gradient is an unbiased estimate of the integral. Therefore, the training data should satisfy IID sampling. However, it is unrealistic to ensure that the data on each mobile device is always IID. Therefore, devices face data heterogeneity in actual applications, resulting in non-independent and identically distributed data on mobile devices. It should be noted that the NonIID data setting will reduce the effectiveness of the federated learning model.

[0006] How to achieve federated learning while ensuring local data privacy and resisting FL inference attacks at the same time is a technical problem that needs to be solved. Summary of the Invention

[0007] The technical task of the present invention is to address the above deficiencies and provide a federated learning method and system based on differential privacy to solve the technical problem of how to achieve federated learning while ensuring local data privacy and resisting FL inference attacks at the same time.

[0008] In the first aspect, a federated learning method based on differential privacy of the present invention includes the following steps: Model construction: Based on an auxiliary classifier GAN, a private generation model is constructed. The private generation model includes a generator and two discriminators, namely the first discriminator and the second discriminator. The generator is used to take real samples and random noise as inputs, generate and output synthetic samples, and use samples from two sources, real samples and synthetic samples, as input samples. The first discriminator is used to distinguish the source of the input samples and output the sample source, and the second discriminator is used to analyze the sample categories of the input samples and output class labels; Model training: Introduce the classification loss of real samples and classification samples to train the private generation model, and obtain the trained generator; Federated learning: For each client participating in federated learning, each client uses the local dataset as real samples, generates synthetic samples through the trained generator, locally trains the local model based on the synthetic samples to generate local model parameters of the local model, and uploads the local model parameters to the server. For each client, the server calculates the sample loss based on its local dataset and the local model parameters uploaded by the client, calculates the weight of each local model parameter based on the sample loss, performs aggregation calculation based on the local model parameters and weights corresponding to each client, obtains the global model parameters, and returns the global model parameters to each client for iterative model training.

[0009] Preferably, the objective function of the first discriminator is expressed as: , The objective function equation of the second discriminator is expressed as: , The objective function of the generator is expressed as: , where is the generated sample, used to simulate the real sample distribution, and both represent whether there is a prior distribution on the noise vector, represents the synthetic sample, represents the objective function used to measure the effectiveness of distinguishing synthetic samples from real samples, the best optimization result of and , is the weight of the classification loss.

[0010] Preferably, the gradient of the generator is expressed as: , where is the gradient, is the sensitivity of the gradient function, is the Gaussian budget, and the gradient is kept within by gradient clipping; Noise obeying the Gaussian distribution is added to the gradient, and the equation of the Gaussian distribution is: , where is the mean of the random variable. If satisfies privacy and , where represents sensitivity, the following is obtained: , Gaussian noise When added to the gradient, there are constants and , so given the sampling probability and the number of steps , where represents the current sampling number, represents the total sample size. For any , for any , the private generation model is differentially private.

[0011] Preferably, during federated learning, a constrained global objective function is constructed, expressed as: , Among them, represents the model parameters of the global model, represents the th local model's model parameters, represents the weight of the th local model's corresponding model parameters, represents the dataset on the server local; For each client, when the client conducts local model training on the local model, it trains the local model parameters and calculates the local objective function , the th round after the local training is completed, it sends the model parameters of the local model to the server. Among them, represents the th client's local dataset; The server aggregates all local models to calculate the model parameters of the global model and updates the model parameters of the local model using the model parameters of the global model .

[0012] Preferably, the server aggregates all local models to calculate the model parameters of the global model and updates the model parameters of the local model using the model parameters of the global model, including the following steps: For the loss of each participant , introduce a balance coefficient to balance the loss calculation. The balance coefficient is expressed as: , , Among them, represents the weight; Based on and calculate the model parameters of the global model. The calculation formula is as follows: , After the server calculates through the calculation formula, it distributes the model parameters of the global model to each client. The client updates the model parameters of the local model based on the model parameters of the global model, so that .

[0013] Second aspect, a federated learning system based on differential privacy according to the present invention is used to implement federated learning through a federated learning method based on differential privacy as described in any item of the first aspect. The system includes a model construction module, a model training module, and a federated learning module; The model construction module is used to perform the following: construct a private generation model based on an auxiliary classifier GAN. The private generation model includes a generator and two discriminators, namely a first discriminator and a second discriminator. The generator is used to take real samples and random noise as inputs, generate and output synthetic samples. Using samples from two sources, real samples and synthetic samples, as input samples, the first discriminator is used to distinguish the source of the input samples and output the sample source, and the second discriminator is used to analyze the sample category of the input samples and output class labels; The model training module is used to perform the following: introduce the classification loss of real samples and classification samples to train the private generation model and obtain a trained generator; The federated learning module is used to support the client and the server to perform the following: for each client participating in federated learning, each client uses the local dataset as real samples, generates synthetic samples through the trained generator, locally trains the local model based on the synthetic samples to generate local model parameters of the local model, and uploads the local model parameters to the server. For each client, the server calculates the sample loss based on its local dataset and the local model parameters uploaded by the client, calculates the weight of each local model parameter based on the sample loss, performs an aggregation calculation based on the local model parameters and weights corresponding to each client to obtain global model parameters, and returns the global model parameters to each client for iterative model training.

[0014] Preferably, the objective function of the first discriminator is expressed as: , The objective function equation of the second discriminator is expressed as: , The objective function of the generator is expressed as: , where is the generated sample, used to simulate the real sample distribution, and both represent whether there is a prior distribution on the noise vector, represents the synthetic sample, represents the objective function used, used to measure the effectiveness of distinguishing synthetic samples from real samples, The best optimization result of and , is the weight of the classification loss.

[0015] Preferably, the gradient of the generator is expressed as: , where is the gradient, is the sensitivity of the gradient function, is the Gaussian budget, and the gradient is kept within by gradient clipping; Noise obeying a Gaussian distribution is added to the gradient, and the equation of the Gaussian distribution is: , where is the mean of the random variable. If satisfies privacy and , where represents sensitivity, the following is obtained: , When Gaussian noise is added to the gradient, there are constants and , so given the sampling probability and the number of steps , where represents the current sampling number, represents the total sample size, for any , for any , the private generation model is differentially private.

[0016] Preferably, during federated learning, the federated learning module is used to construct a constrained global objective function, expressed as: , where represents the model parameters of the global model, represents the model parameters of the th local model, represents the weight of the model parameters corresponding to the th local model, represents the dataset on the server local; For each client, when the federated learning module supports the client to perform local model training on the local model, the local model parameters are trained and the local objective function is calculated. After the local training is completed in the th round, the model parameters of the local model are sent to the server, where Represents the local dataset on the client side; The federated learning module enables the server to aggregate all local models to calculate the model parameters of the global model and update the model parameters of the local model using the model parameters of the global model .

[0017] Preferably, the federated learning module enables the server to perform the following operations to aggregate all local models to calculate the model parameters of the global model and update the model parameters of the local model using the model parameters of the global model: For the loss of each participant , introduce a balance coefficient to balance the loss calculation. The balance coefficient is expressed as: , , where represents the weight of; Based on and calculate the model parameters of the global model. The calculation formula is as follows: , After the server calculates through the calculation formula, it distributes the model parameters of the global model to each client. The client updates the model parameters of the local model based on the model parameters of the global model, so that .

[0018] The federated learning method and system based on differential privacy of the present invention have the following advantages: 1. Use a private generation model of differential privacy (DP) to generate fake data approximating the original data. During the federated learning process, the client trains the local model based on the fake data, achieving privacy protection and resisting inference attacks in FL; 2. During the federated learning process, when the server aggregates, it calculates the sample loss of the local model parameters uploaded by each client through the local dataset of the server, calculates the weight of the local model parameters uploaded by each client based on the sample loss, and aggregates the local model parameters uploaded by all clients based on the weight of the local model parameters, improving the accuracy and achieving non-independent and identically distributed. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] The present invention will be further described below in conjunction with the drawings.

[0021] Figure 1 It is a flowchart of a federated learning method based on differential privacy for Embodiment 1. Specific Embodiments

[0022] The present invention will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it. However, the embodiments cited are not intended to limit the present invention. Without conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0023] The embodiments of the present invention provide a federated learning method and system based on differential privacy, which are used to solve the technical problem of how to implement federated learning while ensuring local data privacy and resisting FL inference attacks at the same time. Embodiment

[0024] A federated learning method based on differential privacy of the present invention includes three steps: model construction, model training, and federated learning.

[0025] Step S100 Model Construction: Based on an auxiliary classifier GAN, a private generation model is constructed. The private generation model includes a generator and two discriminators, namely the first discriminator and the second discriminator. The generator is used to take real samples and random noise as inputs, generate and output synthetic samples, and use samples from two sources, real samples and synthetic samples, as input samples. The first discriminator is used to distinguish the source of the input samples and output the sample source, and the second discriminator is used to analyze the sample category of the input samples and output the class label.

[0026] In this embodiment, the private generation model is a supervised model of GAN improved on the basis of an auxiliary classifier GAN (ACGAN). ACGAN consists of a generator G and a discriminator D and is used for mutual training. Different from GAN, ACGAN embeds random noise and label information into the generator to control the category of the generated samples. Its discriminator can not only distinguish the source of the samples but also distinguish the sample category. In addition, ACGAN enters adversarial training by introducing the classification loss of real samples and generated samples. The objective function of ACGAN consists of two parts and is defined as: , , Among them, is the true sample distribution, is the prior distribution on the noise vector, denotes the expectation of the true sample distribution , is the expectation of noise sampling, represents the objective function for measuring the effectiveness of differentiating samples from real data, represents the objective function for measuring the effectiveness of sample classes, is the sample generated by the generator G, and is the label of and is the conditional probability distribution of the class labels of real samples and generated samples. For G, the goal is to minimize , while D is trained to maximize during the adversarial training process.

[0027] ACGAN can generate samples belonging to specific classes, but the minimax game between the generator and the discriminator may be unstable, and GAN often suffers from mode collapse, so it cannot be guaranteed that the generated samples are real or representatives of the target class. D tries to correctly classify synthetic samples and real samples and is also trained to predict the class label of the input sample. Some hyperparameters between the two functions are shared, which will lead to the above problems.

[0028] The improved private generation model DoubleGAN in this embodiment consists of a generator G, a first discriminator D1, and a second discriminator D2. The role of D1 is to classify synthetic samples and real samples, and the role of D2 is to predict the class label of the input sample.

[0029] Although the private generation model prevents inference attacks by generating fake samples, if the generated samples are close to real samples, privacy leakage will inevitably occur. To further prevent inference attacks, this embodiment adds differential privacy to the generator G of DoubleGAN, rather than in the FL aggregation stage. DoubleGAN generates fake samples and differential privacy and inputs the fake samples into the federated learning process. Specifically, this embodiment adds random noise to the discriminator G, and according to the distribution of interfering with the original data, the gradient of the generator is expressed as:

[0030] Among them, is the gradient, is the sensitivity of the gradient function, is the Gaussian budget, and the gradient is kept within by gradient clipping; Noise that follows a Gaussian distribution is added to the gradient. The equation of the Gaussian distribution is: , where is the mean of the random variable. If satisfies privacy and , where represents sensitivity, and we get the following: , Considering the specific noise distribution at each iteration, when adding Gaussian noise to the gradient, there are constants and . Therefore, given the sampling probability and the number of steps , where represents the current sampling number, represents the total sample size. For any , for any , the private generation model is differentially private.

[0031] Step S200 Model training: The private generation model is trained using the classification loss of real samples and classification samples to obtain the trained generator.

[0032] In this embodiment, the objective function of the first discriminator is expressed as: , where is the generated sample, used to simulate the real sample distribution, and both represent whether there is a prior distribution on the noise vector, represents the synthetic sample, represents the objective function used to measure the effectiveness of distinguishing synthetic samples from real samples, The best optimization result of and ; The objective function equation of the second discriminator is expressed as: , D1 and D2 are two independent models, and both can freely construct models that are more suitable for their respective functions. The hyperparameters between D1 and D2 are not shared and do not interfere with each other.

[0033] The objective function of the generator is expressed as: , is the weight of the classification loss.

[0034] The purpose of G is to generate fake samples that can deceive D1, and the labels of the fake samples should be consistent with the results classified by D2. In this embodiment, it is necessary to maximize the objective function of D1, the objective function of D2, and minimize the objective function of G.

[0035] Step S300 Federated learning: For each client participating in federated learning, each client uses the local dataset as the real sample, generates synthetic samples through the trained generator, locally trains the local model based on the synthetic samples to generate the local model parameters of the local model, and uploads the local model parameters to the server. For each client, the server calculates the sample loss based on its local dataset and the local model parameters uploaded by the client, calculates the weight of each local model parameter based on the sample loss, performs aggregation calculation based on the local model parameters and weights corresponding to each client, obtains the global model parameters, and returns the global model parameters to each client for iterative model training.

[0036] In this embodiment, during federated learning, a constrained global objective function is constructed, which is expressed as: , where, represents the model parameters of the global model, represents the th local model's model parameters, represents the th weight of the model parameters corresponding to the local model, represents the server's local dataset; For each client, when the client locally trains the local model, it trains the local model parameters and calculates the local objective function , the th round when the local training is completed, it sends the model parameters of the local model to the server, where, represents the th client's local dataset; The server aggregates all local models to calculate the model parameters of the global model and updates the model parameters of the local model using the model parameters of the global model 。

[0037] Among them, the server aggregates all local models to calculate the model parameters of the global model and updates the model parameters of the local models using the model parameters of the global model, including the following steps: (1) For the loss of each participant , introduce a balance coefficient to balance the loss calculation, and the balance coefficient is expressed as: , , where represents the weight of; (2) Based on and calculate the model parameters of the global model, and the calculation formula is as follows: , (3) After the server calculates through the calculation formula, it distributes the model parameters of the global model to each client, and the client updates the model parameters of the local model based on the model parameters of the global model, so that .

[0038] The method of this embodiment uses DoubleGAN of differential privacy (DP) to generate fake data approximating the original data, and puts the fake data into the local model training process of federated learning, realizing privacy protection and resisting inference attacks in FL; using a double discriminator in DoubleGAN to improve ACGAN, improving the performance of the generator and the quality of the generated samples; in the process of federated learning, the aggregation weight of each round can be dynamically adjusted, and the global model parameters are calculated by calculating the loss of the local model of the test set in the server, and this strategy effectively improves non-independent and identically distributed.

[0039] Embodiment 2: A federated learning system based on differential privacy according to the present invention includes a model construction module, a model training module, and a federated learning module.

[0040] The model construction module is used to perform the following: construct a private generation model based on an auxiliary classifier GAN, the private generation model includes a generator and two discriminators, the two discriminators are a first discriminator and a second discriminator respectively, the generator is used to take real samples and random noise as inputs, generate and output synthetic samples, take samples from two sources of real samples and synthetic samples as input samples, the first discriminator is used to distinguish the source of the input samples and output the sample source, and the second discriminator is used to analyze the sample category of the input samples and output the class label.

[0041] Among them, the private generation model is a supervised model of GAN improved on the basis of the Auxiliary Classifier GAN (ACGAN). The ACGAN consists of a generator G and a discriminator D and is used for mutual training. Different from GAN, the ACGAN embeds random noise and label information into the generator to control the categories of the generated samples. Its discriminator can not only distinguish the source of the samples but also distinguish the categories of the samples. In addition, the ACGAN enters the adversarial training by introducing the classification losses of real samples and generated samples. The objective function of the ACGAN consists of two parts and is defined as: , , wherein, is the real sample distribution, is the prior distribution on the noise vector, denotes the expectation of the real sample distribution , is the expectation of the noise sampling, denotes the objective function used to measure the effectiveness of distinguishing samples from real data, denotes the objective function used to measure the effectiveness of the sample categories, is the sample generated by the generator G, and is 's label, and are the conditional probability distributions of the class labels of real samples and generated samples. For G, the goal is to minimize , while D is trained to maximize during the adversarial training process.

[0042] The ACGAN can generate samples belonging to specific categories, but the minimax game between the generator and the discriminator may be unstable, and the GAN often suffers from mode collapse, so it cannot be guaranteed that the generated samples are real or representatives of the target class. D tries to correctly classify synthetic samples and real samples and is also trained to predict the class labels of input samples. Some hyperparameters between the two functions are shared, which will lead to the above problems.

[0043] The improved private generation model DoubleGAN in this embodiment consists of a generator G, a first discriminator D1, and a second discriminator D2. The role of D1 is to classify synthetic samples and real samples, and the role of D2 is to predict the class labels of input samples.

[0044] Although private generation models prevent inference attacks by generating fake samples, privacy leakage will inevitably occur if the generated samples are close to real samples. To further prevent inference attacks, in this embodiment, differential privacy is added to the generator G of DoubleGAN, rather than in the FL aggregation phase. DoubleGAN generates fake samples and differential privacy, and inputs the fake samples into the federated learning process. Specifically, in this embodiment, random noise is added to the discriminator G, and according to the distribution of the interfering original data, the gradient of the generator is obtained as follows:

[0045] where, is the gradient, is the sensitivity of the gradient function, is the Gaussian budget, and the gradient is kept within by gradient clipping; Noise obeying the Gaussian distribution is added to the gradient, and the equation of the Gaussian distribution is: , where is the mean of the random variable, if satisfies privacy and where represents sensitivity, the following is obtained: , Considering the specific noise distribution at each iteration, when adding Gaussian noise to the gradient, there are constants and , so given the sampling probability and the number of steps where represents the current sampling number, represents the total sample size, for any , for any , the private generation model is differentially private.

[0046] The model training module is used to perform the following: introducing the classification loss of real samples and classification samples to train the private generation model to obtain the trained generator.

[0047] In this embodiment, the objective function of the first discriminator is expressed as: , where, is the generated sample, which is used to simulate the real sample distribution, and both indicate whether there is a prior distribution on the noise vector, represents the synthetic samples, represents the objective function used to measure the effectiveness of distinguishing synthetic samples from real samples, the best optimization result and ; The objective function equation of the second discriminator is expressed as: , D1 and D2 are two independent models, and both can freely construct models more suitable for their respective functions. The hyperparameters between D1 and D2 are not shared and do not interfere with each other.

[0048] The objective function of the generator is expressed as: , is the weight of the classification loss.

[0049] The purpose of G is to generate fake samples that can deceive D1, and the labels of the fake samples should be consistent with the results classified by D2. In this embodiment, it is necessary to maximize the objective function of D1, the objective function of D2, and minimize the objective function of G.

[0050] The federated learning module is used to support the client and the server to perform the following: for each client participating in federated learning, each client uses the local dataset as the real sample, generates synthetic samples through the trained generator, locally trains the local model based on the synthetic samples to generate the local model parameters of the local model, and uploads the local model parameters to the server. For each client, the server calculates the sample loss based on its local dataset and the local model parameters uploaded by the client, calculates the weight of each local model parameter based on the sample loss, performs aggregation calculation based on the local model parameters and weights corresponding to each client, obtains the global model parameters, and returns the global model parameters to each client for iterative model training.

[0051] In this embodiment, during federated learning, a constrained global objective function is constructed, which is expressed as: , where, represents the model parameters of the global model, represents the th local model's model parameters, represents the th local model's corresponding model parameter weight, represents the server's local dataset; For each client, when the client performs local model training on the local model, it trains the local model parameters and calculates the local objective function , at the round, after the local training is completed, the model parameters of the local model are sent to the server, where represents the local dataset of the th client.

[0052] The federated learning module supports the server to aggregate all local models to calculate the model parameters of the global model and updates the model parameters of the local model using the model parameters of the global model .

[0053] Among them, the federated learning module supports the server to perform the following to aggregate all local models to calculate the model parameters of the global model and update the model parameters of the local model using the model parameters of the global model: (1) For the loss of each participant , introduce a balance coefficient for balanced loss calculation, and the balance coefficient is expressed as: , , where represents the weight of ; (2) Based on and calculate the model parameters of the global model, and the calculation formula is as follows: , (3) After the server calculates through the calculation formula, it distributes the model parameters of the global model to each client, and the client updates the model parameters of the local model based on the model parameters of the global model, so that .

[0054] The system of this embodiment can implement efficient and secure federated learning by executing the method disclosed in Embodiment 1.

[0055] The above has detailedly demonstrated and described the present invention through the drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that more embodiments of the present invention can be obtained by combining the means in the above different embodiments, and these embodiments are also within the protection scope of the present invention.

Claims

1. A federated learning method based on differential privacy, characterized in that: The steps include: Model construction: A private generative model is constructed based on the auxiliary classifier GAN. The private generative model includes a generator and two discriminators, the two discriminators are the first discriminator and the second discriminator. The generator is used to take real samples and random noise as input, generate and output synthetic samples, and take samples from two sources, real samples and synthetic samples, as input samples. The first discriminator is used to distinguish the sample sources of the input samples and output the sample sources. The second discriminator is used to perform sample category analysis on the input samples and output class labels. Model training: introduce the classification loss of real samples and classified samples to train the private generative model and obtain the trained generator; Federated learning: For each client participating in federated learning, each client uses the local data set as the real sample, generates synthetic samples through the trained generator, performs local training on the local model based on the synthetic samples to generate local model parameters of the local model, and uploads the local model parameters to the server. For each client, the server performs sample loss calculation based on its local data set and the local model parameters uploaded by the client, calculates the weight of each local model parameter based on the sample loss, performs aggregation calculation based on the local model parameters and weights corresponding to each client, obtains the global model parameters, and returns the global model parameters to each client for iterative model training.

2. The differential privacy-based federated learning method according to claim 1, characterized in that: The objective function of the first discriminator is expressed as: ， The objective function equation of the second discriminator is expressed as: ， The objective function of the generator is expressed as: ， in, is a generated sample used to simulate the real sample distribution, and Both indicate whether there is a prior distribution on the noise vector. represents a synthetic sample, represents the objective function used to measure the effectiveness of distinguishing synthetic samples from real samples, The best optimization results and , is the weight of the classification loss.

3. The federated learning method based on differential privacy according to claim 1, characterized in that: The gradient of the generator is expressed as: ， in, is the gradient, is the sensitivity of the gradient function, For the Gaussian budget, the gradient is kept at within the scope; Noise that follows a Gaussian distribution is added to the gradient. The equation for the Gaussian distribution is: ， in is the mean of a random variable, if satisfy privacy and ,in express Sensitivity is obtained as follows: ， Gaussian noise When added to the gradient, there is a constant and , so given the sampling probability and number of steps ,in Indicates the current sampling number. represents the total sample size. For any , for any , private generative models are Differential privacy.

4. The federated learning method based on differential privacy according to claim 1, characterized in that: In federated learning, a constrained global objective function is constructed, which is expressed as: ， in, represents the model parameters of the global model, Indicates The model parameters of the local model, Indicates The weights of the model parameters corresponding to the local models, Represents the local data set of the server; For each client, when the client performs local model training on the local model, the local model parameters are trained. And calculate the local objective function , No. After the local training is completed, the model parameters of the local model are Sent to the server, where Indicates A local dataset of the client; The server aggregates all local models to calculate the model parameters of the global model And use the model parameters of the global model to update the model parameters of the local model .

5. The federated learning method based on differential privacy according to claim 4, characterized in that: The server aggregates all local models to calculate the model parameters of the global model And use the model parameters of the global model to update the model parameters of the local model, including the following steps: The loss to each participant , introduce the balance coefficient Balance loss calculation, the balance coefficient is expressed as: ， ， in, express The weight of based on and Calculate the model parameters of the global model, the calculation formula is as follows: ， The server calculates the formula After that, the model parameters of the global model are sent to each client, and the client updates the model parameters of the local model based on the model parameters of the global model, so that .

6. A federated learning system based on differential privacy, characterized in that: Used to implement federated learning through a federated learning method based on differential privacy as described in any one of claims 1 to 5, the system comprising a model building module, a model training module and a federated learning module; The model building module is used to perform the following: construct a private generation model based on the auxiliary classifier GAN, the private generation model includes a generator and two discriminators, the two discriminators are respectively a first discriminator and a second discriminator, the generator is used to take real samples and random noise as input, generate and output synthetic samples, and take samples from two sources, real samples and synthetic samples, as input samples, the first discriminator is used to distinguish the sample sources of the input samples and output the sample sources, and the second discriminator is used to perform sample category analysis on the input samples and output class labels; The model training module is used to perform the following: introduce the classification loss of real samples and classified samples to train the private generation model to obtain a trained generator; The federated learning module is used to support the client and server to perform the following: for each client participating in federated learning, each client uses the local data set as the real sample, generates a synthetic sample through the trained generator, performs local training on the local model based on the synthetic sample to generate local model parameters of the local model, and uploads the local model parameters to the server. For each client, the server performs sample loss calculation based on its local data set and the local model parameters uploaded by the client, calculates the weight of each local model parameter based on the sample loss, performs aggregation calculation based on the local model parameters and weights corresponding to each client, obtains the global model parameters, and returns the global model parameters to each client for iterative model training.

7. The differential privacy-based federated learning system according to claim 6, characterized in that: The objective function of the first discriminator is expressed as: ， The objective function equation of the second discriminator is expressed as: ， The objective function of the generator is expressed as: ， in, is a generated sample used to simulate the real sample distribution, and Both indicate whether there is a prior distribution on the noise vector. represents a synthetic sample, represents the objective function used to measure the effectiveness of distinguishing synthetic samples from real samples, The best optimization results and , is the weight of the classification loss.

8. The differential privacy-based federated learning system according to claim 6, characterized in that: The gradient of the generator is expressed as: ， in, is the gradient, is the sensitivity of the gradient function, For the Gaussian budget, the gradient is kept at within the scope; Noise that follows a Gaussian distribution is added to the gradient. The equation for the Gaussian distribution is: ， in is the mean of a random variable, if satisfy privacy and ,in express Sensitivity is obtained as follows: ， Gaussian noise When added to the gradient, there is a constant and , so given the sampling probability and number of steps ,in Indicates the current sampling number. represents the total sample size. For any , for any , private generative models are Differential privacy.

9. The differential privacy-based federated learning system according to claim 6, characterized in that: In federated learning, the federated learning module is used to construct a constrained global objective function, expressed as: ， in, represents the model parameters of the global model, Indicates The model parameters of the local model, Indicates The weights of the model parameters corresponding to the local models, Represents the local data set of the server; For each client, the federated learning module supports the client to train local model parameters when training the local model. And calculate the local objective function , No. After the local training is completed, the model parameters of the local model are Sent to the server, where Indicates A local dataset of the client; The federated learning module supports the server to aggregate all local models to calculate the model parameters of the global model. And use the model parameters of the global model to update the model parameters of the local model .

10. The differential privacy-based federated learning system according to claim 6, characterized in that: The federated learning module supports the server to perform the following operations to aggregate all local models and calculate the global model parameters: And use the model parameters of the global model to update the model parameters of the local model: The loss to each participant , introduce the balance coefficient Balance loss calculation, the balance coefficient is expressed as: ， ， in, express The weight of based on and Calculate the model parameters of the global model, the calculation formula is as follows: ， The server calculates the formula After that, the model parameters of the global model are sent to each client, and the client updates the model parameters of the local model based on the model parameters of the global model, so that .

Images