Privacy enhanced federal learning method

Abstract

The invention relates to a privacy enhancement federated learning method based on a conditional generative adversarial network, which is used for improving the performance of federated learning and enhancing the privacy protection capability. By designing an auxiliary data generation mechanism based on the CGAN, the training data set of the client is expanded, and the problem of data overfitting is solved. Meanwhile, a double-model coexistence concept of a private model and a public model of the client is provided, and an improved deep mutual learning model interaction training method is designed to ensure that sensitive information is only stored locally. In addition, through a high-performance model aggregation method based on prediction distribution, the performance of the global model is effectively improved. Experimental results show that the method can effectively resist various inference attacks while keeping the performance of the federal learning model, and the privacy protection capability of user data is remarkably enhanced.

Description

Technical Field

[0001] The present invention relates to a privacy-enhanced federated learning method, and belongs to the technical field of federated learning and privacy protection. Background Art

[0002] In recent years, with the rapid development of artificial intelligence and machine learning, especially the widespread application of deep learning technology, data privacy and security issues have gradually attracted widespread attention. In traditional centralized machine learning, all data needs to be uploaded to a central server for training. This approach faces greater privacy risks when processing sensitive information, such as in the fields of medical, financial, and personal data processing. In order to solve this problem, Federated Learning (FL) has emerged as a distributed machine learning method. Since the data in the federated learning system is usually unevenly distributed (i.e., Non-IID), the difference in training data between clients seriously affects the model performance. In addition, with the continuous development of malicious attack methods, the gradient or parameters of the model may also become the target of the attacker, resulting in potential privacy leakage, especially in reconstruction attacks (such as gradient inversion attacks), the attacker may analyze the model gradient to infer the training data, thereby exposing user privacy. Therefore, how to improve the generalization ability and training performance of the model while ensuring privacy security has become a key challenge in current federated learning research.

[0003] Currently, federated learning faces three main challenges: the first is the problem of insufficient data and uneven data distribution. In a federated learning system, the local data of the client is often limited by the device storage and computing power, and these data are usually not independent and identically distributed. This uneven distribution will make it difficult for the model to converge during the joint training process, and even cause performance degradation. To overcome this problem, traditional federated learning methods usually rely on data augmentation or incremental training, but these methods often cannot effectively solve the problems of scarce data or large distribution differences.

[0004] Secondly, privacy protection is another major challenge in federated learning. Although federated learning avoids direct data exchange, the model parameters or gradients uploaded by the client may still expose the user's private information. By analyzing the gradients or model parameters, attackers can infer the user's sensitive data. This "gradient leakage" problem has attracted widespread attention from academia and industry in recent years.

[0005] Finally, performance improvement in the model aggregation process is also an urgent problem to be solved. In traditional federated learning, the model parameters of all clients are usually aggregated using a simple average strategy. However, this method fails to take into account the differences in client data quality and training effects, making the performance of the global model vulnerable to low-quality clients.

[0006] Although existing methods have solved the privacy protection and performance issues in federated learning to a certain extent, they still have some shortcomings and cannot fully meet the needs of practical applications. Summary of the invention

[0007] The purpose of the present invention is to provide a privacy-enhanced federated learning method, which effectively solves the problems of data privacy protection, performance improvement and uneven data distribution based on the combination of conditional generative adversarial networks, dual-model coexistence mechanism and improved weighted model aggregation method, and shows a more significant performance advantage.

[0008] In order to achieve the above objectives / solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0009] In one aspect, the present invention provides a privacy-enhanced federated learning method, comprising:

[0010] Step 1: Each client receives the global model parameters from the server. At the same time, each client generates auxiliary training data through a conditional generative adversarial network.

[0011] Step 2: Each client initializes local private model parameters and public model parameters through the received global model parameters, trains private model parameters through local original data and public model parameters through auxiliary training data, and uses the deep model mutual training method to mutually train public model parameters and private model parameters;

[0012] Step 3: After the training is completed, the public model parameters of each client after training are uploaded to the server. The server infers the public model parameters of each client after training based on the public data set and statistically predicts the results;

[0013] Step 4: The server calculates the weight of each client based on the prediction results;

[0014] Step 5: The server performs weighted aggregation on the public model parameters uploaded by each client according to the weight, updates the global model parameters, and broadcasts the updated global model parameters to each client;

[0015] Step 6: Repeat steps 2 to 5 until the global model parameters meet the requirements;

[0016] Wherein: the generating auxiliary training data by conditional generative adversarial network specifically includes: introducing feature distance loss into the CGAN objective function to expand the auxiliary training data.

[0017] Furthermore, the feature distance loss is introduced into the CGAN objective function to expand the auxiliary training data, and its expression is:

[0018]

[0019] Among them, x is the local original data, and y is the conditional label; is the adversarial objective function between the discriminator D and the generator G in the generative adversarial network, For the distribution of real data p data(x) The expected value of x is obtained by sampling from is the noise distribution p from the prior z(z) The expected value of z is obtained by sampling, P data(x) is the distribution of local original data; logD(x|y) is the logarithmic probability that the discriminator D marks the local original data as true; z is the logarithmic probability of the discriminator D marking the local original data as true from the prior noise distribution P z(z) noise sampled in; G(z|y) is the auxiliary training data generated by the generator G; log(1-D(G(z|y))) is the logarithmic probability that the discriminator G marks the generated auxiliary training data as false; δ is the coefficient controlling the proportion of feature distance loss; g(·) is the function mapping local raw data to features, ∥g(x)-g(G()|y))∥ 2 is the feature distance loss.

[0020] Based on the auxiliary data generation mechanism of conditional generative adversarial networks, this method can generate training data that is similar to the original data but does not contain sensitive information, thereby increasing the amount and diversity of training data on the client. The introduction of auxiliary training data alleviates the problems caused by data scarcity and uneven data distribution, thereby improving the generalization ability of the global model. This mechanism effectively improves model performance when the amount of data is limited or the data differences are large.

[0021] Furthermore, before the client trains the public model parameters using the auxiliary training data, the auxiliary training data may be subjected to data enhancement processing, which is expressed as follows:

[0022] x′=M⊙x a +(1)⊙x;

[0023] y′=ρy a +(1-ρ)y;

[0024] Among them, x′ is the auxiliary training data after data enhancement processing; M is a 0-1 matrix with the same dimension as the local original data; x a is the output data of generator G; y′, y a are x′, x a The label of ; ρ is a random sample from the 0-1 distribution.

[0025] Furthermore, each client trains private model parameters through local original data and public model parameters through auxiliary training data, and uses a deep model mutual training method to mutually train public model parameters and private model parameters, and the expression is:

[0026] L Θ1 =L C1 +W(Θ1,Θ2;

[0027] L Θ2 =L C2 +W(Θ2,Θ1;

[0028] Where: L Θ1 The objective function for training private model parameters for the client; L Θ2 The objective function for training public model parameters for clients; C1 is the loss function for training private model parameters using local raw data, C2 The loss function of the client training the public model parameters; W(Θ1,Θ2) is the objective function of mutual training of public model parameters and private model parameters using the deep model mutual training method;

[0029]

[0030] Where: K is the total number of local original data samples, y k,c is the label of category c of the kth sample in the local original data, p k,c The probability associated with category c of the kth sample of the original data;

[0031]

[0032] Where: H is the total number of auxiliary training data samples, y′ h,c is the label of category c of the hth sample in the auxiliary training data, p′ h,c is the probability associated with category c of the h-th sample in the auxiliary training data;

[0033]

[0034] Where: Θ1 is the private model parameter distribution; Θ2 is the public model parameter distribution; inf is the minimum value of the predicted distribution of private model parameters and public model parameters, Π(Θ1.Θ2) is the set of all possible joint probability distributions formed by combining Θ1 and Θ2, p1 is the predicted distribution of private model parameters, p2 is the predicted distribution of public model parameters, and ∥p1-p2∥ is a distance metric defined in the space of predicted distributions of private model parameters and public model parameters.

[0035] Through the dual-model coexistence mechanism, the parameters of the private model are always saved locally, and only the parameters of the public model are uploaded, which effectively reduces the risk of privacy leakage during data transmission, thereby avoiding the leakage of sensitive data. By generating auxiliary training data to replace real data for training, the privacy protection effect is further enhanced.

[0036] Furthermore, the server infers the public model parameters trained by each client according to the public data set and calculates the prediction results, which specifically include:

[0037] The server obtains the public model parameters W from client i i ,

[0038] Let r i =0, indicating that the initial "correct prediction number" of the public model parameters of the i-th client is 0.

[0039] For public datasets on the server The kth sample x in (k) , do the following:

[0040] Through the public model parameter W i To build the model function Model i (·,

[0041] The kth sample x (k) Enter the model, the expression is:

[0042]

[0043] in: is the predicted probability vector output by the model, Model i (x (k) ; i ) is to calculate the kth sample x in the public dataset (k) Input model W i The obtained label prediction distribution vector,

[0044] Take the category corresponding to the maximum probability as the prediction result, the expression is:

[0045]

[0046] Among them, c traverses all possible categories; if This means that the kth sample prediction is correct, so let: i =r i +1, after completing the reasoning of all M samples, the final r i That is model W i The number of correct examples predicted on a public dataset.

[0047] Furthermore, the server calculates the weight of each client according to the prediction result, specifically including:

[0048]

[0049] Where: i is the weight of client i, r i is the number of correct predictions of client i on the public dataset, and N is the total number of clients;

[0050] Furthermore, the server performs weighted aggregation on the public model parameters uploaded by each client according to the weight, and updates the global model parameters, specifically including:

[0051]

[0052] Where: W′ global is the updated global model parameter, W global is the global model parameter before updating, W i ′ is the public model parameter uploaded by client i.

[0053] The weighted aggregation method based on predictive distribution dynamically adjusts the weight of the client model in the global model according to the training effect of the client model. Through this weighted aggregation strategy, the method avoids the performance instability problem brought by the traditional simple average aggregation method. Weighted aggregation effectively improves the accuracy of the global model and reduces the negative impact of clients with low data quality on the global model.

[0054] Furthermore, each client initializes the local private model parameters and public model parameters through the received global model parameters, specifically including: when federated learning starts or the server completes a round of global model parameter aggregation, the server broadcasts the new global model parameters W to all clients. global After receiving it, the client caches it locally and prepares for initialization. The client's private model parameter is W privAte , let the client's public model parameter be W public ; When federated learning starts, the initial model parameters W sent by the server are used global Directly used as the initial model of private model parameters and public model parameters; in each subsequent communication round, the server sends the updated global model parameters W′ global When , only the updated global model parameters W′ are used global Replace the original as the public model parameter W public as a new public model parameter; instead of the private model parameter W private Initialize.

[0055] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention proposes a high-performance privacy-enhanced federated learning method based on conditional generative adversarial networks, which combines conditional generative adversarial networks, dual-model coexistence mechanism and improved weighted model aggregation method. First, through the auxiliary data generation mechanism based on conditional generative adversarial networks, diversified auxiliary training data is generated. Secondly, the dual-model coexistence mechanism is adopted to separate the training process of private data and public data, avoiding the risk of sensitive information leakage while ensuring the training quality of the public model. Finally, a weighted aggregation method based on predicted distribution is adopted to dynamically adjust the model weight according to the performance of the client, thereby improving the generalization ability and training efficiency of the global model. The method of the present invention effectively solves the problems of data privacy protection, performance improvement and uneven data distribution, and shows a more significant performance advantage. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is the overall framework diagram of the privacy-enhanced federated learning method of the present invention;

[0057] Figure 2 It is a comparison of the attack effects of the design of the present invention and other comparison methods;

[0058] Figure 3 It is the model convergence curve of the design of the present invention and other comparison methods. DETAILED DESCRIPTION

[0059] It should be noted that:

[0060] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0061] The term "and / or" is only a way to describe the association relationship of associated objects. There can be three kinds of relationships. For example, A and / or B can be: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " generally means that the associated objects before and after are in an "or" relationship.

[0062] Example 1

[0063] like Figure 1 An embodiment shown in FIG. 1 is a method for privacy-enhanced federated learning, which specifically includes the following steps:

[0064] Step 1: Each client receives the global model parameters from the server. At the same time, each client generates auxiliary training data through a conditional generative adversarial network, which specifically includes the following steps:

[0065] Step 1.1: At the beginning of federated learning training, the client first receives the global model parameters W from the server global ;These parameters are generated by the server through weighted aggregation of the public model parameters uploaded by all clients in the previous round;

[0066] The client uses the received W global Initialize private model parameters W private and the public model parameter W public ,This step ensures that the starting point of all client models is consistent and provides a stable parameter basis for subsequent local training;

[0067] Step 1.2: The client generates auxiliary training data through conditional generative adversarial networks Generator G receives noise and conditional label y, generating data In order to prevent overfitting of generated data, feature distance loss is introduced into the objective function of the conditional generative adversarial network, namely:

[0068]

[0069] Among them, x is the local original data, and y is the conditional label; is the adversarial objective function between the discriminator D and the generator G in the generative adversarial network, For the distribution of real data p data(x) The expected value of x is obtained by sampling from For the prior noise distribution p z(z) The expected value of z is obtained by sampling, P data(x) is the distribution of local original data; logD(x|y) is the logarithmic probability that the discriminator D marks the local original data as true; z is the logarithmic probability of the discriminator D marking the local original data as true from the prior noise distribution P z(z) noise sampled in; G(z|y) is the auxiliary training data generated by the generator G; log(1-D(G(z|y))) is the logarithmic probability that the discriminator G marks the generated auxiliary training data as false; δ is the coefficient controlling the proportion of feature distance loss; g(·) is the function mapping local raw data to features, ∥g(x)-g(G(z|y))∥ 2 It is the feature distance loss.

[0070] Step 2: Perform data enhancement on the auxiliary training data. The client randomly replaces the generated data with the original data at a ratio of 20% to form an enhanced auxiliary data set. To improve the generalization ability of the model, specifically:

[0071] x′=M⊙x a +(1)⊙x;

[0072] y ′ =ρy a +(1-ρ)y;

[0073] Among them, x ′ is the auxiliary training data after data enhancement processing; M is a 0-1 matrix with the same dimension as the local original data; x a is the output data of generator G; y′, y a are x′, x a The label of; ρ is a random sample distributed between 0 and 1;

[0074] Step 3: The client trains the private model parameters using the local original data and the public model parameters using the auxiliary training data after data enhancement, and uses the deep model mutual training method to mutually train the public model parameters and the private model parameters, including:

[0075] L Θ1 =L C1 +W(Θ1,Θ2;

[0076] L Θ2 =L C2 +W(Θ2,Θ1;

[0077] Where: L Θ1 The objective function for training private model parameters for the client; L Θ2 The objective function for training public model parameters for clients; C1 is the cross entropy loss function for training private model parameters using local raw data, C2 The cross entropy loss function of the client training the public model parameters; W(Θ1,Θ2) is the objective function of mutual training of public model parameters and private model parameters using the deep model mutual training method, that is, the bulldozer distance between the public model parameters and the private model parameters;

[0078]

[0079] Where: K is the total number of local original data samples, y k,c is the label of category c of the kth sample in the local original data, p k,c The probability associated with category c of the kth sample of the original data;

[0080]

[0081] Where: H is the total number of auxiliary training data samples, y′ h,c is the label of category c of the hth sample in the auxiliary training data, p′ h,c is the probability associated with category c of the h-th sample in the auxiliary training data;

[0082]

[0083] Where: Θ1 is the private model parameter distribution; Θ2 is the public model parameter distribution; inf is the minimum value of the predicted distribution of the private model parameters and the public model parameters, Π(Θ1.Θ2) is the set of all possible joint probability distributions formed by combining Θ1 and Θ2, p1 is the predicted distribution of the private model parameters, p2 is the predicted distribution of the public model parameters, ∥p1-p2∥ is the distance metric defined in the space of the predicted distributions of the private model parameters and the public model parameters;

[0084] Step 4: After the training is completed, the public model parameters of each client after training are uploaded to the server. The server infers the public model parameters of each client after training based on the public data set and calculates the prediction results, including:

[0085] The updated public model parameters W public Upload to the server, the server collects the parameters of all clients And use the public data set to infer the models of each client and statistically predict the number of correct samplesξ i , ensuring that the server only receives the desensitized model parameters, while the local original data and private models are always kept locally on the client. Specifically:

[0086] The server obtains the public model parameters W from client i i ,

[0087] Let r i =0, indicating that the initial "correct prediction number" of the public model parameters of the i-th client is 0,

[0088] For public datasets on the server The kth sample x in (k) , do the following:

[0089] Through the public model parameter W i To build the model function Model i (·,

[0090] The kth sample x (k) Enter the model, the expression is:

[0091]

[0092] in: is the predicted probability vector output by the model, Model i (x (k) ; i ) is to calculate the kth sample x in the public dataset (k) Input model Wi The obtained label prediction distribution vector,

[0093] Take the category corresponding to the maximum probability as the prediction result, the expression is:

[0094]

[0095] Among them, c traverses all possible categories; if This means that the kth sample prediction is correct, so let: i =r i +1, after completing the reasoning of all M samples, the final r i That is model W i The number of correct examples predicted on a public dataset.

[0096] Step 5: The server calculates the weight of each client based on the prediction results, including:

[0097]

[0098] Where: i is the weight of client i, r i is the number of correct predictions of client i on the public dataset, and N is the total number of clients;

[0099] By dynamically assigning weights, the server can more accurately identify high-performance models and give them a higher weight during aggregation, thereby improving the overall performance of the global model;

[0100] Step 6: The server performs weighted aggregation on the public model parameters uploaded by each client according to the weights, and updates the global model parameters, which specifically includes:

[0101]

[0102] Where: W′ glonal is the updated global model parameter, W global is the global model parameter before updating, W i ′ is the public model parameter uploaded by client i; the aggregation method of this embodiment not only considers the contribution of each client, but also enhances the stability and generalization ability of the global model by minimizing the deviation and variance of the model parameters. The updated W′ global It will be distributed to all clients to start the next round of federated training.

[0103] like Figures 2-3As shown in Table 1, the high-performance privacy-enhanced federated learning method based on conditional generative adversarial network designed by the present invention is applied in practice. Compared with the three comparison methods of Fed-CDP, FL-AC and Soteria, the design method of the present invention (Ours) and the method of the present invention combined with KD (Ours with KD) are

[0104] Table 1. Reasoning attack success rate under different protection algorithms (unit: %)

[0105]

[0106] The attack success rates of the algorithm of the present invention and its version combined with KD on all three data sets are less than 5%, which highlights that the algorithm of the present invention has a strong privacy protection effect. This is because the present invention introduces feature loss in CGAN to generate auxiliary training data with strong robustness and reliable privacy, and combines IDML model mutual learning to achieve global data desensitization, which can significantly reduce the success rate of malicious client speculation, thereby providing strong security protection for user privacy. In comparison, the reasoning attack success rates of other protection methods such as Fed-CDP, FL-AC and Soteria are generally higher than 74%, among which Soteria has a success rate of more than 90% in all cases, exposing the shortcomings of such methods in resisting member reasoning attacks.

[0107] As shown in Table 2 below, when the communication rounds reach 100, the accuracy performance of the models under different protection algorithms on the MNIST, CIFAR-10, and Fashion-MNIST datasets.

[0108] Table 2 Model accuracy when communication rounds reach 100 (unit: %)

[0109]

[0110] By analyzing the above data, we can deeply understand the performance differences of various protection algorithms. First, this method shows a high accuracy rate on the MNIST and Fashion-MNIST datasets, reaching 72% and 71% respectively. However, on CIFAR-10, the accuracy rates of all methods are low, but the algorithm of the present invention also reaches 37%. This shows that even in the face of potential data privacy and security threats, the algorithm of the present invention can still maintain good model performance. This is because the algorithm of the present invention performs global model aggregation by calculating the accuracy of public data, which can improve the fault tolerance and stability of the aggregated model. Secondly, this method uses EM distance instead of traditional KL divergence for knowledge distillation, which makes the accuracy of Ours with KD method slightly lower than that of the algorithm of the present invention. In contrast, other protection methods such as Fed-CDP and FL-AC perform poorly on the MNIST dataset, with an accuracy rate of only 12% and 11%, indicating that these methods greatly sacrifice the learning ability of the model while protecting privacy.

[0111] In summary, the high-performance privacy-enhanced federated learning algorithm based on conditional generative adversarial networks proposed in the present invention achieves an effective balance between privacy protection and model performance through multi-dimensional collaborative optimization compared to traditional methods. In response to these limitations, the present invention first generates auxiliary training data through conditional generative adversarial networks, expands the local data set and alleviates overfitting in combination with feature distance loss, and reduces the amount of sensitive information directly involved in federated training by replacing the original data, thereby reducing the risk of privacy leakage from the data source. Secondly, a dual-model architecture of private models and public models is introduced to improve the traditional deep mutual learning method using bulldozer distance as the loss function, thereby ensuring the stable transfer of knowledge between the dual models while strictly limiting sensitive information to the local private model, solving the problem that intermediate parameters in traditional methods are easily attacked by reverse attacks. In addition, a dynamic aggregation strategy based on predictive distribution evaluates the contribution of client models through public data sets, and prioritizes the aggregation of high-performance model parameters, which not only improves the convergence speed and accuracy of the global model, but also avoids the interference of low-quality models.

[0112] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A privacy-enhanced federated learning method, characterized in that: include: Step 1: Each client receives the global model parameters from the server. At the same time, each client generates auxiliary training data through a conditional generative adversarial network. Step 2: Each client initializes local private model parameters and public model parameters through the received global model parameters, trains private model parameters through local original data and public model parameters through auxiliary training data, and uses the deep model mutual training method to mutually train public model parameters and private model parameters; Step 3: After the training is completed, the public model parameters of each client after training are uploaded to the server. The server infers the public model parameters of each client after training based on the public data set and statistically predicts the results; Step 4: The server calculates the weight of each client based on the prediction results; Step 5: The server performs weighted aggregation on the public model parameters uploaded by each client according to the weight, updates the global model parameters, and broadcasts the updated global model parameters to each client; Step 6: Repeat steps 2 to 5 until the global model parameters meet the requirements; Wherein: the generating auxiliary training data by conditional generative adversarial network specifically includes: introducing feature distance loss into the CGAN objective function to expand the auxiliary training data.

2. The privacy-enhanced federated learning method according to claim 1, characterized in that: The feature distance loss is introduced into the CGAN objective function to expand the auxiliary training data, and its expression is: Among them, x is the local original data, and y is the conditional label; is the adversarial objective function between the discriminator D and the generator G in the generative adversarial network, For the distribution of real data p data(x) The expected value of x is obtained by sampling from is the noise distribution p from the prior z(z) The expected value of z is obtained by sampling, P data(x) is the distribution of local original data, P z(z) is the prior noise distribution; logD(x|y) is the logarithmic probability that the discriminator D marks the local original data as true; z is the logarithmic probability of the discriminator D marking the local original data as true from the prior noise distribution P z(z) noise sampled in; G(z|y) is the auxiliary training data generated by the generator G; log(1-D(G(z|y))) is the logarithmic probability that the discriminator G marks the generated auxiliary training data as false; δ is the coefficient controlling the proportion of feature distance loss; g(·) is the function mapping local raw data to features, and ∥g(x)-g(G(z|y))∥2 is the feature distance loss.

3. The privacy-enhanced federated learning method according to claim 1, characterized in that: Before the client trains the public model parameters through the auxiliary training data, the auxiliary training data may be subjected to data enhancement processing, which is expressed as follows: x′=M⊙x a +(1-M)⊙x; y′=ρy a +(1-ρ)y; Where x is the local original data, x′ is the auxiliary training data after data enhancement processing; M is a 0-1 matrix with the same dimension as the local original data; x a is the output data of generator G; y′, y a are x′, x a The label of ; ρ is a random sample from the 0-1 distribution.

4. The privacy-enhanced federated learning method according to claim 3, characterized in that: Each client trains private model parameters through local original data and public model parameters through auxiliary training data, and uses the deep model mutual training method to mutually train public model parameters and private model parameters, and the expression is: L Θ1 =L C1 +W(Θ1,Θ2); L Θ2 =L C2 +W(Θ2,Θ1); Where: L Θ1 The objective function for training the local private model parameters for the client; L Θ2 The objective function for training the public model parameters for the client; L C1 is the loss function for training the private model parameters using local raw data, L C2 The loss function of the client training the public model parameters; W(Θ1,Θ2) is the objective function of mutual training of public model parameters and private model parameters using the deep model mutual training method; Where: K is the total number of local original data samples, y k,c is the label of category c of the kth sample in the local original data, p k,c The probability associated with category c of the kth sample of the original data; Where: H is the total number of auxiliary training data, y′ h,c is the label of category c of the kth sample in the auxiliary training data, p′ h,c is the probability associated with category c of the kth sample of the auxiliary training data; Where: Θ1 is the private model parameter distribution; Θ2 is the public model parameter distribution; inf is the minimum value of the predicted distribution of private model parameters and public model parameters, Π(Θ1.Θ2) is the set of all possible joint probability distributions formed by combining Θ1 and Θ2, p1 is the predicted distribution of private model parameters, p2 is the predicted distribution of public model parameters, and ∥p1-p2∥ is a distance metric defined in the space of predicted distributions of private model parameters and public model parameters.

5. The privacy-enhanced federated learning method according to claim 4, characterized in that: The server infers the public model parameters trained by each client according to the public data set and statistically predicts the results, specifically including: The server obtains the public model parameters W from client i i , Let r i =0, indicating that the initial "correct prediction number" of the public model parameters of the i-th client is 0, For public datasets on the server The kth sample x in (k) , do the following: Through the public model parameter W i To build the model function Model i (·), The kth sample x (k) Enter the model, the expression is: in: is the predicted probability vector output by the model, Model i (x (k) ; W i ) is to calculate the kth sample x in the public dataset (k) Input model W i The obtained label prediction distribution vector, Take the category corresponding to the maximum probability as the prediction result, the expression is: Among them, c traverses all possible categories; if This means that the kth sample prediction is correct, so let: r i =r i +1, after completing the reasoning of all M samples, the final r i That is model W i The number of correct examples predicted on a public dataset.

6. The privacy-enhanced federated learning method according to claim 5, characterized in that: The server calculates the weight of each client according to the prediction result, specifically including: Where: i is the weight of client i, r i is the number of correct predictions of client i on the public dataset, and N is the total number of clients.

7. The privacy-enhanced federated learning method according to claim 6, characterized in that: The server performs weighted aggregation on the public model parameters uploaded by each client according to the weights, and updates the global model parameters, specifically including: Where: W′ global is the updated global model parameter, W global is the global model parameter before updating, W i ′ is the public model parameter uploaded by client i.

8. The privacy-enhanced federated learning method according to claim 1, characterized in that: Each client initializes the local private model parameters and public model parameters through the received global model parameters, specifically including: when federated learning starts or the server completes a round of global model parameter aggregation, the server broadcasts the new global model parameters W to all clients global After receiving it, the client caches it first and prepares for initialization. The client's private model parameter is recorded as W private , let the client's public model parameter be W public ; When federated learning starts, the initial model parameters W sent by the server are used global Directly used as the initial model of private model parameters and public model parameters, the server then sends the updated global model parameters W′ in each communication round global When , only the updated global model parameters W′ are used global Replace the original as the public model parameter W public as a new public model parameter; instead of the private model parameter W private Initialize.

Images