Verifiable longitudinal federated learning privacy protection method
By using verifiable privacy protection methods in vertical federated learning, using Sharple value to calculate the contribution value of the passive party and weighted aggregation, the problem of unverifiable local intermediate results of the participants is solved, improving the accuracy and performance of the model, and reducing the accuracy and performance of the model Malicious effects and save computational overhead.
Patent Information
- Application Number
- CN202410776703.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-17
- Publication Date
- 2024-11-15
AI Technical Summary
The local intermediate results of participants in vertical federated learning cannot be verified, resulting in low model training performance, and intermediate results of the aggregation server dishonest or passive party submitting errors may occur, polluting the global model.
Using verifiable vertical federated learning privacy protection method, the key is generated by trusted third parties, and the encryption and decryption and homomorphic encryption are performed. The contribution value of the passive party is calculated using the Sharple value, and its contribution to the global model is verified. It is also used to train the process. Weighted aggregation is performed in the model to ensure the accuracy of the model.
Effectively evaluate the contribution value of the passive party, reduce malicious influence, improve model training performance, save computing overhead, and improve the accuracy of the global model.
Smart Images

Figure CN118965417A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a verifiable vertical federated learning privacy protection method, belonging to the technical field of federated learning. Background Art
[0002] In recent years, with the increasing emphasis on data privacy protection, data exists in the form of isolated islands between individuals, which seriously affects the use value of high-quality data. Federated learning is proposed to achieve privacy protection in data circulation. Federated learning allows multiple participants to jointly train a global model and keep the data locally. The training data and training process of any participant are invisible. In vertical federated learning, it is usually initiated by an active party and assisted by multiple passive parties to jointly train a global model.
[0003] Vertical federated learning provides a technical solution to data silos. However, vertical federated learning still faces the challenge of the inability to verify the local intermediate results of the participants, such as the aggregation server dishonestly aggregating the local model or forging the global model and the participants submitting incorrect intermediate results to pollute the global model, which in turn results in low model training performance.
[0004] Therefore, how to verify the contribution of the passive party's local model to the global model in order to improve the model accuracy has become an urgent problem to be solved. Summary of the invention
[0005] The purpose of the present invention is to solve the problem of low model performance caused by the unverifiable passive party of traditional vertical federated learning, and proposes a verifiable vertical federated learning privacy protection method.
[0006] The objective of the present invention is achieved through the following technical solutions:
[0007] A verifiable vertical federated learning privacy protection method of the present invention is applied to the vertical federated model training of a participant Z including a member node and a trusted third party, and includes the following steps:
[0008] Step 1: Build an initialized federated learning global model and initialize the node parameters of the participant Z and the trusted third party of the member node. The participant Z includes the active party A and the passive party B; the trusted third party is used to generate key pairs and perform key distribution and encryption and decryption.
[0009] Step 2: Perform privacy intersection on the local data of participant Z to achieve sample alignment of training data.
[0010] Step 3: The trusted third party distributes an encryption key pair to participant Z, which is used to encrypt the intermediate result obtained by participant Z in step 5.1;
[0011] Step 4: Party Z uses local features to perform forward training of the local model to obtain intermediate results.
[0012] Step 4.1: Participant Z trains parameters θ for the local model Z Initialize.
[0013] Step 4.2: Participant Z performs local model forward training on the data set trained locally by participant Z, as shown in formula (1).
[0014] Y Z =X z θ Z (1)
[0015] Among them, X Z represents the local data owned by participant Z; θ Z represents the local model parameters owned by participant Z, where participant Z∈{active party A, passive party B}.
[0016] Step 5: Party Z exchanges the intermediate results after homomorphic encryption, and then obtains the exchanged intermediate results.
[0017] Step 5.1: Party Z performs homomorphic encryption on the intermediate result described in step 4 and calculates the encrypted intermediate result ||θ Z X Z ||, where the active party A obtains the encrypted intermediate result ||θ A X A ||; Passive party B obtains the encrypted intermediate result ||θ B X B ||.
[0018] Step 5.2: The active party A and the passive party B exchange the encrypted intermediate results, that is, the active party A sends ‖θ A X A -Y‖ to the passive party B, the passive party B sends ‖θ B X B ‖ to the active party A; where Y represents the label value of the sample.
[0019] Step 6: Calculate the local model gradient for the swapped intermediate results described in step 5.
[0020] Step 6.1: Use the cross entropy loss function calculation method to calculate the active party A and the passive party B loss function as loss(θ A ) and loss(θ B ).
[0021] Step 6.2: Active party A and passive party B calculate the gradient according to the loss function
[0022] Step 6.3: Party Z adds mask M to the swap intermediate result Z , Sent to a trusted third party; where the participants Z∈{active party A, passive party B}.
[0023] Step 6.4: The trusted third party decrypts the And send it to the participant Z, where participant Z∈{active party A, passive party B}.
[0024] Step 7: Party Z calculates the validation set loss function and contribution value.
[0025] Step 7.1: Party Z calculates the loss function loss on the validation set v v (θ Z ) and gradient Among them, the participants Z∈{active party A, passive party B}.
[0026] Step 7.2: Participant Z calculates the contribution value φ of this round based on the Shapley value approximation formula t,z , as shown in formula (2), and sent to the active party A.
[0027]
[0028] Among them G t Represents the global gradient, diag() represents the creation of a diagonal matrix, Represents the set of all validation sets, and E represents the identity matrix. Among them, the participating parties Z∈{active party A, passive party B}.
[0029] Step 8: Party Z performs reverse model update training for the local model.
[0030] Step 8.1: Active party A calculates the weight ω of the local model aggregation of participant Z based on the contribution value Z , as shown in formula (3).
[0031]
[0032] And will send ω B To the passive party B; among them, the participating party Z∈{active party A, passive party B}.
[0033] Step 8.2: The passive party B adds weights to its intermediate results and obtains the weighted gradient value, as shown in formula (4).
[0034]
[0035] Step 8.3: The active party A calculates the weighted gradient value, as shown in formula (5).
[0036]
[0037] Step 8.4: Participant Z uses the gradient descent algorithm to update the local model parameters as shown in equation (6).
[0038]
[0039] Among them l r represents the learning rate, represents the local model parameters of participant Z in round t, G represents the local model parameters of participant Z in round t-1. t Represents the global gradient value of participant Z, where participant Z∈{active party A, passive party B}.
[0040] Step 9: Repeat steps 4 to 8 until the training model reaches the expected convergence accuracy or reaches the pre-agreed maximum number of training rounds, at which point the training is stopped. Thus, verifiable longitudinal federated learning using Shapley values is achieved.
[0041] Beneficial effects:
[0042] 1 The present invention evaluates the contribution value of the passive party, verifies the importance of the information submitted by the passive party to the global model, and uses the contribution value to achieve weighted aggregation of the global model, thereby reducing the negative impact of malicious clients on the global model and improving the model training performance.
[0043] 2 In the process of collaborative training among all participants, the passive party does not need to perform additional training in a global round, which greatly saves the computational overhead of training.
[0044] 3 The vertical federated learning privacy protection method disclosed in the present invention aims to solve the problem that the local model of the passive party in the existing vertical federated learning is unverifiable, resulting in low model performance. A verifiable vertical federated learning privacy protection method is proposed to improve the performance of the model. During the training process, the Shapley value is used to calculate the contribution value of the passive party, thereby verifying the performance of the local model of the passive party and improving the accuracy of the global model. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 It is a schematic diagram of the process of the present invention;
[0046] Figure 2 This is a comparison chart of the model accuracy of traditional vertical federated learning and verifiable vertical federated learning methods. DETAILED DESCRIPTION
[0047] In order to better illustrate the purpose and advantages of the present invention, the invention is further described below in conjunction with the accompanying drawings and examples. It should be noted that the implementation of the present invention is not limited to the following embodiments, and any form of modification or change made to the present invention will fall within the protection scope of the present invention.
[0048] This embodiment provides a verifiable longitudinal federated learning method based on Shapley values under a fully connected neural network MLP. The system initializes an image classification, i.e., a handwriting recognition task and two participants, one active party and one passive party. Furthermore, the local data set is MNIST, an image with a known image classification label. Among them, the image classification label is 0 to 9. Further, a fully connected neural network MLP local model is set.
[0049] Example
[0050] like Figure 1 As shown, the verifiable vertical federated learning privacy protection method of the present invention has the following specific implementation steps:
[0051] Step 1: Build an initialized federated learning global model and initialize the node parameters of the participant Z and the trusted third party of the member node. The participant Z includes the active party A and the passive party B; the trusted third party is used to generate key pairs and perform key distribution and encryption and decryption.
[0052] Step 2: Perform privacy intersection on the local data of participant Z to achieve sample alignment of training data.
[0053] Step 3: The trusted third party distributes an encryption key pair to participant Z, which is used to encrypt the intermediate result obtained by participant Z in step 5.1;
[0054] Step 4: Party Z uses local features to perform forward training of the local model to obtain intermediate results.
[0055] Step 4.1: Participant Z trains parameters θ for the local model Z Initialize.
[0056] Step 4.2: Participant Z performs local model forward training on the data set trained locally by participant Z, as shown in formula (1).
[0057] Y Z =X z θ Z (1)
[0058] Among them, X Z represents the local data owned by participant Z; θ Z represents the local model parameters owned by participant Z, where participant Z∈{active party A, passive party B}.
[0059] Step 5: Party Z exchanges the intermediate results after homomorphic encryption, and then obtains the exchanged intermediate results.
[0060] Step 5.1: Party Z performs homomorphic encryption on the intermediate result described in step 4 and calculates the encrypted intermediate result ||θ Z X Z ||, where the active party A obtains the encrypted intermediate result ||θ A X A ||; Passive party B obtains the encrypted intermediate result ||θ B X B ||.
[0061] Step 5.2: The active party A and the passive party B exchange the encrypted intermediate results, that is, the active party A sends ||θ A X A -Y|| to the passive party B, the passive party B sends ||θ B X B || to the active party A; where Y represents the label value of the sample.
[0062] Step 6: Calculate the local model gradient for the swapped intermediate results described in step 5.
[0063] Step 6.1: Use the cross entropy loss function calculation method to calculate the active party A and the passive party B loss function as loss(θ A ) and loss(θ B ).
[0064] Step 6.2: Active party A and passive party B calculate the gradient according to the loss function
[0065] Step 6.3: Party Z adds mask M to the swap intermediate result Z , Sent to a trusted third party; where the participants Z∈{active party A, passive party B}.
[0066] Step 6.4: The trusted third party decrypts the And send it to the participant Z, where participant Z∈{active party A, passive party B}.
[0067] Step 7: Party Z calculates the validation set loss function and contribution value.
[0068] Step 7.1: Party Z calculates the loss function loss on the validation set v v (θ Z ) and gradient Among them, the participants Z∈{active party A, passive party B}.
[0069] Step 7.2: Participant Z calculates the contribution value of this round based on the Shapley value approximation formula As shown in formula (2), and sent to the active party A.
[0070]
[0071] Among them G t Represents the global gradient, diag() represents the creation of a diagonal matrix, Represents the set of all validation sets, and E represents the identity matrix. Among them, the participating parties Z∈{active party A, passive party B}.
[0072] Step 8: Party Z performs reverse model update training for the local model.
[0073] Step 8.1: Active party A calculates the weight ω of the local model aggregation of participant Z based on the contribution value Z , as shown in formula (3).
[0074]
[0075] And will send ω B To the passive party B; among them, the participating party Z∈{active party A, passive party B}.
[0076] Step 8.2: The passive party B adds weights to its intermediate results and obtains the weighted gradient value, as shown in formula (4).
[0077]
[0078] Step 8.3: The active party A calculates the weighted gradient value, as shown in formula (5).
[0079]
[0080] Step 8.4: Participant Z uses the gradient descent algorithm to update the local model parameters as shown in equation (6).
[0081]
[0082] Among them l r represents the learning rate, represents the local model parameters of participant Z in round t, G represents the local model parameters of participant Z in round t-1. t Represents the global gradient value of participant Z, where participant Z∈{active party A, passive party B}.
[0083] Step 9: Repeat steps 4 to 8 until the training model reaches the expected convergence accuracy or reaches the pre-agreed maximum number of training rounds, at which point the training is stopped. Thus, verifiable longitudinal federated learning using Shapley values is achieved.
[0084] In order to further demonstrate the superiority of the present invention, the present invention conducts the following training experiment for illustration.
[0085] This experiment observes the model performance of traditional vertical federated learning and verifiable vertical federated learning by submitting incorrect results during the training process. The experimental results of the model accuracy of traditional vertical federated learning and verifiable vertical federated learning methods are compared. Figure 2 As shown in the figure, it can be concluded that the global model accuracy of verifiable longitudinal federated learning reaches 81.301%, while the global model accuracy of traditional longitudinal federated learning is only 63.298%, and the model accuracy has been improved by 18.003%.
Claims
1. A verifiable vertical federated learning privacy protection method, applied to the vertical federated model training of a participant Z including a member node and a trusted third party, characterized by: The following steps are included: Step 1: Build an initialized federated learning global model and initialize the node parameters of the participant Z of the member node and the trusted third party; the participant Z includes the active party A and the passive party B; the trusted third party is used to generate key pairs and perform key distribution and encryption and decryption; Step 2: Perform privacy intersection on the local data of participant Z to achieve sample alignment of training data; Step 3: The trusted third party distributes an encryption key pair to participant Z, which is used to encrypt the intermediate result obtained by participant Z in step 5.1; Step 4: Party Z uses local features to perform forward training of the local model to obtain an intermediate result. Step 5: Party Z exchanges the intermediate results after homomorphic encryption, and then obtains the exchanged intermediate results; Step 6: Calculate the local model gradient for the exchanged intermediate results described in step 5; Step 7: Participant Z calculates the validation set loss function and contribution value; Step 8: Party Z performs reverse model update training for the local model; Step 9: Repeat steps 4 to 8 until the training model reaches the expected convergence accuracy or reaches the pre-agreed maximum number of training rounds, at which point the training is stopped; thereby achieving verifiable longitudinal federated learning using Shapley values.
2. A verifiable vertical federated learning privacy protection method as claimed in claim 1, characterized in that: Step 4 is implemented as follows: Step 4.1: Participant Z trains parameters θ for the local model Z Initialize; Step 4.2: Participant Z performs local model forward training on the data set trained locally by Participant Z, as shown in formula (1); Y Z =X z i Z (1) Among them, X Z represents the local data owned by participant Z; θ Z represents the local model parameters owned by participant Z, where participant Z∈{active party A, passive party B}.
3. A verifiable vertical federated learning privacy protection method as claimed in claim 1, characterized in that: Step 5 is implemented as follows: Step 5.1: Party Z performs homomorphic encryption on the intermediate result described in step 4 and calculates the encrypted intermediate result ‖θ Z X Z ‖, where the active party A obtains the encrypted intermediate result ‖θ A X A ‖; Passive party B obtains the encrypted intermediate result ‖θ B X B ‖; Step 5.2: The active party A and the passive party B exchange the encrypted intermediate results, that is, the active party A sends ‖θ A X A -Y|| to the passive party B, the passive party B sends ‖θ B X B ‖ to the active party A; where Y represents the label value of the sample.
4. A verifiable vertical federated learning privacy protection method as claimed in claim 1, characterized in that: Step 6 is implemented as follows: Step 6.1: Use the cross entropy loss function calculation method to calculate the active party A and the passive party B loss function as loss(θ A ) and loss(θ B ); Step 6.2: Active party A and passive party B calculate the gradient according to the loss function Step 6.3: Party Z adds mask M to the swap intermediate result Z , Send to a trusted third party; Among them, the participants Z∈{active party A, passive party B}; Step 6.4: The trusted third party decrypts and obtains. And send it to the participant Z, where participant Z∈{active party A, passive party B}.
5. A verifiable vertical federated learning privacy protection method as claimed in claim 1, characterized in that: Step 7 is implemented as follows: Step 7.1: Party Z calculates the loss function loss on the validation set v v (θ Z ) and gradient Among them, the participants Z∈{active party A, passive party B}; Step 7.2: Participant Z calculates the contribution value of this round based on the Shapley value approximation formula As shown in formula (2), and sent to the active party A; Among them G t Represents the global gradient, diag() represents the creation of a diagonal matrix, Represents the set of all verification sets, E represents the identity matrix; where the participating parties Z∈{active party A, passive party B}.
6. A verifiable vertical federated learning privacy protection method as claimed in claim 1, characterized in that: Step 8 is implemented as follows: Step 8.1: Active party A calculates the weight ω of the local model aggregation of participant Z based on the contribution value Z , as shown in formula (3); And will send ω B To the passive party B; where the participants Z∈{active party A, passive party B}; Step 8.2: The passive party B adds weights to its intermediate results to obtain the weighted gradient value, as shown in formula (4); Step 8.3: The active party A calculates the weighted gradient value, as shown in formula (5); Step 8.4: Participant Z uses the gradient descent algorithm to update the local model parameters, as shown in formula (6); Among them l r represents the learning rate, represents the local model parameters of participant Z in round t, G represents the local model parameters of participant Z in round t-1; t Represents the global gradient value of participant Z, where participant Z∈{active party A, passive party B}.
Citation Information
Cited By
Cross-institution tenant information processing method and system based on federal learning
CN122490564A
