Federal learning privacy protection method based on block chain and zero knowledge proof

Abstract

The invention discloses a federated learning privacy protection method based on a block chain and zero knowledge proof, and relates to the technical field of machine learning privacy protection, and the method comprises the steps that a task publisher determines a training task and a participant set of federated learning, initializes model parameters, and distributes the training task and initial global model parameters to each participant; each participant carries out local model training and carries out noise addition on the gradient of the local model to realize differential privacy protection; generating a zero-knowledge proof related to a local model reasoning process, and uploading the zero-knowledge proof to the block chain; and verifying the received zero-knowledge proof by using an intelligent contract, updating and transmitting the verified local model to a task publisher for global model aggregation, and issuing the local model to honest participants for next training until convergence. Therefore, by adopting the method, the privacy of the individual data can be protected on the premise of not influencing the overall data analysis, and the verification and transparent management of the reliability of the model source can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of privacy protection for machine learning, and in particular to a privacy protection method for federated learning based on blockchain and zero-knowledge proof. Background Art

[0002] As a common research hotspot in the field of artificial intelligence and pattern recognition, machine learning is widely used in the fields of transportation, electricity, etc. Among them, federated learning is a distributed machine learning method that can achieve cross-device or node model collaborative training while ensuring local data storage, avoiding centralized data transmission and storage, and strengthening the protection of data privacy.

[0003] However, the application of federated learning still faces some security and privacy challenges. First, the client may be attacked by malicious attackers when uploading local model updates, resulting in privacy data leakage; second, since the model training process is invisible, the effect of the trained local model is difficult to guarantee. Malicious attackers may disguise themselves as honest local clients and upload tampered malicious models, thereby interfering with the aggregation of the global model of federated learning.

[0004] Therefore, it is necessary to provide a federated learning privacy protection method that can protect individual data privacy while verifying the reliability and transparently managing the model source, so as to provide a reference for building a more secure, reliable, and privacy-protected distributed machine learning system. Summary of the invention

[0005] The purpose of this invention is to provide a privacy protection method for federated learning based on blockchain and zero-knowledge proof, to provide a transparent and tamper-proof verification method for distributed systems, and to enhance transparency and security.

[0006] To achieve the above objectives, the present invention provides a privacy protection method for federated learning based on blockchain and zero-knowledge proof, comprising the following steps:

[0007] S1. The task publisher determines the training task and the set of participants in federated learning, initializes the model parameters, and distributes the training task and initial global model parameters to each participant;

[0008] S2. Each participant uses the local data set for local training, calculates the loss function and back-propagates the gradient, clips the gradient, adds Gaussian noise, and then updates the local model.

[0009] S3. Each participant generates a zero-knowledge proof about the local model reasoning process through the zero-knowledge proof system, and sends the zero-knowledge proof, verification key and setting information to the blockchain;

[0010] S4. The blockchain uses smart contracts to verify the received zero-knowledge proofs, and transmits the verified local model updates to the task publisher, and aggregates them into a global model;

[0011] S5. Distribute the global model to each verified participant, and repeat steps S2-S4 to perform model training and aggregation until convergence.

[0012] Preferably, step S2 includes:

[0013] Clip the gradient, the expression is as follows:

[0014]

[0015] In the formula, is the local model parameter of the ith participant at time t, D i is the local dataset of the ith participant, C is the threshold of the clipped L2 norm;

[0016] Add Gaussian noise to the clipped gradient as follows:

[0017]

[0018] In the formula, is the gradient after adding noise, σ is the noise multiplier, G(0,σ 2 C 2 ) is Gaussian noise.

[0019] Preferably, the local model is updated using the noisy gradient as follows:

[0020]

[0021] Where η is the learning rate of the participant.

[0022] Preferably, the zero-knowledge proof system includes:

[0023] Using local model parameters and zero-knowledge proof to set parameters, the model's reasoning process is converted into an arithmetic circuit, and a set of structured reference strings SRS are pre-generated to provide the public parameters of the circuit;

[0024] Utilize the local data set and local model to generate a witness for the zero-knowledge proof, and utilize the KZG polynomial commitment scheme and the pre-generated structured reference string SRS to generate the proof key and verification key for the zero-knowledge proof, thereby generating a zero-knowledge proof. Send the zero-knowledge proof, verification key, and setting parameters to the blockchain, and use smart contracts to verify the received zero-knowledge proof, identify eligible participants, and receive the local model updates uploaded by them.

[0025] Preferably, the aggregation of the global model is as follows:

[0026]

[0027] In the formula, w global is the global model, M, m i They represent the total amount of global data and the number of local data samples of participants respectively, and n is the number of local models that have passed verification.

[0028] Therefore, the present invention adopts the above-mentioned federated learning privacy protection method based on blockchain and zero-knowledge proof, which has the following technical effects:

[0029] (1) Use local differential privacy technology to add noise to the model gradients during the local training process of participants to prevent malicious attackers from stealing model data and ensure the security of the local model.

[0030] (2) Use zero-knowledge proof technology to generate proof of the model reasoning process, and verify the zero-knowledge proof of the local model reasoning process through blockchain technology, thereby solving the model verification and data privacy issues of federated learning.

[0031] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the overall process of a federated learning privacy protection method based on blockchain and zero-knowledge proof;

[0033] Figure 2 A flowchart of zero-knowledge proof generation in an embodiment of a method for privacy protection of federated learning based on blockchain and zero-knowledge proof;

[0034] Figure 3 It is a flowchart of blockchain verification in an embodiment of a method for protecting privacy in federated learning based on blockchain and zero-knowledge proof;

[0035] Figure 4 The present invention is a schematic diagram of the vehicle networking process in an embodiment of a federated learning privacy protection method based on blockchain and zero-knowledge proof. DETAILED DESCRIPTION

[0036] The present invention can be explained in more detail by the following examples. The purpose of disclosing the present invention is to protect all changes and improvements within the scope of the present invention. The present invention is not limited to the following examples.

[0037] Embodiment 1

[0038] like Figure 1As shown, the present invention provides a method for protecting privacy in federated learning based on blockchain and zero-knowledge proof, including training task release, local model training, zero-knowledge proof generation, blockchain verification, and global model aggregation. The specific steps are as follows:

[0039] S1. The task publisher hopes to improve the generalization ability of the model through a certain degree of training so that it can perform well in a wide range of scenarios. At the same time, in order to meet user privacy requirements, the publisher hopes to cooperate with several participants through federated learning to complete the training task, thereby completing the model training without directly accessing the participants' local data.

[0040] The task publisher determines the training task and the set of federated learning participants Initialize the global model parameters w 0 , the training task and w 0 Distribute to selected participants.

[0041] S2. First, after receiving the global model and training task, the i-th participant uses the local dataset D i Through the loss function Calculate the loss and backpropagate to get the gradient

[0042]

[0043] Secondly, before adding differential privacy noise, the gradients are clipped to ensure that the norms of all gradients before adding noise are within the control range, as follows:

[0044]

[0045] in, is the local model of the ith participant at the tth iteration, and C is the threshold of the clipped L2 norm.

[0046] Then, Gaussian noise G(0,σ 2 C 2 ) to ensure the privacy of individual data without affecting the overall data analysis, as follows:

[0047]

[0048] Among them, σ is the noise multiplier, which represents the multiple of the noise added to the gradient or loss.

[0049] Use the noisy gradient to update the client's local model parameters

[0050]

[0051] Where η is the learning rate of the participant.

[0052] S3, such as Figure 3 As shown in the figure, when the i-th participant completes local training, it starts to build zero-knowledge proof. Specifically, the participant extracts a data sample from the training data set as input, and determines whether to increase the batch dimension according to the actual situation to ensure that the shape of the input data adapts to the batch input requirements of the model. The input data is converted into a numpy array, and then the multi-dimensional numpy array is flattened to facilitate serialization and storage.

[0053] Participants record the parameter information of the local model as zero-knowledge proof setting parameters (Settings), which include the model's input dimension, output dimension, weight, bias, etc., as well as the visibility of input data, model parameters and output results, for subsequent reproduction of the model's reasoning process.

[0054] In this embodiment, the local model w is used i and Settings, and transform the model’s reasoning process into an arithmetic circuit Circuit←(w i ,Settings), recursive proof without trusted settings. In addition, a set of structured reference strings SRS are generated in advance, which provide the public parameters of the circuit, so that all participants can generate and verify proofs in the same trusted environment.

[0055] Using the input sample and the local model, record all the intermediate values ​​and calculations performed when the model runs the input, and generate the witness of the zero-knowledge proof for use in the subsequent generation of the proof. Based on the KZG polynomial commitment scheme provided by the Halo2 proof system, use the previously generated circuit and structured reference string SRS to generate the proof key and verification key (P k ,V k )←(Circuit,SRS). Through the local model w i , Witness, Proof Key P k Generate zero-knowledge proof π about the model reasoning process i ←(w i ,Witness,P k ). Participants will use zero-knowledge proof, verification key and setting parameters (π i ,V k ,Settings) are sent to the blockchain for subsequent verification.

[0056] S4, such as Figure 4 As shown, the blockchain receives and records the information submitted by each participant (π i ,V k,Settings), and use the smart contract to perform the verification process {0,1}←VerifyProve(π i ,V k ,Settings). The verification node verifies whether the zero-knowledge proof meets specific constraints based on the encrypted information contained in the proof, that is, whether it matches the model reasoning result of the client.

[0057] After the verification is completed, the result {0,1} is recorded on the blockchain. If the verification is passed, it means that the reasoning process of the model is authentic and reliable. At this time, the submitted local model is updated for subsequent federated learning global model aggregation; if the verification fails, it is marked as invalid, triggering further review or rejection of the submission.

[0058] According to the zero-knowledge proof verification result, the task publisher receives the local model update sent to it by the verified participants and executes the global model aggregation algorithm as follows:

[0059]

[0060] In the formula, w global is the global model, M, m i They represent the total amount of global data and the number of local data samples of participants respectively.

[0061] S5. The task publisher sends the updated global model to the verified participants so as to continue to optimize and improve the performance of the model in a new round of local training.

[0062] In addition, the task publisher can choose to end the training, or continue to repeat the above steps S2-S4 until the global model converges or reaches the specified number of iterations.

[0063] Embodiment 2

[0064] At present, as an emerging technology, the Internet of Vehicles is becoming a key support in the field of modern transportation. By connecting vehicles, road infrastructure, pedestrians and other devices, the Internet of Vehicles enables information sharing and intelligent decision-making, effectively improving traffic safety, road efficiency and user experience. However, the surge in the number of Internet of Vehicles devices has made the generation and processing of massive data from vehicles, the limitation of communication and computing resources, and the potential risk of privacy leakage a huge challenge.

[0065] To this end, a federated learning privacy protection method based on blockchain and zero-knowledge proof is adopted in the present invention to effectively solve the problems currently faced by the Internet of Vehicles, such as Figure 4 As shown, the specific steps include:

[0066] S1. Distribute federated learning tasks through the roadside units (RSUs) in the Internet of Vehicles, initialize global model parameters, and broadcast the task description and initialized global model to all participating vehicles.

[0067] S2. Each vehicle performs local training using the local dataset according to the given federated learning task and obtains local model updates.

[0068] S3. The vehicle generates a zero-knowledge proof about the local model reasoning process through the zero-knowledge proof system, and sends the proof, verification key and setting information to the blockchain for verification.

[0069] S4. The blockchain verifies the zero-knowledge proof submitted by the vehicle, confirms that the model is trained by an honest vehicle, and saves the verification result on the chain.

[0070] S5. The roadside unit RSU obtains the verification result from the blockchain, receives the local model update provided by the honest vehicle, executes the global model aggregation algorithm, obtains the global model update, and sends the global model update to the honest vehicle for the next round of federated learning training.

[0071] S6. Execute the federated learning task until the global model converges and the training ends.

[0072] Therefore, the present invention adopts the above-mentioned federated learning privacy protection method based on blockchain and zero-knowledge proof, which can ensure the verifiability, security and privacy of the federated learning system.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A privacy protection method for federated learning based on blockchain and zero-knowledge proof, characterized in that: The following steps are involved: S1. The task publisher determines the training task and the set of participants in federated learning, initializes the model parameters, and distributes the training task and initial global model parameters to each participant; S2. Each participant uses the local data set for local training, calculates the loss function and back-propagates the gradient, clips the gradient, adds Gaussian noise, and then updates the local model. S3. Each participant generates a zero-knowledge proof about the local model reasoning process through the zero-knowledge proof system, and sends the zero-knowledge proof, verification key and setting information to the blockchain; S4. The blockchain uses smart contracts to verify the received zero-knowledge proofs, and transmits the verified local model updates to the task publisher, and aggregates them into a global model; S5. Distribute the global model to each verified participant, and repeat steps S2-S4 to perform model training and aggregation until convergence.

2. According to claim 1, a method for protecting privacy in federated learning based on blockchain and zero-knowledge proof, characterized in that: Step S2 includes: Clip the gradient, the expression is as follows: In the formula, is the local model parameter of the ith participant at time t, D i is the local dataset of the ith participant, C is the threshold of the clipped L2 norm; Add Gaussian noise to the clipped gradient as follows: In the formula, is the gradient after adding noise, σ is the noise multiplier, G(0,σ 2 C 2 ) is Gaussian noise.

3. According to a method for protecting privacy in federated learning based on blockchain and zero-knowledge proof according to claim 1, it is characterized in that: The local model is updated using the noisy gradient as follows: Where η is the learning rate of the participant.

4. According to a method for protecting privacy in federated learning based on blockchain and zero-knowledge proof according to claim 1, it is characterized in that: The zero-knowledge proof system includes: Using local model parameters and zero-knowledge proof to set parameters, the model's reasoning process is converted into an arithmetic circuit, and a set of structured reference strings SRS are pre-generated to provide the public parameters of the circuit; Using the local data set and local model, a witness for the zero-knowledge proof is generated, and the proof key and verification key for the zero-knowledge proof are generated using the KZG polynomial commitment scheme and the pre-generated structured reference string SRS, thereby generating a zero-knowledge proof; The zero-knowledge proof, verification key, and setting parameters are sent to the blockchain, and the received zero-knowledge proof is verified using smart contracts to identify eligible participants and receive the local model updates uploaded by them.

5. According to a method for protecting privacy in federated learning based on blockchain and zero-knowledge proof according to claim 1, it is characterized in that: The aggregation of the global model is as follows: In the formula, w global is the global model, M, m i They represent the total amount of global data and the number of local data samples of participants respectively, and n is the number of local models that have passed verification.

Images