Blockchain and zero-knowledge proof-based trusted federated learning system and method
By introducing blockchain and zero-knowledge proofs into the federated learning system, and designing perturbation modules and sharding consensus algorithms, the problems of low efficiency and vulnerability to attacks in privacy protection and model verification in federated learning are solved, and secure, reliable and efficient training and model aggregation are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI JIAOTONG UNIV
- Filing Date
- 2024-08-29
- Publication Date
- 2026-07-21
AI Technical Summary
Existing federated learning systems suffer from inefficiency, lack of reliable verification, and vulnerability to malicious attacks in the process of privacy protection and model verification. This is especially true in blockchain technology, where Gaussian noise is difficult to verify and malicious nodes have many attack methods, leading to a significant increase in communication and computing overhead.
A trusted federated learning system based on blockchain and zero-knowledge proofs is adopted. Noise is added by a perturbation module to generate zero-knowledge proofs. The legitimacy of the noise is verified by a non-interactive zero-knowledge protocol, and the verification efficiency is improved by a sharding consensus algorithm to defend against Byzantine attacks.
It achieves a balance between security, model performance, and training efficiency in federated learning systems, can resist various security threats, significantly reduces communication overhead, and resists Byzantine attacks through a sharding consensus algorithm, ensuring the privacy of trainer data and the trustworthiness of the model.
Smart Images
Figure CN119090028B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, specifically to an algorithmic solution for problems related to security, trustworthiness, privacy protection, and Byzantine fault tolerance in federated learning, and more specifically to a trusted federated learning system and method based on blockchain and zero-knowledge proofs. Background Technology
[0002] In recent years, with advancements in computing hardware and communication technologies, blockchain and federated learning technologies have developed rapidly. Blockchain technology aims to build a trusted data interaction environment for multiple nodes in a network, where nodes collectively maintain consistent block data. Federated learning, on the other hand, involves multiple computing devices building machine learning models. It allows these devices to train locally using private data and then aggregate the trained local models into a final model. Due to their inherent distributed nature, building secure and trustworthy federated learning systems based on blockchain technology has attracted increasing attention.
[0003] In a typical blockchain-based federated learning system, machine learning model parameters are transmitted to multiple trainers. These trainers then use their local, private data as training data to train and transmit local models to validator nodes. To further protect the privacy of their local data, trainers typically add noise to the model parameters before transmitting the local models. Upon receiving the local models, the validator nodes perform a local validation process to evaluate the quality of the local models and, using the blockchain consensus mechanism, aggregate a subset of the local models to obtain the global model.
[0004] While blockchain and differential privacy technologies have enhanced the security of federated learning, they have also introduced new challenges. First, the mechanism for adding noise to the local model requires careful design. On one hand, adding noise can introduce performance loss to the global model to some extent. To comprehensively consider privacy protection and training accuracy, the noise level needs to be carefully controlled in the design, and its impact on training accuracy should be theoretically limited. On the other hand, in federated learning systems incorporating differential privacy, the model accuracy depends on the honesty of training nodes in adding conditional noise. Therefore, feasible schemes need to be designed to supervise the local training process.
[0005] To address the aforementioned challenges, one feasible approach is to apply zero-knowledge proofs (ZKP) to verify the local training process. However, regarding Gaussian noise, which is often added in linear arithmetic operations (FL), no work has yet been able to ensure its verifiability. Furthermore, applying ZKP typically involves converting the designed algorithm into a linear arithmetic circuit, but generating Gaussian noise involves a large number of nonlinear functions, which increases the difficulty of the verification process.
[0006] The second challenge stems from the complexity of the verification process. In blockchain-based federated learning systems, malicious nodes can not only add illegal noise to local models but also employ various attack methods to compromise the learning model's performance, including stochastic gradient attacks and injection techniques. Furthermore, existing methods require all verifiers to participate in the verification of each model, leading to a significant increase in additional computational and communication overhead as the model parameter size and the number of network nodes increase. A feasible solution is to utilize sharding technology to improve verification efficiency.
[0007] Currently, considering the challenges mentioned above, there is still no effective solution for designing an efficient, secure, and reliable federated learning system. Therefore, designing a secure, reliable, and efficient federated learning system based on zero-knowledge proofs and sharded blockchain technology is of paramount practical significance. A solution is needed to address the privacy leaks, inefficiencies, and lack of trusted verification inherent in federated learning.
[0008] Currently, no descriptions or reports of technologies similar to this invention have been found, and no similar information has been collected domestically or internationally. Summary of the Invention
[0009] To address the shortcomings of existing technologies, the purpose of this invention is to provide a trusted federated learning system and method based on blockchain and zero-knowledge proofs.
[0010] According to the present invention, a trusted federated learning system based on blockchain and zero-knowledge proofs includes:
[0011] The perturbation module includes a differential privacy submodule and a zero-knowledge proof submodule, which add noise to the local model and generate zero-knowledge proofs at the same time.
[0012] A non-interactive zero-knowledge protocol is applied to enable the verification of noise added to the local model. The trainer and verifier independently and separately construct the same circuit containing specific Gaussian noise using linear approximation. The trainer uses the circuit, a common generator, and model parameters to construct a proof. The verifier uses the circuit to verify the received proof and compares the verification output field with the noisy model. If the comparison result is correct, the proof is accepted.
[0013] Verification module: Includes local verification and sharding consensus process;
[0014] Validators are uniformly and randomly formed into a set containing multiple shards. In each validation cycle, validators in shards that meet specific criteria use the public dataset in the system to validate the quality of local models in parallel. One validator in a shard proposes a voting block and broadcasts it to other members of the shard. The block content is the selection result of the local model. Validators in the shard submit votes to reach a conclusion, and the process continues until a consensus is reached. Validators select a portion of the local models for global aggregation.
[0015] Preferably, in the disturbance module:
[0016] After initializing the system parameters, a common generator is first built based on the compressed Σ protocol and shared among all trainers and verifiers. Trainers and verifiers then construct consistent circuits using linear estimation based on specific Gaussian noise. During local training, trainers use the circuits, the common generator, and the model parameters to construct proofs. After receiving the proofs, verifiers use the generated circuits to verify the legality of the perturbation model.
[0017] The process of adding noise to the model satisfies (∈,δ) differential privacy, where ∈ and δ are differential privacy parameters, both of which are between 0 and 1.
[0018] Preferably, in the differential privacy submodule:
[0019] The noise follows a mean of 0 and a variance of . The distribution is Gaussian, and the variance is... satisfy:
[0020]
[0021] Where c1, c2, and G are constants, δ, ∈ are differential privacy parameters, T is the number of training rounds, q is the number of trainers, and n is the number of training samples in the public dataset;
[0022] In the zero-knowledge proof submodule:
[0023] The trainer completes local training, repeats the noise calculation steps a total of d times for the model dimension, adds noise to all parameters of the local model, and calculates zero-knowledge proofs by using the compressed Σ-protocol, through the generation circuit and the common generator.
[0024] The objective function F(θ) used in local training is differentiable with respect to the model parameters θ and has the following properties: Strong convexity, G-Lipshitz and L-smoothness properties mean that when the learning rate η = 1 / L, the objective function will expect the error to satisfy:
[0025]
[0026] Where n is the number of training samples, F* Let T be the optimal objective function, T be the number of training rounds, c be a constant, and E be the expected value of the expression within the square brackets.
[0027] Preferably, in the verification module:
[0028] The verification module employs a high-performance sharded blockchain consensus algorithm based on voting, and includes the following steps:
[0029] The verification module evaluates the quality of local models and selects a subset of models for aggregation based on their quality to defend against Byzantine attacks from malicious trainers. During the local verification phase, verifiers calculate the accuracy of both the local model and the previous round's global model, selecting the local model with higher accuracy for aggregation. In the consensus phase, all verifiers in each shard send their generated voting digests to each other. A verifier is randomly selected to package the correct voting results into a block. All verifiers reach a consensus on this block. If the consensus passes, the local model used for aggregation is determined; otherwise, a new round of consensus is initiated.
[0030] The local model used for aggregation first reaches consensus in a shard. Consensus ends when more than a certain proportion of validators submit consistent votes; otherwise, the consensus process is repeated in the next shard. This proportion is denoted as S. l ; The calculation formula is as follows:
[0031]
[0032] Where R = α / V represents the maximum proportion of malicious validators that this module can tolerate, α is the number of malicious nodes among the validators, V is the number of validators, z is a variable for setting system security parameters, j is the index of the blockchain shard in the system, r is the index of the first round of sharding, l is the consensus round number, and h... j Let j be the j-th blockchain shard.
[0033] Preferably, in the verification module:
[0034] For fragmentation Each validator v in j Calculate the local model separately Compared to the previous global model θ t-1 The accuracy;
[0035] For a given trainer τ i and validator v j The verifier obtains the local model of the trainer. The accuracy of this trainer's model is evaluated as follows:
[0036]
[0037] in, Let n be the number of training samples in the public dataset, and x be the number of training samples in the public dataset. k and y k These represent data samples and labels, respectively. The coefficient of the regularization term;
[0038] The verifier simultaneously calculates the previous round's global model θ t-1 Model accuracy:
[0039]
[0040] if If the model is selected for aggregation, then this model will be used; otherwise, this model will not participate in global model aggregation.
[0041] A trusted federated learning method based on blockchain and zero-knowledge proofs, provided by the present invention, includes:
[0042] Step S1: The perturbation module applies a non-interactive zero-knowledge protocol to enable the verification of noise added to the local model; the trainer and verifier independently and separately construct the same circuit containing specific Gaussian noise using linear approximation; the trainer uses the circuit, a common generator, and model parameters to construct a proof; the verifier uses the circuit to verify the received proof and compares the verification output field with the noisy model; if the comparison result is correct, the proof is accepted; including a differential privacy submodule and a zero-knowledge proof submodule, noise is added to the local model while generating zero-knowledge proofs;
[0043] Step S2: The verification module causes verifiers to form a set of multiple shards in a uniformly random manner. In each verification cycle, verifiers in shards that meet specific criteria use the public dataset in the system to verify the quality of their local models in parallel. One verifier in a shard proposes a voting block and broadcasts it to other members of the shard. The block content is the selection result of the local model. Verifiers in the shard submit votes to reach a conclusion, and the process continues until a consensus decision is reached. Verifiers select a portion of the local models for global aggregation. This includes the local verification and shard consensus processes.
[0044] Preferably, in the disturbance module:
[0045] After initializing the system parameters, a common generator is first built based on the compressed Σ protocol and shared among all trainers and verifiers. Trainers and verifiers then construct consistent circuits using linear estimation based on specific Gaussian noise. During local training, trainers use the circuits, the common generator, and the model parameters to construct proofs. After receiving the proofs, verifiers use the generated circuits to verify the legality of the perturbation model.
[0046] The process of adding noise to the model satisfies (∈,δ) differential privacy, where ∈ and δ are differential privacy parameters, both of which are between 0 and 1.
[0047] Preferably, in the differential privacy submodule:
[0048] The noise follows a mean of 0 and a variance of . The distribution is Gaussian, and the variance is... satisfy:
[0049]
[0050] Where c1, c2, and G are constants, δ, ∈ are differential privacy parameters, T is the number of training rounds, q is the number of trainers, and n is the number of training samples in the public dataset;
[0051] In the zero-knowledge proof submodule:
[0052] The trainer completes local training, repeats the noise calculation steps a total of d times for the model dimension, adds noise to all parameters of the local model, and calculates zero-knowledge proofs by using the compressed Σ-protocol, through the generation circuit and the common generator.
[0053] The objective function F(θ) used in local training is differentiable with respect to the model parameters θ and has the following properties: Strong convexity, G-Lipshitz and L-smoothness properties mean that when the learning rate η = 1 / L, the objective function will expect the error to satisfy:
[0054]
[0055] Where n is the number of training samples, F * Let T be the optimal objective function, T be the number of training rounds, c be a constant, and E be the expected value of the expression within the square brackets.
[0056] Preferably, in the verification module:
[0057] The verification module employs a high-performance sharded blockchain consensus algorithm based on voting, and includes the following steps:
[0058] The verification module evaluates the quality of local models and selects a subset of models for aggregation based on their quality to defend against Byzantine attacks from malicious trainers. During the local verification phase, verifiers calculate the accuracy of both the local model and the previous round's global model, selecting the local model with higher accuracy for aggregation. In the consensus phase, all verifiers in each shard send their generated voting digests to each other. A verifier is randomly selected to package the correct voting results into a block. All verifiers reach a consensus on this block. If the consensus passes, the local model used for aggregation is determined; otherwise, a new round of consensus is initiated.
[0059] The local model used for aggregation first reaches consensus in a shard. Consensus ends when more than a certain proportion of validators submit consistent votes; otherwise, the consensus process is repeated in the next shard. This proportion is denoted as S. l ; The calculation formula is as follows:
[0060]
[0061] Where R = α / V represents the maximum proportion of malicious validators that this module can tolerate, α is the number of malicious nodes among the validators, V is the number of validators, z is a variable for setting system security parameters, j is the index of the blockchain shard in the system, r is the index of the first round of sharding, l is the consensus round number, and h... j Let j be the j-th blockchain shard.
[0062] Preferably, in the verification module:
[0063] For fragmentation Each validator v in j Calculate the local model separately Compared to the previous global model θ t-1 The accuracy;
[0064] For a given trainer τ i and validator v j The verifier obtains the local model of the trainer. The accuracy of this trainer's model is evaluated as follows:
[0065]
[0066] in, Let n be the number of training samples in the public dataset, and x be the number of training samples in the public dataset. k and y k These represent data samples and labels, respectively. The coefficient of the regularization term;
[0067] The verifier simultaneously calculates the previous round's global model θ t-1 Model accuracy:
[0068]
[0069] if If the model is selected for aggregation, then this model will be used; otherwise, this model will not participate in global model aggregation.
[0070] Compared with the prior art, the present invention has the following beneficial effects:
[0071] 1. This invention balances the security, model performance, and training efficiency of federated learning, and can resist various security threats. It constructs a novel solution for problems such as security and trustworthiness, privacy protection, and Byzantine fault tolerance in federated learning systems, and has strong practicality and wide applicability.
[0072] 2. This invention can ensure that trainers can verifiably extract additive noise from a predefined Gaussian distribution, and by integrating an iterative sharding consensus algorithm, it can also effectively resist Byzantine attacks from malicious verifiers during model validation.
[0073] 3. This invention can significantly reduce communication overhead in blockchain networks;
[0074] 4. This invention designs a perturbation module based on non-interactive zero-knowledge proof technology, which makes the process of adding noise to the local model efficient and reliable. While ensuring the privacy of trainer data, it can also supervise the trainer's local training process.
[0075] 5. This invention designs a differential privacy federated learning noise addition mechanism, which enables the trainer to add noise to the gradient according to a specified distribution. This algorithm can be proven to achieve differential privacy and can analyze the upper bound of the impact of noise on model performance.
[0076] 6. This invention introduces a model validation process that selects updated models by scoring the trainers' models, thereby aggregating them into a new global model. This mechanism enables the federated learning system to resist Byzantine attacks, improving the system's robustness. Attached Figure Description
[0077] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0078] Figure 1 Flowchart for the incentive module;
[0079] Figure 2 Flowchart for the training module;
[0080] Figure 3 This is a schematic diagram of the system of the present invention. Detailed Implementation
[0081] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.
[0082] Example 1:
[0083] This invention provides a trusted federated learning system and method based on blockchain and zero-knowledge proofs, comprising the following modules: (1) a perturbation module. In the perturbation module, a non-interactive zero-knowledge (NIZK) protocol is applied to make the noise added to the local model verifiable. The trainer and verifier independently and separately construct the same circuit containing specific Gaussian noise using linear approximation. Then, the trainer uses the circuit, a common generator, and model parameters to construct a proof. Finally, the verifier uses their circuit to verify the received proof; (2) a verification module. The verification module constructs a sharded blockchain consensus process. Each verifier first uses a common dataset to verify the accuracy of the local model gradient updates. Then, they use their verification results as a voting digest to form consensus in the network. For block data containing local model selection information, a Byzantine fault-tolerant protocol is introduced to ensure data trustworthiness. The verification module enables the federated learning system to resist Byzantine attacks, improving the robustness of the system.
[0084] According to the present invention, a trusted federated learning method based on blockchain and zero-knowledge proof is provided, such as... Figure 3 ,include:
[0085] Step S1: The perturbation module applies a non-interactive zero-knowledge protocol to enable the verification of noise added to the local model; the trainer and verifier independently and separately construct the same circuit containing specific Gaussian noise using linear approximation; the trainer uses the circuit, a common generator, and model parameters to construct a proof; the verifier uses the circuit to verify the received proof and compares the verification output field with the noisy model; if the comparison result is correct, the proof is accepted; including a differential privacy submodule and a zero-knowledge proof submodule, noise is added to the local model while generating zero-knowledge proofs;
[0086] Specifically, in the disturbance module:
[0087] After initializing the system parameters, a common generator is first built based on the compressed Σ protocol and shared among all trainers and verifiers. Trainers and verifiers then construct consistent circuits using linear estimation based on specific Gaussian noise. During local training, trainers use the circuits, the common generator, and the model parameters to construct proofs. After receiving the proofs, verifiers use the generated circuits to verify the legality of the perturbation model.
[0088] The process of adding noise to the model satisfies (∈,δ) differential privacy, where ∈ and δ are differential privacy parameters, both of which are between 0 and 1.
[0089] Specifically, in the differential privacy submodule:
[0090] The noise follows a mean of 0 and a variance of . The distribution is Gaussian, and the variance is... satisfy:
[0091]
[0092] Where c1, c2, and G are constants, δ, ∈ are differential privacy parameters, T is the number of training rounds, q is the number of trainers, and n is the number of training samples in the public dataset;
[0093] In the zero-knowledge proof submodule:
[0094] The trainer completes local training, repeats the noise calculation steps a total of d times for the model dimension, adds noise to all parameters of the local model, and calculates zero-knowledge proofs by using the compressed Σ-protocol, through the generation circuit and the common generator.
[0095] The objective function F(θ) used in local training is differentiable with respect to the model parameters θ and has the following properties: Strong convexity, G-Lipshitz and L-smoothness properties mean that when the learning rate η = 1 / L, the objective function will expect the error to satisfy:
[0096]
[0097] Where n is the number of training samples, F * Let T be the optimal objective function, T be the number of training rounds, c be a constant, and E be the expected value of the expression within the square brackets.
[0098] Step S2: The verification module causes verifiers to form a set of multiple shards in a uniformly random manner. In each verification cycle, verifiers in shards that meet specific criteria use the public dataset in the system to verify the quality of their local models in parallel. One verifier in a shard proposes a voting block and broadcasts it to other members of the shard. The block content is the selection result of the local model. Verifiers in the shard submit votes to reach a conclusion, and the process continues until a consensus decision is reached. Verifiers select a portion of the local models for global aggregation. This includes the local verification and shard consensus processes.
[0099] Specifically, in the verification module:
[0100] The verification module employs a high-performance sharded blockchain consensus algorithm based on voting, and includes the following steps:
[0101] The verification module evaluates the quality of local models and selects a subset of models for aggregation based on their quality to defend against Byzantine attacks from malicious trainers. During the local verification phase, verifiers calculate the accuracy of the local model and the previous round's global model, selecting the local model with higher accuracy for aggregation. In the consensus phase, all verifiers in each shard send their generated voting digests to each other. A verifier is randomly selected to package the correct voting results into a block. All verifiers reach a consensus on this block. If the consensus passes, the local model used for aggregation is determined; otherwise, a new round of consensus is initiated.
[0102] The local model used for aggregation first reaches consensus in a shard. Consensus ends when more than a certain proportion of validators submit consistent votes; otherwise, the consensus process is repeated in the next shard. This proportion is denoted as S. l ; The calculation formula is as follows:
[0103]
[0104] Where T = α / V represents the maximum proportion of malicious validators that this module can tolerate, α is the number of malicious nodes among the validators, V is the number of validators, z is a variable for setting system security parameters, j is the index of the blockchain shard in the system, r is the index of the first round of sharding, l is the consensus round number, and h... j Let j be the j-th blockchain shard.
[0105] Specifically, in the verification module:
[0106] For fragmentation Each validator v in j Calculate the local model separately Compared to the previous global model θ t-1 The accuracy;
[0107] For a given trainer τ i and validator v j The verifier obtains the local model of the trainer. The accuracy of this trainer's model is evaluated as follows:
[0108]
[0109] in, Let n be the number of training samples in the public dataset, and x be the number of training samples in the public dataset. k and y k These represent data samples and labels, respectively. The coefficient of the regularization term;
[0110] The verifier simultaneously calculates the previous round's global model θ t-1 Model accuracy:
[0111]
[0112] if If the model is selected for aggregation, then this model will be used; otherwise, this model will not participate in global model aggregation.
[0113] Example 2:
[0114] Example 2 is a preferred embodiment of Example 1, and is used to illustrate the present invention in more detail.
[0115] The present invention also provides a trusted federated learning system based on blockchain and zero-knowledge proof. The trusted federated learning system based on blockchain and zero-knowledge proof can be implemented by executing the process steps of the trusted federated learning method based on blockchain and zero-knowledge proof. That is, those skilled in the art can understand the trusted federated learning method based on blockchain and zero-knowledge proof as a preferred implementation of the trusted federated learning system based on blockchain and zero-knowledge proof.
[0116] According to the present invention, a trusted federated learning system based on blockchain and zero-knowledge proofs is provided, such as... Figures 1-3 As shown, it includes:
[0117] The perturbation module includes a differential privacy submodule and a zero-knowledge proof submodule, which add noise to the local model and generate zero-knowledge proofs at the same time.
[0118] A non-interactive zero-knowledge protocol is applied to enable the verification of noise added to the local model. The trainer and verifier independently and separately construct the same circuit containing specific Gaussian noise using linear approximation. The trainer uses the circuit, a common generator, and model parameters to construct a proof. The verifier uses the circuit to verify the received proof and compares the verification output field with the noisy model. If the comparison result is correct, the proof is accepted.
[0119] Specifically, in the disturbance module:
[0120] After initializing the system parameters, a common generator is first built based on the compressed Σ protocol and shared among all trainers and verifiers. Trainers and verifiers then construct consistent circuits using linear estimation based on specific Gaussian noise. During local training, trainers use the circuits, the common generator, and the model parameters to construct proofs. After receiving the proofs, verifiers use the generated circuits to verify the legality of the perturbation model.
[0121] The process of adding noise to the model satisfies (∈,δ) differential privacy, where ∈ and δ are differential privacy parameters, both of which are between 0 and 1.
[0122] Specifically, in the differential privacy submodule:
[0123] The noise follows a mean of 0 and a variance of . The distribution is Gaussian, and the variance is... satisfy:
[0124]
[0125] Where c1, c2, and G are constants, δ, ∈ are differential privacy parameters, T is the number of training rounds, q is the number of trainers, and n is the number of training samples in the public dataset;
[0126] In the zero-knowledge proof submodule:
[0127] The trainer completes local training, repeats the noise calculation steps a total of d times for the model dimension, adds noise to all parameters of the local model, and calculates zero-knowledge proofs by using the compressed Σ-protocol, through the generation circuit and the common generator.
[0128] The objective function F(θ) used in local training is differentiable with respect to the model parameters θ and has the following properties: Strong convexity, G-Lipshitz and L-smoothness properties mean that when the learning rate η = 1 / L, the objective function will expect the error to satisfy:
[0129]
[0130] Where n is the number of training samples, F * Let T be the optimal objective function, T be the number of training rounds, c be a constant, and E be the expected value of the expression within the square brackets.
[0131] Verification module: Includes local verification and sharding consensus process;
[0132] Validators are uniformly and randomly formed into a set containing multiple shards. In each validation cycle, validators in shards that meet specific criteria use the public dataset in the system to validate the quality of local models in parallel. One validator in a shard proposes a voting block and broadcasts it to other members of the shard. The block content is the selection result of the local model. Validators in the shard submit votes to reach a conclusion, and the process continues until a consensus is reached. Validators select a portion of the local models for global aggregation.
[0133] Specifically, in the verification module:
[0134] The verification module employs a high-performance sharded blockchain consensus algorithm based on voting, and includes the following steps:
[0135] The verification module evaluates the quality of local models and selects a subset of models for aggregation based on their quality to defend against Byzantine attacks from malicious trainers. During the local verification phase, verifiers calculate the accuracy of both the local model and the previous round's global model, selecting the local model with higher accuracy for aggregation. In the consensus phase, all verifiers in each shard send their generated voting digests to each other. A verifier is randomly selected to package the correct voting results into a block. All verifiers reach a consensus on this block. If the consensus passes, the local model used for aggregation is determined; otherwise, a new round of consensus is initiated.
[0136] The local model used for aggregation first reaches consensus in a shard. Consensus ends when more than a certain proportion of validators submit consistent votes; otherwise, the consensus process is repeated in the next shard. This proportion is denoted as S. l ; The calculation formula is as follows:
[0137]
[0138] Where R = α / V represents the maximum proportion of malicious validators that this module can tolerate, α is the number of malicious nodes among the validators, V is the number of validators, z is a variable for setting system security parameters, j is the index of the blockchain shard in the system, r is the index of the first round of sharding, l is the consensus round number, and h... j Let j be the j-th blockchain shard.
[0139] Specifically, in the verification module:
[0140] For fragmentation Each validator v in j Calculate the local model separately Compared to the previous global model θ t-1 The accuracy;
[0141] For a given trainer τ i and validator v j The verifier obtains the local model of the trainer. The accuracy of this trainer's model is evaluated as follows:
[0142]
[0143] in, Let n be the number of training samples in the public dataset, and x be the number of training samples in the public dataset. k and y k These represent data samples and labels, respectively. The coefficient of the regularization term;
[0144] The verifier simultaneously calculates the previous round's global model θ t-1 Model accuracy:
[0145]
[0146] if If the model is selected for aggregation, then this model will be used; otherwise, this model will not participate in global model aggregation.
[0147] Example 3:
[0148] Example 3 is a preferred example of Example 1, and is used to illustrate the present invention in more detail.
[0149] The trusted federated learning system and method based on blockchain and zero-knowledge proof proposed in this invention consists of multiple rounds of local training processes. Each round of local training includes the following:
[0150] - A perturbation module, comprising differential privacy and zero-knowledge proof submodules, adds noise to the local model to protect privacy while generating zero-knowledge proofs to ensure the credibility of the model parameters; the perturbation module applies the NIZK protocol to make the perturbations verifiable. To achieve this, a public generator is first created based on a compressed Σ-protocol and shared between the trainer and verifier. The trainer and verifier independently and separately construct the same circuit containing specific Gaussian noise using a linear approximation. The trainer then constructs a proof using the circuit, the public generator, and the model parameters. The verifier then uses their circuit to verify the received proof and compares the verification output field with the noisy model. If the comparison is correct, they accept the proof.
[0151] - A verification module, comprising local verification and shard consensus processes, ensures the reliability of the model aggregation process while saving computational and communication overhead and balancing system operating efficiency; in the verification module, verifiers randomly and uniformly form a set containing H shards. In each validation cycle, validators in the shards that meet the specified criteria utilize the public dataset. The quality of local models is verified in parallel. Afterwards, one validator in the shard (i.e., the leader) proposes a voting procedure (also called a block) and broadcasts it to the other members of the shard. This block decides whether the verified local model should be selected. Validators in the shard then submit their votes, with a result of 1 ("agree") or 0 ("disagree"). If they cannot reach a consensus, a set is involved. In another shard, validators validate local models and vote to reach a conclusion. This process continues until a consensus is reached. Finally, the validators select q local models for global aggregation.
[0152] Furthermore, in the perturbation module, a differential privacy design is employed for stochastic additive gradient perturbation. The added noise follows a mean of 0 and a variance of... Gaussian noise, variance satisfy Where c1, c2, and G are constants, δ, ∈ are differential privacy parameters, T is the number of training rounds, and n min Let n be the minimum number of training samples for q trainers, and n be the number of training samples in the public dataset.
[0153] Furthermore, for the local training process with the aforementioned noise added, it can be proven that when the learning rate η = 1 / L, the objective function will satisfy the expected error. Where n is the number of training samples in the public dataset, F * Let T be the optimal objective function, T be the number of training rounds, and c be a constant.
[0154] Furthermore, the required number of training rounds can be derived from the above training process. Where c3 is a constant.
[0155] Furthermore, in the verification module, the local model used for aggregation first reaches consensus in one shard. Consensus ends when more than a certain proportion of validators submit consistent votes; otherwise, the process repeats in the next shard. This proportion is denoted as S. l .
[0156] Furthermore, the validators compare the accuracy of the global model and the local models from the previous round, and select the local models with higher accuracy than the previous global model for aggregation. The consensus for accuracy calculation is as follows: Where θ are the model parameters. is the coefficient of the regularization term.
[0157] The following is a brief description of the implementation steps of the present invention with reference to the flowchart:
[0158] The implementation steps of the disturbance module are as follows: Figure 1 As shown:
[0159] Step S1: System Initialization. This step specifies the federated learning task, model parameters, public dataset, system security parameters, cryptographic algorithms, node information, etc.
[0160] Step S2: Construct a common generator based on the compressed Σ-protocol and share it among all validators and trainers. The trainer and validator independently construct a circuit with a given Gaussian noise using linear estimation, and this circuit is consistent.
[0161] Step S3: The trainer obtains the global model θ from the previous round. t-1The update gradient of the local model is calculated, and noise is added to it. For each parameter in the local model, noise needs to be calculated individually. The specific steps for calculating the noise are as follows:
[0162] S3.1. Given the trainer τ j and its local dataset Solving gradients using the backpropagation algorithm
[0163] S3.2. Randomly and uniformly sample a random number r in the interval [0, Q). j Generate the corresponding commitment C j and r j and C j public.
[0164] S3.3. Summation using the following formula yields the result. The extension yields a K-dimensional random number vector r. j =(r j,1 ,…,r j,K )∈[0,Q) K And uniformly and randomly select a K-dimensional vector a j =(a j,1 ,…,a j,K )∈[0,Q) K .
[0165] S3.4. For positive integers calculate Then, the Gaussian distribution value I is obtained according to the following formula. j :
[0166]
[0167] Then output g for each model parameter. j =f j +I j .
[0168] Step S4: The trainer completes local training, repeating the noise calculation step a total of d times across the model dimensions, adding noise to all parameters of the local model. Then, using the compressed Σ-protocol, zero-knowledge proofs are calculated through the generator circuit and a common generator.
[0169] Step S5: The verifier uses the circuit they have constructed to verify the received proof and compares the circuit output results. If the comparison results are correct, the proof is accepted.
[0170] The verification module implementation steps are as follows: Figure 2 As shown:
[0171] Step S1: All validators use the public dataset Calculate the global model θ t-1 The model accuracy is expressed as
[0172]
[0173] Step S2: Local verification. For shards Each validator v in j Using the formula Verify model quality. If... Then select the corresponding update gradient G. i (θ t-1 If the model is aggregated, it will be used for aggregation; otherwise, the model will not participate in global model aggregation.
[0174] Step S3: Global consensus.
[0175] S3.1. Consensus Preparation. Each shard Select a leader. (For sharding) Each validator v in j From the fragment h s All validators v k Receive voting summary U k,s And recorded in the local selection summary, where If less than S l-1 If the proportion of validators agree, then this selection result is synchronized to the latest state of the block.
[0176] S3.2. Consensus Implementation. For sharding... Each validator v in j Send his voting results to Collect all validators and record the voting results of other validators to obtain a voting summary.
[0177] S3.3. Consensus Verification. If the percentage of unanimous votes in the voting digest does not exceed... If the next shard is introduced, the process returns to the consensus preparation phase. Otherwise, the validator sends its vote digest to the leader of this shard. Preferably, The calculation formula is as follows:
[0178]
[0179] Where R = α / V represents the proportion of malicious nodes in the validator set, that is, the maximum proportion of malicious validator nodes that this module can tolerate.
[0180] S3.4. Consensus Confirmation. Regarding sharding... The leader in the network checks the correctness of the validators' vote digests and then proposes a block containing the correct vote digest. Subsequently, using the Byzantine Fault Tolerance algorithm, all validators vote on this block. If the vote passes, the local model selected in this round of model aggregation is determined and broadcast to the network; otherwise, sharding is introduced. And return to the consensus preparation phase.
[0181] Step S4: The verification module verifies each local model, repeating steps S2 and S3 until the local model used for aggregation is determined. Then, the updated global model θ is calculated according to the following formula. t :
[0182]
[0183] in, Let be the set of trainers selected for model aggregation in the t-th round of training.
[0184] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.
[0185] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.
Claims
1. A trusted federated learning system based on blockchain and zero-knowledge proofs, characterized in that, include: The perturbation module includes a differential privacy submodule and a zero-knowledge proof submodule, which add noise to the local model and generate zero-knowledge proofs at the same time. By applying a non-interactive zero-knowledge protocol, noise added to the local model can be verified. The trainer and verifier independently and separately construct the same circuit containing specific Gaussian noise using linear approximation. The trainer uses the circuit, a common generator, and model parameters to construct a proof. The verifier uses the circuit to verify the received proof and compares the verification output field with the noisy model. If the comparison is correct, the proof is accepted. Verification module: Includes local verification and sharding consensus process; The validators are uniformly and randomly formed into a set containing multiple shards. In each validation cycle, the validators in the shards that meet specific criteria use the public dataset in the system to validate the quality of the local model in parallel. One validator in a shard proposes a voting procedure block and broadcasts it to the other members of the shard. The block content represents the selection result of the local model; validators in the shard submit votes to reach a conclusion, and the process continues until a consensus is reached; validators select a portion of the local model for global aggregation.
2. The trusted federated learning system based on blockchain and zero-knowledge proofs according to claim 1, characterized in that, In the disturbance module: After initializing the system parameters, a common generator is first built based on the compressed Σ protocol and shared among all trainers and validators. The trainers and validators build consistent circuits based on specific Gaussian noise using linear estimation. During local training, the trainer constructs a proof using circuits, a common generator, and model parameters; after receiving the proof, the verifier uses the generated circuit to verify the legality of the perturbation model. The process of adding noise to the model satisfies (∈,δ) differential privacy, where ∈ and δ are differential privacy parameters, both of which are between 0 and 1.
3. The trusted federated learning system based on blockchain and zero-knowledge proofs according to claim 1, characterized in that: In the differential privacy submodule: The noise follows a mean of 0 and a variance of . The distribution is Gaussian, and the variance is... satisfy: Where c1, c2, and G are constants, δ, ∈ are differential privacy parameters, T is the number of training rounds, q is the number of trainers, and n is the number of training samples in the public dataset; In the zero-knowledge proof submodule: The trainer completes local training, repeats the noise calculation steps a total of d times for the model dimension, adds noise to all parameters of the local model, and calculates zero-knowledge proofs by using the compressed Σ-protocol, through the generation circuit and the common generator. The objective function F(θ) used in local training is differentiable with respect to the model parameters θ and has the following properties: Given the strong convexity, g-Lipschitz, and L-smoothness properties, when the learning rate η = 1 / L, the objective function will expect the error to satisfy: Where n is the number of training samples, F * Let T be the optimal objective function, T be the number of training rounds, c be a constant, and E be the expected value of the expression within the square brackets.
4. The trusted federated learning system based on blockchain and zero-knowledge proofs according to claim 1, characterized in that, In the verification module: The verification module employs a high-performance sharded blockchain consensus algorithm based on voting, and includes the following steps: The verification module evaluates the quality of local models and selects a subset of models for aggregation based on their quality to defend against Byzantine attacks from malicious trainers. During the local verification phase, verifiers calculate the accuracy of both the local model and the previous round's global model, selecting the local model with higher accuracy for aggregation. In the consensus phase, all verifiers in each shard send their generated voting digests to each other. A verifier is randomly selected to package the correct voting results into a block. All verifiers reach a consensus on this block. If the consensus passes, the local model used for aggregation is determined; otherwise, a new round of consensus is initiated. The local model used for aggregation first reaches consensus in a shard. Consensus ends when more than a certain proportion of validators submit consistent votes; otherwise, the consensus process is repeated in the next shard. This proportion is denoted as S. l ; The calculation formula is as follows: Where R = α / V represents the maximum proportion of malicious validators that this module can tolerate, α is the number of malicious nodes among the validators, V is the number of validators, z is a variable for setting system security parameters, j is the index of the blockchain shard in the system, r is the index of the first round of sharding, l is the consensus round number, and h... j Let j be the j-th blockchain shard.
5. The trusted federated learning system based on blockchain and zero-knowledge proofs according to claim 1, characterized in that, In the verification module: For fragmentation Each validator v in j Calculate the local model separately Compared to the previous global model θ t-1 The accuracy; For a given trainer τ i and validator v j The verifier obtains the local model of the trainer. The accuracy of this trainer's model is evaluated as follows: in, Let n be the number of training samples in the public dataset, and x be the number of training samples in the public dataset. k and y k These represent data samples and labels, respectively. The coefficient of the regularization term; The verifier simultaneously calculates the previous round's global model θ t-1 Model accuracy: if If the model is selected for aggregation, then this model will be used; otherwise, this model will not participate in global model aggregation.
6. A trusted federated learning method based on blockchain and zero-knowledge proofs, characterized in that, include: Step S1: The perturbation module applies a non-interactive zero-knowledge protocol to enable the verification of noise added to the local model; The trainer and verifier independently and separately construct the same circuit containing specific Gaussian noise using linear approximation. The trainer uses the circuit, a common generator, and model parameters to construct a proof. The verifier uses the circuit to verify the received proof and compares the verification output field with the noisy model. If the comparison is correct, the proof is accepted. It includes a differential privacy submodule and a zero-knowledge proof submodule, which add noise to the local model while generating zero-knowledge proofs. Step S2: The verification module causes the verifiers to form a set containing multiple shards in a uniform and random manner. In each verification cycle, the verifiers in the shards that meet specific criteria use the public dataset in the system to verify the quality of the local model in parallel. One validator in a shard proposes a voting procedure block and broadcasts it to the other members of the shard. The block content represents the selection result of the local model; validators in the shard submit votes to reach a conclusion, and the process continues until a consensus is reached; validators select a portion of the local model for global aggregation; it includes the local verification and shard consensus processes.
7. The trusted federated learning method based on blockchain and zero-knowledge proofs according to claim 6, characterized in that: In the disturbance module: After initializing the system parameters, a common generator is first built based on the compressed Σ protocol and shared among all trainers and validators. The trainers and validators build consistent circuits based on specific Gaussian noise using linear estimation. During local training, the trainer constructs a proof using circuits, a common generator, and model parameters; after receiving the proof, the verifier uses the generated circuit to verify the legality of the perturbation model. The process of adding noise to the model satisfies (∈,δ) differential privacy, where ∈ and δ are differential privacy parameters, both of which are between 0 and 1.
8. The trusted federated learning method based on blockchain and zero-knowledge proofs according to claim 6, characterized in that: In the differential privacy submodule: The noise follows a mean of 0 and a variance of . The distribution is Gaussian, and the variance is... satisfy: Where c1, c2, and G are constants, δ, ∈ are differential privacy parameters, T is the number of training rounds, q is the number of trainers, and n is the number of training samples in the public dataset; In the zero-knowledge proof submodule: The trainer completes local training, repeats the noise calculation steps a total of d times for the model dimension, adds noise to all parameters of the local model, and calculates zero-knowledge proofs by using the compressed Σ-protocol, through the generation circuit and the common generator. The objective function F(θ) used in local training is differentiable with respect to the model parameters θ and has the following properties: Strong convexity, G-Lipshitz and L-smoothness properties mean that when the learning rate η = 1 / L, the objective function will expect the error to satisfy: Where n is the number of training samples, F * Let T be the optimal objective function, T be the number of training rounds, c be a constant, and E be the expected value of the expression within the square brackets.
9. The trusted federated learning method based on blockchain and zero-knowledge proofs according to claim 6, characterized in that: In the verification module: The verification module employs a high-performance sharded blockchain consensus algorithm based on voting, and includes the following steps: The verification module evaluates the quality of local models and selects a subset of models for aggregation based on their quality to defend against Byzantine attacks from malicious trainers. During the local verification phase, verifiers calculate the accuracy of both the local model and the previous round's global model, selecting the local model with higher accuracy for aggregation. In the consensus phase, all verifiers in each shard send their generated voting digests to each other. A verifier is randomly selected to package the correct voting results into a block. All verifiers reach a consensus on this block. If the consensus passes, the local model used for aggregation is determined; otherwise, a new round of consensus is initiated. The local model used for aggregation first reaches consensus in a shard. Consensus ends when more than a certain proportion of validators submit consistent votes; otherwise, the consensus process is repeated in the next shard. This proportion is denoted as S. l ; The calculation formula is as follows: Where R = α / V represents the maximum proportion of malicious validators that this module can tolerate, α is the number of malicious nodes among the validators, V is the number of validators, z is a variable for setting system security parameters, j is the index of the blockchain shard in the system, r is the index of the first round of sharding, l is the consensus round number, and h... j Let j be the j-th blockchain shard.
10. The trusted federated learning method based on blockchain and zero-knowledge proofs according to claim 6, characterized in that: In the verification module: For fragmentation Each validator v in j Calculate the local model separately Compared to the previous global model θ t-1 The accuracy; For a given trainer τ i and validator v j The verifier obtains the local model of the trainer. The accuracy of this trainer's model is evaluated as follows: in, Let n be the number of training samples in the public dataset, and x be the number of training samples in the public dataset. k and y k These represent data samples and labels, respectively. The coefficient of the regularization term; The verifier simultaneously calculates the previous round's global model θ t-1 Model accuracy: if If the model is selected for aggregation, then this model will be used; otherwise, this model will not participate in global model aggregation.