Credible federal learning system and method based on block chain and zero knowledge proof

Abstract

The invention provides a credible federal learning system and method based on a block chain and zero knowledge proof. The credible federal learning system comprises a disturbance module. In the disturbance module, a non-interactive zero knowledge protocol is applied, so that noise added to a local model can be verified. A trainer and a verifier construct the same circuit containing specific Gaussian noise independently and respectively using linear approximation, and the trainer constructs proof using the circuit and a common generator as well as model parameters. The verifier verifies the received attestation using their circuitry; and a verification module. And the verification module constructs a fragment block chain consensus process. And each verifier firstly verifies the precision of local model gradient updating by using the public data set. And forming a consensus in the network by taking the verification result as a voting abstract. For block data containing local model selection information, a Byzantine fault-tolerant protocol is introduced to ensure data credibility. The method can be proved to realize differential privacy, and can analyze the upper bound of the influence of noise on the model performance.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to an algorithmic solution for problems such as security and trustworthiness, privacy protection, and Byzantine fault tolerance in federated learning, and more particularly, to a trusted federated learning system and method based on blockchain and zero-knowledge proof. Background Art

[0002] In recent years, with the advancement of computing hardware and communication technology, blockchain technology and federated learning technology have developed rapidly. Blockchain technology aims to build a trusted data interaction environment for multiple nodes in the network, and the nodes in the network jointly maintain consistent block data; while federated learning is to build machine learning models by multiple computing devices, which allows computing devices to train locally with private data and aggregate the trained local models into the final model. Due to the inherent distributed characteristics of both, building a secure and reliable federated learning system based on blockchain technology has received more and more attention.

[0003] In a typical blockchain-based federated learning system, machine learning model parameters are transmitted to multiple trainers. These trainers then use their local private data as training data to train and transmit local models to the verification nodes. In order to further protect the privacy and security of local data, trainers usually add noise to the model parameters before transmitting the local model. After receiving the local model, the verification node performs a local verification process to evaluate the quality of the local model, and with the help of the blockchain consensus mechanism, selects some local models to aggregate and obtain the global model.

[0004] Although blockchain and differential privacy technologies have improved the security of federated learning, they have also introduced new challenges. First, the mechanism for adding noise to local models needs to be carefully designed. On the one hand, adding noise will introduce performance loss to the global model to a certain extent. In order to comprehensively consider privacy protection and training accuracy, the noise level needs to be carefully controlled when designing the solution, and the impact of noise on training accuracy needs to be limited in theory. On the other hand, in a federated learning system that introduces differential privacy, the accuracy of its model depends on the ability of the training nodes to honestly add qualified noise. Therefore, it is necessary to design a feasible solution to supervise the local training process.

[0005] For the above challenges, a feasible approach is to apply zero-knowledge proof (ZKP) to verify the local training process. However, as for the Gaussian noise often added in FL, there is no work that can ensure its verifiability. In addition, applying ZKP usually involves converting the designed algorithm into a linear arithmetic circuit, but generating Gaussian noise involves a large number of nonlinear functions, which increases the difficulty of the verification process.

[0006] The second challenge comes from the complexity of the verification process. In a blockchain-based federated learning system, in addition to adding illegal noise to the local model, malicious nodes can also use various attack methods to destroy the performance of the learning model, including random gradient attacks, injection techniques, etc. In addition, existing methods require all validators to participate in each model verification, which leads to a significant increase in the additional computing and communication overhead generated in the verification process as the scale of model parameters and the number of network nodes increase. A feasible solution is to use sharding technology to improve verification efficiency.

[0007] Considering the above challenges, there is no effective solution to design an efficient, secure, and reliable federated learning system. Therefore, it is of great practical significance to design a secure, reliable, and efficient federated learning system based on zero-knowledge proof and sharded blockchain technology. A solution is needed to solve the problems of privacy leakage, inefficiency, lack of reliable verification, etc. in federated learning.

[0008] At present, no description or report of similar technology to the present invention has been found, and similar information at home and abroad has not been collected yet. Summary of the invention

[0009] In view of the defects in the prior art, the purpose of the present invention is to provide a trusted federated learning system and method based on blockchain and zero-knowledge proof.

[0010] A trusted federated learning system based on blockchain and zero-knowledge proof provided by the present invention includes:

[0011] Perturbation module: includes differential privacy submodule and zero-knowledge proof submodule, adds noise to the local model and generates zero-knowledge proof;

[0012] A non-interactive zero-knowledge protocol is applied to enable the noise added to the local model to be verified. The trainer and the verifier independently and separately use linear approximation to construct the same circuit containing specific Gaussian noise. The trainer uses the circuit and the public generator as well as the model parameters to construct the proof. The verifier uses the circuit to verify the received proof and compares the verification output field with the noise model. If the comparison result is correct, the proof is accepted.

[0013] Verification module: includes local verification and sharding consensus process;

[0014] Validators uniformly and randomly form a set containing multiple shards. In each verification cycle, validators in the shards that meet specific criteria use the public data set in the system to verify the quality of local models in parallel; a validator in the shard proposes a voting program block and broadcasts it to other members of the shard; the block content is the selection result of the local model; the validators in the shard submit votes to reach a conclusion, and the process continues until a consensus decision is reached; the validator selects some local models for global aggregation.

[0015] Preferably, in the disturbance module:

[0016] After initializing the system parameters, a public generator is first constructed based on the compressed Σ protocol and shared among all trainers and verifiers. The trainers and verifiers respectively construct consistent circuits based on specific Gaussian noise using linear estimation. During local training, the trainer constructs a proof using the circuit, public generator, and model parameters. After receiving the proof, the verifier uses the generated circuit to verify the legitimacy of the perturbation model.

[0017] The process of adding noise to the model satisfies (∈,δ) differential privacy, where ∈ and δ are differential privacy parameters, and their values ​​are both between 0 and 1.

[0018] Preferably, in the differential privacy submodule:

[0019] The noise has a mean of 0 and a variance of The Gaussian distribution with variance satisfy:

[0020]

[0021] Where c1, c2, G are all constants, δ,∈ is the differential privacy parameter, T is the number of training rounds, q is the number of trainers, and n is the number of training samples in the public dataset;

[0022] In the zero-knowledge proof submodule:

[0023] The trainer completes local training, repeats the noise calculation steps for a total of d model dimensions, adds noise to all parameters of the local model, and uses the compressed Σ-protocol to calculate the zero-knowledge proof through the generation circuit and the public generator;

[0024] The objective function F(θ) used in local training is differentiable with respect to the model parameter θ and has Strong convexity, G Lipschitz and L smoothness properties, when the learning rate η = 1 / L, the objective function will expect the error to satisfy:

[0025]

[0026] Among them, n is the number of training samples, F* is the optimal objective function, T is the number of training rounds, c is a constant, and E represents the mathematical expectation of the expression in the square brackets.

[0027] Preferably, in the verification module:

[0028] The verification module uses a high-performance sharding blockchain consensus algorithm based on voting, which includes the following steps:

[0029] The verification module evaluates the quality of local models and selects some models for aggregation based on the model quality to defend against Byzantine attacks from malicious trainers. In the local verification phase, the verifier calculates the accuracy of the local model and the global model of the previous round respectively, and selects the local model with better accuracy than the global model for aggregation. In the consensus phase, all validators in each shard send their generated voting summaries to each other, randomly select a validator, and package the correct voting results into a block. All validators reach a consensus on this block. If the consensus is passed, the local model for aggregation is determined; otherwise, the next round of consensus is repeated.

[0030] The local model for aggregation first reaches consensus in one shard. When more than a certain proportion of validators submit consistent votes, the consensus ends. Otherwise, the next shard is introduced to repeat the consensus. The proportion is represented by S. l ; The calculation formula is as follows:

[0031]

[0032] Among them, R = α / V represents the maximum ratio of malicious verification nodes that this module can tolerate, α is the number of malicious nodes among the verifiers, V is the number of verifiers, z is the variable for setting system security parameters, j is the subscript of the blockchain shard in the system, r is the subscript of the first round of shards, l is the number of consensus rounds, h is the number of rounds of consensus, and j is the jth blockchain shard.

[0033] Preferably, in the verification module:

[0034] For sharding Each validator v in j , respectively calculate the local model and the previous global model θ t-1 The accuracy of

[0035] For a given trainer τ i and the validator v j , the verifier obtains the local model of the trainer The model accuracy of this trainer is evaluated as:

[0036]

[0037] in, is a public dataset, n is the number of training samples in the public dataset, x k and k represent data samples and labels respectively, is the regularization term coefficient;

[0038] The verifier simultaneously calculates the global model θ of the previous round t-1 Model accuracy:

[0039]

[0040] if If the model is not included in the global model aggregation, then this model is selected for aggregation; otherwise, the model does not participate in the global model aggregation.

[0041] A trusted federated learning method based on blockchain and zero-knowledge proof provided by the present invention includes:

[0042] Step S1: The perturbation module applies a non-interactive zero-knowledge protocol so that the noise added to the local model can be verified; the trainer and the verifier independently and separately use linear approximation to construct the same circuit containing specific Gaussian noise. The trainer uses the circuit and the public generator as well as the model parameters to build the proof. The verifier uses the circuit to verify the received proof and compares the verification output field with the noise model. If the comparison result is correct, the proof is accepted. It contains a differential privacy submodule and a zero-knowledge proof submodule to add noise to the local model and generate a zero-knowledge proof at the same time.

[0043] Step S2: The verification module allows the validators to uniformly and randomly form a set containing multiple shards. In each verification cycle, the validators in the shards that meet the specific criteria use the public data set in the system to verify the quality of the local model in parallel; a validator in the shard proposes a voting program block and broadcasts it to other members of the shard; the block content is the selection result of the local model; the validators in the shard submit votes to reach a conclusion, and the process continues until a consensus decision is reached; the validators select some local models for global aggregation; including local verification and shard consensus process;

[0044] Preferably, in the disturbance module:

[0045] After initializing the system parameters, a public generator is first constructed based on the compressed Σ protocol and shared among all trainers and verifiers. The trainers and verifiers respectively construct consistent circuits based on specific Gaussian noise using linear estimation. During local training, the trainer constructs a proof using the circuit, public generator, and model parameters. After receiving the proof, the verifier uses the generated circuit to verify the legitimacy of the perturbation model.

[0046] The process of adding noise to the model satisfies (∈,δ) differential privacy, where ∈ and δ are differential privacy parameters, and their values ​​are both between 0 and 1.

[0047] Preferably, in the differential privacy submodule:

[0048] The noise has a mean of 0 and a variance of The Gaussian distribution with variance satisfy:

[0049]

[0050] Where c1, c2, G are all constants, δ,∈ is the differential privacy parameter, T is the number of training rounds, q is the number of trainers, and n is the number of training samples in the public dataset;

[0051] In the zero-knowledge proof submodule:

[0052] The trainer completes local training, repeats the noise calculation steps for a total of d model dimensions, adds noise to all parameters of the local model, and uses the compressed Σ-protocol to calculate the zero-knowledge proof through the generation circuit and the public generator;

[0053] The objective function F(θ) used in local training is differentiable with respect to the model parameter θ and has Strong convexity, G Lipschitz and L smoothness properties, when the learning rate η = 1 / L, the objective function will expect the error to satisfy:

[0054]

[0055] Among them, n is the number of training samples, F * is the optimal objective function, T is the number of training rounds, c is a constant, and E represents the mathematical expectation of the expression in the square brackets.

[0056] Preferably, in the verification module:

[0057] The verification module uses a high-performance sharding blockchain consensus algorithm based on voting, which includes the following steps:

[0058] The verification module evaluates the quality of local models and selects some models for aggregation based on the model quality to defend against Byzantine attacks from malicious trainers. In the local verification phase, the verifier calculates the accuracy of the local model and the global model of the previous round respectively, and selects the local model with better accuracy than the global model for aggregation. In the consensus phase, all validators in each shard send their generated voting summaries to each other, randomly select a validator, and package the correct voting results into a block. All validators reach a consensus on this block. If the consensus is passed, the local model for aggregation is determined; otherwise, the next round of consensus is repeated.

[0059] The local model for aggregation first reaches consensus in one shard. When more than a certain proportion of validators submit consistent votes, the consensus ends. Otherwise, the next shard is introduced to repeat the consensus. The proportion is represented by S. l ; The calculation formula is as follows:

[0060]

[0061] Among them, R = α / V represents the maximum ratio of malicious verification nodes that this module can tolerate, α is the number of malicious nodes among the verifiers, V is the number of verifiers, z is the variable for setting system security parameters, j is the subscript of the blockchain shard in the system, r is the subscript of the first round of shards, l is the number of consensus rounds, h is the number of rounds of consensus, and j is the jth blockchain shard.

[0062] Preferably, in the verification module:

[0063] For sharding Each validator v in j , respectively calculate the local model and the previous global model θ t-1 The accuracy of

[0064] For a given trainer τ i and the validator v j , the verifier obtains the local model of the trainer The model accuracy of this trainer is evaluated as:

[0065]

[0066] in, is a public dataset, n is the number of training samples in the public dataset, x k and k represent data samples and labels respectively, is the regularization term coefficient;

[0067] The verifier simultaneously calculates the global model θ of the previous round t-1 Model accuracy:

[0068]

[0069] if If the model is not included in the global model aggregation, then this model is selected for aggregation; otherwise, the model does not participate in the global model aggregation.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] 1. The present invention takes into account the security, model performance and training efficiency of federated learning, can resist various security threats, and constructs a new solution for problems such as security and trustworthiness, privacy protection and Byzantine fault tolerance in the federated learning system. It has strong practicality and a wide range of applications.

[0072] 2. The present invention can ensure that the trainer can verifiably extract additive noise from a predefined Gaussian distribution. At the same time, by integrating the iterative sharding consensus algorithm, it can also effectively resist Byzantine attacks from malicious validators during the model verification process.

[0073] 3. The present invention can significantly reduce the communication overhead in the blockchain network;

[0074] 4. The present invention designs a perturbation module based on non-interactive zero-knowledge proof technology, so that the process of adding noise to the local model is efficient and reliable. While ensuring the data privacy of the trainer, the trainer's local training process can also be supervised;

[0075] 5. The present invention designs a differential privacy federated learning noise adding mechanism, which enables the trainer to add noise to the gradient according to the specified distribution. The algorithm can be proven to achieve differential privacy and can analyze the upper bound of the impact of noise on model performance;

[0076] 6. This invention introduces a model verification process, which selects model updates by scoring the trainer's model, thereby aggregating into a new global model. This mechanism enables the federated learning system to resist Byzantine attacks and improves the robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0078] Figure 1 It is the flow chart of the incentive module;

[0079] Figure 2 This is the flow chart of the training module;

[0080] Figure 3 Schematic diagram of the system of the present invention. DETAILED DESCRIPTION

[0081] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0082] Embodiment 1:

[0083] The present invention provides a trusted federated learning system and method based on blockchain and zero-knowledge proof, comprising the following modules: (1) a perturbation module. In the perturbation module, a non-interactive zero-knowledge (NIZK) protocol is applied to make the noise added to the local model verifiable. The trainer and the verifier independently and separately use linear approximation to construct the same circuit containing specific Gaussian noise, and then the trainer uses the circuit and a public generator and model parameters to construct a proof. Finally, the verifier uses their circuit to verify the received proof; (2) a verification module. The verification module constructs a sharded blockchain consensus process. Each verifier first uses a public data set to verify the accuracy of the local model gradient update. Then, its own verification result is used as a voting summary to form a consensus in the network. For block data containing local model selection information, a Byzantine fault-tolerant protocol is introduced to ensure data credibility. The verification module enables the federated learning system to resist Byzantine attacks and improves the robustness of the system.

[0084] According to a trusted federated learning method based on blockchain and zero-knowledge proof provided by the present invention, Figure 3 ,include:

[0085] Step S1: The perturbation module applies a non-interactive zero-knowledge protocol so that the noise added to the local model can be verified; the trainer and the verifier independently and separately use linear approximation to construct the same circuit containing specific Gaussian noise. The trainer uses the circuit and the public generator as well as the model parameters to build the proof. The verifier uses the circuit to verify the received proof and compares the verification output field with the noise model. If the comparison result is correct, the proof is accepted. It contains a differential privacy submodule and a zero-knowledge proof submodule to add noise to the local model and generate a zero-knowledge proof at the same time.

[0086] Specifically, in the disturbance module:

[0087] After initializing the system parameters, a public generator is first constructed based on the compressed Σ protocol and shared among all trainers and verifiers. The trainers and verifiers respectively construct consistent circuits based on specific Gaussian noise using linear estimation. During local training, the trainer constructs a proof using the circuit, public generator, and model parameters. After receiving the proof, the verifier uses the generated circuit to verify the legitimacy of the perturbation model.

[0088] The process of adding noise to the model satisfies (∈,δ) differential privacy, where ∈ and δ are differential privacy parameters, and their values ​​are both between 0 and 1.

[0089] Specifically, in the differential privacy submodule:

[0090] The noise has a mean of 0 and a variance of The Gaussian distribution with variance satisfy:

[0091]

[0092] Where c1, c2, G are all constants, δ,∈ is the differential privacy parameter, T is the number of training rounds, q is the number of trainers, and n is the number of training samples in the public dataset;

[0093] In the zero-knowledge proof submodule:

[0094] The trainer completes local training, repeats the noise calculation steps for a total of d model dimensions, adds noise to all parameters of the local model, and uses the compressed Σ-protocol to calculate the zero-knowledge proof through the generation circuit and the public generator;

[0095] The objective function F(θ) used in local training is differentiable with respect to the model parameter θ and has Strong convexity, G Lipschitz and L smoothness properties, when the learning rate η = 1 / L, the objective function will expect the error to satisfy:

[0096]

[0097] Among them, n is the number of training samples, F * is the optimal objective function, T is the number of training rounds, c is a constant, and E represents the mathematical expectation of the expression in the square brackets.

[0098] Step S2: The verification module allows the validators to uniformly and randomly form a set containing multiple shards. In each verification cycle, the validators in the shards that meet the specific criteria use the public data set in the system to verify the quality of the local model in parallel; a validator in the shard proposes a voting program block and broadcasts it to other members of the shard; the block content is the selection result of the local model; the validators in the shard submit votes to reach a conclusion, and the process continues until a consensus decision is reached; the validators select some local models for global aggregation; including local verification and shard consensus process;

[0099] Specifically, in the verification module:

[0100] The verification module uses a high-performance sharding blockchain consensus algorithm based on voting, which includes the following steps:

[0101] The verification module evaluates the quality of local models and selects some models for aggregation based on the model quality to defend against Byzantine attacks from malicious trainers. In the local verification phase, the verifier calculates the accuracy of the local model and the global model of the previous round respectively, and selects the local model with better accuracy than the global model for aggregation. In the consensus phase, all validators in each shard send their generated voting summaries to each other, randomly select a validator, and package the correct voting results into a block. All validators reach a consensus on this block. If the consensus is passed, the local model for aggregation is determined; otherwise, the next round of consensus is repeated.

[0102] The local model for aggregation first reaches consensus in one shard. When more than a certain proportion of validators submit consistent votes, the consensus ends. Otherwise, the next shard is introduced to repeat the consensus. The proportion is denoted as S. l ; The calculation formula is as follows:

[0103]

[0104] Where T = α / V represents the maximum ratio of malicious verification nodes that this module can tolerate, α is the number of malicious nodes among the verifiers, V is the number of verifiers, z is the variable for setting system security parameters, j is the subscript of the blockchain shard in the system, r is the subscript of the first round of shards, l is the number of consensus rounds, and h is the number of rounds of consensus. j is the jth blockchain shard.

[0105] Specifically, in the verification module:

[0106] For sharding Each validator v in j , respectively calculate the local model and the previous global model θ t-1 The accuracy of

[0107] For a given trainer τ i and the validator v j , the verifier obtains the local model of the trainer The model accuracy of this trainer is evaluated as:

[0108]

[0109] in, is a public dataset, n is the number of training samples in the public dataset, x k and k represent data samples and labels respectively, is the regularization term coefficient;

[0110] The verifier simultaneously calculates the global model θ of the previous round t-1 Model accuracy:

[0111]

[0112] if If the model is not included in the global model aggregation, then this model is selected for aggregation; otherwise, the model does not participate in the global model aggregation.

[0113] Embodiment 2:

[0114] Embodiment 2 is a preferred example of Embodiment 1, and is used to illustrate the present invention in more detail.

[0115] The present invention also provides a trusted federated learning system based on blockchain and zero-knowledge proof. The trusted federated learning system based on blockchain and zero-knowledge proof can be implemented by executing the process steps of the trusted federated learning method based on blockchain and zero-knowledge proof, that is, those skilled in the art can understand the trusted federated learning method based on blockchain and zero-knowledge proof as a preferred implementation of the trusted federated learning system based on blockchain and zero-knowledge proof.

[0116] According to a trusted federated learning system based on blockchain and zero-knowledge proof provided by the present invention, Figure 1-Figure 3 As shown, including:

[0117] Perturbation module: includes differential privacy submodule and zero-knowledge proof submodule, adds noise to the local model and generates zero-knowledge proof;

[0118] A non-interactive zero-knowledge protocol is applied to enable the noise added to the local model to be verified. The trainer and the verifier independently and separately use linear approximation to construct the same circuit containing specific Gaussian noise. The trainer uses the circuit and the public generator as well as the model parameters to construct the proof. The verifier uses the circuit to verify the received proof and compares the verification output field with the noise model. If the comparison result is correct, the proof is accepted.

[0119] Specifically, in the disturbance module:

[0120] After initializing the system parameters, a public generator is first constructed based on the compressed Σ protocol and shared among all trainers and verifiers. The trainers and verifiers respectively construct consistent circuits based on specific Gaussian noise using linear estimation. During local training, the trainer constructs a proof using the circuit, public generator, and model parameters. After receiving the proof, the verifier uses the generated circuit to verify the legitimacy of the perturbation model.

[0121] The process of adding noise to the model satisfies (∈,δ) differential privacy, where ∈ and δ are differential privacy parameters, and their values ​​are both between 0 and 1.

[0122] Specifically, in the differential privacy submodule:

[0123] The noise has a mean of 0 and a variance of The Gaussian distribution with variance satisfy:

[0124]

[0125] Where c1, c2, G are all constants, δ,∈ is the differential privacy parameter, T is the number of training rounds, q is the number of trainers, and n is the number of training samples in the public dataset;

[0126] In the zero-knowledge proof submodule:

[0127] The trainer completes local training, repeats the noise calculation steps for a total of d model dimensions, adds noise to all parameters of the local model, and uses the compressed Σ-protocol to calculate the zero-knowledge proof through the generation circuit and the public generator;

[0128] The objective function F(θ) used in local training is differentiable with respect to the model parameter θ and has Strong convexity, G Lipschitz and L smoothness properties, when the learning rate η = 1 / L, the objective function will expect the error to satisfy:

[0129]

[0130] Among them, n is the number of training samples, F * is the optimal objective function, T is the number of training rounds, c is a constant, and E represents the mathematical expectation of the expression in the square brackets.

[0131] Verification module: includes local verification and sharding consensus process;

[0132] Validators uniformly and randomly form a set containing multiple shards. In each verification cycle, validators in the shards that meet specific criteria use the public data set in the system to verify the quality of local models in parallel; a validator in the shard proposes a voting program block and broadcasts it to other members of the shard; the block content is the selection result of the local model; the validators in the shard submit votes to reach a conclusion, and the process continues until a consensus decision is reached; the validator selects some local models for global aggregation.

[0133] Specifically, in the verification module:

[0134] The verification module uses a high-performance sharding blockchain consensus algorithm based on voting, which includes the following steps:

[0135] The verification module evaluates the quality of local models and selects some models for aggregation based on the model quality to defend against Byzantine attacks from malicious trainers. In the local verification phase, the verifier calculates the accuracy of the local model and the global model of the previous round respectively, and selects the local model with better accuracy than the global model for aggregation. In the consensus phase, all validators in each shard send their generated voting summaries to each other, randomly select a validator, and package the correct voting results into a block. All validators reach a consensus on this block. If the consensus is passed, the local model for aggregation is determined; otherwise, the next round of consensus is repeated.

[0136] The local model for aggregation first reaches consensus in one shard. When more than a certain proportion of validators submit consistent votes, the consensus ends. Otherwise, the next shard is introduced to repeat the consensus. The proportion is represented by S. l ; The calculation formula is as follows:

[0137]

[0138] Among them, R = α / V represents the maximum ratio of malicious verification nodes that this module can tolerate, α is the number of malicious nodes among the verifiers, V is the number of verifiers, z is the variable for setting system security parameters, j is the subscript of the blockchain shard in the system, r is the subscript of the first round of shards, l is the number of consensus rounds, h is the number of rounds of consensus, and j is the jth blockchain shard.

[0139] Specifically, in the verification module:

[0140] For sharding Each validator v in j , respectively calculate the local model and the previous global model θ t-1 The accuracy of

[0141] For a given trainer τ i and the validator v j , the verifier obtains the local model of the trainer The model accuracy of this trainer is evaluated as:

[0142]

[0143] in, is a public dataset, n is the number of training samples in the public dataset, x k and k represent data samples and labels respectively, is the regularization term coefficient;

[0144] The verifier simultaneously calculates the global model θ of the previous round t-1 Model accuracy:

[0145]

[0146] if If the model is not included in the global model aggregation, then this model is selected for aggregation; otherwise, the model does not participate in the global model aggregation.

[0147] Embodiment 3:

[0148] Embodiment 3 is a preferred example of Embodiment 1, and is used to illustrate the present invention in more detail.

[0149] The trusted federated learning system and method based on blockchain and zero-knowledge proof proposed in the present invention, the federated learning process of the framework consists of multiple rounds of local training processes, each round of local training process includes the following contents:

[0150] -Perturbation module, which contains differential privacy and zero-knowledge proof submodules, adds noise to the local model to protect privacy, and generates zero-knowledge proofs to ensure that the model parameters are credible; the perturbation module applies the NIZK protocol to make the perturbation verifiable. To achieve this goal, a public generator is first created based on the compressed Σ-protocol and shared between the trainer and the verifier. The trainer and the verifier independently and separately use linear approximation to build the same circuit containing specific Gaussian noise. The trainer then builds a proof using the circuit and the public generator and the model parameters. The verifier then uses their circuit to verify the received proof and compares the verification output field with the noise model. If the comparison result is correct, they accept the proof.

[0151] - Verification module, which includes local verification and shard consensus process to ensure the credibility of the model aggregation process, while saving computing and communication overhead and taking into account the system operation efficiency; in the verification module, the verifier randomly and evenly forms a set of H shards In each validation cycle, validators in shards that meet the specified criteria use the public dataset Verify the quality of local models in parallel. Afterwards, one validator in the shard (the leader) proposes a voting procedure (also called a block) and broadcasts it to the other members of the shard. The block determines whether the verified local model should be selected. The validators in the shard then submit their votes, which are either 1 (for "agree") or 0 (for "disagree"). If they cannot reach a unanimous decision, the collective Another shard in , where the validators validate the local models and vote to reach a conclusion. This process continues until a consensus decision is reached. Finally, the validators select q local models for global aggregation.

[0152] Furthermore, in the perturbation module, a random additive gradient perturbation is designed using differential privacy. The added noise has a mean of 0 and a variance of Gaussian noise with variance satisfy Where c1, c2, G are all constants, δ,∈ is the differential privacy parameter, T is the number of training rounds, n min is the minimum number of training samples for q trainers, and n is the number of training samples in the public dataset.

[0153] Furthermore, for the local training process with the above noise added, it can be proved that when the learning rate η = 1 / L, the objective function will satisfy the expected error Among them, n is the number of training samples in the public dataset, F * is the optimal objective function, T is the number of training rounds, and c is a constant.

[0154] Furthermore, for the above training process, the number of training rounds required can be derived: Wherein, c3 is a constant.

[0155] Furthermore, in the verification module, the local model used for aggregation first reaches a consensus in one shard. When more than a certain proportion of validators submit consistent votes, the consensus ends. Otherwise, the next shard is introduced to repeat the consensus. The proportion is represented by S l .

[0156] Furthermore, the verifier compares the accuracy of the global model and the local model in the previous round, and selects the local model with better accuracy than the global model in the previous round for aggregation. The accuracy calculation consensus is Where θ is the model parameter, is the regularization coefficient.

[0157] The following is a brief description of the implementation steps of the present invention in conjunction with the flow chart:

[0158] The disturbance module implementation steps are as follows: Figure 1 As shown:

[0159] Step S1: System initialization. Specify the federated learning tasks, model parameters, public data sets, system security parameters, cryptographic algorithms, node information, etc.

[0160] Step S2: A common generator is constructed based on the compressed Σ-protocol and shared among all verifiers and trainers. Trainers and verifiers independently construct a circuit with a given Gaussian noise using linear estimation, and this circuit is consistent.

[0161] Step S3: The trainer obtains the global model θ of the previous round t-1, calculate the updated gradient of the local model and add noise to it. For each parameter in the local model, the noise needs to be calculated separately. The specific steps for calculating the noise are as follows:

[0162] S3.1. Given a trainer τ j and its local dataset Using the back-propagation algorithm to solve the gradient

[0163] S3.2. Randomly and uniformly sample a random number r in the interval [0, Q) j , generating the corresponding commitment C j , and r j and C j public.

[0164] S3.3. Use the following formula to sum up and get Expand to get K-dimensional random number vector r j =(r j,1 ,…,r j,K )∈[0,Q) K , and uniformly randomly extract a K-dimensional vector a j =(a j,1 ,…,a j,K )∈[0,Q) K .

[0165] S3.4. For positive integers calculate Then, the Gaussian distribution value I is obtained according to the following formula j :

[0166]

[0167] Then output g for each model parameter j =f j +I j .

[0168] Step S4: The trainer completes local training and repeats the noise calculation step for a total of d model dimensions to add noise to all parameters of the local model. Then, the zero-knowledge proof is calculated by using the compressed Σ-protocol, generating circuits and public generators.

[0169] Step S5: The verifier verifies the received proof using the circuit it constructed and compares the circuit output results. If the comparison result is correct, the proof is accepted.

[0170] The verification module implements the steps as follows: Figure 2 As shown:

[0171] Step S1: All validators use the public dataset Calculate the global model θ t-1 The model accuracy is expressed as

[0172]

[0173] Step S2: Local verification. For shards Each validator v in j , using the formula Verify model quality. If Then choose the corresponding updated gradient G i (θ t-1 ) for aggregation, otherwise, the model does not participate in global model aggregation.

[0174] Step S3: Global consensus.

[0175] S3.1. Consensus preparation. Each shard Elect a leader. For a shard Each validator v in j , from the slice h s All validators in v k Receive voting summary U k,s and recorded in the local selection summary, where If less than S l-1 If a certain percentage of validators agree, the selection result will be synchronized to the latest state of the block.

[0176] S3.2. Consensus Implementation. For Sharding Each validator v in j , sending his voting results to All validators in the network are included, and the voting results of other validators are recorded to obtain the voting summary.

[0177] S3.3. Consensus Verification. If the percentage of unanimous votes in the voting summary does not exceed Then the next shard is introduced and the consensus pre-preparation phase is returned. Otherwise, the validator sends his voting summary to the leader of this shard. Preferably, The calculation formula is as follows:

[0178]

[0179] Among them, R = α / V represents the proportion of malicious nodes in the validator set, that is, the maximum proportion of malicious validator nodes that this module can tolerate.

[0180] S3.4. Consensus confirmation. Sharding The leader in the network checks the correctness of the validator's voting summary and then proposes a final block containing the correct voting summary. Subsequently, all validators vote on this block using the Byzantine fault tolerance algorithm. If the vote is passed, the local model selected by the model aggregation in this round is determined and broadcast to the network; otherwise, sharding is introduced. And return to the consensus pre-preparation stage.

[0181] Step S4: The verification module verifies each local model and repeats steps S2 and S3 until the local model for aggregation is determined. Then, the updated global model θ is calculated according to the following formula: t :

[0182]

[0183] in, is the set of trainers selected for model aggregation in the tth round of training.

[0184] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0185] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A trusted federated learning system based on blockchain and zero-knowledge proof, characterized in that: include: Perturbation module: includes differential privacy submodule and zero-knowledge proof submodule, adds noise to the local model and generates zero-knowledge proof; Applying a non-interactive zero-knowledge protocol enables the noise added to the local model to be verified; The trainer and the verifier independently and separately construct the same circuit containing specific Gaussian noise using linear approximation. The trainer uses the circuit and the public generator and model parameters to construct the proof. The verifier uses the circuit to verify the received proof and compares the verification output field with the noise model. If the comparison result is correct, the proof is accepted. Verification module: includes local verification and sharding consensus process; Validators uniformly and randomly form a set of multiple shards. In each verification cycle, validators in the shards that meet specific criteria use the public dataset in the system to verify the quality of local models in parallel. A validator in the shard proposes a voting program block and broadcasts it to other members of the shard; The block content is the selection result of the local model; the validators in the shard submit votes to reach a conclusion, and the process continues until a consensus decision is reached; the validators select some local models for global aggregation.

2. The trusted federated learning system based on blockchain and zero-knowledge proof according to claim 1, characterized in that: In the perturbation module: After initializing the system parameters, a public generator is first constructed based on the compressed Σ protocol and shared among all trainers and verifiers. The trainers and verifiers respectively construct consistent circuits based on specific Gaussian noise using linear estimation. During local training, the trainer constructs a proof using the circuit, public generator, and model parameters. After receiving the proof, the verifier uses the generated circuit to verify the legitimacy of the perturbation model. The process of adding noise to the model satisfies (∈,δ) differential privacy, where ∈ and δ are differential privacy parameters, and their values ​​are both between 0 and 1.

3. The trusted federated learning system based on blockchain and zero-knowledge proof according to claim 1, characterized in that: In the differential privacy submodule: The noise has a mean of 0 and a variance of The Gaussian distribution with variance satisfy: Where c1, c2, G are all constants, δ,∈ is the differential privacy parameter, T is the number of training rounds, q is the number of trainers, and n is the number of training samples in the public dataset; In the zero-knowledge proof submodule: The trainer completes local training, repeats the noise calculation steps for a total of d model dimensions, adds noise to all parameters of the local model, and uses the compressed Σ-protocol to calculate the zero-knowledge proof through the generation circuit and the public generator; The objective function F(θ) used in local training is differentiable with respect to the model parameter θ and has Strong convexity, gLipschitz and L smoothness properties, when the learning rate η=1 / L, the objective function will expect the error to satisfy: Among them, n is the number of training samples, F * is the optimal objective function, T is the number of training rounds, c is a constant, and E represents the mathematical expectation of the expression in the square brackets.

4. The trusted federated learning system based on blockchain and zero-knowledge proof according to claim 1, characterized in that: In the verification module: The verification module uses a high-performance sharding blockchain consensus algorithm based on voting, which includes the following steps: The verification module evaluates the quality of local models and selects some models for aggregation based on the model quality to defend against Byzantine attacks from malicious trainers. In the local verification phase, the verifier calculates the accuracy of the local model and the global model of the previous round respectively, and selects the local model with better accuracy than the global model for aggregation. In the consensus phase, all validators in each shard send their generated voting summaries to each other, randomly select a validator, and package the correct voting results into a block. All validators reach a consensus on this block. If the consensus is passed, the local model for aggregation is determined; otherwise, the next round of consensus is repeated. The local model for aggregation first reaches consensus in one shard. When more than a certain proportion of validators submit consistent votes, the consensus ends. Otherwise, the next shard is introduced to repeat the consensus. The proportion is represented by S. l ; The calculation formula is as follows: Among them, R = α / V represents the maximum ratio of malicious verification nodes that this module can tolerate, α is the number of malicious nodes among the verifiers, V is the number of verifiers, z is the variable for setting system security parameters, j is the subscript of the blockchain shard in the system, r is the subscript of the first round of shards, l is the number of consensus rounds, h is the number of rounds of consensus, and j is the jth blockchain shard.

5. The trusted federated learning system based on blockchain and zero-knowledge proof according to claim 1, characterized in that: In the verification module: For sharding Each validator v in j , respectively calculate the local model and the previous global model θ t-1 The accuracy of For a given trainer τ i and the validator v j , the verifier obtains the local model of the trainer The model accuracy of this trainer is evaluated as: in, is a public dataset, n is the number of training samples in the public dataset, x k and k represent data samples and labels respectively, is the regularization term coefficient; The verifier simultaneously calculates the global model θ of the previous round t-1 Model accuracy: if If the model is not included in the global model aggregation, then this model is selected for aggregation; otherwise, the model does not participate in the global model aggregation.

6. A trusted federated learning method based on blockchain and zero-knowledge proof, characterized in that: include: Step S1: The perturbation module applies a non-interactive zero-knowledge protocol so that the noise added to the local model can be verified; The trainer and the verifier independently and separately use linear approximation to construct the same circuit containing specific Gaussian noise. The trainer uses the circuit and the public generator and model parameters to build the proof. The verifier uses the circuit to verify the received proof and compares the verification output field with the noise model. If the comparison result is correct, the proof is accepted. It contains a differential privacy submodule and a zero-knowledge proof submodule to add noise to the local model and generate a zero-knowledge proof at the same time. Step S2: The verification module uniformly and randomly forms a set of multiple shards for the verifiers. In each verification cycle, the verifiers in the shards that meet the specific criteria use the public data set in the system to verify the quality of the local model in parallel. A validator in the shard proposes a voting program block and broadcasts it to other members of the shard; The block content is the selection result of the local model; the validators in the shard submit votes to reach a conclusion, and the process continues until a consensus decision is reached; the validators select part of the local model for global aggregation; it includes local verification and shard consensus process.

7. The trusted federated learning method based on blockchain and zero-knowledge proof according to claim 6, characterized in that: In the perturbation module: After initializing the system parameters, a public generator is first constructed based on the compressed Σ protocol and shared among all trainers and verifiers. The trainers and verifiers respectively construct consistent circuits based on specific Gaussian noise using linear estimation. During local training, the trainer constructs a proof using the circuit, public generator, and model parameters. After receiving the proof, the verifier uses the generated circuit to verify the legitimacy of the perturbation model. The process of adding noise to the model satisfies (∈,δ) differential privacy, where ∈ and δ are differential privacy parameters, and their values ​​are both between 0 and 1.

8. The trusted federated learning method based on blockchain and zero-knowledge proof according to claim 6, characterized in that: In the differential privacy submodule: The noise has a mean of 0 and a variance of The Gaussian distribution with variance satisfy: Where c1, c2, G are all constants, δ,∈ is the differential privacy parameter, T is the number of training rounds, q is the number of trainers, and n is the number of training samples in the public dataset; In the zero-knowledge proof submodule: The trainer completes local training, repeats the noise calculation steps for a total of d model dimensions, adds noise to all parameters of the local model, and uses the compressed Σ-protocol to calculate the zero-knowledge proof through the generation circuit and the public generator; The objective function F(θ) used in local training is differentiable with respect to the model parameter θ and has Strong convexity, G Lipschitz and L smoothness properties, when the learning rate η = 1 / L, the objective function will expect the error to satisfy: Among them, n is the number of training samples, F * is the optimal objective function, T is the number of training rounds, c is a constant, and E represents the mathematical expectation of the expression in the square brackets.

9. The trusted federated learning method based on blockchain and zero-knowledge proof according to claim 6, characterized in that: In the verification module: The verification module uses a high-performance sharding blockchain consensus algorithm based on voting, which includes the following steps: The verification module evaluates the quality of local models and selects some models for aggregation based on the model quality to defend against Byzantine attacks from malicious trainers. In the local verification phase, the verifier calculates the accuracy of the local model and the global model of the previous round respectively, and selects the local model with better accuracy than the global model for aggregation. In the consensus phase, all validators in each shard send their generated voting summaries to each other, randomly select a validator, and package the correct voting results into a block. All validators reach a consensus on this block. If the consensus is passed, the local model for aggregation is determined; otherwise, the next round of consensus is repeated. The local model for aggregation first reaches consensus in one shard. When more than a certain proportion of validators submit consistent votes, the consensus ends. Otherwise, the next shard is introduced to repeat the consensus. The proportion is represented by S. l ; The calculation formula is as follows: Among them, R = α / V represents the maximum ratio of malicious verification nodes that this module can tolerate, α is the number of malicious nodes among the verifiers, V is the number of verifiers, z is the variable for setting system security parameters, j is the subscript of the blockchain shard in the system, r is the subscript of the first round of shards, l is the number of consensus rounds, h is the number of rounds of consensus, and j is the jth blockchain shard.

10. The trusted federated learning method based on blockchain and zero-knowledge proof according to claim 6, characterized in that: In the verification module: For sharding Each validator v in j , respectively calculate the local model and the previous global model θ t-1 The accuracy of For a given trainer τ i and the validator v j , the verifier obtains the local model of the trainer The model accuracy of this trainer is evaluated as: in, is a public dataset, n is the number of training samples in the public dataset, x k and k represent data samples and labels respectively, is the regularization term coefficient; The verifier simultaneously calculates the global model θ of the previous round t-1 Model accuracy: if If the model is not included in the global model aggregation, then this model is selected for aggregation; otherwise, the model does not participate in the global model aggregation.

Images