Blockchain-based privacy federated learning method and system against byzantine attacks

Abstract

The application provides a kind of anti-bayesian attack privacy federated learning method and system based on block chain, including incentive step: by work node to bid node offer, bid node selects winning work node according to the reputation value of work node, and winning work node is randomly divided into training node and verification node.Training step: training node requests global model to block chain and uses local privacy data set to carry out model training, and adds Gaussian noise to the gradient of loss function to obtain local model and sends to verification node, verification node calculates the accuracy of local model on local data set as model score, then according to the score result carries out model aggregation, finally, the score result and the global model obtained by aggregation are added to the block chain through the bayesian consensus mechanism.The application overcomes the disadvantages of centralized federated learning system, designs transparent, traceable decentralized federated learning system framework in training process, and gives consideration to model safety, efficiency and performance.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to a privacy-preserving federated learning method and system based on blockchain and resistant to Byzantine attacks. More particularly, it relates to algorithmic solutions for problems related to incentive mechanisms, privacy protection, and Byzantine fault tolerance in federated learning. Background Technology

[0002] In recent years, with the development of machine learning, especially deep learning, machine learning methods have become increasingly important for tasks such as prediction and content generation in fields like computer vision, natural language processing, and decision support. As a data-driven approach, machine learning requires large-scale, high-quality data for model training, which presents a data privacy constraint. Specifically, schemes where model training nodes collect and train data from data owners are facing increasingly severe challenges. To address this, the academic community has proposed federated learning algorithms, where multiple data owners train models locally and then send model updates, rather than training data, to a server for model aggregation, thereby achieving the goal of privacy-preserving machine learning.

[0003] As research into federated learning deepens, various inference attack schemes targeting model updates have been proposed, forcing federated learning algorithms to incorporate reliable privacy protection mechanisms. Differential privacy, homomorphic encryption, and secure multi-party computation are among the privacy protection mechanisms that have received widespread attention. However, due to the complexity of the federated learning computation process, and the computational or communication efficiency limitations of homomorphic encryption and secure multi-party computation schemes, the aforementioned privacy-preserving federated learning based on these two algorithms is inefficient and impractical. Conversely, differential privacy, due to its efficient computation process, is considered a feasible solution for achieving privacy-preserving federated learning and has attracted widespread attention.

[0004] Because the performance of federated learning models is closely related to the amount and quality of data during training, how to incentivize more worker nodes, especially those holding high-quality data, to participate in the training process has always been a crucial issue in federated learning algorithm design. The training process of federated learning models requires worker nodes to perform the training locally and send the results to the server, which consumes the computing power and network bandwidth of the worker nodes. Therefore, incentives for training nodes are necessary. Currently, most mainstream incentive schemes only consider the computational load of worker nodes, i.e., the amount of incentive is proportional to the amount of data, while ignoring the factor of data quality. This leads to worker nodes actively lowering the quality of data to increase rewards, thus requiring the design of more effective incentive mechanisms.

[0005] Furthermore, using centralized servers for federated learning model aggregation can lead to performance bottlenecks, single points of failure, and vulnerability to attacks. Blockchain technology, as the underlying technology of cryptocurrency systems, offers superior characteristics such as decentralization, data transparency, and immutability. Therefore, designing a decentralized federated learning system using blockchain technology is considered a feasible solution to these problems. Moreover, to prevent Byzantine attacks on federated learning worker nodes and improve system robustness, designing highly robust federated learning algorithms based on blockchain is also an important research topic.

[0006] Patent document CN114239862A discloses a federated learning method that protects user data privacy against Byzantine attacks. The method divides the server into two parts: a computation server and a service provider server. These two servers jointly perform secure computation and each obtains a validity detection result and a cosine similarity result that reflects the similarity between the server's gradient direction and the local gradient direction.

[0007] Currently, research and solutions exist for certain aspects of the aforementioned problems. However, considering privacy protection, robustness, incentive mechanisms, and the design of a decentralized federated learning system based on blockchain technology, no effective method has yet been found to solve this problem.

[0008] Therefore, taking into account the above issues, designing a privacy-preserving, Byzantine-robust incentive-based federated learning framework based on blockchain has extremely important practical significance. Summary of the Invention

[0009] To address the shortcomings of existing technologies, the purpose of this invention is to provide a privacy-preserving federated learning method and system based on blockchain that is resistant to Byzantine attacks.

[0010] A privacy-preserving federated learning method based on blockchain and resistant to Byzantine attacks, provided by the present invention, includes:

[0011] Incentive steps: Work nodes bid to bidding nodes, and bidding nodes select winning work nodes based on the reputation value of the work nodes, and randomly divide the winning work nodes into training nodes and verification nodes;

[0012] Training steps: The training node requests a global model from the blockchain and trains the model using a local privacy dataset. Gaussian noise is added to the gradient of the loss function to obtain a local model, which is then sent to the verification node. The verification node calculates the accuracy of the local model on the local dataset as a model score. The model is then aggregated based on the score. Finally, the score and the aggregated global model are added to the blockchain through the Byzantine consensus mechanism.

[0013] Preferably, the incentive step includes a differential privacy reverse bidding sub-step and a reputation value mechanism sub-step, including:

[0014] Step S1.1: Cost and amount of data acquisition;

[0015] Step S1.2: The worker node calculates the cost required to participate in this round of training based on the cost and volume of the data;

[0016] Step S1.3: Calculate the corresponding bid based on the required cost and send it to the bidding node;

[0017] Step S1.4: The bidding node calculates the reputation value of the corresponding working node based on the scoring result data stored in the blockchain;

[0018] Step S1.5: Calculate the probability of selecting the corresponding working node based on the reputation value;

[0019] Step S1.6: Select a worker node as the winning worker node according to the probability and calculate the training cost of the winning worker node. Randomly divide the worker node into training nodes and verification nodes.

[0020] The formula for calculating the training cost is as follows:

[0021]

[0022] Where, p i Let b represent the training cost of the i-th worker node. i Let b represent the bid of the i-th worker node. m b i The maximum value of r i Let r represent the reputation of the i-th worker node. m Indicates r i The maximum value;

[0023] Preferably, for each worker node i, the probability of selecting the corresponding winning worker node is calculated using the following formula:

[0024]

[0025] Where W and R represent the set of worker nodes and the reputation set corresponding to the worker nodes, respectively; b and r represent the bid and reputation of the worker node, respectively; and the index j is used to traverse each worker node in W.

[0026] Preferably, the maximum benefit is obtained when the bid of the corresponding working node is equal to the cost required to participate in this round of training, and the benefit is greater than 0.

[0027] Preferably, the training step includes a Gaussian mechanism differential privacy model optimization sub-step and a local accuracy verification sub-step, including:

[0028] Step S2.1: Obtain the global model on the blockchain and train it using the local dataset;

[0029] Step S2.2: The training node adds random Gaussian noise to the calculated loss function gradient and sends it as a local model to the validation node. The random Gaussian noise has a mean of 0 and a variance of σ. 2 I d ,and

[0030] Among them, I d Let denot be a d×d dimensional matrix, ΔG denote the upper bound of the difference of the loss function on adjacent sets, and δ G ,∈ G Represents (∈ G ,δ G - Differential privacy boundary parameters;

[0031] Step S2.3: The verification node scores the local model by executing the local accuracy verification sub-step, and obtains the corresponding score result;

[0032] Step S2.4: Verify the node aggregation model, and store the global model and the scoring results on the blockchain using the Byzantine formula;

[0033] Step S2.5: Take the highest working node in the current training cycle as the bidding node for the next cycle, and repeat steps S2.1 to S2.4 until the training ends.

[0034] Preferably, the formula for the verification node aggregation model is as follows:

[0035]

[0036] Where, θ t This represents the model parameters, where η represents the learning rate, and n... i n represents the size of the local data at training node i. j The amount of local data used to iterate through each training node, G i () indicates the model update for the i-th training node;

[0037] The models participating in the aggregation are the 2 / 3*p local models with the highest model scores, where p is the number of training nodes.

[0038] According to the present invention, a blockchain-based privacy-preserving federated learning system resistant to Byzantine attacks includes:

[0039] Incentive module: Work nodes bid to bidding nodes, and bidding nodes select winning work nodes based on the reputation value of the work nodes, and randomly divide the winning work nodes into training nodes and verification nodes;

[0040] Training Module: The training node requests a global model from the blockchain and trains the model using a local privacy dataset. It adds Gaussian noise to the gradient of the loss function to obtain a local model, which is then sent to the verification node. The verification node calculates the accuracy of the local model on the local dataset as a model score. Based on the score, the model is aggregated. Finally, the score and the aggregated global model are added to the blockchain through the Byzantine consensus mechanism.

[0041] Preferably, the incentive module includes a differential privacy reverse bidding submodule and a reputation value mechanism submodule, comprising:

[0042] Module M1.1: Cost and volume of data acquisition;

[0043] Module M1.2: The worker node calculates the cost required to participate in this round of training based on the cost and volume of the data;

[0044] Module M1.3: Calculates the corresponding bid based on the required cost and sends it to the bidding node;

[0045] Module M1.4: The bidding node calculates the reputation value of the corresponding working node based on the scoring result data stored in the blockchain;

[0046] Module M1.5: Calculates the probability of selecting the corresponding working node based on the reputation value;

[0047] Module M1.6: Select a worker node as the winning worker node according to the probability and calculate the training cost of the winning worker node, and randomly divide the worker node into training nodes and verification nodes;

[0048] The formula for calculating the training cost is as follows:

[0049]

[0050] Where, p i Let b represent the training cost of the i-th worker node. i Let b represent the bid of the i-th worker node. m b i The maximum value of r i Let r represent the reputation of the i-th worker node. m Indicates r i The maximum value;

[0051] Preferably, for each worker node i, the probability of selecting the corresponding winning worker node is calculated using the following formula:

[0052]

[0053] Where W and R represent the set of worker nodes and the reputation set corresponding to the worker nodes, respectively; b and r represent the bid and reputation of the worker node, respectively; and the index j is used to traverse each worker node in W.

[0054] Preferably, the maximum benefit is obtained when the bid of the corresponding working node is equal to the cost required to participate in this round of training, and the benefit is greater than 0.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] 1. This invention uses blockchain technology to overcome the drawbacks of centralized federated learning systems and designs a decentralized federated learning system framework with transparent and traceable training process, taking into account model security, efficiency and performance.

[0057] 2. This invention uses a differential privacy reverse bidding algorithm to incentivize worker nodes to participate in model training. It comprehensively considers factors such as the amount and quality of data of worker nodes, satisfies the completeness of incentives and individual rationality, and protects the bidding privacy of worker nodes.

[0058] 3. This invention designs a differential privacy federated learning algorithm that adds Gaussian noise to the model updates of training nodes. This algorithm can be proven to achieve differential privacy and can analyze the upper bound of the impact of noise on model performance.

[0059] 4. This invention introduces a model verification process, which selects model updates based on the model scores of training nodes to aggregate into a new global model, enabling the federated learning system to resist Byzantine attacks and have higher robustness.

[0060] 5. This invention has strong practicality and a wide range of applications. Attached Figure Description

[0061] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0062] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0063] Figure 2 This is a schematic diagram of the excitation module's workflow in this invention.

[0064] Figure 3 This is a schematic diagram of the training module workflow in this invention.

[0065] Figure 4 This is a schematic diagram of the on-chain block data structure in this invention. Detailed Implementation

[0066] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0067] According to the present invention, a privacy-preserving federated learning method based on blockchain and resistant to Byzantine attacks is provided, such as... Figure 1 As shown, it includes:

[0068] Incentive process: Worker nodes bid to bidding nodes. Bidding nodes select a winning worker node based on the worker nodes' reputation score and randomly assign the winning worker node to training nodes or validation nodes. For example... Figure 2 As shown, the incentive steps include a differential privacy reverse bidding sub-step and a reputation value mechanism sub-step, including node reputation value updating, winning worker node selection, and training cost determination. The specific steps are as follows:

[0069] Step S1.1: Cost and amount of data acquisition.

[0070] Step S1.2: The worker node calculates the cost required to participate in this round of training based on the cost and volume of data.

[0071] Step S1.3: Calculate the corresponding bid based on the required cost and send it to the bidding node. Specifically, according to the incentive completeness rule, that is, the incentive completeness and individual rationality of the incentive mechanism, the maximum benefit is obtained when the bid of the corresponding work node is equal to the cost required to participate in this round of training and the benefit is greater than 0.

[0072] Step S1.4: The bidding nodes calculate the reputation value of the corresponding working nodes through the reputation value mechanism sub-step based on the scoring result data stored in the blockchain.

[0073] Step S1.5: Calculate the probability of selecting the corresponding working node based on the reputation value.

[0074] Step S1.6: Select winning worker nodes based on probability and calculate the training cost for each winning worker node. Randomly divide the worker nodes into training nodes and validation nodes. For example, the bidding node randomly selects 25% of the winning worker nodes as validation nodes, and the remaining nodes as training nodes.

[0075] The formula for calculating training costs is as follows:

[0076]

[0077] Where, p i Let b represent the training cost of the i-th worker node. i Let b represent the bid of the i-th worker node. m b i The maximum value of r i Let r represent the reputation of the i-th worker node. m Indicates r i The maximum value. This method satisfies individual rationality and incentive completeness, and protects the cost data privacy of worker nodes by selecting winning worker nodes to participate in training through probability.

[0078] The working node is selected based on probability (∈ E ,δ E The differential privacy index method, for each worker node i, calculates the probability of selecting the corresponding winning worker node, as shown in the following formula:

[0079]

[0080] Where W and R represent the set of worker nodes and the reputation set corresponding to the worker nodes, respectively; b and r represent the bid and reputation of the worker node, respectively; and the index j is used to traverse each worker node in W.

[0081] Training steps: The training node requests the global model from the blockchain and trains the model using a local privacy dataset. Gaussian noise is added to the gradient of the loss function to obtain a local model, which is then sent to the verification node. The verification node calculates the accuracy of the local model on its local dataset as a model score. Models are then aggregated based on the score, and finally, the score and the aggregated global model are added to the blockchain via a Byzantine consensus mechanism. Specifically, the score is broadcast to other verification nodes, and models from several nodes with high scores are selected to participate in model updates, aggregating into a new global model. The information of the winning worker, the score, and the global model are then stored on the blockchain. Furthermore, system nodes update their reputation values ​​based on the score, and the training node with the highest local model score in this round of training is selected as the bidding node for the next round. Figure 2 As shown, the training steps are a differential privacy, Byzantine robust federated learning training process, including a Gaussian mechanism differential privacy model optimization sub-step and a local accuracy verification sub-step, including:

[0082] Step S2.1: Obtain the global model on the blockchain and train it using the local dataset.

[0083] Step S2.2: The training node adds random Gaussian noise to the calculated loss function gradient and sends it as a local model to the validation node. The mean of the added random Gaussian noise is 0, and the variance is σ. 2 I d ,and Among them, I d Let denot be a d×d dimensional matrix, ΔG denote the upper bound of the difference of the loss function on adjacent sets, and δ G ,∈ G Represents (∈ G ,δ G - Differential privacy boundary parameters. Specifically, training nodes use a common loss function and calculate its gradient based on the current state-of-the-art global model. Then, worker nodes add Gaussian noise to this gradient to obtain a perturbed gradient, which is then sent as a local model to the validation nodes. This differential privacy algorithm, designed by adding Gaussian noise, can achieve (∈ G ,δ G - Differential privacy.

[0084] Step S2.3: The validation nodes score the local model by performing the local accuracy verification sub-step, obtaining the corresponding score results. The validation nodes use local data as the test set, calculate the model score of the training node's local model on the test set, score the training node's model updates according to a unified rule, and share the score results among the validation nodes. For a specific training node, the median score of its local model scores from all validation nodes is taken as its final model score.

[0085] Step S2.4: Verify the node aggregation model by storing the global model and scoring results on the blockchain using the Byzantine formula. Other nodes in the network update the reputation value of the winning worker node based on the scoring results. The formula for verifying the node aggregation model is as follows:

[0086]

[0087] Where, θ t n represents the model parameters, η represents the learning rate, and n i n represents the size of the local data at training node i. j The amount of local data used to iterate through each training node, G i () represents the model update of the i-th training node. The models participating in the aggregation are the top 2 / 3*p local models with the highest model scores, where p is the number of training nodes, thus ensuring that the federated learning system can resist Byzantine attacks such as stochastic gradients, backpropagation, and label flipping. By adding Gaussian noise of the aforementioned size through this aggregation step, in application scenarios with uniformly distributed training data, after... After training rounds, the upper bound of the difference between the trained model and the real model is: Where c1 and c2 are constants, and n is the total amount of training data.

[0088] Step S2.5: Select the highest-scoring working node in the current training cycle as the bidding node for the next cycle, and repeat steps S2.1 to S2.4 until training ends. In other words, the training node with the highest model score in this round becomes the bidding node for the next training round, and the incentive cost for each bidding node is a constant B. The first bidding node is determined during system initialization.

[0089] Furthermore, the training step employs a Gaussian mechanism differential privacy design for privacy protection and a Byzantine robust federated learning method. This step consists of three parts: model training, model validation, and model aggregation. In model training, training node t... i The loss function is During the t-th round of training, the training node first obtains the current global model θ from the blockchain. t The stochastic gradient descent optimization algorithm is employed, taking the local dataset and the current model as input to calculate the gradient of the loss function. Then, an additive condition following the order N(0,σ) is added to this gradient. 2 I d A local model is obtained by using Gaussian noise distributed in a specific pattern and sent to the validation nodes. During model validation, the validation nodes score the local model of each training node according to its accuracy and broadcast the scores to other validation nodes. The test data is the validation nodes' local dataset. For a specific training node, the median score from all validation nodes is taken as the model score. In model aggregation, the validation nodes select the models from the top 2 / 3 of the training nodes with the highest scores, weighted according to their training data volume, to participate in model aggregation, generating a new global model. This new global model, along with the scoring results, is added to the blockchain via the Byzantine consensus mechanism.

[0090] Furthermore, the present invention will be specifically described below with reference to the accompanying drawings:

[0091] like Figure 2 As shown, the excitation module:

[0092] Step 1: System Initialization. This step defines system security parameters, cryptographic algorithms, federated learning tasks, model parameters, node reputation values, and the nodes selected for the first round of bidding.

[0093] Step 2: Working node w i Calculate the cost c for participating in this round of training based on the local dataset information used for training. i The cost is proportional to the amount of data n i That is, c i =(λ+ξ) i )n i , where ξ iλ is a cost coefficient related to data quality, measuring the relationship between data volume and computational cost. The system allows different worker nodes to hold different values ​​for ξ. i However, they share the same λ. Subsequently, in accordance with the principle of incentive completeness, the worker node sends its training cost as its bid to the bidding node.

[0094] Step 3: The bidding node obtains the local model score results stored on the blockchain and updates the worker node reputation value. Then, based on the worker node's bid and reputation value, it calculates the probability of that worker node being selected and selects the winning worker node according to the probability. Finally, the bidding node calculates the training cost for each winning worker node. The specific steps are as follows:

[0095] Step 3.1. The reputation value of the system's working nodes is initialized to 1. For a system with p training nodes and q validation nodes in each training round, based on the local model scores in each round, the ranking i of each training node (i = 0, 1, ..., p-1) can be obtained in ascending order of scores. The reputation value update algorithm for a specific training node is as follows:

[0096]

[0097] Where c1 is a constant and c1>1, r ave ,r min These are the mean and minimum reputation values ​​of the nodes in this round of training, respectively. Similarly, for a specific verification node, its reputation value update algorithm is as follows:

[0098]

[0099] Where c2 is a constant and c2>1, and i is the ranking of the validation node scoring results, which is based on the number of validation node scoring results selected as model scores.

[0100] Step 3.2. The bidding node calculates the probability of being selected as the winning worker node based on the worker node's reputation value and bid. The algorithm is as follows:

[0101]

[0102] Then, based on probability, one worker node is selected to enter the winning worker node set.

[0103] Step 3.3. Calculate the training cost for the selected winning working node.

[0104]

[0105] Then, determine whether the number of elements in the winning worker node set has reached K. If not, return to S3.2; otherwise, proceed to the next step.

[0106] Step 4: The bidding node randomly selects p elements from the set of winning worker nodes as training nodes, and the rest are verification nodes.

[0107] like Figure 3 As shown, the training module:

[0108] Step 1: In the t-th round of training, the training node obtains the global model θ on the blockchain. t Using the local dataset D i Training is performed on dataset D. i The number of samples is n i The loss function for training the local model of a node is:

[0109]

[0110] Step 2: Calculate the local model Z t To obey N(0,σ) 2 I d The Gaussian noise is distributed in a specific pattern. The local model is then sent to the verification node.

[0111] Step 3: The validation nodes score the local model of the training nodes, that is, using the local dataset as the test set, they calculate the accuracy of the local model. The validation nodes broadcast the score results to other validation nodes, and for a specific training node, they select the median of the scores given by the validation nodes as the local model score.

[0112] Step 4: The validation node sorts the local models of the training nodes from highest to lowest score, selects the top 2 / 3 of the local models with the highest scores, and aggregates them to obtain the global model. The aggregation algorithm is as follows:

[0113]

[0114] Subsequently, through the Byzantine consensus mechanism, the scoring results of all validating nodes, as well as the calculated new global model, are stored on the blockchain.

[0115] Step 5: The working node with the highest score in the t-th round of training is used as the bidding node in the next round. Repeat the above process until the training ends.

[0116] In this embodiment, the block structure is as follows: Figure 4 As shown. A block contains a block header and transaction content information. The block header includes: the hash of the previous block, the Merkle tree root, the timestamp, the block height, and federated learning information. The transaction content includes incentive information, scoring results, and global model information.

[0117] Federated learning information includes the learning task and model parameters, i.e., the number of training rounds.

[0118] The incentive information includes the bidding node ID, the winning worker node ID and fee, and signature information.

[0119] The scoring result information includes information on q verification nodes, the scoring results of q verification nodes, and their signatures.

[0120] The global model information includes the IDs of 2 / 3*p training nodes participating in model aggregation, the global model and the signature of the verification node. The IDs of the training nodes participating in model aggregation are sorted from highest to lowest according to their score results, so the first ID is the node for the next round of bidding.

[0121] The present invention also provides a blockchain-based privacy-preserving federated learning system resistant to Byzantine attacks. This blockchain-based privacy-preserving federated learning system can be implemented by executing the process steps of a blockchain-based privacy-preserving federated learning method. That is, those skilled in the art can understand the blockchain-based privacy-preserving federated learning method as a preferred implementation of the blockchain-based privacy-preserving federated learning system.

[0122] According to the present invention, a blockchain-based privacy-preserving federated learning system resistant to Byzantine attacks includes:

[0123] Incentive Module: Worker nodes bid to bidding nodes, which select the winning worker node based on the worker nodes' reputation score. The winning worker node is then randomly assigned to training or validation nodes. The incentive module includes a differential privacy reverse bidding submodule and a reputation score mechanism submodule, including:

[0124] Module M1.1: Cost and volume of data acquisition. Module M1.2: Worker nodes calculate the cost required to participate in this round of training based on the cost and volume of data. Module M1.3: Calculate the corresponding bid based on the required cost and send it to the bidding nodes. Module M1.4: Bidding nodes calculate the reputation value of the corresponding worker node based on the scoring results data stored in the blockchain. Module M1.5: Calculate the probability of selecting the corresponding worker node based on the reputation value. Module M1.6: Select the winning worker node based on the probability and calculate the training cost of the winning worker node. Worker nodes are randomly divided into training nodes and validation nodes. The formula for calculating the training cost is as follows:

[0125]

[0126] Where, p i Let b represent the training cost of the i-th worker node. i Let b represent the bid of the i-th worker node. m b i The maximum value of r iLet r represent the reputation of the i-th worker node. m Indicates r i The maximum value.

[0127] For each worker node i, the probability of selecting the corresponding winning worker node is calculated using the following formula:

[0128]

[0129] Where W and R represent the set of worker nodes and the reputation set corresponding to each worker node, respectively; b and r represent the bid and reputation of each worker node, respectively; and the index j is used to traverse each worker node in W. The maximum benefit, greater than 0, is achieved when the bid of the corresponding worker node equals the cost required to participate in this round of training.

[0130] Training Module: Training nodes request the global model from the blockchain and train the model using a local privacy dataset. Gaussian noise is added to the gradient of the loss function to obtain a local model, which is then sent to the verification nodes. Verification nodes calculate the accuracy of the local model on the local dataset as a model score. Based on the score, the models are aggregated. Finally, the score and the aggregated global model are added to the blockchain via a Byzantine consensus mechanism. The training module includes a Gaussian mechanism differential privacy model optimization submodule and a local accuracy verification submodule, including:

[0131] Module M2.1: Obtains the global model on the blockchain and trains it using the local dataset. Module M2.2: The training node adds random Gaussian noise to the calculated loss function gradient and sends it as a local model to the validation node. The random Gaussian noise has a mean of 0 and a variance of σ. 2 I d ,and Among them, I d Let denot be a d×d dimensional matrix, ΔG denote the upper bound of the difference of the loss function on adjacent sets, and δ G ,∈ G Represents (∈ G ,δ G - Differential privacy boundary parameters. Module M2.3: The verification node scores the local model by triggering the local accuracy verification submodule, obtaining the corresponding score result. Module M2.4: The verification node aggregates the model, storing the global model and score results on the blockchain using the Byzantine formula. The formula for the verification node to aggregate the model is as follows:

[0132]

[0133] Where, θ t This represents the model parameters, where η represents the learning rate, and n... i n represents the size of the local data at training node i. jThe amount of local data used to iterate through each training node, G i () indicates the model update for the i-th training node. The models participating in the aggregation are the 2 / 3*p local models with the highest model scores, where p is the number of training nodes. Module M2.5: Takes the highest working node in the current training cycle as the bidding node for the next cycle, and repeatedly triggers modules M2.1 to M2.4 until training ends.

[0134] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0135] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for privacy federated learning based on blockchain, which is resistant to Byzantine attacks, characterized in that, Comprising: Incentive step: the worker nodes bid to the bidding nodes, the bidding nodes select the winning worker nodes according to the reputation value of the worker nodes, and randomly divide the winning worker nodes into training nodes and verification nodes; Training step: the training nodes request global model from the blockchain and train the model using local private data set, add Gaussian noise to the gradient of the loss function to obtain local model and send it to the verification nodes, the verification nodes calculate the accuracy of the local model on the local data set as the model score, then perform model aggregation according to the score result, finally add the score result and the global model obtained by aggregation to the blockchain through the Byzantine consensus mechanism; The training step comprises: Step S2.1: obtaining the global model on the blockchain, training using the local data set; Step S2.2: The training node adds random Gaussian noise to the computed loss function gradient, with mean 0 and variance , and , wherein, denotes a matrix of weights, denotes an upper bound on the difference of the loss function on adjacent sets, , denotes a differential privacy bound parameter; Step S2.3: the verification node scores the local model by executing the local accuracy checking sub-step to obtain the corresponding score result; Step S2.4: the verification node aggregates the model, and stores the global model and the score result on the blockchain through the Byzantine formula; Step S2.5: the highest worker node in the current training period is taken as the bidding node in the next period, and the steps S2.1 to S2.4 are repeatedly executed until the training is completed.

2. The method of claim 1, wherein, The incentive step includes differential privacy reverse bidding sub-step and reputation value mechanism sub-step, comprising: Step S1.1: obtaining the cost and data volume of the data; Step S1.2: the worker node calculates the required cost for participating in the current training according to the cost and data volume of the data; Step S1.3: calculate the corresponding bid according to the required cost and send it to the bidding node; Step S1.4: the bidding node calculates the reputation value of the corresponding worker node according to the score result data stored in the blockchain; Step S1.5: calculate the probability of selecting the corresponding worker node through the reputation value; Step S1.6: select the worker node as the winning worker node according to the probability and calculate the training fee of the winning worker node, and randomly divide the worker node into training node and verification node; Wherein, the calculation formula of the training fee is as follows: wherein, denotes the training cost of the i-th worker node, denotes the bid of the i-th worker node, denotes the maximum of denotes the reputation of the i-th worker node, denotes the maximum of the maximum of​ 3. The method of claim 2, wherein, When the corresponding worker node bid is equal to the required cost for participating in the current training, the obtained income is maximum and greater than 0.

4. A blockchain-based privacy federated learning system against Byzantine attacks, characterized in that, Comprising: Incentive module: the worker nodes bid to the bidding nodes, the bidding nodes select the winning worker nodes according to the reputation value of the worker nodes, and randomly divide the winning worker nodes into training nodes and verification nodes; Training module: the training nodes request global model from the blockchain and train the model using local private data set, add Gaussian noise to the gradient of the loss function to obtain local model and send it to the verification nodes, the verification nodes calculate the accuracy of the local model on the local data set as the model score, then perform model aggregation according to the score result, finally add the score result and the global model obtained by aggregation to the blockchain through the Byzantine consensus mechanism; The training module comprises: Module M2.1: obtaining the global model on the blockchain, training using the local data set; Module M2.2: The training node adds random Gaussian noise to the computed gradient of the loss function, with mean 0 and variance , and , wherein, denotes a matrix of weights, denotes an upper bound on the difference of the loss function on adjacent sets, , denotes a differential privacy bound parameter; Module M2.3: The verification node scores the local model by performing the local accuracy verification sub-step to obtain a corresponding score result; Module M2.4: The verification node aggregates the model, and stores the global model and the score result on the blockchain through the Byzantine formula; Module M2.5: The highest working node in the current training period is taken as the bidding node in the next period, and the modules M2.1 to M2.4 are repeatedly executed until the training is completed.

5. The blockchain-based Byzantine-attack-resistant privacy federated learning system according to claim 4, wherein, The incentive module includes a differential privacy reverse bidding sub-module and a reputation value mechanism sub-module, which includes: Module M1.1: Obtain the cost and data volume of data; Module M1.2: The working node calculates the required cost for participating in the current round of training according to the cost and data volume of the data; Module M1.3: Calculate the corresponding bid according to the required cost and send it to the bidding node; Module M1.4: The bidding node calculates the reputation value of the corresponding working node according to the score result data stored on the blockchain; Module M1.5: Calculate the probability of selecting the corresponding working node through the reputation value; Module M1.6: Select the working node as the winning working node according to the probability and calculate the training fee of the winning working node, and randomly divide the working node into a training node and a verification node; Wherein, the calculation formula of the training fee is as follows: wherein, represents the training cost of the i-th worker node, represents the bid of the i-th worker node, represents the maximum value of represents the reputation of the i-th worker node, represents the maximum value of .​ 6. The blockchain-based Byzantine-attack-resistant privacy federated learning system according to claim 5, wherein, When the corresponding working node bid is equal to the required cost for participating in the current round of training, the obtained income is maximum and greater than 0.

Images