Blockchain-based byzantine robust federated learning method and system

Abstract

The present disclosure relates to a kind of blockchain-based byzantine robust federated learning method and system, the method is not only based on the committee aggregation algorithm of feedback mechanism, by selecting a committee node in each round of learning process to carry out the contribution calculation of model update, and according to the contribution of each node to determine the aggregation strategy, thereby reducing the influence of malicious model update on global model, after completing aggregation stage, a round of feedback verification is added, only the global model of this aggregation is verified and approved by most nodes, training can formally enter next round.The present application solves the problem of untrusted central server, and can be used to resist the poisoning attack of malicious client.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, and in particular relates to a blockchain-based Byzantine robust federated learning method and system. Background Technology

[0002] With the improvement of hardware computing performance and the growth of data, deep learning algorithms have been widely applied in various industries, including image recognition, natural language processing, and recommendation systems. These algorithms require massive amounts of data for training, but obtaining high-quality data is very difficult in most fields because data often has commercial value, and organizations are reluctant to share data directly. At the same time, data privacy and security issues are becoming increasingly important, and countries are enacting laws to protect personal data privacy and rights. These new data regulations and laws pose challenges to the traditional centralized data processing model of artificial intelligence. Therefore, a new machine learning paradigm—federated learning—has attracted widespread attention from researchers.

[0003] Federated learning is a special distributed machine learning approach designed to train models by leveraging data distributed across different devices and locations while protecting data privacy and security. The core idea of ​​federated learning is to build machine learning models on multiple devices while preventing data leakage across these devices. Different data owners collaborate on training without sharing the data itself, sharing the value of the data but not the data itself. Federated learning has been deployed in many large companies and plays a crucial role in training data for privacy-sensitive applications distributed at the edge.

[0004] However, traditional federated learning is vulnerable to attacks from malicious clients and servers. For example... Figure 1As shown, malicious servers may tamper with the training process or control the global model obtained by participants. Federated learning systems are also vulnerable to poisoning attacks by malicious participants. Attackers can intentionally alter their local data or the parameters of their uploaded models to attack the convergence of the global model or implant backdoors into it. In recent years, research on poisoning attacks in federated learning has become an important direction. For example, some works have proposed robust aggregation methods based on model update differences, designing a robust aggregation rule to reduce the interference of malicious gradients on the global model. Other works require the central server to have a validation set, using the validation dataset to verify the model updates uploaded by participants, thereby filtering out malicious model updates. However, these defenses all rely on the trustworthiness of the central server and perform poorly when the data is not independent and identically distributed. To reduce dependence on the central server, some works have focused on designing a decentralized federated learning framework, utilizing the decentralized characteristics of blockchain and writing corresponding smart contracts to coordinate the federated training of various participants, thereby preventing attacks from malicious servers. However, these works rarely consider the possibility of poisoning attacks by malicious participants.

[0005] In summary, designing a Byzantine robust federated learning method presents two main challenges:

[0006] 1. The assumption of a trusted server does not reflect reality.

[0007] In federated learning, the central server is usually responsible for controlling and coordinating the training of local devices and the aggregation of global models. If the server is malicious, it may tamper with the computation in the federated learning protocol, causing the Byzantine fault-tolerant aggregation algorithm to fail and affecting the performance of the entire federated learning system.

[0008] 2. Non-independent and identically distributed data reduces the success rate of malicious detection models.

[0009] The Byzantine Fault Model assumes the existence of a small number of malicious actors who may engage in poisoning attacks, deliberately tampering with their local models or uploading fake model parameters to influence the training of the global model. Since data among participants in federated learning is typically not independent and identically distributed, detecting malicious model updates becomes even more difficult.

[0010] In summary, how to solve the problem of untrusted central servers in traditional federated learning systems, and how to provide effective methods for detecting malicious models under non-independent and identically distributed data distributions, have become urgent technical problems to be solved. Summary of the Invention

[0011] To address the vulnerability of existing Byzantine fault-tolerant aggregation algorithms to poisoning attacks due to the untrustworthiness of the central server, this invention proposes a blockchain-based Byzantine robust federated learning method. This method solves the problem of untrustworthy central servers and defends against poisoning attacks from malicious clients. Furthermore, to address the issue of malicious node poisoning attacks, this invention proposes a committee aggregation algorithm based on a feedback mechanism. In each round of learning, a committee node is selected to calculate its contribution to model updates, and the aggregation strategy is determined based on each node's contribution, thereby reducing the impact of malicious model updates on the global model. Moreover, to ensure the trustworthiness of the committee nodes, a feedback verification round is added after the aggregation phase. Only when a majority of nodes verify and approve the global model of this aggregation will training officially enter the next round. The number of nodes that pass verification in this round is maintained on-chain as the approval score of the committee node in this round; nodes with higher approval scores have a higher probability of being elected as committee nodes. The diversity of committee data is ensured through the replacement of committee nodes in each round.

[0012] The technical solution of this invention is as follows:

[0013] A blockchain-based Byzantine robust federated learning method employs a training node, wherein the training node belongs to the set of training nodes participating in the i-th round of training events, and the method includes:

[0014] Obtain the parameter data needed during the training of a model;

[0015] Committee node C was detected i After the i-th round of training events broadcast in the blockchain, based on the data from the committee node C... i The obtained global model parameters Mi and the parameter data required during model training are used to train the model, thus obtaining the model parameters. Where p represents the training node number, and the committee node Ci is the training node that participated in the i-th round of training events. The committee node C1 is elected from among the members. The initial committee node C1 is either the task initiating node that initiates the model training task or a training node designated by the task initiating node.

[0016] The model parameters Transmitted to the committee node C i So that the committee node C i Based on the model parameters obtained from local training and all received model parameters Calculate the set of training nodes participating in the (i+1)th round of training events and the global model parameters M. i+1 ;

[0017] Receive and verify the global model parameter M i+1The verification results are then broadcast to the blockchain to coordinate the training nodes participating in the i-th round of training events. The verification results yielded the global model parameters M. i+1 The accuracy rate;

[0018] In the global model parameter M i+1 If the accuracy is below a certain accuracy threshold, the training nodes jointly participating in the i-th round of training events... Re-election Committee Node C i And return to the model parameters. Transmitted to the committee node C i ;

[0019] In the global model parameter M i+1 If the accuracy exceeds a certain threshold and the training node does not belong to the set of training nodes participating in the (i+1)th round of training events, the training task is terminated.

[0020] In the global model parameter M i+1 If the accuracy is higher than a certain threshold, and the training node belongs to the set of training nodes participating in the (i+1)th round of training events, then let i = i+1, and return to the listening committee node C. i The i-th round of training events broadcast in the blockchain.

[0021] Furthermore, the committee node C i Based on the model parameters obtained from local training and all received model parameters Calculate the set of training nodes participating in the (i+1)th round of training events and the global model parameters M. i+1 ,include:

[0022] Based on the model parameters obtained from local training and any training node Model parameters Calculate the training node The degree of contribution;

[0023] In the set of training nodes participating in the i-th round of training events, training nodes whose contribution is less than a contribution threshold are filtered out. To obtain the set of training nodes participating in the (i+1)th round of training events;

[0024] For the set of training nodes participating in the (i+1)th round of training events, and in conjunction with the aforementioned contribution, adjust the corresponding model parameters. Perform aggregation to obtain the global model parameters M. i+1 .

[0025] Furthermore, the model parameters obtained based on local training and any training node Model parameters Calculate the training node Contribution

[0026] Calculate the model parameters obtained from the local training and the model parameters respectively. The modulus length;

[0027] The model parameters obtained during local training and the model parameters If the difference in modulus length is greater than a length threshold, the model parameters will be... The contribution level is set to 0;

[0028] The model parameters obtained from local training and the model parameters If the difference in modulus length is not greater than the length threshold, the model parameters are calculated using cosine similarity. With the model parameters Similarity;

[0029] The similarity is normalized to obtain the corresponding training nodes. The degree of contribution.

[0030] Furthermore, the training nodes that collaboratively participate in the i-th round of training events The verification results yielded the global model parameters M. i+1 The accuracy includes:

[0031] For each training node Obtain the total amount of data N in the validation set, the number of label types K, and the total amount of data corresponding to each label k;

[0032] Calculate the label weight for each label k

[0033] For each data point q in the validation set, obtain the true label y. q and in the global model parameter M i+1 Predicted labels

[0034] Calculate global model parameters M i+1 Corresponding accuracy Where [·] indicates that the value is 1 when the condition is true, and 0 otherwise. This represents the label weight wk corresponding to data q.

[0035] Furthermore, the basis from the committee node C i The obtained global model parameters M i The model is trained using the parameter data required during the model training process to obtain the model parameters. Previously, it also included:

[0036] Obtain the global model parameter M from the blockchain. i The hash value; wherein, the global model parameter M i The hash value is that of the committee node C i Uploaded to the blockchain;

[0037] In the global model parameter M i The hash value and the global model parameter M i In the corresponding case, jump to the method based on the committee node C. i The obtained global model parameters M i The model is trained using the parameter data required during the model training process to obtain the model parameters.

[0038] A blockchain-based Byzantine robust federated learning device, wherein the blockchain node corresponding to the device belongs to the set of training nodes participating in the i-th round of training events, the device comprising:

[0039] The data acquisition module is used to acquire the parameter data required during the training of a model.

[0040] The model training module is used to monitor committee node C. i After the i-th round of training events broadcast in the blockchain, based on the data from the committee node C... i The obtained global model parameters M i The model is trained using the parameter data required during the model training process to obtain the model parameters. Where p represents the training node number and C represents the committee node number. i It starts from the training node that participated in the i-th round of training events. The committee node C1 is elected from among the members. The initial committee node C1 is either the task initiating node that initiates the model training task or a training node designated by the task initiating node.

[0041] The parameter verification module is used to verify the model parameters. Transmitted to the committee node C i So that the committee node C i Based on the model parameters obtained from local training and all received model parameters Calculate the set of training nodes participating in the (i+1)th round of training events and the global model parameters M. i+1 ; Receive and verify the global model parameter M i+1 The verification results are then broadcast to the blockchain to coordinate the training nodes participating in the i-th round of training events. The verification results yielded the global model parameters M. i+1 The accuracy; in the global model parameter Mi+1 If the accuracy is below a certain accuracy threshold, the training nodes jointly participating in the i-th round of training events... Re-election Committee Node C i And return to the model parameters. Transmitted to the committee node C i ; in the global model parameter M i+1 If the accuracy exceeds a certain threshold and the training node does not belong to the set of training nodes participating in the (i+1)th round of training events, the training task is terminated; the global model parameters M... i+1 If the accuracy is higher than a certain threshold, and the training node belongs to the set of training nodes participating in the (i+1)th round of training events, then let i = i+1, and return to the listening committee node C. i The i-th round of training events broadcast in the blockchain.

[0042] A computer device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the method described in any of the preceding descriptions.

[0043] A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method described in any of the preceding claims.

[0044] A blockchain-based Byzantine robust federated learning system, the system comprising:

[0045] Blockchain;

[0046] A training node is used to acquire parameter data required during model training; wherein, when the training node belongs to the set of training nodes participating in the i-th round of training events, the training node is further used for:

[0047] Committee node C was detected i After the i-th round of training events broadcast in the blockchain, based on the data from the committee node C... i The obtained global model parameters M i The model is trained using the parameter data required during the model training process to obtain the model parameters. Where p represents the training node number and C represents the committee node number. i It starts from the training node that participated in the i-th round of training events. The committee node C1 is elected from among the members. The initial committee node C1 is either the task initiating node that initiates the model training task or a training node designated by the task initiating node.

[0048] The model parameters Transmitted to the committee node C i So that the committee node C i Based on the model parameters obtained from local training and all received model parameters Calculate the set of training nodes participating in the (i+1)th round of training events and the global model parameters M. i+1 ;

[0049] Receive and verify the global model parameter M i+1 The verification results are then broadcast to the blockchain to coordinate the training nodes participating in the i-th round of training events. The verification results yielded the global model parameters M. i+1 The accuracy rate;

[0050] In the global model parameter M i+1 If the accuracy is below a certain accuracy threshold, the training nodes jointly participating in the i-th round of training events... Re-election Committee Node C i And return to the model parameters. Transmitted to the committee node C i ;

[0051] In the global model parameter M i+1 If the accuracy exceeds a certain threshold and the training node does not belong to the set of training nodes participating in the (i+1)th round of training events, the training task is terminated.

[0052] In the global model parameter M i+1 If the accuracy is higher than a certain threshold, and the training node belongs to the set of training nodes participating in the (i+1)th round of training events, then let i = i+1, and return to the listening committee node C. i The i-th round of training events broadcast in the blockchain.

[0053] Compared with the prior art, the positive effects of the present invention are as follows:

[0054] 1) To address the problem that existing Byzantine fault-tolerant aggregation algorithms cannot resist poisoning attacks due to the untrustworthiness of the central server, this chapter proposes a decentralized federated learning method to prevent malicious servers from interfering with the federated training process.

[0055] 2) To address the issues of low efficiency and high overhead caused by storing model parameters on the blockchain, this invention proposes a scheme that uses off-chain storage of model parameters and on-chain storage of model parameter hashes to reduce on-chain storage costs, and uses an election committee for aggregation to reduce system computation and communication overhead.

[0056] 3) To address the difficulty in detecting malicious model updates under non-independent and identically distributed data, this chapter proposes a committee aggregation algorithm based on a feedback mechanism. This algorithm uses weighted aggregation based on committee evaluation and participant verification to ensure the global model is protected from malicious updates. Furthermore, to guarantee the algorithm's effectiveness on non-independent and identically distributed data, a new committee election will be conducted in each round to ensure the diversity of committee data. Attached Figure Description

[0057] Figure 1 This diagram illustrates the various attack risks inherent in the federated learning process of this invention.

[0058] Figure 2 This is the overall framework of the blockchain-based Byzantine robust federated learning method in this embodiment of the invention.

[0059] Figure 3 This is an execution flowchart of the blockchain-based Byzantine robust federated learning method according to an embodiment of the present invention.

[0060] Figure 4 This is a system architecture for blockchain-based Byzantine robust federated learning, as described in an embodiment of the present invention. Detailed Implementation

[0061] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0062] The present invention provides a blockchain-based Byzantine robust federated learning method. Figure 2 This provides the overall framework for the method. Figure 3 This is a workflow for a blockchain-based Byzantine robust federated learning method, which includes the following steps:

[0063] Step 1: Task Initialization Phase: Before training begins, the task initiator needs to define the deep learning model network structure to be trained, the format of the dataset required by the model, the hyperparameters for local model training, etc., and then propagate this information to each training node through the blockchain network.

[0064] Step 2: Initialization Phase of Training: At the start of a training round, the committee nodes (in the first round, the model provider, and thereafter, committee nodes will be elected through a committee selection algorithm) upload the hash values ​​of the current model parameters. Then, the corresponding training events are triggered via a blockchain smart contract.

[0065] Step 3: Model Federated Training Phase: On the training node side, upon receiving a corresponding training event, the training node will download the global model parameters from the committee node and verify the consistency of the model parameters by comparing them with the hash values ​​of the model parameters stored on the blockchain. Then, it will use its local dataset to perform the training task.

[0066] Step 4: Model Contribution Calculation Phase: After the training node completes its current training round, it uploads the hash value of the model parameters to the blockchain for storage and transmits the model parameters to the committee node. The committee node calculates the contribution value of the model trained by the training node in this round and stores the corresponding model contribution on the blockchain. These contribution values ​​will be beneficial for subsequent aggregation and the distribution of benefits among various participants after the training task is completed.

[0067] Step 5: Global Model Aggregation Phase: After determining the contribution of model updates from each training node, a contribution-based aggregation algorithm is used to aggregate the global model. Model updates with contribution values ​​below a set threshold are considered malicious updates and are rejected during aggregation to prevent poisoning attacks. For the remaining model updates, weighted aggregation is performed based on contribution values; updates with higher contribution values ​​have higher weights in the aggregation. After obtaining the latest global model, its hash value is uploaded to the blockchain for storage.

[0068] Step 6: Model Validation Phase: After the blockchain smart contract stores the global model hash value, it will trigger a model validation event. All training nodes in this round, upon hearing this event, will retrieve the global model from the committee node and use their own test sets to validate the global model. They will then upload their evaluations of the latest model. Only training nodes exceeding a specified threshold will consider the latest model not to have been poisoned, and the model aggregation for this round will then take effect. Otherwise, a new election of committee nodes will be conducted, and steps (4), (5), and (6) will be repeated. After the model aggregation takes effect, the committee nodes will still be replaced to ensure that the update of the global model is not affected by data from a single committee node.

[0069] All nodes participating in the task are called participant nodes. Participant nodes can play different roles, namely:

[0070] • Task Initiating Node: The task initiator is responsible for defining the deep learning model network structure required during training, the format of the dataset needed for the model, the hyperparameters for local model training, etc. The entire federated learning task is started by the task initiator, who also sets the termination conditions. The termination conditions can be set as the model's accuracy reaching a certain level or after a certain number of training rounds.

[0071] • Committee Nodes: Committee nodes are responsible for aggregating model updates from all training nodes. During training, they wait for local model updates from all training nodes and calculate the contribution value for each training node's model update during the model contribution calculation phase. Then, in the global model aggregation phase, the corresponding aggregation algorithm is used to aggregate the model and obtain the latest global model. The parameter hash value of this global model is uploaded to the blockchain, and then the committee node broadcasts the global model to the next round of training nodes to start a new round of training. In traditional federated learning systems, the central server is responsible for aggregation and remains unchanged. However, in the system of this invention, the committee node initially acts as the task initiator node, and then a new committee node is elected before the start of each round to be responsible for the aggregation and contribution value calculation for that round, thereby ensuring the reliability of the aggregation nodes and the fairness of the contribution value calculation.

[0072] • Training Nodes: Each training node contributes its data value through federated learning. During model initialization, each participating node obtains the task defined by the task initiator through the blockchain network. Then, in each training round, it obtains the latest global model parameters from the committee nodes and performs local model training based on the hyperparameters set for the task, such as the number of epochs, optimizer, and learning rate. After each training round, the model's hash value is uploaded to the blockchain to record the federated model training process. The actual model parameters are sent to the committee nodes off-chain. At the end of each training round, to prevent committee nodes from cheating by uploading malicious global models, each training node uses its own validation set to evaluate the global model update for that round. Only training nodes exceeding a specified threshold consider the latest global model not to have been poisoned, and the aggregation result for that round is valid. The evaluation of the global model is recorded on the blockchain as a basis for committee election.

[0073] The blockchain used in this invention is a consortium blockchain. The openness of a consortium blockchain lies between that of a public blockchain and a private blockchain. The access restrictions and identity authentication of a consortium blockchain help achieve accountability in federated learning and improve system security. Each blockchain node stores a local copy of the complete transaction data in the form of blocks. Even if some nodes are damaged, the operation of the entire system will not be affected. Compared to the centralized server architecture used in traditional federated learning, this improves system reliability and prevents the concentration of power through a decentralized design. The blockchain uses smart contracts to ensure that the rules for computation, uploading, and coordination in federated learning are uniformly executed within the network, eliminating the need for a third-party institution to ensure trust relationships between nodes. In each round of the federated learning training process, the global and local model parameters transmitted between nodes are stored locally on each node. Simultaneously, the blockchain stores the hash values ​​of the model parameters. Although only the hash values ​​of the models are recorded on the chain, it still ensures that the records of on-chain and off-chain models are immutable and transparent to stakeholders. Off-chain models can be verified using the hash values ​​on the chain, thus enabling model traceability. Smart contracts run on the blockchain, and each round of committee nodes interacts with the training nodes through smart contracts. Furthermore, the smart contract retains the participant node's identity, current task ID, and information such as model update hash, aggregated model hash, current training round number, and approval rating for each training round. The smart contract will provide the following functions: node registration management, task information management, round-by-round information management, model update submission and query, approval rating submission and query, and aggregated submission and query.

[0074] During the model contribution calculation phase, to defend against poisoning attacks, training nodes are responsible for their own updates. Committee nodes calculate the contribution of all updates, identifying potentially malicious training nodes. Furthermore, updates with high contributions are assigned higher weights in this round of aggregation. This mechanism encourages training nodes to actively use their own good datasets for training. From the committee node's perspective, its model training is honest; therefore, model updates similar to its own training results will improve the model's accuracy on its own dataset. Thus, the calculation of model update contribution depends on the similarity between the update and its own training results. Updates with model parameters closer to the committee node in this round will receive higher scores. Updates that differ significantly from its own model updates will be suspected of being malicious and will be ignored in this round of aggregation. To prevent good updates from being consistently assigned low weights and failing to realize their data value when there are no poisoning attacks, due to the non-independent and identically distributed nature of the data, committee nodes are replaced in each round. This allows the global model to extract as many effective features as possible from diverse data, improving the global model's generalization ability. Unlike existing federated learning scoring algorithms, the method in this chapter does not require a centralized test dataset or a trusted third-party institution to act as a scorer. Committee nodes consider their own updates to be normal when filtering model updates, assuming they will train their models using the prescribed training algorithm and their own dataset. Therefore, committee nodes believe that the model parameters they train are beneficial to the global model update, and thus, they will use their own trained model updates as the standard for a two-stage contribution evaluation. In the first stage, committee nodes filter model updates that differ significantly from their own, judging this by comparing the magnitude of the model parameters. If the magnitude of a participant's model update is too large or too small compared to their own, it is considered a malicious update and is filtered out in the first stage. In the second stage, the similarity between their own model update and the remaining model updates is calculated using cosine similarity, and the normalized similarity is used as the contribution. The contribution calculation formula is as follows:

[0075]

[0076] Where, x c x represents the updated parameters of the model obtained from training the committee nodes. k This represents the model update obtained from training at the training nodes, where n represents the total number of parameters in the model. Then, a weighted aggregation of the global model is performed based on the proportion of contribution.

[0077] During the model validation phase, if a committee node is malicious or a malicious model update bypasses the committee node's detection, continued iteration would cause the model training process to fail to converge or affect the final model accuracy. In this case, each training node can collaborate to validate the global model using its own model updates and private validation sets to enhance the robustness of federated training. Each training node will validate the global model after one round of global model aggregation and upload its decision on whether to agree to the aggregation to the blockchain. Due to the diversity of training node data, adding a training node model validation step is equivalent to having a centralized validation dataset, thereby increasing the probability of detecting malicious model updates. Furthermore, since the validation step only needs to upload whether the model aggregated in this round has been poisoned and does not need to upload the dataset, there is no risk of leaking the data privacy of training nodes during the model validation phase. The validation of the global model by training nodes is divided into two phases. In the first phase, the training node will judge the global model that differs too much from its own model update as malicious by comparing the magnitude of the model parameters, and then reject the global model update in this round. In the second phase, each training node uses its own validation set to test the accuracy of the global model. Only if the calculated model accuracy is not lower than a set threshold will the current aggregation update be approved. This threshold depends on the accuracy of the global model evaluation in the previous round. Each participant node contains its own dataset, which must not be disclosed. Participants divide their local datasets into two parts: a training set and a validation set. The validation set is used for model validation triggered after a round of model aggregation. At this time, each training node needs to download the global model and validate it using its own validation set. The validation set cannot participate in model training. However, due to the non-independent and identically distributed nature of the data, the data distribution of each participant is different, resulting in significant differences in the number of samples in each category within the validation set. Because the number of labels in the validation set is uneven, using the validation set to evaluate model accuracy can lead to models that are good at identifying categories with a large number of labels for their own samples having higher evaluation accuracy, thus making the model validation inaccurate. To improve the fairness of node evaluation, this invention uses the validation dataset for evaluation and assigns weights based on the proportion of sample categories. Specifically, categories with a larger proportion in the validation set are assigned lower weights, while categories with a smaller proportion are assigned higher weights. The accuracy calculation method is as follows:

[0078]

[0079]

[0080] Where N represents the total amount of data in the validation set, K represents the number of tag types, and n k w represents the number of k-tags. kRepresents the weight, y i This represents the true label of the i-th sample. This represents the label predicted by the model for the i-th sample. The value is 1 if the condition is true, and 0 otherwise. Training nodes determine if the model is poisoned by comparing the evaluation accuracy of the global model obtained in this round of aggregation with that of the previous round. Aggregation of the current model only takes effect if a training node that exceeds a set threshold considers the latest global model not to be poisoned. Otherwise, a new committee node will be elected to re-aggregate the model for this round of training updates.

[0081] Based on the same concept, this invention also provides a blockchain-based Byzantine robust federated learning system, which can be based on... Figure 4 The system shown primarily comprises a task initiation module, a blockchain module, a blockchain connection module, a participant network module, and a private data module. The specific functions of each module are as follows:

[0082] • Task Initiator: To facilitate task initiators in creating and submitting tasks without needing to concern themselves with underlying details such as federated learning and privacy protection, allowing them to focus on their business needs, this system includes a task publishing module. This module provides a user-friendly, white-screen interface on the front end for task initiators to write tasks. In the corresponding backend service, the service parses the relevant parameters, constructs the federated learning task, and uploads it to the blockchain network. Participating nodes can download the task, parse it, and choose whether to participate in the federated learning task.

[0083] • Blockchain: The system is built using the Hyperledger Fabric consortium blockchain and corresponding smart contracts have been developed to provide node registration management, task information management, management of each round of training information, model update submission and query, approval submission and query, and aggregation submission and query functions. The blockchain, through smart contracts, ensures that the rules for computation, uploading, and coordination of federated learning within the network are uniformly executed, eliminating the need for a third-party institution to ensure trust relationships between nodes.

[0084] • Blockchain Connection Module: This module is responsible for communicating with the blockchain to obtain information and upload updates such as training model updates. It maintains the identity certificate information of previously registered users for connecting to the blockchain network and provides corresponding API interfaces to package participant requests into transactions and send them to the blockchain. This module also subscribes to events on the blockchain; whenever an event is received, the program parses its payload and publishes it to subscribers based on the event type.

[0085] • Participant Network Module: This module is responsible for node communication, task execution, committee election, model validation for training nodes, and model aggregation and evaluation for committee nodes. Node communication provides a web server and client for receiving and sending requests to other nodes. When a participant acts as a trainer node, they use the aggregation client in the aggregation module for model transfer and key supply. When a participant acts as a committee node, they use the aggregation server for model aggregation and then global model distribution. Task execution is responsible for parsing and executing tasks, setting the corresponding model and training parameters based on the parsed parameters.

[0086] Then, the PyTorch framework is used for model training and evaluation. The committee election is responsible for conducting committee elections and voting.

[0087] After each training round, each training node elects a committee node for the next round based on a voting algorithm. When a training node receives a specified threshold of votes, it is successfully elected and begins serving as a committee node for the next round, initiating the next training round via a smart contract. Model validation, model aggregation, and model evaluation are implemented using a committee aggregation algorithm based on a feedback mechanism.

[0088] • Private Data Module: This module is responsible for storing and managing the data of each participant node.

[0089] The following example illustrates the implementation process using n participant nodes that need to coordinate the training of a machine learning model:

[0090] Step 1: Build a blockchain network using a consortium blockchain. After establishing the corresponding channel on the consortium blockchain, the first step is node registration. Participating nodes will use the blockchain connection module to register on the smart contract of this channel using their corresponding user identities. The registration information mainly includes the node name and URL, which will be used for communication between nodes.

[0091] Step 2: To begin a federated learning task, the task publisher will write the task description. Using the white-screen task creation page provided by the task publishing module, the task publisher can quickly publish a federated learning task by selecting the task name, neural network model, model training parameters, and federated learning parameters according to their needs.

[0092] Step 3: After the task is published, the blockchain will generate a task publication event. All participating nodes that have joined this channel will receive this event and can then choose whether to join the task. If enough participating nodes participate, the task will start normally; otherwise, it will result in a task anomaly. Next, multiple rounds of federated learning training will begin. Each round of training consists of several stages: training initialization, model federated training, model contribution calculation, global model aggregation, and model validation.

[0093] Step 3.1: During the training initialization phase, the committee node (in the first round, the model provider, and thereafter, committee nodes will be elected through a committee selection algorithm) will upload the hash values ​​of the current model parameters to the blockchain. Then, the corresponding training event is triggered via the blockchain smart contract. Other participating nodes will download the global model parameters from the committee node and compare them with the on-chain model parameter hashes for verification.

[0094] Step 3.2: In the federated training phase of the model, the training nodes use their own local datasets for training tasks.

[0095] The obtained model parameter hash value is then uploaded to the blockchain, and the model parameters are then transmitted to the committee node.

[0096] Step 3.3: Model Contribution Calculation Phase. The committee nodes will evaluate the contribution of the models trained by the nodes in this round of training. First, some model parameters will be filtered using Euclidean distance, and then cosine distance will be used to calculate the contribution of each participating node's model parameters to the committee's model parameters.

[0097] Step 3.4: Global Model Aggregation Phase. After determining the contribution of each training node's model update, a contribution-based aggregation algorithm is used to aggregate the global model. Model updates with contribution values ​​below a set threshold are considered malicious updates and are rejected during aggregation to prevent poisoning attacks. For the remaining model updates, weighted aggregation is performed based on contribution values; updates with higher contribution values ​​have higher weights in the aggregation. After obtaining the latest global model, its hash value is uploaded to the blockchain for storage.

[0098] Step 3.5: Model Validation Phase. After the blockchain smart contract stores the global model hash value, it will trigger a model validation event. All training nodes in this round, upon hearing this event, will retrieve the global model from the committee node and use their own test sets to validate the global model. They will then upload their evaluations of the latest model. Only training nodes exceeding a specified threshold will consider the latest model not to have been poisoned, and the model aggregation for this round will then take effect. Otherwise, a new committee node election will be conducted, and the above steps will be repeated. Even after model aggregation takes effect, committee node replacement will continue to ensure that the update of the global model is not solely influenced by data from a single committee node.

[0099] Step 4: After the federated learning task training is completed, the task status changes to "completed," and the task issuer can click on the corresponding task to download the final global model parameters. Each participating node can view the current blockchain operation status through a blockchain explorer. The explorer can monitor the Fabric network status in real time, including the number of transactions, block height, and chaincode data. It also provides detailed information on each block and transaction during the blockchain-based federated learning process, thereby improving the transparency and credibility of the training process and making it traceable and verifiable.

[0100] In summary, this invention proposes a blockchain-based Byzantine robust federated learning method and designs a corresponding system. This method eliminates the problem of untrustworthy central servers in federated learning and makes the federated learning process open, transparent, and traceable. To prevent malicious nodes from poisoning attacks, a committee aggregation algorithm based on a feedback mechanism is proposed. This algorithm selects committee nodes to verify the model data in the current round and determines the aggregation strategy based on their contributions. At the end of each training round, verification by the participants helps resist poisoning attacks from malicious nodes. This system facilitates data collaboration among different organizations, leverages data value, and has high practicality.

[0101] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A blockchain-based Byzantine robust federated learning method, characterized in that, The method comprises the following steps: Obtaining parameter data required in a model training process; The committee node C is listened to i After the i-th round of training events broadcast in the blockchain, based on the committee node C i The global model parameters M i and the parameter data required in the model training process are used for model training to obtain model parameters Where p represents the number of training nodes, the committee node C i is elected from the training nodes participating in the i-th round of training events The initial committee node C1 is the task initiating node initiating the model training task or a training node designated by the task initiating node; The model parameters Transmitted to the committee node C i So that the committee node C i Based on the model parameters obtained from local training and all received model parameters Calculate the set of training nodes participating in the (i+1)th round of training events and the global model parameters M. i+1 ; wherein, the committee node C i Based on the model parameters obtained from local training and all received model parameters Calculate the set of training nodes participating in the (i+1)th round of training events and the global model parameters M. i+1 ,include: based on the model parameters obtained by local training and model parameters of any training node based on the model parameters obtained by local training and model parameters of any training node based on the model parameters obtained by local training and model parameters of any training node based on the model parameters obtained by local training and model parameters of any training node In the training node set participating in the i-th round of training events, the training nodes with the contribution degree less than a contribution threshold are screened out to obtain a training node set participating in the i+1-th round of training events; For the training node set participating in the i+1th round of training events, in combination with the contribution degree, the corresponding model parameters are aggregated to obtain the global model parameters M i+1 ;​ receiving and verifying the global model parameters M i+1 and broadcasting the verification result into the blockchain to coordinate the training nodes participating in the i-th round of training events the verification result of the training nodes participating in the i-th round of training events, obtaining the global model parameters M i+1 accuracy rate; In the case where the accuracy of the global model parameter M i+1 , the training node participating in the i-th round of training event re-elects the committee node C i , and returns to the transmission of the model parameter M to the committee node C i ; In the global model parameter M i+1 If the accuracy is higher than the accuracy threshold and the training node does not belong to the training node set participating in the i+1th round of training events, the training task is exited. In the global model parameter M i+1 If the accuracy is higher than a threshold and the training node belongs to the training node set participating in the i+1th round of training events, then let i=i+1, and return to the committee node C i The i-th round of training events broadcast in the blockchain.

2. The method of claim 1, wherein, The model parameters obtained based on local training and any training node Model parameters Calculate the training node The contributions include: respectively, and the model parameters obtained by the local training length of the module; In a case where a difference between a length of the model parameter and a length of the model parameter obtained by the local training is greater than a length threshold, a contribution degree of the model parameter is set to 0. In a case where a difference between a length of the model parameter obtained by the local training and a length of the model parameter is not greater than the length threshold, the similarity between the model parameter and the model parameter is calculated by using a cosine similarity. normalizing the similarity to obtain a contribution degree of the corresponding training node .

3. The method of claim 1, wherein, The training node participating in the i-th round of training events cooperatively The verification result of the global model parameter M i+1 The accuracy rate, comprising: For each training node Obtaining the total data amount N of the validation set, the label category number K, and the total data amount corresponding to each label k. calculating a label weight for each label k For each data q in the validation set, obtain the true label y q and the predicted label i+1 under the global model parameters M Compute global model parameters M i+1 Corresponding accuracy where [·] denotes the value is 1 if the condition is true, otherwise 0, denotes the label weight w corresponding to data q k .

4. The method according to any one of claims 1 to 3, characterized in that, The global model parameter M is obtained based on the committee node C i The global model parameter M is obtained based on the committee node C i The model parameter is obtained by performing model training based on the parameter data required in the model training process Before, also comprising: obtaining the global model parameters M from the blockchain i ; wherein the hash value of the global model parameters M i is uploaded to the blockchain by the committee nodes C i ; In the case that the hash value of the global model parameter M i corresponds to the global model parameter M i obtained from the committee node C i , the model training is performed based on the global model parameter M i and the parameter data required in the model training process, and the model parameter M 5. A blockchain-based Byzantine robust federated learning device, characterized in that, The device corresponds to a blockchain node belonging to a training node set participating in an i-th round of training events, and the device comprises: A data acquisition module is configured to obtain parameter data required in a model training process; A model training module is configured to listen to committee nodes C i After the i-th round of training events broadcast in the blockchain, based on the committee nodes C i The global model parameters M i And the parameter data required in the model training process, the model training is carried out to obtain the model parameters Wherein, p represents the number of training nodes, the committee nodes C i Is elected from the training nodes participating in the i-th round of training events The initial committee node C1 is the task initiating node initiating the model training task or a training node designated by the task initiating node; The parameter verification module is used to verify the model parameters. Transmitted to the committee node C i So that the committee node C i Based on the model parameters obtained from local training and all received model parameters Calculate the set of training nodes participating in the (i+1)th round of training events and the global model parameters M. i+1 ; Receive and verify the global model parameter M i+1 The verification results are then broadcast to the blockchain to coordinate the training nodes participating in the i-th round of training events. The verification results yielded the global model parameters M. i+1 The accuracy; in the global model parameter M i+1 If the accuracy is below a certain accuracy threshold, the training nodes jointly participating in the i-th round of training events... Re-election Committee Node C i And return to the model parameters. Transmitted to the committee node C i ; in the global model parameter M i+1 If the accuracy exceeds a certain threshold and the training node does not belong to the set of training nodes participating in the (i+1)th round of training events, the training task is terminated; the global model parameters M... i+1 If the accuracy is higher than a certain threshold, and the training node belongs to the set of training nodes participating in the (i+1)th round of training events, then let i = i+1, and return to the listening committee node C. i The i-th round of training events broadcast in the blockchain; Wherein, the committee node C i Based on the model parameters obtained by local training and all received model parameters Calculate the set of training nodes participating in the i+1th round of training events and the global model parameters M i+1 , comprising: Based on the model parameters obtained from local training and any training node Model parameters Calculate the training node The degree of contribution; In the training node set participating in the i-th round of training events, the training nodes with the contribution degree less than a contribution threshold are screened out to obtain a training node set participating in the i+1-th round of training events; For the training node set participating in the i+1th round of training events, in combination with the contribution degree, the corresponding model parameters are aggregated to obtain the global model parameters M . i+1 .

6. A computer device, comprising: The computer device comprises a processor and a memory storing computer program instructions; and the processor executes the computer program instructions to implement the method of any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer program instructions, and the computer program instructions are executed by the processor to implement the method of any one of claims 1-4. 8.A blockchain-based Byzantine robust federated learning system, characterized in that, The system comprises: A blockchain; A plurality of training nodes configured to obtain parameter data required in a model training process; wherein, in the case that the training node belongs to a training node set participating in an i-th round of training events, the training node is further configured to: The committee node C is listened to i After the i-th round of training events broadcast in the blockchain, based on the committee node C i The global model parameters M i And the parameter data required in the model training process, the model parameters are obtained by model training Where p represents the number of training nodes, the committee node C i Is elected from the training nodes participating in the i-th round of training events The initial committee node C1 is the task initiating node initiating the model training task or a training node designated by the task initiating node; The model parameters Transmitted to the committee node C i So that the committee node C i Based on the model parameters obtained from local training and all received model parameters Calculate the set of training nodes participating in the (i+1)th round of training events and the global model parameters M. i+1 ; wherein, the committee node C i Based on the model parameters obtained from local training and all received model parameters Calculate the set of training nodes participating in the (i+1)th round of training events and the global model parameters M. i+1 ,include: Based on the model parameters obtained from local training and any training node Model parameters Calculate the training node The degree of contribution; In the training node set participating in the i-th round of training events, the training nodes with the contribution degree less than a contribution threshold are screened out to obtain a training node set participating in the i+1-th round of training events; For the training node set participating in the i+1th round of training events, in combination with the contribution degree, the corresponding model parameters are aggregated to obtain the global model parameters M i+1 ;​ receiving and verifying the global model parameters M i+1 and broadcasting the verification result into the blockchain to coordinate the training nodes participating in the i-th round of training events the verification result of the i-th round of training events, obtaining the global model parameters M i+1 accuracy rate; In the case where the accuracy of the global model parameters M i+1 is lower than an accuracy threshold, the training nodes participating in the i-th round of training events re-elect the committee nodes C i , and return to the transmission of the model parameters M to the committee nodes C i ; In the global model parameter M i+1 If the accuracy is higher than the accuracy threshold and the training node does not belong to the training node set participating in the i+1th round of training events, the training task is exited. In the global model parameter M i+1 If the accuracy is higher than a threshold and the training node belongs to the training node set participating in the i+1th round of training events, then let i=i+1, and return to the committee node C i The i-th round of training events broadcast in the blockchain; A blockchain connection module is configured to communicate and connect the blockchain and the training node.

Images