Byzantine robustness federated learning method based on block chain

Abstract

The invention discloses a Byzantine robustness federated learning method based on a block chain. The method comprises the steps of 1, initializing an initial global model; 2, m clients are selected from the n clients to serve as training clients; step 3, each training client obtains a current global model from the block chain, and performs local model training by using a local data set; step 4, the training client updates and uploads each obtained local model to the block chain; 5, the intelligent contract aggregates the local models uploaded in the step 4 according to an aggregation algorithm; step 6, distributing the global model obtained by aggregation to each verification client for verification; 7, the verification client verifies the received global model and returns a verification result to the block chain; 8, if the verification is passed, recording the current global model on the block chain; and if the verification is not passed, returning to the step 5 to re-aggregate the local model.

Description

Technical Field

[0001] The present invention relates to the field of federated learning security technology, and in particular to a Byzantine robust federated learning method based on blockchain. Background Art

[0002] As artificial intelligence (AI) technology continues to advance, it is demonstrating significant vitality across diverse frontiers. However, the current development of AI is constrained by data. On the one hand, the emergence of data silos and privacy issues has hindered the development of traditional AI technologies, and big data processing methods have encountered bottlenecks. On the other hand, the massive amounts of data held by institutions, enterprises, and organizations hold enormous potential for application. Consequently, federated learning, a technology designed to protect privacy and data security, has emerged. Federated learning is a distributed learning algorithm that effectively addresses the data silo problem. In federated learning, clients train local models locally and upload these updates to a central server, which then aggregates the local model updates to form a global model. However, federated learning presents security issues, making it vulnerable to malicious client attacks. To mitigate Byzantine attacks, numerous Byzantine-robust aggregation rules and anomaly detection mechanisms have been proposed. However, experiments have shown that these defenses are still ineffective against Byzantine attacks.

[0003] As a decentralized, immutable, and shared distributed ledger and database, blockchain offers many advantages suitable for research related to federated learning. Blockchain is a decentralized, distributed database where all nodes in the network have equal rights. This avoids the potential for data leakage in centralized systems and strengthens privacy and data security. The information in the blockchain is tamper-proof and traceable, which, to a certain extent, prevents malicious nodes from damaging the system. Summary of the Invention

[0004] To achieve this objective, this paper proposes a blockchain-based Byzantine-robust federated learning framework, combining blockchain and federated learning to defend against Byzantine attacks. The core idea is to leverage the immutability and traceability of blockchain information to store relevant information, including local model gradients and the aggregated global model, uploaded by training clients during each round. The framework then continuously backtracks the intermediate aggregation process based on the verification results of the current global model from the verification clients to obtain the optimal global model.

[0005] The technical solutions adopted in the present invention are as follows:

[0006] A Byzantine robust federated learning method based on blockchain, characterized by comprising the following steps:

[0007] Step 1: Initialize the initial global model on the blockchain;

[0008] Step 2: The blockchain selects m clients from n clients as training clients, v clients as training clients, and the remaining clients as idle clients;

[0009] Step 3: Each training client obtains the current global model from the blockchain and uses the local dataset of the training client to perform local model training to obtain a trained local model;

[0010] Step 4: The training client uploads each local model update to the blockchain;

[0011] Step 5: Update and aggregate the local model uploaded in step 4 according to the aggregation algorithm recorded in the smart contract on the blockchain to obtain the global model;

[0012] Step 6: Distribute the global model obtained by aggregation in step 5 to each verification client for verification;

[0013] Step 7: The verification client verifies the received global model based on its own local dataset and returns the voting results to the blockchain;

[0014] Step 8: Determine the voting results received. If more than half of the clients vote to accept the current global model, the verification is successful and the current global model is recorded on the blockchain. Otherwise, the verification fails and the process returns to step 5.

[0015] Step 9: Repeat steps 2 to 8 until convergence, and output the final global model.

[0016] Furthermore, the aggregation algorithm described in step 5 includes the following steps:

[0017] Step 51: Calculate the Euclidean distances between the local model update of each training client and the local model updates of other training clients and sum them up to obtain the sum of the Euclidean distances of the local models of each training client;

[0018] Step 52: Sort the obtained Euclidean distance sums from small to large, and score the local model updates of each training client based on the size of the Euclidean distance sums. The smaller the Euclidean distance sum, the smaller the score, indicating that it is more likely to be a local model update of an honest client. Conversely, the larger the Euclidean distance sum, the higher the score, indicating that it is more likely to be a model update of a malicious client.

[0019] Step 53: Delete the sorted Euclidean distances and the largest K local model updates;

[0020] Step 54: The remaining local model updates are weighted and summed according to their scores to obtain a new global model.

[0021] Furthermore, the formula for calculating the Euclidean distance in step 51 is:

[0022]

[0023] in, Represents a local model update.

[0024] Furthermore, the scoring formula in step 52 is:

[0025]

[0026] in, Update the score of the client local model for round t of training.

[0027] The beneficial effects of the present invention are:

[0028] First, this invention combines blockchain with federated learning. By making the information in the blockchain immutable and traceable, it prevents malicious aggregation by a central server, thereby enhancing privacy protection and data security.

[0029] Second, the present invention verifies the obtained global model through a verification mechanism to ensure that the model can still learn an accurate global model even when it is attacked by malicious attacks, effectively resisting various malicious attacks, thereby improving the robustness of federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The flowchart of the method proposed in the present invention is shown in FIG.

[0031] Figure 2 This is the flow chart of the aggregation rule algorithm.

[0032] Figure 3 This is a comparison chart of the results of different aggregation methods under untargeted attack in the embodiment.

[0033] Figure 4 This is a comparison chart of the results of different aggregation methods under label flipping attack in the embodiment. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0035] This paper addresses the Byzantine attack problem in federated learning and, in combination with blockchain technology, provides a blockchain-based Byzantine robust federated learning method, which specifically includes the following steps:

[0036] Step 1: Initialize the global model;

[0037] Step 2: Randomly select m clients from n clients as training clients, select v clients as training clients, and the rest as idle clients. Each training client obtains the initialized global model from the blockchain;

[0038] Step 3: Each training client uses the current global model obtained on the local dataset to train a local model;

[0039] Step 4: Each training client uploads the trained local model update to the blockchain;

[0040] Step 5: Calculate the Euclidean distance between each training client's local model update and other training client's local model updates and sum them to obtain the corresponding Euclidean distance sum. The Euclidean distance calculation formula is:

[0041]

[0042] in, represents a local model update;

[0043] Step 6: Sort the obtained Euclidean distance sums from small to large, and score the local model updates of each training client based on the size of the Euclidean distance sums. The smaller the Euclidean distance sum, the lower the score, indicating that the local model update is more similar to other local model updates and is more likely to be a local model update of an honest client. The larger the Euclidean distance sum, the higher the score, indicating that the model update is more likely to be a model update of a malicious client.

[0044] The score is calculated as follows:

[0045]

[0046] in, Update the score of the client local model for round t of training;

[0047] Step 7: Remove the K largest local model gradients corresponding to the sorted Euclidean distances. The value of K is determined by the number of malicious clients and is generally set to 1 or 2.

[0048] Step 8: Perform weighted summation of the remaining local model gradients according to the scores and aggregate them to obtain a new global model;

[0049] Step 9: Distribute the new global model obtained by the current aggregation to the verification client for verification;

[0050] Step 10: The verification client verifies the current global model based on its own dataset. If the verification accuracy of the global model trained on its own test set is high, the model is considered to have passed the verification. Otherwise, the model fails and the verification result is returned to the blockchain.

[0051] Step 11: If the new global model fails verification, the current global model is discarded, and the sorted Euclidean distances and the largest K local model gradients are removed from the remaining local model gradients. The new global model is obtained by aggregation and then verified again until the current global model passes the verification of the verification client.

[0052] Step 12: The verified current global model is recorded on the blockchain, completing the training of this round of global model;

[0053] Step 13: Repeat steps 2 to 12 for t rounds, and finally output the optimal global model.

[0054] Example

[0055] In order to further verify the effect and performance of the method proposed in the present invention, the following simulation experiments were carried out.

[0056] 1. Experimental Setup

[0057] In this experiment, a total of 100 clients are set up, and 10 clients are selected as training clients and 5 clients are selected as verification clients each time. Twenty percent of the clients are set as malicious clients.

[0058] 2. Experimental process

[0059] The proposed method was compared with the FedAVG aggregation method. In the absence of an attack, the FedAVG method achieved an accuracy rate of 96.04%, while the proposed method achieved an accuracy rate of 95.03%. The proposed method was also compared with the classic Byzantine robust aggregation rule, Trim-mean. The test results are shown in Table 1.

[0060] Table 1

[0061]

[0062]

[0063] As shown in Table 2-3, in the case of an untargeted attack with 20% malicious clients, FedAvg's accuracy is 84.59, while our method's accuracy is 95.33. In the case of a label flipping attack with 20% malicious clients, FedAvg's accuracy is 83.24, while our method's accuracy is 94.50.

[0064] As shown in Tables 2 and 3, this experiment also compared the model accuracy of the two aggregation rules, Trim-mean and FedAvg (without attack), for different attacks when there are different proportions of malicious clients. It can be seen that this method has a clear advantage when the proportion of malicious clients reaches about 25%.

[0065] Table 2

[0066]

[0067] Table 3

[0068] Label flipping attack FedAvg w / o attacks Trim-mean BRFL 0 96.04 95.04 95.03 5 96.04 94.22 94.58 10 96.04 94.44 94.47 15 96.04 94.53 94.21 20 96.04 90.71 94.50 25 96.04 90.50 94.47 30 96.04 76.45 94.33 35 96.04 72.87 94.68 40 96.04 43.66 94.85 45 96.04 27.99 94.64

[0069] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A Byzantine robust federated learning method based on blockchain, characterized by: The following steps are involved: Step 1: Initialize the initial global model on the blockchain; Step 2: The blockchain selects m clients from n clients as training clients, v clients as training clients, and the remaining clients as idle clients; Step 3: Each training client obtains the current global model from the blockchain and uses the local dataset of the training client to perform local model training to obtain a trained local model; Step 4: The training client uploads each local model update to the blockchain; Step 5: Update and aggregate the local model uploaded in step 4 according to the aggregation algorithm recorded in the smart contract on the blockchain to obtain the global model; Step 6: Distribute the global model obtained by aggregation in step 5 to each verification client for verification; Step 7: The verification client verifies the received global model based on its own local dataset and returns the voting results to the blockchain; Step 8: Determine the voting results received. If more than half of the clients vote to accept the current global model, the verification is successful and the current global model is recorded on the blockchain. Otherwise, the verification fails and the process returns to step 5. Step 9: Repeat steps 2 to 8 until convergence, and output the final global model.

2. A blockchain-based Byzantine robust federated learning method according to claim 1, characterized in that: The aggregation algorithm described in step 5 includes the following steps: Step 51: Calculate the Euclidean distances between the local model update of each training client and the local model updates of other training clients and sum them up to obtain the sum of the Euclidean distances of the local models of each training client; Step 52: Sort the obtained Euclidean distance sums from small to large, and score the local model updates of each training client based on the size of the Euclidean distance sums. The smaller the Euclidean distance sum, the smaller the score, which means it is more likely to be a local model update of an honest client. Conversely, the larger the Euclidean distance sum, the higher the score, which means it is more likely to be a model update of a malicious client. Step 53: Delete the sorted Euclidean distances and the largest K local model updates; Step 54: The remaining local model updates are weighted and summed according to their scores to aggregate them into a new global model.

3. The blockchain-based Byzantine robust federated learning method according to claim 2, characterized in that: The formula for calculating the Euclidean distance in step 51 is: in, Represents a local model update.

4. A blockchain-based Byzantine robust federated learning method as claimed in claim 3, characterized in that: The scoring formula in step 52 is: in, Update the score of the client local model for round t of training.

Images