A blockchain-based privacy protection asynchronous federated learning method

Abstract

The present application is directed to the challenges of low efficiency, vulnerable to malicious attacks and single point failure faced by federated learning, and proposes a privacy protection asynchronous federated learning framework based on blockchain to ensure the required security and efficiency. Federated learning allows model training on multiple devices without transmitting data to a central server, effectively protecting data privacy. Blockchain ensures that model data cannot be tampered with, solves single point failure and untrusted aggregation; asynchronous learning accelerates global aggregation; differential privacy can improve the robustness of the framework. At the same time, the entropy weight method, an objective weighting method, is used to evaluate the credit of the consensus node, effectively preventing malicious behavior. A large number of experimental results show that, compared with existing schemes, the framework proposed by the present application has higher efficiency and performance, and can effectively guarantee the accuracy of the system.

Description

Technical Field

[0001] This invention relates to the fields of blockchain and federated learning, specifically a privacy-preserving asynchronous federated learning method based on blockchain. Background Technology

[0002] Federated learning allows multiple participants to collaborate on machine learning tasks without directly exchanging data, thus protecting data privacy to some extent. However, traditional federated learning still faces many challenges in web applications. Firstly, there are security issues. Although federated learning doesn't directly share raw data, model updates may contain sensitive information that could be inferred by malicious attackers, leading to data breaches and privacy problems. Furthermore, federated learning typically relies on a central server to aggregate models, which not only increases the risk of single points of failure but also risks corrupting the entire model due to malicious behavior on the central server. Secondly, there are efficiency issues. The synchronous updates in traditional federated learning require all devices to update in every training round, which can lead to low overall learning efficiency due to uneven device performance or network latency.

[0003] Blockchain technology boasts superior data security due to its decentralized, transparent, and immutable characteristics. As a distributed learning architecture, federated learning can provide a foundation for integrating blockchain technology. To enhance security, blockchain is becoming an important complementary technology to federated learning. By using blockchain technology, the failure of a single central server can be prevented, ensuring decentralized and secure data storage. Combining blockchain technology with federated learning not only solves the problems of centralized aggregation but also improves system transparency and reliability, reducing the risk of attacks. Differential privacy technology protects the privacy of local model data by adding noise during model updates, preventing malicious attackers from inferring private information of participating devices through inference attacks. Asynchronous federated learning allows each device to independently perform local training and updates without waiting for other devices to complete, thus reducing the latency issues caused by synchronous updates. Summary of the Invention

[0004] To address the security issues faced by traditional federated learning methods, this invention provides a privacy-preserving asynchronous federated learning method based on blockchain.

[0005] This invention combines blockchain with asynchronous federated learning. Blockchain ensures that model data is not tampered with, while differential privacy protects the federated model training process from inference attacks. Furthermore, the asynchronous federated learning strategy allows each device to upload its local model when global aggregation requires rapid convergence, without waiting for other devices to complete, reducing latency issues caused by synchronous updates and accelerating global model aggregation. The invention also employs entropy weighting, an objective weighting method, to evaluate the credibility of consensus nodes, improving the objectivity and accuracy of the evaluation, effectively preventing malicious behavior, and ensuring the system's stability and robustness.

[0006] The blockchain-based privacy-preserving asynchronous federated learning method in this invention has the following two parts in its overall architecture:

[0007] (1) Application Layer: Composed of various basic network devices, located at the top of the entire architecture; To protect data privacy, network devices participating in federated learning tasks use local datasets for model training; These devices are all connected to the consensus nodes of the blockchain; In each iteration, the device obtains the latest global model from the blockchain system and uses local data for a new round of model training; After training is completed, the device uploads the updated model to the nearby consensus node and then initiates the global arbitration process; This process ensures that all devices can obtain the latest data from the neighboring consensus node, including the update information of the global model; After each device uploads the complete global model, it starts a new iteration cycle until the accuracy and convergence of the global model reach a satisfactory level.

[0008] (2) Blockchain Layer: The core of the entire architecture, responsible for reaching consensus among consensus nodes and generating blocks; the distributed ledger is maintained by all consensus nodes in the network, who record the model in federated learning; each consensus node is responsible for generating or checking transactions, executing the consensus mechanism and creating blocks to ensure the accuracy of the model; among them, the leader consensus node (the consensus node with the highest reputation) plays a key role in the consensus mechanism, guiding other consensus nodes to reach consensus and being responsible for proposing new blocks; each block consists of a header and a body, the header containing a link to the next block, and the body containing a series of verified transaction information; in each iteration, consensus nodes receive information such as local models and global models from nearby devices, and store this information in the distributed ledger after cross-validating it through the blockchain; the decentralized and tamper-proof characteristics of the blockchain ensure the reputation of the consensus nodes and the security of the model stored in it. Attached Figure Description

[0009] Figure 1 A privacy-preserving asynchronous federated learning framework based on blockchain.

[0010] Figure 2 Accuracy of different federated learning algorithms. Detailed Implementation

[0011] A blockchain-based privacy-preserving asynchronous federated learning method includes the following steps:

[0012] (1) Blockchain network initialization; determine the initialization parameters and write them into the genesis block;

[0013] (2) Local model update; the device performs model training on its local data, iteratively updating the global model to the local model; the goal of local training is to minimize the loss function: The local model for the l-th global epoch and the h-th local iteration is represented as: ;

[0014] (a) The client device initializes the model parameters as follows: Set the number of iterations for local training to E, and perform stochastic gradient descent with a learning rate λ.

[0015] (b) Set privacy parameters based on differential privacy mechanism and And ensure that the privacy cost does not exceed the maximum value S, for each iteration e, if If so, then the process terminates;

[0016] (c) Otherwise, update the model parameters. Calculate the sensitivity s and noise scale. Then return via

[0017] Over-disturbance processing parameters ;

[0018] (3) Upload local models;

[0019] (a) The device uploads the updated local model to the consensus node associated with it;

[0020] (b) Consensus nodes will conduct preliminary verification and broadcast these model updates to the blockchain network;

[0021] (4) Leader selection; consensus nodes select a leader node based on credibility; the leader node is responsible for further verifying the consensus nodes.

[0022] The node uploads the local model and aggregates it into a global model;

[0023] (a) Using cosine similarity to determine the differences between models; the model of consensus node k in time h and the global model in time h-1.

[0024] The cosine similarity between models is: ;

[0025] (b) Cosine similarity is calculated using a neural network; for consensus node k, Indicates the j-th index

[0026] The normalized value of the target, the proportion is: ;

[0027] (c) Then calculate the entropy weight of index j. The larger the entropy weight, the better the information...

[0028] The greater the contribution to reputation;

[0029] (d) The reputation of consensus node k in period h is obtained by the following formula: ;

[0030] (5) Local model cross-validation;

[0031] (a) After receiving the local model uploaded and broadcast by the consensus nodes, the leader node performs detailed cross-validation to ensure the model is valid.

[0032] The accuracy of model updates;

[0033] (b) Once verified, the leader node records it in a new block and broadcasts it to other nodes in the blockchain network;

[0034] (6) Global model aggregation;

[0035] (a) After the leader node has completed the verification of all local models, it will aggregate these models to update the global model.

[0036] parameter;

[0037] (b) In asynchronous federated learning, the aggregation process uses a weighted average method to combine all validated local models.

[0038] Create a new global model;

[0039] (c) The aggregation process is defined as follows: The larger the weight, the better the new model is.

[0040] The greater the impact of global model updates;

[0041] (7) Block generation and chain entry;

[0042] (a) After completing the global model aggregation, the leader node will generate a node containing the updated global model parameters and consensus nodes.

[0043] A new block for reputation;

[0044] (b) The leader node executes the consensus mechanism to ensure that the content of the new block is agreed upon by all participants in the network;

[0045] (c) Once consensus is reached, the new block will be added to the blockchain and broadcast to all nodes in the blockchain network for use.

[0046] All consensus nodes and devices download and verify;

[0047] (8) Global model download;

[0048] (a) After the system status is updated, all devices download the latest global model from the blockchain network;

[0049] (b) Then begin a new round of local model training until the global model converges or reaches the required accuracy.

Claims

1. A privacy-preserving asynchronous federated learning method based on blockchain, comprising the following steps: (1) Blockchain network initialization; determine the initialization parameters and write them into the genesis block; (2) Local model update; the device performs model training on its local data, iteratively updating the global model to the local model; the goal of local training is to minimize the loss function: ; the local model of the lth global epoch and the hth local iteration is represented as: ; (a) the client device initializes model parameters to , sets the number of iterations for local training to E, and performs stochastic gradient descent with learning rate λ; (b) set a privacy parameter according to a differential privacy mechanism and and ensure that the privacy cost does not exceed a maximum value S, for each iteration e, if then terminate; (c) Otherwise, update model parameters , compute sensitivity s and noise scale Then return the refined Parameters of the over-disturbance processing ; (3) Local model upload; (a) The device uploads the updated local model to the consensus node associated with it; (b) The consensus node will conduct preliminary verification and broadcast these model updates to the blockchain network; (4) Leader selection; the consensus node selects the leader node according to the credibility; the leader node is responsible for further verifying the local model uploaded by the consensus node and aggregating the global model; (a) Use cosine similarity to judge the difference between models; the more the consensus node k contributes to the credibility of the model at period h and the model at period h-1; (5) Local model cross-validation; The cosine similarity between the local models is: ; (b) calculating cosine similarity using neural network; for consensus node k, represents the jth finger Target normalized value, in proportion to: ; (c) then calculate the entropy weight of index j ; the greater the entropy weight, the greater the impact on the (a) After the leader node receives the local model uploaded and broadcast by the consensus node, it performs detailed cross-validation to ensure the accuracy of the model update; (d) the reputation of consensus node k in epoch h is obtained by the following formula: ; (b) After verification, the leader node records it in a new block and broadcasts it to other nodes in the blockchain network; (6) Global model aggregation; (a) After completing the verification of all local models, the leader node will aggregate these models to update the global model parameters; (b) In asynchronous federated learning, the aggregation process uses a weighted average method to combine all verified local models into a new global model; The greater the impact of global model update; (7) Block generation and chain entry; (a) After completing the global model aggregation, the leader node will generate a new block containing the updated global model parameters and the credibility of the consensus node; (b) The leader node performs a consensus mechanism to ensure that the contents of the new block are agreed upon by all participants in the network; (c) Once consensus is reached, the new block will be added to the blockchain and broadcast to all nodes in the blockchain network for all consensus nodes and devices to download and verify; (c) The polymerization process is defined as follows: The greater the weight, the greater the new model's (8) Global model download; (a) After the system state is updated, all devices download the latest global model from the blockchain network; (b) Then start a new round of local model training until the global model converges or reaches the required accuracy. ​ ​ ​ ​ ​ ​ ​

Images