A trusted federated learning method based on blockchain
By introducing blockchain and committee consensus mechanisms into federated learning, the stability and security issues caused by central servers are resolved, the fairness and consensus efficiency of the system are enhanced, the privacy protection and regulatory requirements of domestic consortium blockchains are met, interference from malicious nodes is resisted, and the reliability of the training process is ensured.
Patent Information
- Application Number
- CN202410987333.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-23
AI Technical Summary
The stability, fairness, and security issues of the central server in existing federated learning have not been effectively resolved. Malicious nodes and offline nodes have a significant impact on the training process. Furthermore, existing blockchain technology is not adapted to the performance, privacy protection, and regulatory compliance requirements of domestic consortium blockchains.
A blockchain-based trusted federated learning approach is adopted. By constructing and updating model blocks, honest nodes are selected to form a committee for verification. A committee consensus mechanism (CCM) is designed and an incentive mechanism is introduced to solve the above problems.
It improves consensus efficiency, reduces network congestion, enhances system stability and security, adapts to the privacy protection and regulatory requirements of domestic consortium blockchains, resists interference from malicious nodes, and ensures the fairness and reliability of the training process.
Smart Images

Figure CN118520052B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a trusted federated learning method based on blockchain, and belongs to the technical field of mobile communications. Background Art
[0002] With the introduction of the EU General Data Protection Regulation (GDPR), industry and academia have begun to pay more attention to the privacy protection of machine learning. The implementation of GDPR marks a new era in the legal framework for personal data protection, and imposes strict requirements on how companies handle personal data. This change has prompted the industry to re-examine its data processing practices to ensure compliance, while also driving the demand for privacy protection technologies.
[0003] In the field of machine learning, privacy protection has become an important issue. Since machine learning models usually require a large amount of data for training, and this data often contains sensitive information, how to conduct effective model training without leaking personal privacy has become a hot topic of research. Among the current mainstream technologies, the most common one is Federated Learning (FL), which is a distributed machine learning method that allows multiple participants to collaborate to train a shared model while maintaining the privacy and localization of their respective data. In federated learning, data does not need to leave the local environment of the participants, but is trained by exchanging model parameter updates (such as gradients or weight changes). This method is particularly suitable for scenarios involving sensitive data, such as healthcare, financial services, and personal devices, because it reduces the risk of data leakage and abuse. Federated learning usually includes two main roles: participating devices (or clients) and central servers. Participating devices are responsible for training models using their local data and generating updates to send to the central server; the central server is responsible for aggregating updates from all devices and generating global models to distribute to participating devices for the next round of local training. The goal of federated learning is to enhance the generalization ability and performance of the model using data from multiple parties without sacrificing data privacy.
[0004] However, federated learning often only proposes a distributed training model, including two roles: participating devices and a central server. Instead of uploading private data, nodes update the global model locally and then upload the model update (i.e., local gradient). The central server collects these updates and integrates them into an updated model. Due to this privacy feature, federated learning has attracted increasing attention from researchers in recent years. In the FL setting, the server performs the central operations of update aggregation, client selection, and global model maintenance. The server needs to collect updates from many clients to complete the aggregation operation, and also needs to broadcast the new global model to these clients, which places high demands on network bandwidth. In addition, cloud-based servers are affected by the stability of cloud service providers. The central server can tilt the global model by favoring certain clients. In addition, some malicious central servers can pollute the model and even collect the client's private data from the update. Therefore, the stability, fairness, and security of the central server are crucial to FL.
[0005] In the existing public texts, for example, the application with publication number CN115640305A discloses a fair and reliable federated learning method based on blockchain, including: the model demander publishes training tasks and transaction contract transfer tasks on the blockchain, the client trains the global model, generates local model parameters, and encrypts and transmits the local model parameters to the corresponding miners on the blockchain; the corresponding miners propagate and verify the encrypted local model parameters, the corresponding miners aggregate the verified local model parameters, update the global model according to the aggregation results, generate new blocks based on the update results, and broadcast the new blocks; all miners verify the new blocks and reach a consensus; the incentive contract calculates the client's contribution based on the consensus results and generates the latest global model; repeat S2-S6 until the training end conditions are met to obtain the optimized model. The transaction contract transmits the optimized global model to the model demander.
[0006] However, although the above existing technologies use blockchain technology to enhance the transparency and security of federated learning, they do not clearly propose a consensus mechanism that is suitable for the domestic alliance chain ecosystem. Domestic alliance chains usually have special requirements for performance, privacy protection, and regulatory compliance; at the same time, although the above text mentions the use of incentive contracts to encourage client contributions, it may not fully consider the impact of malicious nodes on the system and the potential interference of offline nodes on the training process. Malicious nodes may upload incorrect model updates in an attempt to disrupt the model training process; while offline nodes may cause incomplete model updates, affecting the accuracy and generalization ability of the global model.
[0007] Therefore, it is necessary to propose a new learning method that can solve the stability, fairness and security problems brought by the central server, while the efficiency of consensus is also improved. Summary of the invention
[0008] The present invention provides a trusted federated learning method based on blockchain, which solves the stability, fairness and security problems brought by the central server, and the efficiency of consensus is also improved.
[0009] The technical solution adopted by the present invention to solve its technical problem is:
[0010] A trusted federated learning method based on blockchain, specifically comprising the following steps:
[0011] Step S1: construct a storage model based on blockchain, which includes two blocks, one block is a model block, and the other block is an update block;
[0012] Step S2: Select some initial nodes to form a committee, and the remaining nodes actively obtain the current global model and perform local training at any time; this also includes offline nodes;
[0013] Step S3: Send the gradient information obtained after the remaining nodes perform local training to the committee for verification;
[0014] Step S4: The verified gradient information is placed in the update block; after the qualified gradient information is packaged into the update block, a token reward is given;
[0015] Step S5: Several consecutive update blocks are aggregated by the committee to form a new global model, which is packaged into the model block of the blockchain;
[0016] Step S6: During committee verification, the training results of the nodes are assigned scores, and a new committee is elected based on the scores of the nodes in the previous round during a new round of verification and update;
[0017] Step S7: Repeat steps S3-S6 and perform the next round of training based on the latest global model;
[0018] As a further preferred embodiment of the present invention, in step S1, the model block includes a block header, a t The round number and global model, the update block includes the block header, t The round number, the gradient information obtained after executing local training, the uploader address, and the distribution score;
[0019] Set the number of updates per training round to , numbered The block contains t The wheel model is the model block, numbered The block contains t The update of the round is the update block, where ;
[0020] As a further preferred embodiment of the present invention, in step S2, the initial node constituting the committee is an honest node, and the way to form the committee is that after the gradient information that has been verified by the committee is placed in the update block, the committee regards the local update as a verification set, and uses its verification accuracy as a score. After combining the scores of each committee member, the median is the updated distribution score. At the beginning of the next round of training, a new committee is elected according to the scores of the nodes in the previous round;
[0021] As a further preferred embodiment of the present invention, for offline nodes, they can participate in the previous global model and perform local training in an offline state. When the offline nodes are reconnected to the network, they submit the updates obtained by their training and are verified by the committee.
[0022] As a further preferred embodiment of the present invention, the initial node constitutes a global model, trains and is responsible for node management, that is, acts as an administrator;
[0023] If a device submits misleading updates or spreads private models, the administrator uses the blacklist mode to kick the device out of the global model;
[0024] As a further preferred embodiment of the present invention, an incentive mechanism is built into the learning method, including licensing fees and profit sharing;
[0025] The license fee means that each device pays for access to the global model. The fee is retained by the administrator. After payment, the node has unlimited access to the latest global model in the blockchain.
[0026] Profit sharing means that after each round of aggregation, the administrator distributes rewards to the corresponding nodes based on the updated scores submitted.
[0027] Through the above technical solution, compared with the prior art, the present invention has the following beneficial effects:
[0028] 1. The blockchain-based trusted federated learning method provided by the present invention provides a blockchain-based federated framework that can replace the architecture that relies on the central server and solve the stability, fairness and security problems brought by the central server;
[0029] 2. The blockchain-based trusted federated learning method provided by the present invention proposes a committee consensus mechanism (CCM), which improves consensus efficiency and reduces network congestion by selecting a portion of honest nodes for verification and block packaging, and adapts to the performance, privacy protection and regulatory requirements of domestic alliance chains;
[0030] 3. The blockchain-based trusted federated learning method provided by the present invention fully considers the impact of malicious nodes on the system and the potential interference of offline nodes on the training process when encouraging client contributions through incentive contracts. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0032] Figure 1 It is a preferred storage mode design framework diagram provided by the present invention. DETAILED DESCRIPTION
[0033] The present invention will now be described in further detail with reference to the accompanying drawings.
[0034] At present, in the field of machine learning, in order to conduct effective model training without leaking personal privacy, several mainstream technologies have been formed. In addition to the federated learning mentioned in the background technology, there are also secure multi-party computing (SMPC), homomorphic encryption (HE), and differential privacy (DF). For secure multi-party computing, its computational complexity is high and requires a large amount of computing resources, which is particularly prominent in resource-constrained environments such as mobile devices; secondly, the design and implementation of the SMPC protocol is relatively complex and requires a high degree of expertise. In addition, as the number of participants increases, the coordination and communication overhead will also increase significantly. Homomorphic encryption has relatively low computational efficiency, especially when processing large-scale data sets, its performance may become a bottleneck; secondly, the implementation of the HE algorithm usually requires specific hardware support, which may increase the deployment cost of the system; in addition, HE technology may encounter compatibility issues when processing complex models and deep learning algorithms. Similarly, differential privacy also faces some problems in practical applications. First, determining appropriate privacy parameters (such as privacy budget ε) is a challenge because it requires a trade-off between privacy protection and data accuracy. Second, differential privacy may lead to reduced practicality of data, especially under high privacy protection requirements. In addition, the implementation and effect evaluation of differential privacy also require expertise and experience.
[0035] Therefore, in order to solve and avoid the above problems, this application proposes a trusted federated learning method based on blockchain. As one of the innovations of this application, it is based on blockchain. Federated learning (FL) allows the training of machine learning algorithms between multiple distributed clients without exchanging their data samples. In the original FL setting, a central server controls the training process, including client management, global model maintenance, and gradient aggregation. In each round of training, the server broadcasts the current model to some participating nodes. After receiving the model, the node updates it locally using local data and submits the updated gradient to the server. Then, the server aggregates and applies the local gradient to the next round of the model. However, the architecture that relies on the central server needs to be wary of the stability, fairness, and security issues brought by the central server.
[0036] The study found that the function of the central server can be replaced by a smart contract (SC) and activated through transactions on the blockchain. Therefore, the decentralized nature of the blockchain can replace the position of the central server. In order to achieve the above purpose, this application builds a federated learning framework based on the blockchain and adopts committee consensus.
[0037] Specifically include the following:
[0038] First, a storage model based on blockchain is constructed. The storage model includes two blocks, one for the model block and the other for the update block.
[0039] In order to enable permission control, the storage model built is a consortium blockchain system, where only authorized devices can access FL training content. On the blockchain, two different blocks are designed to store global models and local updates (collectively referred to as learning information), namely model blocks and update blocks.
[0040] From the implementation point of view, the model block includes the block header, t The round number and global model, the update block includes the block header, t The number of the round, the gradient information obtained after executing local training, the uploader address, and the distribution score; the specific explanation will be given during the subsequent training. For simplicity, it is assumed that only one learning information is placed in each block. At the beginning, a randomly initialized model is placed in block #0, and then round 0 training begins. The node accesses the current model and performs local training, and puts the verified local gradient into a new update block. By numbering the model block and the update block, the order relationship of the model block and the update block in the newly added block in each iteration is reflected (continuous k In order to illustrate how to reasonably allocate the blockchain storage model during the training and updating process of a model, we use the numbering method combined with Figure 1To intuitively reflect the order relationship between the newly added model blocks and the update blocks, specifically, the number of updates per round of training is set to , numbered The block contains t The wheel model is the model block, numbered The block contains t The update of the round is the update block, where . Figure 1 As shown, if , then #0 means The model block at the time of , which contains the update block number of round 0 ,Right now Figure 1 Among #1, #2…#k. Figure 1 middle Indicates t The model block of the round, then contains the t The update block number of the round is When there are enough consecutive update blocks, the smart contract triggers aggregation, generates a new model for the next round and places it on the chain. FL training only relies on the latest model block, and historical blocks are stored for fault recovery and block verification.
[0041] The blockchain storage solution mentioned above can quickly find the latest global model; although historical models and updates can provide disaster recovery functions, they also occupy a huge storage space. Therefore, this application immediately proposes a simple and feasible storage overhead reduction solution: nodes with insufficient storage capacity can locally delete historical blocks and only retain the latest models and updates of the current round. In this way, the problem of insufficient storage space on some nodes can be solved while retaining the ability of disaster recovery and block verification on core nodes. However, the disadvantages of this method are also obvious. As nodes are deleted, the credibility of the blockchain decreases. In a mutually distrustful training community, each node may not use this solution for security reasons. Therefore, a trustworthy and reliable third-party storage may be a better solution. The blockchain only maintains the location network address and modification operation records of each model or update file. Other nodes interact with centralized storage to obtain the latest model or upload updates. This centralized storage will be responsible for disaster recovery backup and distributed file storage services.
[0042] The chain structure of blockchain guarantees immutability, therefore, appending the correct blocks to the chain is a key component, which is the work of the consensus mechanism. The competition-based consensus mechanism first appends the blocks to the chain and then reaches a consensus. In contrast, the communication-based generation mechanism reaches an agreement before appending the blocks. Considering the computational and communication costs of consensus, this application adopts an efficient and secure committee consensus mechanism (CCM) to verify the local gradient before appending it to the chain. Specifically, some initial nodes are selected to form a committee, and the initial nodes that constitute the committee are honest nodes. The remaining nodes actively obtain the current global model at any time and perform local training.
[0043] Common ways to form a committee include random election, election by score, or multi-factor optimization. Random election means that new committee members are randomly selected from verified nodes. From the perspective of machine learning, this method improves the generalization ability of the model and reduces overfitting. However, when malicious nodes disguise themselves as normal nodes, the ability to resist malicious attacks is weak. Election by score means that the providers with the highest verification scores form a new committee, which may aggravate the uneven distribution of samples caused by some nodes not entering the committee. However, for malicious node attacks, this method significantly increases the cost of attack and brings more security and stability. Multi-factor optimization This method considers multiple factors of the device (i.e., network transmission rate) and verification score to make the best election. However, this optimization will bring additional computing overhead. Therefore, this method should be applied according to real-world scenarios and related requirements.
[0044] One of the innovations of this application is that after the gradient information that has passed the committee's verification is placed in the update block, the committee regards the local update as a verification set, and uses its verification accuracy as a score. After combining the scores of each committee member, the median is the updated distribution score. At the beginning of the next round of training, a new committee is elected based on the scores of the nodes in the previous round, which means that the committee will not be reused.
[0045] The above method is firstly efficient, as only a few nodes will verify the update instead of broadcasting to every node and reaching an agreement. Secondly, the committee members will not participate in local training in this round, so the local data of the committee is used as a validation set. As the committee members rotate in each round, the validation set will also change. Under this setting, K-fold cross-validation of FL is achieved. Finally, based on the validation score, the corresponding nodes with better performance will be elected by the smart contract and constitute a new committee for the next round of training, which means that the selected local data distribution is gregarious, the nodes are not malicious, and malicious behavior is resisted.
[0046] As mentioned above, all nodes except the committee perform local training in each round. In federated learning, for security and privacy, the original data will be kept locally on the nodes, and these nodes only upload gradient information to the blockchain.
[0047] That is to say, only a certain number of local updates are required in each round. The committee consensus mechanism can maximize the generalization ability of the global model by using the data distribution of committee members to verify local updates, which solves the problem that the local data distribution may not be independent and identically distributed (non-IID).
[0048] The remaining nodes actively obtain the current global model and perform local training at any time. This is an active local learning progress designed for the nodes in this application. The gradient information obtained after the remaining nodes perform local training is sent to the committee for verification, and the qualified gradient information is placed in the update block; among them, after the qualified gradient information is packaged into the update block, a token can be attached as a reward.
[0049] Obviously, by pre-selecting a part of honest nodes to form a committee, these nodes are responsible for verifying the model updates submitted by other nodes. This mechanism not only improves the efficiency of consensus, but also reduces the impact of malicious nodes on the system through centralized verification, and also adapts to the domestic alliance chain's regulatory compliance needs.
[0050] Next, since each round of updates requires a certain number of valid updates, when the committee verifies a sufficient number of local updates, the aggregation process is activated, and these verified updates are aggregated by the committee into a new global model. Aggregation can be performed on local gradients or local models, and the network transmission consumption of these two methods is equal. After the new global model is packaged on the blockchain, the committee will be elected again and the next round of training will begin.
[0051] The entire training process relies on mutual promotion between nodes, and node management is also a key part of this application. Participating nodes can not only access the global model, but also upload updates to influence the global model. The aforementioned token rewards are the incentive mechanism designed by this application, which encourages nodes to actively participate in model training and updating, thereby incentivizing nodes to contribute to the improvement of the global model. In order to control permissions, the initial node is designated to form the training community and is responsible for node management, that is, as an administrator, each device must be verified by the administrator before joining the training community. This verification adopts a blacklist mode: if the device is kicked out of the community due to improper behavior (such as submitting misleading updates, spreading private models), it will be rejected. According to the proposed blockchain storage structure, new nodes can quickly find the latest global model on the chain after joining. Nodes can immediately use this model to complete local tasks, or use local data to update the model and obtain scores on the chain after verification by the consensus committee. It is worth noting that only a certain number of valid updates are required for each round of aggregation, and only some nodes participate online. Therefore, as long as the node actively submits updates, it is possible to participate in the global model update and obtain scores.
[0052] In the background technology, the applicant pointed out another defect of the prior art, that is, the problem of offline nodes was not taken into account. The present application allows nodes to obtain the current global model and perform local training at any point in time, and participate in model training even when offline; when the node is reconnected to the network, it can submit the updates obtained from its training and participate in the update of the global model through verification by the committee; this design improves the flexibility and robustness of the system, ensuring that the federated learning process can continue even when some nodes are offline.
[0053] In addition to the rewards for tokens, considering that nodes in the community can always use the model without submitting updates, it is also necessary to encourage nodes to provide updates to the global model. Then the incentive mechanism also includes licensing fees and profit sharing. Licensing fees: Each device should pay for access to the global model, and these fees are retained by the administrator. The node can then access the latest model in the community without restrictions. Profit sharing: After each round of aggregation, the administrator distributes rewards to the corresponding nodes based on the score of submitted updates. The effect is that frequent updates can get more rewards, and the continuously updated global model will attract more nodes to participate. This incentive mechanism is highly scalable and can adapt to different real-world applications.
[0054] From the above, it can be seen that the trusted federated learning method based on blockchain provided by this application replaces the central server in federated learning by designing a blockchain-based model and updating the storage architecture, thereby solving the single point failure problem of the central server and the problem of vulnerability to attacks and privacy leakage; by designing a committee consensus mechanism, the consensus efficiency is improved and security attacks are resisted; by designing a model training scheme and proposing an incentive mechanism, the problem of malicious nodes or offline nodes is solved, and the robustness and security of the system are improved.
[0055] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.
[0056] The meaning of "and / or" described in this application means that the situations where each exists alone or both exist at the same time are included.
[0057] The term “connection” as used in this application may mean a direct connection between components or an indirect connection between components via other components.
[0058] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.
Claims
1. A trusted federated learning method based on blockchain, characterized by: The specific steps include: Step S1: construct a storage model based on blockchain, which includes two blocks, one block is a model block, and the other block is an update block; The model block includes the block header, t The round number and global model, the update block includes the block header, t The round number, the gradient information obtained after executing local training, the uploader address, and the distribution score; Set the number of updates per training round to k , , numbered The block contains t The wheel model is the model block, numbered The block contains t The update of the round is the update block, in which, ; When there are enough consecutive update blocks, the smart contract triggers aggregation, generates a new model for the next round and places it on the chain. Nodes with insufficient storage capacity can delete historical blocks locally and only retain the latest model and update for the current round. FL training only relies on the latest model block. Historical blocks are stored for fault recovery and block verification. The blockchain only maintains the location network address and modification operation records of each model or update file. Other nodes interact with the centralized storage to obtain the latest model or upload updates. This centralized storage will be responsible for disaster recovery backup and distributed file storage services. Step S2: Select some initial nodes to form a committee, and the remaining nodes actively obtain the current global model and perform local training at any time; this also includes offline nodes; The initial nodes that make up the committee are honest nodes. The way to form the committee is that after the gradient information that has been verified by the committee is placed in the update block, the committee regards the local update as a verification set, and uses its verification accuracy as a score. After combining the scores of each committee member, the median is the updated distribution score. At the beginning of the next round of training, a new committee is elected based on the scores of the nodes in the previous round. Step S3: Send the gradient information obtained after the remaining nodes perform local training to the committee for verification; Step S4: The qualified gradient information is put into the update block; after the qualified gradient information is packaged into the update block, token rewards are given; the committee's local data is used as the verification set, and the verification set will change with the alternation of committee members in each round; Step S5: Several consecutive update blocks are aggregated by the committee to form a new global model, which is packaged into the model block of the blockchain; Step S6: During committee verification, the training results of the nodes are assigned scores, and a new committee is elected based on the scores of the nodes in the previous round during a new round of verification and update; Step S7: Repeat steps S3-S6 and perform the next round of training based on the latest global model.
2. The blockchain-based trusted federated learning method according to claim 1, characterized in that: For offline nodes, they can participate in the previous global model and perform local training in an offline state. When the offline node reconnects to the network, it submits the updates obtained from its training and is verified by the committee.
3. The blockchain-based trusted federated learning method according to claim 1, characterized in that: The initial nodes constitute the global model, train and are responsible for node management, i.e., act as administrators; If a device submits misleading updates or spreads private models, the administrator uses the blacklist mode to kick the device out of the global model.
4. The blockchain-based trusted federated learning method according to claim 1, characterized in that: Building incentives into learning methods, including licensing fees and profit sharing; The license fee means that each device pays for access to the global model. The fee is retained by the administrator. After payment, the node has unlimited access to the latest global model in the blockchain. Profit sharing means that after each round of aggregation, the administrator distributes rewards to the corresponding nodes based on the updated scores submitted.
