Robust efficient decentralized federated learning method and system based on committee consensus
Through the committee consensus mechanism and smart contract based on contribution proof, the problems of single-point failure and malicious participant attacks in federated learning are solved, the accuracy and efficiency of model aggregation are improved, and the greater robustness and decentralization are achieved Metamorphosis.
Patent Information
- Application Number
- CN202411419064.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing federated learning framework based on blockchain has problems such as single point failure and untrustworthy central servers, and low proof of work and waste of computing resources. Proof of stake leads to continuous exclusive accounting rights, and malicious participants may invest through Poison attacks affect the accuracy and availability of global models.
Using a committee consensus mechanism based on the degree of contribution proof, the committee members can be elected through a verifiable random function, dual verification of mutual information of model parameters, smart contracts are introduced for global model aggregation, calculate the contribution of each participant, and select honest participants for consensus. , avoid single point of failure and malicious model updates.
It improves the accuracy and efficiency of model aggregation, effectively defends against poisoning attacks by malicious participants, reduces the risk of single point failure, improves the robustness and decentralization of the system, and reduces waste of computing resources.
Smart Images

Figure CN118940865A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a robust and efficient decentralized federated learning method and system based on committee consensus, and belongs to the technical field of federated learning. Background Art
[0002] Machine learning is a method of automatically learning models from data. Its main purpose is to discover laws and patterns through data analysis in order to make predictions or decisions. Machine learning has been widely used in many fields, such as natural language processing, computer vision, and intelligent speech. With the continuous advancement of artificial intelligence technology and the continuous expansion of application scenarios, machine learning will have greater potential and broader application prospects.
[0003] Machine learning requires a large amount of data for training and verification. Traditional machine learning requires servers to collect a large amount of data for model training, but these data often contain the privacy information of the data provider, and centralized storage will bring the risk of privacy data leakage. In order to protect the privacy data of machine learning data providers, McMahan et al. first proposed a federated learning framework, which allows multiple parties to jointly train machine learning models without exposing the original data, while maintaining user trust and reducing costs. Federated learning reduces the risk of privacy leakage caused by data upload to a certain extent, but requires a central server to issue and aggregate the global model, and receive the model parameters uploaded by the client, which is at the core of the federated learning framework. Therefore, federated learning often faces the problems of single point failure and unreliable central server. If the central server fails, or attempts to destroy the global model or infer the client's privacy information by some means, it will cause unpredictable losses. Relevant researchers have conducted a series of studies on the above issues, using blockchain to replace the central server in federated learning, effectively solving the problems of single point failure and unreliable central server in traditional federated learning. However, there are still some defects:
[0004] In the existing blockchain-based federated learning framework, proof of work and proof of stake are mostly used as consensus mechanisms. However, proof of work uses the workload, that is, the computing power of the node, as the standard for the competition between nodes for the blockchain bookkeeping rights. Proof of work can provide higher security and a higher degree of decentralization, but it is inefficient and wastes too many computing resources. Proof of stake competes for bookkeeping rights based on the amount of stake owned by the node. Those with the highest stake are often more likely to obtain bookkeeping rights. Proof of stake solves the shortcomings of low efficiency and waste of resources of proof of work, but it will cause the problem of continuous monopoly of bookkeeping rights, weakening the decentralization of the network.
[0005] Malicious participants may exist in federated learning, and the global model is vulnerable to poisoning attacks from malicious participants. Attackers can disrupt the federated learning process by tampering with local models or sending false models, thereby affecting the accuracy and availability of the global model. In response to poisoning attacks, a variety of Byzantine robust federated learning algorithms have been proposed, such as Trimmed Mean, Krum, and Multi-Krum. These algorithms use different mechanisms to enhance the system's ability to resist malicious models. Trimmed Mean reduces the impact of malicious data by trimming extreme values, thereby calculating a more reliable model update; the Krum algorithm reduces the impact of malicious participants on the final model by selecting the gradient closest to other gradients; and Multi-Krum further extends the idea of Krum and improves the robustness of the system by introducing multiple Krum processes. These methods can effectively detect and mitigate malicious poisoning attacks in theory, but they each have certain limitations, such as computational overhead, robustness range, and complexity of practical applications. Summary of the invention
[0006] To solve the above problems, the present invention proposes a robust and efficient decentralized federated learning method and system based on committee consensus. Using a committee consensus mechanism based on proof of contribution, a contribution calculation method is designed. A part of honest committee members are selected for consensus based on their contributions, which can reduce the waste of computing resources while ensuring the verification and consensus efficiency in federated learning. The committee members perform double verification of the trainer's model parameters based on mutual information, which can effectively filter malicious model updates and resist poisoning attacks from malicious participants. The introduction of smart contracts to aggregate the global model and calculate the contribution of each participant replaces the central server in traditional federated learning, which can effectively avoid single point failure problems and improve the accuracy and efficiency of model aggregation.
[0007] The present invention is applicable to a federated learning scenario in which multiple participants perform joint training.
[0008] The technical solution of the present invention is: Robust and efficient decentralized federated learning method based on committee consensus, including: 1) Elect committee members; the initial contribution of each participant is zero. In the first round of elections, the committee members are randomly selected using a verifiable random function (VRF). After a round of training, each participant obtains his or her own contribution. Starting from the second round of elections, committee members are selected based on the contribution of each participant. 2) Both the trainer and committee members use the global model to train local models; 3) Committee members verify the trainer's model and score the trainer who passes the verification. The score will become part of the trainer's contribution; 4) Committee members upload their own information and the information of the trainers they have verified to the blockchain; when there are enough consecutive update blocks, the smart contract will be automatically called to aggregate the model updates on the blockchain, generate a new global model, and calculate the contribution of each participant in this round of training. The committee members for the next round of training are selected based on the contribution, and the new global model, the contribution of each participant, and the role of the participant in the next round of federated learning are uploaded to the blockchain; 5) Each participant downloads the latest block and uses the new global model as the initial model in the next round of training; the above operations are repeated until the model converges.
[0009] Robust and efficient decentralized federated learning method based on committee consensus, including: Step 1: Initial global model issuance and first round of committee member elections: Task publisher To every participant in federated learning Send the initial global model , and use a verifiable random function to initialize the roles and randomly select the committee members for the first round of federated learning and trainers ; Step 2: Local model training: Trainer and committee members Based on their own local data and Train the local models and obtain their respective local models and ; Trainers have The committee has a total of There are 1,000 participants indivual, ; Step 3: Model Parameter Validation: Trainer After training, use your local model The parameters and training related information are sent to the committee members Conduct verification; committee member According to the verification mechanism, based on the global model And the local model you trained yourself For trainers Local Model The parameters are verified and the local model is verified The parameters are used to calculate the score ; Step 4: Committee Members Trainers who will be verified Related information and committee members Upload relevant information to the blockchain; Step 5: Smart contract aggregates the global model and calculates the trainer and committee members Contribution: When there are enough consecutive update blocks, the smart contract is triggered to aggregate the model updates and obtain the global model of the current round. , and for trainers and committee members Computational Trainer and committee members Contribution of this round; Step 6: The smart contract selects the next round of trainers and committee gathering :Based on the calculated contribution, all participants Sort by contribution and select the next round of trainers and committee gathering ; The global model , Trainer Collection and committee gathering and each participant The contribution is uploaded to the blockchain; Step 7: Global Model As the initial model for the next round of training, repeat the above steps until the global model converges.
[0010] Preferably, in step 1, the initial global model is issued and the first round of committee members are elected; specifically, the following steps are included: Task publisher To participants in federated learning Send the initial global model ; Each participant Everyone has their own unique and a public-private key pair , using a verifiable random function, based on its own and private key Generate a unique random hash value , ; is a verifiable random function, and Input to Generate a random hash value ; Based on each participant Initialize the role with the random hash value: Sort the obtained random hash values in ascending order from small to large, and select the one with the highest sorting position Participants became initial committee members for the first round of training , Refers to the collection of committee members, including all committee members ; The rest Participants became trainers in the first round of training , Refers to the set of trainers, including all trainers .
[0011] Preferably, according to the present invention, in step 2, local model training includes: trainer Based on your own local data Train the local model and get the trainer Local Model ; Committee Members Based on your own local data Train the local model and get committee members Local Model ;in, is the learning rate, is the loss function, Represents the gradient.
[0012] Preferably, in step 3, model parameter verification includes: First, the trainer After the training, the committee members Send Message ,in, Indicates trainer The time spent on training, Indicates the size of the local data set, Indicates the number of training rounds; Then, committee members Received from trainer News After that, calculate the trainer Local Model and committee members Local Model Mutual information between ,in, Is a trainer Local Model The entropy of Is a member of the committee Local Model The entropy of yes and The joint entropy of Similarly, computational trainers Local Model and the initial global model Mutual information between , is the initial global model The entropy of yes and The joint entropy of Finally, judge and Are they both true? If true, then the trainer Local Model By verification, committee members For a verified local model Parameters for calculating the score .
[0013] According to the preferred embodiment of the present invention, in step 4, the committee members Trainers who will be verified Related information and committee members The relevant information is uploaded to the blockchain; specifically including: Committee Members Update Trainer The information is , and committee members Related information Packaged together into a message , and the message Uploaded to the blockchain, where Member of the Committee Local training time, Member of the Committee For trainers The time it took to perform the verification.
[0014] According to the preferred embodiment of the present invention, in step 5, the smart contract aggregates the global model and calculates the trainer and committee members Contribution: Specifically include: When there are enough consecutive update blocks, the smart contract is automatically triggered; enough consecutive update blocks means more than Continuously update blocks; First, check the trainer Local Model Whether it is passed by more than Verification of committee members, if yes, using The aggregation algorithm updates all the models that have passed the validation Perform aggregation to obtain the global model of the current round ,in, Update the number for models that pass validation; otherwise, the trainer Local Model Will not be used for aggregation in the global model; Next, according to the participants The role is different for each trainer and committee members Calculate contribution; based on the message And the aggregated results of this round to count the trainers and committee members Contributions to this round of training, not used for the final global model aggregation trainer The contribution of this round is zero; the specific calculation method is as follows: trainer Contribution : ,in, For all committee members For trainers The average of the scores; Committee Members Contribution: ,in, Member of the Committee The number of verified models that were finally used for global model aggregation, and The contribution ratio of two different roles helps prevent participants from Continuing membership of the committee .
[0015] According to the preferred embodiment of the present invention, in step 6, the smart contract selects the next round of trainers and committee gathering ; Specifically include: First, sort all contributions in descending order from largest to smallest. Participants Those who have made great contributions to federated learning and are considered to be credible participants will become members of the next round of committees. Saved to the committee members collection middle; Next, the remaining participants become trainers in the next round of training. Saved in the trainer collection middle; Finally, the global model , the selected trainer set and committee gathering and the contributions of all participants Upload to blockchain.
[0016] According to the preferred embodiment of the present invention, in step 7, the federated learning participants Download the latest block, confirm your role in the next round of federated learning, and obtain the latest global model , and the latest global model As the initial model for the next round of training, repeat the above steps until the final global model converges.
[0017] A robust and efficient decentralized federated learning system based on committee consensus, including: The committee member election module is configured to: conduct committee member elections; each participant’s initial contribution is zero. In the first round of elections, committee members are randomly selected using a verifiable random function. After a round of training, each participant obtains his or her own contribution; starting from the second round of elections, committee members are selected based on each participant’s contribution; The local model training module is configured as follows: both the trainer and committee members use the global model for local model training; The verification and scoring module is configured as follows: committee members verify the trainer's model and score the trainer who passes the verification, and the score will become part of the trainer's contribution; The aggregation and contribution calculation module is configured as follows: committee members upload their own information and the information of the trainers they have verified to the blockchain; when there are enough consecutive update blocks, the smart contract will be automatically called to aggregate the model updates on the blockchain, generate a new global model, and calculate the contribution of each participant in this round of training. The committee members for the next round of training are selected based on the contribution, and the new global model, the contribution of each participant, and the role of the participant in the next round of federated learning are uploaded to the blockchain.
[0018] The beneficial effects of the present invention are: 1. The present invention proposes a robust and efficient decentralized federated learning method and system based on committee consensus. The system introduces a committee consensus mechanism and smart contracts to verify local model updates and aggregate and calculate the contribution of the global model. While effectively solving the single point failure caused by the existence of the central server and the malicious server, it improves the accuracy, speed and efficiency of model aggregation. In addition, the existence of the committee consensus mechanism ensures the effective identification of malicious models, effectively defends against poisoning attacks launched by malicious participants, reduces the risk of malicious participants, improves the robustness of the federated learning system, and ensures the performance of the global model of federated learning.
[0019] 2. The present invention proposes a model parameter verification mechanism based on mutual information. Different from other detection methods based on the distance between model parameters, the distance-based detection method will filter out some contributing model updates. The present invention not only calculates the mutual information value between the model parameters of committee members and trainers, but also calculates the mutual information value between the latest global model and the trainer model parameters. After double verification, malicious model updates can be effectively filtered to prevent malicious participants from damaging the process of federated learning global model training by sharing toxic model updates.
[0020] 3. The proof of work is inefficient and wastes too many computing resources. The proof of rights will cause the problem of continuous monopoly of the right to record, which weakens the decentralization of the network. Both have certain limitations. Proof of contribution is a new consensus mechanism. Taking the contribution of the node as the basis for obtaining the right to record, it can reduce the waste of computing resources and ensure the efficiency of block generation. The present invention uses a committee consensus mechanism based on proof of contribution. Based on the contribution made by each participant in the federated learning process, a part of honest participants are selected to become committee members in the federated learning process. The committee members verify the model update, which improves the verification and consensus efficiency, and can effectively prevent malicious model updates from being used for the aggregation of the global model, thereby improving the robustness of the system. In order to ensure the reliability of the committee members, the present invention designs a contribution calculation method, designs different contribution calculation bases and contribution proportions for participants with different roles, reasonably quantifies the contribution of the participants, is conducive to the replacement of committee members, ensures the fairness of the federated learning participants in the federated learning process, and encourages participants to honestly participate in the federated learning process, thereby providing a strong guarantee for the high-quality aggregation of the global model. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 A schematic diagram of the model validation process for committee members; Figure 2 A schematic diagram of the process of aggregating and calculating contribution for smart contracts; Figure 3This is a schematic diagram of the decentralized federated learning architecture; Figure 4 This is a schematic diagram for comparing the results of the attack1 experiment; Figure 5 This is a schematic diagram for comparing the attack2 experimental results. DETAILED DESCRIPTION
[0022] The present invention will be further defined below in conjunction with the accompanying drawings and embodiments, but is not limited thereto.
[0023] Example 1 Terminology explanation: Verifiable Random Function (VRF) is a cryptographic function with verifiability and random output. It is generated based on data input and comes with a certificate that can be verified by anyone. VRF usually contains a pair of public and private keys and a random seed. When the private key and random seed are input into VRF together, a pseudo-random number and a certificate of this random number will be generated. Anyone can use the public key and certificate corresponding to the private key to verify the pseudo-random number to determine whether it is calculated by the private key holder. The pseudo-random number, private key and seed are all uniquely determined, and no one can deduce the private key based on the random seed and pseudo-random number.
[0024] Blockchain: Blockchain is a shared, decentralized, and immutable distributed ledger technology. It is essentially a special chain data structure where blocks are interconnected through the hash value of the parent block, designed to facilitate transaction records and asset tracking processes in business networks. The consensus mechanism is used to solve the consistency problem in the blockchain distributed ledger system and ensure the consistency of each transaction in each accounting node. The consensus mechanism is a rule that all nodes must follow. The consensus mechanism is a program used in the blockchain system to implement a distributed protocol about the state of the ledger. It is an algorithm that reaches a consensus on the order of things within a time period and is the core of the application of blockchain technology. Smart contracts are automated programs or protocols on the blockchain that are activated when certain preset conditions are met. Running smart contracts in a decentralized infrastructure can avoid single point failure risks, remove centralized intermediaries, and neither any party to the contract nor the system administrator can tamper with the results. When certain conditions are met, smart contracts will automatically execute, thereby improving accuracy, speed and efficiency.
[0025] Mutual information: Mutual information refers to the degree of correlation between two random variables, and its calculation formula is: ,in Mutual information can be used to calculate the correlation between different models trained by clients with non-IID data, which helps to detect abnormal models more accurately, thereby reducing the misjudgment of contributing models. At the same time, it effectively prevents malicious models from being used to aggregate global models and prevents global models from being poisoned.
[0026] : It is a model aggregation algorithm proposed by McMahan et al. for the federated learning distributed framework, which aggregates model parameters through weighted averaging. The basic idea is to upload the parameters of the client local model to the server. In order to ensure the accuracy of model aggregation, The algorithm uses weighted averaging to aggregate models. Specifically, each model parameter uploaded by a client is assigned a weight and then weighted averaged. The weight of the model parameter uploaded by the client is assigned based on the amount of local data on the device. The client with more data has a greater weight. The server then calculates the average of all model parameters based on the weights of each client, and then broadcasts this average to all clients. This process can be iterated multiple times until the model converges, and the final converged model approaches the model of the centralized machine learning result.
[0027] Robust and efficient decentralized federated learning method based on committee consensus, including: 1) Elect committee members; the initial contribution of each participant is zero. In the first round of elections, the committee members are randomly selected using a verifiable random function (VRF). After a round of training, each participant obtains his or her own contribution. Starting from the second round of elections, committee members are selected based on the contribution of each participant. 2) Both the trainer and committee members use the global model to train local models; 3) Committee members verify the trainer's model and score the trainer who passes the verification. The score will become part of the trainer's contribution; 4) Committee members upload their own information and the information of the trainers they have verified to the blockchain; when there are enough consecutive update blocks, the smart contract will be automatically called to aggregate the model updates on the blockchain, generate a new global model, and calculate the contribution of each participant in this round of training. The committee members for the next round of training are selected based on the contribution, and the new global model, the contribution of each participant, and the role of the participant in the next round of federated learning are uploaded to the blockchain; 5) Each participant downloads the latest block and uses the new global model as the initial model in the next round of training; the above operations are repeated until the model converges.
[0028] Specifically, a robust and efficient decentralized federated learning method based on committee consensus, such as Figure 3 As shown, including: Step 1: Initial global model issuance and first round of committee member elections: Task publisher To every participant in federated learning Send the initial global model , and use a verifiable random function to initialize the roles and randomly select the committee members for the first round of federated learning and trainers ; Step 2: Local model training: Trainer and committee members Based on their own local data and Train the local models and obtain their respective local models and ; Trainers have The committee has a total of There are 1,000 participants indivual, ; Step 3: Model Parameter Validation: Trainer After training, use your local model The parameters and training related information are sent to the committee members Conduct verification; committee member According to the verification mechanism, based on the global model And the local model you trained yourself For trainers Local Model The parameters are verified and the local model is verified The parameters are used to calculate the score ; Step 4: Committee Members Trainers who will be verified Related information and committee members Upload relevant information to the blockchain; Step 5: Smart contract aggregates the global model and calculates the trainer and committee members Contribution: When there are enough consecutive update blocks, the smart contract is triggered to aggregate the model updates and obtain the global model of the current round. , and for trainers and committee members Computational Trainer and committee members Contribution of this round; Step 6: The smart contract selects the next round of trainers and committee gathering :Based on the calculated contribution, all participants Sort by contribution and select the next round of trainers and committee gathering ; The global model , Trainer Collection and committee gathering and each participant The contribution is uploaded to the blockchain; Step 7: Global Model As the initial model for the next round of training, repeat the above steps until the global model converges.
[0029] Example 2 The robust and efficient decentralized federated learning method based on committee consensus described in Example 1 is different in that: In step 1, the initial global model is issued and the first round of committee members are elected; specifically: Task publisher To participants in federated learning Send the initial global model ; Each participant Everyone has their own unique and a public-private key pair , using a verifiable random function (VRF), based on its own and private key Generate a unique random hash value , ; is a verifiable random function, and Input to Generate a random hash value ; Based on each participant Initialize the role with the random hash value: Sort the obtained random hash values in ascending order from small to large, and select the one with the highest sorting position Participants became initial committee members for the first round of training , Refers to the collection of committee members, including all committee members ; The rest Participants became trainers in the first round of training , Refers to the set of trainers, including all trainers .
[0030] In step 2, local model training; including: trainer Based on your own local data Train the local model and get the trainer Local Model ; Likewise, committee members Based on your own local data Train the local model and get committee members Local Model ;in, is the learning rate, is the loss function, Represents the gradient.
[0031] In step 3, model parameters are verified; Figure 1 As shown, specifically including: First, the trainer After the training, the committee members Send Message ,in, Indicates trainer The time spent on training, Indicates the size of the local data set, Indicates the number of training rounds; it will be used as the basis for calculating contribution later.
[0032] Then, committee members Received from trainer News After that, calculate the trainer Local Model and committee members Local Model Mutual information between ,in, Is a trainer Local Model The entropy of Is a member of the committee Local Model The entropy of yes and The joint entropy of Similarly, computational trainers Local Model and the initial global model Mutual information between , is the initial global model The entropy of yes and The joint entropy of Finally, judge and Are they both true? If true, then the trainer Local Model By verification, committee members For a verified local model Parameters for calculating the score .
[0033] In step 4, committee members Trainers who will be verified Related information and committee members The relevant information is uploaded to the blockchain; specifically including: Committee Members Update Trainer The information is , and committee members Related information Packaged together into a message , and the message Uploaded to the blockchain, where Member of the Committee Local training time, Member of the Committee For trainers The time it took to perform the verification.
[0034] In step 5, the smart contract aggregates the global model and calculates the trainer and committee members Contribution: Figure 2 As shown, specifically including: When there are enough continuous update blocks, the smart contract will be automatically triggered to execute related programs; enough continuous update blocks means more than Continuously update blocks; First, check the trainer Local Model Whether it is passed by more than Verification of committee members, if yes, using The aggregation algorithm updates all the models that have passed the validation Perform aggregation to obtain the global model of the current round ,in, Update the number for models that pass validation; otherwise, the trainer Local Model Will not be used for aggregation in the global model; Next, according to the participants The role is different for each trainer and committee members Calculate contribution; based on the message And the aggregated results of this round to count the trainers and committee members Contributions to this round of training, not used for the final global model aggregation trainer The contribution of this round is zero; the specific calculation method is as follows: trainer Contribution : ,in, For all committee members For trainers The average of the scores; Committee Members Contribution: ,in, Member of the Committee The number of verified models that were finally used for global model aggregation, and The contribution ratio of two different roles helps prevent participants from Continuing membership of the committee .
[0035] In step 6, the smart contract selects the next round of trainers and committee gathering ; Specifically include: First, sort all contributions in descending order from largest to smallest. Participants Those who have made great contributions to federated learning and are considered to be credible participants will become members of the next round of committees. Saved to the committee members collection middle; Next, the remaining participants become trainers in the next round of training. Saved in the trainer collection middle; Finally, the global model , the selected trainer set and committee gathering and the contributions of all participants Upload to blockchain.
[0036] In step 7, federated learning participants Download the latest block, confirm your role in the next round of federated learning, and obtain the latest global model , and the latest global model As the initial model for the next round of training, repeat the above steps until the final global model converges.
[0037] Based on the above, the present invention conducted experiments on the non-independent and identically distributed MNIST dataset, comparing the present invention with traditional centralized federated learning and a distributed federated learning scheme CMFL. The experiment used centralized federated learning without attack as a benchmark, and tested the accuracy of the present invention, CMFL, and traditional federated learning methods in two different poisoning attack scenarios. The two attack types are gradient scaling attack (attack1) and same-value attack (attack2), and the experimental results correspond to Figure 4 and Figure 5 . It can be seen that under poisoning attacks, traditional federated learning methods are difficult to converge, and the CMFL solution can achieve better results than traditional federated learning, and the model will gradually converge. Compared with CMFL, the solution of the present invention can achieve better results than it, can converge in a short time and obtain a higher accuracy rate. The overall accuracy rate is higher than the CMFL solution, and is closer to the traditional federated learning baseline in a non-attack scenario. It has good convergence and superior robustness, and can better resist poisoning attacks from malicious clients.
[0038] Example 3 A robust and efficient decentralized federated learning system based on committee consensus, including: The committee member election module is configured to: conduct committee member elections; each participant’s initial contribution is zero. In the first round of elections, committee members are randomly selected using a verifiable random function. After a round of training, each participant obtains his or her own contribution; starting from the second round of elections, committee members are selected based on each participant’s contribution; The local model training module is configured as follows: both the trainer and committee members use the global model for local model training; The verification and scoring module is configured as follows: committee members verify the trainer's model and score the trainer who passes the verification, and the score will become part of the trainer's contribution; The aggregation and contribution calculation module is configured as follows: committee members upload their own information and the information of the trainers they have verified to the blockchain; when there are enough consecutive update blocks, the smart contract will be automatically called to aggregate the model updates on the blockchain, generate a new global model, and calculate the contribution of each participant in this round of training. The committee members for the next round of training are selected based on the contribution, and the new global model, the contribution of each participant, and the role of the participant in the next round of federated learning are uploaded to the blockchain.
Claims
1. A robust and efficient decentralized federated learning method based on committee consensus, characterized by: include: Step 1: Initial global model issuance and first round of committee member elections: Task issuer To every participant in federated learning Send the initial global model , and use a verifiable random function to initialize the roles and randomly select the committee members for the first round of federated learning and trainers ; Step 2: Local model training: Trainer and committee members Based on their own local data and Train the local models and obtain their respective local models and ; Trainers have The committee has a total of There are 1,000 participants indivual, ; Step 3: Model Parameter Validation: Trainer After training, use your local model The parameters and training related information are sent to the committee members Conduct verification; committee member According to the verification mechanism, based on the global model And the local model you trained yourself For trainers Local Model The parameters are verified and the local model is verified The parameters are used to calculate the score ; Step 4: Committee Members Trainers who will be verified Related information and committee members Upload relevant information to the blockchain; Step 5: Smart contract aggregates the global model and calculates the trainer and committee members Contribution: When there are enough consecutive update blocks, the smart contract is triggered to aggregate the model updates and obtain the global model of the current round. , and for trainers and committee members Computational Trainer and committee members Contribution of this round; Step 6: The smart contract selects the next round of trainers and committee gathering :Based on the calculated contribution, all participants Sort by contribution and select the next round of trainers and committee gathering ; The global model , Trainer Collection and committee gathering and each participant The contribution is uploaded to the blockchain; Step 7: Global Model As the initial model for the next round of training, repeat the above steps until the global model converges.
2. The robust and efficient decentralized federated learning method based on committee consensus according to claim 1, characterized in that: In step 1, the initial global model is issued and the first round of committee members are elected; specifically: Task publisher To participants in federated learning Send the initial global model ; Each participant Everyone has their own unique and a public-private key pair , using a verifiable random function, based on its own and private key Generate a unique random hash value , ; is a verifiable random function, and Input to Generate a random hash value ; Based on each participant Initialize the role with the random hash value: Sort the obtained random hash values in ascending order from small to large, and select the one with the highest sorting position Participants became initial committee members for the first round of training , Refers to the collection of committee members, including all committee members ; The rest Participants became trainers in the first round of training , Refers to the set of trainers, including all trainers .
3. The robust and efficient decentralized federated learning method based on committee consensus according to claim 1, characterized in that: In step 2, local model training; including: trainer Based on your own local data Train the local model and get the trainer Local Model ; Committee Members Based on your own local data Train the local model and get committee members Local Model ;in, is the learning rate, is the loss function, Represents the gradient.
4. The robust and efficient decentralized federated learning method based on committee consensus according to claim 1, characterized in that: In step 3, model parameter verification is performed, including: First, the trainer After the training, the committee members Send Message ,in, Indicates trainer The time spent on training, Indicates the size of the local data set, Indicates the number of training rounds; Then, committee members Received from trainer News After that, calculate the trainer Local Model and committee members Local Model Mutual information between ,in, Is a trainer Local Model The entropy of Is a member of the committee Local Model The entropy of yes and The joint entropy of Similarly, computational trainers Local Model and the initial global model Mutual information between , is the initial global model The entropy of yes and The joint entropy of Finally, judge and Are they both true? If true, then the trainer Local Model By verification, committee members For a verified local model Parameters for calculating the score .
5. The robust and efficient decentralized federated learning method based on committee consensus according to claim 1, characterized in that: In step 4, committee members Trainers who will be verified Related information and committee members The relevant information is uploaded to the blockchain; specifically including: Committee Members Update Trainer The information is , and committee members Related information Packaged together into a message , and the message Uploaded to the blockchain, where Member of the Committee Local training time, Member of the Committee For trainers The time it took to perform the verification.
6. The robust and efficient decentralized federated learning method based on committee consensus according to claim 1, characterized in that: In step 5, the smart contract aggregates the global model and calculates the trainer and committee members Contribution: Specifically include: When there are enough consecutive update blocks, the smart contract is automatically triggered; enough consecutive update blocks means more than Continuously update blocks; Check trainer Local Model Whether it is passed by more than Verification of committee members, if yes, using The aggregation algorithm updates all the models that have passed the validation Perform aggregation to obtain the global model of the current round ,in, Update the number for models that pass validation; otherwise, the trainer Local Model Will not be used for aggregation in the global model; According to the participants The role is different for each trainer and committee members Calculate contribution; based on the message And the aggregated results of this round to count the trainers and committee members Contributions to this round of training, not used for the final global model aggregation trainer The contribution of this round is zero.
7. The robust and efficient decentralized federated learning method based on committee consensus according to claim 6, characterized in that: The specific calculation method is as follows: trainer Contribution : ,in, For all committee members For trainers The average of the scores; Committee Members Contribution: ,in, Member of the Committee The number of verified models that were finally used for global model aggregation, and The contribution ratio of two different roles helps prevent participants from Continuing membership of the committee .
8. The robust and efficient decentralized federated learning method based on committee consensus according to claim 1, characterized in that: In step 6, the smart contract selects the next round of trainers and committee gathering ; Specifically include: First, sort all contributions in descending order from largest to smallest. Participants Those who have made great contributions to federated learning and are considered to be credible participants will become members of the next round of committees. Saved to the committee members collection middle; Next, the remaining participants become trainers in the next round of training. Saved in the trainer collection middle; Finally, the global model , the selected trainer set and committee gathering and the contributions of all participants Upload to blockchain.
9. The robust and efficient decentralized federated learning method based on committee consensus according to any one of claims 1 to 8, characterized in that: In step 7, federated learning participants Download the latest block, confirm your role in the next round of federated learning, and obtain the latest global model , and the latest global model As the initial model for the next round of training, repeat the above steps until the final global model converges.
10. A robust and efficient decentralized federated learning system based on committee consensus, characterized by: include: The committee member election module is configured to: conduct committee member elections; each participant’s initial contribution is zero. In the first round of elections, committee members are randomly selected using a verifiable random function. After a round of training, each participant obtains his or her own contribution; starting from the second round of elections, committee members are selected based on each participant’s contribution; The local model training module is configured as follows: both the trainer and committee members use the global model for local model training; The verification and scoring module is configured as follows: committee members verify the trainer's model and score the trainer who passes the verification, and the score will become part of the trainer's contribution; The aggregation and contribution calculation module is configured as follows: committee members upload their own information and the trainers they have verified to the blockchain; When there are enough consecutive update blocks, the smart contract will be automatically called to aggregate the model updates on the blockchain, generate a new global model, calculate the contribution of each participant in this round of training, select the committee members for the next round of training based on the contribution, and upload the new global model, the contribution of each participant, and the role of the participant in the next round of federated learning to the blockchain.
Citation Information
Cited By
Block chain-based scientific research record opening method and system
CN119577828A
Protein phosphorylation site prediction method based on federal learning
CN119943140A
