Decentralized verifiable federated learning method and device based on block chain

Abstract

The invention provides a decentralized verifiable federated learning method and device based on a block chain. Comprising the following steps: an activity initiator issues a task, a client calculates a local gradient, divides and transmits the local gradient to a server node, and calculates and uploads a gradient verification value to a verification main node; and the server nodes aggregate the gradient shares to obtain a part of global gradients, and calculate a part of verification values to be stored in the block chain. And the aggregation main node further aggregates to obtain a global gradient sum value for verification of participants in the next round. By comparing verification values on the block chain, the participant confirms whether the gradient is tampered or not, if not, the model is updated, and if not, the model is reported. And the verification main node traces and deletes the untrusted nodes by using the block chain. According to the method, the problems of single-point failure and bandwidth bottleneck possibly caused by a centralized architecture of traditional federated learning are effectively avoided; the safety of the user data in the training process is ensured; through a linear homomorphic Hash technology, verification of an aggregation result and tracking of a malicious server are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning privacy protection, and particularly relates to a decentralized verifiable federated learning method and device based on a blockchain. Background Art

[0002] With the rapid development of artificial intelligence technology, data-driven machine learning models have been widely applied in fields such as finance, transportation, and healthcare. However, large-scale data is scattered and stored in different institutions, forming "data silos", which seriously restricts the rapid development of artificial intelligence.

[0003] Federated learning is an innovative machine learning paradigm proposed to solve the data silo problem. Its core architecture supports data holders to participate in joint modeling under local storage conditions. Specifically, when implemented, after the participants train the model based on local data, they only submit the encrypted gradient parameters to the aggregation server for collaborative optimization. This mechanism maximizes data utility on the premise of meeting data security regulations, providing a reliable solution for cross-institutional data collaboration. However, in the current federated learning framework, the centralized federated learning architecture faces problems such as single-point failures and bandwidth bottlenecks; there are security risks in the frequent parameter interaction process between the participants and the aggregation server, and attackers can infer the sensitive information of the participants by analyzing the model parameters. In addition, malicious servers will tamper with the aggregation results and launch attacks against specific participants. These security threats seriously affect the healthy development of federated learning in actual scenarios.

[0004] To address these problems, some researchers have proposed a decentralized distributed federated learning architecture. However, the existing decentralized distributed federated learning architectures fail to balance privacy protection, computational efficiency, communication overhead, and system robustness. For example, although some decentralized frameworks avoid single-point failures, they lack an aggregation result verification mechanism, and malicious nodes can submit incorrect parameters without being detected; although blockchain-based solutions can trace tampering behaviors, they do not solve the problem of gradient privacy leakage. Therefore, there is an urgent need for a federated learning method that can achieve strict privacy protection and verify the correctness of aggregation results. Summary of the Invention

[0005] The present invention provides a decentralized verifiable federated learning method and device based on a blockchain to solve the defect of gradient privacy leakage in the prior art.

[0006] A decentralized verifiable federated learning method based on a blockchain includes:

[0007] Step 1: The activity initiator publishes a training task statement on the blockchain composed of server nodes for the client to determine whether it participates in the training;

[0008] Step 2: The participating clients calculate local gradients based on local data, split the local gradients into several gradient shares, and transmit them to the corresponding multiple server nodes respectively;

[0009] Calculate the linear homomorphic hash value of the local gradient to obtain a gradient verification value, and send the gradient verification value to the verification master node;

[0010] Step 3: The verification master node linearly combines all the received gradient verification values to obtain an initial verification value, and uploads the initial verification value and the participating user set to the blockchain;

[0011] Step 4: Each server node aggregates all the received gradient shares in the participating user set to obtain a partial global gradient, and sends the partial global gradient to the designated aggregation master node;

[0012] Calculate the linear homomorphic hash value of the partial global gradient to obtain a partial verification value, and submit the verification gradient information containing the partial verification value to the blockchain network for evidence storage;

[0013] Step 5: The aggregation master node aggregates the partial global gradients to obtain a first global gradient sum value, and sends the first global gradient sum value to the participants of the next round of training for the participants of the next round of training to calculate the linear homomorphic hash value of the first global gradient sum value to obtain a final verification value, and upload the final verification value to the blockchain;

[0014] Step 6: The participants of the next round of training obtain the initial verification value from the blockchain, and compare and verify the initial verification value with the final verification value to confirm whether the first global gradient sum value has been illegally tampered with. If there is no tampering, update the model parameters for the activity initiator to evaluate the termination condition of model training; if there is tampering, report the tampering information to the blockchain for the blockchain to notify the verification master node of the tampering information;

[0015] Step 7: The verification master node uses the information stored on the blockchain for traceability to confirm which of the aggregation master node and the server nodes is untrusted, and delete the untrusted nodes.

[0016] Further, for the blockchain-based decentralized verifiable federated learning method as described above, the step 1 includes:

[0017] The activity initiator publishes a training task announcement and publicity information on the blockchain; the training task announcement includes: the data types and scale requirements required by the training participants; the publicity information includes: the initial global model, public parameters; the public parameters include: the learning rate and the parameters of the linear homomorphic hash;

[0018] The training participant verifies whether they have the required data type through a non-interactive zero-knowledge proof mechanism. After the proof is submitted to the blockchain node, the smart contract deployed on the blockchain node verifies the zero-knowledge proof of the training participant; each training participant who passes the verification receives the network addresses and serial number information of the verification master node and all other server nodes provided by the system for subsequent connection establishment.

[0019] Further, for the blockchain-based decentralized verifiable federated learning method as described above, step two includes:

[0020] The training participant trains based on the initial global model, public parameters, and local data, and calculates the local gradient of the current round.

[0021] The training participant uses the Shamir threshold secret sharing mechanism to securely split the local gradient into several gradient shares, and transmits each gradient share and its corresponding serial number to the corresponding server node.

[0022] Each training participant calculates the gradient verification value of its own local gradient, and after distributing a certain number of gradient shares, sends the gradient verification value to the verification master node.

[0023] Further, for the blockchain-based decentralized verifiable federated learning method as described above, in step four, the information including the partial verification values includes: the current round of training, the server node ID, the user ID, and the partial verification values.

[0024] Further, for the blockchain-based decentralized verifiable federated learning method as described above, step five includes:

[0025] The aggregation master node uses the recovery mechanism of the Shamir threshold secret sharing mechanism to perform an aggregation recovery operation on all partial global gradients to obtain the total global gradient value.

[0026] Further, for the blockchain-based decentralized verifiable federated learning method as described above, step seven includes:

[0027] The verification master node recollects all partial global gradients of the current round from the server nodes, aggregates them to obtain the second total global gradient value, and calculates the linear homomorphic hash value of the second total global gradient value to obtain the master node verification value;

[0028] The verification master node compares the master node verification value with the final verification value obtained from the blockchain to determine whether they are consistent. If they are not consistent, it determines that the aggregated master node is untrusted and removes the untrusted aggregated master node.

[0029] Further, in the decentralized verifiable federated learning method based on blockchain as described above, step seven further includes:

[0030] Based on the verification gradient information obtained from the blockchain, the verification master node sends server node verification information to all clients included in the user ID, for all clients included in the user ID to calculate the share verification values corresponding to the server nodes to be verified and send the share verification values to the verification master node; the share verification value is the linear homomorphic hash value of the gradient share previously sent by the client to the server node to be verified; the server node verification information includes the ID of the server node to be verified.

[0031] The verification master node linearly combines the share verification values of all clients included in the user ID to obtain the server node verification value.

[0032] The verification master node compares and verifies the server node verification value with the partial verification value obtained from the blockchain to determine whether they are consistent. If they are not consistent, it determines that the corresponding server node is untrusted and removes the untrusted server node.

[0033] A decentralized verifiable federated learning device based on blockchain includes:

[0034] A publishing unit for the activity initiator to publish a training task statement on the blockchain composed of server nodes for clients to determine whether they participate in the training.

[0035] A first calculation unit for clients participating in the training to calculate local gradients based on local data.

[0036] A splitting unit for clients participating in the training to split the local gradient into several gradient shares.

[0037] A first transmission unit for clients participating in the training to respectively transmit the several gradient shares to the corresponding multiple server nodes.

[0038] The calculation unit is further used for clients participating in the training to calculate the linear homomorphic hash value of the local gradient to obtain a gradient verification value.

[0039] The transmission unit is further used for clients participating in the training to send the gradient verification value to the verification master node.

[0040] Combination unit, used to verify that the master node linearly combines all received gradient verification values to obtain an initial verification value;

[0041] Second transmission unit, used to verify that the master node uploads the initial verification value and the participating user set to the blockchain;

[0042] First aggregation unit, used for each server node to aggregate all received gradient shares to obtain a partial global gradient;

[0043] Third transmission unit, used for each server node to send the partial global gradient to a designated aggregation master node;

[0044] Second calculation unit, used for each server node to calculate the linear homomorphic hash value of the partial global gradient to obtain a partial verification value;

[0045] The third transmission unit is also used for each server node to submit verification gradient information containing the partial verification value to the blockchain network for evidence storage;

[0046] Second aggregation unit, used for the aggregation master node to aggregate the partial global gradients to obtain a first global gradient total value;

[0047] Fourth transmission unit, used to send the first global gradient total value to the participants in the next round of training for the participants in the next round of training to calculate the linear homomorphic hash value of the first global gradient total value to obtain a final verification value, and upload the final verification value to the blockchain;

[0048] Verification unit, used for the participants in the next round of training to obtain the initial verification value from the blockchain and compare and verify the initial verification value with the final verification value to confirm whether the first global gradient total value has been illegally tampered with;

[0049] Update unit, used for the participants in the next round of training to update the model parameters for the activity initiator to evaluate the termination condition of model training in the case of no tampering;

[0050] Reporting unit, used to report tampering information to the blockchain in the case of tampering for the blockchain to notify the verification master node of the tampering information;

[0051] Tracing unit, used for the verification master node to trace using the information stored on the blockchain to confirm which of the aggregation master node and the server nodes is untrusted and delete the untrusted nodes.

[0052] The decentralized verifiable federated learning method and device based on blockchain provided by the present invention utilize blockchain technology to deploy multiple server nodes as aggregation computing nodes at the network edge, reducing the system risk of the centralized architecture; construct a distributed ledger based on the consortium chain to achieve the transparency and traceability of the aggregation process; design an intelligent contract automated execution verification mechanism to enhance the system credibility. At the same time, a verifiable privacy protection mechanism based on Shamir threshold secret sharing is proposed, splitting the local gradient into multiple gradient shares to ensure the secure transmission of model parameters; under the Byzantine fault tolerance assumption, even if some server nodes are compromised, the original model cannot be restored; use linear homomorphic hashing technology to achieve efficient verification of the aggregation result; utilize the immutability of blockchain to record the verification process and support tracing and locating malicious server nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 One of the flowcharts of the decentralized verifiable federated learning method based on blockchain provided by the present invention;

[0054] Figure 2 Another flowchart of the decentralized verifiable federated learning method based on blockchain provided by the present invention;

[0055] Figure 3 The system model diagram of the decentralized verifiable federated learning method based on blockchain provided by the present invention;

[0056] Figure 4 The timing diagram of the decentralized verifiable federated learning method based on blockchain provided by the present invention;

[0057] Figure 5 The structural schematic diagram of the decentralized verifiable federated learning device based on blockchain provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without making creative efforts based on the embodiments in the present invention fall within the protection scope of the present invention.

[0059] The blockchain-based decentralized verifiable federated learning method, at the system design level, uses blockchain technology to build a decentralized model aggregation platform, effectively avoiding the single-point failure and bandwidth bottleneck problems that may be brought about by the centralized architecture of traditional federated learning; in terms of privacy protection, the Shamir threshold secret sharing mechanism is adopted to encrypt the local model, ensuring the security of user data during the training process; in terms of verifiability, the linear homomorphic hashing technology is used to verify the aggregation result, and smart contracts are used to trace and locate malicious server nodes.

[0060] Figure 3 The system model diagram of the blockchain-based decentralized verifiable federated learning method provided by the present invention is as Figure 3 shown. In the embodiment of the present invention, the server node includes: a verification master node and an aggregation master node. The server node is both a model aggregator and a blockchain consensus node, and they work together to perform secure decentralized aggregation of the model for the client. The client is responsible for local training and submits the training result to the server node for aggregation.

[0061] The server node constructs a consortium blockchain network through a consensus mechanism. The shared ledger therein serves both as a data sharing platform and a supervisor for decentralized model aggregation. The system deploys smart contracts on the consortium chain to record the verification data of the process and detect malicious behaviors. The client, as a data collector and holder, uses its local data to train a local model, then iteratively optimizes the local model through the stochastic gradient descent algorithm, and uses the Shamir threshold secret sharing mechanism to securely split the model parameters into multiple shares and distribute them to different server nodes. In addition, the client can also use the linear homomorphic hashing technology to verify the correctness of the aggregation result, thus ensuring the reliability of the training process.

[0062] The following combines Figure 3 , and gives an overview of the blockchain-based decentralized verifiable federated learning method provided by the present invention:

[0063] Release activity: The system starts a new aggregation cycle and announces relevant parameters for the client to refer to.

[0064] Data collection and preliminary verification: The server node collects the local training data from the client and conducts preliminary verification. This step ensures the legality and validity of the data.

[0065] First aggregation: The data that has passed the preliminary verification is passed to other server nodes for the first aggregation.

[0066] Second aggregation: The result of the first aggregation is further processed for the second aggregation.

[0067] Partial global gradient: After two aggregations, the system generates a partial global gradient.

[0068] Global gradient sum value: All partial global gradients are aggregated into a global gradient sum value.

[0069] Gradient share verification: The server node compares the global gradient sum value with the previously recorded gradient shares to verify the correctness of the aggregation result. This step is crucial for ensuring data consistency and integrity.

[0070] Verify the global gradient: Finally, the system verifies the global gradient. This step records the verification data on the consortium blockchain through a smart contract and detects any potential malicious behavior. Once the verification passes, the global gradient is broadcast to all participants, marking the end of the entire aggregation cycle.

[0071] The following combines Figure 1 , Figure 2 , and details the blockchain-based decentralized verifiable federated learning method provided by the present invention, specifically including the following steps:

[0072] Step 1: The activity initiator publishes a training task statement on the blockchain composed of server nodes for the client to determine whether to participate in the training.

[0073] Specifically, first, initialization is performed, that is: the activity initiator publishes a training task statement on the blockchain, and the training task statement contains task metadata such as hyperparameter specifications and data quality requirements for the client to make a subscription decision; the client determined to participate in the training will obtain configuration information including server node identifiers and network topology parameters. To support decentralized model aggregation, the system generates cryptographic public parameters and writes them into an immutable ledger. The blockchain network is deployed in a server node cluster, and the consistency of the distributed state machine is guaranteed through the Byzantine fault-tolerant consensus protocol, and at the same time, a smart contract is used to record the process verification data and detect malicious behavior.

[0074] Step 2: The client participating in the training calculates the local gradient based on the local data, divides the local gradient into several gradient shares, and transmits them to the corresponding multiple server nodes respectively. Calculate the linear homomorphic hash value of the local gradient to obtain a gradient verification value, and send the gradient verification value to the verification master node;

[0075] Specifically, the participating clients train the model based on the announced parameters and their respective local datasets. First, a local model is obtained through local training, and then the local gradient is calculated using the Stochastic Gradient Descent (SGD) algorithm. To protect data privacy, the clients use the Shamir threshold secret sharing mechanism to securely split the local gradient and distribute the gradient shares to different server nodes. At the same time, the clients generate linear homomorphic hash values based on the local gradient and synchronously submit them to the verification master node. By splitting the local gradient into multiple gradient shares, even if a certain server node is compromised, the attacker cannot restore the original model parameters. This design not only protects the privacy of users but also ensures the effectiveness and accuracy of model training.

[0076] In the decentralized verifiable federated learning method based on blockchain provided by the present invention, the activity initiator publishes a training task on the blockchain, and the clients independently decide whether to participate in the training and split the local gradient into multiple gradient shares and send them to different server nodes. This design avoids the problem that a single node undertakes too many responsibilities in the traditional centralized architecture because the data is no longer centrally stored on a single server but is dispersed among multiple server nodes. In this way, even if a certain node fails or is attacked, it will not cause a catastrophic impact on the entire system.

[0077] Step 3: The verification master node linearly combines all the received gradient verification values to obtain an initial verification value, and uploads the initial verification value and the set of participating users to the blockchain.

[0078] Specifically, the verification master node collects the gradient verification values sent from all the clients participating in the training, linearly combines them to calculate the initial verification value, and then uploads the initial verification value and the set of users participating in the training to the blockchain.

[0079] Step 4: Each server node aggregates all the received gradient shares to obtain a partial global gradient, and sends the partial global gradient to the designated aggregation master node; calculates the linear homomorphic hash value of the partial global gradient to obtain a partial verification value, and submits the verification gradient information containing the partial verification value to the blockchain network for deposit;

[0080] Specifically, the server node aggregates all the gradient shares to obtain a partial global gradient, calculates the partial verification value through linear homomorphic hashing, and uploads it to the blockchain in a specific format. And calculates its linear homomorphic hash value and submits it to the blockchain network for deposit, so that the verification master node can use the information it uploads later to verify whether the corresponding server node is trustworthy.

[0081] Step 5: The aggregation master node aggregates the partial global gradients to obtain the first total global gradient value, and sends the first total global gradient value to the participants in the next round of training for the participants in the next round of training to calculate the linear homomorphic hash value of the first total global gradient value to obtain the final verification value, and upload the final verification value to the blockchain.

[0082] Specifically, the smart contract dynamically elects the aggregation master node and the standby node (for dealing with the situation where the aggregation master node fails) through predefined policies. All partial global gradients are directionally transmitted to the aggregation master node, which performs the reconstruction calculation of the global gradient.

[0083] Step 6: The participants in the next round of training obtain the initial verification value from the blockchain, and compare and verify the initial verification value with the final verification value to confirm whether the first total global gradient value has been illegally tampered with. If there is no tampering, the model parameters are updated for the activity initiator to evaluate the termination condition of the model training; if there is tampering, the tampering information is reported to the blockchain for the blockchain to notify the verification master node of the tampering information.

[0084] Specifically, the participants in the next round of training start a two-stage verification mechanism after receiving the data: perform data integrity verification based on linear homomorphic hashing to confirm that the global gradient has not been illegally tampered with; after passing the verification, calculate the global model initialization parameters for the next training cycle. Subsequently, the termination condition evaluation module is activated. If it is satisfied, the system will terminate the training; otherwise, a new round of training scheduling instructions is generated.

[0085] Step 7: The verification master node uses the information stored on the blockchain for traceability to confirm which of the aggregation master node and the server node is untrusted, and deletes the untrusted node.

[0086] Specifically, when the verification master node, the server node, and the aggregation master node execute their respective tasks, they all need to upload relevant information to the blockchain. For example, the verification master node uploads the initial verification value and the set of participating users, the server node submits partial verification values for archiving, and the aggregation master node is responsible for generating the total global gradient value and providing verification for the participants in the next round of training. The above solution jointly constructs a distributed ledger based on the consortium chain, making the entire aggregation process transparent and traceable. Any node can view the records on the blockchain at any time to verify whether the behavior of other nodes meets the expectations.

[0087] In addition, through the automated execution verification mechanism of smart contracts in this application, the system can automatically detect and respond to potential tampering behaviors without relying on manual intervention. When the participants in the next round of training discover that the total value of the global gradient has been illegally tampered with, they can report the tampering information to the blockchain, and the verification master node uses the information stored on the blockchain for tracing to confirm which node is untrusted and delete it. This mechanism greatly improves the credibility of the system because it ensures that the system can maintain its integrity and security even in the most adverse circumstances.

[0088] Moreover, this application adopts the linear homomorphic hashing technology, which can achieve efficient verification of the aggregation result. At the same time, under the Byzantine fault tolerance assumption, even if some server nodes are compromised and attempt to send incorrect information, the system can detect and correct these errors by comparing the verification values. This design ensures that the system can remain stable and reliable in the face of malicious attacks. And by utilizing the immutability of the blockchain, it can record the entire verification process and support tracing and locating malicious server nodes. This means that once a node is proven to be untrusted or a malicious actor, it will be permanently recorded on the blockchain and receive corresponding penalties. This mechanism helps to maintain the fairness and security of the system.

[0089] In summary, the decentralized verifiable federated learning method based on blockchain provided by the present invention utilizes blockchain technology to deploy multiple server nodes as aggregation computing nodes at the network edge, reducing the system risk of the centralized architecture; constructs a distributed ledger based on the consortium chain to achieve the transparency and traceability of the aggregation process; designs an automated execution verification mechanism of smart contracts to improve the credibility of the system. At the same time, a verifiable privacy protection mechanism based on threshold secret sharing is proposed to split the local gradient into multiple gradient shares to ensure the secure transmission of model parameters; under the Byzantine fault tolerance assumption, even if some server nodes are compromised, the original model cannot be restored; the linear homomorphic hashing technology is adopted to achieve efficient verification of the aggregation result; the immutability of the blockchain is used to record the verification process and support tracing and locating malicious server nodes.

[0090] Further, the specific implementation of step one is as follows:

[0091] The activity initiator publishes a training task announcement and publicity information on the blockchain; the training task announcement includes: the data type and scale requirements required by the training participants; the publicity information includes: the initial global model, public parameters; the public parameters include: the learning rate and the parameters of the linear homomorphic hash ; The training participants verify whether they have the required data types through a non-interactive zero-knowledge proof mechanism. After the proof is submitted to the blockchain node, the smart contract deployed on the blockchain node verifies the zero-knowledge proof of the training participants; each verified training participant receives the network addresses and serial number information of the verification master node and all other server nodes provided by the system for subsequent connection establishment.

[0092] Specifically, the activity initiator publishes a training activity announcement on the blockchain, clearly stating the required data types and their scale requirements. At the same time, the initial global model and parameters (including the learning rate and the parameters of the linear homomorphic hash ) are uploaded to the blockchain for public display. After other clients view the activity announcement, they can decide whether to subscribe. The client verifies that it has the required data types through a non-interactive zero-knowledge proof mechanism (user ). After the proof is submitted to the blockchain node, the smart contract will verify the user's zero-knowledge proof. After verification, the user will be regarded as a training participant. The system then provides each participant with the network addresses and serial number information of the verification master node and all other server nodes (a total of nodes) for subsequent connection establishment.

[0093] Furthermore, the above-mentioned step two specifically includes the following solutions:

[0094] The training participants train based on the initial global model, public parameters, and local data, and calculate the local gradient of the current round; the training participants use the Shamir threshold secret sharing mechanism to securely split the local gradient, split it into several gradient shares, and transmit each gradient share and its corresponding serial number to the corresponding server node; the training participants calculate the gradient verification value of each local gradient, and after distributing a certain number of gradient shares, send all gradient verification values to the verification master node.

[0095] Specifically, the participant ( ) trains based on the global model , public parameters , and local data to calculate the local gradient of the current round. Subsequently, the participant performs secret sharing on the local gradient ( ), splits it into gradient shares, , and sends them to the server node corresponding to the serial number. At the same time, the participant Calculate the local gradient of the linear homomorphic hash value , and after completing the distribution of the gradient shares, send to the verification master node . The master node receives linear homomorphic hash values, and performs a linear combination of the homomorphic hash values to generate an initial verification value (the initial verification value in the round is obtained by linearly combining the linear homomorphic hash values of the local gradients of participants). Finally, upload and the set of participating users to the blockchain. and the set of participating users to the blockchain.

[0096] Furthermore, each server node aggregates all the received gradient shares to obtain a partial global gradient, and sends the partial global gradient to the designated aggregation master node, specifically including: the server node aggregates the gradient shares to obtain a partial global gradient, calculates a partial verification value through linear homomorphic hashing, and then uploads it to the blockchain in a specific format. After completing the above steps, the server node sends the partial global gradient to the aggregation master node for the aggregation master node to calculate the sum of the global gradients using the recovery mechanism of Shamir secret sharing.

[0097] Specifically, the server node performs an aggregation operation on all the gradient shares in the set of participating users to obtain a partial global gradient , (the result of aggregating the gradient shares sent by the participants in the server in the round); and calculates the corresponding linear homomorphic hash value according to the partial global gradient to obtain a partial verification value ; the server node uploads the training round t, the server node E ID , the user U ID , the partial verification value to the blockchain in a specific format, and sends the partial global gradient to the designated aggregation master node ; the aggregation master node uses the recovery mechanism of Shamir secret sharing for all the partial global gradients ; the aggregation master node uses the recovery mechanism of Shamir secret sharing for all the partial global gradients ​Perform an aggregation recovery operation to obtain the global gradient sum value ; (In the round, partial global gradients are reconstructed into the global gradient sum using the recovery mechanism of Shamir secret sharing.)

[0098] Furthermore, the fifth step includes: the aggregation master node performs an aggregation recovery operation on all partial global gradients using the recovery mechanism of the Shamir threshold secret sharing mechanism to obtain the global gradient sum value.

[0099] In the embodiment of the present invention, there are two aggregations. The first aggregation: the server node aggregates all gradient shares in the participating user set to obtain partial global gradients . Subsequently, calculate the linear homomorphic hash value of the partial global gradient to obtain the partial verification value , and this information will be uploaded to the blockchain in a specific format. The data format is (round node user ). After completing the above steps, the server node sends the partial global gradients to the designated aggregation master node . The second aggregation: the aggregation master node collects all partial global gradients and performs a final aggregation recovery operation using the recovery mechanism of Shamir secret sharing to calculate the global gradient sum value .

[0100] Furthermore, the sixth step specifically includes:

[0101] The participants in the next round of training receive the global gradient sum value sent by the aggregation master node and the scale of the participating user set ; the participants in the next round calculate the linear homomorphic hash value of the global gradient sum value , , and compare it with the initial verification value . After passing the verification, calculate the global gradient using the average gradient, ; where the condition for passing the verification is: ; Calculate the global initial model for the round.

[0102] Aggregation master node Distribute the total global gradient value and the scale of the set of participating users to the participants in the next round of training (assuming the number is ). The participants in the next round first calculate the linear homomorphic hash value of and compare and verify it with the initial verification value . After passing the verification ( ), calculate the global gradient using the average gradient. Subsequently, calculate the global initial model for the round (using the initial global model for the round , the globally trained gradient and the learning rate to calculate the initial global model for the round ).

[0103] After completing the above steps, the activity initiator will evaluate the termination condition of the model training: if the model reaches the convergence standard, submit a request to end the training to the system; if it does not converge and the preset number of training rounds is not reached, continue with the next round of training.

[0104] Furthermore, if it is found in step six that the global gradient has been illegally tampered with, trace it using the information stored on the blockchain. First, check whether data tampering occurred during the second aggregation process. Second, check whether data tampering occurred during the first aggregation process to identify possible malicious server nodes. The system can accurately identify malicious server nodes that perform data tampering through the above double-layer verification. Once a malicious server node is confirmed, the system will immediately remove it from the subsequent training process until the necessary security maintenance work is completed.

[0105] Next, how to verify which of the aggregation master node and the server node is untrusted will be introduced. First, how to verify whether the aggregation master node is trusted will be introduced, and the specific solutions are as follows:

[0106] Verify that the master node re-collects all partial global gradients of the current round from the server nodes, aggregates them to obtain the second total global gradient value, and calculates the linear homomorphic hash value of the second total global gradient value to obtain the master node verification value; verify that the master node compares the master node verification value with the final verification value obtained from the blockchain to determine whether they are consistent. If they are not consistent, it is determined that the aggregation master node is untrusted, and the untrusted aggregation master node is removed.

[0107] Specifically, check whether data tampering occurs during the second aggregation process, that is, check whether the aggregation master node is malicious by comparing with the verification value obtained after the verification master node performs the second aggregation to see if they are the same. If the results are the same ( ), it is determined that the aggregation master node is trustworthy. Otherwise, it indicates that the aggregation master node tampered with the data during the second aggregation process.

[0108] The following introduces how to verify whether a server node is trustworthy, specifically including the following steps:

[0109] Based on the verification gradient information obtained from the blockchain, the verification master node sends server node verification information to all clients included in the user ID, so that all clients included in the user ID can calculate the share verification value corresponding to the server node to be verified and send the share verification value to the verification master node; the share verification value is the linear homomorphic hash value of the gradient share sent by the client to the server node to be verified; the server node verification information includes the ID of the server node to be verified; the verification master node linearly combines the share verification values of all clients included in the user ID to obtain the server node verification value; the verification master node compares and verifies the server node verification value with the partial verification value obtained from the blockchain to determine whether they are the same. If they are not the same, it is determined that the corresponding server node is untrustworthy and the untrustworthy server node is excluded.

[0110] Specifically, from the verification gradient information obtained by the verification master node from the blockchain, obtain the current training round, server node ID, user ID, and partial verification value, and based on the user ID, send server node verification information to all clients included in it (for example: the server node verification information is: the verification information for verifying server node 1), so that all clients can calculate the hash value corresponding to the gradient share sent to server node 1 before, thereby obtaining the share verification value corresponding to this server node 1 and sending the share verification value to the verification master node;

[0111] The verification master node linearly combines the share verification values calculated by all clients for server node 1 to obtain the server node verification value; then, the verification master node compares and verifies the server node verification value for server node 1 with the partial verification value obtained from the blockchain to determine whether they are the same. If they are not the same, it is determined that server node 1 is untrustworthy and server node 1 is excluded.

[0112] Specifically, check whether data tampering occurs during the first aggregation process to identify possible malicious server nodes. The verification master node is based on the information recorded on the blockchain (round Node User ), perform tracking, and request the linear homomorphic hash value corresponding to the gradient share initially submitted by the relevant client . Through linear homomorphic hashing operations, verify and match the calculation result with the partial verification value stored in the blockchain ( ), so as to accurately identify the malicious server node that has performed the tampering operation

[0113] Through the above verification steps, the system can accurately identify the malicious server node that has tampered with the data through the above double-layer verification. Once the malicious server node is confirmed, the system will immediately remove it from the subsequent training process until the necessary security maintenance work is completed

[0114] Next, in combination with Figure 2 、 Figure 4 The decentralized verifiable federated learning method based on blockchain provided by the present invention is introduced as a whole, including the following steps

[0115] Step 1. The activity initiator publishes a training activity announcement on the blockchain, clearly stating the required data type and its scale requirements. At the same time, upload the initial global model and parameters (including the learning rate and the parameters of the linear homomorphic hash ) to the blockchain for public notice. After other clients view the activity announcement, they can decide whether to subscribe. The client uses the non-interactive zero-knowledge proof mechanism (user ) to verify that it has the required data type. After the proof is submitted to the blockchain node, the smart contract will verify the user's zero-knowledge proof. After the verification passes, the user will be regarded as a training participant. Subsequently, the system provides each participant with the network addresses and serial number information of the verification master node and all other server nodes (a total of ) for subsequent connection establishment

[0116] Step 2. The participant ( ) performs training based on the global model , public parameters and local data to calculate the local gradient of the current round . Among them, the local gradient of the current round represents the local gradient generated by the client through local training in the th round of training. Subsequently, the participant performs secret sharing on the local gradient ( ), split into gradient shares, , representing the th round, where the local gradients of the participants are split through Shamir secret sharing and sent to the server as shares, and sent to the server nodes corresponding to the serial numbers. Meanwhile, the participants calculate the linear homomorphic hash value of the local gradient (indicating that in the th round, the homomorphic hash value of the local gradient of the participant is calculated using the linear homomorphic hash algorithm), and after completing the distribution of the gradient shares, is sent to the verification master node (the server nodes are all represented by , and represents the server node for verification). After receiving the linear homomorphic hash values, the master node performs a linear combination of the homomorphic hash values to generate the initial verification value . Finally, and the set of participating users are uploaded to the blockchain. . and the set of participating users are uploaded to the blockchain.

[0117] Step 3.1. First aggregation: The server node performs an aggregation operation on all the gradient shares in the set of participating users to obtain the partial global gradient . Subsequently, calculate the linear homomorphic hash value of the partial global gradient to obtain the partial verification value , and this information will be uploaded to the blockchain in a specific format, with the data format being (round node user ). After completing the above steps, the server node sends the partial global gradient to the designated aggregation master node .

[0118] Step 3.2. Second aggregation: The aggregation master node collects all the partial global gradients and uses the recovery mechanism of Shamir secret sharing to perform the final aggregation recovery operation to calculate the total global gradient value .

[0119] Step 4. The aggregation master node compares the total global gradient value with the size of the set of participating users Distribute to the participants in the next round of training (assuming the number is ). The participants in the next round first calculate 's linear homomorphic hash value , and compare it with the initial verification value (in the th round, the initial value used to verify the final result) for verification. After passing the verification ( ), calculate the global gradient using the average gradient. Subsequently, calculate the global initial model for the th round.

[0120] After completing the above steps, the activity initiator will evaluate the termination condition of model training: if the model reaches the convergence standard, submit a request to end the training to the system; if it does not converge and does not reach the preset number of training rounds, continue to execute the next round of training.

[0121] Step 5.1. Check whether data tampering occurs during the second aggregation process, that is, check whether the aggregation master node is malicious by comparing with the verification value obtained by the verification master node to see if they are consistent ( , ). If the results are consistent ( ), the aggregation master node is determined to be trustworthy. Otherwise, it means that the aggregation master node tampered with the data during the second aggregation process.

[0122] Step 5.2. Check whether data tampering occurs during the first aggregation process to identify possible malicious server nodes. The verification master node traces the information based on the blockchain record (round node user ), and requests the linear homomorphic hash value corresponding to the initial submitted gradient share from the relevant client. Through linear homomorphic hashing operations, verify and match the calculation results with the partial verification values stored in the blockchain ( ), so as to accurately identify the malicious server nodes that performed the tampering operation.

[0123] Through the above verification steps, the system can accurately identify the malicious server nodes that perform data tampering through the above double-layer verification. Once a malicious server node is confirmed, the system will immediately remove it from the subsequent training process until the necessary security maintenance work is completed.

[0124] The decentralized verifiable federated learning method based on blockchain provided by the present invention has the following advantages:

[0125] Privacy protection: The local gradients of each client are strictly protected. Neither the server nodes nor other clients can obtain the local gradient information of any specific client. Even in the case of collusion among some server nodes, the privacy protection mechanism of the system cannot be breached.

[0126] Robustness: The system has strong fault tolerance and operational stability. Specifically, it can tolerate the delayed submission or mid-way exit of clients; it can still operate normally when some server nodes are offline (attacked or faulty); it ensures the correct execution of the aggregation operation.

[0127] Verifiability and traceability: It has a comprehensive result verification and tracing mechanism. In terms of correctness verification, the client can verify the correctness of the aggregation result returned by the server node to prevent malicious server nodes from providing forged results. At the same time, it has the tracing ability. Once it detects that the aggregation result has been maliciously tampered with, it can locate and identify the relevant malicious server nodes to ensure the secure operation of the system.

[0128] Low loss of model accuracy: While achieving the above properties, it is also necessary to ensure that the privacy protection mechanism does not significantly affect the accuracy of the model.

[0129] The method of the present invention, by combining technologies such as blockchain, threshold secret sharing, and linear homomorphic hashing, realizes the verifiability of the model aggregation result and the traceability of malicious servers while protecting user privacy in a decentralized federated learning framework.

[0130] As Figure 5 shown, the present invention also provides a decentralized verifiable federated learning device based on blockchain, including:

[0131] A publishing unit 5101, configured to enable an activity initiator to publish a training task statement on a blockchain composed of server nodes for clients to determine whether they participate in the training;

[0132] A first computing unit 5201, configured to enable clients participating in the training to calculate local gradients based on local data;

[0133] A splitting unit 5202, configured to enable clients participating in the training to split the local gradients into several gradient shares;

[0134] A first transmission unit 5203, configured to enable clients participating in the training to respectively transmit the several gradient shares to corresponding multiple server nodes;

[0135] The first computing unit 5201 is further configured to enable clients participating in the training to calculate a linear homomorphic hash value of the local gradient to obtain a gradient verification value;

[0136] The first transmission unit 5203 is further configured to enable the client participating in the training to send the gradient verification value to the verification master node;

[0137] The combination unit 5301 is configured to enable the verification master node to linearly combine all the received gradient verification values to obtain an initial verification value;

[0138] The second transmission unit 5302 is configured to enable the verification master node to upload the initial verification value and the participating user set to the blockchain;

[0139] The first aggregation unit 5401 is configured to enable each server node to aggregate all the received gradient shares to obtain a partial global gradient;

[0140] The third transmission unit 5402 is configured to enable each server node to send the partial global gradient to a designated aggregation master node;

[0141] The second calculation unit 5403 is configured to enable each server node to calculate a linear homomorphic hash value of the partial global gradient to obtain a partial verification value;

[0142] The third transmission unit 5402 is further configured to enable each server node to submit verification gradient information including the partial verification value to the blockchain network for evidence storage;

[0143] The second aggregation unit 5501 is configured to enable the aggregation master node to aggregate the partial global gradients to obtain a first global gradient total value;

[0144] The fourth transmission unit 5502 is configured to send the first global gradient total value to the participants in the next round of training for the participants in the next round of training to calculate a linear homomorphic hash value of the first global gradient total value to obtain a final verification value, and upload the final verification value to the blockchain;

[0145] The verification unit 5204 is configured to enable the participants in the next round of training to obtain the initial verification value from the blockchain and compare and verify the initial verification value with the final verification value to confirm whether the first global gradient total value has been illegally tampered with;

[0146] The update unit 5205 is configured to enable the participants in the next round of training to update the model parameters for the activity initiator to evaluate the termination condition of the model training in the case of no tampering;

[0147] The reporting unit 5206 is configured to report the tampering information to the blockchain in the case of tampering for the blockchain to notify the verification master node of the tampering information;

[0148] A tracing unit 5303 is used to verify that the master node performs tracing by using the information stored on the blockchain to confirm which of the aggregated master node and the server node is untrusted, and delete the untrusted node.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A decentralized and verifiable federated learning method based on blockchain, characterized in that: include: Step 1: The activity initiator publishes a training task statement on the blockchain composed of server nodes for the client to determine whether it is participating in the training; Step 2: The client participating in the training calculates the local gradient based on the local data, and divides the local gradient into several gradient shares and transmits them to the corresponding multiple server nodes respectively; Calculate the linear homomorphic hash value of the local gradient to obtain a gradient verification value, and send the gradient verification value to the verification master node; Step 3: The verification master node linearly combines all received gradient verification values ​​to obtain an initial verification value, and uploads the initial verification value and the participating user set to the blockchain; Step 4: Each server node aggregates all the gradient shares received from the participating user set to obtain a partial global gradient, and sends the partial global gradient to the designated aggregation master node; Calculate the linear homomorphic hash value of the partial global gradient to obtain a partial verification value, and submit the verification gradient information including the partial verification value to the blockchain network for evidence storage; Step 5: The aggregation master node aggregates the partial global gradients to obtain a first global gradient sum value, and sends the first global gradient sum value to the participants of the next round of training, so that the participants of the next round of training can calculate the linear homomorphic hash value of the first global gradient sum value, obtain the final verification value, and upload the final verification value to the blockchain; Step 6: The participants of the next round of training obtain the initial verification value from the blockchain, and compare and verify the initial verification value with the final verification value to confirm whether the first global gradient sum value has been illegally tampered with. If not, the model parameters are updated for the activity initiator to evaluate the termination conditions of the model training; If tampered, report the tampered information to the blockchain so that the blockchain can notify the verification master node of the tampered information; Step 7: Verify that the master node uses the information stored on the blockchain to trace back to confirm which of the aggregation master node and server node is untrustworthy, and delete the untrustworthy node.

2. The decentralized verifiable federated learning method based on blockchain according to claim 1, characterized in that: The step one comprises: The activity initiator publishes a training task announcement and public information on the blockchain; the training task announcement includes: the data type and scale requirements required for the training participants; the public information includes: the initial global model, public parameters; the public parameters include: the learning rate and the parameters of the linear homomorphic hash; The training participant verifies whether he or she has the required data type through a non-interactive zero-knowledge proof mechanism. After the proof is submitted to the blockchain node, the smart contract deployed on the blockchain node verifies the zero-knowledge proof of the training participant; each verified training participant receives the network address and serial number information of the verification master node and all other server nodes provided by the system for subsequent connection establishment.

3. The decentralized verifiable federated learning method based on blockchain according to claim 2, characterized in that: The second step comprises: The training participants perform training based on the initial global model, public parameters and local data, and calculate the local gradient of the current round; The training participant uses the Shamir threshold secret sharing mechanism to securely split the local gradient into a plurality of gradient shares, and transmits each of the gradient shares and its corresponding sequence number to a corresponding server node; Each of the training participants calculates the gradient verification value of its own local gradient, and after completing the distribution of a certain number of gradient shares, sends the gradient verification value to the verification master node.

4. The decentralized verifiable federated learning method based on blockchain according to claim 1, characterized in that: In the step 4, the verification gradient information includes: the current training round, the server node ID, the user ID, and the partial verification value.

5. The decentralized verifiable federated learning method based on blockchain according to claim 3, characterized in that: The step five comprises: The aggregation master node uses the recovery mechanism of the Shamir threshold secret sharing mechanism to perform an aggregation recovery operation on all partial global gradients, thereby obtaining the global gradient sum value.

6. The decentralized verifiable federated learning method based on blockchain according to claim 3, characterized in that: The step seven comprises: The verification master node recollects all partial global gradients of the current round from the server node, aggregates them, obtains the second global gradient sum value, and calculates the linear homomorphic hash value of the second global gradient sum value, thereby obtaining the master node verification value; The verification master node compares and verifies the master node verification value with the final verification value obtained from the blockchain to determine whether the two are consistent. If they are inconsistent, the aggregation master node is determined to be untrustworthy and the untrustworthy aggregation master node is eliminated.

7. The decentralized verifiable federated learning method based on blockchain according to claim 4, characterized in that: The step seven also includes: The verification master node sends server node verification information to all clients included in the user ID based on the verification gradient information obtained from the blockchain, so that all clients included in the user ID can calculate the share verification value corresponding to the server node to be verified, and send the share verification value to the verification master node; the share verification value is the linear homomorphic hash value of the gradient share previously sent by the client to the server node to be verified; the server node verification information includes the server node ID to be verified; The verification master node linearly combines the share verification values ​​of all clients contained in the user ID to obtain the server node verification value; The verification master node compares and verifies the server node verification value with the partial verification value obtained from the blockchain to determine whether the two are consistent. If they are inconsistent, the corresponding server node is determined to be untrustworthy and the untrustworthy server node is removed.

8. A decentralized verifiable federated learning device based on blockchain, characterized in that: include: A publishing unit, which is used by the activity initiator to publish the training task statement on the blockchain composed of server nodes so that the client can determine whether it is participating in the training; A first computing unit, used for the client participating in the training to calculate a local gradient based on local data; A segmentation unit, used for the client participating in the training to segment the local gradient into a plurality of gradient shares; A first transmission unit, configured for the client participating in the training to transmit the plurality of gradient shares to the corresponding plurality of server nodes respectively; The computing unit is further used for the client participating in the training to calculate the linear homomorphic hash value of the local gradient to obtain a gradient verification value; The transmission unit is also used for the client participating in the training to send the gradient verification value to the verification master node; The combination unit is used to verify that the master node linearly combines all the received gradient verification values ​​to obtain an initial verification value; A second transmission unit is used to verify that the master node uploads the initial verification value and the participating user set to the blockchain; A first aggregation unit, configured for each server node to aggregate all received gradient shares to obtain a partial global gradient; A third transmission unit, configured for each server node to send the part of the global gradient to a designated aggregation master node; A second calculation unit is used for each server node to calculate the linear homomorphic hash value of the partial global gradient to obtain a partial verification value; The third transmission unit is further configured for each server node to submit the verification gradient information including the partial verification value to the blockchain network for storage; A second aggregation unit, configured for the aggregation master node to aggregate the partial global gradients to obtain a first global gradient sum value; a fourth transmission unit, configured to send the first global gradient sum value to participants of the next round of training, so that the participants of the next round of training can calculate the linear homomorphic hash value of the first global gradient sum value, obtain a final verification value, and upload the final verification value to the blockchain; A verification unit, used for a participant in the next round of training to obtain the initial verification value from the blockchain, and compare and verify the initial verification value with the final verification value to confirm whether the first global gradient sum value has been illegally tampered with; The updating unit is used by the participants of the next round of training to update the model parameters without tampering so that the activity initiator can evaluate the termination conditions of the model training; A reporting unit, used to report the tampering information to the blockchain in case of tampering, so that the blockchain notifies the verification master node of the tampering information; The traceability unit is used to verify that the master node uses the information stored on the blockchain for traceability to confirm which of the aggregate master node and server node is untrustworthy and delete the untrustworthy node.

Images