A hybrid aggregation mechanism for federated learning under blockchain architecture

By adopting a mixed aggregation mechanism under the blockchain architecture in the federal study, combining sub -sampling, shuffling and Shape value, the problem of user privacy and data usability in federal learning has been solved, and efficient privacy protection and data assessment have been achieved.

CN115987612BActive Publication Date: 2025-01-28DALIAN UNIV OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211642700.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-20
Publication Date
2025-01-28
Estimated Expiration
2042-12-20

AI Technical Summary

Technical Problem

In federated learning, user privacy faces attacks, differential privacy perturbations reduce data availability, and blockchain-based federated learning computing overhead is high and cannot accurately evaluate user contributions.

Method used

The federal learning mixed aggregation mechanism under the blockchain architecture is combined with sub -samples, shuffle and Shapping interest value. Through local differential privacy disturbances and gradient selection to protect the privacy of users, and the secondary sub -sampling and washing at the reshuffle server is performed at the shuffle server Card, reduce the amount of disturbance and improve data usability.

Benefits of technology

While ensuring user privacy, improve data usability, realize the accuracy of model prediction and reasonable allocation of rewards, and solve the problem of balanced availability of user data privacy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115987612B_ABST
    Figure CN115987612B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of machine learning privacy and security technology, and provides a federated learning hybrid aggregation mechanism under a blockchain architecture. Using local servers as management and miner nodes, the network is divided into multiple local domains consisting of local servers and terminal devices, and an inter-domain network consisting of local server connections. In the local domain, a coding algorithm that satisfies local differential privacy is used to select gradients to defend against gradient exponential attacks; a perturbation algorithm that satisfies local differential privacy is used to perturb gradients to defend against gradient content attacks; the public key of the local server is used to encrypt the gradient to defend against eavesdropping attacks; a shuffle server is used to shuffle and sample user updates to defend against user identity attacks. A local server is used to decrypt and aggregate updates within the domain, and an update evaluation and consensus mechanism based on Shapley values ​​is used between domains to issue rewards and update models. Users within the domain share the rewards equally to ensure user privacy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a federated learning hybrid aggregation mechanism based on blockchain and differential privacy amplification mechanism, and belongs to the field of machine learning security domain privacy technology. Background Art

[0002] Machine learning can provide personalized services to users by centrally collecting large amounts of data and training multiple prediction models. However, in practical applications, data is often distributed across different organizations. Federated learning aggregates users' model updates after local training, rather than the original data to update the global model. This solves the data isolation problem in traditional machine learning and provides better data privacy, but it still faces a variety of privacy attacks.

[0003] In the federated learning task model, local users upload gradients as model updates after training, and the dimension of the gradient increases with the dimension of the training data. In order to improve the update efficiency, each update of federated learning will select some gradients to upload, but this may leak user privacy, because attackers can conduct privacy attacks on the content of local model updates and the identity of their uploaders through monitoring and inference based on background knowledge to obtain user updated content or locate user records. In general, user privacy still faces privacy threats from user identity, gradient index, and gradient content.

[0004] User identity privacy refers to the threat that an attacker can obtain user update behavior or identity information based on monitoring or inference attacks. For example, an attacker can determine when a user starts updating, whether a user participates in a round of uploads, or which update corresponds to a specific user based on channel monitoring and background knowledge inference attacks. Gradient index privacy refers to the threat that an attacker can infer user data or identity based on the obtained gradient index. For example, an attacker determines whether a gradient belongs to a larger gradient of a user, or whether certain information is contained in the user's training data. Gradient content privacy refers to the threat that an attacker can infer user training data based on the obtained gradient content. For example, an attacker can determine whether a specific user participates in uploads based on their salient features, or infer user privacy based on background knowledge of multiple updates.

[0005] To mitigate privacy threats in federated learning, differential privacy is widely used to perturb updates to resist various inference attacks because of its strict privacy constraints. In a federated learning model that satisfies differential privacy, a trusted central server aggregates the original local updates from users and performs privacy perturbations before publishing the global model. In a more private federated learning model that satisfies local differential privacy, users can perturb and upload their updates to an untrusted central server synchronously or asynchronously, avoiding privacy inference attacks. While reducing privacy threats in federated learning, differential privacy brings new problems in terms of data availability.

[0006] On the one hand, perturbation-based differential privacy reduces data availability in federated learning training. Because differential privacy regulates the amount of perturbation by introducing a privacy budget, there is no conflict between data privacy and availability. The increase in local update dimensions leads to a decrease in the budget allocated to each dimension, an increase in perturbations, and thus a decrease in data availability. At the same time, local differential privacy introduces more perturbations than differential privacy. On the other hand, a large amount of perturbations brings availability issues to federated learning data evaluation, because the errors introduced by perturbations bring challenges to the quality evaluation of user updates and the incentive mechanism of federated learning. In federated learning, user devices generate a lot of overhead when performing local model training, perturbations, and uploads. Therefore, a reasonable update evaluation and incentive mechanism is necessary to increase the enthusiasm of users to participate in updates. However, a large amount of perturbations makes it difficult for the server to evaluate the user's updates and reasonably distribute the user's rewards. Worse, the privacy and credibility of the central server used for update aggregation and evaluation may be destroyed by the service provider, attackers, or malicious servers.

[0007] In order to solve the privacy and data evaluation availability issues brought by the server, blockchain-based federated learning has been widely studied because the blockchain network structure can eliminate the single point of failure and trust issues brought by the central server. At the same time, the blockchain consensus mechanism based on the quality of user updates coincides with the incentive mechanism of federated learning, which makes the update of the federated learning model of the blockchain network efficiently deployed. However, the consensus on the quality of each update in the blockchain-based federated learning model requires a high computational overhead. In addition, the disturbance introduced by local differential privacy makes it impossible for blockchain-based federated learning to accurately evaluate user contributions to achieve reasonable reward distribution. Therefore, blockchain-based federated learning does not solve the problem of reduced data availability caused by local differential privacy, which even weakens the consensus mechanism of blockchain-based federated learning. Summary of the invention

[0008] In order to solve the challenges of data privacy and availability, the present invention proposes a federated learning hybrid aggregation mechanism based on subsampling, shuffling and Shapley value under the blockchain architecture, which enhances user privacy while ensuring data availability. The local servers in the network are used as management nodes and miner nodes, and the network is divided into multiple local domains consisting of local servers and connected terminal devices, and an inter-domain blockchain network consisting of multiple local servers. In order to improve the data availability in training while ensuring privacy, the gradient is perturbed to meet the local differential privacy at the user end to protect the gradient privacy, and the gradient is subsampled and selected to amplify the protection of the gradient index privacy; the shuffle server is used to perform secondary subsampling and shuffling on the gradient updates uploaded by the user to amplify the protection of user identity privacy, thereby reducing the amount of perturbation and improving the data availability in training. In order to improve the data availability in evaluation while ensuring privacy, the updates in the local domain are privacy aggregated at the local server, and then the Shapley value is used between local servers to effectively evaluate the aggregated updates and blockchain consensus, realizing a hybrid aggregation mechanism based on privacy aggregation within the local domain and efficient evaluation between local domains, while ensuring privacy, giving an average reward to the update uploader of each local domain.

[0009] In order to achieve the above object, the technical solution adopted by the present invention is:

[0010] A hybrid aggregation mechanism for federated learning under a blockchain architecture, specifically a privacy protection and amplification mechanism based on local differential privacy and gradient selection, a local differential privacy amplification mechanism based on subsampling and shuffling, and an inter-domain update evaluation aggregation mechanism based on hybrid aggregation and Shapley value. First, define the variables shown in Table 1:

[0011] Table 1 Commonly used variables and descriptions

[0012]

[0013]

[0014] The specific steps are as follows:

[0015] (1) First, after the user collects local training data, local training is performed based on the federated learning task model published in the network to obtain the model update gradient. In order to improve data privacy and availability, gradient selection is performed based on an encoding algorithm that satisfies local differential privacy, and gradient perturbation is performed using a perturbation algorithm that satisfies local differential privacy. The gradient is then encrypted using the public key of the local server and sent to the shuffle server. The selected gradient defends against gradient exponential attacks, the perturbed gradient defends against gradient speculation attacks, and the encrypted gradient defends against content attacks from outside the local server.

[0016] The construction of privacy protection and amplification mechanism based on local differential privacy and gradient selection is as follows:

[0017] (1.1) First, the user locally trains the gradient update to be uploaded. The local user obtains the federated learning task model and global parameters stored in the local server, trains the federated learning task model using the locally stored data as training data, and extracts the trained gradient;

[0018] (1.2) Select the uploaded gradient. Get the privacy budget ε of the gradient selection k , based on the gradient selection coding algorithm that satisfies local differential privacy, the gradient to be uploaded is selected to defend against privacy attacks on the gradient index. First, the gradient is divided into k gradients with high update rates and dk gradients with low update rates according to the absolute value. To defend against privacy attacks on the gradient index, the privacy budget ε for gradient selection is k , the probability difference of the attacker judging that a gradient in two adjacent data sets belongs to a high update rate gradient should be less than

[0019]

[0020] Among them, x and x′ represent the adjacent local user gradient updates with a gradient difference; represents the gradient selection encoding algorithm that satisfies local differential privacy; topk represents the set of high update rate gradients; and

[0021] They represent the probability that the gradient belongs to the high update gradient after gradient selection for x and x′ respectively.

[0022] Therefore, based on the privacy budget ε k Construct the probability formulas for selecting high update rate gradients and low update rate gradients respectively, so that the gradient selection satisfies ε k Local Differential Privacy:

[0023]

[0024] in, represents the i-th gradient x in x i The probability of being selected by the stochastic gradient selection algorithm, ε k The privacy budget chosen for the gradient;

[0025] (1.3) Perform privacy perturbation on the selected gradient. Obtain the privacy budget ε of the gradient perturbation l , and ε l Evenly distributed to the gradient perturbation algorithm, the perturbation budget for each gradient is Where k is the number of selected gradients, and the selected gradients are perturbed to satisfy localized differential privacy to defend against privacy attacks on gradient content;

[0026] (1.4) Calculate and compare the privacy amplification constraints. For the local differential privacy constraint (ε l ,δ l ), ε l After being evenly distributed to each gradient perturbation, Based on the combination principle of differential privacy and the amplification principle of subsampling, the privacy amplification constraint (ε dk ,δ dk )for:

[0027]

[0028]

[0029] in, privacy composition constraints that represent local perturbations;

[0030] represents the sampling parameter of gradient subsampling; ε k Represents the privacy budget of the gradient selection mechanism.

[0031] (2) In order to further improve data privacy and availability and defend against privacy attacks on user identities, privacy is further amplified at the shuffle server based on update subsampling and shuffle mechanisms, and the association between update records and users is disconnected, thereby achieving the purpose of identity privacy protection and improved availability.

[0032] The specific process of privacy amplification at the shuffle server is as follows:

[0033] (2.1) First, the shuffle server collects enough user updates. The shuffle server sets t as the threshold for starting shuffle and subsampling, so that users achieve the most basic t-anonymity in the domain;

[0034] (2.2) After the shuffle server collects the threshold t user updates, it collects the user IDs, constructs a random shuffle sequence of length t to shuffle the collected t user updates, and calculates and compares the privacy amplification effect of the shuffle. For the gradient selection, the amplification is (ε dk ,δ dk ) privacy constraint, and the privacy amplification constraint (ε kt ,δ kt )for:

[0035]

[0036] δ kt =δdk (t+1)

[0037] (2.3) Subsample the t user updates after shuffling. Subsample s user updates from the t shuffled user updates with uniform probability, and calculate and compare the privacy amplification effect of subsampling. For the privacy amplification after shuffling (ε kt ,δ kt ), based on the subsampling amplification principle, the privacy amplification constraint (ε, δ) of subsampling s user updates from t user updates is:

[0038]

[0039] δ=η2δ kt =η2δ dk (t+1)=η2η1δ l (t+1)

[0040] in, represents the privacy constraint after shuffling; ε dk =log(1+η1(e ε '-1)) represents the privacy constraint after gradient selection subsampling; privacy composition constraints that represent local perturbations; represents the sampling parameters of sub-sampling after shuffling; represents the sampling parameters of gradient subsampling;

[0041] (3) The local server obtains the t user IDs collected by the shuffler and the s user updates after shuffling and subsampling, decrypts and aggregates the intra-domain updates, and performs hybrid aggregation of the global federated learning task model and inter-domain update evaluation based on Shapley value with other local servers;

[0042] The specific process of mixed aggregation and update evaluation is as follows:

[0043] (3.1) The local server decrypts and aggregates user updates one by one, broadcasts the aggregated updates, and receives aggregated updates from other local servers;

[0044] (3.2) The local server updates the aggregation within a time interval, taking the test accuracy of the federated learning task model as the utility function, calculates the Shapley value of each node and broadcasts it;

[0045] (3.3) The node with the highest Shapley value distributes rewards and generates blocks. Rewards are distributed to each node according to the Shapley value. The global federated learning task model is obtained based on the learning rate of federated learning and the parameters of the current federated learning task model. A new block node is generated and added to the blockchain.

[0046] (3.4) To ensure user privacy and reward distribution, rewards are evenly distributed to users who participate in update aggregation within the same local server domain.

[0047] Beneficial effects of the present invention: Federated learning is widely used to provide personalized services to users, but the complex network environment makes it difficult to protect user privacy. For the blockchain-based federated learning mechanism, the present invention ensures the gradient privacy, gradient index privacy and user identity privacy of user data in training while ensuring the availability of data, that is, the accuracy of model prediction, through local privacy perturbation, based on privacy amplification mechanisms such as gradient selection, subsampling, and shuffling; inter-domain evaluation based on hybrid aggregation and Shapley value ensures the privacy of user data in aggregation while ensuring the availability of data, that is, reasonable distribution of rewards. The present invention solves the problem of available balance of user data privacy to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is an organizational chart of the federated learning hybrid aggregation mechanism under the blockchain architecture described in the present invention.

[0049] Figure 2 This is a flowchart of privacy disturbance and amplification for the user side of the present invention.

[0050] Figure 3 This is a privacy amplification flow chart of the shuffling server described in the present invention.

[0051] Figure 4 This is a flow chart of the hybrid aggregation and update evaluation according to the present invention. DETAILED DESCRIPTION

[0052] In order to express the purpose, technical solutions and advantages of the present invention more clearly, the present invention will be further described in detail through examples and drawings. A hybrid aggregation mechanism for federated learning under a blockchain architecture, the method includes user-side privacy perturbation and amplification based on local differential privacy and subsampling, shuffle server-side privacy amplification based on shuffle and subsampling, and global update aggregation and data evaluation based on blockchain and Shapley value.

[0053] Reference Figure 2 ,The specific operation process of privacy disturbance and privacy amplification on the user side is as follows:

[0054] Step 1. Collect local training data.

[0055] Step 2: Receive the federated learning task in the network from the local server and train it based on the local training data to obtain the gradient x = {g1, g2, …, g d}.

[0056] Step 3. Set the gradient x = {g1, g2, …, g d}Arrange from largest to smallest according to absolute value.

[0057] Step 4. Encode the first k bits of the sorted gradient as 1 and the last dk bits as 0.

[0058] Step 5. Privacy budget ε selected based on gradient k And the encoding algorithm that satisfies local differential privacy, calculate the gradient selection probability, and the calculation formula is:

[0059]

[0060] Among them, g i represents the i-th gradient g in step 4 i The encoded gradient value, is a coding algorithm that satisfies local differential privacy. Refers to the probability of a gradient being selected after gradient selection using an encoding algorithm that satisfies local differential privacy.

[0061] Step 6. Probably select the gradients according to the selection probability. If the output of the selection algorithm is not 1, continue to step 6. If the output is 1, proceed to the next step.

[0062] Step 7. Add the gradient whose output is 1 using the encoding algorithm that satisfies local differential privacy to the selected gradient set.

[0063] Step 8. Compare the number of selected gradients with k. If the number of selected gradients is less than k, proceed to step 5. If the number of selected gradients is greater than or equal to k, proceed to the next step.

[0064] Step 9. Clip the gradients. That is, arrange the selected gradients from large to small in absolute value, and cancel the selection of gradients exceeding k digits.

[0065] Step 10. Privacy budget ε based on local perturbation l , use the Laplace perturbation mechanism to perturb the selected gradient, encrypt the local update with the local server's public key and upload it to the shuffler. The gradient perturbation budget is ε l / k, the perturbation formula is:

[0066] y i =g i +Lap(ε l / k)

[0067] Where Lap represents the Laplace distribution function; y i represents the gradient g after perturbation i .

[0068] Step 11. Calculate the privacy amplification constraint. For the privacy constraint (ε l ,δ l )’s local differential privacy perturbation algorithm, the privacy amplification constraint (ε dk ,δ dk )for:

[0069]

[0070]

[0071] in, privacy composition constraints that represent local perturbations;

[0072] represents the sampling parameter of gradient subsampling; ε k Represents the privacy budget of the gradient selection mechanism.

[0073] Reference Figure 3 ,The specific process of privacy amplification of the shuffle server is as follows:

[0074] Step 12. The shuffle server collects the encrypted domain user updates c i .

[0075] Step 13. Update the number of users in the domain stored in the shuffle server n(c i ) is compared with the threshold value t. If the threshold value t is not reached, proceed to step 11. If the threshold value t is reached, proceed to the next step.

[0076] Step 14. Collect and record the IDs of the t domain users who participate in the update.

[0077] Step 15. Generate a shuffle sequence using a random number mechanism

[0078] Step 16. Use the shuffle sequence to shuffle the subsampled user updates and obtain the shuffled user update p(c s ).

[0079] Step 17. Calculate the privacy amplification constraint after shuffling. For the privacy constraint (ε dk ,δ dk ) gradient selection and perturbation, and the privacy amplification constraint (ε kt ,δ kt )for:

[0080]

[0081] δ kt =δ dk (t+1)

[0082] Step 18. Sample user updates with equal probability without replacement, i.e., when there are t updates, each update is sampled with probability 1 / t, when there are t-1 updates, each update is sampled with probability 1 / (t-1), and so on.

[0083] Step 19. Compare the sampled update number with s. If the sampled update number is less than s, continue to step 14. If the sampled update number is equal to s, continue to the next step.

[0084] Step 20. Calculate the privacy amplification constraint after subsampling. For the privacy constraint (ε kt ,δ kt ) Privacy amplification mechanism, the privacy amplification constraint (ε, δ) after s subsampling is:

[0085]

[0086] δ=η2δ kt =η2δ dk (t+1)=η2η1δ l (t+1)

[0087] in, represents the privacy constraint after shuffling; ε dk =log(1+η1(e ε '-1)) represents the privacy constraint after gradient selection subsampling; privacy composition constraints that represent local perturbations; represents the sampling parameters of sub-sampling after shuffling; represents the sampling parameters of gradient subsampling;

[0088] Step 21. Upload the collected t user IDs and the sub-sampled and shuffled s user updates to the local server.

[0089] Reference Figure 4 ,The specific operation process of mixed aggregation and data evaluation is as follows:

[0090] Step 22. The local server collects the user ID and user update from the shuffler in the domain, decrypts the user update using the private key and aggregates the update values ​​to obtain the aggregated update Z in the local domain. s ;

[0091] Step 23. The local server broadcasts the aggregated domain updates to other connected local servers, and collects broadcast updates from other local servers;

[0092] Step 24. If the time for collecting updates does not reach the time of an update interval, continue to step 23. If it reaches the time of an update interval, proceed to the next step;

[0093] Step 25. The local server calculates the Shapley value of each local server based on the collected updates. The calculation formula is as follows:

[0094]

[0095] Where N represents the number of local servers; I = {1, ..., N} represents the set of all local servers; i represents the i-th local server; S represents the combination of multiple local servers; u represents the utility function, that is, the test accuracy of the global model after using the current gradient update;

[0096] Step 26. Receive update content broadcasted by other nodes;

[0097] Step 27. Each node determines and elects the node with the highest Shapley value as the model update and block generation node. If it is the node with the highest Shapley value, it proceeds to the next step. If it is not the node with the highest Shapley value, it directly proceeds to step 31.

[0098] Step 28. The node with the maximum Shapley value is used as the global model update node. The aggregated updates are globally aggregated based on the learning rate. For the federated learning model with a learning rate of r, the global aggregation formula is:

[0099]

[0100] Among them, Z si represents the i-th gradient of the aggregate update; W Gi represents the i-th parameter of the global model;

[0101] Step 29. The node with the largest Shapley value is used as the blockchain generation and update node. It uses the aggregated global update to generate a new block and connects the hash address of the previous block to join the blockchain.

[0102] Step 30. The node with the largest Shapley value is used as the reward distribution and issuance node. It obtains the reward for model update and distributes it to each node according to the Shapley value. The reward of each node is issued to each local server in the form of blockchain transactions. The formula for reward distribution according to the Shapley value is:

[0103]

[0104] Among them, reward represents the reward for updating aggregation in this round of federated learning; reward i represents the domain reward obtained by the i-th local server;

[0105] Step 31. After receiving the reward, the local server evenly distributes the reward to each user node participating in the update. For the i-th local server, the reward distribution formula for users in its domain is:

[0106]

[0107] Among them, reward useri represents the user reward in the i-th local server domain; t represents the number of users participating in this round of updates in the i-th local server domain.

Claims

1. A hybrid aggregation method for federated learning under a blockchain architecture, characterized in that: Here are the steps: (1) First, after the user collects local training data, local training is performed based on the federated learning task model published in the network to obtain the model update gradient; in order to improve data privacy and availability, gradient selection is performed based on the gradient selection encoding algorithm that satisfies local differential privacy, and gradient perturbation is performed using the perturbation algorithm that satisfies local differential privacy. The gradient is then encrypted using the public key of the local server and sent to the shuffling server; the selected gradient is protected against gradient exponential attacks, the perturbed gradient is protected against gradient speculation attacks, and the encrypted gradient is protected against content attacks from outside the local server; The construction of privacy protection and amplification mechanism based on local differential privacy and gradient selection is as follows: (1.1) First, the user locally trains the gradient update to be uploaded; the local user obtains the federated learning task model and global parameters stored in the local server, trains the federated learning task model using the locally stored data as training data, and extracts the trained gradient; (1.2) Select the uploaded gradient; obtain the privacy budget ε for gradient selection k , based on the gradient selection coding algorithm that satisfies local differential privacy, the gradient to be uploaded is selected to defend against privacy attacks on the gradient index; first, the gradient is divided into k gradients with high update rates and dk gradients with low update rates according to the absolute value, where d is the number of locally updated gradients; to defend against privacy attacks on the gradient index, the privacy budget ε for gradient selection is k , the probability difference of the attacker judging that a gradient in two adjacent data sets belongs to a high update rate gradient should be less than Among them, x and x′ represent the adjacent local user gradient updates with a gradient difference; represents the gradient selection encoding algorithm that satisfies local differential privacy; topk represents the set of high update rate gradients; and They represent the probability that the gradient belongs to the high update gradient after gradient selection for x and x′ respectively; Therefore, based on the privacy budget ε k Construct the selection probability formulas of high update rate gradient and low update rate gradient respectively, so that the gradient selection satisfies ε k Local Differential Privacy: in, represents the i-th gradient x in x i The probability of being selected by the stochastic gradient selection algorithm, ε k The privacy budget chosen for the gradient; (1.3) Perform privacy perturbation on the selected gradient; obtain the privacy budget ε of the gradient perturbation l , and ε l Evenly distributed to the perturbation algorithm that satisfies local differential privacy, the perturbation budget for each gradient is Where k is the number of selected gradients, and the selected gradients are perturbed to satisfy localized differential privacy to defend against privacy attacks on gradient content; (1.4) Calculate and compare privacy amplification constraints; for the local differential privacy constraint (ε l ,δ l ), ε l After being evenly distributed to each gradient perturbation, Based on the combination principle of differential privacy and the amplification principle of subsampling, the privacy amplification constraint (ε dk ,δ dk )for: in, privacy composition constraints that represent local perturbations; represents the sampling parameter of gradient subsampling; ε k represents the privacy budget of the gradient selection mechanism; (2) In order to further improve data privacy and availability and defend against privacy attacks on user identities, privacy is further amplified at the shuffle server based on update subsampling and shuffle mechanisms, and the association between update records and users is disconnected, thereby achieving the purpose of identity privacy protection and improved availability. The specific process of privacy amplification at the shuffle server is as follows: (2.1) First, the shuffle server collects user updates; the shuffle server sets t as the threshold for starting shuffle and subsampling, so that users achieve the most basic t-anonymity in the domain; (2.2) After the shuffle server collects the threshold t user updates, it collects the user IDs, constructs a random shuffle sequence of length t to shuffle the collected t user updates, and calculates and compares the privacy amplification effect of the shuffle; for the gradient selection, the amplification is (ε dk ,δ dk ) privacy constraint, and the privacy amplification constraint (ε kt ,δ kt )for: d kt =d dk (t+1) (2.3) Subsample the t user updates after shuffling; subsample s user updates from the t shuffled user updates with uniform probability, calculate and compare the privacy amplification effect of subsampling; for the privacy amplification after shuffling (ε kt ,δ kt ), based on the subsampling amplification principle, the privacy amplification constraint (ε, δ) of subsampling s user updates from t user updates is: δ=η2δ kt =η2δ dk (t+1)=η2η1δ l (t+1) in, represents the privacy constraint after shuffling; ε dk =log(1+η1(e ε' -1)) represents the privacy constraint after gradient selection subsampling; privacy composition constraints that represent local perturbations; represents the sampling parameters of sub-sampling after shuffling; represents the sampling parameters of gradient subsampling; (3) The local server obtains the t user IDs collected by the shuffler and the s user updates after shuffling and subsampling, decrypts and aggregates the user updates, and performs hybrid aggregation of the global federated learning task model and inter-domain update evaluation based on Shapley value with other local servers; The specific process of mixed aggregation and update evaluation is as follows: (3.1) The local server decrypts and aggregates user updates one by one, broadcasts the aggregated updates, and receives aggregated updates from other local servers; (3.2) The local server updates the aggregation within a time interval, taking the test accuracy of the federated learning task model as the utility function, calculates the Shapley value of each node and broadcasts it; (3.3) The node with the highest Shapley value distributes rewards and generates blocks; rewards are distributed to each node according to the Shapley value, and the global federated learning task model is obtained based on the learning rate of federated learning and the parameters of the current federated learning task model, and a new block node is generated and added to the blockchain; (3.4) To ensure user privacy and reward distribution, rewards are evenly distributed to users who participate in update aggregation within the same local server domain.

Citation Information

Patent Citations

  • Model stealing defense method under vertical federation based on block chain technology

    CN112738035A

  • Federal learning method and device based on splitting shuffling, equipment and medium

    CN115442036A