A privacy protection method based on alliance chain federated learning

Abstract

The present application relates to a kind of privacy protection method based on alliance chain federal learning, belong to federal learning privacy protection technical field.The present application adds adaptive laplace noise satisfying differential privacy according to the correlation of data characteristics and model by adaptive differential privacy technology, under the condition of protecting data privacy, try to reduce model accuracy reduction.Furthermore, the distributed federal learning privacy protection scheme is realized to solve single point failure problem by alliance chain technology.The method has good privacy protection and the performance of training model accuracy.The method uses adaptive differential privacy technology, adds adaptive noise satisfying differential privacy in the process of local model training, guarantees the accuracy of model training in the case of guaranteeing client data privacy, solves the centralization problem faced by traditional federal learning privacy protection scheme, enhances the privacy protection performance of system.

Description

TECHNICAL FIELD

[0001] The present application relates to a privacy protection method based on alliance chain federated learning, belonging to the technical field of federated learning privacy protection. BACKGROUND

[0002] With the increasingly rapid digitalization of human society and the rapid development of the Internet, a large amount of information data has been generated. Through machine learning technology, the treasures hidden in the data can be automatically mined. Machine learning models trained by a large amount of data have been applied in various practical scenarios and are changing our world, such as precision medicine, clinical auxiliary diagnosis, new drug research and development, portrait recognition, voiceprint recognition, picture voice, natural language and multi-modal learning. In application, the precision and generalization ability of the model are crucial, which relies on the learning of a large amount of data by the machine. Especially in the field of information privacy protection, due to the constraints of legal regulations, policy supervision, business secrets, personal privacy and other data privacy security, multiple data source parties cannot directly exchange data, forming a "data island" phenomenon, which restricts the further improvement of the ability of artificial intelligence models. The birth of federated learning is to solve this problem.

[0003] Federated learning is a distributed machine learning technology, whose core idea is to conduct distributed model training among multiple data sources with local data, to build a global model based on virtual fused data by exchanging model parameters or intermediate results without exchanging local individual or sample data, so as to achieve the balance between data privacy protection and data sharing calculation.

[0004] Traditional federated learning carries out efficient machine learning among multiple participants or nodes under the premise of guaranteeing the information security of big data exchange, protecting user data and personal data privacy, and ensuring legal compliance. However, the current federated learning privacy protection scheme supporting privacy calculation still has the problem of model parameter leakage leading to privacy leakage.

[0005] At present, the traditional federated learning privacy protection scheme mainly includes anonymization technology, cryptography technology such as secure multi-party computation and differential privacy technology. The anonymization technology mainly acts on the original data set before training, and has certain limitations in federated learning; although the cryptography technology has high security, due to the huge calculation overhead and complexity, it still cannot meet the needs of practical deployment; compared with the anonymization technology and the cryptography technology, the differential privacy technology has better applicability in actual deployment. However, there are still two main key problems in the federated learning privacy protection technology based on differential privacy. First, in the local model training process, the strong opposition problem between data privacy and model accuracy caused by not considering data correlation needs to be solved; second, in the traditional federated learning, once the center server is attacked, the single point failure problem will be caused, thereby destroying the whole federated learning system.

[0006] In the data information privacy protection, in order to solve the single point failure problem caused by the center server and the strong opposition problem between data privacy leakage and model accuracy in the existing federated learning privacy protection scheme, it is necessary to find a secure and high-accuracy federated learning data privacy protection scheme supporting privacy calculation. SUMMARY

[0007] The purpose of the present application is to solve the technical defects such as data privacy leakage and single point failure existing in the existing federated learning privacy protection scheme supporting privacy calculation, adopt adaptive differential privacy technology and alliance chain technology, and creatively propose a data information privacy protection scheme based on alliance chain federated learning supporting privacy calculation, to solve the problems of data privacy leakage and single point failure in the privacy calculation process.

[0008] The innovation point of the present application is that: through the adaptive differential privacy technology, the adaptive Laplace noise meeting the differential privacy is added according to the correlation between the data characteristics and the model, so as to reduce the model accuracy as much as possible under the condition of protecting the data privacy. Through the alliance chain technology, the distributed federated learning privacy protection scheme is further realized to solve the single point failure problem.

[0009] In order to achieve the above purpose, the present application adopts the following technical scheme.

[0010] Firstly, the related definitions are explained:

[0011] Definition 1: Alliance chain network refers to the infrastructure network used for the operation of the federated learning client, including the client node providing data, the intelligent contract and the incentive mechanism, and the distributed storage network system reached through the consensus algorithm.

[0012] Definition 2: Intelligent contract refers to the account node in the alliance chain which undertakes the deployment of the intelligent contract business and simultaneously performs the full synchronization of the global model of the federated learning.

[0013] Definition 3: Consensus node refers to a node responsible for collecting local client training model parameters, verifying global model parameters, and generating blocks for packaging.

[0014] Definition 4: Ledger node refers to a node responsible for verifying blocks generated by consensus nodes and storing blocks.

[0015] Definition 5: Client refers to a user who stores data and participates in federated learning model training.

[0016] Definition 6: Task publisher refers to the demander of global model training in federated learning.

[0017] Definition 7: Incentive mechanism refers to a reward and punishment mechanism provided by the alliance chain based on the model parameters trained by the client.

[0018] A privacy protection method based on federated learning of alliance chain, comprising the following steps:

[0019] Step 1: System initialization, including uploading of initial global model in federated learning, client registration.

[0020] Specifically, step 1 can adopt the following scheme:

[0021] Step 1.1: The task publisher publishes the initial global model in the alliance chain.

[0022] Step 1.2: The client with data registers the information in the alliance chain.

[0023] Step 1.3: Select a group of clients with strong computing and communication capabilities as candidate consensus nodes.

[0024] Step 2: Select the leader node in the alliance chain.

[0025] The leader node works as a representative of the consensus node, generating and verifying block data in DBFT (Delegated Byzantine Fault Tolerance, delegated Byzantine fault tolerance consensus algorithm). From the representative, a delegated consensus node (i.e. a representative) is randomly selected as the leader consensus node of the current consensus round, and the remaining representatives act as followers in the consensus process.

[0026] Step 3: Train the federated learning local model based on adaptive differential privacy.

[0027] Specifically, step 3 can adopt the following scheme:

[0028] Step 3.1: The local client downloads the initial global training model from the alliance chain.

[0029] Step 3.2: The local client calculates the relevance R of the data set to the global training model using the hierarchical relevance propagation algorithm.

[0030] Step 3.3: The local client uses the relevance R and the hierarchical differential privacy neural network to train the model and obtain the locally optimal training model parameter W.

[0031] Step 3.4: The local client uploads the locally optimal training model parameter W to the representative node of the alliance chain.

[0032] Step 4: Update the global model in the verification alliance chain.

[0033] Specifically, step 4 can use the following scheme:

[0034] Step 4.1: The representative node estimates the quality of the locally optimal training model W based on the test data set, thereby verifying the legitimacy of the local client.

[0035] If the local client is not legitimate, the locally optimal training model parameter of the local client is deleted. If the local client is legitimate, the locally optimal training model parameter of the local client is retained.

[0036] Step 4.2: After time T, the leader consensus node collects all locally optimal training models, aggregates the global training model, and generates a new block with the global training model and a digital signature.

[0037] Step 4.3: The leader consensus node broadcasts the new block to other consensus nodes as followers.

[0038] Step 5: The follower receives the new block with the digital signature and verifies the candidate block verification.

[0039] Specifically, step 5 contains the following steps:

[0040] Step 5.1: The follower consensus node generates a verification result for the block data.

[0041] Step 5.2: The follower consensus node sends the verification result and its digital signature to other follower consensus nodes.

[0042] Step 5.3: Each follower compares its own verification result with the results received from other followers, and sends a confirmation to the leader consensus node to indicate its verification result.

[0043] Step 5.4: The leader consensus node receives the verification results from different follower consensus nodes, the leader consensus node verifies the information, and checks whether more than 2 / 3 of the followers have the same conclusion on the block data. If more than 2 / 3 of the followers have the same conclusion, the block data will be recorded into the consortium blockchain; otherwise, the block data is discarded.

[0044] Step 6: Global model training of federated learning.

[0045] Specifically, step 6 includes the following steps:

[0046] Step 6.1: In federated learning, all client participants download new block data from the consortium chain.

[0047] Step 6.2: The client node calculates the weighted average of all qualified local model updates as their new global model.

[0048] Step 6.3: The client participant uses the new global model as the initialization model to perform global model training for the next iteration.

[0049] Step 7: When the global model training is completed, record the client who continuously submits qualified local model updates and give a reward. If a malicious client is found, a penalty will be given.

[0050] After steps 1 to 7, the privacy protection of data information is realized.

[0051] Advantages

[0052] Compared with the prior art, the method of the present application has the following advantages:

[0053] 1. The method has good privacy protection and training model accuracy performance. The adaptive differential privacy technology is used in the method, and adaptive noise satisfying differential privacy is added in the process of local model training. The accuracy of model training is ensured while the privacy of client data is ensured.

[0054] 2. The present application proposes a federated learning aggregation method based on consortium chain, which solves the centralization problem faced by the traditional federated learning privacy protection scheme and enhances the privacy protection performance of the system. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a schematic diagram of the method of the present application. DETAILED DESCRIPTION

[0056] The technical solutions of the present application will be described clearly and completely in combination with the drawings and examples.

[0057] Examples

[0058] As Figure 1 shown, a privacy protection method based on a consortium chain federated learning supporting privacy calculation.

[0059] Step 1: system initialization, including federated learning initial global model uploading and client registration.

[0060] Specifically, step 1 includes the following steps:

[0061] Step 1.1: the task publisher publishes the initial global model in the consortium chain system.

[0062] Step 1.2: the client with data registers the information in the consortium chain system.

[0063] Step 1.3: select a group of clients with strong computing and communication capabilities as candidate consensus nodes.

[0064] Step 2: selection of leader nodes in the consortium chain. The leader node generates and verifies block data in DBFT as a representative of the consensus node. A delegated consensus node (i.e. a representative) is randomly selected from the representatives as the leader consensus node of the current consensus round, and the remaining representatives act as followers in the consensus process.

[0065] Step 3: federated learning local model training based on adaptive differential privacy.

[0066] Specifically, step 3 includes the following steps:

[0067] Step 3.1: the local client downloads the initial global training model from the consortium chain.

[0068] Step 3.2: the local client calculates and layer-related propagation algorithm according to the local data set to calculate the relevance R of the data set and model training.

[0069] Step 3.3: the local client uses the relevance R and hierarchical differential privacy neural network to train the model to obtain the local optimal training model parameter W.

[0070] Specifically, step 3.3 includes the following steps:

[0071] Step 3.3.1: assume that there are n clients participating in the iterative learning process of the tthround model training at present, 1≤n≤N', N' represents the total number of clients participating in federated learning;

[0072] Where, the model parameter obtained by the client i after local training is w i (t), and the model training parameter obtained by the server is: α iLet αi represent the participation rate of client i in the t-th round of model training. For simplicity, let α1 = α2 = ... = αt. n =α.

[0073] Step 3.3.2: The affine transformation of a deep learning neuron h∈h0 is represented as: x i Let W represent the data element in dataset D, W represent the input parameters of neuron h, and b represent the basis vector.

[0074] Step 3.3.3: Based on data element x i The relevance of model training in affine transformation Adaptive Gaussian noise is added. In this method, the privacy budget ratio α of the j-th data feature is calculated. j and privacy budget ε j The relationship between them is expressed as: ε j =α j ×ε2, where d represents the total number of features in the dataset. εj represents the correlation between the j-th data feature and the global model, and ε2 represents the privacy budget for the added local differential privacy.

[0075] Step 3.3.4: Based on the privacy budget ratio α j and privacy budget ε j In the original data element x i By adding adaptive Gaussian noise to the basis vector b, we obtain an affine transformation that is privacy-safe. Indicates Gaussian noise. Indicates in data element x i Add noise that follows a Gaussian distribution.

[0076] in, δ²∈(0,1); N represents the Gaussian distribution function, σ² represents the standard deviation of the Gaussian distribution, c² represents the coefficient relating the standard deviation of the Gaussian distribution to the privacy budget, and ε... j Let ε2 represent the privacy budget for adaptive differential privacy, ε2 represent the privacy budget for added local differential privacy, δ2 represent the standard deviation of the Gaussian distribution, and σ2 represent the standard deviation of the Gaussian distribution. 2j This represents the standard deviation of Gaussian noise added when the j-th client performs federated learning.

[0077] Based on the feature values ​​x of all input data ij We set Δh to represent the sensitivity between adjacent datasets. This ultimately yields the privacy-preserving local model training parameters.

[0078] Step 3.4: The local client uploads the locally optimal training model parameters W to the representative node of the consortium blockchain.

[0079] Step 4: Verify the global model update in the consortium blockchain.

[0080] Specifically, step 4 includes the following steps:

[0081] Step 4.1: The representative node verifies the legitimacy of the local client by estimating the quality of the locally optimal training model W based on the test dataset. If the local client is illegitimate, its locally optimal training model parameters are removed. If the local client is legitimate, its locally optimal training model parameters are retained.

[0082] Step 4.2: After time T, the leader consensus node aggregates all the locally optimal training models collected to obtain the global training model and generates a new block with the global training model and digital signature.

[0083] Step 4.3: The leader consensus node broadcasts the new block to the other consensus nodes that are followers.

[0084] Step 5: Followers receive the new block with the digital signature and verify the candidate block verification.

[0085] Specifically, step 5 includes the following steps:

[0086] Step 5.1: Follower consensus nodes generate verification results for block data.

[0087] Step 5.2: The follower consensus node sends the verification result and its digital signature to other follower consensus nodes.

[0088] Step 5.3: Each follower compares its own verification result with the results received from other followers and sends an acknowledgment to the leader consensus node to indicate its verification result.

[0089] Step 5.4: The leader consensus node receives verification results from different follower consensus nodes. The leader consensus node verifies the information and checks whether more than 2 / 3 of the followers have reached the same conclusion regarding the block data. If more than 2 / 3 of the followers have reached the same conclusion, the block data will be recorded in the consortium blockchain; otherwise, the block data will be discarded.

[0090] Step 6: Global model training for federated learning.

[0091] Specifically, step 6 includes the following steps:

[0092] Step 6.1: In federated learning, all client participants download new block data from the consortium blockchain.

[0093] Step 6.2: The client nodes calculate the weighted average of all eligible local model updates as their new global model.

[0094] Step 6.3: Client participants use the new global model as the initialization model to train the global model for the next iteration.

[0095] Step 7: Once the global model training is complete, clients that continuously submit qualified local model updates will be recorded and rewarded. Clients found to be malicious will be penalized.

[0096] The described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

Claims

1. A privacy protection method based on consortium blockchain federated learning, characterized in that, Includes the following steps: Step 1: System initialization, including uploading the initial global model for federated learning and client registration; Step 2: Select the leader node in the consortium blockchain; In this process, the leader node acts as a representative of the consensus nodes, generating and verifying block data in DBFT; one delegated consensus node is randomly selected from the representatives to serve as the leader consensus node for the current consensus round, while the remaining representatives act as followers during the consensus process. Step 3: Train a local model based on adaptive differential privacy federated learning; Step 3 includes the following steps: Step 3.1: The local client downloads the initial global training model from the consortium blockchain; Step 3.2: The local client uses the hierarchical correlation propagation algorithm to calculate the correlation R between the dataset and the global training model; Step 3.3: The local client uses the correlation R and the hierarchical differential privacy neural network to train the model and obtain the locally optimal training model parameters W; Step 3.3 includes the following steps: Step 3.3.1: Suppose there are currently n clients participating in the iterative learning process of the t-th round of model training, 1≤n≤N', where N' represents the total number of clients participating in federated learning; where client i obtains model parameters w through local training. i (t), the model training parameters obtained by the server are: α i This represents the participation rate of client i in the t-th round of model training, simplified as α1=α2=...=α n =α; Step 3.3.2: The affine transformation of a deep learning neuron h∈h0 is represented as: x i Let W represent the data element in dataset D, W represent the input parameters of neuron h, and b represent the basis vector; Step 3.3.3: Based on data element x i The relevance of model training in affine transformation Add adaptive Gaussian noise; Calculate the privacy budget ratio α for the j-th data feature. j and privacy budget ε j The relationship between them is expressed as: ε j =α j ×ε2, where d represents the total number of features in the dataset. εj represents the correlation between the j-th data feature and the global model, and ε2 represents the privacy budget for the added local differential privacy. Step 3.3.4: Based on the privacy budget ratio α j and privacy budget ε j In the original data element x i By adding adaptive Gaussian noise to the basis vector b, we obtain an affine transformation that is privacy-safe. Indicates Gaussian noise. Indicates in data element x i Add noise that follows a Gaussian distribution; in, δ²∈(0,1); N represents the Gaussian distribution function, σ² represents the standard deviation of the Gaussian distribution, c² represents the coefficient relating the standard deviation of the Gaussian distribution to the privacy budget, and ε... j Let ε2 represent the privacy budget for adaptive differential privacy, ε2 represent the privacy budget for added local differential privacy, δ2 represent the standard deviation of the Gaussian distribution, and σ2 represent the standard deviation of the Gaussian distribution. 2j This represents the standard deviation of Gaussian noise added when the j-th client performs federated learning. Based on the feature values ​​x of all input data ij By setting Δh as the sensitivity between adjacent datasets, we can finally obtain the privacy-preserving local model training parameters. Step 3.4: The local client uploads the locally optimal trained model parameters W to the representative node of the consortium blockchain; Step 4: Update and verify the global model in the consortium blockchain; Step 5: Followers receive the new block with the digital signature and verify the candidate block verification; Step 6: Global model training for federated learning; Step 7: Once the global model training is complete, clients that continuously submit qualified local model updates will be recorded and rewarded. If any malicious clients are found, they will be punished.

2. The privacy protection method based on consortium blockchain federated learning as described in claim 1, characterized in that, Step 1 includes the following steps: Step 1.1: The task publisher publishes the initial global model to the consortium blockchain; Step 1.2: Clients possessing the data register the information with the consortium blockchain; Step 1.3: Select a group of clients with strong computing and communication capabilities as candidate consensus nodes.

3. The privacy protection method based on consortium blockchain federated learning as described in claim 1, characterized in that, Step 4 includes the following steps: Step 4.1: The representative node verifies the legitimacy of the local client by estimating the quality of the locally optimal training model W based on the test dataset; If the local client is invalid, its locally optimal training model parameters are deleted; if the local client is valid, its locally optimal training model parameters are retained. Step 4.2: After time T, the leader consensus node collects all the locally optimal training models, aggregates them to obtain the global training model, and generates a new block with the global training model and digital signature; Step 4.3: The leader consensus node broadcasts the new block to the other consensus nodes that are followers.

4. The privacy protection method based on consortium blockchain federated learning as described in claim 1, characterized in that, Step 5 includes the following steps: Step 5.1: Follower consensus nodes generate verification results for block data; Step 5.2: The follower consensus node sends the verification result and its digital signature to other follower consensus nodes; Step 5.3: Each follower compares its own verification result with the results received from other followers and sends an acknowledgment to the leader consensus node to indicate its verification result; Step 5.4: The leader consensus node receives the verification results from different follower consensus nodes. The leader consensus node verifies the information and checks whether more than 2 / 3 of the followers have reached the same conclusion on the block data. If more than 2 / 3 of the followers have reached the same conclusion, the block data will be recorded in the consortium blockchain; otherwise, the block data will be discarded.

5. The privacy protection method based on consortium blockchain federated learning as described in claim 1, characterized in that, Step 6 includes the following steps: Step 6.1: In federated learning, all client participants download new block data from the consortium blockchain; Step 6.2: Client nodes calculate the weighted average of all eligible local model updates as their new global model; Step 6.3: Client participants use the new global model as the initialization model to train the global model for the next iteration.

Images