A distributed federated learning method for data privacy protection and high model quality

Abstract

The application discloses a kind of data privacy protection and high model quality's distributed federal learning method, this method includes: system setting, equipment initialization, task release, training and aggregation.The method proposed in the application aims at the problem of low data privacy and model quality in federal learning.The method mainly uses multi-key homomorphic encryption, trusted execution environment and blockchain technology to realize.Using distributed idea, clients can directly exchange encrypted local models without the server having to bear the aggregation task, fundamentally solving the single point failure problem.The method can also effectively detect whether the updates received by the client are low-quality model updates.In addition, the multi-key homomorphic encryption scheme is used to establish a secure channel between clients to enhance data privacy.The method also supports client joining and leaving, and effectively avoids low-quality model updates, preventing malicious clients from tampering with and uploading false models.The application meets the characteristics of strong security, high model accuracy, etc.

Description

Technical Field

[0001] This invention relates to the fields of information security and federated learning technology, specifically addressing the problem of distributed federated learning with data privacy protection and model reliability. It involves homomorphic encryption, federated learning, trusted execution environments, and blockchain, particularly multi-key homomorphic encryption, trusted execution environment data protection, blockchain communication, and a distributed federated learning method for data privacy protection and high model quality. Background Technology

[0002] In recent years, neural network-based deep learning has made significant progress in various tasks such as speech recognition, image recognition, and autonomous driving. To achieve higher accuracy requirements, it is necessary to provide deep learning models with a large amount of training data. However, with increasing awareness of data privacy, it has become difficult to centralize the training data of all participants. To address this issue, many privacy-preserving deep learning frameworks have emerged. Among these frameworks, federated learning is a widely used system. It allows users to participate in collaborative training without uploading their local data. The traditional federated learning architecture is a centralized structure, consisting of a central server and multiple participants. Participants train their models using local data and upload the resulting model updates to the central server. After receiving updates from all participants, the central server aggregates them and returns an aggregated global model. Participants then adjust their local models based on the global model returned by the central server. This process iterates over multiple rounds until the global model converges.

[0003] While federated learning enables participants to collaboratively train without disclosing local data, it still faces several challenges. A single point of failure and malicious behavior by the central server can severely damage the system. The central server is responsible for model aggregation. If it disconnects or becomes corrupted, the entire federated learning process will halt. Existing research in training tasks largely assumes the central server is honest or semi-honest. However, some studies do not protect the global model, and its security remains to be verified. Once the central server obtains an unprotected global model, attackers can exploit it for profit. More importantly, once the central server is compromised, attackers can arbitrarily disrupt the training task. Even if the central server performs its tasks normally, it can infer a large amount of sensitive information from the gradients of model updates through privacy inference attacks. Furthermore, in some scenarios (e.g., when the client is an edge mobile device), sending data to the central server each time incurs significant costs. Therefore, distributed federated learning is an effective solution to address these issues. This approach fundamentally solves the problems of single point of failure and malicious behavior by the server. However, this framework lacks effective privacy protection methods and model quality detection techniques. Therefore, more effective methods are needed to protect the data privacy and model reliability of distributed federated learning frameworks. Summary of the Invention

[0004] This invention primarily discusses distributed federated learning. In this setup, clients can directly exchange models without a central server, avoiding single points of failure and malicious server behavior, and allowing clients to join or leave during the learning process.

[0005] The purpose of this invention is to address the issues of data privacy and model reliability in distributed federated learning by proposing an effective distributed federated learning scheme that improves data privacy and ensures model reliability.

[0006] The specific solution to achieve the objective of this invention is as follows:

[0007] A distributed federated learning method that protects data privacy and provides high model quality includes the following entities: a client, a trusted authority (CA), and a task publisher; the client is responsible for training the local model and undertaking the aggregation task; the trusted authority (CA) is responsible for generating parameters and certificates for the system; the task publisher is responsible for selecting task regions and publishing tasks; the distributed federated learning method includes the following steps:

[0008] Step 1: System Settings

[0009] CA selects a security parameter λ and executes the multi-key homomorphic encryption parameter generation algorithm MKHE.Setup(1 λ Generate system parameters pp;

[0010] Step 2: Device Initialization

[0011] Client P i Generate your own public / private key pair (pk i ,sk i The client has a local dataset D. i Trusted Execution Environment (TEE) i The CA generates a Reputation Certificate (RC) for the client. i Client-side verification of the local trusted execution environment (TEE) i Identify and establish a secure channel; clients jointly maintain a blockchain for synchronizing information;

[0012] Step 3: Task Issuance

[0013] The task publisher generates a test dataset according to the task requirements; the task publisher initializes the global model and selects the task region; the task publisher divides the task region into sub-regions, and clients in the same sub-region are grouped together; the task publisher remotely verifies the client's Trusted Execution Environment (TEE) and establishes a secure channel, and then uploads the test dataset to the client's TEE.

[0014] Step 4: Training and Aggregation Phase

[0015] In step 3, the client trains a local model using the local dataset and then uploads the local model and its public key to the local TEE via a secure channel. The TEE encrypts and signs the model and uploads the results to the blockchain. The client downloads the encrypted model information from other clients in the same group from the blockchain and aggregates it to generate a ciphertext sub-global model. Clients in the same group collaboratively decrypt the ciphertext sub-global model and upload it to the blockchain. Finally, they download the plaintext sub-global models corresponding to other groups and calculate the global model. After obtaining the global model, the client first uses the TEE to evaluate the quality of the local model, updates the client's reputation value based on the evaluation results, and finally updates the local model using the global model. After T rounds, training ends.

[0016] Step 2 specifically includes:

[0017] Client P i The key generation algorithm MKHE.KeyGen(pp) is executed to generate a public-private key pair (pk). i ,sk i Client P i Initialize local dataset D i and Local Trusted Execution Environment (TEE) i The CA generates a reputation certificate for the client. Where id i It is the client's identity, rvi It's the client's reputation score, au i It contains all the auxiliary information for the client. Is CA in (id) i ,pk i ,rv i ,au i The signature on the TEE; client verification of the local TEE i The identity information is collected and a secure channel is established;

[0018] Step 3 specifically includes:

[0019] The task publisher selects a training task I, initializes the global model ω0, and selects a task region, dividing this task region into K sub-regions; the task publisher then sends the task information (location, tt, σ) to the task region. T This is broadcast to clients in the region, where `location` is the task region and information about how the region is divided into K sub-regions, `tt` represents other task information, including `ω0`, `T`, and `F(·)`, where `ω0` is the initial global model, `T` is the maximum number of training epochs, `F(·)` is the loss function, and `σ` is the target value. T It is the signature of the task publisher on the task information location,tt; any client interested in task I responds to the message broadcast by the task publisher by sending a response message; if client P i It is interested in the task and sends a response message to the task publisher. Among them RC i It is P i Reputation certificate, location i It is P i Location, It is P i In (RC) i ,location i The signature on the RM1, ..., RM1; if the response information received by the task publisher is RM1, ..., RM2 N The task publisher selects clients based on the device's location and reputation value. The task publisher sends specific tasks to the selected clients, remotely verifies the client's TEE and establishes a secure channel. Then, the task publisher pre-stores the test dataset into the TEE of all selected clients through the secure channel.

[0020] Step 4 specifically includes:

[0021] In step 3, the client trains a local model using the local dataset, and then uploads the local model and its public key to the local TEE via a secure channel. The TEE encrypts the model using the MKHE.Enc encryption algorithm, generates an encrypted model, and signs the encrypted result using its own internal private key. Each client downloads the packaged encrypted model and signature information from the local TEE and uploads it to the blockchain. The specific steps are as follows:

[0022] There are t rounds of training. When t=1, the client updates the local model using the local dataset and its size. The specific steps are as follows:

[0023] 1) Update the local model using the local dataset. It is the local model generated by client P in round t, η

[0024] It's the learning rate. This indicates that the gradient descent algorithm is being executed, and the calculation process is as follows:

[0025]

[0026] 2) Update the local model using the local dataset size s as weights:

[0027]

[0028] When t>1, the client uses the accuracy of the previous model as weights to update the local model. The specific steps are as follows:

[0029] 1) Update the local model using the local dataset:

[0030]

[0031] 2) Utilize the accuracy of the previous model Used as weights to update the local model

[0032]

[0033] Upload the updated local model to TEE. TEE encrypts and signs the model. The specific steps are as follows:

[0034] 1) The client uploads the local model to the local TEE via a secure channel;

[0035] 2) The local TEE uses the MKHE.Enc encryption model and the Sign signature algorithm to sign the encryption result:

[0036]

[0037] 3) The client downloads the file in its local TEE. And upload To blockchain;

[0038] When information is uploaded to the blockchain, the smart contract first verifies the correctness of the signature with the TEE service provider. Only after successful verification can the information be uploaded to the blockchain. In model aggregation, each client downloads the encrypted model information of other clients in the same group from the blockchain and aggregates it to generate a ciphertext sub-global model. Clients in the same group collaboratively decrypt the ciphertext sub-global model and upload it to the blockchain. Finally, they download the plaintext sub-global models corresponding to other groups and calculate the global model. The specific execution steps are as follows:

[0039] If the clients in a group are It is P in round t q The client in the model corresponds to the generated encrypted local model;

[0040] 1) Ciphertext using the MKHE.Add multi-key homomorphic encryption addition algorithm aggregation model:

[0041]

[0042] 2) The client uses the threshold decryption algorithm MKHE.PartDec to perform threshold decryption on the aggregation result:

[0043]

[0044] 3) The client will display the decryption result. Upload to the blockchain and download from the blockchain

[0045] 4) Each client within the group uses the threshold aggregation algorithm MKHE.FinDec to calculate the sub-global model and uploads the generated plaintext sub-global model to the blockchain:

[0046]

[0047] 5) The client sums all sub-global models to obtain the global model:

[0048]

[0049] In the model quality assessment, the task publisher evaluates the quality of the local model of each participating client. After each round of model aggregation, the task publisher uploads the global model and threshold to the client's TEE via a secure channel. The model evaluation algorithm is run in the TEE to evaluate the local model uploaded by the client. After the evaluation, the TEE returns the evaluation results to the task publisher via a secure channel. The task publisher then selects participating clients based on the evaluation results. The specific steps are as follows:

[0050] 1) The client downloads the global model ω from the blockchain.t And transmit the threshold and ω through a secure channel t TEE uploaded to the client;

[0051] 2) Evaluate model accuracy using the test dataset in TEE:

[0052]

[0053] 3) Let client P i Reputation value Update its reputation value:

[0054]

[0055] To ensure clients actively and honestly participate in training tasks, a fair incentive mechanism is needed to encourage users. The client's reputation value is the fundamental basis for this incentive mechanism design. Two factors influence this reputation value: the quality of the local model and the completion status of the training task. The task publisher will decide whether to select a client to participate in the training task based on their reputation value. The specific steps are as follows:

[0056] If the reputation reward is m and the penalty coefficient is c, for client P i ,T(P i ) is P i The time required to perform the aggregation and decryption tasks in this round, T q It is a time limit;

[0057] 1) If the client is within the specified time T q If you complete the task within the specified time, your reputation will increase.

[0058]

[0059] 2) Otherwise, its reputation score will be affected:

[0060]

[0061] This invention enables clients to directly exchange their models and effectively detects whether the models received by the client are of low quality. Furthermore, multi-key homomorphic encryption is used to protect client model information, enhancing data privacy. The method also supports client joining and leaving, effectively avoiding single points of failure and malicious behavior on the server, and ensuring model reliability with virtually no loss of accuracy in the generated global model. This invention offers strong security and high model accuracy. Attached Figure Description

[0062] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0063] The present invention will now be described in detail with reference to the accompanying drawings.

[0064] The distributed federated learning method for data privacy protection and model reliability of this invention includes the following entities: a client, a trusted authority (CA), and a task publisher. The client is responsible for training the local model and undertaking the aggregation task; the trusted authority (CA) is responsible for generating necessary parameters and certificates for the system; and the task publisher is responsible for selecting task regions and publishing tasks, etc. Specifically, it includes the following steps:

[0065] (1) System Settings

[0066] CA selects a security parameter λ and executes the multi-key homomorphic encryption parameter generation algorithm MKHE.Setup(1 λ Generate system parameters pp;

[0067] (2) Device initialization

[0068] Client P i The key generation algorithm MKHE.KeyGen(pp) is executed to generate a public-private key pair (pk). i ,sk i Client P i Initialize local dataset D i and Local Trusted Execution Environment (TEE) i The CA generates a reputation certificate for the client. Where id i It is the client's identity, rv i It's the client's reputation score, au i It contains all the auxiliary information for the client. Is CA in (id) i ,pk i ,rv i ,au i The signature on the TEE; client verification of the local TEE i The identity information is collected and a secure channel is established;

[0069] (3) Task Issuance

[0070] The task publisher selects a training task I, initializes the global model ω0, and selects a task region, dividing this task region into K sub-regions; the task publisher then sends the task information (location, tt, σ) to the task region. T This is broadcast to clients in the region, where `location` is the task region and information about how the region is divided into K sub-regions, `tt` represents other task information, including `ω0`, `T`, and `F(·)`, where `ω0` is the initial global model, `T` is the maximum number of training epochs, `F(·)` is the loss function, and `σ` is the target value.T It is the signature of the task publisher on the task information location,tt; any client interested in task I responds to the message broadcast by the task publisher by sending a response message; if client P i It is interested in the task and sends a response message to the task publisher. Among them RC i It is P i Reputation certificate, location i It is P i Location, It is P i In (RC) i ,location i The signature on the RM1, ..., RM2; assuming the response information received by the task publisher is RM1, ..., RM3. N The task publisher selects clients based on the device's location and reputation value. The task publisher sends specific tasks to the selected clients, remotely verifies the client's TEE and establishes a secure channel. Then, the task publisher pre-stores the test dataset into the TEE of all selected clients through the secure channel.

[0071] (4) Training and aggregation phase

[0072] In step 3, the client trains a local model using the local dataset, and then uploads the local model and its public key to the local TEE via a secure channel. The TEE encrypts the model using the MKHE.Enc encryption algorithm, generates an encrypted model, and signs the encrypted result using its own internal private key. Each client downloads the packaged encrypted model and signature information from the local TEE and uploads it to the blockchain. The specific steps are as follows:

[0073] There are t rounds of training. When t=1, the client updates the local model using the local dataset and its size. The specific steps are as follows:

[0074] 1) Update the local model using the local dataset. It is client P i The local model generated in round t, where η is the learning rate. This indicates that the gradient descent algorithm is being executed, and the calculation process is as follows:

[0075]

[0076] 2) Utilize the local dataset size s i Used as weights to update the local model:

[0077]

[0078] When t>1, the client uses the accuracy of the previous model as weights to update the local model. The specific steps are as follows:

[0079] 1) Update the local model using the local dataset:

[0080]

[0081] 2) Utilize the accuracy of the previous model Used as weights to update the local model

[0082]

[0083] Upload the updated local model to TEE. TEE encrypts and signs the model. The specific steps are as follows:

[0084] 1) The client uploads the local model to the local TEE via a secure channel;

[0085] 2) The local TEE uses the MKHE.Enc encryption model and the Sign signature algorithm to sign the encryption result:

[0086]

[0087] 3) The client downloads the file in its local TEE. And upload To blockchain;

[0088] When information is uploaded to the blockchain, the smart contract first verifies the correctness of the signature with the TEE service provider. Only after successful verification can the information be uploaded to the blockchain. In model aggregation, each client downloads the encrypted model information of other clients in the same group from the blockchain and aggregates it to generate a ciphertext sub-global model. Clients in the same group collaboratively decrypt the ciphertext sub-global model and upload it to the blockchain. Finally, they download the plaintext sub-global models corresponding to other groups and calculate the global model. The specific execution steps are as follows:

[0089] If the clients in a group are It is P in round t q The client in the model corresponds to the generated encrypted local model;

[0090] 1) Ciphertext using the MKHE.Add multi-key homomorphic encryption addition algorithm aggregation model:

[0091]

[0092] 2) The client uses the threshold decryption algorithm MKHE.PartDec to perform threshold decryption on the aggregation result:

[0093]

[0094] 3) The client will decrypt the result ρ qi Upload to the blockchain and download {ρ} from the blockchain q1 ,…,ρ qi-1 ,ρ qi+1 ,…,ρ qa};

[0095] 4) Each client within the group uses the threshold aggregation algorithm MKHE.FinDec to calculate the sub-global model, and then generates the plaintext sub-model.

[0096] Global model upload to blockchain:

[0097]

[0098] 5) The client sums all sub-global models to obtain the global model:

[0099]

[0100] In the model quality assessment, the task publisher evaluates the quality of the local model of each participating client. After each round of model aggregation, the task publisher uploads the global model and threshold to the client's TEE via a secure channel. The model evaluation algorithm is run in the TEE to evaluate the local model uploaded by the client. After the evaluation, the TEE returns the evaluation results to the task publisher via a secure channel. The task publisher then selects participating clients based on the evaluation results. The specific steps are as follows:

[0101] 1) The client downloads the global model ω from the blockchain. t And transmit the threshold and ω through a secure channel t TEE uploaded to the client;

[0102] 2) Evaluate model accuracy using the test dataset in TEE:

[0103]

[0104] 3) Let the reputation value of client P be... Update its reputation value:

[0105]

[0106] To ensure clients actively and honestly participate in training tasks, a fair incentive mechanism is needed to encourage users. The client's reputation value is the fundamental basis for this incentive mechanism design. Two factors influence this reputation value: the quality of the local model and the completion status of the training task. The task publisher will decide whether to select a client to participate in the training task based on their reputation value. The specific steps are as follows:

[0107] If the reputation reward is m and the penalty coefficient is c, for client P, T(P) is the time required for P to perform the aggregation and decryption tasks in this round. q It is a time limit;

[0108] 1) If the client is within the specified time T q If you complete the task within the specified time, your reputation will increase.

[0109]

[0110] 2) Otherwise, its reputation score will be affected:

[0111]

Claims

1. A distributed federated learning method for data privacy protection and high model quality, comprising the following entities: a client, a trusted authority (CA), and a task publisher; wherein the client is responsible for training the local model and undertaking the aggregation task; the trusted authority (CA) is responsible for generating parameters and certificates for the system; and the task publisher is responsible for selecting the task region and publishing the task; characterized in that, The distributed federated learning method includes the following steps: Step 1: System Settings CA selects a security parameter λ and executes the multi-key homomorphic encryption parameter generation algorithm MKHE.Setup. Generate system parameters pp; Step 2: Device Initialization Client Generate your own public / private key pair ( , The client has a local dataset. Trusted Execution Environment The CA generates a reputation certificate for the client. ; Client verifies local trusted execution environment Identify and establish a secure channel; clients jointly maintain a blockchain for synchronizing information; Step 3: Task Issuance The task publisher generates a test dataset according to the task requirements; the task publisher initializes the global model and selects the task region; the task publisher divides the task region into sub-regions, and clients in the same sub-region are grouped together; the task publisher remotely verifies the client's Trusted Execution Environment (TEE) and establishes a secure channel, and then uploads the test dataset to the client's TEE. Step 4: Training and Aggregation Phase In step 3, the client uses the local dataset to train and generate a local model, and then uploads the local model and its public key to the local TEE via a secure channel; TEE encrypts and signs the model and uploads the results to the blockchain; The client downloads encrypted model information from other clients in the same group from the blockchain and aggregates it to generate a ciphertext sub-global model. Clients in the same group collaboratively decrypt the ciphertext sub-global model and upload it to the blockchain. Finally, they download the plaintext sub-global models corresponding to other groups and calculate the global model. After obtaining the global model, the client first uses a TEE to evaluate the quality of its local model, updates its reputation value based on the evaluation results, and finally updates its local model using the global model. After T rounds, training ends. Wherein: Step 4 specifically includes: In step 3, the client trains a local model using the local dataset, and then uploads the local model and its public key to the local TEE via a secure channel. The TEE encrypts the model using the MKHE.Enc encryption algorithm, generates an encrypted model, and signs the encrypted result using its own internal private key. Each client downloads the packaged encrypted model and signature information from the local TEE and uploads it to the blockchain. When information is uploaded to the blockchain, the smart contract first verifies the correctness of the signature with the TEE service provider. Only after the verification is successful can the information be uploaded to the blockchain. In model aggregation, each client downloads the encrypted model information of other clients in the same group from the blockchain and aggregates it to generate a ciphertext subglobal model. Clients in the same group collaborate to decrypt the ciphertext subglobal model and upload it to the blockchain. Finally, they download the plaintext subglobal models corresponding to other groups and calculate the global model. In the model quality assessment, the task publisher evaluates the quality of the local model of each participating client. After each round of model aggregation, the task publisher uploads the global model and threshold to the client's TEE via a secure channel. The model evaluation algorithm is run in the TEE to evaluate the local model uploaded by the client. After the evaluation is completed, the TEE returns the evaluation results to the task publisher via a secure channel. The task publisher then selects participating clients based on the evaluation results.

Images