Secure and Verifiable Federated Learning Method

Abstract

The present invention discloses a secure and verifiable federated learning scheme with low communication volume and resistance to tampering attacks and deletion attacks, comprising the following steps: (1) initialization: a trusted third party generates initialization parameters and a public-private key pair for each user and distributes them to the corresponding users; (2) secret sharing: each user generates a mask and shares the mask with each user through secret sharing technology; (3) encryption and proof generation: the user uses the mask to encrypt the local update to be uploaded and generates a proof for the unencrypted local update for subsequent correctness verification and resistance to spoofing attacks; (4) aggregation verification: the trusted third party collects the user's proof to generate a verification proof, and the server requests to restore the mask of the offline user; (5) aggregation update: the user verifies the server's request. If the verification is successful, the server restores the mask and then aggregates the updates uploaded by the user; (6) correctness verification: the user verifies the correctness of the aggregation result sent by the server.

Description

Technical Field

[0001] The present invention relates to federated learning, privacy protection, and verification of aggregated results, and more specifically to a verifiable secure aggregated federated learning method in a cloud-edge collaborative environment. Background Art

[0002] In the process of federated learning, there are problems of privacy leakage, incorrect aggregation results, and user disconnection. That is, the parameters uploaded by users may leak their privacy, malicious or lazy servers may return incorrect aggregation results, and users at the terminal nodes may be disconnected due to network fluctuations.

[0003] Federated learning, a distributed machine learning method, protects users' local data during training, making data available but invisible. In a classic federated learning process, there are generally two entities: n users and one cloud server. All users train their local datasets according to a global model. The trained parameters are then shared with the cloud server, which aggregates the user parameters and returns the aggregated results to the user. The user then updates the global model based on the aggregated results and begins the next round of training. This prevents the leakage of the user's local dataset and ensures the privacy of the user's local dataset. However, this process still presents some challenges. First, there is the issue of privacy leakage. With the advancement of attack techniques, research has demonstrated that attackers can infer users' local training data from uploaded parameters. Second, there is the issue of incorrect aggregation results. If the cloud server is lazy, it may use a simpler but incorrect aggregation method to aggregate the user's uploaded parameters to save computing resources, resulting in incorrect aggregation results. Furthermore, if the cloud server is malicious, it may directly tamper with the parameters and return incorrect aggregation results. Finally, users may be disconnected at any time due to power or network issues on their end devices (mobile phones, computers, and other IoT devices).

[0004] The following are some of the technologies used in verifiably secure aggregated federated learning in a cloud-edge collaborative environment:

[0005] Homomorphic hash functions can verify our aggregation results. Assuming that a file is divided into n file blocks, homomorphic hashing can calculate a hash value for each file block. Then we can aggregate the hash values of all file blocks to finally get the hash value of the original file. Therefore, for data that needs to be aggregated into one piece, we can first calculate the hash value of each part of the data, and finally aggregate the hash values of the parts of the data to get the hash value of the entire data. Specifically, where a iis a file block and is a vector of length m, and p is a random large prime number.

[0006] In previous work, the federated learning method using the double-masking protocol is a classic privacy-preserving federated learning method. This method achieves security by masking local updates twice, protecting data during transmission and preventing servers from eavesdropping on local updates. Double masking involves users adding noise to their local updates twice: the first noise added is called the primary mask, and the second noise added is called the secondary mask. In this method, users generate and share the primary mask between each other through a secure protocol, while the secondary mask is randomly generated by each user. Each user encrypts their local updates by adding the primary mask. However, in practice, some users may experience disconnection due to network instability. This can lead to inaccurate aggregation results due to the inability to fully offset the primary mask. Therefore, secret sharing techniques are used to address disconnection issues. However, this process poses a risk of spoofing attacks, where the server can deceive users to recover the primary mask and subsequently calculate their local updates, infringing their privacy. Therefore, federated learning using a double-masking protocol introduces a secondary mask to combat spoofing attacks and ultimately achieves correct aggregation results. However, this approach further increases communication overhead and fails to consider the possibility that the server could maliciously tamper with the aggregation results, impacting learning efficiency.

[0007] In most aggregation result verification technologies, users generally send local updates and verification certificates of local updates (hereinafter referred to as certificates) to the server. The server then aggregates the local updates and certificates separately and sends them back to the user. Users can verify the aggregation results of local updates and certificates through a designed algorithm to determine whether the aggregation results have been tampered with. The above method can effectively resist the above-mentioned tampering attacks, but it cannot resist deletion attacks. That is, when the server deletes the local updates and certificates of certain users during aggregation, the user cannot detect these problems. Deletion attacks can also cause inaccurate aggregation results, affecting learning efficiency. Summary of the Invention

[0008] The present invention includes three types of entities: cloud servers, edge servers, and users. The federated learning model in the present invention is a federated learning model suitable for a cloud-edge collaborative environment. The cloud server is malicious or lazy and may tamper with the aggregation results to affect the training of the model. The edge server is also malicious or lazy and may perform more efficient but incorrect calculations to save computing resources. Each user is honest and curious. It will honestly train and upload the correct parameters, but will be interested in other users' data and try to infer their private data based on the parameters they upload. Neither the central server nor the trusted third party colludes with the central server.

[0009] To address the above technical issues, the present invention provides a secure and verifiable federated learning method with low communication volume and resistance to tampering and deletion attacks. Regarding encryption security, this method proposes a one-masking protocol. Compared to double-masking, this method only requires a single mask, thus achieving privacy protection while also reducing communication overhead. Regarding verification of the correctness of aggregated results, the present invention proposes an aggregate verification method. Compared to traditional aggregate verification, this method can resist both tampering and deletion attacks, thus ensuring the correctness of aggregated results in multiple ways.

[0010] The technical solution of the present invention to solve the above technical problems includes the following steps:

[0011] (1) Initialization: A trusted third party generates initialization parameters and each user's public and private key pair and distributes them to the corresponding users;

[0012] (2) Secret sharing: Each user generates a mask and shares the mask with every other user through secret sharing technology;

[0013] (3) Encryption and proof generation: The user uses a mask to encrypt the local update to be uploaded and generates a proof for the unencrypted local update for subsequent correctness verification and defense against spoofing attacks;

[0014] (4) Aggregate Verification: A trusted third party collects user certificates to generate a verification certificate, and the server requests to restore the mask of the disconnected user;

[0015] (5) Aggregate updates: The user verifies the server's request. If the verification is successful, the server restores the mask and then aggregates the updates uploaded by the user.

[0016] (6) Correctness verification: The user verifies the correctness of the aggregation results sent by the server.

[0017] In the above-mentioned secure and verifiable federated learning method, in step (1), the initialization phase mainly involves the generation and distribution of initialization parameters. The specific process is as follows:

[0018] Trusted Third Party:

[0019] 1-1) Considering that secure communication is required between any two entities, the trusted third party can adopt the DH protocol. The trusted third party generates a key pair for each user according to the DH protocol and u ∈ U, where U is the set of all users, and and are both public keys, and are both private keys. Specifically, given g and p, and g is a primitive root of the prime number p, the trusted third party generates a random number x for user A A <p as the private key Calculate the public key as Similarly, we can get

[0020] 1-2) At the same time, for the resistance to deception attacks and the verification of the correctness of the aggregation result in subsequent training, the trusted third party generates many hash functions H used by the Bloom filter i , where 1 ≤ i ≤ k, the length L of the Bloom filter, and the homomorphic hash function HHF. Finally, the trusted third party sends the initialization parameters to user u;

[0021] User u:

[0022] 1-3) After receiving Then send the public key to the server through a secure channel;

[0023] Server:

[0024] 1-4) After receiving the public key of each user u Then Distribute it to other users so that each user can obtain the public keys of other users and the id number u.

[0025] For the above secure and verifiable federated learning method, the specific process of secret sharing in step (2) is as follows:

[0026] User u:

[0027] 2-1) After receiving the public key of other user v and the id number v;

[0028] 2-2) Use its own private key and the public key of user v According to the DH protocol for secure communication, negotiate a shared random number r u,v ;

[0029] 2-3) Using a pseudo-random number generator, r u,v Generates a pseudo-random number PRG(r u,v );

[0030] 2-4) Use secret sharing technology to convert the pseudo-random number PRG(r u,v ) to other users, that is, {s u,v} v∈n ←Shamir.Share(PRG(s u,v ),t). Where {s u,v} v∈n That is, n shares of secret sharing;

[0031] 2-5) Using your own private key and the public key of user v According to the DH protocol, secure communication is performed and the share {s u,v} v∈n Sent to user v.

[0032] In the above secure and verifiable federated learning method, the specific process of encryption and proof generation in step (3) is as follows:

[0033] User u:

[0034] 3-1) According to the global model, train on the local data set to obtain the local gradient a u ;

[0035] 3-2) Encrypt local gradient:

[0036] 3-3) To defend against server spoofing, tampering, and deletion attacks, the following operations are required:

[0037] Calculate B u =H i (u), where B u is the Bloom filter containing the user u mapping;

[0038] Calculate the hash value of the local gradient

[0039] 3-4) {B u ,HHF(a u )} u∈U Send to a trusted third party, Send to the server.

[0040] In the above secure and verifiable federated learning method, the specific process of aggregate verification in step (4) is as follows:

[0041] server:

[0042] 4-1) Receive information from a group of users, and use U1 to represent this group of users. And |U1|≥t, otherwise stop this round of training;

[0043] 4-2) Send to all users That is, send the offline user ID to all users;

[0044] Trusted Third Party:

[0045] 4-3) Receive information from a group of users, and use U1 to represent this group of users. Where |U1|≥t, otherwise stop this round of aggregation

[0046] 4-4) Calculation Computational Proof Where B is the union of all users’ Bloom filters, which is used to defend against subsequent spoofing attacks; Proof is used to verify the correctness of the aggregation results;

[0047] 4-5) Send B and Proof to all users.

[0048] In the above secure and verifiable federated learning method, the specific process of aggregation update in step (5) is as follows:

[0049] User u:

[0050] 5-1) Receive B and Proof;

[0051] 5-2) Verify H i (v) Is it on B? If it is on B, stop this training. If it is not on B, change the share Send to the server;

[0052] server:

[0053] 5-3) Receive sub-secrets from at least t users Otherwise, stop training. According to the secret sharing algorithm, recover the secret

[0054] 5-4) Calculate the aggregation results

[0055] 5-5) Send A to all users.

[0056] In the above-mentioned secure and verifiable federated learning method, the specific process of correctness verification in step (6) is as follows:

[0057] User u:

[0058] 6-1) After receiving A, perform aggregation result verification to determine whether Proof is equal to HHF(A). If so, verification succeeds, receive A, and start the next round of training. Otherwise, stop training.

[0059] The beneficial effects of the present invention are:

[0060] 1. This paper proposes a one-masking security encryption method in a dual-server environment. In addition to achieving security protection for uploaded data during federated learning, it can also resist server deception attacks.

[0061] 2. Compared with the double-masking encryption protocol, the present invention reduces one masking during the method design process, so the present invention can effectively reduce communication overhead.

[0062] 3. The present invention proposes a server aggregation result verification method in a dual-server environment, which can resist server tampering attacks, and compared with traditional aggregation result verification, the present invention can also resist server deletion attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is the overall flow chart of the present invention.

[0064] Figure 2 This is a system framework diagram of the present invention.

[0065] Figure 3 This is a schematic diagram of initialization of the present invention.

[0066] Figure 4 Schematic diagram of secret sharing of the present invention.

[0067] Figure 5 This is a schematic diagram of the encryption and proof generation, aggregate verification, and aggregate update of the present invention. DETAILED DESCRIPTION

[0068] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0069] like Figure 2 As shown, Figure 2 This is the system framework diagram of the present invention. The system includes three entities: a trusted third party, a central server, and users. The trusted third party primarily generates initialization parameters and aggregates proofs and Bloom filters. The central server, which is malicious, primarily aggregates local updates to generate global updates. Users train based on the global updates and also verify the correctness of the final aggregated results. The trusted third party and the central server do not collude.

[0070] like Figure 1As shown in the figure, a secure and verifiable federated learning method, characterized by comprising the following steps:

[0071] (1) Initialization: The trusted third party generates the initialization parameters and the public-private key pairs of each user, and distributes them to the corresponding users. As Figure 3 shown in the figure, the specific process of initialization is as follows:

[0072] Trusted third party:

[0073] 1-1) Considering that secure communication needs to be carried out between any two entities, the trusted third party can adopt the DH protocol. The trusted third party generates the key pairs for each user according to the DH protocol and u ∈ U, where U is the set of all users, and and are both public keys, and are both private keys. Specifically, given g and p, and g is a primitive root of the prime number p, the trusted third party generates a random number x for user A A <p as the private key Calculate the public key as Similarly, we can get

[0074] 1-2) At the same time, in order to resist deception attacks and verify the correctness of the aggregation results in subsequent training, the trusted third party generates many hash functions H used by the Bloom filter i , where 1 ≤ i ≤ k, the length L of the Bloom filter, and the homomorphic hash function HHF. Finally, the trusted third party sends the initialization parameters to user u;

[0075] User u:

[0076] 1-3) After receiving then send the public key to the server through a secure channel;

[0077] Server: [[ID=**54**]]

[0078] 1-4) After receiving the public key of each user u then distribute it to other users again, so that each user can obtain the public keys of other users and the id number u.

[0079] (2) Secret sharing: Each user generates a mask and shares the mask with each user through the secret sharing technology. As Figure 4 shown in the figure, the specific process of user secret sharing is as follows: <00002**68**>User u: Note: There seems to be a duplicate tag number in the original text (both

[0078] and

[0080] are repeated in the translation). Please check the original text for accuracy. Also, the specific mathematical expressions and cryptographic concepts represented by these tags may need further context understanding for a more comprehensive and accurate translation. This translation attempts to maintain the integrity of the original text structure and tag usage as much as possible.

[0081] 2-1) Receive the public key of other user v and id number v;

[0082] 2-2) Using your own private key and the public key of user v Perform secure communication according to the DH protocol and negotiate a shared random number r u,v ;

[0083] 2-3) Using a pseudo-random number generator, r u,v Generates a pseudo-random number PRG(r u,v );

[0084] 2-4) Use secret sharing technology to convert the pseudo-random number PRG(r u,v ) to other users, that is, {s u,v} v∈n ←Shamir.Share(PRG(s u,v ),t). Where {s u,v} v∈n That is, n shares of secret sharing;

[0085] 2-5) Using your own private key and the public key of user v According to the DH protocol, secure communication is performed and the share {s u,v} v∈n Sent to user v.

[0086] (3) Encryption and proof generation: The user uses a mask to encrypt the local update to be uploaded and generates a proof for the unencrypted local update for subsequent correctness verification and defense against spoofing attacks;

[0087] (4) Aggregate Verification: A trusted third party collects user certificates to generate a verification certificate, and the server requests to restore the mask of the disconnected user;

[0088] (5) Aggregate updates: The user verifies the server's request. If the verification is successful, the server restores the mask and then aggregates the updates uploaded by the user.

[0089] (6) Correctness verification: The user verifies the correctness of the aggregation results sent by the server;

[0090] like Figure 5 As shown, the specific process of encryption and proof generation, aggregation verification, aggregation update and correctness verification is as follows:

[0091] User u:

[0092] 3-1) According to the global model, train on the local data set to obtain the local gradient au ;

[0093] 3-2) Encrypt local gradient:

[0094] 3-3) To defend against server spoofing, tampering, and deletion attacks, the following operations are required:

[0095] Calculate B u =H i (u), where B u is the Bloom filter containing the user u mapping;

[0096] Calculate the hash value of the local gradient

[0097] 3-4) {B u ,HHF(a u )} u∈U Send to a trusted third party, Send to the server.

[0098] server:

[0099] 4-1) Receive information from a group of users, and use U1 to represent this group of users. And |U1|≥t, otherwise stop this round of training;

[0100] 4-2) Send to all users That is, send the offline user ID to all users;

[0101] Trusted Third Party:

[0102] 4-3) Receive information from a group of users, denoted by U1. Where |U1|≥t, otherwise terminate this round of aggregation.

[0103] 4-4) Calculation Computational Proof Where B is the union of all users’ Bloom filters, which is used to defend against subsequent spoofing attacks; Proof is used to verify the correctness of the aggregation results;

[0104] 4-5) Send B and Proof to all users.

[0105] User u:

[0106] 5-1) Receive B and Proof;

[0107] 5-2) Verify H i (v) Is it on B? If it is on B, stop this training. If it is not on B, change the share Send to the server;

[0108] server:

[0109] 5-3) Receive sub-secrets from at least t users Otherwise, stop training. According to the secret sharing algorithm, recover the secret

[0110] 5-4) Calculate the aggregation results

[0111] 5-5) Send A to all users.

[0112] User u:

[0113] 6-1) After receiving A, perform aggregation result verification to determine whether Proof is equal to HHF(A). If so, verification succeeds, receive A, and start the next round of training. Otherwise, stop training.

Claims

1. A secure and verifiable federated learning method, characterized by: The following steps are involved: (1) Initialization: A trusted third party generates initialization parameters and each user's public and private key pair and distributes them to the corresponding users; (2) Secret sharing: Each user generates a mask and shares the mask with every other user through secret sharing technology; (3) Encryption and proof generation: The user uses a mask to encrypt the local update to be uploaded and generates a proof for the unencrypted local update for subsequent correctness verification and defense against spoofing attacks; (4) Aggregate Verification: A trusted third party collects user certificates to generate a verification certificate, and the server requests to restore the mask of the disconnected user; (5) Aggregate updates: The user verifies the server's request. If the verification is successful, the server restores the mask and then aggregates the updates uploaded by the user. (6) Correctness verification: The user verifies the correctness of the aggregation results sent by the server; The specific process of secret sharing in step (2) is as follows: User u: 2-1) Receive the public key of other user v and id number v; 2-2) Using your own private key and the public key of user v Perform secure communication according to the DH protocol and negotiate a shared random number r u,v ; 2-3) Using a pseudo-random number generator, r u,v Generates a pseudo-random number PRG(r u,v ); 2-4) Use secret sharing technology to convert the pseudo-random number PRG(r u,v ) to other users, that is, {s u,v } v∈n ←Shamir.Share(PRG(s u,v ),t), where {s u,v } v∈n That is, n shares of secret sharing; 2-5) Using your own private key and the public key of user v According to the DH protocol, secure communication is performed and the share {s u,v } v∈n Sent to user v.

2. The secure and verifiable federated learning method according to claim 1, characterized in that: The initialization phase in step (1) is mainly about a trusted third party generating initialization parameters, and the specific process is as follows: Trusted Third Party: 1-1) The trusted third party generates a key pair for each user according to the DH protocol and for u ∈ U, where U is the set of all users, and both are public keys, and both are private keys; specifically, given g and p, where g is a primitive root of the prime number p, the trusted third party generates a random number x for user A A <p is used as the private key calculate the public key as Similarly, we can get 1-2) A trusted third party generates the hash functions H used by the Bloom filter i , where 1≤i≤k, the length L of the Bloom filter and the homomorphic hash function HHF; 1-3) The trusted third party will initialize the parameters Sent to user u, the trusted third party only sends the initialization parameters to the corresponding user, that is, the initialization parameters received by user u only contain its own public and private keys, and do not contain the public and private keys of other users; User u: 1-4) Receive Then the public key Send to the server via a secure channel; server: 1-5) Receive the public key of each user u Then Then distribute it to other users so that each user can obtain the public key of other users And id number u.

3. The secure and verifiable federated learning method according to claim 2, characterized in that: The specific process of encryption and proof generation in step (3) is as follows: User u: 3-1) According to the global model, train on the local data set to obtain the local gradient a u ; 3-2) Encrypt local gradient: 3-3) To defend against server spoofing, tampering, and deletion attacks, the following operations are required: Calculate B u =H i (u), where B u is the Bloom filter containing the user u mapping; Calculate the hash value of the local gradient 3-4) {B u ,HHF(a u )} u∈U Send to a trusted third party Send to the server.

4. The secure and verifiable federated learning method according to claim 3, characterized in that: In step (4), the aggregation verification process is: server: 4-1) Receive information from a group of users, which is represented by U1. And |U1|≥t, otherwise stop this round of training; 4-2) Send to all users That is, send the offline user ID to all users; Trusted Third Party: 4-3) Receive information from a group of users, and use U1 to represent this group of users, where |U1| ≥ t, otherwise stop this round of aggregation; 4-4) Calculation Computational Proof Where B is the union of all users’ Bloom filters, which is used to defend against subsequent spoofing attacks, and Proof is used to verify the correctness of the aggregation results; 4-5) Send B and Proof to all users.

5. The secure and verifiable federated learning method according to claim 4, characterized in that: In step (5), the aggregation update process is: User u: 5-1) Receive B and Proof; 5-2) Verify H i (v) Is it on B? If it is on B, stop this training. If it is not on B, change the share Send to the server; server: 5-3) Receive sub-secrets from at least t users Otherwise, stop training and recover the secret according to the secret sharing algorithm. 5-4) Calculate the aggregation results 5-5) Send A to all users.

6. The secure and verifiable federated learning method according to claim 5, characterized in that: In step (6), the correctness verification process is: User u: 6-1) After receiving A, perform aggregation result verification to determine whether Proof is equal to HHF(A). If so, verification succeeds, receive A, and start the next round of training. Otherwise, stop training.

Images