Verifiable security federal learning method for unreliable user
By adopting a five-stage method in the federated learning framework, combining European-style distance function and secret sharing technology, processing the gradient of unreliable users and encrypting data, the problem of unreliable users' training efficiency and privacy protection of model is solved, and data privacy is realized Dual guarantees and results verification.
Patent Information
- Application Number
- CN202411449355.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2024-12-27
AI Technical Summary
When facing unreliable users, low-quality data leads to reduced model training efficiency and privacy risks, and existing privacy protection technologies are difficult to take into account user verification and data privacy protection.
Using five-stage methods: initialization, key sharing, mask input, mask elimination and aggregation, verification stages, user weight is calculated through the European distance function, gradients and weights are encrypted using secret sharing and symmetric encryption technology, verification labels are generated and verification is verified. Correctness of the aggregation results.
Mitigate the negative impact of unreliable users on the global model, protect data privacy, reduce communication overhead, and ensure the verifiability and accuracy of aggregated results.
Smart Images
Figure CN119203238A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a verifiable security method for unreliable users, and in particular to a verifiable security federated learning method for unreliable users. Background Art
[0002] The information provided in this section is for the purpose of generally presenting the background of the present disclosure. To the extent described in this section, the work of the presently named inventors and aspects of the description that may not constitute prior art at the time of filing are neither explicitly nor implicitly admitted to be prior art to the present disclosure.
[0003] With the widespread application of data and artificial intelligence, data privacy protection has become a key issue that cannot be ignored. As personal and sensitive information is collected and processed in large quantities, the risk of data leakage is increasing. In response to this challenge, governments and regulators around the world have implemented strict data privacy regulations. These regulations have had a significant impact on traditional machine learning methods that rely on centralized data collection and analysis, especially data privacy issues in model training. Against this background, federated learning came into being and became an effective way to protect data privacy. Federated learning was proposed by Google Labs and allows distributed model training on multiple devices.
[0004] However, in the existing federated learning framework, there is still a potential risk of privacy leakage when the gradient model is shared between users and servers. In addition, federated learning faces another important challenge - unreliable users. The low-quality data provided by these users not only reduces the convergence speed of the model, but may even cause the model to fail. In addition, the wrong aggregation results returned by the cloud server may further deviate from the user's learning results, so the user needs a mechanism to verify the correctness of the aggregation results.
[0005] Existing federated learning privacy protection technologies mainly include multi-party secure computing and homomorphic encryption. These technologies can effectively protect the privacy of individual users, but they are weak in dealing with the negative impact of unreliable users. The low-quality data generated by unreliable users may seriously hinder the training of the global model and even affect its accuracy. In order to solve the problem of unreliable users, some studies have proposed the use of differential privacy technology to protect privacy by interfering with the loss function of the model. However, differential privacy often finds it difficult to strike a balance between privacy protection and model utility, resulting in impaired model performance. In addition, there are methods based on the Euclidean distance algorithm, which assigns weights to users by calculating the distance between the user gradient and the global gradient to reduce the impact of unreliable users.
[0006] In terms of dealing with cloud servers returning erroneous aggregation results, some studies have proposed verifiable methods based on homomorphic hash functions, but their computational overhead is huge. Another verification method based on Lagrange interpolation also faces the bottleneck of high communication overhead. Existing technologies either focus on mitigating the impact of unreliable users or try to protect privacy, but it is difficult to take into account the user's verification needs for the results returned by the cloud server. Therefore, there is an urgent need for a method that can simultaneously protect data privacy and mitigate the impact of unreliable users. Summary of the invention
[0007] The purpose of the present invention is to provide a verifiable secure federated learning method for unreliable users, which can further ensure the verifiability of the calculation results while reducing the adverse effects of the unreliable user's gradient on the global model and protecting the user's data privacy.
[0008] In order to achieve the above object, the present invention provides a verifiable secure federated learning method for unreliable users, which includes the following five stages. In each stage, the number of online users is not less than a threshold value t, otherwise the calculation is terminated; it is assumed that there are n users participating in the federated learning process, and the set of all users is represented as U = {u i , i∈[1,n]}, where u i represents the i-th customer or customer i; set U i represents the set of users online in the i-1th stage, that is,
[0009] (S100) Phase 0: Initialization Phase
[0010] Before all users start training, the authority generates the gradient parameters x 0 ; Then, the authority generates the user's public key / private key (pk i ,sk i ), where pk i Represents the public key, sk i Represents a private key, which is used to generate a shared key between users; the authority generates a random seed b∈GF(q), a random vector a and a random number μ; a and μ are visible to all users but not to the cloud server; the random number μ is used by the user to mask the plaintext; GF(q) represents a finite field, q represents a large prime number; the authoritative institution releases system parameters p={x 0 ,a,b,μ,pk i ,sk i} and go offline, the user shares his public key through the cloud server, and the cloud server records the group of users received as U 1 ;
[0011] (S200) Phase 1: Key Sharing Phase
[0012] When the iteration round of model training is the Tth round, each user u i ∈U 1 Get local gradients through local model training By using The Euclidean distance function obtains the distance between the user's local gradient and the global model, and then calculates the weight w i , get the weighted gradient
[0013] Each user randomly generates two masks and And using the secret partitioning algorithm and Generate shares, user u i The secret share sent to other users is expressed as
[0014] The user uses symmetric encryption technology to encrypt the secret share Then the user sends it to the cloud server; the cloud server records the received group of users as U 2 , and e i，j Distribute to U 2 The corresponding user in;
[0015] (S300) Stage 2: Mask Input Stage
[0016] Each user uses a mask and Encrypt weighted gradients separately and weight w i , and use b for secondary masking to prevent the server from knowing the aggregation result, and obtain the encrypted gradient and encryption weight Where B = PRG(b), and PRG() is a pseudo-random generator;
[0017] User Utilization Generate verification tags and get and in is the Hadamard product;
[0018] User uploaded encrypted gradient Weight and the corresponding validation tags to the cloud server; when the cloud server receives user u i When the information is i Join the online user group U 3; Then, the cloud server will broadcast the user set U 3 of online users;
[0019] (S400) Stage 3: Mask removal and aggregation stage
[0020] User i The encrypted e is received from the cloud server i，j , then decrypt to get the corresponding and User Computing and And send it to the cloud server. When the cloud server receives at least t users, this group of users is recorded as U 4 ;
[0021] The cloud server reconstructs the and in
[0022] The cloud server receives information from each user u i ∈U 3 Received Obtained by aggregation and Among them, the two summary masks X agg represents the sum of the encrypted weighted gradients received from the user, W agg Indicates the encrypted weight sum received from the user;
[0023] Then, the cloud server aggregates the verification tags of all users to obtain and
[0024] The cloud server aggregates the results X agg , W agg , verification tag R 1 , R 2 Broadcast to U 4 Users in
[0025] (S500) Stage 4: Verification Stage
[0026] After receiving the aggregation result, the user verifies the proof and the aggregation mask, and finally performs the aggregation result verification to check whether the following equations 1) and 2) hold;
[0027]
[0028] If equations 1) and 2) are both true, it means that the user gets the correct aggregation result: Otherwise, the verification fails and the run is aborted.
[0029] Preferably, in step (S200), the weight w i Obtained using the truth discovery algorithm.
[0030] Preferably, in step (S200), the user u i The weight w i Calculated by the following formula:
[0031]
[0032] in, is the coefficient used to scale the user weight, where |M| is , χ is the chi-square distribution, and α is the corresponding significance level.
[0033] Preferably, the is the Euclidean distance function, used to calculate user u i Local gradient in round T and the global model x at round T-1 T-1 The Euclidean distance between 0 is provided by an authoritative organization; Euclidean distance function
[0034] Preferably, the gradient parameter x 0 Calculated from public auxiliary databases.
[0035] Preferably, in step (S200), each user has a corresponding random number β i , will be shared in advance Random number beta i It is the address value corresponding to each user in secret sharing.
[0036] Secret Splitting Algorithm Include:
[0037] The secret s is divided into n shares. Only when at least t shares are obtained can the secret be obtained. In the (t, n) threshold scheme proposed by Shamir, the secret in is a finite field, 0<t≤n<q;
[0038] Given a secret s, a threshold t, and the set of all users U = {u 1 , ..., u n};from Randomly select t-1 positive integers and let the polynomial f(x) = s + a 1 x+a 2x+…+a t-1 x t-1 mod q, use f(x) to calculate f(β i ); the share of secret s is calculated as
[0039] Preferably, in step (S400), the secret reconstruction algorithm Include:
[0040] Given any t (β i ,f(β i )), the secret s can be reconstructed by Lagrange interpolation method, and the secret can be efficiently recovered by the following formula:
[0041]
[0042] Among them, j represents the index of the sum, which is iterated from 0 to t and is used to traverse β j And the function f(β j ) value; m represents the index of the product term, iterating from 0 to t, excluding j, for traversing except x j All x m Value to calculate Lagrange interpolation weight; x m represents the horizontal coordinate of the mth point used to calculate the Lagrange interpolation, that is, a point of the interpolation node; x j Represents the horizontal coordinate of the,th term in the summation, that is, a point of the interpolation node currently being processed.
[0043] Preferably, in step (S200), the symmetric encryption technology consists of the following two algorithms: a symmetric encryption algorithm AE.Enc, which, given a key and plain text as input, will be encrypted into ciphertext; and a symmetric decryption algorithm AE.De c , given a key and ciphertext as input, the plaintext will be decrypted; for any given key h and any plaintext message y, the symmetric encryption algorithm can always ensure that the equation AE.Dec(h, AE.Enc(h, y)) = y holds.
[0044] The verifiably secure federated learning method for unreliable users of the present invention has the following advantages:
[0045] The method of the present invention processes the gradient provided by the user under the federated learning framework to reduce the adverse effects of the gradient obtained through low-quality data training on the overall model; masks the weighted gradient to avoid data leakage and the deduction of the real source data, while reducing the communication overhead; the user makes consistency judgments based on the returned aggregation results to ensure that the received aggregation results have not been tampered with. While reducing the adverse effects of unreliable users' gradients on the global model and protecting the user's data privacy, the present invention further ensures the verifiability of the calculation results. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 This is a basic model diagram of the verifiable secure federated learning method for unreliable users of the present invention.
[0047] Figure 2 This is a diagram of the secure weighted federated learning model for unreliable users of the present invention. DETAILED DESCRIPTION
[0048] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0049] The features mentioned in the present invention can be combined arbitrarily, as long as there is no contradiction in the combination of these features, all possible combinations should be considered as the scope of this specification. Each feature disclosed in the specification can be replaced by any alternative feature that can provide the same, equal or similar purpose. Therefore, unless otherwise specified, the disclosed features are only general examples of equal or similar features.
[0050] The present invention proposes a verifiably secure federated learning method for unreliable users, which combines a truth discovery algorithm and a masking technique to improve the security and reliability of federated learning. Specifically, the truth discovery algorithm is used to process the gradients submitted by unreliable users, thereby reducing their adverse effects on the global model. The masking technique is used to encrypt the weighted gradients and weights to ensure that data privacy is protected. In addition, the verification tags generated by the user are used to verify the accuracy and security of the aggregation results. Ultimately, even in the presence of unreliable users, this method not only achieves efficient use of data, but also ensures the verifiability of the aggregation results, ensuring the accuracy and security of the model.
[0051] The following is a detailed description of a verifiable secure federated learning method for unreliable users provided by the present invention through Example 1.
[0052] Example 1
[0053] A verifiable secure federated learning method for unreliable users, the method includes three types of entities: authority, cloud server and user. The method includes the following five stages: (S100) Stage 0: initialization stage; (S200) Stage 1: key sharing stage; (S300) Stage 2: mask input stage; (S400) Stage 3: mask removal and aggregation stage; (S500) Stage 4: verification stage. In each stage, the number of online users is not less than the threshold t, otherwise the calculation is terminated. Assume that there are n users participating in the process of federated learning, and the set of all users is represented as U = {u i , i∈[1,n]}, where u i represents the i-th customer or customer i. Here we assume that the symbol U i represents the set of users online in the i-1th stage, that is,
[0054] The verifiable secure federated learning method for unreliable users of the present invention is as follows:
[0055] S100: Initialization phase
[0056] Before the user starts training, the authority generates the gradient parameters x 0 , where x 0 It can be calculated through the public auxiliary database. Then, the user's public key / private key (pk i ,sk i ), where pk i Represents the public key, sk i Represents the private key, which is used to generate shared keys between users. The authority generates a random seed b∈GF(q), a vector a, and a random number μ; a and μ are visible to users but not to the cloud server. The random number μ is used by users to mask the plaintext. GF(q) represents a finite field, and q represents a large prime number. The authority releases the system parameter p=(x 0 ,a,b,μ,pk i ,sk i} and go offline. Users share their public keys through the cloud server, and the cloud server records the group of users received as U 1 .
[0057] S200: Key sharing phase
[0058] When the iteration round of model training is the Tth round, each user u i ∈U 1 Get local gradients through local model training By using The Euclidean distance function obtains the distance between the user's local gradient and the global model, and then calculates the weight w i , get the weighted gradient
[0059] The calculation of weights uses the truth discovery algorithm, as follows:
[0060] Calculate the user's aggregate weight, user u i The weight w i Can be Calculate, where is the coefficient used to scale the user weight, the coefficient Can be calculated. Where |M| is , χ is the chi-square distribution, and α is the corresponding significance level. is a public parameter. Once α and |M| are determined, all users can calculate is the Euclidean distance function, used to calculate user u i Local gradient in round T and the global model x at round T-1 T-1 In the first round, x 0 It is provided by an authoritative organization. Euclidean distance function
[0061] Before weighted gradients and weights are encrypted, each user randomly generates a mask and And use and Generate shares, user u i The secret share that needs to be sent to other users is expressed as
[0062] Among them, the secret sharing technology (Secret Sharing, referred to as SS) is introduced as follows:
[0063] Secret sharing is to divide the secret s into n shares. Only when at least t shares are obtained can the secret be obtained. Specifically, in the (t, n) threshold scheme proposed by Shamir, the secret in is a finite field, with GF(q) For the same finite field, 0<t≤n<q.
[0064] In the method of the present invention, n users U = {u i , i∈[1,n]}, and each user has a corresponding random number β i , random number β iIt is the address value corresponding to each user in secret sharing, which will be shared in advance This method consists of two parts: secret distribution SS.Sh(·) and secret reconstruction SS.Re(·). The specific algorithm is as follows:
[0065] Secret Splitting Algorithm Given a secret s, a threshold t, and the set of all users U = {u 1 ,…,u n}.from Randomly select t-1 positive integers and let the polynomial f(x) = s + a 1 x+a 2 x+…+a t-1 x t-1 mod q, use f(x) to calculate f(β i ). The share of secret s can be calculated as
[0066] Secret reconstruction algorithm Given any t (β i ,f(β i )), the secret s can be reconstructed by Lagrange interpolation. The following formula can efficiently recover the secret Among them, j represents the index of the sum, which is iterated from 0 to t and is used to traverse β j And the function f(β j ) value; m represents the index of the product term, iterating from 0 to t, excluding j (i.e. m≠j), used to traverse all products except x j All x m Value to calculate Lagrange interpolation weight; x m represents the horizontal coordinate of the mth point used to calculate the Lagrange interpolation, that is, a point of the interpolation node; x j Represents the horizontal coordinate of the,th term in the summation, that is, a point of the interpolation node currently being processed.
[0067] The user uses symmetric encryption to encrypt the secret share as Then the user sends it to the cloud server. KA.agree() is the key negotiation algorithm that can obtain the shared secret value. The cloud server records the group of users received as U 2 , and e i，j Distribute to U 2 The corresponding user in .
[0068] Among them, symmetric encryption technology consists of the following two algorithms:
[0069] Symmetric encryption algorithm AE.Enc, given the key (ie KA.agree (sk i , pk j)) and plain text (i.e. ) as input, will be encrypted into ciphertext.
[0070] Symmetric decryption algorithm AE.Dec, given a key and ciphertext as input, will decrypt the plaintext.
[0071] For any given key h and any plaintext message y, the symmetric encryption algorithm can always ensure that the equation AE.Dec(h, AE.Enc(h, y)) = y holds.
[0072] S300: Mask input stage
[0073] Double aggregation is used to achieve the verifiability of the aggregation gradient. The first aggregation calculates the aggregation gradient itself, and the second aggregation is used to prove the correctness of the first aggregation result.
[0074] Each user uses a mask and Encrypt weighted gradients separately and weight w i , and use b for secondary masking to prevent the server from knowing the aggregation result. The first aggregation calculation obtains the encrypted gradient and encryption weight Wherein B=PRG(b), and PRG() is a pseudo-random generator.
[0075] User Utilization Generate verification tags, obtained through secondary aggregation and in is the Hadamard product.
[0076] User uploaded encrypted gradient Weight and the corresponding validation tags to the cloud server. When the cloud server receives user u i When the information is i Included in the online user set U 3 Then, the cloud server will broadcast the online user set U 3 Online users.
[0077] S400: Elimination of masking and aggregation phases
[0078] User i Receive the encrypted e from the cloud server i，j , then decrypt to get the corresponding and calculate and And send it to the cloud server.
[0079] When the cloud server receives enough users, it records this group of users as U 4 The cloud server reconstructs the and in
[0080] The cloud server receives data from each user u i ∈U 3 Received Obtained by aggregation and In addition, the two summary masks
[0081] Then, the cloud server aggregates the verification tags of all users to obtain and
[0082] The cloud server will aggregate the results X agg , W agg , verification tag R 1 , R 2 Broadcast to other users.
[0083] S500: Verification phase
[0084] After receiving the aggregation result, the user verifies the proof and the aggregation mask, and finally performs the aggregation result verification, that is, checks and Is it true?
[0085] If all the equations hold, it means that the user gets the correct aggregation result: Otherwise, the verification fails and the run is aborted.
[0086] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.
Claims
1. A verifiably secure federated learning method for unreliable users, characterized in that: The method includes the following five stages. In each stage, the number of online users is not less than the threshold t, otherwise the calculation is terminated. It is assumed that there are n users participating in the federated learning process, and the set of all users is represented as U = {u i ,i∈[1,n]}, where u i represents the i-th customer or customer i; set U i represents the set of users online in the i-1th stage, that is, (S100) Phase 0: Initialization Phase Before all users start training, the authority generates the gradient parameters x 0 ; Then, the authority generates the user's public key / private key (pk i ,sk i ), where pk i Represents the public key, sk i represents a private key, which is used to generate a shared key between users; the authority generates a random seed b∈GF(q), a random vector a and a random number μ; a and μ are visible to all users but not to the cloud server; The random number μ is used by the user to mask the plain text; GF(q) represents a finite field, and q represents a large prime number; the authoritative organization issues the system parameter p={x 0 , a, b, μpk i ,sk i } and go offline, the user shares his public key through the cloud server, and the cloud server records the group of users received as U1; (S200) Phase 1: Key Sharing Phase When the iteration round of model training is the Tth round, each user u i ∈U1 obtains local gradient through local model training By using The Euclidean distance function obtains the distance between the user's local gradient and the global model, and then calculates the weight w i , get the weighted gradient Each user randomly generates two masks and And using the secret partitioning algorithm and Generate shares, user u i The secret share sent to other users is expressed as The user uses symmetric encryption technology to encrypt the secret share Then the user sends it to the cloud server; the cloud server records the group of users received as U2 and i，j Distribute to corresponding users in U2; (S300) Stage 2: Mask Input Stage Each user uses a mask and Encrypt weighted gradients separately and weight w i , and use b for secondary masking to prevent the server from knowing the aggregation result, and obtain the encrypted gradient and encryption weight Where B = PRG(b), and PRG() is a pseudo-random generator; User Utilization Generate verification tags and get and in is the Hadamard product; User uploaded encrypted gradient Weight and the corresponding validation tags to the cloud server; when the cloud server receives user u i When the information is i Included in the online user group U3; then, the cloud server will broadcast the online users of user set U3; (S400) Stage 3: Mask removal and aggregation stage User i The encrypted e is received from the cloud server i，j , then decrypt to get the corresponding and User Computing and And send it to the cloud server. When the cloud server receives no less than t users, this group of users is recorded as U4; The cloud server reconstructs the and in The cloud server receives information from each user u i ∈U3 received Obtained by aggregation and Among them, the two summary masks X agg represents the sum of the encrypted weighted gradients received from the user, W agg Indicates the encrypted weight sum received from the user; Then, the cloud server aggregates the verification tags of all users to obtain and The cloud server aggregates the results X agg , W agg , verification tag R 1 , R 2 Broadcast to users in U4; (S500) Stage 4: Verification Stage After receiving the aggregation result, the user verifies the proof and the aggregation mask, and finally performs the aggregation result verification to check whether the following equations 1) and 2) hold; If equations 1) and 2) are both true, it means that the user gets the correct aggregation result: Otherwise, the verification fails and the run is aborted.
2. The verifiably secure federated learning method for unreliable users according to claim 1, characterized in that: In step (S200), the weight w i Obtained using the truth discovery algorithm.
3. The verifiably secure federated learning method for unreliable users according to claim 2, characterized in that: In step (S200), the user u i The weight w i Calculated by the following formula: in, is the coefficient used to scale the user weight, where |M| is , χ is the chi-square distribution, and α is the corresponding significance level.
4. The verifiably secure federated learning method for unreliable users according to claim 3, characterized in that: Said is the Euclidean distance function, used to calculate user u i Local gradient in round T and the global model x at round T-1 T-1 The Euclidean distance between 0 It is provided by an authoritative organization; Euclidean distance function 5. The verifiably secure federated learning method for unreliable users according to claim 4, characterized in that: The gradient parameter x 0 Calculated from public auxiliary databases.
6. The verifiably secure federated learning method for unreliable users according to claim 1, characterized in that: In step (S200), each user has a corresponding random number β i , will be shared in advance Random number beta i It is the address value corresponding to each user in secret sharing; Secret Splitting Algorithm Include: The secret s is divided into n shares. Only when at least t shares are obtained can the secret be obtained. In the (t, n) threshold scheme proposed by Shamir, the secret in is a finite field, 0<t≤n<q; Given a secret s, a threshold t, and a set of all users U = {u1, ..., u n };from Randomly select t-1 positive integers from the equation, and let the polynomial f(x) = s + a1x + a2x + ... + a t-1 x t-1 mod q, use f(x) to calculate f(β i ); the share of secret s is calculated as 7. The verifiably secure federated learning method for unreliable users according to claim 6, characterized in that: In step (S400), the secret reconstruction algorithm Include: Given any t (β i ,f(β i )), the secret s can be reconstructed by Lagrange interpolation method, and the secret can be efficiently recovered by the following formula: Among them, j represents the index of the sum, which is iterated from 0 to t and is used to traverse β j And the function f(β j ) value; m represents the index of the product term, iterating from 0 to t, excluding j, for traversing except x j All x m Value to calculate Lagrange interpolation weight; x m represents the horizontal coordinate of the mth point used to calculate the Lagrange interpolation, that is, a point of the interpolation node; x j Represents the abscissa of the jth term in the sum, i.e., a point of the interpolation node currently being processed.
8. The verifiably secure federated learning method for unreliable users according to claim 1, characterized in that: In step (S200), the symmetric encryption technology consists of the following two algorithms: Symmetric encryption algorithm AE.Enc, given a key and plaintext as input, will encrypt into ciphertext; Symmetric decryption algorithm AE.Dec, given a key and ciphertext as input, will decrypt the plaintext; For any given key h and any plaintext message y, the symmetric encryption algorithm can always ensure that the equation AE.Dec(h,AE.Enc(h,y))=y holds.