Safe and efficient federated learning method for resisting enhanced gradient attack

Abstract

The invention relates to the technical field of federated learning, enhanced gradient attack resistance and privacy protection, in particular to a safe and efficient federated learning method for resisting enhanced gradient attack. According to the method, on the premise that privacy of participants is not leaked, combined defense is carried out through constraint of the component upper bound and the L2 norm of the integer update vectors of the participants, and enhanced gradient poisoning attacks of malicious participants can be efficiently eliminated. According to the method, Boolean sharing and Diffie-Hellman key exchange protocols are combined, the privacy of model update vectors of participants is protected, meanwhile, the privacy protection operation of the participants is also lightweight Boolean operation, expensive encryption operation is not needed, and the privacy protection operation does not further increase the communication overhead of the participants. The invention provides a method for using an alternative casual transmission protocol, Boolean sharing owned by a server is converted into corresponding arithmetic sharing, and the server uses the arithmetic sharing to aggregate a federal learning global model.

Description

Technical Field

[0001] The present invention relates to the technical fields of federated learning, anti-enhanced gradient attack and privacy protection, and in particular to a safe and efficient federated learning method that is anti-enhanced gradient attack. Background Art

[0002] Federated learning is an emerging privacy-preserving machine learning approach that can be trained efficiently on large datasets. In a typical federated learning process, developers run a server to maintain a global machine learning model, which iteratively updates the model by computing a collection of gradient updates sent by participants after local training.

[0003] In a federated learning system without malicious users, federated learning allows participants to retain full ownership of their data, and application developers can effectively train high-quality ML models. However, in actual federated learning systems, some participants may maliciously manipulate updates to negatively affect the quality of the global model, i.e., poisoning attacks. Due to the large number of participants in cross-device federated learning, it is difficult to verify whether they are malicious one by one, which makes it easy for malicious participants to launch a series of attacks on the system.

[0004] Over the years, a large body of work has identified two desirable security properties: (1) Privacy of individual gradients. It is well known that the gradients submitted by participants leak information about their local datasets. Therefore, to be usable in privacy-sensitive applications, FL must protect the privacy of individual gradients and, therefore, the privacy of the dataset. (2) Filtering out augmented gradients from malicious participants. Aggregation is at the heart of FL, and it is very sensitive to disproportionate values. Without any defenses, even a single malformed gradient (e.g., with a very high norm) can arbitrarily bias the global model. Malicious participants can boost their gradients (expand them to a large norm) to corrupt the global model, and many model poisoning attacks rely on this strategy. An effective FL solution must defend against augmented gradients. Recent prior work has shown that filtering gradients based on their L2 norm is effective against a large number of sophisticated poisoning attacks under realistic threat models for production FL.

[0005] Most existing studies either provide privacy or defend against malformed gradients. Since then, some attempts have been made to achieve both properties at the same time, but they are either very inefficient or rely on weak threat models. These protocols can be divided into two categories, one is deployed on a single aggregation server for gradient aggregation, and the other is deployed on two servers. When the global model aggregation operation of federated learning is deployed on two servers, it can effectively handle the problem of participants being offline. Summary of the invention

[0006] The purpose of the present invention is to solve the above problems. A secure and efficient federated learning method that is resistant to enhanced gradient attacks is proposed. Boolean sharing and the Diffie-Hellman key exchange protocol are used to protect the privacy of the participant model update vector. Arithmetic sharing and the Diffie-Hellman key exchange protocol are used to allow the participants to generate an arithmetic share of a square-related group between two servers. The two servers implement the conversion of the integer update vector from Boolean to arithmetic sharing, as well as the verification of the square-related group. The L2 norm of the participant's integer update vector is calculated. Combined with the upper bound of the components of the integer update vector and the upper bound of its L2 norm, the enhanced gradient submitted by malicious participants can be effectively eliminated, and the retained arithmetic sharing is aggregated and the global model parameters are updated.

[0007] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions:

[0008] A secure and efficient federated learning method that is resistant to enhanced gradient attacks, comprising the following steps:

[0009] S10, system platform construction phase: The system administrator SM builds the system platform, generates public system parameters, and publicly publishes the task list of federated learning; each participant, server S1 and S2 uses the public parameters to calculate their corresponding pair of public and private keys; participants can apply to participate in the training of some federated learning tasks on this platform;

[0010] S20, participant encoding model parameter stage: Each participant uses the local data set to train the local model parameters, uses the global model parameters of the previous round to calculate the local model update vector, uses the update vector to calculate the corresponding symbol vector, and encodes the real update vector into the corresponding integer update vector. The participant then combines his own symbol vector and integer update vector into a binary (σ i ,ω i );

[0011] S30, protecting the privacy of local model parameters: Each participant obtains the public keys of servers S1 and S2, uses its own private key to calculate two corresponding session keys, inputs these two session keys into the pseudo-random number generator to generate two random numbers, and performs an XOR operation on these two random numbers and the integer update vector to obtain the blinded update vector ω i * , and then the symbol vector and the blind update vector form a new two-tuple (σ i ,ω i * );

[0012] S40, generating arithmetic shares of square related groups: Each participant uses two session keys to generate two arithmetic shares of square related groups, and then i ,ω i * ) and the blinded squared correlation group c i * To form a triple (σ i ,ω i * ,c i * ) is sent to server S1;

[0013] S50, servers S1 and S2 calculate the Boolean sharing phase: Server S1 calculates the triplet (σ i ,ω i * ,c i * ) for verification. If the verification is successful, the symbol vector σ of this participant is i Send it to server S2, otherwise discard the triplet of this participant; server S1 uses its own private key and the public key of the legitimate participant to calculate the session key, input this session private key into the pseudo-random generator, output a random number, and compare this random number with the blind update vector ω i * Perform an XOR operation to obtain the Boolean shared vector ω of this participant i '; Server S2 uses its own private key and the public key of the legitimate participant to calculate the session key, inputs the session private key into the pseudo-random generator, outputs a random number, and obtains a Boolean shared value s for this participant i,2 ;

[0014] S60, verification of square correlation group stage: servers S1 and S2 use the session key and pseudo-random number generator agreed with the participants to calculate the arithmetic share of the square correlation group of the participants, and use the second half of the square correlation group to verify the correctness of the first half of the square correlation group, eliminate the Boolean share of malicious participants, and discard the κ+3 high-order bits of the arithmetic share of the qualified square correlation group, and retain its low-order u bits;

[0015] S70, conversion phase from Boolean sharing to arithmetic sharing: servers S1 and S2 use the two-choice oblivious transfer protocol to calculate a specified number of double verification bits according to the Boolean sharing number of each participant; servers S1 and S2 use the double verification bits to convert their respective Boolean sharing values ​​into corresponding arithmetic sharing values;

[0016] S80, stage of removing enhanced gradients: servers S1 and S2 use the arithmetic sharing of the squared related group of participants and the arithmetic sharing of the integer update vector to calculate the arithmetic sharing of the squared L2 norm of the update vector, and then calculate the squared L2 norm thereof. If the squared L2 norm exceeds the specified upper bound, the arithmetic sharing of the participant is deleted;

[0017] S90, global model parameter calculation stage: servers S1 and S2 combine the symbol vectors of the undeleted participants into a symbol matrix, combine the arithmetic shares of the undeleted participants into an arithmetic share matrix, and use the arithmetic share matrix and the symbol matrix to calculate their respective aggregation vectors; server S2 sends its own aggregation vector to server S1; server S1 uses S2's aggregation vector to calculate the global model parameters;

[0018] S100, local model update phase: Participants repeatedly execute steps S20 to S90 until the number of training rounds reaches the specified number, and the system administrator and all participants obtain the final model parameters.

[0019] Furthermore, the step S10 includes:

[0020] S101. System administrator SM builds a platform, selects a statistical security parameter κ, a large prime number p and a random number g∈Z p * , Z p * ={1,2,…,p-1}; SM sets the initial model parameter w0 of d dimensions, selects a positive integer scaling factor λ, a positive integer L and two pseudo-random number generators PRG1(·) and PRG2(·), and sets a positive integer u such that 2 u ≥p, construct a ring of L2 norm Set the L2 norm upper bound μ, publish the task list of federated learning, set the maximum number of rounds T for global model training, and participants and servers need to register on the platform;

[0021] S102, let U i represents the i-th participant, participant U i Apply to participate in a federated learning task and select a random number r i ∈Z p * , as its private key sk i =r i And keep it secret, calculate the public key Form a pair of public and private keys (pk i ,sk i ); Server S1 selects a random number r1'∈Z p* , as its private key ssk1 = r1' and keep it secret, calculate the public key A pair of public and private keys (spk1, ssk1) is formed; server S2 selects a random number r2'∈Z p * , as its private key ssk2 = r2' and keep it secret, calculate the public key Form a pair of public and private keys (spk2, ssk2); the server and participants use their public and private key pairs to build an authenticated channel for data transmission between them.

[0022] Further, step S20 includes:

[0023] S201. Assume that participant U i Perform the t-th round of model training (1≤t≤T). When t=1, U i Get the initial global model parameters w from the system manager SM t-1 =w0, when t>1, U i Get the global model parameter w of round t-1 from server S1 t-1 , use the local data set to train the local model parameters w t,i , calculate the local model update vector

[0024] S202, Order is a vector The jth component (1≤j≤d) of , let the sign function sgn(x)∈Z2, Z2={0,1}, when x≥0, sgn(x)=0, when x<0, sgn(x)=1, that is, when x is non-negative, sgn(x) outputs bit 0, when x is negative, sgn(x) outputs bit 1;

[0025] S203, Participant U i Using the sign function sgn(x), we can Calculate the corresponding symbol value for each component in Get vector The symbol vector σ i =(σ i,1 ,σ i,2 ,···,σ i,d );

[0026] S204, Participant U i Use a positive integer scaling factor λ to convert the real number Round to integer ω i,j , so that the integer ω i,j ≤2 L -1, integer ω i,jEncoded as an Lbit binary number, the integer update vector ω is obtained i =(ω i,1 ,ω i,2 ,···,ω i,d ), forming a tuple (σ i ,ω i ). Rounding is based on If the decimal part is greater than or equal to 0.5, it is rounded up; if the decimal part is less than 0.5, it is rounded down.

[0027] Further, step S30 includes:

[0028] S301, Participant U i Use your own private key sk i , server S1's public key spk1 and S2's public key spk2, calculate the random seed r i,1 and r i,2 Input the pseudo-random number generator PRG1(·) to get two binary random numbers s of length L. i,1 =PRG1(r i,1 ) and s i,2 =PRG1(r i,2 );

[0029] S302, in order to blind the vector ω i Each component ω i,j (1≤j≤d), participant U i calculate in is the XOR operator symbol, forming a blinded update vector ω i * =(ω i,1 * ,ω i,2 * ,···,ω i,d * ), we get a binary group (σ i ,ω i * ).

[0030] Further, step S40 includes:

[0031] S401, Participant U i calculate For the d-dimensional update vector ω i , the random seed r i,1 Input to the pseudo-random number generator PRG2 (r i,1 ), generate 2d random numbers Ui The random seed r i,2 Input to the pseudo-random number generator PRG2 (r i,2 ), generate 2d random numbers

[0032] S402, Participant U i calculate At this time, the square correlation group is (a i,j (1) +a i,j (2) ,c i,j ), calculate the blinded value of the square correlation group The blinding vector c that constitutes the square correlation group i * =(c i,1 * ,c i,2 * ,···,c i,2d * ); Participant U i The privacy-protected binary (σ i ,ω i * ) and vector c i * To form a triple (σ i ,ω i * ,c i * ) is sent to server S1.

[0033] Further, step S50 includes:

[0034] S501, server S1 receives the triple (σ i ,ω i * ,c i * ), verification vector ω i * Each component ω i,j * Is the binary length of L? If each component ω i,j * The binary length is equal to L, and server S1 puts the symbol vector σ i Send to server S2, otherwise discard the triplet (σ i ,ω i * ,c i * ), which ensures that the participant U i The update vector ω iEach component ω i,j ≤2 L -1;

[0035] S502, server S1 uses its private key ssk1 and participant U i The public key pk i , calculate the random seed r i,1 Input the pseudo-random number generator PRG1(·) to generate a binary random number s with a length of L bits. i,1 =PRG1(r i,1 ), for the blinded vector ω i * Each component ω i,j * (1≤j≤d), calculate Get the vector ω i The corresponding Boolean shared vector ω i '=(ω i,1 ',ω i,2 ',···,ω i,d ');

[0036] S503: Server S2 receives symbol vector σ from server S1 i , using its private key ssk2 and participant U i The public key pk i , calculate the random seed r i,2 Input the pseudo-random number generator PRG1(·) to generate a binary random number s with a length of L bits. i,2 =PRG1(r i,2 ), we get the vector ω i Each component ω i,j Corresponding to a Boolean shared value s i,2 .

[0037] Further, step S60 includes:

[0038] S601, server S1 sends random seed r i,1 Input to the pseudo-random number generator PRG2 (r i,1 ), generate 2d random numbers calculate Arithmetic sharing of the square correlation group

[0039] S602, server S2 sends the random seed r i,2 Input to the pseudo-random number generator PRG2 (r i,2 ), generate 2d random numbers The arithmetic share of the square correlation group (a i,j (2) ,a i,j (2) ), 1≤j≤2d;

[0040] S603, server S1 selects a random odd number For 1≤j≤d, calculate Get the d-dimensional vector h i (1) =(h i,1 (1) ,h i,2 (1) ,···,h i,d (1) ), the odd number h and the vector h i (1) Send to server S2;

[0041] S604, for 1≤j≤d, server S2 calculates Get the d-dimensional vector h i (2) =(h i,1 (2) ,h i,2 (2) ,···,h i,d (2) ), the vector h i (2) Send to server S1;

[0042] S605: Server S1 receives vector h i (2) , for 1≤j≤d, calculate Form a vector Δ i =(Δ i,1 ,Δ i,2 ,···,Δ i,d ), sending vector Δ i To server S2;

[0043] S606: Server S2 receives vector h i (1) , for 1≤j≤d, calculate Δ i,j '=h 2 a i,j (2) -a i,(j+d) (2) –2h·h i,j a i,j (2) , forming a vector Δ i'=(Δ i,1 ',Δ i,2 ',···,Δ i,d '), send vector Δ i 'To server S1;

[0044] S607, servers S1 and S2 calculate Δ i,j * =Δ i,j +Δ i,j ', for 1≤j≤d, if there exists Δ i,j * Not 0, indicating that participant U i A square correlation group that does not meet the requirements is generated, namely If the square correlation group meets the requirements, then Let a i,j (1) +a i,j (2) =a i,j , a i,(j+d) (1) +a i,(j+d) (2) =a i,(j+d) ,So

[0045] will hold; then S1 discards participant U i The Boolean shared vector ω i ', S2 discards participant U i The Boolean shared value s i,2 ;

[0046] S608, due to Arithmetic sharing of squared correlation groups The binary form of bits, server S1 discards the arithmetic share of the square-related groups The kappa + 3 high-order bits of the u-bit square correlation group are obtained. Right now Server S2 performs a similar operation, discarding the arithmetic share of the square-related group (a i,j (2) ,a i,j (2) ) of the κ+3 high-order bits, we get the arithmetic sharing of the square correlation group of u bits (a i,j (2) ,a i,j (2) ),Right now

[0047] Further, step S70 includes:

[0048] S701, server S1 selects a random bit b1∈{0,1}, two random numbers z,z'∈Z p * , and then send the random number z' to S2;

[0049] S702, server S2 selects a pair of random public and private keys (pk, sk), calculates pk'=z'-pk, selects a random bit b2∈{0,1}, and converts a public key pair (pk0 * ,pk1 * ) is sent to server S1. The value of the public key pair has two cases: when b2 = 0, pk0 * =pk,pk1 * =pk'; when b2 = 1, pk0 * =pk',pk1 * =pk;

[0050] S703, the server S1 receives the public key pair (pk0 * ,pk1 * ), using the RSA public key encryption algorithm [1] , Calculate the ciphertext ε0=Enc(z,pk0 * ) and ε1=Enc(z+b1,pk1 * ), send the ciphertext (ε0,ε1) to S2;

[0051] S704. Server S2 receives the ciphertext (ε0, ε1) and decrypts the ciphertext using the RSA public key encryption algorithm. If b2 = 0, sk is used to decrypt ε0:

[0052] Dec(ε0,sk)=Dec(Enc(z,pk0 * ),sk)=Dec(Enc(z,pk),sk)=z,

[0053] Otherwise, decrypt ε1 with sk:

[0054] Dec(ε1,sk)=Dec(Enc(z+b1,pk1 * ),sk)=Dec(Enc(z+b1,pk),sk)=z+b1,

[0055] S2 can only obtain one of z and z+b1, and the obtained value can be expressed as z+b1b2(modp), let a=z+b1b2(modp);

[0056] S705, server S1 calculates b1'=b1+2z(modp), realizes the conversion of Boolean share b1 to arithmetic share b1', and forms a pair of double verification bits (b1, b1'); server S2 calculates b2'=b2-2a(modp)=b2-2z-2b1b2(modp), realizes the conversion of Boolean share b2 to arithmetic share b2', and forms a pair of double verification bits (b2, b2'); it is explained here that: because Servers S1 and S2 execute steps S701 to S705 to convert Boolean shares b1 and b2 into their corresponding arithmetic shares b1' and b2';

[0057] S706, since steps S701 to S705 can convert a Boolean shared value b1 into a corresponding arithmetic shared value b1', the Boolean shared value ω i,j 'There are L bits, server S1 executes steps S701 to S705 L times, and converts L Boolean shared values ​​b 1,k Convert to the corresponding arithmetic shared value b 1,k ', get L pairs of double verification bits (b 1,k ,b 1,k ')(1≤k≤L); Server S2 performs a similar operation process, because the Boolean shared value s i,2 There are L bits, and steps S701 to S705 are executed L times to convert the L Boolean shared values ​​b 2,k Convert to the corresponding arithmetic shared value b 2,k ', get L pairs of double verification bits (b 2,k ,b 2,k ')(1≤k≤L);

[0058] S707, order ω i,j,k ' is the component ω i,j The kth binary digit of ', s i,2,k Yes i,2 The kth binary bit (1≤k≤L) of the server S1 uses a pair of double verification bits (b 1,k ,b 1,k ') and a Boolean shared bit ω i,j,k ',calculate Send v1 to S2; Server S2 uses a pair of double verification bits (b 2,k ,b 2,k ') and a Boolean shared bit s i,2,k ,calculate Send v2 to S1; S1 and S2 each calculate S1 calculates the arithmetic shared value of the kth bit (modp), S2 calculates the arithmetic shared value of the kth bit because It can be calculated

[0059] so Established, servers S1 and S2 thus realize the conversion of single-bit Boolean sharing to corresponding arithmetic sharing;

[0060] S708, server S1 uses L arithmetic shared values calculate Composition of participants i An arithmetic shared vector of Server S2 uses L arithmetic shared values calculate Composition of participants i An arithmetic shared vector of

[0061] Further, step S80 includes:

[0062] S801, server S1 uses U i The arithmetic shared set of square-related groups and update vector ω i Arithmetic shared vector calculate Form a vector e i (1) =(e i,1 (1) ,e i,2 (1) ,···,e i,d (1) ), and transform this vector e i (1) Send to server S2;

[0063] S802, server S2 uses U i The arithmetic shared set of square-related groups {(a i,j (2) ,a i,j (2) )(1≤j≤d)} and update vector ω i Arithmetic shared vector calculate Form a vector e i (2) =(e i,1 (2) ,e i,2 (2) ,···,ei,d (2) ), the vector e i (2) Send to server S1;

[0064] S803: Server S1 receives vector e i (2) , calculate e i,j =e i,j (1) +e i,j (2) mod 2 u (1≤j≤d), get the update vector ω i The arithmetic sharing of the L2 norm squared And l i (1) Sent to server S2; server S2 receives vector e i (1) , calculate e i,j =e i,j (1) +e i,j (2) mod 2 u (1≤j≤d), get the update vector ω i The arithmetic sharing of the L2 norm squared And l i (2) Send to server S1;

[0065] S804, known a i,j (1) +a i,j (2) =a i,j

[0066] Servers S1 and S2 calculate the participant U i Update vector ω i The square of the L2 norm Compare it with the square μ of the upper bound of the L2 norm 2 For comparison; if l i >μ 2 , represents the participant U i The update vector ω i If the L2 norm of i Submitted the enhanced gradient, then server S1 deletes participant U i Arithmetic shared vector Server S2 deletes participant U i Arithmetic shared vector

[0067] Further, step S90 includes:

[0068] S901, assuming that the participant U is not deleted i The number of participants is m, and server S1 will delete the undeleted participants U i Arithmetic shared vector of (1≤i≤m) Form an m-row d-column arithmetic sharing matrix W m×d ; Server S2 will not delete the participant U i Arithmetic shared vector of (1≤i≤m) Form an arithmetic shared matrix S with m rows and d columns m×d ;

[0069] S902, server S1 sends the symbol vector σ of m participants i =(σ i,1 ,σ i,2 ,···,σ i,d ), forming a symbol matrix F with m rows and d columns m×d , let F j Represents the symbol matrix F m×d The jth column of j Denotes the arithmetic sharing matrix W m×d The jth column of

[0070] S903. For j=1, 2, ···, d, when F j The i-th component σ i,j = 0 (1≤i≤m), server S1 takes W j The i-th component in the column calculate w 1,j + W j Specified in the column When F j The i-th component σ i,j =1 (1≤i≤m), server S1 takes W j The i-th component in the column calculate W j Specified in the column The sum of two vectors w1 + =(w 1,1 + ,w 1,2 + ,···,w 1,d + ) and w1 - =(w 1,1 - ,w1,2 - ,···,w 1,d - );

[0071] S904, server S2 sends the symbol vector σ of m participants i =(σ i,1 ,σ i,2 ,···,σ i,d ), forming a symbol matrix F with m rows and d columns m×d , let F j Represents the symbol matrix F m×d The jth column of S j Denotes the arithmetic sharing matrix S m×d The jth column of

[0072] S905. For j=1, 2, ···, d, when F j The i-th component σ i,j = 0 (1≤i≤m), server S2 takes S j The i-th component in the column calculate w 2,j + For S j Specified in the column When F j The i-th component σ i,j =1 (1≤i≤m), server S2 takes S j The i-th component in the column calculate w 2,j - For S j Specified in the column The sum of two vectors w2 + =(w 2,1 + ,w 2,2 + ,···,w 2,d + ) and w2 - =(w 2,1 - ,w 2,2 - ,···,w 2,d - ), and w2 + and w2 - Send to S1;

[0073] S906, server S1 is represented by vector w1 + ,w2 + ,w1- and w2 - For each component of and the scaling factor λ, calculate:

[0074] w j + =(w 1,j + +w 2,j + modp) / 10 λ ,

[0075] w j - =(w 1,j - +w 2,j - modp) / 10 λ ,

[0076] Get two vectors w + =(w1 + ,w2 + ,···,w d + ) and vector w - =(w1 - ,w1 - ,···,w d - );

[0077] S907, server S1 calculates the global model parameter w of this round t =w t-1 +(w + -w - ) / m.

[0078] Further, step S100 includes:

[0079] S1001. When the number of model training rounds t≤T, participant U i Repeat the above steps S20 to S90. Otherwise, the participants end the federated learning task. After the federated learning task is completed, the system administrator and all participants obtain the final model parameter w T .

[0080] References: [1] Katz J, Lindell Y. Introduction to modern cryptography: principles and protocols [M]. Chapman and hall / CRC, 2007.

[0081] Table 1 shows the symbols in the present invention

[0082]

[0083]

[0084]

[0085] The secure and efficient federated learning method for resisting enhanced gradient attacks described in the present invention adopts the above technical solution and has the following technical effects compared with the prior art:

[0086] (1) The present invention provides a secure and efficient federated learning method that is resistant to enhanced gradient attacks. Without leaking the privacy of participants, the method performs joint defense by constraining the upper bounds of the components of the participants' integer update vectors and the L2 norm, and can effectively eliminate enhanced gradient poisoning attacks by malicious participants.

[0087] (2) The present invention provides a secure and efficient federated learning method that is resistant to enhanced gradient attacks, which combines Boolean sharing and the Diffie-Hellman key exchange protocol to protect the privacy of the participants' model update vectors. At the same time, the privacy protection operations performed by the participants are also lightweight Boolean operations that do not require expensive encryption operations, and the privacy protection operations do not further increase the communication overhead of the participants.

[0088] (3) The present invention provides a secure and efficient federated learning method that is resistant to enhanced gradient attacks, and proposes a method using a two-choice oblivious transfer protocol to convert the Boolean shares owned by the server into their corresponding arithmetic shares, and the server then uses the arithmetic shares to aggregate the global model of federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 A flowchart of a secure and efficient federated learning method that is resistant to enhanced gradient attacks provided by the present invention.

[0090] Figure 2 A system structure diagram of a secure and efficient federated learning method that is resistant to enhanced gradient attacks provided by the present invention. DETAILED DESCRIPTION

[0091] The present invention is further explained below in detail with reference to the accompanying drawings so that those skilled in the art can more deeply understand the present invention and be able to implement it. However, the following reference examples are only used to explain the present invention and are not intended to limit the present invention.

[0092] Example 1

[0093] See also Figure 1 to Figure 2 The present invention provides a technical solution, a secure and efficient federated learning method that resists enhanced gradient attacks, such as Figure 1As shown, it includes: building a system platform, participants encoding model parameters, protecting the privacy of local model parameters, generating arithmetic shares of square-related groups, server computing Boolean shares, verifying square-related groups, converting Boolean shares to arithmetic shares, removing enhanced gradients, calculating global model parameters, and updating local models. Figure 2 As shown in the figure, the system model of a secure and efficient federated learning method that resists enhanced gradient attacks consists of four entities: system administrator SM, participants U i , Server S1 and Server S2. The system administrator SM is a trusted entity and will correctly perform the system initialization steps. i Most of them are semi-honest, and there are also some malicious participants. Semi-honest participants will correctly perform local model training, encoding, privacy protection, and generate arithmetic sharing of square-related groups, but will infer other private information from the protocol. Malicious participants will not correctly execute the above protocol steps, specifically generating wrong square-related groups and poisoning attacks on enhanced gradients. Servers S1 and S2 are semi-honest, and will correctly verify the triples sent by the participants, verify the square-related groups, complete the conversion from Boolean to arithmetic sharing, eliminate the enhanced gradients, and finally aggregate the retained arithmetic sharing, calculate and update the global model parameters, but semi-honest servers will try to infer the private information of the participants from the protocol. As long as this method ensures that any server and other entities do not collude, the private information of the participants will not be exposed.

[0094] This embodiment provides a secure and efficient federated learning method that is resistant to enhanced gradient attacks, which specifically includes the following steps:

[0095] S10, system platform construction phase: The system administrator SM builds the system platform, generates public system parameters, and publicly publishes the task list of federated learning; each participant, server S1 and S2 uses the public parameters to calculate their corresponding pair of public and private keys; participants can apply to participate in the training of some federated learning tasks on this platform;

[0096] S20, participant encoding model parameter stage: Each participant uses the local data set to train the local model parameters, uses the global model parameters of the previous round to calculate the local model update vector, uses the update vector to calculate the corresponding symbol vector, and encodes the real update vector into the corresponding integer update vector. The participant then combines his own symbol vector and integer update vector into a binary (σ i ,ω i );

[0097] S30, protecting the privacy of local model parameters: Each participant obtains the public keys of servers S1 and S2, uses its own private key to calculate two corresponding session keys, inputs these two session keys into the pseudo-random number generator to generate two random numbers, and performs an XOR operation on these two random numbers and the integer update vector to obtain the blinded update vector ω i * , and then the symbol vector and the blind update vector form a new two-tuple (σ i ,ω i * );

[0098] S40, generating arithmetic shares of square related groups: Each participant uses two session keys to generate two arithmetic shares of square related groups, and then i ,ω i * ) and the blinded squared correlation group c i * To form a triple (σ i ,ω i * ,c i * ) is sent to server S1;

[0099] S50, servers S1 and S2 calculate the Boolean sharing phase: Server S1 calculates the triplet (σ i ,ω i * ,c i * ) for verification. If the verification is successful, the symbol vector σ of this participant is i Send it to server S2, otherwise discard the triplet of this participant; server S1 uses its own private key and the public key of the legitimate participant to calculate the session key, input this session private key into the pseudo-random generator, output a random number, and compare this random number with the blind update vector ω i * Perform an XOR operation to obtain the Boolean shared vector ω of this participant i '; Server S2 uses its own private key and the public key of the legitimate participant to calculate the session key, inputs the session private key into the pseudo-random generator, outputs a random number, and obtains a Boolean shared value s for this participant i,2 ;

[0100] S60, verification of square correlation group stage: servers S1 and S2 use the session key and pseudo-random number generator agreed with the participants to calculate the arithmetic share of the square correlation group of the participants, and use the second half of the square correlation group to verify the correctness of the first half of the square correlation group, eliminate the Boolean share of malicious participants, and discard the κ+3 high-order bits of the arithmetic share of the qualified square correlation group, and retain its low-order u bits;

[0101] S70, conversion phase from Boolean sharing to arithmetic sharing: servers S1 and S2 use the two-choice oblivious transfer protocol to calculate a specified number of double verification bits according to the Boolean sharing number of each participant; servers S1 and S2 use the double verification bits to convert their respective Boolean sharing values ​​into corresponding arithmetic sharing values;

[0102] S80, stage of removing enhanced gradients: servers S1 and S2 use the arithmetic sharing of the squared related group of participants and the arithmetic sharing of the integer update vector to calculate the arithmetic sharing of the squared L2 norm of the update vector, and then calculate the squared L2 norm thereof. If the squared L2 norm exceeds the specified upper bound, the arithmetic sharing of the participant is deleted;

[0103] S90, global model parameter calculation stage: servers S1 and S2 combine the symbol vectors of the undeleted participants into a symbol matrix, combine the arithmetic shares of the undeleted participants into an arithmetic share matrix, and use the arithmetic share matrix and the symbol matrix to calculate their respective aggregation vectors; server S2 sends its own aggregation vector to server S1; server S1 uses S2's aggregation vector to calculate the global model parameters;

[0104] S100, local model update phase: Participants repeatedly execute steps S20 to S90 until the number of training rounds reaches the specified number, and the system administrator and all participants obtain the final model parameters.

[0105] The step S10 specifically includes the following steps:

[0106] S101. System administrator SM builds a platform, selects a statistical security parameter κ, a large prime number p and a random number g∈Z p * , Z p * ={1,2,…,p-1}; SM sets the initial model parameter w0 of d dimensions, selects a positive integer scaling factor λ, a positive integer L and two pseudo-random number generators PRG1(·) and PRG2(·), and sets a positive integer u such that 2 u ≥p, construct a ring of L2 norm Set the L2 norm upper bound μ, publish the task list of federated learning, set the maximum number of rounds T for global model training, and participants and servers need to register on the platform;

[0107] S102, let U i represents the i-th participant, participant U i Apply to participate in a federated learning task and select a random number r i ∈Z p * , as its private key sk i =r i And keep it secret, calculate the public key Form a pair of public and private keys (pk i ,sk i ); Server S1 selects a random number r1'∈Z p * , as its private key ssk1 = r1' and keep it secret, calculate the public key A pair of public and private keys (spk1, ssk1) is formed; server S2 selects a random number r2'∈Z p * , as its private key ssk2 = r2' and keep it secret, calculate the public key Form a pair of public and private keys (spk2, ssk2); the server and participants use their public and private key pairs to build an authenticated channel for data transmission between them.

[0108] The step S20 specifically includes the following steps:

[0109] S201. Assume that participant U i Perform the t-th round of model training (1≤t≤T). When t=1, U i Get the initial global model parameters w from the system manager SM t-1 =w0, when t>1, U i Get the global model parameter w of round t-1 from server S1 t-1 , use the local data set to train the local model parameters w t,i , calculate the local model update vector

[0110] S202, Order is a vector The jth component (1≤j≤d) of , let the sign function sgn(x)∈Z2, Z2={0,1}, when x≥0, sgn(x)=0, when x<0, sgn(x)=1, that is, when x is non-negative, sgn(x) outputs bit 0, when x is negative, sgn(x) outputs bit 1;

[0111] S203, Participant U i Using the sign function sgn(x), we can Calculate the corresponding symbol value for each component in Get vector The symbol vector σ i =(σ i,1 ,σ i,2 ,···,σ i,d );

[0112] S204, Participant U i Use a positive integer scaling factor λ to convert the real number Round to integer ω i,j , so that the integer ω i,j ≤2 L -1, integer ω i,j Encoded as an Lbit binary number, the integer update vector ω is obtained i =(ω i,1 ,ω i,2 ,···,ω i,d ), forming a tuple (σ i ,ω i ). Rounding is based on If the decimal part is greater than or equal to 0.5, it is rounded up; if the decimal part is less than 0.5, it is rounded down.

[0113] The step S30 specifically includes the following steps:

[0114] S301, Participant U i Use your own private key sk i , server S1's public key spk1 and S2's public key spk2, calculate the random seed r i,1 and r i,2 Input the pseudo-random number generator PRG1(·) to get two binary random numbers s of length L. i,1 =PRG1(r i,1 ) and s i,2 =PRG1(r i,2 );

[0115] S302, in order to blind the vector ω i Each component ω i,j (1≤j≤d), participant U i calculate in is the XOR operator symbol, forming a blinded update vector ω i * =(ω i,1* ,ω i,2 * ,···,ω i,d * ), we get a binary group (σ i ,ω i * ).

[0116] The step S40 specifically includes the following steps:

[0117] S401, Participant U i calculate For the d-dimensional update vector ω i , the random seed r i,1 Input to the pseudo-random number generator PRG2 (r i,1 ), generate 2d random numbers U i The random seed r i,2 Input to the pseudo-random number generator PRG2 (r i,2 ), generate 2d random numbers

[0118] S402, Participant U i calculate At this time, the square correlation group is (a i,j (1) +a i,j (2) ,c i,j ), calculate the blinded value of the square correlation group The blinding vector c that constitutes the square correlation group i * =(c i,1 * ,c i,2 * ,···,c i,2d * ); Participant U i The privacy-protected binary (σ i ,ω i * ) and vector c i * To form a triple (σ i ,ω i * ,c i * ) is sent to server S1.

[0119] The step S50 specifically includes the following steps:

[0120] S501, server S1 receives the triple (σ i ,ωi * ,c i * ), verification vector ω i * Each component ω i,j * Is the binary length of L? If each component ω i,j * The binary length is equal to L, and server S1 puts the symbol vector σ i Send to server S2, otherwise discard the triplet (σ i ,ω i * ,c i * ), which ensures that the participant U i The update vector ω i Each component ω i,j ≤2 L -1;

[0121] S502, server S1 uses its private key ssk1 and participant U i The public key pk i , calculate the random seed r i,1 Input the pseudo-random number generator PRG1(·) to generate a binary random number s with a length of L bits. i,1 =PRG1(r i,1 ), for the blinded vector ω i * Each component ω i,j * (1≤j≤d), calculate Get the vector ω i The corresponding Boolean shared vector ω i '=(ω i,1 ',ω i,2 ',···,ω i,d ');

[0122] S503: Server S2 receives symbol vector σ from server S1 i , using its private key ssk2 and participant U i The public key pk i , calculate the random seed r i,2 Input the pseudo-random number generator PRG1(·) to generate a binary random number s with a length of L bits. i,2 =PRG1(r i,2 ), we get the vector ω i Each component ω i,jCorresponding to a Boolean shared value s i,2 .

[0123] The step S60 specifically includes the following steps:

[0124] S601, server S1 sends random seed r i,1 Input to the pseudo-random number generator PRG2 (r i,1 ), generate 2d random numbers calculate Arithmetic sharing of the square correlation group 2d;

[0125] S602, server S2 sends the random seed r i,2 Input to the pseudo-random number generator PRG2 (r i,2 ), generate 2d random numbers The arithmetic share of the square correlation group (a i,j (2) ,a i,j (2) ), 1≤j≤2d;

[0126] S603, server S1 selects a random odd number For 1≤j≤d, calculate Get the d-dimensional vector h i (1) =(h i,1 (1) ,h i,2 (1) ,···,h i,d (1) ), the odd number h and the vector h i (1) Send to server S2;

[0127] S604, for 1≤j≤d, server S2 calculates Get the d-dimensional vector h i (2) =(h i,1 (2) ,h i,2 (2) ,···,h i,d (2) ), the vector h i (2) Send to server S1;

[0128] S605: Server S1 receives vector h i (2) , for 1≤j≤d, calculate Form a vector Δi =(Δ i,1 ,Δ i,2 ,···,Δ i,d ), sending vector Δ i To server S2;

[0129] S606: Server S2 receives vector h i (1) , for 1≤j≤d, calculate Δ i,j '=h 2 a i,j (2) -a i,(j+d) (2) –2h·h i,j a i,j (2) , forming a vector Δ i '=(Δ i,1 ',Δ i,2 ',···,Δ i,d '), send vector Δ i 'To server S1;

[0130] S607, servers S1 and S2 calculate Δ i,j * =Δ i,j +Δ i,j ', for 1≤j≤d, if there exists Δ i,j * Not 0, indicating that participant U i A square correlation group that does not meet the requirements is generated, namely If the square correlation group meets the requirements, then Let a i,j (1) +a i,j (2) =a i,j , a i,(j+d) (1) +a i,(j+d) (2) =a i,(j+d) ,So

[0131] will hold; then S1 discards participant U i The Boolean shared vector ω i ', S2 discards participant U i The Boolean shared value s i,2 ;

[0132] S608, due to Arithmetic sharing of squared correlation groups The binary form of bits, server S1 discards the arithmetic share of the square-related groups The kappa + 3 high-order bits of the u-bit square correlation group are obtained. Right now Server S2 performs a similar operation, discarding the arithmetic share of the square-related group (a i,j (2) ,a i,j (2) ) of the κ+3 high-order bits, we get the arithmetic sharing of the square correlation group of u bits (a i,j (2) ,a i,j (2) ),Right now

[0133] The step S70 specifically includes the following steps:

[0134] S701, server S1 selects a random bit b1∈{0,1}, two random numbers z,z'∈Z p * , and then send the random number z' to S2;

[0135] S702, server S2 selects a pair of random public and private keys (pk, sk), calculates pk'=z'-pk, selects a random bit b2∈{0,1}, and converts a public key pair (pk0 * ,pk1 * ) is sent to server S1. The value of the public key pair has two cases: when b2 = 0, pk0 * =pk,pk1 * =pk'; when b2 = 1, pk0 * =pk',pk1 * =pk;

[0136] S703, the server S1 receives the public key pair (pk0 * ,pk1 * ), using the RSA public key encryption algorithm, calculate the ciphertext ε0 = Enc(z,pk0 * ) and ε1=Enc(z+b1,pk1 * ), send the ciphertext (ε0,ε1) to S2;

[0137] S704. Server S2 receives the ciphertext (ε0, ε1) and decrypts the ciphertext using the RSA public key encryption algorithm. If b2 = 0, sk is used to decrypt ε0:

[0138] Dec(ε0,sk)=Dec(Enc(z,pk0 *),sk)=Dec(Enc(z,pk),sk)=z,

[0139] Otherwise, decrypt ε1 with sk:

[0140] Dec(ε1,sk)=Dec(Enc(z+b1,pk1 * ),sk)=Dec(Enc(z+b1,pk),sk)=z+b1,

[0141] S2 can only obtain one of z and z+b1, and the obtained value can be expressed as z+b1b2(modp), let a=z+b1b2(modp);

[0142] S705, server S1 calculates b1'=b1+2z(modp), realizes the conversion of Boolean share b1 to arithmetic share b1', and forms a pair of double verification bits (b1, b1'); server S2 calculates b2'=b2-2a(modp)=b2-2z-2b1b2(modp), realizes the conversion of Boolean share b2 to arithmetic share b2', and forms a pair of double verification bits (b2, b2'); it is explained here that: because Servers S1 and S2 execute steps S701 to S705 to convert Boolean shares b1 and b2 into their corresponding arithmetic shares b1' and b2';

[0143] S706, since steps S701 to S705 can convert a Boolean shared value b1 into a corresponding arithmetic shared value b1', the Boolean shared value ω i,j 'There are L bits, server S1 executes steps S701 to S705 L times, and converts L Boolean shared values ​​b 1,k Convert to the corresponding arithmetic shared value b 1,k ', get L pairs of double verification bits (b 1,k ,b 1,k ')(1≤k≤L); Server S2 performs a similar operation process, because the Boolean shared value s i,2 There are L bits, and steps S701 to S705 are executed L times to convert the L Boolean shared values ​​b 2,k Convert to the corresponding arithmetic shared value b 2,k ', get L pairs of double verification bits (b 2,k ,b 2,k ')(1≤k≤L);

[0144] S707, order ω i,j,k ' is the component ω i,j The kth binary digit of ', s i,2,k Yes i,2The kth binary bit (1≤k≤L) of the server S1 uses a pair of double verification bits (b 1,k ,b 1,k ') and a Boolean shared bit ω i,j,k ',calculate Send v1 to S2; Server S2 uses a pair of double verification bits (b 2,k ,b 2,k ') and a Boolean shared bit s i,2,k ,calculate Send v2 to S1; S1 and S2 each calculate S1 calculates the arithmetic shared value of the kth bit (modp), S2 calculates the arithmetic shared value of the kth bit because It can be calculated

[0145] so Established, servers S1 and S2 thus realize the conversion of single-bit Boolean sharing to corresponding arithmetic sharing;

[0146] S708, server S1 uses L arithmetic shared values calculate Composition of participants i An arithmetic shared vector of Server S2 uses L arithmetic shared values j,k (1≤k≤L), calculate Composition of participants i An arithmetic shared vector of

[0147] The step S80 specifically includes the following steps:

[0148] S801, server S1 uses U i The arithmetic shared set of square-related groups and update vector ω i Arithmetic shared vector calculate Form a vector e i (1) =(e i,1 (1) ,e i,2 (1) ,···,e i,d (1) ), and transform this vector e i (1)Send to server S2;

[0149] S802, server S2 uses U i The arithmetic shared set of square-related groups {(a i,j (2) ,a i,j (2) )(1≤j≤d)} and update vector ω i Arithmetic shared vector calculate Form a vector e i (2) =(e i,1 (2) ,e i,2 (2) ,···,e i,d (2) ), the vector e i (2) Send to server S1;

[0150] S803: Server S1 receives vector e i (2) , calculate e i,j =e i,j (1) +e i,j (2) mod 2 u (1≤j≤d), get the update vector ω i The arithmetic sharing of the L2 norm squared And l i (1) Sent to server S2; server S2 receives vector e i (1) , calculate e i,j =e i,j (1) +e i,j (2) mod 2 u (1≤j≤d), get the update vector ω i The arithmetic sharing of the L2 norm squared And l i (2) Send to server S1;

[0151] S804, known a i,j (1) +a i,j (2) =a i,j

[0152] Servers S1 and S2 calculate the participant U iUpdate vector ω i The square of the L2 norm Compare it with the square μ of the upper bound of the L2 norm 2 For comparison; if l i >μ 2 , represents the participant U i The update vector ω i If the L2 norm of i Submitted the enhanced gradient, then server S1 deletes participant U i Arithmetic shared vector Server S2 deletes participant U i Arithmetic shared vector

[0153] The step S90 specifically includes the following steps:

[0154] S901, assuming that the participant U is not deleted i The number of participants is m, and server S1 will delete the undeleted participants U i Arithmetic shared vector of (1≤i≤m) Form an m-row d-column arithmetic sharing matrix W m×d ; Server S2 will not delete the participant U i Arithmetic shared vector of (1≤i≤m) Form an arithmetic shared matrix S with m rows and d columns m×d ;

[0155] S902, server S1 sends the symbol vector σ of m participants i =(σ i,1 ,σ i,2 ,···,σ i,d ), forming a symbol matrix F with m rows and d columns m×d , let F j Represents the symbol matrix F m×d The jth column of j Denotes the arithmetic sharing matrix W m×d The jth column of

[0156] S903. For j=1, 2, ···, d, when F j The i-th component σ i,j = 0 (1≤i≤m), server S1 takes W j The i-th component in the column calculate w 1,j + W j Specified in the column When F jThe i-th component σ i,j =1 (1≤i≤m), server S1 takes W j The i-th component in the column calculate w 1,j - W j Specified in the column The sum of two vectors w1 + =(w 1,1 + ,w 1,2 + ,···,w 1,d + ) and w1 - =(w 1,1 - ,w 1,2 - ,···,w 1,d - );

[0157] S904, server S2 sends the symbol vector σ of m participants i =(σ i,1 ,σ i,2 ,···,σ i,d ), forming a symbol matrix F with m rows and d columns m×d , let F j Represents the symbol matrix F m×d The jth column of S j Denotes the arithmetic sharing matrix S m×d The jth column of

[0158] S905. For j=1, 2, ···, d, when F j The i-th component σ i,j = 0 (1≤i≤m), server S2 takes S j The i-th component in the column calculate w 2,j + For S j Specified in the column When F j The i-th component σ i,j =1 (1≤i≤m), server S2 takes S j The i-th component in the column calculate p,w 2,j - For S j Specified in the column The sum of two vectors w2 +=(w 2,1 + ,w 2,2 + ,···,w 2,d + ) and w2 - =(w 2,1 - ,w 2,2 - ,···,w 2,d - ), and w2 + and w2 - Send to S1;

[0159] S906, server S1 is represented by vector w1 + ,w2 + ,w1 - and w2 - For each component of and the scaling factor λ, calculate:

[0160] w j + =(w 1,j + +w 2,j + modp) / 10 λ ,

[0161] w j - =(w 1,j - +w 2,j - modp) / 10 λ ,

[0162] Get two vectors w + =(w1 + ,w2 + ,···,w d + ) and vector w - =(w1 - ,w1 - ,···,w d - );

[0163] S907, server S1 calculates the global model parameter w of this round t =w t-1 +(w + -w - ) / m.

[0164] The step S100 specifically includes the following steps:

[0165] S1001. When the number of model training rounds t≤T, participant U i Repeat the above steps S20 to S90. Otherwise, the participants end the federated learning task. After the federated learning task is completed, the system administrator and all participants obtain the final model parameter w T .

[0166] Some of the terms mentioned in this embodiment are explained below:

[0167] Authentication channel: Authentication channel can be realized through digital signature technology, which means using digital signature mechanism to ensure the integrity, authenticity and non-repudiation of data without relying on traditional encryption channel, and prevent data from being tampered or forged during communication. The main properties are as follows: Integrity: When the sender generates a message and signs it with a private key, the receiver can verify the signature with the sender's public key. If the message is tampered with during transmission, the signature verification will fail, and the receiver can detect that the integrity of the message has been compromised. Authenticity: Since only the sender holds the private key, the signature can prove that the message is indeed from the sender, preventing others from impersonating the sender to forge data. Non-repudiation: The sender uses the private key to generate the signature, which is unique and only held by the legitimate sender, which means that the sender cannot deny that he has sent the message. Even if the sender tries to deny it later, the signature and public key can still prove the source of the message.

[0168] L2 norm: also known as Euclidean norm, is a measure of a vector or matrix in Euclidean space, often used to measure the length of a vector or the size of a matrix element. It is usually expressed as the square root of the sum of the squares of the vector components. For vectors (x1, x2, ···, x n ), its L2 norm can be expressed as

[0169] RSA public key encryption algorithm: RSA algorithm is one of the most widely used public key encryption algorithms, mainly used for data encryption and digital signatures. It is based on the large integer decomposition problem in number theory. Its core idea is to achieve encryption and decryption key separation while maintaining security. The basic principles of the RSA algorithm are as follows: (1) Key generation: select two large prime numbers p and q, calculate their product n = p × q, let n be the modulus of RSA; calculate the Euler function Choose one with Let the mutually prime integer e be the public key exponent. Calculate the private key d so that (2) Encryption: The sender uses the receiver’s public key (n, e) to encrypt the plaintext m, and the ciphertext c = m e mod n. (3) Decryption: After the receiver receives the ciphertext c sent by the sender, it uses its own private key (n, d) to decrypt it into plaintext m = cd mod n.

[0170] Arithmetic sharing: Arithmetic sharing is to decompose a secret value into multiple shares through addition operations, and these shares are distributed to multiple participants. Each share is meaningless alone, and only by adding all the shares can the original secret value be restored. For example, the secret value x∈Z p * The arithmetic sharing is x1,x2,···,x n ∈Z p * , then x1+x2+···+x n modp=x.

[0171] Boolean sharing: Boolean sharing is to decompose a secret value into multiple fixed-length binary shares through Boolean operations, and distribute these fixed-length binary numbers to multiple participants. A single share cannot provide any information about the original secret, but all shares can restore the original secret through Boolean operations. The Boolean share of Then it satisfies where x and x1,x2,···,x n A binary number of l bit length.

[0172] Diffie-Hellman key exchange protocol: This protocol is a cryptographic protocol for secure communication that allows two communicating parties who have not shared a key in advance to generate a shared secret key over an insecure channel. This key can be used for subsequent symmetric encryption communications. The security of the Diffie-Hellman key exchange protocol is based on the computational difficulty of the discrete logarithm problem. The basic principle is as follows: (1) Generate public parameters. The two parties negotiate a large prime number p and a random number g∈Z p * , these two parameters are public. (2) In the key exchange process, one party randomly selects a random number r1∈Z p * , as its private key sk1, calculate the public key Send it to the other party. The other party randomly selects a random number r2∈Z p * , as its private key sk2, calculate the public key Send it to the other party. (3) Shared key generation: After receiving the other party’s public key, both parties use their own private keys to calculate the shared key. The other party calculates the shared key modp. At this time, the shared key r calculated by the two parties 1,2 =r 2,1 .

[0173] Square correlation group: When it is known that x∈Z p * and y∈Z p * When the secret share of x·y needs to be calculated, this secure multiplication usually uses the Beaver triple. The Beaver triple consists of three elements (α, β, γ), where α, β∈Z p * , γ=α·β, the secret share of the Beaver triple can be used to calculate the secret share of x·y. When β=α in the Beaver triple, γ=α 2 , then a square correlation group of the form (α,γ) is formed. For example, (3,9) is a square correlation group.

[0174] The specific implementation scheme described above further describes in detail the purpose, technical scheme and beneficial effects of the present invention. It should be understood that the above is only a specific implementation scheme of the present invention and is not intended to limit the scope of the present invention. Any equivalent changes and modifications made by any technician in the field without departing from the concept and principle of the present invention should fall within the scope of protection of the present invention.

Claims

1. A secure and efficient federated learning method that is resistant to enhanced gradient attacks, characterized in that: The following steps are involved: S10, system platform construction phase: The system administrator SM builds the system platform, generates system public parameters, and publicly publishes the task list of federated learning; each participant, server S1 and S2 use the public parameters to calculate their corresponding pair of public and private keys; participants can apply to participate in the training of some federated learning tasks on this platform; S20, participant encoding model parameter stage: Each participant uses the local data set to train the local model parameters, uses the global model parameters of the previous round to calculate the local model update vector, uses the update vector to calculate the corresponding symbol vector, and encodes the real update vector into the corresponding integer update vector. The participant then combines his own symbol vector and integer update vector into a binary (σ i ,ω i ); S30, protecting the privacy of local model parameters: Each participant obtains the public keys of servers S1 and S2, uses its own private key to calculate two corresponding session keys, inputs these two session keys into the pseudo-random number generator to generate two random numbers, and performs an XOR operation on these two random numbers and the integer update vector to obtain the blinded update vector ω i * , and then the symbol vector and the blind update vector form a new two-tuple (σ i ,ω i * ); S40, generating arithmetic shares of square related groups: Each participant uses two session keys to generate two arithmetic shares of square related groups, and then i ,ω i * ) and the blinded squared correlation group c i * To form a triple (σ i ,ω i * ,c i * ), sent to server S1; S50, servers S1 and S2 calculate the Boolean sharing phase: Server S1 calculates the triplet (σ i ,ω i * ,c i * ) for verification. If the verification is successful, the symbol vector σ of this participant is i Send it to server S2, otherwise discard the triplet of this participant; server S1 uses its own private key and the public key of the legitimate participant to calculate the session key, input this session private key into the pseudo-random generator, output a random number, and compare this random number with the blind update vector ω i * Perform an XOR operation to obtain the Boolean shared vector ω of this participant i '; Server S2 uses its own private key and the public key of the legitimate participant to calculate the session key, inputs the session private key into the pseudo-random generator, outputs a random number, and obtains a Boolean shared value s for this participant i,2 ; S60, verification of square correlation group stage: servers S1 and S2 use the session key and pseudo-random number generator agreed with the participants to calculate the arithmetic share of the square correlation group of the participants, and use the second half of the square correlation group to verify the correctness of the first half of the square correlation group, eliminate the Boolean share of malicious participants, and discard the κ+3 high-order bits of the arithmetic share of the qualified square correlation group, and retain its low-order u bits; S70, conversion phase from Boolean sharing to arithmetic sharing: servers S1 and S2 use the two-choice oblivious transfer protocol to calculate a specified number of double verification bits according to the Boolean sharing number of each participant; servers S1 and S2 use the double verification bits to convert their respective Boolean sharing values ​​into corresponding arithmetic sharing values; S80, stage of removing enhanced gradients: servers S1 and S2 use the arithmetic sharing of the squared related group of participants and the arithmetic sharing of the integer update vector to calculate the arithmetic sharing of the squared L2 norm of the update vector, and then calculate the squared L2 norm thereof. If the squared L2 norm exceeds the specified upper bound, the arithmetic sharing of the participant is deleted; S90, global model parameter calculation stage: servers S1 and S2 combine the symbol vectors of the undeleted participants into a symbol matrix, combine the arithmetic shares of the undeleted participants into an arithmetic share matrix, and use the arithmetic share matrix and the symbol matrix to calculate their respective aggregation vectors; server S2 sends its own aggregation vector to server S1; server S1 uses S2's aggregation vector to calculate the global model parameters; S100, local model update phase: Participants repeatedly execute steps S20 to S90 until the number of training rounds reaches the specified number, and the system administrator and all participants obtain the final model parameters.

2. A secure and efficient federated learning method against enhanced gradient attacks according to claim 1, characterized in that: The step S10 comprises the following steps: S101. System administrator SM builds a platform, selects a statistical security parameter κ, a large prime number p and a random number g∈Z p * , Z p * ={1,2,…,p-1}; SM sets the initial model parameter w0 of d dimensions, selects a positive integer scaling factor λ, a positive integer L and two pseudo-random number generators PRG1(·) and PRG2(·), and sets a positive integer u such that 2 u ≥p, construct a ring of L2 norm Set the L2 norm upper bound μ, publish the task list of federated learning, set the maximum number of rounds T for global model training, and participants and servers need to register on the platform; S102, let U i represents the i-th participant, participant U i Apply to participate in a federated learning task and select a random number r i ∈Z p * , as its private key sk i =r i And keep it secret, calculate the public key Form a pair of public and private keys (pk i ,sk i ); Server S1 selects a random number r1'∈Z p * , as its private key ssk1 = r1' and keep it secret, calculate the public key A pair of public and private keys (spk1, ssk1) is formed; server S2 selects a random number r2'∈Z p * , as its private key ssk2 = r2' and keep it secret, calculate the public key Form a pair of public and private keys (spk2, ssk2); the server and participants use their public and private key pairs to build an authenticated channel for data transmission between them.

3. A secure and efficient federated learning method against enhanced gradient attacks according to claim 2, characterized in that: The step S20 comprises the following steps: S201. Assume that participant U i Perform the t-th round of model training, 1≤t≤T, when t=1, U i Get the initial global model parameters w from the system manager SM t-1 =w0, when t>1, U i Get the global model parameter w of round t-1 from server S1 t-1 , use the local data set to train the local model parameters w t,i , calculate the local model update vector S202, Order is a vector The jth component of , 1≤j≤d, let the sign function sgn(x)∈Z2, Z2={0,1}, when x≥0, sgn(x)=0, when x<0, sgn(x)=1, that is, when x is a non-negative number, sgn(x) outputs bit 0, when x is a negative number, sgn(x) outputs bit 1; S203, Participant U i Using the sign function sgn(x), we can Calculate the corresponding symbol value for each component in Get vector The symbol vector σ i =(σ i,1 ,σ i,2 ,···,σ i,d ); S204, Participant U i The real number Round to integer ω i,j , so that the integer ω i,j ≤2 L -1, integer ω i,j Encoded as an Lbit binary number, the integer update vector ω is obtained i =(ω i,1 ,ω i,2 ,···,ω i,d ), forming a tuple (σ i ,ω i ).

4. A secure and efficient federated learning method against enhanced gradient attacks according to claim 3, characterized in that: The step S30 comprises the following steps: S301, Participant U i Use your own private key sk i , server S1's public key spk1 and S2's public key spk2, calculate the random seed r i,1 and r i,2 Input the pseudo-random number generator PRG1(·) to get two binary random numbers s of length L. i,1 =PRG1(r i,1 ) and s i,2 =PRG1(r i,2 ); S302, in order to blind the vector ω i Each component ω i,j , 1≤j≤d, participant U i calculate Where ⊕ is the XOR operator symbol, forming a blinded update vector ω i * =(ω i,1 * ,ω i,2 * ,···,ω i,d * ), we get a binary group (σ i ,ω i * ).

5. A secure and efficient federated learning method against enhanced gradient attacks according to claim 4, characterized in that: The step S40 comprises the following steps: S401, Participant U i calculate For the d-dimensional update vector ω i , the random seed r i,1 Input to the pseudo-random number generator PRG2 (r i,1 ), generate 2d random numbers U i The random seed r i,2 Input to the pseudo-random number generator PRG2 (r i,2 ), generate 2d random numbers S402, Participant U i calculate Get the blind value c of the square correlation group i,j * =c i,j -a i, 1≤j≤2d, forming the blinding vector c of the square correlation group i * =(c i,1 * ,c i,2 * ,···,c i,2d * ); Participant U i The binary group (σ i ,ω i * ) and vector c i * To form a triple (σ i ,ω i * ,c i * ) is sent to server S1.

6. A secure and efficient federated learning method against enhanced gradient attacks according to claim 5, characterized in that: The step S50 comprises the following steps: S501, server S1 receives the triple (σ i ,ω i * ,c i * ), verification vector ω i * Each component ω i,j * Is the binary length of L? If each component ω i,j * The binary length is equal to L, and server S1 puts the symbol vector σ i Send to server S2, otherwise discard the triplet (σ i ,ω i * ,c i * ); S502, server S1 uses its private key ssk1 and participant U i The public key pk i , calculate the random seed r i,1 Input the pseudo-random number generator PRG1(·) to generate a binary random number s with a length of L bits. i,1 =PRG1(r i,1 ), for the blinded vector ω i * Each component ω i,j * , 1≤j≤d, calculate Get the vector ω i The corresponding Boolean shared vector ω i '=(ω i,1 ',ω i,2 ',···,ω i,d '); S503: Server S2 receives symbol vector σ from server S1 i , using its private key ssk2 and participant U i The public key pk i , calculate the random seed r i,2 Input the pseudo-random number generator PRG1(·) to generate a binary random number s with a length of L bits. i,2 =PRG1(r i,2 ), we get the vector ω i Each component ω i,j Corresponding to a Boolean shared value s i,2 .

7. A secure and efficient federated learning method against enhanced gradient attacks according to claim 6, characterized in that: The step S60 comprises the following steps: S601, server S1 sends random seed r i,1 Input to the pseudo-random number generator PRG2 (r i,1 ), generate 2d random numbers calculate Arithmetic sharing of the square correlation group S602, server S2 sends the random seed r i,2 Input to the pseudo-random number generator PRG2 (r i,2 ), generate 2d random numbers The arithmetic share of the square correlation group (a i,j (2) ,a i,j (2) ), 1≤j≤2d; S603, server S1 selects a random odd number For 1≤j≤d, calculate h i,j (1) =h·a i,j (1) -a i,(j+d) (1) mod Get the d-dimensional vector h i (1) =(h i,1 (1) ,h i,2 (1) ,···,h i,d (1) ), the odd number h and the vector h i (1) Send to server S2; S604, for 1≤j≤d, server S2 calculates Get the d-dimensional vector h i (2) =(h i,1 (2) ,h i,2 (2) ,···,h i,d (2) ), the vector h i (2) Send to server S1; S605: Server S1 receives vector h i (2) , for 1≤j≤d, calculate Form a vector Δ i =(Δ i,1 ,Δ i,2 ,···,Δ i,d ), sending vector Δ i To server S2; S606: Server S2 receives vector h i (1) , for 1≤j≤d, calculate Δ i,j '=h 2 a i,j (2) -a i,(j+d) (2) –2h·h i,j a i,j (2) , forming a vector Δ i '=(Δ i,1 ',Δ i,2 ',···,Δ i,d '), send vector Δ i 'To server S1; S607, for 1≤j≤d, servers S1 and S2 calculate Δ i,j * =Δ i,j +Δ i,j ', if Δ i,j * ≠0, then S1 discards participant U i The Boolean shared vector ω i ', S2 discards participant U i The Boolean shared value s i,2 ; S608, due to Arithmetic sharing of squared correlation groups The binary form has u+κ+3 bits, and server S1 discards the arithmetic share of the square-related group The kappa + 3 high-order bits of the u-bit square correlation group are obtained. Right now 1≤j≤d; Server S2 performs similar operations, discarding the arithmetic share of the square-related group (a i,j (2) ,a i,j (2) ) of the κ+3 high-order bits, we get the arithmetic sharing of the square correlation group of u bits (a i,j (2) ,a i,j (2) ),Right now 8. A secure and efficient federated learning method against enhanced gradient attacks according to claim 7, characterized in that: The step S70 includes the following steps: S701, server S1 selects a random bit b1∈{0,1}, two random numbers z,z'∈Z p * , and then send the random number z' to S2; S702, server S2 selects a pair of random public and private keys (pk, sk), calculates pk'=z'-pk, selects a random bit b2∈{0,1}, and converts a public key pair (pk0 * ,pk1 * ) is sent to server S1. The value of the public key pair has two cases: when b2 = 0, pk0 * =pk,pk1 * =pk'; when b2 = 1, pk0 * =pk',pk1 * =pk; S703, the server S1 receives the public key pair (pk0 * ,pk1 * ), using the RSA public key encryption algorithm, calculate the ciphertext ε0 = Enc(z,pk0 * ) and ε1=Enc(z+b1,pk1 * ), send the ciphertext (ε0,ε1) to S2; S704. Server S2 receives the ciphertext (ε0, ε1) and decrypts the ciphertext using the RSA public key encryption algorithm. If b2 = 0, sk is used to decrypt ε0: Dec(ε0,sk)=Dec(Enc(z,pk0 * ),sk)=Dec(Enc(z,pk),sk)=z, Otherwise, decrypt ε1 with sk: Dec(ε1,sk)=Dec(Enc(z+b1,pk1 * ),sk)=Dec(Enc(z+b1,pk),sk)=z+b1, S2 obtains only one value from z and z+b1, which can be expressed in the form of z+b1b2(modp), let a=z+b1b2(modp); S705. To convert the Boolean share b1 into the corresponding arithmetic share b1', the server S1 calculates b1'=b1+2z(modp) to form a pair of double verification bits (b1, b1'); to convert the Boolean share b2 into the corresponding arithmetic share b2', the server S2 calculates b2'=b2-2a(modp)=b2-2z-2b1b2(modp) to form a pair of double verification bits (b2, b2'); S706, since steps S701 to S705 can convert a Boolean shared value b1 into a corresponding arithmetic shared value b1', the Boolean shared value ω i,j 'There are L bits, server S1 executes steps S701 to S705 L times, and converts L Boolean shared values ​​b 1,k Convert to the corresponding arithmetic shared value b 1,k ', get L pairs of double verification bits (b 1,k ,b 1,k ')(1≤k≤L); Server S2 performs a similar operation process, because the Boolean shared value s i,2 There are L bits, and steps S701 to S705 are executed L times to convert the L Boolean shared values ​​b 2,k Convert into the corresponding arithmetic value share b 2,k ', get L pairs of double verification bits (b 2,k ,b 2,k '), 1≤k≤L; S707, order ω i,j,k ' is the component ω i,j The kth binary digit of ', s i,2,k is a binary number s i,2 The kth binary bit of , 1≤k≤L; Server S1 uses a pair of double verification bits (b 1,k ,b 1,k ') and a Boolean shared bit ω i,j,k ', calculate v1 = ω i,j,k '⊕b 1,k , v1 is sent to S2; server S2 uses a pair of double verification bits (b 2,k ,b 2,k ') and a Boolean shared bit s i,2,k , calculate v2 = s i,2,k ⊕b 2,k , v2 is sent to S1; S1 and S2 each calculate v = v1 ⊕ v2, and S1 calculates the arithmetic shared value of the kth bit -2vb 1,k '(modp), S2 calculates the arithmetic shared value of the kth bit S708, server S1 uses L arithmetic shared values 1≤k≤L, calculate Composition of participants i An arithmetic shared vector of Server S2 uses L arithmetic shared values j,k , 1≤k≤L, calculate Composition of participants i An arithmetic shared vector of 9. A secure and efficient federated learning method against enhanced gradient attacks according to claim 8, characterized in that: The step S80 includes the following steps: S801, server S1 uses U i The arithmetic shared set of square-related groups and update vector ω i Arithmetic shared vector calculate 1≤j≤d, forming a vector e i (1) =(e i,1 (1) ,e i,2 (1) ,···,e i,d (1) ), and transform this vector e i (1) Send to server S2; S802, server S2 uses U i The arithmetic shared set of square-related groups {(a i,j (2) ,a i,j (2) )(1≤j≤d)} and update vector ω i Arithmetic shared vector calculate 1≤j≤d, forming a vector e i (2) =(e i,1 (2) ,e i,2 (2) ,···,e i,d (2) ), the vector e i (2) Send to server S1; S803: Server S1 receives vector e i (2) , calculate e i,j =e i,j (1) +e i,j (2) mod 2 u , 1≤j≤d, calculate the arithmetic sharing of the L2 norm square And l i (1) Sent to server S2; server S2 receives vector e i (1) , calculate e i,j =e i,j (1) +e i,j (2) mod 2 u , 1≤j≤d, calculate the arithmetic sharing of the L2 norm square And l i (2) Send to server S1; S804, servers S1 and S2 calculate the participant U i Update vector ω i The square of the L2 norm l i = l i (1) +l i (2) mod2 u , if l i >μ 2 , then server S1 deletes participant U i Arithmetic shared vector Server S2 deletes participant U i Arithmetic shared vector 10. A secure and efficient federated learning method against enhanced gradient attacks according to claim 9, characterized in that: The step S90 includes the following steps: S901, assuming that the participant U is not deleted i The number of participants is m, and server S1 will delete the undeleted participants U i , 1≤i≤m, the arithmetic shared vector Form an m-row d-column arithmetic shared matrix W m×d ; Server S2 will not delete the participant U i , 1≤i≤m, the arithmetic shared vector Form an arithmetic shared matrix S with m rows and d columns m×d ; S902, server S1 sends the symbol vector σ of m participants i =(σ i,1 ,σ i,2 ,···,σ i,d ), forming a symbol matrix F with m rows and d columns m×d , let F j Represents the symbol matrix F m×d The jth column of j Denotes the arithmetic sharing matrix W m×d The jth column of S903. For j=1, 2, ···, d, when F j The i-th component σ i,j =0, 1≤i≤m, server S1 takes W j The i-th component in the column 1≤i≤m, calculate When F j The i-th component σ i,j =1, 1≤i≤m, server S1 takes W j The i-th component in the column 1≤i≤m, calculate Get two vectors w1 + =(w 1,1 + ,w 1,2 + ,···,w 1,d + ) and w1 - =(w 1,1 - ,w 1,2 - ,···,w 1,d - ); S904, server S2 sends the symbol vector σ of m participants i =(σ i,1 ,σ i,2 ,···,σ i,d ), forming a symbol matrix F with m rows and d columns m×d , let F j Represents the symbol matrix F m×d The jth column of S j Denotes the arithmetic sharing matrix S m×d The jth column of S905. For j=1, 2, ···, d, when F j The i-th component σ i,j =0, 1≤i≤m, server S2 takes S j The i-th component in the column 1≤i≤m, calculate When F j The i-th component σ i,j =1, 1≤i≤m, server S2 takes S j The i-th component in the column 1≤i≤m, calculate Get two vectors w2 + =(w 2,1 + ,w 2,2 + ,···,w 2,d + ) and w2 - =(w 2,1 - ,w 2,2 - ,···,w 2,d - ), and w2 + and w2 - Send to S1; S906, server S1 is represented by vector w1 + ,w2 + ,w1 - and w2 - For each component of and the scaling factor λ, calculate: In j + =(in 1,j + +in 2,j + modp) / 10 λ , w j - =(w 1,j - +w 2,j - mod p) / 10 λ , Get two vectors w + =(w1 + ,w2 + ,···,w d + ) and vector w - =(w1 - ,w1 - ,···,w d - ); S907, server S1 calculates the global model parameter w of this round t =w t-1 +(w + -w - ) / m; The step S100 includes the following steps: S1001, when the number of model training rounds t≤T, the participant U i Repeat the above steps S20 to S90. Otherwise, the participants end the federated learning task. After the federated learning task is completed, the system administrator and all participants obtain the final model parameter w T .

Images