Privacy protection federated learning security aggregation method and device, equipment and medium

Abstract

The invention discloses a privacy protection federated learning security aggregation method, apparatus and device, and a medium, a double-server federated learning architecture is constructed based on a double trap door homomorphic encryption technology, and it is ensured that the privacy of client-side related information is not disclosed in a poisoning attack detection and global model aggregation process. On the premise of strictly guaranteeing the data privacy of a client, the difference value between elements of the gradient is directly calculated in a ciphertext state, and the difference of the gradient in element dimensions is effectively captured, so that the accurate detection of the encryption model poisoning attack in a heterogeneous data scene is realized, and finally, the detection accuracy is improved. Through a dynamic threshold Byzantine robust aggregation scheme based on an average value, the technical problem of difficult poisoning gradient elimination caused by gradient threshold dynamic change in a heterogeneous data scene is solved.

Description

Technical Field

[0001] The present application relates to the field of federated learning technology, and in particular to a privacy-preserving federated learning security aggregation method, apparatus, device, and medium. Background Art

[0002] Privacy-preserving federated learning is a distributed machine learning paradigm that minimizes the risk of privacy breaches by leveraging distributed data for model training and combining cryptographic primitives for data encryption and secure aggregation. In a typical privacy-preserving federated learning scheme, the server performs secure aggregation based on encrypted local updates with strong privacy guarantees. With its significant advantages in privacy protection and resource efficiency, privacy-preserving federated learning has become a key machine learning paradigm in fields such as the Industrial Internet of Things.

[0003] However, privacy-preserving federated learning is inherently vulnerable to poisoning attacks launched by Byzantine attackers. Existing solutions typically employ methods such as cosine similarity, Euclidean distance, and gradient clipping to defend against poisoning attacks. However, these methods have limited applicability in scenarios with heterogeneous data. In heterogeneous data scenarios, there is a lack of clear distinction between benign and poisoned gradients. Furthermore, as the global model iterates, the threshold between gradients uploaded by malicious and benign clients changes dynamically, posing a significant challenge to the effective detection and defense against poisoned gradients. Summary of the Invention

[0004] This application provides a privacy-preserving federated learning security aggregation method, apparatus, equipment and medium. It constructs a dual-server federated learning architecture based on double-trapdoor homomorphic encryption technology. Under the premise of strictly protecting the privacy of client data, it effectively captures the differences in the gradient dimensions of each element by directly calculating the differences between the gradient elements in the ciphertext state, thereby realizing accurate detection of poisoning attacks on encryption models in heterogeneous data scenarios.

[0005] In view of this, the first aspect of the present application provides a privacy-preserving federated learning secure aggregation method, the method comprising:

[0006] S1. Generate a public-private key pair (pk, sk) based on security parameters, and split the private key into sk0, sk1, ...sk using a key splitting algorithm. n+1 n+2 copies;

[0007] S2, send the public key pk to each client and server, send the private key sk0 to server S0, send the private key sk to server S1 n+1 , send the private key sk to each client i ,i∈[1,n];

[0008] S3. Initialize the global model Server S0 uses the pure root dataset to evaluate the global model Perform training to obtain pure gradient G t , and then use the public key pk to convert the global model Encrypted to And partially decrypt using the private key sk0 get Then send it to each client as well as

[0009] S4, the client passes the private key sk i Partial decryption get And according to as well as The result of full decryption is

[0010] S5, client passes Train the local model and pass the gradient Encrypted using the public key Send to server S0;

[0011] S6, servers S0 and S1 collaborative computing With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance greater than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i );

[0012] S7, server S0 calculates Enc(Score based on the properties of double trapdoor homomorphic encryption i )and The product of Then, according to the homomorphic additive property of double trapdoor homomorphic encryption, Calculated by servers S0 and S1 Weighting in the global model aggregation process

[0013] S8 and server S0 perform global model aggregation based on the homomorphic additivity of double trapdoor homomorphic encryption.

[0014] Optionally, step S1 specifically includes:

[0015] Generate a public-private key pair (pk, sk) based on the security parameter ε and the public-private key generation algorithm, where pk = (g, N), g = N + 1, sk = λ, N = p·q, λ = lcm(p-1,q-1), and μ = λ - 1modN, p = 2p' + 1, q = 2q' + 1, where p and q are two unequal strong prime numbers generated according to the security parameter ε;

[0016] The private key sk is split into sk0, sk1, ..., sk through the key splitting algorithm (n+1) =λ0,λ1,...,λ (n+1) n+2 copies, where (λ0+λ1)=0modλ and (λ0+λ1)=1modN, (λ0+λ2)=0modλ and (λ0+λ2)=1modN,..., (λ0+λ (N+1) =0modλ and (λ0+λ (N+1) )=0modN, δ satisfies δ=0modλ and δ=1modN, λ0 is a random number of any σ bits, λ1,λ2,...,λ n+1 Satisfying λ1,λ2,...,λ n+1 =λ·μ+α·N-λ0, where α is any non-negative integer.

[0017] Optionally, step S6 specifically includes:

[0018] Server S0 and S1 collaborative computing With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance less than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i )

[0019] Servers S0 and S1 combine the scalar multiplication property, homomorphic additive property and secure absolute value calculation protocol of double trapdoor homomorphic encryption to calculate With Enc(G t )

[0020] Servers S0 and S1 jointly execute the secure division protocol SDIV (Enc (MD i ),Enc(n))=Enc(MD i / n), get Enc(MD i )’s average value Enc(AvgScore);

[0021] Servers S0 and S1 use a secure comparison protocol to remove data whose Manhattan distance is less than the average Enc (AvgScore). Corresponding client;

[0022] Server S0 calculates the corresponding gradient of the unremoved client Gradient score Enc(Score i )=1 / Relu(Enc(MD i ) -1 Enc(AvgScore)).

[0023] Optionally, in step S7, the server S0 calculates Enc(Score) based on the properties of double trapdoor homomorphic encryption. i )and The product of Specifically include:

[0024] Server S0 generates random numbers r1 and r2;

[0025] Server S0 calculates the gradient score Enc (Score) of each client based on random numbers r1 and r2. i )as well as Obfuscate to get Enc(Score i )·Enc(r1) -1 =Enc(Score i -r1) and

[0026] Server S0 uses private key sk0 to partially decrypt and obtain Enc(Score i -r1)' and

[0027] Server S0 will Enc(Score i -r1)', and Enc(Score i -r1) sent to server S1;

[0028] Server S1 uses private key sk1 to and Enc(Score i -r1) after partial decryption and Enc(Score i -r1)";

[0029] Server S1 obtains the full decryption function and Score i -r1;

[0030] Server S1 performs the calculation And use the public key pk to encrypt the calculation result;

[0031] Server S1 performs the calculation Afterwards

[0032] Server S1 will be distributed Send to server S0;

[0033] Server S0 performs calculations Get noise cancellation

[0034] A second aspect of the present application provides a privacy-preserving federated learning security aggregation device, the device comprising:

[0035] The key generation unit is used to generate a public and private key pair (pk, sk) based on the security parameters, and split the private key into sk0, sk1, ...sk through the key splitting algorithm n+1 n+2 copies;

[0036] The key distribution unit is used to send the public key pk to each client and server, send the private key sk0 to server S0, and send the private key sk to server S1. n+1 , send the private key sk to each client i ,i∈[1,n];

[0037] Initial encryption unit, used to initialize the global model Server S0 uses the pure root dataset to evaluate the global model Perform training to obtain pure gradient G t , and then use the public key pk to convert the global model Encrypted to And partially decrypt using the private key sk0 get Then send it to each client as well as

[0038] Decryption unit, used by the client through the private key sk i Partial decryption get And according to as well as The result of full decryption is

[0039] Local training unit, used by the client through Train the local model and pass the gradient Encrypted using the public key Send to server S0;

[0040] The first processing unit is used for collaborative computing between servers S0 and S1 With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance greater than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i );

[0041] The second processing unit is used for server S0 to calculate Enc(Score based on the properties of double trapdoor homomorphic encryption i )and The product of Then, according to the homomorphic additive property of double trapdoor homomorphic encryption, Calculated by servers S0 and S1 Weighting in the global model aggregation process

[0042] Aggregation unit, used by server S0 to perform global model aggregation based on homomorphic additivity of double trapdoor homomorphic encryption

[0043] Optionally, the key generation unit is specifically configured to:

[0044] Generate a public-private key pair (pk, sk) based on the security parameter ε and the public-private key generation algorithm, where pk = (g, N), g = N + 1, sk = λ, N = p·q, λ = lcm(p-1,q-1), and μ = λ -1 modN, p = 2p' + 1, q = 2q' + 1, where p and q are two unequal strong prime numbers generated according to the security parameter ε;

[0045] The private key sk is split into sk0, sk1, ..., sk through the key splitting algorithm (n+1) =λ0,λ1,...,λ (n+1) n+2 copies, where (λ0+λ1)=0modλ and (λ0+λ1)=1modN, (λ0+λ2)=0modλ and (λ0+λ2)=1modN,..., (λ0+λ (N+1) =0modλ and (λ0+λ (N+1) )=0modN, δ satisfies δ=0modλ and δ=1modN, λ0 is a random number of any σ bits, λ1,λ2,...,λ n+1 Satisfying λ1,λ2,...,λ n+1 =λ·μ+α·N-λ0, where α is any non-negative integer.

[0046] Optionally, the first processing unit is specifically configured to:

[0047] Server S0 and S1 collaborative computing With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance less than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i )

[0048] Servers S0 and S1 combine the scalar multiplication property, homomorphic additive property and secure absolute value calculation protocol of double trapdoor homomorphic encryption to calculate With Enc(G t )

[0049] Servers S0 and S1 jointly execute the secure division protocol SDIV (Enc (MD i ),Enc(n))=Enc(MD i / n), get Enc(MD i )’s average value Enc(AvgScore);

[0050] Servers S0 and S1 use a secure comparison protocol to remove data whose Manhattan distance is less than the average Enc (AvgScore). Corresponding client;

[0051] Server S0 calculates the corresponding gradient of the unremoved client Gradient score Enc(Score i )=1 / Relu(Enc(MD i ) -1 Enc(AvgScore)).

[0052] Optionally, the server S0 in the second processing unit calculates Enc(Score) based on the properties of double trapdoor homomorphic encryption. i )and The product of Specifically include:

[0053] Server S0 generates random numbers r1 and r2;

[0054] Server S0 calculates the gradient score Enc (Score) of each client based on random numbers r1 and r2. i )as well as Obfuscate to get Enc(Score i )·Enc(r1) -1 =Enc(Score i -r1) and

[0055] Server S0 uses private key sk0 to partially decrypt and obtain Enc(Score i -r1)' and

[0056] Server S0 will Enc(Score i -r1)', and Enc(Score i -r1) sent to server S1;

[0057] Server S1 uses private key sk1 to And Enc(Scorei-r1) is partially decrypted to get and Enc(Score i -r1)";

[0058] Server S1 obtains the full decryption function and Score i -r1;

[0059] Server S1 performs the calculation And use the public key pk to encrypt the calculation result;

[0060] Server S1 performs the calculation Afterwards

[0061] Server S1 will be distributed Send to server S0;

[0062] Server S0 performs calculations Get noise cancellation

[0063] A third aspect of the present application provides a privacy-preserving federated learning security aggregation device, the device comprising a processor and a memory:

[0064] The memory is used to store program code and transmit the program code to the processor;

[0065] The processor is used to execute the steps of the privacy-preserving federated learning security aggregation method as described in the first aspect according to the instructions in the program code.

[0066] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the method described in the first aspect.

[0067] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0068] In this application, a privacy-preserving federated learning security aggregation method, apparatus, device and medium are provided. A dual-server federated learning architecture is constructed based on double trapdoor homomorphic encryption technology to ensure that the privacy of client-related information is not disclosed during poisoning attack detection and global model aggregation. Under the premise of strictly protecting client data privacy, the difference between each element of the gradient is directly calculated in the ciphertext state, and the difference in the gradient in each element dimension is effectively captured, thereby achieving accurate detection of poisoning attacks on encrypted models in heterogeneous data scenarios. Finally, a dynamic threshold Byzantine robust aggregation scheme based on average value is used to solve the technical problem of difficulty in removing poisoned gradients caused by dynamic changes in gradient thresholds in heterogeneous data scenarios. DETAILED DESCRIPTION

[0069] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0070] This application designs a privacy-preserving federated learning security aggregation method, device, equipment and medium, and constructs a dual-server federated learning architecture based on double trapdoor homomorphic encryption technology. Under the premise of strictly protecting the privacy of client data, by directly calculating the difference between each element of the gradient in the ciphertext state, the difference in the gradient in each element dimension is effectively captured, thereby realizing accurate detection of encryption model poisoning attacks in heterogeneous data scenarios.

[0071] For ease of understanding, the privacy-preserving federated learning security aggregation method in the embodiment of this application is specifically as follows:

[0072] S1. Generate a public-private key pair (pk, sk) based on security parameters, and split the private key into sk0, sk1, ...sk using a key splitting algorithm. n+1 n+2 copies;

[0073] S2, send the public key pk to each client and server, send the private key sk0 to server S0, send the private key sk to server S1 n+1 , send the private key sk to each client i,i∈[1,n];

[0074] S3. Initialize the global model Server S0 uses the pure root dataset to evaluate the global model Perform training to obtain pure gradient G t , and then use the public key pk to convert the global model Encrypted to And partially decrypt using the private key sk0 get Then send it to each client as well as

[0075] S4, the client passes the private key sk i Partial decryption get And according to as well as The result of full decryption is

[0076] S5, client passes Train the local model and pass the gradient Encrypted using the public key Send to server S0;

[0077] S6, servers S0 and S1 collaborative computing With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance greater than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i );

[0078] S7, server S0 calculates Enc(Score based on the properties of double trapdoor homomorphic encryption i )and The product of Then, according to the homomorphic additive property of double trapdoor homomorphic encryption, Calculated by servers S0 and S1 Weighting in the global model aggregation process

[0079] S8 and server S0 perform global model aggregation based on the homomorphic additivity of double trapdoor homomorphic encryption.

[0080] Furthermore, the step S1 specifically includes:

[0081] Generate a public-private key pair (pk, sk) based on the security parameter ε and the public-private key generation algorithm, where pk = (g, N), g = N + 1, sk = λ, N = p·q, λ = lcm(p-1,q-1), and μ = λ -1 modN, p = 2p' + 1, q = 2q' + 1, where p and q are two unequal strong prime numbers generated according to the security parameter ε;

[0082] The private key sk is split into sk0, sk1, ..., sk through the key splitting algorithm (n+1) =λ0,λ1,...,λ (n+1) n+2 copies, where (λ0+λ1)=0modλ and (λ0+λ1)=1modN, (λ0+λ2)=0modλ and (λ0+λ2)=1modN,..., (λ0+λ (N+1) =0modλ and (λ0+λ (N+1) )=0modN, δ satisfies δ=0modλ and δ=1modN, λ0 is a random number of any σ bits, λ1,λ2,...,λ n+1 Satisfying λ1,λ2,...,λ n+1 =λ·μ+α·N-λ0, where α is any non-negative integer.

[0083] Furthermore, the step S6 specifically includes:

[0084] Server S0 and S1 collaborative computing With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance less than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i )

[0085] Servers S0 and S1 combine the scalar multiplication property, homomorphic additive property and secure absolute value calculation protocol of double trapdoor homomorphic encryption to calculate With Enc(G t )

[0086] Servers S0 and S1 jointly execute the secure division protocol SDIV (Enc (MD i ),Enc(n))=Enc(MD i / n), get Enc(MD i)’s average value Enc(AvgScore);

[0087] Servers S0 and S1 use a secure comparison protocol to remove data whose Manhattan distance is less than the average Enc (AvgScore). Corresponding client;

[0088] Server S0 calculates the corresponding gradient of the unremoved client Gradient score Enc(Score i )=1 / Relu(Enc(MD i ) -1 Enc(AvgScore)).

[0089] Furthermore, in step S7, the server S0 calculates Enc(Score i )and The product of Specifically include:

[0090] Server S0 generates random numbers r1 and r2;

[0091] Server S0 calculates the gradient score Enc (Score) of each client based on random numbers r1 and r2. i )as well as Obfuscate to get Enc(Score i )·Enc(r1) -1 =Enc(Score i -r1) and

[0092] Server S0 uses private key sk0 to partially decrypt and obtain Enc(Score i -r1)' and

[0093] Server S0 will Enc(Score i λr1)'、 and Enc(Score i -r1) sent to server S1;

[0094] Server S1 uses private key sk1 to And Enc(Scorei-r1) is partially decrypted to get and Enc(Score i -r1)";

[0095] Server S1 obtains the full decryption function and Score i -r1;

[0096] Server S1 performs the calculation And use the public key pk to encrypt the calculation result;

[0097] Server S1 performs the calculation Afterwards

[0098] Server S1 will be distributed Send to server S0;

[0099] Server S0 performs calculations Get noise cancellation

[0100] It should be noted that the algorithms included in this application solution are:

[0101] Public-private key generation algorithm: Input security parameter ε and generate two unequal strong prime numbers p and q, where p = 2p'+1 and q = 2q'+1. Calculate N = p·q, λ = lcm(p-1,q-1), and μ = λ -1 mod N. The public key is pk = (g, N), where g = N + 1, and the private key is sk = λ.

[0102] Encryption algorithm: Input plaintext x, encrypt x using pk to obtain Enc(x) = g x ·r N modN 2 .

[0103] Decryption algorithm: Input ciphertext Enc(x) and sk=λ, calculate x=L(Enc(x)modN 2 )·μmodN. Where L(x)=(x-1) / N.

[0104] Partial decryption algorithm: input Enc(x) and sk0 or sk i , i∈[1,n+1]. Calculate or

[0105] Full decryption algorithm: Input Enc(x)' and Enc(x)", calculate x = L(Enc(x)'·Enc(x)" mod N 2 ). Where L(x)=(x-1) / N.

[0106] Secure Division Protocol: Input Enc(x) and Enc(y), server S0 and server S1 jointly execute the secure division protocol SDIV(Enc(x), Enc(y)) to obtain

[0107] Secure comparison protocol: Input Enc(x) and Enc(y), server S0 and server S1 jointly execute the secure comparison protocol SCMP(Enc(x), Enc(y)) to obtain Enc(P). If P = 0, then x ≥ y, otherwise x ≤ y.

[0108] Secure Absolute Value Protocol: Input Enc(x), S0 and server S1 jointly execute the Secure Absolute Value Protocol SSBA(Enc(x)) to obtain Enc(|x|).

[0109] This solution constructs a dual-server federated learning architecture, which successfully implements effective detection of poisoning attacks on encrypted models while ensuring that no private information of the client is disclosed.

[0110] In scenarios with heterogeneous data, local models trained on the client often move in different directions, making it difficult to distinguish between benign and poisoned gradients. Existing solutions struggle to accurately distinguish between these two types of gradients. Therefore, the poisoning attack detection method proposed in this solution, based on Manhattan distance, can accurately detect poisoned gradients, addressing the shortcomings of existing solutions.

[0111] Because real-world scenarios are more similar to data heterogeneity, the threshold between benign gradients and poisoned gradients constantly changes dynamically. The proposed solution performs well in this scenario, successfully excluding gradients uploaded by malicious clients. Even in situations where a Byzantine attacker controls 70% of the malicious clients, it can still effectively defend against poisoning attacks launched by Byzantine attackers.

[0112] The privacy-preserving federated learning security aggregation device in the embodiment of the present application is specifically:

[0113] The key generation unit is used to generate a public and private key pair (pk, sk) based on the security parameters, and split the private key into sk0, sk1, ...sk through the key splitting algorithm n+1 n+2 copies;

[0114] The key distribution unit is used to send the public key pk to each client and server, send the private key sk0 to server S0, and send the private key sk to server S1. n+1 , send the private key sk to each client i ,i∈[1,n];

[0115] Initial encryption unit, used to initialize the global model Server S0 uses the pure root dataset to evaluate the global model Perform training to obtain pure gradient G t , and then use the public key pk to convert the global model Encrypted to And partially decrypt using the private key sk0 get Then send it to each client as well as

[0116] Decryption unit, used by the client through the private key sk i Partial decryption get And according to as well as The result of full decryption is

[0117] Local training unit, used by the client through Train the local model and pass the gradient Encrypted using the public key Send to server S0;

[0118] The first processing unit is used for collaborative computing between servers S0 and S1 With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance greater than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i );

[0119] The second processing unit is used for server S0 to calculate Enc(Score based on the properties of double trapdoor homomorphic encryption i )and The product of Then, according to the homomorphic additive property of double trapdoor homomorphic encryption, Calculated by servers S0 and S1 Weighting in the global model aggregation process

[0120] Aggregation unit, used by server S0 to perform global model aggregation based on homomorphic additivity of double trapdoor homomorphic encryption

[0121] Furthermore, the key generation unit is specifically used to:

[0122] Generate a public-private key pair (pk, sk) based on the security parameter ε and the public-private key generation algorithm, where pk = (g, N), g = N + 1, sk = λ, N = p·q, λ = lcm(p-1,q-1), and μ = λ -1modN, p = 2p' + 1, q = 2q' + 1, where p and q are two unequal strong prime numbers generated according to the security parameter ε;

[0123] The private key sk is split into sk0, sk1, ..., sk through the key splitting algorithm (n+1) =λ0,λ1,...,λ (n+1) n+2 copies, where (λ0+λ1)=0modλ and (λ0+λ1)=1modN, (λ0+λ2)=0modλ and (λ0+λ2)=1modN,..., (λ0+λ (N+1) =0modλ and (λ0+λ (N+1) )=0modN, δ satisfies δ=0modλ and δ=1modN, λ0 is a random number of any σ bits, λ1,λ2,...,λ n+1 Satisfying λ1,λ2,...,λ n+1 =λ·μ+α·N-λ0, where α is any non-negative integer.

[0124] Furthermore, the first processing unit is specifically configured to:

[0125] Server S0 and S1 collaborative computing With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance less than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i )

[0126] Servers S0 and S1 combine the scalar multiplication property, homomorphic additive property and secure absolute value calculation protocol of double trapdoor homomorphic encryption to calculate With Enc(G t )

[0127] Servers S0 and S1 jointly execute the secure division protocol SDIV (Enc (MD i ),Enc(n))=Enc(MD i / n), get Enc(MD i )’s average value Enc(AvgScore);

[0128] Servers S0 and S1 use a secure comparison protocol to remove data whose Manhattan distance is less than the average Enc (AvgScore). Corresponding client;

[0129] Server S0 calculates the corresponding gradient of the unremoved client Gradient score Enc(Score i )=1 / Relu(Enc(MD i ) -1 Enc(AvgScore)).

[0130] Furthermore, the server S0 in the second processing unit calculates Enc(Score i )and The product of Specifically include:

[0131] Server S0 generates random numbers r1 and r2;

[0132] Server S0 calculates the gradient score Enc (Score) of each client based on random numbers r1 and r2. i )as well as Obfuscate to get Enc(Score i )·Enc(r1) -1 =Enc(Score i -r1) and

[0133] Server S0 uses private key sk0 to partially decrypt and obtain Enc(Score i -r1)' and

[0134] Server S0 will Enc(Score i -r1)', and Enc(Score i -r1) sent to server S1;

[0135] Server S1 uses private key sk1 to and Enc(Score i -r1) after partial decryption and Enc(Score i -r1)";

[0136] Server S1 obtains the full decryption function and Score i -r1;

[0137] Server S1 performs the calculation And use the public key pk to encrypt the calculation result;

[0138] Server S1 performs the calculation Afterwards

[0139] Server S1 will be distributed Send to server S0;

[0140] Server S0 performs calculations Get noise cancellation

[0141] This embodiment of the present application also provides another privacy-preserving federated learning security aggregation device, the device comprising:

[0142] One or more processors and memory, with one processor being used as an example for the following description. The processor and memory may be connected via a bus or other means.

[0143] The processor is used to implement various control logics of the device. It can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. In addition, the processor can also be any traditional processor, microprocessor or state machine. The processor can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP, and / or any other such configuration.

[0144] Memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions corresponding to the method for constructing a multilingual phoneme representation model in the embodiments of the present invention. The processor executes the non-volatile software programs, instructions, and modules stored in memory to execute various functional applications and data processing of the device, thereby implementing the method for constructing a multilingual phoneme representation model in the aforementioned method embodiments.

[0145] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the device, etc. In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0146] One or more units are stored in the memory and, when executed by one or more processors, implement the following steps:

[0147] S1. Generate a public-private key pair (pk, sk) based on security parameters, and split the private key into sk0, sk1, ...sk using a key splitting algorithm. n+1 n+2 copies;

[0148] S2, send the public key pk to each client and server, send the private key sk0 to server S0, send the private key sk to server S1 n+1 , send the private key sk to each client i ,i∈[1,n];

[0149] S3. Initialize the global model Server S0 uses the pure root dataset to evaluate the global model Perform training to obtain pure gradient G t , and then use the public key pk to convert the global model Encrypted to And partially decrypt using the private key sk0 get Then send it to each client as well as

[0150] S4, the client passes the private key sk i Partial decryption get And according to as well as The result of full decryption is

[0151] S5, client passes Train the local model and pass the gradient Encrypted using the public key Send to server S0;

[0152] S6, servers S0 and S1 collaborative computing With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance greater than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i );

[0153] S7, server S0 calculates Enc(Score based on the properties of double trapdoor homomorphic encryptioni )and The product of Then, according to the homomorphic additive property of double trapdoor homomorphic encryption, Calculated by servers S0 and S1 Weighting in the global model aggregation process

[0154] S8 and server S0 perform global model aggregation based on the homomorphic additivity of double trapdoor homomorphic encryption.

[0155] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, any one of the implementation methods of the privacy-preserving federated learning security aggregation method described in the above embodiments is implemented.

[0156] As examples, non-volatile storage media can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) as external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM, (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0157] In the embodiments of the present application, a privacy-preserving federated learning security aggregation method, apparatus, device and medium are provided. A dual-server federated learning architecture is constructed based on double trapdoor homomorphic encryption technology to ensure that the privacy of client-related information is not disclosed during poisoning attack detection and global model aggregation. Under the premise of strictly protecting the privacy of client data, the difference between each element of the gradient is directly calculated in the ciphertext state, and the difference in the gradient in each element dimension is effectively captured, thereby achieving accurate detection of poisoning attacks on encrypted models in heterogeneous data scenarios. Finally, a dynamic threshold Byzantine robust aggregation scheme based on average value is used to solve the technical problem of difficulty in removing poisoned gradients caused by dynamic changes in gradient thresholds in heterogeneous data scenarios.

[0158] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0159] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0160] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0161] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0162] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0163] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0164] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English full name: Read-Only Memory, English abbreviation: ROM), random access memory (English full name: Random Access Memory, English abbreviation: RAM), magnetic disk or optical disk and other media that can store program code.

[0165] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A privacy-preserving federated learning secure aggregation method, characterized by: include: S1. Generate a public-private key pair (pk, sk) based on security parameters, and split the private key into sk0, sk1, ...sk using a key splitting algorithm. n+1 n+2 copies; S2, send the public key pk to each client and server, send the private key sk0 to server S0, send the private key sk to server S1 n+1 , send the private key sk to each client i ,i∈[1,n]; S3. Initialize the global model W i t , server S0 uses the pure root dataset to analyze the global model Perform training to obtain pure gradient G t , and then use the public key pk to convert the global model Encrypted to And partially decrypt using the private key sk0 get Then send it to each client as well as S4, the client passes the private key sk i Partial decryption Get Enc(W i t )” and based on and Enc(W i t )” and then decrypt the result to get W i t ; S5, the client through W i t Train the local model and pass the gradient Encrypted using the public key Send to server S0; S6, servers S0 and S1 collaborative computing With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance greater than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i ); S7, server S0 calculates Enc(Score based on the properties of double trapdoor homomorphic encryption i )and The product of Then, according to the homomorphic additive property of double trapdoor homomorphic encryption, Calculated by servers S0 and S1 Weighting in the global model aggregation process S8 and server S0 perform global model aggregation based on the homomorphic additivity of double trapdoor homomorphic encryption.

2. The privacy-preserving federated learning secure aggregation method according to claim 1, characterized in that: The step S1 specifically includes: Generate a public-private key pair (pk, sk) based on the security parameter ε and the public-private key generation algorithm, where pk = (g, N), g = N + 1, sk = λ, N = p·q, λ = lcm(p-1,q-1), and μ = λ -1 modN, p = 2p' + 1, q = 2q' + 1, where p and q are two unequal strong prime numbers generated according to the security parameter ε; The private key sk is split into sk0, sk1, ..., sk through the key splitting algorithm (n+1) =λ0,λ1,...,λ (n+1) n+2 copies, where (λ0+λ1)=0modλ and (λ0+λ1)=1modN, (λ0+λ2)=0modλ and (λ0+λ2)=1modN,..., (λ0+λ (N+1) =0modλ and (λ0+λ( N+1) )=0modN, δ satisfies δ=0modλ and δ=1modN, λ0 is a random number of any σ bits, λ1,λ2,...,λ n+1 Satisfying λ1,λ2,...,λ n+1 =λ·μ+α·N-λ0, where α is any non-negative integer.

3. The privacy-preserving federated learning secure aggregation method according to claim 2, characterized in that: The step S6 specifically includes: Server S0 and S1 collaborative computing With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance less than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i ) Servers S0 and S1 combine the scalar multiplication property, homomorphic additive property and secure absolute value calculation protocol of double trapdoor homomorphic encryption to calculate With Enc(G t ) Servers S0 and S1 jointly execute the secure division protocol SDIV (Enc (MD i ),Enc(n))=Enc(MD i / n), get Enc(MD i )’s average value Enc(AvgScore); Servers S0 and S1 use a secure comparison protocol to remove data whose Manhattan distance is less than the average Enc (AvgScore). Corresponding client; Server S0 calculates the corresponding gradient of the unremoved client Gradient score Enc(Score i )=1 / Relu(Enc(MD i ) -1 Enc(AvgScore)).

4. The privacy-preserving federated learning secure aggregation method according to claim 3, characterized in that: In step S7, the server S0 calculates Enc(Score) based on the properties of double trapdoor homomorphic encryption. i )and The product of Specifically include: Server S0 generates random numbers r1 and r2; Server S0 calculates the gradient score Enc (Score) of each client based on random numbers r1 and r2. i )as well as Obfuscate to get Enc(Score i )·Enc(r1) -1 =Enc(Score i -r1) and Server S0 uses private key sk0 to partially decrypt and obtain Enc(Score i -r1)' and Server S0 will Enc(Score i -r1)', and Enc(Score i -r1) sent to server S1; Server S1 uses private key sk1 to and Enc(Score i -r1) after partial decryption and Enc(Score i -r1)"; Server S1 obtains the full decryption function and Score i -r1; Server S1 performs the calculation And use the public key pk to encrypt the calculation result; Server S1 performs the calculation Afterwards Server S1 will be distributed Send to server S0; Server S0 performs calculations Get noise cancellation 5. A privacy-preserving federated learning security aggregation device, characterized in that: include: The key generation unit is used to generate a public and private key pair (pk, sk) based on the security parameters, and split the private key into sk0, sk1, ...sk through the key splitting algorithm n+1 n+2 copies; The key distribution unit is used to send the public key pk to each client and server, send the private key sk0 to server S0, and send the private key sk to server S1. n+1 , send the private key sk to each client i ,i∈[1,n]; Initial encryption unit, used to initialize the global model W i t , server S0 uses the pure root dataset to calculate the global model W i t Perform training to obtain pure gradient G t , and then use the public key pk to convert the global model W i t Encrypted as Enc(W i t ), and use the private key sk0 to partially decrypt Enc(W i t ) to get Enc(W i t )', then send Enc(W i t ) as well as Decryption unit, used by the client through the private key sk i Partial decryption Enc(W i t ) to get Enc(W i t )” and based on and Enc(W i t )” and then decrypt the result to get W i t ; Local training unit, used by the client through Train the local model and pass the gradient Encrypted using the public key Send to server S0; The first processing unit is used for collaborative computing between servers S0 and S1 With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance greater than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i ); The second processing unit is used for server S0 to calculate Enc(Score based on the properties of double trapdoor homomorphic encryption i )and The product of Then, according to the homomorphic additive property of double trapdoor homomorphic encryption, Calculated by servers S0 and S1 Weighting in the global model aggregation process Aggregation unit, used by server S0 to perform global model aggregation based on homomorphic additivity of double trapdoor homomorphic encryption 6. The privacy-preserving federated learning security aggregation device according to claim 5, characterized in that: The key generation unit is specifically used to: Generate a public-private key pair (pk, sk) based on the security parameter ε and the public-private key generation algorithm, where: pk=(g,N), g=N+1, sk=λ, N=p·q, λ=lcm(p-1,q-1) and μ=λ -1 modN, p = 2p' + 1, q = 2q' + 1, where p and q are two unequal strong prime numbers generated according to the security parameter ε; The private key sk is split into sk0, sk1, ..., sk through the key splitting algorithm (n+1) =λ0,λ1,...,λ (n+1) n+2 copies, where (λ0+λ1)=0modλ and (λ0+λ1)=1modN, (λ0+λ2)=0modλ and (λ0+λ2)=1modN,..., (λ0+λ (N+1) =0modλ and (λ0+λ (N+1) )=0modN, δ satisfies δ=0modλ and δ=1modN, λ0 is a random number of any σ bits, λ1,λ2,...,λ n+1 Satisfying λ1,λ2,...,λ n+1 =λ·μ+α·N-λ0, where α is any non-negative integer.

7. The privacy-preserving federated learning security aggregation device according to claim 6, characterized in that: The first processing unit is specifically configured to: Server S0 and S1 collaborative computing With Enc(G t ) between the Manhattan distance Enc(MD i ) of the average value Enc(AvgScore)=Enc(MD i / n), and remove those with Manhattan distance less than the average Enc (AvgScore) After that, calculate the client gradient score Enc(Score i ) Servers S0 and S1 combine the scalar multiplication property, homomorphic additive property and secure absolute value calculation protocol of double trapdoor homomorphic encryption to calculate With Enc(G t ) Servers S0 and S1 jointly execute the secure division protocol SDIV (Enc (MD i ),Enc(n))=Enc(MD i / n), get Enc(MD i )’s average value Enc(AvgScore); Servers S0 and S1 use a secure comparison protocol to remove data whose Manhattan distance is less than the average Enc (AvgScore). Corresponding client; Server S0 calculates the corresponding gradient of the unremoved client Gradient score Enc(Score i )=1 / Relu(Enc(MD i ) -1 Enc(AvgScore)).

8. The privacy-preserving federated learning security aggregation device according to claim 7, characterized in that: In the second processing unit, server S0 calculates Enc(Score based on the properties of double trapdoor homomorphic encryption) i )and The product of Specifically include: Server S0 generates random numbers r1 and r2; Server S0 calculates the gradient score Enc (Score) of each client based on random numbers r1 and r2. i )as well as Obfuscate to get Enc(Score i )·Enc(r1) -1 =Enc(Score i -r1) and Server S0 uses private key sk0 to partially decrypt and obtain Enc(Score i -r1)' and Server S0 will Enc(Score i -r1)', and Enc(Score i -r1) sent to server S1; Server S1 uses private key sk1 to And Enc(Scorei-r1) is partially decrypted to get and Enc(Score i -r1)"; Server S1 obtains the full decryption function and Score i -r1; Server S1 performs the calculation And use the public key pk to encrypt the calculation result; Server S1 performs the calculation Afterwards Server S1 will be distributed Send to server S0; Server S0 performs calculations Get noise cancellation 9. A privacy-preserving federated learning security aggregation device, characterized in that: The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the privacy-preserving federated learning security aggregation method according to any one of claims 1 to 4 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used to execute the privacy-preserving federated learning security aggregation method according to any one of claims 1 to 4.

Images