Multi-party joint reasoning method for privacy protection

Abstract

The invention discloses a multi-party joint reasoning method for privacy protection, and aims to solve the problems of convergence difficulty and privacy leakage caused by data heterogeneity and model heterogeneity in federated learning. The method comprises the following steps: a task initiator generates a public key and a private key, divides the private key into two parts, and respectively sends the two parts to an aggregation platform and a client; and the initiator encrypts the task data set and sends the task data set to the client, and the client performs ciphertext reasoning to obtain an encrypted category score, performs partial decryption and then uploads a result to the aggregation platform. And the aggregation platform uses the partial private key to decrypt the partial decryption result uploaded by the client, and calculates probability distribution through a security calculation protocol. And finally, the aggregation platform aggregates the probability distribution of all the clients according to an aggregation algorithm to obtain a final reasoning result. According to the method, multi-party joint reasoning is efficiently and fairly carried out on the premise of protecting privacy through a security computing protocol and an aggregation algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of information security, and in particular to a multi-party joint inference method for privacy protection. Background Art

[0002] With the advent of the big data era, the security and privacy protection of data have become key issues in the information society. In the field of machine learning, especially in federated learning, data is usually scattered among multiple clients, and these clients may not be able to centralize the data for model training due to data privacy regulations and privacy concerns. Federated learning, as a distributed machine learning architecture, allows for model training through local training and collaborative updates without sharing the original data. However, federated learning faces the problems of data heterogeneity and model heterogeneity, which lead to difficulties in model convergence and aggregation obstacles. In addition, federated learning may also face the risk of privacy leakage during the model aggregation process.

[0003] Traditional federated learning methods often adopt methods such as client clustering, federated optimization, meta-learning, and multi-task learning when dealing with data heterogeneity and model heterogeneity. However, these methods are difficult to learn general features when dealing with non-independent and identically distributed data, resulting in poor generalization ability on the data of other clients. For the problem of model heterogeneity, existing federated learning frameworks need to perform group operations, knowledge distillation, or only support some different model architectures, and these methods have different limitations in practical applications.

[0004] In addition, federated learning requires a large amount of computing resources and communication resources during the training and inference processes. When encountering a heterogeneous environment, the training efficiency may be further reduced. Some existing works improve the communication efficiency and reduce resource consumption by compressing the transmitted model, but in a heterogeneous environment, the efficiency and accuracy of model training are still a challenge.

[0005] Therefore, in view of the problems existing in the existing solutions, there is an urgent need for an efficient and accurate joint inference method. Summary of the Invention

[0006] The object of the present invention is to overcome the deficiencies of the prior art. The present invention proposes a multi-party joint inference for privacy protection, aiming to solve the problems of data heterogeneity and model heterogeneity in federated learning, while protecting data privacy. By designing a secure computing protocol and aggregation algorithm, efficient and fair multi-party joint inference is achieved under the premise of protecting privacy.

[0007] To achieve the above object, the technical solution provided by the present invention is: a multi-party joint inference method for privacy protection. The multi-party joint inference includes a task initiator, an aggregation platform, and M clients. The method is that the task initiator creates a task including a dataset to be inferred, and then sends the dataset to be inferred to the online clients through the aggregation platform. The clients perform local inference to obtain the scores of each class and send them to the aggregation platform. Finally, the aggregation platform designs a fair aggregation protocol to obtain the final inference result and sends it to the task initiator. Among them, in order to protect the privacy of the inference dataset of the task initiator, the task initiator uses a Paillier cryptosystem with threshold decryption to protect its data and sends it to the aggregation platform. To protect the local inference data of the task initiator from being leaked, the aggregation platform and the clients jointly execute a secure computing protocol to obtain the final inference result

[0008] Further, the multi-party joint inference method for privacy protection includes the following steps:

[0009] S1. The task initiator initializes a Paillier cryptosystem with threshold decryption, publishes the public key pk of this system, and divides the private key λ into two parts, namely partial private keys sk 1 and sk 2 , and sends them to the aggregation platform and all clients respectively;

[0010] S2. The task initiator publishes a task, which includes a dataset D to be inferred t . The task initiator encrypts the dataset D using a Paillier cryptosystem with threshold decryption t , where represents the encryption operation, and then sends the encrypted dataset D to be inferred t to the aggregation platform;

[0011] S3. The aggregation platform sends the encrypted dataset to be inferred to the online clients C = [c 1 , c 2 ,..., c i ,..., c M , where c i represents client i. Client i performs local ciphertext inference through a designed secure algorithm protocol to obtain the encrypted scores of each category where k represents category k and K represents the number of all categories. Then, after partially decrypting the encrypted scores of each category, [s i = [[s i，0 , [s i，1 ,..., [s i，k ,..., [s i，K is obtained and sent to the aggregation platform;

[0012] S4. The aggregation platform decrypts all the fraction corresponding to each category of the partially decrypted data sent by the clients by using the Paillier cryptosystem with threshold decryption to obtain Then, the probability distribution of each client is obtained by using the softmax function And by designing a fair aggregation mechanism, the final inference result y is output, and the result is sent to the task initiator.

[0013] Furthermore, in step S1, the task initiator initializes the Paillier cryptosystem with threshold decryption, selects two large prime numbers p and q, calculates the integer N = p·q and the generator g = N + 1, and calculates the private key λ = lcm(p - 1, q - 1), where lcm() represents the least common multiple function, and publishes the public key pk = (g, N) of the Paillier cryptosystem with threshold decryption. The task initiator divides the private key into sk 1 and sk 2 , sk 1 +sk 2 = 0 mod λ, sk 1 +sk 2 = 1 mod N, sk 1 is a random number.

[0014] Furthermore, in step S2, the task initiator encrypts the dataset D based on the Paillier cryptosystem with threshold decryption using the public key pk t , obtaining D t ← Enc(pk, D t ).

[0015] Furthermore, in step S3, there are M clients on the aggregation platform. The aggregation platform and client i design a secure ReLU protocol (Secure ReLU) based on the Paillier cryptosystem with threshold decryption. The input of the Secure ReLU protocol is the ciphertext x. The specific steps are as follows:

[0016] S311. The aggregation platform selects a random number r 1 ← {0, 1} σ \{0}, the random number r 2 ≤ N / 2, such that r 1 + r 2 > N / 2, where σ is the security parameter; the aggregation platform flips a random coin to generate π, i.e., π = {0, 1};

[0017] S312. If π = 0, then encrypt r 1 + r 2 using the public key pk based on the Paillier cryptosystem with threshold decryption to obtain r1 +r 2 ← Enc(pk, r 1 +r 2 ), then partial decryption can be obtained If π = 1, then encrypt r using the public key pk based on the Paillier cryptosystem with threshold decryption 1 +r 2 Get r 1 +r 2 ← Enc(pk, r 1 +r 2 ), then

[0018] S313. The aggregation platform uses the partial private key sk 1 and partial decryption D to obtain decryption D 1 ← PDec(sk 1 , D), and finally send the data D, D 1 , x to the client i;

[0019] S314. After receiving the data, the client i uses the partial private key sk 2 Based on the Paillier cryptosystem with threshold decryption for partial decryption, obtain decryption D 2 ← PDec(sk 2 , D 1 ), and based on D 1 , D 2 Decrypt to obtain the intermediate parameter

[0020] S315. If then the intermediate parameter μ 0 ← 1; otherwise, μ 0 ← 0; The client i calculates the intermediate parameter d 0 , that is and send it to the aggregation platform;

[0021] S316. The aggregation platform calculates the maximum value under the ciphertext, and gets max(x, 0) ← x π d 0 1-2π ;

[0022] Furthermore, in the described privacy - protected multi - party joint inference method, it is characterized in that in step S3, it includes an aggregation platform and M clients. The aggregation platform and client i calculate the encrypted score for each category based on the scalar multiplication protocol and the secure ReLU protocol of the Paillier cryptosystem with threshold decryption

[0023] S321. The client receives the encrypted dataset D t and the number of classes C, and client i has the model parameters The client takes the encrypted dataset as the input D t , and the input of the 0th layer is denoted as a 0 = D t ;

[0024] S322. For each layer l from 1 to L, the client locally calculates the intermediate parameters through the Paillier encryption system with a threshold The aggregation platform and the client jointly execute based on the Secure ReLU protocol (Secure ReLU) to obtain the output of the, layer, where σ is the ReLU function;

[0025] S323. Client i and the aggregation platform jointly decrypt a by calling the partial decryption function PDec and the threshold decryption function TDec L to obtain

[0026] S324. The aggregation platform obtains the probability value p of client i i = softmax(s i ), where

[0027] Furthermore, in the described privacy - protected multi - party joint inference method, in step S4, the aggregation algorithm includes a maximum - value aggregation algorithm and an addition - aggregation algorithm. The maximum - value aggregation algorithm determines the final inference result y by calculating the maximum probability value of each class to determine the final inference result y, and the addition - aggregation algorithm determines the final inference result y by calculating the total probability value of each class .

[0028] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0029] 1. Through offline training, the present invention significantly reduces the online training time and improves the inference efficiency. In the federated learning environment, this high efficiency is crucial for practical deployment and can effectively reduce the computing and communication costs.

[0030] 2. The present invention adopts the threshold Paillier encryption system and secure computing protocols (such as the Secure ReLU protocol) to achieve secure machine - learning inference while protecting data privacy. Through the partial decryption and joint - decryption mechanisms, it ensures that the private data of the client and the requester is not leaked.

[0031] 3. The present invention designs a fair aggregation algorithm that can fairly consider the contributions of all clients and avoid unfair problems caused by data or model heterogeneity. This fairness helps improve the generalization ability and convergence of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flowchart of the method of the present invention.

[0033] Figure 2 It is a system model diagram of the method of the present invention.

[0034] Figure 3 It is a schematic diagram of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] The present invention will be further described below in conjunction with specific embodiments.

[0036] As Figures 1 to 2 shown, this embodiment provides a privacy - protected multi - party joint inference method. The mobile crowd sensing includes a task initiator, an aggregation platform, and 3 clients. The method is that the task initiator creates a task including a data set to be inferred, then sends the data set to be inferred to the online clients through the aggregation platform. The clients perform local inferences to obtain the scores for each class and send them to the aggregation platform. Finally, the aggregation platform designs a fair aggregation protocol to obtain the final inference result and sends it to the task initiator. Among them, in order to protect the privacy of the inference data set of the task initiator, the task initiator uses a threshold - decryption Paillier cryptosystem to protect its data and sends it to the aggregation platform. To protect the local inference data of the task initiator from being leaked, the aggregation platform and the clients jointly execute a secure computing protocol to obtain the final inference result.

[0037] Furthermore, the privacy - protected multi - party joint inference method includes the following steps:

[0038] S1. The task initiator initializes a Paillier cryptosystem with threshold decryption, publishes the public key pk of this system, and divides the private key λ into two parts, namely partial private keys sk 1 and sk 2 , and sends them to the aggregation platform and all clients respectively;

[0039] S2. The task initiator publishes a task, which includes a data set D t ={5, 2, 3} to be inferred. The task initiator encrypts the data set D t using a Paillier cryptosystem with threshold decryption, where represents the encryption operation, and then sends the encrypted data set D t to the aggregation platform;

[0040] S3. The aggregation platform sends the encrypted data set to be inferred to the online client C = [c 1 , c 2 , c 3 . The client 1 performs local ciphertext inference through the designed secure algorithm protocol and obtains the encrypted scores s 1 = [614556256553860506000547166032349667986584058613299889883015, 839786741518535489251552746097780886701399609483695052383343, 43811194630584545942267520678103605626006057179755104276952]. The client 2 performs local ciphertext inference through the designed secure algorithm protocol and obtains the encrypted scores s 2 = [5729644723651735478263024129410233812376149908007141031149198310034198576312345678901234567890123456789, 87766554438856198470254317596410312345678901234567890123456789012345678901234567890123456789, 1928374651928374651928374651928374651928374651928374651928374651928374651928374651928374651928374651928]. The client 3 performs local ciphertext inference through the designed secure algorithm protocol and obtains the encrypted scores s 3= [41582714367426835869834198237465948319271236841928374651928374651928374651928374651928374651.8675492185478621983746521982374659483192712368419283746519283746519283746519283746519283746519,2947853128374651928374651928374651928374651928374651928374651928374651928374651928374651928375], and then decrypt the fractional part of each encrypted category to obtain [s i = [[s i，0 , [s i,1 ,..., [s i，k ,..., [s i，K , and send it to the aggregation platform;

[0041] S4. The aggregation platform uses the Paillier cryptosystem with threshold decryption to decrypt the scores corresponding to each category of the partially decrypted data sent by all clients to obtain Then, use the softmax function for the probability distribution of each client And output the final inference result y = 2 by designing a fair aggregation mechanism, and then send the result to the task initiator.

[0042] Furthermore, in step S1, the task initiator initializes the Paillier cryptosystem with threshold decryption, selects two large prime numbers p, q, and calculates the integer N = p·q and the generator g = N + 1, and calculates the private key λ = lcm(p - 1, q - 1), where lcm() represents the least common multiple function, and publishes the public key pk = (g, N) of the Paillier cryptosystem with threshold decryption. The task initiator divides the private key into sk 1 and sk 2 , sk 1 + sk 2 = 0 mod λ, sk 1 + sk 2 = 1 mod N, sk 1 is a random number.

[0043] Furthermore, in step S2, the task initiator encrypts the dataset D based on the Paillier cryptosystem with threshold decryption using the public key pk t, obtain D t ←Enc(pk, D t ).

[0044] Furthermore, in step S3, it includes aggregating M clients. The aggregation platform and client i design a secure ReLU protocol (Secure ReLU) based on the Paillier cryptosystem with threshold decryption. The input of the Secure ReLU protocol is the ciphertext x, and the specific steps are as follows:

[0045] S311. The aggregation platform selects a random number r 1 ←{0, 1} σ \{0}, and the random number r 2 ≤N / 2, such that r 1 + r 2 > N / 2, where σ is the security parameter; the aggregation platform flips a random coin to generate π, i.e., π = {0, 1};

[0046] S312. If π = 0, then encrypt r 1 + r 2 using the public key pk based on the Paillier cryptosystem with threshold decryption to obtain r 1 + r 2 ←Enc(pk, r 1 + r 2 ), then partial decryption can be obtained If π = 1, then encrypt r 1 + r 2 using the public key pk based on the Paillier cryptosystem with threshold decryption to obtain r 1 + r 2 ←Enc(pk, r 1 + r 2 ), then

[0047] S313. The aggregation platform uses the partial private key sk 1 and the partial decryption D to obtain the decryption D 1 ←PDec(sk 1 , D), and finally sends the data of D, D 1 , x to client i;

[0048] S314. After receiving the data, client i uses the partial private key sk 2 to perform partial decryption based on the Paillier cryptosystem with threshold decryption to obtain the decryption D 2 ←PDec(sk 2 , D 1 ), and based on D 1, D 2 Decrypt to obtain the intermediate parameter

[0049] S315. If then the intermediate parameter μ 0 ← 1; otherwise, μ 0 ← 0; Client i calculates the intermediate parameter d 0 , that is and sends it to the aggregation platform;

[0050] S316. The aggregation platform calculates the maximum value under the ciphertext to obtain max(x,0) ← x π d 0 1-2π ;

[0051] Furthermore, in the described privacy-preserving multi-party joint inference method, it is characterized in that in step S3, it includes an aggregation platform and M clients. The aggregation platform and client i calculate the encrypted scores for each category based on the scalar multiplication protocol and the secure ReLU protocol of the Paillier cryptosystem with threshold decryption The specific steps are as follows:

[0052] S321. The client receives the encrypted dataset D t and the number of categories C, and client i has the model parameters The client uses the encrypted dataset as the input D t , and the input of the 0th layer is denoted as a 0 = D t ;

[0053] S322. For each layer l = 1 to L, the client locally calculates the intermediate parameter through the Paillier encryption system with a threshold The aggregation platform and the client jointly execute based on the secure ReLU protocol (secure ReLU) to obtain the output of the, layer, where σ is the ReLU function;

[0054] S323. Client i and the aggregation platform jointly decrypt a L by calling the partial decryption function PDec and the threshold decryption function TDec to obtain

[0055] S324. The aggregation platform obtains the probability value p of client i i = softmax(s i ), where

[0056] Further, the multi-party joint inference method for privacy protection is characterized in that, in step S4, the aggregation algorithm includes a maximum value aggregation algorithm and an addition aggregation algorithm, wherein the maximum value aggregation algorithm determines the final inference result y = 2 by calculating the maximum probability value of each category and the addition aggregation algorithm determines the final inference result y = 2 by calculating the total probability value of each category to determine the final inference result y = 2.

[0057] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A privacy-preserving multi-party joint reasoning method, the multi-party joint reasoning method comprising a task initiator, an aggregation platform, and M clients; characterized in that: The method is that the task initiator creates a task containing a data set that needs to be inferred, and then sends the data set that needs to be inferred to the online client through the aggregation platform. The client performs local reasoning to obtain the score of each class and sends it to the aggregation platform. Finally, the aggregation platform designs a fair aggregation protocol to obtain the final reasoning result and sends it to the task initiator. Among them, in order to protect the privacy of the task initiator's reasoning data set, the task initiator uses the Paillier cryptographic system with threshold decryption to protect his own data and send it to the aggregation platform. In order to protect the task initiator's local reasoning data from being leaked, the aggregation platform and the client jointly execute a secure computing protocol to obtain the final reasoning result.

2. A privacy-preserving multi-party joint reasoning method according to claim 1, characterized in that: The following steps are involved: S1. The task initiator initializes the Paillier cryptosystem with threshold decryption, publishes the public key pk of this system, and divides the private key λ into two parts, namely partial private keys sk1 and sk2, and sends them to the aggregation platform and all clients respectively; S2: The task initiator publishes a task, which includes the data set D that needs to be reasoned t , the task initiator uses the Paillier cryptosystem with threshold decryption to encrypt the dataset in Represents the encryption operation, and then the encrypted data set that needs to be inferred Send to aggregation platform; S3. The aggregation platform sends the encrypted data set that needs to be inferred to the online client C = [c1, c2, ..., c i , ..., c M ], where c i Represents client i. Client i performs local ciphertext inference through the designed security algorithm protocol and obtains the score of each encrypted category Where k represents category k, K represents the number of all categories, and then the encrypted fraction of each category is decrypted to obtain [s i ]=[[s i，0 ],[s i，1 ],...,[s i，k ],...,[s i，K ]], sent to the aggregation platform; S4. The aggregation platform uses the Paillier cryptographic system with threshold decryption to decrypt the partially decrypted scores of each category sent by all clients. Then use the softmax function to calculate the probability distribution of each client. And by designing a fair aggregation mechanism, the final reasoning result y is output, and the result is sent to the task initiator.

3. A privacy-preserving multi-party joint reasoning method according to claim 2, characterized in that: In step S1, the task initiator initializes the Paillier cryptosystem with threshold decryption, selects two large prime numbers p and q, and calculates the integer N=p·q and the generator g=N+1, as well as the private key λ=1cm(p-1, q-1), where 1cm() represents the least common multiple function, and discloses the public key pk=(g, N) of the Paillier cryptosystem with threshold decryption. The task initiator divides the private key into sk1 and sk2, sk1+sk2=0modλ, sk1+sk2=1modN, and sk1 is a random number.

4. The privacy-preserving multi-party joint reasoning method according to claim 2, characterized in that: In step S2, the task initiator encrypts the dataset D using the public key pk based on the Paillier cryptosystem with threshold decryption. t ,get 5. A privacy-preserving multi-party joint reasoning method according to claim 2, characterized in that: In step S3, the aggregation platform includes M clients, the aggregation platform and client i design a secure ReLU protocol (secure ReLU) based on the Paillier cryptosystem with threshold decryption. The input of the secure ReLU protocol is the ciphertext The specific steps are as follows: S311, the aggregation platform selects a random number r1←{0, 1} σ \{0}, random number r2≤N / 2, such that r1+r2>N / 2, where σ is a security parameter; the aggregation platform throws a random coin to generate π, that is, π={0,1}; S312, if π=0, then encrypt r1+r2 using the public key pk based on the Paillier cryptosystem with threshold decryption to obtain Then you can get partial decryption If π = 1, then encrypt r1 + r2 using the public key pk based on the Paillier cryptosystem with threshold decryption to obtain Then you can get S313, the aggregation platform uses the partial private key sk1 and the partial decryption D based on the Paillier cryptographic system with threshold decryption to obtain the decryption D1←PDec(sk1,D), and finally D, D1, These data are sent to client i; S314, after receiving the data, client i uses the partial private key sk2 to partially decrypt the data based on the Paillier cryptographic system with threshold decryption, and obtains the decrypted data D2←PDec(sk2, D1), and decrypts the data based on D1 and D2 to obtain the intermediate parameters S315, if Then the intermediate parameter μ0←1; otherwise, μ0←0; client i calculates the intermediate parameter d0, that is And send it to the aggregation platform; S316, the aggregation platform calculates the maximum value under the ciphertext and obtains 6. A privacy-preserving multi-party joint reasoning method according to claim 5, characterized in that: In step S3, the aggregation platform and the M clients calculate the encrypted scores of each category based on the scalar multiplication protocol and the secure ReLU protocol of the Paillier cryptosystem with threshold decryption. The specific steps are as follows: S321. The client receives the encrypted data set and the number of categories C, and client i has the model parameters The client takes the encrypted dataset as input The input of layer 0 is recorded as S322, for each layer l = 1 to L, the client locally calculates the intermediate parameters through the Paillier encryption system with a threshold Aggregation platform and client jointly execute based on secure ReLU protocol To obtain the output of the lth layer, where σ is the ReLU function; S323, client i and aggregation platform jointly decrypt by calling partial decryption function PDec and threshold decryption function TDec get S324: The aggregation platform obtains the probability value p of client i i =softmax(s i ),in 7. The privacy-preserving multi-party joint reasoning method according to claim 2, characterized in that: In step S4, the aggregation algorithm includes a maximum aggregation algorithm and an addition aggregation algorithm, wherein the maximum aggregation algorithm calculates the maximum probability value of each category. To determine the final inference result y, the additive aggregation algorithm calculates the total probability value of each category To determine the final inference result y.

Images