Three-party robust privacy-preserving machine learning training method with one privileged party
Through the vector space secret sharing technology and the design of the public matrix Φ, the problems of privileged party dominance and insufficient data are solved, and the training control when the privileged party is not involved and the training continued after the assistant exits is achieved, ensuring that only the privileged party obtains the results Privacy protection machine learning training method.
Patent Information
- Application Number
- CN202210807840.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-11
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-07-11
Smart Images

Figure CN115348009B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of data security technology, and in particular relates to a three-party robust privacy-preserving machine learning training method with a privileged party. Background Art
[0002] Secure multi-party computing technology[1] enables a group of mutually untrusting parties to securely compute an agreed function without relying on a trusted third party, without leaking original sensitive information other than the result. Privacy-preserving machine learning based on secure multi-party computing allows multiple parties to jointly perform machine learning on their private data while protecting the privacy of the data.
[0003] The current privacy-preserving machine learning framework based on secure multi-party computation usually follows a peer-to-peer architecture. That is, each participant has the same opportunity to process the results, including the trained results and the final model. However, in actual application scenarios, there is usually a powerful party to dominate the business scenario. For example, when a company (privileged party) wants to train a better-performing model but lacks sufficient data, it usually seeks cooperation with other companies (assisting parties). However, the privileged party has two requirements: 1) Without his participation, even if the assisting parties collude with each other, training cannot continue, and only the privileged party can obtain the results. 2) To avoid the training process being terminated due to the disconnection of some assisting parties, even if some assisting parties exit the training, the privileged party and the remaining assisting parties can still continue training and obtain the final results.
[0004] The present invention uses vector space secret sharing technology [2] to meet the above two requirements. is a group of participants, Γ={B0,B1…,B k}, for The access structure on which B j The authorized subset is the secret value that can only be recovered if the participants in the authorized set cooperate with each other. Let p be a large prime number, a positive integer for Suppose there exists a function Satisfy the following properties:
[0005] (1,0,0) can be represented by the set {Φ(P i )|P i ∈B j Linear representation of the elements in}
[0006] Then there exists a set of constants c0,c1,…,c m-1 , where m is B j The number of participants in
[0007]
[0008] Secret distribution phase: The party holding the secret value x generates d-1 random numbers and constructs the vector Then calculate Sent to the corresponding participant P i .
[0009] Secret recovery phase: When an authorized subset B j When you want to recover the secret value x. From the above secret distribution phase, we know but
[0010]
[0011] References
[0012] [1]ACYao, "Protocols for secure computations," 23rd Annual Symposiumon Foundations of Computer Science (sfcs 1982), 1982, pp.160-164.
[0013] [2]Brickell E F. Some ideal secret sharing schemes[C] / / Workshop on the Theory and Application of Cryptographic Techniques. Springer, Berlin, Heidelberg, 1989: 468-475. Summary of the invention
[0014] The purpose of the present invention is to provide a robust privacy-preserving machine learning training method based on vector space secret sharing for a privileged party and two assisting parties, so as to jointly train a machine learning model under the premise that the original data is available but invisible. And two conditions are met: 1) If there is no participation of the privileged party, even if the assisting parties collude with each other, the training cannot continue, and only the privileged party can obtain the results. 2) One of the assisting parties is allowed to be offline during the training process.
[0015] The present invention proposes a three-party robust privacy-preserving machine learning training method with one privileged party, and the specific steps are as follows:
[0016] (1) Three participants determine a public matrix Φ in the vector space secret sharing technology, wherein the three participants are the privileged party P0, the first assisting party P1, and the second assisting party P2; and encrypt the original data using the vector space secret sharing technology, wherein the privileged party holds two copies of the shared value, and each assisting party holds one copy of the shared value;
[0017] (1.1) The privileged party P0, the first assisting party P1, and the second assisting party P2 jointly determine a 4×3 public matrix Φ; for each participant P i (i=0,1,2), the i-th row Φ(i) in the public matrix Φ is the corresponding public vector; in addition, the privileged party P0 additionally has a public vector Φ(3);
[0018] (1.2) The public matrix Φ needs to satisfy the following three conditions:
[0019] ·(1,0,0) can be linearly represented by the public vectors in the set Φ(0),Φ(1),Φ(2), and Φ(0),Φ(1),Φ(2) are linearly independent. Therefore, there are three non-zero constants c0, c1, c2 that satisfy (1,0,0) = c0×Φ(0)+c1×Φ(1)+c2×Φ(2);
[0020] Φ(3) is linearly represented by Φ(1) and Φ(2), that is, there exist non-zero constants a1 and a2 such that Φ(3) = a1×Φ(1) + a2×Φ(2); therefore, (1,0,0) can also be linearly represented by the public vectors in the sets Φ(0), Φ(1), Φ(3) and Φ(0), Φ(2), Φ(3), respectively; thus, there exist non-zero constants c′0, c′1, c′3, c″0, c″2, c″3 such that (1,0,0) = c′0×Φ(0) + c′1×Φ(1) + c′3×Φ(3), (1,0,0) = c″0×Φ(0) + c″2×Φ(2) + c″3×Φ(3);
[0021] (1,0,0) cannot be linearly represented by the public vectors in the sets Φ(0),Φ(3) and Φ(1),Φ(2);
[0022] (2) Design robust computational primitives based on the above encryption scheme to support the training of three commonly used machine learning algorithms: linear regression, logistic regression, and neural network;
[0023] (2.1) If no assisting party is offline:
[0024] (2.1.1) Addition and constant multiplication: Given a constant a and a vector space secret shared value <x>and <y>,get <z> =a× <x> + <y>; Since the above secret sharing satisfies the additive homomorphic property, the addition operation and constant multiplication operation are performed locally; each participant P i Local Computing <z> i =a× <x> i + <y> i , Privileged party P0 additional calculation <z> 3=a× <x> 3+ <y>3;
[0025] (2.1.2) Multiplication operation: Given a vector space secret shared value <x>and <y>,get <z>=<x×y> ; The multiplication operation requires communication between the three parties to complete, and the operation method is similar to the multiplication operation under additive secret sharing; the privileged party P0 needs to calculate additional <z>3. Its calculation method is consistent with other shared values;
[0026] (2.1.3) Comparison operation: given a vector space secret shared value <x>,get <z>=<(x<0)? 1:0>; The comparison operation first converts the three vector space secret sharing values <x> i (i=0,1,2) local multiplication by coefficient c i Convert to additive secret shared value [x] i =c i × <x> i ; Then, the existing comparison operation method based on additive secret sharing is used to calculate [z] = [(x<0)? 1:0]; then the additive secret sharing value [z] i (i=0,1,2) local division by coefficient c i Convert to vector space secret shared value <z> i =[z] i / c i ; In addition, the three participants pre-generate a vector space secret shared value of a random number k <k> j (j=0,1,2,3); assisting parties P1 and P2 calculate <z> 1+ <k> 1, <z> 2+ <k>2, and sent to the privileged party P0; since Φ(3)=a1×Φ(1)+a2×Φ(2), <z> 3=a1× <z> 1+a2× <z> 2, <k> 3=a1× <k> 1+a2× <k>2, then the privileged party P0 can calculate locally <z>3=a1×(<z>1+ <k> 1)+a2×( <z> 2+ <k> 2)- <k>3 to get the additional shared value held <z>3;
[0027] (2.2) If one of the assisting parties is offline:
[0028] (2.2.1) Addition and constant multiplication and multiplication: The privileged party and the remaining assisting parties perform the same operations as if no assisting party was offline;
[0029] (2.2.2) Comparison operation: If P2 is offline, the privileged party P0 calculates [x]0 = c′0× <x>0,[x]3=c′3× <x>3, P1 calculates [x]1 = c′1× <x>1. Convert the vector space secret shared value into an additive secret shared value; then use the existing comparison operation method to calculate [z]0, [z]1, [z]3; finally, the privileged party P0 locally calculates <z>0=[z]0 / c′0, <z>3=[z]3 / c′3, P1 calculation <z>1 = [z]1 / c′1, thereby converting the additive secret shared value back to the vector space secret shared value for subsequent calculations. If P1 is disconnected, the situation is similar to that of P2;
[0030] (3) Restore the training results. Only privileged persons can obtain the final results.
[0031] In the present invention, the original data in the hands of the three participants in step (1) is encrypted using the vector space secret sharing technology, as follows:
[0032] (1) Three Participants P i (i=0,1,2) respectively for the original data d in their hands i , construct a three-dimensional vector Where s1, s2 are random numbers;
[0033] (2) Each participant P i Calculate the original data d held by each i Shared value Privileged party P0 sends <d0> 1, <d0>2 to the assisting parties P1 and P2, and keep it for yourself <d0> 0, <d0>3. The first assisting party P1 sends <d1> 0, <d1>3 To the privileged party P0, send <d1>2 to the second assisting party P2, and keep it yourself <d1>1; The second assisting party P2 sends <d2> 0, <d2>3 To the privileged party P0, send <d2>1 is given to the first assisting party P1, and you keep it <d2>2.
[0034] In the present invention, the training result described in step (3) is restored, and only the privileged party can restore the final result, as follows:
[0035] (1) If no assisting party is offline, the first assisting party P1 and the second assisting party P2 will share their own training results. <r> 1、 <r>2 is sent to the privileged party, and the privileged party recovers the final result r = c0× <r> 0+c1× <r> 1+c2× <r>2;
[0036] (2) If the second assistant P2 is offline, the first assistant P1 will share the shared value of its training result. <r>1 is sent to the privileged party, and the privileged party recovers the final result r = c′0× <r> 0+c′1× <r> 1+c′3× <r>3;
[0037] (3) If the first assisting party P1 is offline, the situation is similar to that of the second assisting party P2 being offline.
[0038] The beneficial effects of the present invention are: without the participation of the privileged party, the other two assisting parties will not be able to conduct training, and even if the two assisting parties collude, they cannot recover the final result; allowing one assisting party to be disconnected during training, the remaining two participating parties can continue training, and only the privileged party can obtain the final result. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 Schematic diagram of the robust privacy-preserving machine learning training architecture based on vector space secret sharing between one privileged party and two assisting parties. DETAILED DESCRIPTION
[0040] The present invention will be described in detail below with reference to the accompanying drawings.
[0041] Example 1: Figure 1 In the example, the privileged party P0 wants to train a machine learning model (linear regression, logistic regression or neural network) with the first assisting party P1 and the second assisting party P2, and two conditions must be met: 1) If there is no participation of the privileged party, even if the assisting parties collude with each other, the training cannot continue, and only the privileged party can get the result. 2) During the training process, even if one assisting party quits the training, the privileged party and the remaining assisting party can still continue the training, and the privileged party will get the final result. In the overall training process, none of the three participants can know the original data of other participants.
[0042] (1) The three participants encrypt their original data through vector space secret sharing technology, where the privileged party holds two shared values and each assisting party holds one shared value.
[0043] (2) The three participants use their shared values as input, and the data in the entire training process exists in the form of secret sharing in the hands of each participant.
[0044] (3) The three participants use the shared values in their hands to call the vector space secret sharing primitives in sequence according to the process of the trained machine learning model, such as addition and constant multiplication primitives, multiplication primitives, and comparison primitives, and gradually complete the training of the machine learning model. During the training process, one assisting party is allowed to be offline.
[0045] (4) After the machine learning training, the assisting party sends its shared value to the privileged party, which then recovers the final result to obtain the trained machine learning model.< / r> < / r> < / r> < / r> < / r> < / r> < / r> < / r> < / r> < / d2> < / d1> < / d0> < / d0> < / z> < / z> < / z> < / x> < / x> < / x> < / z> < / k> < / k> < / z> < / k> < / z> < / k> < / k> < / k> < / z> < / z> < / z> < / k> < / z> < / k> < / z> < / k> < / z> < / x> < / x> < / z> < / x> < / z> < / z> < / y> < / x> < / y> < / x> < / z> < / y> < / x> < / z> < / y> < / x> < / z> < / y> < / x>
Claims
1. A three-party robust privacy-preserving machine learning training method with one privileged party, characterized in that The specific steps are: (1) using three participants to determine a public matrix Φ in the vector space secret sharing technology, wherein the three participants are the privileged party P0, the first assisting party P1, and the second assisting party P2; and encrypting the original data using the vector space secret sharing technology, wherein the privileged party holds two copies of the shared value, and each assisting party holds one copy of the shared value; (1.1) The privileged party P0, the first assisting party P1, and the second assisting party P2 jointly determine a 4×3 public matrix Φ; for each participant P i (i=0,1,2), the i-th row Φ(i) in the public matrix Φ is the corresponding public vector; in addition, the privileged party P0 has an additional public vector Φ(3); (1.2) The public matrix Φ needs to satisfy the following three conditions: (1,0,0) is linearly represented by the public vectors in the set Φ(0),Φ(1),Φ(2), and Φ(0),Φ(1),Φ(2) are linearly independent; therefore, there are three non-zero constants c0, c1, c2 that satisfy (1,0,0) = c0×Φ(0)+c1×Φ(1)+c2×Φ(2); Φ(3) is linearly represented by Φ(1) and Φ(2), that is, there exist non-zero constants a1 and a2 such that Φ(3) = a1×Φ(1) + a2×Φ(2); therefore, (1,0,0) is also linearly represented by the public vectors in the sets Φ(0), Φ(1), Φ(3) and Φ(0), Φ(2), Φ(3), respectively; thus, there exist non-zero constants c′0, c′1, c′3, c″0, c″2, c″3 such that (1,0,0) = c′0×Φ(0) + c′1×Φ(1) + c′3×Φ(3), (1,0,0) = c″0×Φ(0) + c″2×Φ(2) + c″3×Φ(3); (1,0,0) cannot be linearly represented by the public vectors in the sets Φ(0),Φ(3) and Φ(1),Φ(2); (2) Design robust computational primitives based on the above encryption scheme to support the training of three commonly used machine learning algorithms: linear regression, logistic regression, and neural network; (2.1) If no assisting party is offline: (2.1.1) Addition and constant multiplication: Given a constant a and a vector space secret shared value <x>and <y>,get <z> =a× <x> + <y>; Since the above secret sharing satisfies the additive homomorphic property, the addition operation and constant multiplication operation are directly performed locally; each participant P i Local Computing <z> i =a× <x> i + <y> i , Privileged party P0 additional local calculation <z> 3=a× <x> 3+ <y> 3;< / y> < / x> < / z> < / y> < / x> < / z> < / y> < / x> < / z> < / y> < / x> (2.1.2) Multiplication operation: Given a vector space secret shared value <x>and <y>,get <z>=<x×y> ; The multiplication operation requires communication between the three parties to complete, and the operation method is similar to the multiplication operation under additive secret sharing; the privileged party P0 needs to calculate additional <z> 3. Its calculation method is consistent with other shared values;< / z> < / z> < / y> < / x> (2.1.3) Comparison operation: given a vector space secret shared value <x>,get <z>=<(x<0)? 1:0>; The comparison operation first converts the three vector space secret sharing values <x> i (i=0,1,2) local multiplication by coefficient c i Convert to additive secret shared value [x] i =c i × <x> i ; Then, the existing comparison operation method based on additive secret sharing is used to calculate [z] = [(x<0)? 1:0]; then the additive secret sharing value [z] i (i=0,1,2) local division by coefficient c i Convert to vector space secret shared value <z> i =[z] i / c i ; In addition, the three participants pre-generate a vector space secret shared value of a random number k <k> j (j=0,1,2,3); assisting parties P1 and P2 calculate <z> 1+ <k> 1, <z> 2+ <k>2, and sent to the privileged party P0; since Φ(3)=a1×Φ(1)+a2×Φ(2), <z> 3=a1× <z> 1+a2× <z> 2, <k> 3=a1× <k> 1+a2× <k>2, then the privileged party P0 can calculate locally <z> 3=a1×( <z> 1+ <k> 1)+a2×( <z> 2+ <k> 2)- <k>3 to get the additional shared value held <z> 3;< / z> < / k> < / k> < / z> < / k> < / z> < / z> < / k> < / k> < / k> < / z> < / z> < / z> < / k> < / z> < / k> < / z> < / k> < / z> < / x> < / x> < / z> < / x> (2.2) If one of the assisting parties is offline: (2.2.1) Addition and constant multiplication and multiplication: The privileged party and the remaining assisting parties perform the same operations as if no assisting party was offline; (2.2.2) Comparison operation: If P2 is offline, the privileged party P0 calculates [x]0 = c′0× <x>0,[x]3=c′3× <x>3, P1 calculates [x]1 = c′1× <x>1. Convert the vector space secret shared value into an additive secret shared value; then use the existing comparison operation method to calculate [z]0, [z]1, [z]3; finally, the privileged party P0 locally calculates <z>0=[z]0 / c′0, <z>3=[z]3 / c′3, P1 calculation <z> 1 = [z]1 / c′1, thereby converting the additive secret shared value back to the vector space secret shared value for subsequent calculations; if P1 is disconnected, the situation is similar to that of P2;< / z> < / z> < / z> < / x> < / x> < / x> (3) Restore the training results. Only privileged persons can obtain the final results.
2. The three-party robust privacy-preserving machine learning training method with one privileged party according to claim 1, characterized in that: The original data in the hands of the three participants described in step (1) is encrypted using the vector space secret sharing technology, as follows: (1) Three Participants P i (i=0,1,2) respectively for the original data d in their hands i , construct a three-dimensional vector Where s1, s2 are random numbers; (2) Each participant P i Calculate the original data d held by each i Shared value Privileged party P0 sends <d0> 1, <d0>2 to the assisting parties P1 and P2, and keep it for yourself <d0> 0, <d0>3. The first assisting party P1 sends <d1> 0, <d1>3 To the privileged party P0, send <d1>2 to the second assisting party P2, and keep it yourself <d1>1; The second assisting party P2 sends <d2> 0, <d2>3 To the privileged party P0, send <d2>1 is given to the first assisting party P1, and you keep it <d2> 2。< / d2> < / d2> < / d1> < / d0> < / d0> 3. The three-party robust privacy-preserving machine learning training method with one privileged party according to claim 1, characterized in that: The training results described in step (3) are restored, and only privileged parties can restore the final results, as follows: (1) If no assisting party is offline, the first assisting party P1 and the second assisting party P2 will share their own training results. <r> 1、 <r>2 is sent to the privileged party, and the privileged party recovers the final result r = c0× <r> 0+c1× <r> 1+c2× <r> 2;< / r> < / r> < / r> < / r> < / r> (2) If the second assistant P2 is offline, the first assistant P1 will share the shared value of its training result. <r>1 is sent to the privileged party, and the privileged party recovers the final result r = c′0× <r> 0+c′1× <r> 1+c′3× <r> 3;< / r> < / r> < / r> < / r> (3) If the first assisting party P1 is offline, the situation is similar to that of the second assisting party P2 being offline.
Citation Information
Patent Citations
Privacy protection linear regression method based on secret sharing and random disturbance
CN113065145A
Key-value pair model safety training and reasoning method based on safety multi-party calculation
CN113535808A
Privacy-preserving machine learning in the three-server model
US20220092216A1
Multi-party computation services
US9577829B1
Privacy-preserving machine learning
WO2018174873A1
