A generic architecture and system for privacy protection

Abstract

This invention provides a general architecture and system for protecting privacy. The general architecture includes a secure computation server and several participating servers, each pre-configured with the same random number generator. The random number generator generates random numbers for each corresponding participating server. The specific steps for the participating servers to perform federated computation are as follows: determining the application scenario of the federated computation; determining a masking scheme based on the application scenario of the participating servers; the masking scheme includes a joint addition protocol, a joint multiplication protocol, a joint division protocol, a joint exponentiation protocol, and a joint comparison protocol. Based on these basic protocols, secure computation schemes for various activation functions are designed, along with a horizontal federated algorithm and a vertical linear regression algorithm. This invention solves the problem of limited adaptability of existing privacy-preserving general architectures to various scenarios.

Description

Technical Field

[0001] This invention relates to the field of data security and privacy technology, and in particular to a general architecture and system for protecting privacy. Background Technology

[0002] With the rapid development of AI, data is playing an increasingly prominent role as a new factor of production. At the same time, various laws and regulations concerning data security and privacy protection have been introduced both domestically and internationally. How to use data securely, discreetly, and compliantly is a problem that needs to be addressed. Privacy-preserving computing offers a technological solution.

[0003] Existing general privacy-preserving architectures typically consist of participant servers and cryptographic computation servers. Participant servers mask the data before passing it to the cryptographic computation server for processing. The cryptographic computation server then returns the result to the participant servers for demasking to obtain the final result. This architecture can be used for multi-party secure computation, federated learning, and federated data analysis. However, current masking mechanisms cannot solve the computation problems of certain activation functions and still pose a risk of information leakage when the number of participant servers is large.

[0004] There is currently no effective solution to the problem that the general privacy architecture in existing related technologies has limited adaptability to various scenarios. Summary of the Invention

[0005] This invention provides a general architecture and system for protecting privacy, which addresses the shortcomings of existing general privacy architectures in terms of limited adaptability to various scenarios.

[0006] In a first aspect, the present invention provides a general architecture for protecting privacy, comprising: a secret-state computation server and several participating server servers, each of which is pre-configured with the same random number generator; The random number generator is used to generate random numbers for each corresponding participating server; The specific steps taken by the participating server during federated computation are as follows: Identify the application scenarios for federated computing; The masking scheme is determined based on the application scenario of the participating server; the masking scheme includes a joint addition protocol, a joint multiplication protocol, a joint division protocol, a joint exponentiation protocol, and a joint comparison protocol; The random number is selected and the data of the participating server is masked according to the determined masking scheme, and federated computation is performed.

[0007] According to a general privacy-preserving architecture provided by the present invention, when the participating server performs horizontal federated learning or vertical linear regression algorithm, the masking scheme is determined to be a joint addition protocol; When the participating server performs deep neural network fully connected layer or deep neural network convolution calculation, the masking scheme is determined to be a joint multiplication protocol; When the participating server performs activation function calculation for a deep neural network, the masking scheme comprehensively uses a joint addition protocol, a joint multiplication protocol, a joint division protocol, a joint exponentiation protocol, or a joint comparison protocol.

[0008] According to a general privacy-protecting architecture provided by the present invention, when the masking scheme of the participating server is determined to be the joint addition protocol: The data of the participating server is summed, and the summation result is arithmetically sharded. The participating server generates a mask and assigns the mask to the data of the participating server to obtain the target data; The dense state computing server sums all the target data, performs arithmetic sharding on the summation result, and sends the sharding result to each of the participating server.

[0009] According to a general privacy-preserving architecture provided by the present invention, when the masking scheme of the participating server is determined to be the federated multiplication protocol: The data from the participating party servers are jointly calculated, and the calculation results are arithmetically sharded and sent to each of the participating party servers. Each participating server generates a corresponding number of masks based on the number of participating servers, and assigns the masks to the data of the participating server to obtain the target data; The participating server sends the target data to the secret state calculation server; The dense state computing server performs joint processing on all target data, performs arithmetic sharding on the processing results, and sends the sharding results to each of the participating server parties.

[0010] According to a general privacy-protecting architecture provided by the present invention, when the masking scheme of the participating server is determined to be the joint division protocol: The data from the participating server is jointly calculated, and the calculation results are arithmetically sharded. The participating server generates a random number and assigns the random number to the data of the participating server to obtain the target data; The participating server sends the target data to the secret state computing server; The dense state computing server generates intermediate processing results based on the target data, then segments the intermediate processing results and sends them to the corresponding participating server.

[0011] According to a general privacy-preserving architecture provided by the present invention, when the masking scheme of the participating server is determined to be the joint index protocol: The data from the participating party servers are jointly calculated, and the calculation results are arithmetically fragmented and then sent to each of the participating party servers. The participating server generates a random number, performs exponential processing on the random number, and sends it to the secret state calculation server. The dense state computing server generates intermediate data based on the exponential processing result, and then sends the intermediate data to each of the participating server after performing arithmetic sharding.

[0012] According to a general privacy-protecting architecture provided by the present invention, when the masking scheme of the participating server is determined to be the joint comparison protocol: Each of the participating servers generates the same random number mask; The participating server processes the data using the random number mask according to its own preset processing rules, and sends the processing result to the secret state calculation server. The dense state computing server performs data summation on the received processing results and compares the data based on the positive or negative value of the summation results to obtain a comparison result.

[0013] According to a privacy-preserving general architecture provided by the present invention, when the federated addition protocol is applied to the horizontal federated learning scenario: Each participating server masks its own model gradient with the corresponding random number. Each participating server generates several identical random numbers and determines a verification value based on the random numbers and the gradient value of the model gradient. The masked model gradient and the verification value are sent to the aggregation server for aggregation processing to obtain the aggregated value, and the aggregated value is returned to each of the participating party servers. Each participating server redetermines the checksum based on the aggregated value and the sum of all masks; When the joint addition protocol is applied to the longitudinal linear regression algorithm: Both the passive party and the active party with tags in the participating party server generate random numbers; Both the active party and the passive party process the random number to obtain dense state data and send the dense state data to the dense state calculation server. The dense state calculation server sums the received dense state data and sends the summation result to the passive party and the active party; The passive party and the active party perform demasking on the received summation result and update the model gradient based on the processing result.

[0014] According to a privacy-preserving general architecture provided by the present invention, the joint multiplication protocol is applied to fully connected layers or convolutional computations in deep neural networks: The participating server masks the matrix that needs to be interactively multiplied using the random number and then sends it to the secret state calculation server. The dense state computing server segments the masking results based on matrix operation rules and multiplication protocols, and sends the segmentation results to the participating server. The participating server performs demasking on the received sharding results and performs addition and aggregation to determine the sharding of the product matrix; When the joint comparison protocol is applied to the activation function calculation of a deep neural network: For an activation function in the form of a piecewise function, or an activation function that can be approximated by a piecewise function, each of the participating servers generates the same random number mask, processes the random number mask, and sends the data processing result to the secret state calculation server. The dense state calculation server sums the received processing results and performs a joint comparison based on the sign of the summation results to obtain a joint comparison result. Based on the joint comparison result, it selects different piecewise functions for processing. For the sigmoid function, the participating server generates a random number, masks the random number, and then sends it to the secret state calculation server. The dense state calculation server performs calculations on the received mask processing results to obtain intermediate results; The dense state computing server performs arithmetic fragmentation on the intermediate results and then sends them to the participating server. The participating server performs demasking on the sharding results; For Gelu functions, the arithmetic slices of the subfunction are determined based on the input of the subfunction; The product of the arithmetic pieces is determined based on the joint multiplication protocol, and the product calculation result is obtained.

[0015] Secondly, the present invention also provides a general system for protecting privacy, comprising: a secret state calculation server and several participating server servers, each of which is pre-configured with the same random number generator; The random number generator is used to generate random numbers for each corresponding participating server; The specific steps taken by the participating server during federated computation are as follows: Identify the application scenarios for federated computing; The masking scheme is determined based on the application scenario of the participating server; the masking scheme includes a joint addition protocol, a joint multiplication protocol, a joint division protocol, a joint exponentiation protocol, and a joint comparison protocol; The random number is selected and the data of the participating server is masked according to the determined masking scheme, and federated computation is performed.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The privacy-preserving general architecture provided by this invention, by adjusting the appropriate masking scheme according to different application scenarios, is more suitable for federated computing scenarios with various activation functions, and can reduce the possibility of information leakage to a certain extent, thus solving the problem of limited adaptability of the privacy general architecture in existing related technologies. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of a general architecture for protecting privacy in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0020] This invention provides a general architecture for protecting privacy, comprising: a secure computation server and several participating servers, each of which is pre-configured with a random number generator. The random number generator is used to generate random numbers for each corresponding participating server, and the random number generators of each participating server are identical, with the same initial seed.

[0021] The specific steps taken by the participating servers during federated computation are as follows: Step S101: Determine the application scenarios for federated computing; Step S102: Determine the masking scheme based on the application scenario of the participating server; the masking scheme includes joint addition protocol, joint multiplication protocol, joint division protocol, joint exponentiation protocol and joint comparison protocol; Step S103: Select a random number and mask the data of the participating server according to the determined masking scheme, and perform federated computation.

[0022] This architecture consists of a dense computing server and several participating party servers, such as Figure 1 As shown, Figure 1 This is a schematic diagram of a general privacy-protecting architecture in an embodiment of the present invention, which includes a secret-state computation server P0 and n participating party servers P. i Let ...

[0023] In some of these embodiments, the masking scheme is determined to be a joint addition protocol when the participating server performs horizontal federated learning or vertical linear regression algorithms.

[0024] Based on this embodiment, when the masking scheme of the participating server is determined to be a joint addition protocol: the data of the participating server is summed, and the summation result is arithmetically fragmented; the participating server generates a mask and assigns the mask to the data of the participating server to obtain the target data; the secret state calculation server sums all the target data, performs arithmetic fragmentation on the summation result, and sends the fragmentation result to each participating server.

[0025] For example, assume each participating server P i There is data They need to jointly calculate The results are then arithmetically partitioned and distributed to each participant to facilitate further computation. Each participant generates n masks. And according to its own server number i Send to the dense-state computation server. The dense-state server performs the computation. And perform arithmetic partitioning on s: The data is then sent to each participating server, and each participating server calculates its own value. = ,i=1,...,n.

[0026] When the joint multiplication protocol is applied to a horizontal federated learning scenario: each participating server masks its own model gradient with the corresponding random number; each participating server generates several identical random numbers and determines a check value based on the random numbers and the gradient values ​​of the model gradient; the masked model gradient and check value are sent to the aggregation server for aggregation processing to obtain the aggregated value, and the aggregated value is returned to each participating server; each participating server redetermines the check value based on the aggregated value and the sum of all masks.

[0027] Specifically, in a horizontal federation, a joint addition protocol can be used to calculate the sum of gradients. Each participating server is configured with a random number generator and the same initial seed. Each participating server runs a machine learning algorithm locally and masks its own model gradient with a random number matching its own ID, i.e.:

[0028] in, Let represent the j-th gradient of participant server i, and This represents the mask generated by the random number generator for this gradient. Then, each participating server generates several identical random numbers, and selects several gradient values ​​based on these random numbers, performing arithmetic summation as the check value. That is, participating server i calculates... = The random number determines which gradients are selected to participate in the checksum calculation. The masked gradients and checksums are then sent to the aggregation server. The aggregation server aggregates the masked gradients and checksums from each party according to the horizontal federation aggregation rules and returns them to each participating server. Each participating server subtracts the sum of all random masks from the aggregated value to obtain the aggregated plaintext gradient. Then, based on the random number used to generate the checksum, it selects the corresponding aggregated gradient to recalculate the checksum and compares it with the received checksum to detect whether the aggregation server has tampered with the gradient value. If the checksum passes verification, the aggregated gradient value is used to update the local model parameters. Alternatively, the checksum can be masked for protection, similar to gradient masking, such as using the parity of the generated random number to determine whether the checksum sign changes, etc., which will not be elaborated further. In this scheme, as long as the random mask is secure, entities outside the participating servers can only access the random number as the mask, so the gradients of each participating server are secure. To prevent attacks from between participating servers, the security of the masked gradient ciphertext needs to be protected. To address this, different encrypted channels are used between the participating servers and the aggregation server to prevent participating parties from obtaining the gradient ciphertext of other participating parties. The aggregation server runs within a trusted execution environment to prevent it from sending the masked ciphertext to some participating servers, thus avoiding collusion attacks. By designing a random number mechanism with a distribution significantly different from the gradient distribution, this scheme can detect which participating server has gone offline by analyzing the gradient distribution after demasking. After subtracting the random number mask of that participating server, the system can continue operating; that is, the scheme can tolerate participating server outages.

[0029] When the joint addition protocol is applied to the longitudinal linear regression algorithm: both the passive party and the labeled active party in the participating server generate random numbers; both the active and passive parties process the random numbers to obtain dense-state data and send the dense-state data to the dense-state calculation server; the dense-state calculation server sums the received dense-state data and sends the summation result to the passive and active parties; the passive and active parties demask the received summation result and update the model gradient based on the processing result.

[0030] The loss function calculation of the longitudinal linear regression algorithm can be decomposed into a locally computeable part and a part requiring interactive computation. The interactive computation part is implemented using joint multiplication in vector form.

[0031] Specifically, in the vertical federated linear regression scenario, each passive party and the labeled active party generate random numbers. n. Active side calculation + Passive side calculation + , n-1, and sent to the dense state computing server P0, where It is the sum of the results calculated by the passive party i's model based on m data samples, while It is the sum of the results calculated by the active party n's model based on m data samples. These are the labels for the active data sample j. and These are the model parameters for the passive party i and the active party n, respectively. Let be the attribute vectors of data sample j for passive party i and active party n, respectively, i=1,2,...,n-1. The dense-state computation server P0 receives the dense-state data from each passive and active party, sums them, and then sends the summed result back to each passive and active party. The masks are then removed from the data of each passive and active party. The model parameters are then calculated and updated using the results obtained, following the gradient calculation method for linear regression.

[0032] From the above process, it can be seen that when the number of samples updated in one go is large enough, or the total number of participants is greater than 2, the arithmetic sum after two or three information compressions does not lead to the leakage of training data, labels, or model parameters to other parties. For ease of description, we will take the calculation of the loss function involving two parties as an example. The calculation of the loss function can be divided into the part that each participant can calculate locally and the part that needs to be calculated interactively. The interactive part can be defined as follows: This can be viewed as an inner product operation of two vectors, which can be performed using the method for calculating the product term in matrix multiplication, except that the results obtained from each side... After receiving its own fragment, the passive party sends its fragment to the active party, and the active party adds the two fragments together to obtain the desired result. When there are multiple participating servers, the process for each participating server is similar. The dense state computation server P0 can calculate the product term by combining the participating servers in pairs, which will not be elaborated here.

[0033] In some of these embodiments, when the participating server performs fully connected layers or convolutional computations in a deep neural network (DNN), the masking scheme is determined to be a joint multiplication protocol.

[0034] Based on this embodiment, when the masking scheme of the participating server is determined to be a joint multiplication protocol: joint calculation is performed on the data of the participating server, the calculation result is arithmetically fragmented and sent to each participating server; each participating server generates a corresponding number of masks according to the number of participating servers, and assigns the masks to the data of the participating server to obtain the target data; the participating server sends the target data to the secret-state calculation server; the secret-state calculation server performs joint processing on all target data, performs arithmetic fragmentation on the processing result, and sends the fragmentation result to each participating server.

[0035] For example, suppose there are n participating servers, and suppose each participating server i has data. They need to jointly calculate The results are then arithmetically partitioned and distributed to each participating server to facilitate further computation. Each participating server calculates n masks. And according to its own server number i Send to the dense-state computation server. The dense-state server calculates p. And perform arithmetic partitioning on p: The data is then sent to each participating server. Upon receiving the fragment, server i calculates... ,i=1,...,n. This is a new method for product calculation that does not utilize Beaver triples, and it can support product calculation of any number of numbers. In contrast, the multiplication scheme using Beaver triples only supports the product of two numbers, and if you want to multiply multiple data, you need to execute the Beaver process multiple times.

[0036] When the joint multiplication protocol is applied to fully connected layers or convolutional computations in deep neural networks: the participating server masks the matrix to be interactively multiplied using random numbers and sends it to the dense computation server; the dense computation server segments the masking result based on matrix operation rules and the multiplication protocol, and sends the segmented result to the participating server; the participating server demasks the received segmented result and performs summation to determine the segmentation of the product matrix.

[0037] Specifically, in DNNs based on multiparty computation (MPC), there are multiplication operations involving two-way data fragments, such as those in fully connected layers and convolutional layers. Traditionally, these operations require the use of Beaver triples. In this embodiment, the fragmented multiplication operation can be divided into a locally computeable part and an interactive part. Assume that the matrices that P1 and P2 need to perform interactive multiplication are denoted as X and Y. P1 sends each element of the involved matrix to the dense computation server P0 after masking the multiplication with different generated random numbers. P2 sends each element of the involved matrix to the aggregation server P0 after masking it with different generated random numbers. P0 calculates the product of each pair of elements according to matrix operation rules and sends the result fragments to P1 and P2 according to the multiplication protocol. P1 and P2 remove the corresponding masks from each received element according to the joint multiplication protocol and then add and aggregate related terms according to the matrix multiplication rules to obtain a fragment of one element of the product matrix. After all elements have been processed, P1 and P2 each obtain their fragments of the product matrix. The communication volume of this algorithm is size(size of P1 matrix mn) + size(size of P2 matrix nv) + size(2mnv), requiring only one round of communication, and there is no need to generate Beaver triples offline or online.

[0038] In some of these embodiments, when the participating server performs activation function calculations for a deep neural network, the masking scheme uses a combination of joint addition protocols, joint multiplication protocols, joint division protocols, joint exponentiation protocols, or joint comparison protocols.

[0039] Based on this embodiment, when the masking scheme of the participating server is determined to be a joint division protocol: the participating server generates a random number and assigns the random number to the data of the participating server to obtain the target data; the participating server sends the target data to the secret state calculation server; the secret state calculation server generates intermediate data based on the target data, and sends the intermediate data to each participating server after fragmenting it.

[0040] For example, suppose there are two participating servers, and suppose each participating server i has data. x= They need to jointly calculate y The results are then arithmetically sharded and distributed to the participating servers to facilitate further calculations. P1 generates random numbers a, b, and c, and calculates a. and put a The integers z=b+c are sent to the dense-state computation server. P2 generates random numbers a, b, and c, and calculates a. c, and put a Send to the secret-state computation server. The secret-state server calculates ax. a +a -z, generates a random number r, and combines r with... Send the data to P1 and P2 respectively. P1 and P2 calculate ar and 1 / x-ar respectively.

[0041] When the masking scheme of the participating server is determined to be the joint exponentiation protocol: the participating server generates a random number, performs exponential processing on the random number, and sends it to the secret state calculation server; the secret state calculation server generates intermediate data based on the exponential processing result, performs arithmetic fragmentation on the intermediate data, and sends it to each participating server.

[0042] Assume there are two participating servers, and each participating server i has data. ,x= They need to jointly calculate y The results are then arithmetically partitioned and distributed to the participating servers to facilitate further calculations. P1 generates random numbers a and b, and calculates... Then send it to P0. P2 generates random numbers a and b, and calculates... Then it is sent to P0. The secret server P0 calculates... Regenerate random number r, and combine r with... -r is sent to P1 and P2 respectively. P1 and P2 calculate r / , -r / .

[0043] When the masking scheme of the participating servers is determined to be a joint comparison protocol: for activation functions in the form of piecewise functions, or activation functions that can be approximated by piecewise functions, each participating server generates the same random number mask; the participating servers process the data using the random number mask according to their respective preset processing rules, and send the processing results to the secret-state computation server; the secret-state computation server performs data summation on the received processing results, and compares the data according to the positive or negative value of the summation results to obtain the joint comparison result, and selects different piecewise functions for processing based on the joint comparison result.

[0044] Activation functions used in neural networks range from simple ones like ReLU, LReLU, PreLU, ELU, and SELU to more complex ones like sigmoid, tanh, geLU, and the Mish function. Many papers fit complex activation functions using piecewise polynomial functions. Therefore, activation functions require comparison operations involving sliced ​​data and a constant c. The private compare method from the SecureNN paper can be used, but it requires permutation operations on a set of k (k=64) elements for each input data point to the activation function, resulting in significant computational overhead. When the joint comparison protocol is applied to activation function computation in deep neural networks: each participating server generates the same random number mask, processes the random number mask, and sends the processing results to the secret-state computation server; the secret-state computation server sums the received processing results and performs a joint comparison based on the sign of the sum, obtaining the joint comparison result.

[0045] Specifically, this embodiment designs the following scheme: Two servers, P1 and P2, each generate the same random number mask a>0. One of the servers, for example, P1, calculates X= -c), while P2 is calculated All of these data are sent to P0. P0 sums the received data to obtain a(xc), and determines whether x > c based on the sign of this value. It should be noted that although this mask obscures the specific value of x, it does leak the range information of x. However, other protection measures will be provided later based on the application scenario.

[0046] Simple activation functions like ReLU can be designed directly based on joint comparison protocols and basic arithmetic partitioning rules. The following example uses SeLU, a slightly more complex function, to illustrate the process. Although SeLU uses an exponential function and appears complex, it can still be designed using comparison protocols and basic arithmetic partitioning rules. The SeLU function is defined as follows: P1, P2, and P0 run a comparison protocol to determine... In this case, P1 and P2 generate random number masks a and b respectively, and then mask the fragments accordingly. +a, +b is sent concurrently to P0. If P0 calculation Calculate x + a + b = x + a + b. Then generate random numbers. Send to P1, - Send to P2. P1 and P2 calculate respectively. -1), - Otherwise, P0 generates random numbers. Send to P1, x+a+b- Send to P2. P1 and P2 calculate respectively. ( -a), (x+a+b- -b). ReLU, LRELU, PRELU, and ELU can all be handled similarly, and will not be elaborated here. Similarly, since the interval information of whether x is greater than 0 is leaked, random permutation is also needed to change the position of the activation function input.

[0047] The activation functions sigmoid and tanh can be calculated as follows: the participating server generates a random number, masks the random number, and sends it to the secret-state computation server; the secret-state computation server performs calculations on the received masking result to obtain an intermediate result; the secret-state computation server performs arithmetic fragmentation on the intermediate result and sends it to the participating server; the participating server demasks the fragmented result.

[0048] Specifically, since tanh(x) = 2sigmoid(2x) - 1, we will take sigmoid as an example, where Sigmoid(x) = P0 possesses an additive homomorphic cryptographic algorithm, such as the Paillier algorithm, and publishes its public key for encryption. P1 has... P2 has Two servers simultaneously generate random numbers a and b. P1 calculates... And calculate c= using P0's public key. ab Then Send c to P0, and send c to P2. P2 calculates b. And calculate d= using P0's public key. ab Calculate cd, then b cd is sent to P0. P0 decrypts cd using its private key to obtain ab. +ab and calculate ( (b) ab ab =ab b. Generate a random number r and calculate... Then r and Send the message to P1 and P2 respectively. P1 and P2 calculate abr respectively. This completes the calculation of the sigmoid activation function. Tanh(x) can be calculated similarly, and will not be elaborated here.

[0049] The computation of complex activation functions like Gelu or Mish can be performed as follows: Assume these activation functions have been approximated using piecewise polynomials. First, according to the joint comparison protocol, P0 determines which interval x belongs to, and thus selects the polynomial f(x) for the corresponding interval. P1 and P2 then generate masks m and compute m respectively. ,m The data is then sent to P0. P0 recovers mx, calculates its powers, and then performs arithmetic partitioning on each power, distributing these partitions to P1 and P2. P1 and P2 demask the received power partitions and calculate their own f(x) partition values ​​according to the definition of f(x). Although x is masked during the comparison protocol to hide its specific value, its interval information is still leaked. This interval information is equivalent to quantizing x, and an attacker might use these quantized intermediate values ​​to build a new model that mimics the model trained by P1 and P2. To resist this potential attack, P1 and P2 can use random permutations to change the position of the activation function input. Since random permutation masking is a known technique, it will not be elaborated further.

[0050] Piecewise polynomial approximation inevitably introduces some errors. In scenarios requiring high accuracy, a stepwise decomposition method can be used to compute activation functions like Gelu or Mish. Taking Gelu as an example, firstly, the arithmetic slices of the subfunction are determined based on its input; then, the product of the arithmetic slices is determined based on a joint multiplication protocol, yielding the product calculation result.

[0051] Specifically, the Gelu function is defined as follows: or For ease of description, this embodiment uses... Taking this as an example, the first expression can be described similarly. The main idea is to decompose the function into polynomials of subfunctions, first calculating the slices of each subfunction, and then calculating the slices of the function. Specifically, first, based on the arithmetic slices of x... and Local computation of 1.702x arithmetic partitioning 1.702 and 1.702 Then, the sigmoid function piecewise calculation method is used to calculate... Arithmetic partitioning, denoted as Then, the joint multiplication protocol is used to calculate the product of the two variables in pieces, that is, to calculate using piecewise multiplication. ( ). ( ( )= + + + ,in Each can be calculated locally. + The aforementioned federated multiplication protocol can be used, thus solving the problem of calculating Gelu(x).

[0052] Since the basic algorithms for addition and multiplication operations have been given in the above embodiments, various common activation functions can be calculated by using these basic operations through a stepwise compounding method.

[0053] The present invention also provides a general system for protecting privacy. The general system for protecting privacy provided by the present invention will be described below. The general system for protecting privacy described below can be referred to in correspondence with the general architecture for protecting privacy described above. The system includes: a secret state computing server and several participating server servers, each of which is pre-configured with the same random number generator. The random number generator is used to generate random numbers for each corresponding participating server; The specific steps taken by the participating servers during federated computation are as follows: Identify the application scenarios for federated computing; The masking scheme is determined based on the application scenario of the participating servers; the masking scheme includes joint addition protocol, joint multiplication protocol, joint division protocol, joint exponentiation protocol and joint comparison protocol; Select random numbers and mask the data of the participating servers according to the determined masking scheme, and then perform federated computation.

[0054] This system consists of a dense-state computation server and several participating party servers, such as... Figure 1 As shown, Figure 1 This is a schematic diagram of a general privacy-protecting architecture in an embodiment of the present invention, which includes a secret-state computation server P0 and n participating party servers P. i Let ...

[0055] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A generic architecture for privacy protection, characterized in that, include: The system comprises a secret-state computing server and several participating server servers, each of which is pre-configured with the same random number generator; The random number generator is used to generate random numbers for each corresponding participating server; The specific steps taken by the participating server during federated computation are as follows: Identify the application scenarios for federated computing; The masking scheme is determined based on the application scenario of the participating server; the masking scheme includes a joint addition protocol, a joint multiplication protocol, a joint division protocol, a joint exponentiation protocol, and a joint comparison protocol; The random number is selected and the data of the participating server is masked according to the determined masking scheme, and federated computation is performed.

2. The privacy-preserving general architecture of claim 1, wherein, When the participating server performs horizontal federated learning or vertical linear regression algorithms, the masking scheme is determined to be a joint addition protocol; When the participating server performs deep neural network fully connected layer or deep neural network convolution calculation, the masking scheme is determined to be a joint multiplication protocol; When the participating server performs activation function calculation for a deep neural network, the masking scheme comprehensively uses a joint addition protocol, a joint multiplication protocol, a joint division protocol, a joint exponentiation protocol, or a joint comparison protocol.

3. The privacy-preserving general architecture of claim 2, wherein, When the masking scheme of the participating server is determined to be the joint addition protocol: The participating server generates a mask and assigns the mask to the data of the participating server to obtain the target data; The dense state computing server sums all the target data, performs arithmetic sharding on the summation result, and sends the sharding result to each of the participating server.

4. The general architecture for protecting privacy according to claim 2, characterized in that, When the masking scheme of the participating server is determined to be the joint multiplication protocol: Each participating server generates a corresponding number of masks based on the number of participating servers, and assigns the masks to the data of the participating server to obtain the target data; The participating server sends the target data to the secret state calculation server; The dense state computing server performs joint processing on all target data, performs arithmetic sharding on the processing results, and sends the sharding results to each of the participating server parties.

5. The general architecture for protecting privacy according to claim 2, characterized in that, When the masking scheme of the participating server is determined to be the joint division protocol: The participating server generates a random number and assigns the random number to the data of the participating server to obtain the target data; The participating server sends the target data to the secret state computing server; The dense state computing server generates intermediate processing results based on the target data, then segments the intermediate processing results and sends them to the corresponding participating server.

6. The general architecture for protecting privacy according to claim 2, characterized in that, When the masking scheme of the participating server is determined to be the joint index protocol: The participating server generates a random number, performs exponential processing on the random number, and sends it to the secret state calculation server. The dense state computing server generates intermediate data based on the exponential processing result, and then sends the intermediate data to each of the participating server after performing arithmetic sharding.

7. The general architecture for protecting privacy according to claim 2, characterized in that, When the masking scheme of the participating server is determined to be the joint comparison protocol: Each of the participating servers generates the same random number mask; The participating server processes the data using the random number mask according to its own preset processing rules, and sends the processing result to the secret state calculation server. The dense state computing server performs data summation on the received processing results and compares the data based on the positive or negative value of the summation results to obtain a comparison result.

8. The general architecture for protecting privacy according to claim 2, characterized in that, When the joint addition protocol is applied to the horizontal federated learning scenario: Each participating server masks its own model gradient with the corresponding random number. Each participating server generates several identical random numbers and determines a verification value based on the random numbers and the gradient value of the model gradient; The masked model gradient and the verification value are sent to the aggregation server for aggregation processing to obtain the aggregated value, and the aggregated value is returned to each of the participating party servers. Each participating server redetermines the checksum based on the aggregated value and the sum of all masks; When the joint addition protocol is applied to the longitudinal linear regression algorithm: Both the passive party and the active party with tags in the participating party server generate random numbers; Both the active party and the passive party process the random number to obtain dense state data and send the dense state data to the dense state calculation server. The dense state calculation server sums the received dense state data and sends the summation result to the passive party and the active party; The passive party and the active party perform demasking on the received summation result and update the model gradient based on the processing result.

9. The general architecture for protecting privacy according to claim 2, characterized in that, When the joint multiplication protocol is applied to fully connected layers or convolutional computations in deep neural networks: The participating server masks the matrix that needs to be interactively multiplied using the random number and then sends it to the secret state calculation server. The dense state computing server segments the masking results based on matrix operation rules and multiplication protocols, and sends the segmentation results to the participating server. The participating server performs demasking on the received sharding results and performs addition and aggregation to determine the sharding of the product matrix; When the joint comparison protocol is applied to the activation function calculation of a deep neural network: For an activation function in the form of a piecewise function, or an activation function that can be approximated by a piecewise function, each of the participating servers generates the same random number mask, processes the random number mask, and sends the data processing result to the secret state calculation server. The dense state calculation server sums the received processing results and performs a joint comparison based on the sign of the summation results to obtain a joint comparison result. Based on the joint comparison result, it selects different piecewise functions for processing. For the sigmoid function, the participating server generates a random number, masks the random number, and then sends it to the secret state calculation server. The dense state calculation server performs calculations on the received mask processing results to obtain intermediate results; The dense state computing server performs arithmetic fragmentation on the intermediate results and then sends them to the participating server. The participating server performs demasking on the sharding results; For Gelu functions, the arithmetic slices of the subfunction are determined based on the input of the subfunction; The product of the arithmetic pieces is determined based on the joint multiplication protocol, and the product calculation result is obtained.

10. A universal system for protecting privacy, characterized in that, include: The system comprises a secret-state computing server and several participating server servers, each of which is pre-configured with the same random number generator; The random number generator is used to generate random numbers for each corresponding participating server; The specific steps taken by the participating server during federated computation are as follows: Identify the application scenarios for federated computing; The masking scheme is determined based on the application scenario of the participating server; the masking scheme includes a joint addition protocol, a joint multiplication protocol, a joint division protocol, a joint exponentiation protocol, and a joint comparison protocol; The random number is selected and the data of the participating server is masked according to the determined masking scheme, and federated computation is performed.

Images