Safe tripartite calculation method and system supporting complex nonlinear function

Abstract

The invention discloses a safe three-party computing method and system supporting a complex nonlinear function, and relates to the field of safe multi-party computing. According to the method, part of data is calculated in advance in the offline stage, so that the online operation speed in multi-party copy secret sharing is increased, the online communication traffic is reduced, and the query response time of a user is optimized. Moreover, the sigmoid function is approximately expressed by adopting a Fourier series expansion technology, so that a gradient explosion phenomenon caused in an approximate interval boundary region is avoided, and better communication efficiency is realized at the same time. Secondly, based on linear conversion characteristics of sigmoid and a tanh function, obtaining a safe three-party calculation method capable of being reused to the tanh function; finally, an ordinary differential equation is constructed to replace a conventional approximation algorithm, a softmax safe three-party calculation method is provided, and higher accuracy and lower communication complexity are achieved.

Description

Technical Field

[0001] The present invention relates to the field of secure multi-party computing, and in particular to a secure three-party computing method and system supporting complex nonlinear functions. Background Art

[0002] In recent years, many technology and Internet companies have launched cloud service platforms that can provide machine learning services to users with limited computing resources. However, this service model also brings security risks: it may lead to user data leakage and platform model data leakage.

[0003] To address this challenge, the concept of privacy-preserving neural networks was proposed. Some of these studies have used secure multi-party computation (MPC) technology to achieve secure reasoning and training of neural networks.

[0004] The initial research on secure multi-party computation focused on convolutional neural networks with simple activation functions (such as ReLU). Although subsequent work has been extended to more complex nonlinear activation functions, most of them rely on polynomial approximation methods (such as Taylor expansion) or use simpler activation functions as substitutes, resulting in insufficient calculation accuracy of nonlinear activation functions. In addition, both linear function operations and nonlinear activation functions have the problem of high computational and communication overhead. How to reduce communication costs while ensuring computational accuracy has become a core problem that needs to be solved in the field of privacy-preserving neural networks.

[0005] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0006] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a secure three-party computing method and system that supports complex nonlinear functions is provided, aiming to solve the problem of large online communication volume of linear function operations and nonlinear activation functions in existing secure multi-party computing.

[0007] The technical solution adopted by the present invention to solve the problem is as follows:

[0008] In a first aspect, an embodiment of the present invention provides a secure three-party calculation method supporting complex nonlinear functions, the method comprising: a secure multiplication method of linear functions and a secure three-party calculation method of nonlinear functions;

[0009] The secret share of each participant's private input is set to include: the public parameter, and the secret share of the random number obtained based on the standard replication secret sharing method;

[0010] The secure multiplication operation method of the linear function includes: in the offline stage, each of the participating parties samples to obtain the secret shares of the random numbers of the linear function, and generates the first pre-computed data based on the secret shares of the random numbers of the two private inputs; in the online stage, each of the participating parties calculates the secret shares of the linear function values of the two private inputs based on the secret shares of the random numbers of the linear function, the secret shares corresponding to the two private inputs respectively, and the first pre-computed data.

[0011] The secure three-party calculation method of the non-linear function includes at least one of the following methods:

[0012] The secure three-party calculation method of the sigmoid function includes: approximately representing the sigmoid function through the Fourier series expansion technique; in the offline stage, each of the participating parties generates multiple random arrays and generates the second pre-computed data; in the online stage, each of the participating parties calculates the secret shares of the sigmoid function values of the private inputs based on the secret shares of the private inputs and the second pre-computed data.

[0013] The secure three-party calculation method of the tanh function includes: each of the participating parties, based on the linear relationship between the tanh function and the sigmoid function, calls the secure three-party calculation method of the sigmoid function to calculate the secret shares of the tanh function values of the private inputs.

[0014] The secure three-party calculation method of the softmax function includes: each of the participating parties iteratively calculates the secret shares of the softmax function values of the private inputs through the softmax function approximately expressed by the ordinary differential equation.

[0015] In a second aspect, an embodiment of the present invention further provides a secure three-party calculation system supporting complex non-linear functions. The system includes several participating parties, and each of the participating parties performs data interaction and calculation based on the secure three-party calculation method supporting complex non-linear functions as described in any one of the above.

[0016] In a third aspect, an embodiment of the present invention further provides a computer terminal, characterized in that the computer terminal includes a memory and more than one processor; the memory stores more than one program; the program includes instructions for executing the secure three-party calculation method supporting complex non-linear functions as described in any one of the above; the processor is used to execute the program.

[0017] Advantages of the present invention: In the offline phase of the embodiments of the present invention, some data is pre-computed, which speeds up the online operation speed in multi-party replicated secret sharing, reduces the online communication volume, and optimizes the query response time of users. Moreover, by using the Fourier series expansion technique to approximately represent the sigmoid function, the gradient explosion phenomenon caused in the boundary region of the approximation interval is avoided, and at the same time, better communication efficiency is achieved. Secondly, based on the linear conversion characteristics of the sigmoid and tanh functions, a secure three-party computing method that can be reused for the tanh function is obtained. Finally, by constructing an ordinary differential equation to replace the conventional approximation algorithm, a secure three-party computing method for softmax is proposed, achieving higher accuracy and lower communication complexity. Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a schematic flowchart of a secure three-party computing method supporting complex non-linear functions provided by the embodiments of the present invention.

[0020] Figure 2 It is a comparison schematic diagram between the sigmoid function approximately represented by the polynomial approximation method and the original sigmoid function provided by the embodiments of the present invention.

[0021] Figure 3 It is a principle block diagram of a terminal provided by the embodiments of the present invention. Detailed Embodiments

[0022] The present invention discloses a secure three-party computing method and system supporting complex non-linear functions. To make the purpose, technical solutions and effects of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0023] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present invention means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the art to which the present invention pertains. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as herein.

[0024] In view of the above-mentioned deficiencies of the prior art, the present invention provides a secure three-party computing method that supports complex non-linear functions, such as Figure 1 shown, the method includes a secure multiplication operation method for linear functions and a secure three-party computing method for non-linear functions; specifically includes the following steps:

[0025] Step S100, setting the secret shares of the private inputs of each participant includes: public parameters, and the secret shares of random numbers obtained based on the standard replicated secret sharing method.

[0026] The method of this embodiment is an improved scheme for the commonly used replicated secret sharing method (RSS). For the convenience of understanding the scheme, the common form of RSS is first described. In secure three-party computing, calculations are usually carried out in the replicated secret sharing (RSS) framework. There are two common forms of RSS: the first replicated secret sharing method, i.e., [·]-sharing; the second replicated secret sharing method, i.e., <·>-sharing. The commonly used second replicated secret sharing method is used as the standard replicated secret sharing method that will be used in the execution process of this embodiment. Assume in the Boolean ring (or arithmetic ring ), among three participants P i , i ∈ {0, 1, 2}, then the two forms of RSS are defined as follows:

[0027] 1. [·]-sharing: In the Boolean ring , [·]-sharing is the exclusive-or sharing of the Boolean private input value v ∈ {0, 1}, that is, v consists of three random values , and each participant P i holds the share denoted as [v] i , and [v] i = v i , such that represents the exclusive - or operation. In the arithmetic ring , assuming there is an arithmetic private input value then the [·] - sharing is an additive sharing of the arithmetic private input value v, that is, v consists of three random values and each participant P i holds a share [v] i = v i such that v = v0 + v1 + v2 (mod 2 l , indicating modular arithmetic).

[0028] 2. <·> - sharing: In the boolean ring , the <·> - sharing is an exclusive - or sharing of the boolean private input value v ∈ {0, 1}, that is, v consists of three random values and each participant P i holds a share <v> i =(v i , v i+1 ), that is, each participant holds two of the random numbers, and there is Therefore, any two of the three participants can construct the secret value v. In the arithmetic ring , assuming there are arithmetic private input values Then the <·>-sharing is an additive sharing of the arithmetic private input value v, that is, v consists of three random values , and each participant P i holds the share <v> i =(v i , v i+1 ), such that v = v0 + v1 + v2 (mod 2 l ).

[0029] [·]-sharing and <·>-sharing both support linear operations. Taking <·>-sharing as an example, let (c1, c2, c3) be Boolean publicly known constant coefficients, and x, y be two <·>-shared Boolean private inputs, and their secret shares are denoted as ( <x> ， <y>), each participating party P i holds <x> i =(x i , x i+1 ) and <y> i = (y i , y i+1 ), where the index subscript is taken modulo, that is, when i + 1 is greater than 2, take i + 1 - 3. The linear operation result Z of the Boolean private inputs x, y and the public coefficients (c1, c2, c3) can be expressed as Secret share <z>Can be obtained locally by each participating party P i through calculation Similarly, for the arithmetic ring only need to replace the exclusive OR operation with the addition "+" and take mod 2 l , the secret shares of the linear operation can still be obtained through local calculation.

[0030] Both the above [·]-sharing and <·>-sharing need to execute the sharing protocol after receiving the input and execute the reconstruction protocol after the calculation to obtain the calculation result. In other words, all the operation steps of these two RSS schemes are completed only after the input is obtained, that is, all the operation steps are related to the input, so the computational amount and communication volume in the online phase are very large.

[0031] Based on the use of [·]-sharing and <·>-sharing, this embodiment proposes a new replicated secret sharing scheme, denoted as the improved replicated secret sharing method, that is -sharing:

[0032] A Boolean private input value 's -sharing consists of the public parameter m v and the random number λ under <·>-sharing v in two parts. Taking the -sharing in the Boolean ring as an example, for the Boolean private input value v ∈ {0, 1}, the share of each participating party P i is denoted as where i ∈ {0, 1, 2}, <λ v > means that λ v is secretly shared. λ v is a random number secretly shared among P0, P1, P2 (that is, each participating party P i holds the share of λ v ) and the public parameter m is a known parameter for all participating parties. Then the Boolean private input value v satisfies v under -sharing Similarly, in the arithmetic ring , for an l-bit arithmetic private input value satisfies v = m v + λ v (mod 2 l ), where

[0033] - Sharing is also linear for Boolean rings and arithmetic rings. For example, for Boolean sharing, let (c1, c2, c3) be Boolean public constant coefficients, and x, y be two - shared Boolean private inputs, whose secret shares are denoted as That is denotes the secret share of x. Then the result of the linear operation on the Boolean private inputs x, y and the public coefficients (c1, c2, c3) All parties P i can obtain the secret share of z through local calculation of and where i ∈ {0, 1, 2}. In addition, for the complement value of the Boolean private input v (that is ), the public parameter m of the Boolean private input v can be modified only v to obtain the public parameter of the Boolean private input That is That is

[0034] In one implementation, - The basic protocol under sharing includes:

[0035] Sharing protocol: - The secret sharing protocol under sharing can enable the holder of the private input to generate the - sharing of its private input x. The function implements the - sharing protocol under sharing Any party that holds a private input and needs to use it under - sharing can call the function. Taking the Boolean ring as an example, in the offline phase, all parties jointly call the random number generation function under <·>- sharing to sample a random number λ x and such that where the private input holder has λ x and all its secret shares. In the online phase, the private input holder calculates the public parameter of the Boolean private input x under and discloses m to other parties x (in the arithmetic ring, m x = x - λ x (mod 2 l ).

[0036] Reconstruction protocol: - The reconstruction protocol under sharing Enables each participating party to reconstruct private values. The reconstruction function implements - The reconstruction protocol under <·>-sharing where <·>-sharing random number λ x can be obtained by calling . The core calculation steps of the reconstruction protocol are the opposite of the sharing protocol. For the boolean ring, each participating party locally calculates In the arithmetic ring, each participating party locally calculates x = m x + λ x (mod 2 l ).

[0037] Multiplication operation: - The two-number multiplication operation protocol under <·>-sharing Enables three participating parties to jointly calculate the multiplication of two private input values and obtain their shares of the operation result under <·>-sharing. The multiplication operation function implements - The multiplication operation protocol under <·>-sharing Assume that for two arithmetic rings the private input shares are multiplied and the calculation result shares are output, where z = x·y, m x , m y , m z are the public parameters of x, y, z under <·>-sharing respectively, and λ x , λ y , λ z are the random numbers of x, y, z under <·>-sharing respectively. Then there is:

[0038] m z = x·y - λ z = (m x + λ x )(m y + λ y ) - λ z

[0039] = m x m y + λ x m y + λ y m x + λ x λ y - λ z ;

[0040] In the above formula, except for λ x λ y All terms other than can be locally computed by the participants because m x and m y are known to all participants. Therefore, the main problem to be solved is how to compute <λ x > and <λ y > given <λ x λ y >. Since λ x and λ y are independent of the input, <λ x λ y > can be computed in the offline phase using computation. In the offline phase, the parties interact to generate <λ z >, <λ x > and <λ y >, and pre-compute <λ x λ y >; in the online phase, the parties compute and reconstruct <m z >, which is the secret share of the public parameter of the linear function value z of the private inputs x and y.

[0041] Step S200, the secure multiplication operation method of the linear function, includes:

[0042] Step S201, in the offline phase, each of the participants samples to obtain the secret share of the random number of the linear function, and generates the first pre-computed data based on the secret shares of the random numbers of the two private inputs;

[0043] Step S202, in the online phase, each of the participants computes the secret share of the linear function value of the two private inputs based on the secret share of the random number of the linear function, the secret shares corresponding to the two private inputs respectively, and the first pre-computed data.

[0044] Generally speaking, in order to reduce the computational overhead in the online phase, this embodiment divides the secure multiplication operation of the linear function into two phases. In the offline phase, mainly random number generation and pre-computation of data independent of the input are performed, and the time-consuming multiplication operation is advanced to the offline phase, thereby reducing the computational amount in the online phase and accelerating the privacy protection computation in the online phase. In the online phase, privacy computation can be performed based on the pre-computed data, and the final secret share of the linear function is computed using the pre-computed data in the offline phase and the input shares. The final result is held by each participant in the form of a secret share without revealing the original input.

[0045] Further, step S201 specifically includes:

[0046] Step S2011: During the offline phase, each of the participating parties samples the secret shares of the random numbers of the linear function under the standard replicated secret sharing method;

[0047] Step S2012: Through the multiplication operation function of the standard replicated secret sharing method, based on the secret shares of the random numbers corresponding to the two private inputs respectively, calculate the secret share of the product of the random numbers of the two private inputs;

[0048] Step S2013: Use the secret share of the product of the random numbers as the first pre-computed data.

[0049] Step S202 specifically includes:

[0050] Furthermore, in Step S2021: During the online phase, each of the participating parties, based on the secret shares of the random numbers of the linear function, the secret shares corresponding to the two private inputs respectively, and the first pre-computed data, calculates the secret share of the intermediate variable through the standard replicated secret sharing method;

[0051] Step S2022: Reconstruct the intermediate variable according to the secret share of the intermediate variable;

[0052] Step S2023: Calculate the public parameter of the linear function according to the reconstructed intermediate variable and the public parameters corresponding to the two private inputs respectively;

[0053] Step S2024: Generate the secret share of the linear function value of the two private inputs based on the public parameter of the linear function and the random number of the linear function held by itself.

[0054] For example, for the multiplication operation in the arithmetic ring The input data is: the - sharing shares of the arithmetic private inputs x and y The output data is: the - sharing shares of the arithmetic private output Z

[0055] During the offline phase: Each participating party P i respectively samples the random number <λ z > under the <·>-sharing using the pre-negotiated random seed, where i ∈ {0, 1, 2}. Then, each participating party uses the function to jointly generate <λ x λ y >, that is, the secret share of the product of the random numbers of the private inputs x and y under the <·>-sharing.

[0056] During the online phase: Each participating party P i Using the public parameters and the secret shares of the random numbers held by each party under <·>-sharing, locally compute the intermediate variable m Δ The secret share <m under <·>-sharing Δ > i = m y <λ x > i + m x <λ y > i + <λ x λ y > i - <λ z >i, where i ∈ {0, 1, 2}, and m x is the public parameter of the private input x, and m y is the public parameter of the private input y, and λ x and λ y are the secret shares of the random numbers of the private inputs x and y respectively. <λ x >represents that λ x is secretly shared.

[0057] Each participating party uses the reconstruction function under <·>-sharing to reconstruct m Δ and calculates the public parameter m of the arithmetic private output z under -sharing z = m Δ + m x m y .

[0058] Each participating party P i holds the share where λ z is the secret share of the random number of the private output z under <·>-sharing.

[0059] Step S300, the secure three-party calculation method of the non-linear function includes at least one of the following methods:

[0060] Step S301, the secure three-party calculation method of the sigmoid function;

[0061] Step S302, the secure three-party calculation method of the tanh function;

[0062] Step S303, the secure three-party calculation method of the softmax function.

[0063] In a neural network with a relatively complex structure, the sigmoid function, tanh function, and softmax function are usually used as non-linear activation functions. Therefore, this embodiment also provides corresponding secure three-party calculation methods for these non-linear functions.

[0064] Among them, step S301, the secure three-party computation method for the sigmoid function, includes:

[0065] Step S3011: Approximately represent the sigmoid function through Fourier series expansion technology; in the offline stage, each of the participating parties generates multiple random arrays and generates second pre-computed data;

[0066] Step S3012: In the online stage, each of the participating parties calculates the secret share of the sigmoid function value of the private input based on the secret share of the private input and the second pre-computed data.

[0067] Generally speaking, the sigmoid function compresses the output in the real number range to between (0, 1), representing the probability of a certain category, and its formula is as follows:

[0068]

[0069] In the formula, x represents the variable of the sigmoid, and e represents the natural constant.

[0070] For example, in a binary classification task, the output value can be interpreted as the probability that the sample belongs to the positive class (such as "yes"). Sigmoid is the core of logistic regression. By optimizing the parameters through maximum likelihood estimation, it directly models the probability that a sample belongs to a certain class, and is usually also used in the output layer of a neural network model to generate probability values.

[0071] Since previous secure multi-party computation work usually roughly replaces the sigmoid function with a piecewise function, the calculation accuracy is low; or uses a polynomial approximation function for approximation, which will produce very large errors for a small number of inputs falling on the interval edge and may have a completely opposite impact on the calculation result, as Figure 2 shown. Therefore, in this embodiment, a secure three-party computation method for approximately representing the sigmoid function through Fourier series expansion technology is used to solve the Runge problem of the polynomial approximation method.

[0072] The expansion of the Fourier series requires the original function to be periodic, but the sigmoid function is not periodic throughout the domain. Considering that in actual secure computation applications, calculations are usually performed in a finite field, an interval [-L, L] can be selected and the entire function can be shifted downward by 1 / 2 to obtain a new function At this time, f(x) can be regarded as an odd function symmetric about the origin, where L can be the common bit width in secure computation. Since f(x) is an odd function, f(x) can be expanded as where k is the number of terms in the expansion, and the constant term is the amplitude of zero frequency, and b k is the amplitude of the sine component. The premise for designing the sigmoid approximation expression in this embodiment is that it satisfies being as close as possible to the original curve within the approximation interval, and the error at the edge of the approximation interval is not too large. Therefore, in this embodiment, the Fourier series is expanded into six terms (where five terms are sine components), and the corresponding b k The calculation formula of is as follows:

[0073]

[0074] In the formula, dx is the integration element, and the Chinese definitions of the remaining characters can be found in the previous part.

[0075] Taking the common bit width in safety calculations, that is, L = 16 and substituting it into the formula, we get:

[0076]

[0077] From this, the coefficients of the five sine components can be calculated (represented by a five-dimensional vector here): Substituting x = 0 into the formula, we can get a0 = 1. For the sake of simple expression, the coefficients before the variables in the sine components in the formula are expressed in the form of a column vector (a five-dimensional vector here), that is So far, the approximate expression of the sigmoid function is obtained as follows:

[0078]

[0079] Therefore, the three parties only need to interactively calculate sin x to implement the sigmoid function. In trigonometric functions, sin(a + b) = sin a cos b + sin b cos a, and in -sharing, there is x = m x + λ x (mod 2 l ), so sin x = sin(m x + λ x ) = sin m x cosλ x + sin λ x cos m x , and m x is a public parameter held by each party, and sin m x and cos m x can be calculated locally by each party. So far, the key point of the protocol lies in how to calculate cosλ x and sinλ x . And λ x is a random number that satisfies <·>-sharing and is independent of the input, that is, λ x = λ x0 +λ x1 +λ x2 (mod 2 l ),therefore sinλ x =sin(λ x0 +λ x1 +λ x2 ), cosλ x =cos(λ x0 +λ x1 +λ x2 ).

[0080] After approximately representing the sigmoid function through the Fourier series expansion technique, in order to reduce the computational overhead in the online stage, the secure three-party computation method of the sigmoid function in this embodiment is also divided into two stages: in the offline stage, each participating party is mainly responsible for generating random arrays and pre-computation, especially pre-computing sinλ x and cosλ x ; in the online stage, each participating party calculates the secret share of the sigmoid function value of the private input based on the secret share of the private input and the pre-computed data.

[0081] Furthermore, step S3011 specifically includes:

[0082] Step S30111: In the offline stage, each of the participating parties generates its own local random array and jointly generates a common random array with the other participating parties;

[0083] Step S30112: Each of the participating parties performs trigonometric function calculations based on the random numbers of the private input to obtain the offline trigonometric function calculation results, and generates local calculation data according to the offline trigonometric function calculation results and the local random array, and sends the local calculation data to the other participating parties;

[0084] Step S30113: Each of the participating parties generates second pre-computed data based on the local random array, the common random array, and the local calculation data received from the other participating parties; and sends the second pre-computation to the other participating parties according to a preset sending rule.

[0085] Furthermore, step S3012 specifically includes:

[0086] Step S30121: In the online stage, each of the participating parties performs trigonometric function calculations based on the public parameters of the private input to obtain the online trigonometric function calculation results;

[0087] Step S30122: According to the second pre-computed data held by itself and the online trigonometric function calculation result, call the standard secret sharing scheme to generate the secret share of the trigonometric function value of the private input;

[0088] Step S30123: Set the public parameter of the private output to a preset value; based on the secret share of the trigonometric function value of the private input and the preset value, generate the secret share of the sigmoid function value of the private input.

[0089] Illustrate with an example for the secure three-party computation of the sigmoid function Input data: The private input x is at - Secret share under sharing Output data: The private output y is at - Secret share under sharing where y ≈ sigmoid(x);

[0090] Related random numbers: Party P0 locally generates random numbers s0, s1, c0, c1 (i.e., the local random number array), and Party P1 locally generates random numbers r0, r1, t0, t1; each party P i Collectively generates random numbers α0, α1, α2, μ0, μ1, μ2 (i.e., the collective random number array), such that α0 + α1 + α2 = 0, μ0 + μ1 + μ2 = 0, where i ∈ {0, 1, 2}.

[0091] λ x = λ x0 + λ x1 + λ x2 , λ x represents the random number of the private input x at - Secret share under sharing;

[0092] During the offline phase: Party P0 locally calculates sin(λ x0 + λ x1 ) and cos(λ x0 + λ x1 ), sets s2 = sin(λ x0 + λ x1 ) - s0 - s1 (mod 2 l ), c2 = cos(λ x0 + λ x1 ) - c0 - c1 (mod 2 l ), and sends (s1, s2), (c1, c2) to Party P1, and sends (s0, s2), (c0, c2) to Party P2.

[0093] Party P1 locally calculates sinλ x2 and cosλ x2 and set \(r2 = \sin\lambda\) x2 \(-r0 - r1\ (\text{mod} 2\) l ), \(t2 = \cos\lambda\) x2 \(-t0 - t1\ (\text{mod} 2\) l ), and send \((r0, r1)\), \((t0, t1)\) to participant \(P0\), and send \((r0, r2)\), \((t0, t2)\) to participant \(P2\).

[0094] Each participant \(P\) i calculates:

[0095] \(\eta\) i \(= s\) i \(\cdot t\) i \(+ s\) i \(\cdot t\) i+1 \(+ s\) i+1 \(\cdot t\) i \(+ c\) i \(\cdot r\) i \(+ c\) i \(\cdot r\) i+1 \(+ c\) i+1 \(\cdot r\) i \(+ \alpha\) i ,

[0096] \(\theta\) i \(= c\) i \(\cdot t\) i \(+ c\) i \(\cdot t\) i+1 \(+ c\) i+1 \(\cdot t\) i \(- s\) i \(\cdot r\) i \(- s\) i \(\cdot r\) i+1 \(- S\) i+1 \(\cdot r\) i \(+ \mu\) i ,

[0097] where \(i\in\{0, 1, 2\}\); \(\eta\) i , \(\theta\) i together serve as the second pre - calculation data.

[0098] Each participant \(P\) i sends \(\eta\) i , \(\theta\) i to participant \(P\) i-1 (all subscript indices adopt a modulo - 3 cyclic mechanism), and at this time each participant \(P\) i holds \((\eta\) i , \(\eta\) i+1 ), \((\theta\) i , \(\theta\) i+1 ).

[0099] Online phase: Each participant \(P\) i Local calculation of sin m x , cos m x , at this time, <sin x> is obtained i =(cos m x ·η i +sin m x ·η i , cos m x ·η i+1 +sin m x ·η i+1 );

[0100] Take Obtain

[0101] Among them, step S302, the secure three-party calculation method of the tanh function, includes:

[0102] Step S3021, each of the participating parties, based on the linear relationship between the tanh function and the sigmoid function, calls the secure three-party calculation method of the sigmoid function to calculate the secret share of the tanh function value of the private input.

[0103] The formula of the tanh function is as follows;

[0104]

[0105] There is a linear relationship between the tanh function and the sigmoid function, so the secure three-party calculation protocol of the sigmoid function can be directly reused to calculate the output of the tanh function under -sharing represents the secret share of the private output in the secure three-party calculation of the tanh function, that is, the secret share of the tanh function value of the private input.

[0106] Furthermore, step S3021, each of the participating parties, based on the linear relationship between the tanh function and the sigmoid function, calls the secure three-party calculation method of the sigmoid function to calculate the secret share of the tanh function value of the private input, specifically including:

[0107] Step S30211, each of the participating parties modifies the secret share of the private input based on the linear relationship between the tanh function and the sigmoid function;

[0108] Step S30212, according to the modified secret share of the private input, calls the secure three-party calculation method of the sigmoid function to calculate the secret share of the tanh function value of the private input.

[0109] Specifically, the input of the sigmoid function has a two-fold relationship with the input of the tanh function, and the -sharing has linear properties. Therefore, only the private input needs to be modified according to the linear relationship between the sigmoid function and the tanh function, and then the secure three-party computation method of the sigmoid function can be used to obtain the output of the tanh function under -sharing

[0110] For example, assume there is an existing private input where m x is a public parameter, λ x is a random number under <·>-sharing, and it is necessary to calculate y = tanh(x). Only the private input needs to be changed to and then call the secure three-party computation method of the sigmoid function to obtain the output of the tanh function under -sharing That is denotes -sharing the secret share of the tanh function value.

[0111] Among them, step S303, the secure three-party computation method of the softmax function, includes:

[0112] Step S3031, each of the participating parties iteratively calculates the secret share of the softmax function value of the private input through the softmax function approximately expressed based on ordinary differential equations.

[0113] The Softmax function is often used in the output layer of multi-classification problems, and its formula is as follows:

[0114]

[0115] Among them, the variable of the Softmax function is the vector x i denotes the i-th component in, j ∈ {1, 2,..., m}, and m represents the dimension of the function output. It converts the output of the network into a probability distribution, with each category corresponding to a probability value, and the sum of all probability values is 1. For example, in image classification, the Softmax function can be used to obtain the probability that the input image belongs to each category; when training a neural network, using the combination of the Softmax function and the cross-entropy loss function can effectively perform gradient descent optimization. In privacy-preserving machine learning, usually, the ReLU function that is more easily processed in MPC is used to replace the Softmax function, making the function calculation process easier, that is, using the formula to calculate the probability distribution of the output result.

[0116] However, since the output of the ReLU function is non - negative, negative inputs are completely suppressed, which may cause the model to fail to correctly handle cases where there are negative numbers in the original output. At the same time, for all - negative inputs that occur with extremely low probability, it will also cause a division - by - zero error. While Softmax performs exponential operations on all positive and negative inputs, so each class has a non - zero probability and retains the gradient of negative inputs.

[0117] In addition, replacing the Softmax function with the ReLU function will also cause gradient problems during training. The gradient of ReLU is 0 for negative inputs, which may cause some neurons to not be updated during training. If used in the output layer, when the ReLU output of some nodes is 0, the corresponding gradient will also disappear, affecting parameter update. While the gradient calculation of Softmax involves all nodes, even if the output of a certain node is very small, it will still contribute to the gradient, which helps for more stable training. Therefore, in this embodiment, an approximate calculation of the original Softmax is performed, and a method for approximating the calculation of the Softmax function based on ordinary differential equations (Euler's formula can be selected) is proposed.

[0118] It can be seen from the formula of the Softmax function that Softmax is a function with multi - variable inputs, while the ordinary differential equation based on Euler's formula is a calculation method with single - variable inputs. Therefore, it is first necessary to convert multi - variable inputs into single - variable inputs. To convert the multi - variable softmax function into a single - variable function, a single - variable function with input t is defined When t = 0, that is, the output probability of each element is equal; when t = 1, that is, the target result, where t is used as a variable and iterates from 0 to 1, and the number of iterations is r, is the input variable of the original Softmax function, and m is the dimension of the function output. According to the first - order difference equation formula, the derivative of the function can be obtained Then there is f(x + Δ)=f′(x + Δ)·Δ + f(x), where Δ represents the difference of the independent variable x. Next, take the partial derivative of with respect to the variable t. According to the fraction derivative rule, it can be obtained where represents the vector inner product, * represents scalar multiplication, represents the operation of converting a scalar variable into a vector.

[0119] According to the above derivation, the single - variable input function can be further transformed, and the final result of Softmax is calculated using an iterative method. From and the two derivation formulas f(x + Δ)=f′(x + Δ)·Δ + f(x), the multi - variable function can be converted into a single - variable function wherein is an iterative solution, that is i ∈ {0, 1, 2} represents the number of iterations, the output of is an m-dimensional vector, r is a hyperparameter, and the output after r iterations is the approximate result of the Softmax function.

[0120] Further, in step S3031, each of the participating parties iteratively calculates the secret share of the Softmax function value of the private input through the Softmax function approximately expressed based on ordinary differential equations, specifically including:

[0121] Step S30311, each of the participating parties holds the secret share of its respective iterative initial solution;

[0122] Step S30312, each of the participating parties, under the secure multiplication operation method and local calculation of the linear function, calculates the secret share of the Softmax function value of the private input according to the preset number of iterations through the Softmax function approximately expressed based on ordinary differential equations.

[0123] Illustrate by example the secure three-party calculation of the Softmax function The input data is: private input Under - the secret share under sharing and the public hyperparameter r; the output data is: private output Under - the secret share under sharing wherein

[0124] At the beginning of the calculation, set the initial solution of the iteration

[0125] Participant P0 holds Participant P1 holds Participant P2 holds

[0126] for t ∈ {1, 2,..., r} (the for instruction and the end for pointer are used to control the program pointer to loop and execute the steps between for and end for, t represents the current iteration round, and r represents the total number of iterations):

[0127] Each participant P i collectively calls the function to calculate the first intermediate variable where i ∈ {0, 1, 2} and j ∈ {1, 2,..., m}.

[0128] Each participant P i Locally calculate the second intermediate variable where \(i\in\{0, 1, 2\}\) and \(j\in\{1, 2, \ldots, m\}\).

[0129] Each participating party \(P\) i Collectively call a function Calculate the third intermediate variable

[0130]

[0131] In the formula, \(i\in\{0, 1, 2\}\) and \(j\in\{1, 2, \ldots, m\}\), denotes the \(j\)-th component of

[0132] Each participating party P i Locally calculate the iterative solution for the current iteration round

[0133] end for;

[0134] After completing the above iteration, each participating party \(P\) i holds

[0135] In one implementation, the method further includes:

[0136] Step S400: Construct a linear layer based on the secure multiplication operation method of the linear function, and construct a non - linear layer based on the secure three - party calculation method of the non - linear function;

[0137] Step S500: Construct a neural network for secure three - party calculation through the linear layer and the non - linear layer.

[0138] This embodiment also constructs a neural network for secure three - party calculation. For the complete secure three - party inference and training of the neural network, the calculation can be divided into a linear layer and a non - linear layer. The linear layer includes convolutional layers (or other building blocks involving matrix operations), and the non - linear layer includes activation functions, pooling layers, and normalization layers, etc. In an actual scenario, the linear layer uses the improved replicated secret sharing for calculation, that is, uses - sharing for calculation; the non - linear layer uses at least one of the secure three - party calculation methods of the above sigmoid function, tanh function, and softmax function for calculation. It should be noted that this embodiment mainly involves the optimization of the algorithms in the privacy - protected neural network process, and both the linear layer and the non - linear layer protocols are three - party calculations. However, in some specific application scenarios, the number of participating parties can be extended, that is, the number of participating parties \(N\) is greater than 3.

[0139] In summary, the advantages of the present invention are as follows:

[0140] (1) The present invention provides an improved replicated secret sharing method, which can achieve low online communication volume. In the offline phase, each participant batch generates input-independent random variables (such as multiplication triples or random number seeds, etc.) without knowing the input data. In the online phase, each participant uses the values pre-generated in the offline phase to jointly complete the final calculation, thereby significantly accelerating the online operation speed and reducing the online communication volume. The present invention reduces the online communication volume in the secret sharing scheme used in the secure inference method and optimizes the query response time of users.

[0141] (2) The present invention provides a secure three-party computing method for complex non-linear functions, which can achieve high computing accuracy and low communication complexity:

[0142] Secure three-party computing method for sigmoid function: The sigmoid function is approximated using Fourier series. By the periodicity and boundedness of the Fourier series, the consistency of computing accuracy is maintained within the domain interval, avoiding the problems of oscillation error and gradient explosion, achieving low communication complexity, and eliminating the computing redundancy and accuracy loss caused by multi-level operations. After testing, the present invention can reduce the communication rounds of the secure three-party computing of the sigmoid function from 16 to 1 round.

[0143] Secure three-party computing method for tanh function, according to the linear relationship between the tanh function and the sigmoid function, reusing the secure three-party computing method of the sigmoid function;

[0144] Secure three-party computing method for softmax function, using ordinary differential equations to design a softmax secure three-party computing protocol, eliminating the computing redundancy and accuracy loss caused by multi-level operations, with higher accuracy and lower communication complexity than traditional approximate computing. After testing, the present invention can reduce the communication rounds of the secure three-party computing of the softmax function from 11n (n is the bit width of the floating point number) to 2r rounds (related to the number of iterations r).

[0145] Based on the above embodiments, the present invention also provides a secure three-party computing system supporting complex non-linear functions. The system includes several participants, and each participant performs data interaction and calculation based on the secure three-party computing method for complex non-linear functions as described in any one of the above.

[0146] Based on the above embodiments, the present invention also provides a computer terminal, and its principle block diagram can be as Figure 3 As shown. The computer terminal includes a processor, a memory, a network interface, and a display screen connected by a system bus. Among them, the processor of the computer terminal is used to provide computing and control capabilities. The memory of the computer terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer terminal is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a secure three-party computing method for supporting complex non-linear functions. The display screen of the computer terminal can be a liquid crystal display screen or an electronic ink display screen.

[0147] Those skilled in the art can understand that Figure 3 the block diagram of the principle shown in is only the block diagram of the partial structure related to the solution of the present invention, and does not constitute a limitation on the computer terminal to which the solution of the present invention is applied. The specific computer terminal may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0148] In one implementation, more than one program is stored in the memory of the computer terminal, and is configured to be executed by more than one processor. The more than one program includes instructions for performing a secure three-party computing method for supporting complex non-linear functions.

[0149] Those of ordinary skill in the art can understand that all or part of the process of implementing the method in the above embodiments can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0150] In summary, the present invention discloses a secure three-party computing method and system for supporting complex non-linear functions, which relates to the field of secure multi-party computing. The method pre-computes part of the data in the offline stage, which speeds up the online operation speed in multi-party replicated secret sharing, reduces the online communication volume, and optimizes the user's query response time. And, by using the Fourier series expansion technology to approximately represent the sigmoid function, the gradient explosion phenomenon caused in the boundary region of the approximation interval is avoided, and at the same time, a better communication efficiency is achieved. Secondly, based on the linear conversion characteristics of the sigmoid and tanh functions, a secure three-party computing method that can be reused for the tanh function is obtained. Finally, by constructing an ordinary differential equation to replace the conventional approximation algorithm, a secure three-party computing method for softmax is proposed, achieving higher accuracy and lower communication complexity.

[0151] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or modifications can be made according to the above description, and all such improvements and modifications shall fall within the protection scope of the appended claims of the present invention.< / z> < / y> < / x> < / y> < / x> < / v> < / v>

Claims

1. A secure three-party computation method supporting complex non-linear functions, characterized in that, The method includes: a secure multiplication operation method for linear functions and a secure three-party computation method for non-linear functions; Setting the secret shares of the private inputs of each participant includes: public parameters, and the secret shares of random numbers obtained based on the standard replicated secret sharing method; The secure multiplication operation method for linear functions includes: in the offline phase, each participant samples to obtain the secret shares of the random numbers of the linear function, and generates first pre-computed data based on the secret shares of the random numbers of two private inputs; in the online phase, each participant calculates the secret shares of the linear function values of the two private inputs based on the secret shares of the random numbers of the linear function, the secret shares corresponding to the two private inputs respectively, and the first pre-computed data; The secure three-party computation method for non-linear functions includes at least one of the following methods: The secure three-party computation method for the sigmoid function includes: approximately representing the sigmoid function through Fourier series expansion technology; in the offline phase, each participant generates multiple random arrays and generates second pre-computed data; in the online phase, each participant calculates the secret shares of the sigmoid function values of the private inputs based on the secret shares of the private inputs and the second pre-computed data; The secure three-party computation method for the tanh function includes: each participant calculates the secret shares of the tanh function values of the private inputs by invoking the secure three-party computation method for the sigmoid function based on the linear relationship between the tanh function and the sigmoid function; The secure three-party computation method for the softmax function includes: each participant iteratively calculates the secret shares of the softmax function values of the private inputs through the softmax function approximately expressed based on ordinary differential equations.

2. The secure three-party computation method for supporting complex non-linear functions according to claim 1, wherein In the offline phase, each participant samples to obtain the secret shares of the random numbers of the linear function, and generates first pre-computed data based on the secret shares of the random numbers of two private inputs, including: In the offline phase, each participant samples to obtain the secret shares of the random numbers of the linear function under the standard replicated secret sharing method; Through the multiplication operation function of the standard replicated secret sharing method, based on the secret shares of the random numbers corresponding to the two private inputs respectively, calculates the secret shares of the product of the random numbers of the two private inputs; Taking the secret shares of the product of the random numbers as the first pre-computed data.

3. The secure three-party computation method for supporting complex non-linear functions according to claim 1 or 2, wherein In the online phase, each participant calculates the secret shares of the linear function values of the two private inputs based on the secret shares of the random numbers of the linear function, the secret shares corresponding to the two private inputs respectively, and the first pre-computed data, including: In the online phase, each participant calculates the secret shares of the intermediate variable through the standard replicated secret sharing method based on the secret shares of the random numbers of the linear function, the secret shares corresponding to the two private inputs respectively, and the first pre-computed data; Reconstructing the intermediate variable according to the secret shares of the intermediate variable; Calculate the public parameters of the linear function according to the public parameters corresponding to the reconstructed intermediate variables and the two private inputs respectively. Generate the secret shares of the linear function values of the two private inputs based on the public parameters of the linear function and the random numbers of the linear function held by itself.

4. The secure three-party computation method for supporting complex non-linear functions according to claim 1, characterized in that In the offline phase, each of the participating parties generates multiple random arrays and generates second pre-computed data, including: In the offline phase, each of the participating parties generates its own local random array and jointly generates a common random array with other participating parties. Each of the participating parties performs trigonometric function calculations based on the random numbers of the private inputs to obtain the offline trigonometric function calculation results, and generates local calculation data according to the offline trigonometric function calculation results and the local random array, and sends the local calculation data to other participating parties. Each of the participating parties generates second pre-computed data based on the local random array, the common random array, and the local calculation data received from other participating parties; sends the second pre-computed data to other participating parties according to a preset sending rule.

5. The secure three-party computing method for supporting complex non-linear functions according to claim 1 or 4, characterized in that, In the online phase, each of the participating parties calculates the secret shares of the sigmoid function values of the private inputs based on the secret shares of the private inputs and the second pre-computed data, including: In the online phase, each of the participating parties performs trigonometric function calculations based on the public parameters of the private inputs to obtain the online trigonometric function calculation results. According to the second pre-computed data held by itself and the online trigonometric function calculation results, call the standard secret sharing scheme to generate the secret shares of the trigonometric function values of the private inputs. Set the public parameters of the private output to a preset value; generate the secret shares of the sigmoid function values of the private inputs based on the secret shares of the trigonometric function values of the private inputs and the preset value.

6. The secure three-party computation method for supporting complex non-linear functions according to claim 1, wherein Each of the participating parties calculates the secret shares of the tanh function values of the private inputs by calling the secure three-party calculation method of the sigmoid function based on the linear relationship between the tanh function and the sigmoid function, including: Each of the participating parties modifies the secret shares of the private inputs based on the linear relationship between the tanh function and the sigmoid function. According to the modified secret shares of the private inputs, call the secure three-party calculation method of the sigmoid function to calculate the secret shares of the tanh function values of the private inputs.

7. The secure three-party computing method for supporting complex non-linear functions according to claim 1, characterized in that Each of the participating parties iteratively calculates the secret shares of the softmax function values of the private inputs through the softmax function approximately expressed by ordinary differential equations, including: Each of the participating parties holds the secret shares of its own iterative initial solutions. Each of the participating parties calculates the secret shares of the softmax function values of the private inputs according to a preset number of iterations through the softmax function approximately expressed by ordinary differential equations under the secure multiplication operation method of the linear function and local calculations.

8. The secure three-party computation method for supporting complex non-linear functions according to claim 1, characterized in that The method further includes: Construct a linear layer based on the secure multiplication operation method of the linear function, and construct a non-linear layer based on the secure three-party calculation method of the non-linear function. Construct a neural network for secure three-party computation through the linear layer and the non-linear layer.

9. A secure three-party computing system that supports complex non-linear functions, characterized in that, The system includes several participating parties, and each participating party conducts data interaction and computation based on the secure three-party computation method for supporting complex non-linear functions as described in any one of claims 1-8.

10. A computer terminal, characterized in that, The computer terminal includes a memory and more than one processor; the memory stores more than one program; the program contains instructions for executing the secure three-party computation method for supporting complex non-linear functions as described in any one of claims 1-8; the processor is used to execute the program.

Images