Data normalization processing method and device based on multi-party secure computation

Abstract

The embodiments of this specification provide a data normalization processing method and device based on multi-party secure computation, which are applicable to the process of securely determining the normalization vector of an m-dimensional first vector x stored in n data parties in a shared manner based on the normalization exponential function softmax. The basic idea includes: setting the initial normalization vector as the mean normalization vector of m-dimensional unit vectors, and iteratively correcting it through multiple update rounds for the initial normalization vector to approximate the normalization result of the normalization exponential function. In a single update round, each data party determines the offset of the current update round based on secure multiplication, and during the iteration process, the calculation results of each item are kept in a shared form. In this way, softmax can be approximated more accurately during the multi-party secure computation process, thereby improving the accuracy of softmax data normalization.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of secure computing technology, and in particular, to a data normalization processing method and apparatus based on multi-party secure computing. Background Art

[0002] Multi-party secure computing, also known as secure multi-party computation and abbreviated as MPC for short, means that multiple parties jointly calculate the result of a function without revealing the input data of each party to this function, and the calculation result is made public to one or more of them. Multi-party secure computing can be applied to, for example, private set intersection, joint training of machine learning models, data query, and so on. A typical application is the joint statistical analysis and machine learning of multi-party data with privacy protection. Multi-party secure computing enables the participating parties to calculate statistical results and machine learning results based on the joint data of all parties without exposing their respective original data. The function of multi-party secure computing here can be a function of a statistical operation, a machine learning algorithm, etc., such as the softmax function of normalization exponential. However, for complex functions, other methods convenient for multi-party cooperation are usually adopted for approximation in the multi-party secure computing process. Summary of the Invention

[0003] One or more embodiments of this specification describe a data normalization processing method and apparatus based on multi-party secure computing to solve one or more problems mentioned in the background art.

[0004] According to a first aspect, there is provided a data normalization processing method based on multi-party secure computing, which is applicable to the process of securely determining the normalization vector obtained by normalizing an m-dimensional first vector x stored in a sum-sharing manner by n data parties based on the softmax function of normalization exponential. This process is realized through k update rounds of iterative correction for an initial normalization vector y 0 wherein the initial normalization vector is the mean normalization vector of m-dimensional unit vectors, k is the preset total number of update rounds, and the first party among the n data parties holds the first shard of the first vector x; the method is executed by the first party. In the current t-th update round, where t is an integer greater than 0, the method includes: obtaining the first shard of the current normalization vector y t-1 wherein the first shard of the current normalization vector y t-1 and the other shards held by other data parties form the current normalization vector y t-1 in the sum-sharing form of n data parties; using the first shard of the first vector x and the current normalization vector y t-1The first shard of, and the iteration number k, calculate the offset vector of the current update round with other data parties based on secure multiplication, so as to obtain the first shard of the offset vector locally, and the first shard of the offset vector and other shards obtained by other data parties form a sum-sharing form of the offset vector; correct the current normalized vector y with the first shard of the offset vector t-1 The first shard of, so as to use the correction result as the current normalized vector y updated in the current update round t The first shard of, the current normalized vector y after update t The first shard of and other shards obtained by other data parties form the current normalized vector y after update t In a sum-sharing form of.

[0005] In one embodiment, when t = 1, the current normalized vector y t-1 Is the initial normalized vector, and the first shard of the current normalized vector y t-1 Is randomly split by the semi-trusted service party for the initial normalized vector as the total amount and then distributed to the first party.

[0006] In one embodiment, using the first shard of the first vector x, the first shard of the current normalized vector y t-1 The first shard of, and the iteration number k, calculating the offset of each dimension in the current update round with other data parties includes: using the first shard of the first vector x, the first shard of the current normalized vector y t-1 The first shard of, based on secure multiplication with other data parties, determine the magnitude vector describing the respective offset magnitudes corresponding to each dimension, so as to obtain the first shard of the magnitude vector locally; according to the first shard of each offset magnitude and the first shard of the current normalized vector y t-1 The first shard of, based on secure multiplication corresponding to each dimension with other data parties, determine the respective offsets corresponding to each dimension in the t-th iteration update, so as to obtain the first shard of each offset locally.

[0007] In a further embodiment, using the first shard of the first vector x, the first shard of the current normalized vector y t-1 The first shard of, based on secure multiplication with other data parties, determine the magnitude vector describing the respective offset magnitudes corresponding to each dimension, so as to obtain the first shard of the magnitude vector locally includes: determining the first vector x and the current normalized vector y with other data parties based on secure multiplication t-1The inner product is obtained to get the first shard of the inner product locally; the first shard of the inner product is extended to the first shard of the first reference vector by using an m-dimensional unit vector; the first difference vector between the first shard of the first vector x and the first shard of the first reference vector is determined as the first shard of the amplitude vector.

[0008] In a further embodiment, the step of extending the first shard of the inner product to the first shard of the first reference vector by using an m-dimensional unit vector includes: obtaining the first shard of the m-dimensional first reference vector with each dimension being the first shard of the inner product based on the product of the first shard of the inner product and the m-dimensional unit vector.

[0009] In another further embodiment, the step of determining the inner product of the first vector x and the current normalized vector y with other data parties based on secure multiplication t-1 to obtain the first shard of the inner product locally includes: for the m dimensions of the first vector x and the current normalized vector y t-1 respectively determining the first shards of m products based on secure multiplication with other data parties; summing up the first shards of the m products as the first shard of the inner product obtained locally.

[0010] In an embodiment, based on secure multiplication with each of the other data parties, the first shard of the offset corresponding to each dimension in the t-th iterative update is determined according to the first shard of the amplitude vector and the first shard of the current normalized vector y t-1 so as to obtain the first shard of the offset vector describing each offset locally. For a single dimension of the current normalized vector y t-1 based on secure multiplication with each of the other data parties, the first shard of the product of the value of the amplitude vector and the current normalized vector y t-1 on this single dimension is determined; the first shard of the product of the value on this single dimension is averaged by k, and the first shard of the offset corresponding to this single dimension is determined according to the average result; the first shard of the offset vector describing each offset is obtained according to the first shards of the m offsets corresponding to the m dimensions.

[0011] In an embodiment, the step of correcting the first shard of the current normalized vector y by the first shard of the offset vector t-1 includes: superimposing the first shard of the offset on the first shard of the current normalized vector y t-1 so as to correct the first shard of the current normalized vector y t-1 of the first shard.

[0012] According to a second aspect, there is provided a data normalization processing device based on multi-party secure computation, which is applicable to the process of securely determining and sharing the normalization vector of an m-dimensional first vector x stored in n data parties based on the normalization exponential function softmax. This process is achieved through k rounds of iterative correction for the initial normalization vector y 0 wherein the initial normalization vector is the mean-normalized vector of m-dimensional unit vectors, k is the preset total number of update rounds, and the first party among the n data parties holds the first shard of the first vector x; the device is provided in the first party and includes an acquisition unit, an offset determination unit, and an update unit. At the current t-th update round:

[0013] The acquisition unit is configured to acquire the first shard of the current normalization vector y t-1 where the first shard of the current normalization vector y t-1 and the other shards held by other data parties form the current normalization vector y t-1 in the form of sum sharing among n data parties;

[0014] The offset determination unit is configured to use the first shard of the first vector x, the first shard of the current normalization vector y t-1 and the number of iterations k to perform secure multiplication with other data parties in the form of sum sharing to calculate the offset vector for the current update round, thereby obtaining the first shard of the offset vector locally;

[0015] The update unit is configured to correct the first shard of the current normalization vector y t-1 through the first shard of the offset vector, and use the correction result as the first shard of the current normalization vector y t after being updated in the current update round. The first shard of the updated current normalization vector y t and the other shards obtained by other data parties form the sum sharing form of the updated current normalization vector y t .

[0016] According to a third aspect, there is provided a computer-readable storage medium, on which a computer program is stored. When the computer program is executed on a computer, the computer is made to execute the method of the first aspect.

[0017] According to a fourth aspect, there is provided a computing device, including a memory and a processor, characterized in that the memory stores executable code, and when the processor executes the executable code, the method of the first aspect is implemented.

[0018] Through the method and device provided in the embodiments of this specification, during the softmax process of the multi-party secure computing normalized exponential function, based on the mean-normalized vector of the m-dimensional unit vector, and via the vector x to be normalized, the offset based on the current normalized vector is gradually determined, thereby correcting the current normalized vector so that it approximates the softmax-normalized vector after k update rounds. Among them, the offset of a single update round is jointly determined by the current normalized vector and the vector x to be normalized, and is based on secure multiplication. During the calculation process, the calculation results of each item are kept in a shared form. In this way, the softmax function can be more accurately approximated during the multi-party secure computing process, thereby improving the accuracy of data normalization processing through softmax. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1 Shows a schematic diagram of an implementation architecture according to the technical concept of this specification;

[0021] Figure 2 Shows a schematic diagram of the operation flow of a single data party in a single update round during the data normalization process based on multi-party secure computing according to an embodiment;

[0022] Figure 3 Shows a flowchart of the operation of a single data party during the process of determining the offset in a single update round according to an embodiment;

[0023] Figure 4 Shows a flowchart of the method for the interactive execution of both parties in the two-party secure multiplication of a specific example;

[0024] Figure 5 Shows an error schematic diagram of a specific experimental result of comparing the technical solution provided in this specification and the ASM solution with softmax respectively;

[0025] Figure 6 Shows a schematic block diagram of the data normalization processing device based on multi-party secure computing provided in the first party according to an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The following describes the technical solutions provided in this specification with reference to the drawings.

[0027] First, some concepts that may be involved in this specification are described.

[0028] Sharing: In two-party secure computation, an integer \(x\) in the range of \(0\) to \(M - 1\) is distributed and stored in two parties in the form of \(x=x L +x R modulo \(M\), such that one party does not know \(x R , and the other party does not know \(x L , \(x R , \(x L are called the shards of the integer \(x\) in the sharing form, and neither party can obtain the complete form of \(x\); furthermore, the two parties can be extended to multiple parties.

[0029] Secure computation in sharing form: Keep the sharing form for computation. During the computation process, a single party can never infer the parts of each data stored in other parties, and the computation result is still stored in the sharing manner.

[0030] Normalized exponential function softmax: Normalize the \(m\)-dimensional data in an exponential way, and its definition is: where \(x i represents the \(i\)-th dimensional data.

[0031] Since the normalized exponential function can map multiple values to values between \(0\) and \(1\), and the sum of these values is \(1\), therefore, the normalized exponential function softmax is often used for probability prediction in multiple business categories, and thus is used in classification models or prediction models for probability prediction and ranking in multiple business categories. For example, in the scenario of jointly training a business model based on multi-party secure computation, the corresponding business model can use the normalized exponential function softmax. Here, the business model can be, for example, various models in which multiple business parties in multi-party secure computation jointly process relevant business data while protecting privacy. For example, in the e-commerce field, each shopping platform, as the data holder, takes the local consumption data as private data, takes consumers as the sample subjects, extracts the input features of a training sample from the single-consumer user's single consumption data (such as user basic information, historical browsing data, historical search data, click data), and uses the final purchase data as the sample label, so as to jointly build a prediction model for predicting the interested products of users when they click, browse, and search on the shopping platform next time. The prediction model can normalize the processing results of various business features through softmax in the layer before the output layer, so as to map them to the interval of \(0 - 1\).

[0032] In the scenario of multi-party secure computation, the calculation of softmax includes the base and the exponent. Usually, first calculate \(\exp(x i ), and then calculate softmax, and fixed-point numbers are used during the calculation process, that is, the position of the decimal point is fixed. However, during the calculation process, since \(x iStored in each data party in a shared manner, when the shared shard value is large, it is easy to cause out-of-bounds.

[0033] In this regard, conventional techniques can use the ASM function approximation method to safely calculate softmax, that is: Among them, the ASM function and softmax have the following common points: the numerical size of each dimension data is positively correlated with the normalization result; the normalization results are all positive numbers; the sum of the normalization values is 1. Therefore, the ASM function can be used to approximate softmax. However, since the ASM function uses the rectified linear unit ReLU to discard the values less than 0 (ReLU(j) = max(0, j)), therefore, the normalization result approximated by the ASM function has a large error compared with the softmax normalization.

[0034] In view of this, this specification proposes an iterative method, which converts the calculation of softmax into a safe calculation containing only multiplication and addition during the iteration process, and effectively avoids the out-of-bounds problem on the basis of ensuring accuracy in the shared form.

[0035] Under the technical concept of this specification, the problem of multi-party secure calculation of softmax can be transformed into an iterative operation containing only secure multiplication and secure addition, thereby avoiding the out-of-bounds problem existing in exponential calculation and improving the approximation accuracy of exponential function normalization.

[0036] Specifically, for an m-dimensional vector x to be normalized, denoted as x = (x1, x2... x m ) T , an initial normalized vector can be defined. This initial normalized vector is, for example, the normalization of an m-dimensional unit vector (denoted as I m ) according to the mean value. For example, each dimension is 1 / m, such as y 0 = (1 / m, 1 / m... 1 / m) T = 1 / m(1, 1... 1) T = I m / m = (y 0 1, y 0 2... y 0 m ) T . On the basis of this mean-normalized vector, the influence of x is superimposed, that is, the offset on the current normalized vector is determined based on x and the current normalized vector, and the current normalized vector is corrected based on this offset. Iterating in this way makes the mean-normalized vector gradually approach the normalization result of the softmax normalization exponential function. Assuming the number of iterations is k, the first iteration is performed on the basis of the initial y 0 , and the normalized vector after iteration is y 1, and so on. The t-th iteration is performed based on y t-1 . The normalized vector after iteration is y t . Among them, the integer t satisfies: 1 ≤ t ≤ k, where k is the preset number of iterations, such as 10.

[0037] Figure 1 shows a schematic diagram of a specific implementation architecture of this specification. In the specific implementation scenario of multi-party secure computing, the shards of x at each data party (hereinafter represented by n data parties) are in the form of sum sharing of x. The preset total number of iterations k, the vector dimension m, and the unit vector I m can all be constants publicly available to n data parties. The shards of y t at n data parties are in the form of sum sharing. During the calculation process, the initial normalized vector y 0 can be randomly split into n shards by a third party and distributed to n data parties. Thus, the initial normalized vector y 0 forms a sum sharing form among n data parties.

[0038] During a single iteration process, it involves the superposition of the current normalized vector y t-1 and the offset updated in the current iteration. Since the current normalized vector y t-1 and the offset updated in the current iteration both have shards in the form of sum sharing at a single data party, therefore, a single data party can, by superimposing the local shards in the form of sum sharing, serve as the shard of the corrected current normalized vector and form a sum sharing form of the corrected current normalized vector with the shards obtained by other data parties.

[0039] Among them, the offset during a single iteration process can be determined based on the vector x to be normalized and the current normalized vector y t-1 . Or rather, it is determined based on the processing result of processing the current normalized vector y t-1 by the vector x to be normalized. Among them, this processing process can be realized through vector multiplication, vector addition or a combination thereof between two vectors. For example, the normalized vector x multiplies the current normalized vector y t-1 dimension by dimension, the normalized vector x takes the scalar inner product with the current normalized vector y t-1 and multiplies the current normalized vector y t-1 to expand it into an m-dimensional vector, and so on.

[0040] In one embodiment, the scalar inner product of the normalized vector x and the current normalized vector y t-1 can be extended to an extended vector with the same value for each dimension of m through the unit vector, and then the difference between x and the extended vector with the same value for each dimension is compared. The difference vector describing this difference is combined with the current normalized vector y t-1Multiply by dimension and consider the influence of the number of iterations on the offset to determine the corresponding offset in a single update process.

[0041] As an example, the offset for the t-th iteration (1 ≤ t ≤ k) is: P t =(x - <x, y t-1 >I m )·y t-1 / k, then the t-th iteration (1 ≤ t ≤ k) can be carried out based on y t-1 , for example:

[0042] y t =y t-1 +P t =y t-1 +(x - <x, y t-1 >I m )·y t-1 / k.

[0043] Among them, in the case of t = k, y k is the calculation result of the softmax of the normalization exponential function for the m-dimensional vector x, <x, y t-1 > represents the inner product (scalar) of the vector x and the vector y t-1 , (x - <x, y t-1 >I m )·y t-1 represents the vector obtained by multiplying the corresponding dimensions of the vector (x - <x, y t-1 >I m ) and the vector y t-1 .

[0044] In this way, for the t-th iteration process, there is a first term y t-1 in the form of addition, and a second term (x - <x, y t-1 >I m )·y t-1 / k. Since k is a constant and is publicly available for each data party, each data party can hold 1 / k as a constant. The part of the second term that actually performs secure calculation is (x - <x, y t-1 >I m )·y t-1 . Substituting each iteration in turn, the k-th iteration is:

[0045]

[0046] Since k is a predetermined value and the reciprocal of k is a publicly available constant, the multi-party secure calculation in the iteration process only involves addition and multiplication in the form of summation and sharing, effectively avoiding the out-of-bounds problem in the division operation of softmax.

[0047] The following analyzes the traffic volume under the iterative scheme provided in this specification and the accuracy of the softmax secure calculation results in combination with further embodiments.

[0048] Figure 2 Shows the data normalization process based on multi-party secure calculation in an embodiment. As can be seen from the foregoing, the data normalization processing scheme provided in this specification is used to simulate the normalization result of the normalization exponential function softmax. Specifically, exponential normalization is performed on the m-dimensional first vector x, and the respective shards of the first vector x among n data parties are in a composition and sharing form. Among them, the n data parties can respectively hold n shards of the first vector x, such as respectively holding each m-dimensional vector x 1 、x 2 ……x n ,and x 1 +x 2 +……+x n =x. The normalization result is still an m-dimensional vector and is in a composition and sharing form among n data parties. For example, the normalization result is denoted as the m-dimensional vector y, and each of the n data parties locally obtains the m-dimensional vector y 1 、y 2 ……y n ,and y 1 +y 2 +……+y n =y.

[0049] Here, it can be assumed that any data party is denoted as the first party, and the shard of the first vector x held by it is denoted as the first shard of the first vector x (such as denoted as x 1 ). The n data parties can iteratively update the current normalized vector through k update rounds. Figure 2 In the shown process, taking the current as the t-th (t is an integer greater than 0) update round as an example, the data remains in a composition and sharing form among n data parties, and the current normalized vector y t-1 is processed by the first vector x, t so as to obtain the normalized vector y t-1 of the t-th iteration, which is used to update y t as the new current normalized vector. Given that various calculation results remain in a composition and sharing form among n data parties, the first party can obtain a shard of the new current normalized vector y t , such as denoted as the first shard (y 1 ).

[0050] Specifically, as Figure 2 shown, taking the t-th update round as an example, the operations performed by the first party may include the following steps:

[0051] Step 201, obtain the current normalized vector y t-1The first shard of, where the current normalized vector is y t-1 The first shard of forms the current normalized vector y together with other shards held by other data parties t-1 In the sum-sharing form among n data parties; Step 202, using the first shard of the first vector x, the first shard of the current normalized vector y t-1 The first shard of, and the iteration number k, calculate the offset vector for the current update round with other data parties based on secure multiplication, so as to locally obtain the first shard of the offset vector. The first shard of the offset vector and other shards obtained by other data parties form the sum-sharing form of the offset vector; Step 203, correct the first shard of the current normalized vector y t-1 The first shard of, and use the correction result as the current normalized vector y updated in the current update round t The first shard of, the updated current normalized vector y t The first shard of forms the updated current normalized vector y together with other shards obtained by other data parties t In the sum-sharing form

[0052] First, through Step 201, obtain the first shard of the current normalized vector y t-1 The first shard of

[0053] It can be understood that in the case where t = 1 and the current iteration period is the first update round, the current normalized vector can be the initial mean-normalized vector y 0 = I m / m. In the case where t > 1 and the current iteration period is not the first update round, the current normalized vector can be the normalized vector y updated in the previous update round t-1 . Among them, n data parties can hold each shard of the current normalized vector y t-1 To form the sum-sharing form of the current normalized vector y t-1 The first party holds the first shard of the current normalized vector y t-1 , for example, denoted as (y t-1 ) 1 . The first shard (y t-1 ) t-1 Of the current normalized vector y 1 And other shards held by other data parties form the sum-sharing form among n data parties. The first party can obtain this first shard locally

[0054] Step 202, using the first shard of the first vector x, the first shard of the current normalized vector y t-1 The first shard of, and the iteration number k, calculate the offset vector for the current update round with other data parties based on secure multiplication, so as to locally obtain the first shard of the offset vector

[0055] Since mean normalization is a normalization result in a special case and is usually applicable to the case where the values participating in the normalization are the same, while in actual business scenarios, the values participating in the normalization are often different. Therefore, it is necessary to adjust based on the actual vector to be normalized, such as the first vector x, on the basis of the current normalized vector. And how much adjustment is needed in the current update round is usually determined by the current normalized vector. For example, it is adjusted according to a certain proportion or a certain amplitude of the current normalized vector. Therefore, the offset vector (composed of the offsets of each dimension) in the current update round can be jointly determined by the first vector x and the current normalized vector. In some embodiments, considering the influence of the iteration number k, the offset of a single update round can also be adjusted by k. For example, the offset is obtained by dividing the processing result of the first vector x on the current normalized vector by k.

[0056] The processing process of the first vector x on the current normalized vector can be carried out in various reasonable ways.

[0057] According to an optional implementation manner, the current normalized vector y can be determined first t-1 The offset amplitude of each dimension, and then the offset of each dimension is determined according to the product of the offset amplitude and the current normalized vector y t-1 Among them, the m offsets can be represented by an m-dimensional vector corresponding to the current normalized vector y t-1 Such as denoted as the offset vector.

[0058] In one embodiment, the offset amplitude can be determined by the first vector x. The offset amplitude of each dimension is, for example, positively correlated with the value size of each dimension of x. For example, the offset amplitude of a single dimension is the value of the corresponding dimension of the first vector x itself, or the value obtained by dividing by the iteration number k. At this time, the offset vector can be x, or x / k. In this way, the first party can, according to the first shard held locally in the sum sharing form of the first vector x, determine the first shard of the offset vector locally, such as denoted as (P t ) 1 . The first shard of this offset vector and the other shards obtained locally by other data parties form the sum sharing form of the offset vector.

[0059] In another embodiment, n data parties can jointly determine the offset amplitude corresponding to each of the m dimensions based on the first vector x and the current normalized vector y t-1 For example, using the product of the corresponding multiplication of each dimension of the first vector x and the current normalized vector y t-1 As the offset amplitude of the corresponding dimension, and each offset amplitude constitutes an offset vector. At this time, the first party can use the first shard of the first vector x, the current normalized vector y t-1For the first shard, based on secure multiplication with each of the other data parties, determine the magnitude vector describing the respective offset magnitudes corresponding to each dimension, so as to obtain the first shard of the magnitude vector locally.

[0060] In yet another embodiment, the scalar inner product of the first vector x and the current normalized vector y t-1 can be extended to an m-dimensional vector and then processed with the first vector x to obtain the offset vector. As in the previous example, the offset vector can be P = x - <x, y t-1 >I m .

[0061] Figure 3 Illustrates the process by which the first party determines the offset vector in this case. As Figure 3 shown, the process by which the first party determines the offset vector is as follows:

[0062] Step 301, use the first shard of the first vector x and, via secure multiplication with other data parties, determine the inner product of the first vector x and the current normalized vector y t-1 to obtain the first shard of this inner product locally.

[0063] It can be understood that the inner product of the first vector x and the current normalized vector y t-1 is a scalar, which can be understood as the fusion result of adding up the products corresponding to each dimension of the first vector x and the current normalized vector y . Since in the sum-sharing form, the vectors x and y as multipliers t-1 both have the sum-sharing form among n data parties, therefore, the secure calculation process of this inner product is based on secure multiplication in the sum-sharing form. t-1

[0064] Figure 4 Illustrates the secure multiplication calculation process in the sum-sharing form between two data parties. As Figure 4 shown, two-party secure calculation is performed with the assistance of a semi-trusted service party. Among them, when both data parties are in the sum-sharing form, the semi-trusted service party as the multiplier is used to assist in generating the random numbers used in the secure multiplication process. In Figure 4 , A and B represent two data parties, x0 and x1 are the sum-sharing forms of the multiplier x, and y0 and y1 are the sum-sharing forms of the multiplier y. The semi-trusted service party generates three random numbers u, v, and a that satisfy the constraint uv = a, and splits the three random numbers into sum-sharing forms and provides the relevant shards to data parties A and B respectively. Data parties A and B exchange t0 = x0 - u0, s0 = y0 - v0, t1 = x1 - u1, and s1 = y1 - v1 calculated based on their local shards. Thus, both parties A and B obtain a shard w0 and w1 of xy = w respectively.

[0065] Specifically, for the m-dimensional vectors x and y formed and shared by n data parties t-1 , u0, u1, v0, v1, t0, t1, s0, and s1 in Figure 4 can all be replaced with their corresponding vector forms, while the vector inner products or inner product slices a0, a1, w0, and w1 are scalar values. Assume that the first slice of the first vector x is denoted as x 1 , and the first slice of the sliced current normalized vector y t-1 (y t-1 ) 1 , while the other slices of the first vector x and the current normalized vector y t-1 are distributed among other data parties. To calculate the inner product of x×y t-1 , it is necessary to securely compute by n data parties and obtain the corresponding slices in the corresponding sum and share form on a single data party through calculation.

[0066] During the multi-party secure computing process, this computing process can be split into:

[0067]

[0068] Assume that the first party corresponds to the slice with superscript 1. Then the first party can locally compute x 1 × (y t-1 ) 1 , and perform two-party secure computing with the second party for x 1 × (y t-1 ) 2 , x 2 × (y t-1 ) 1 , and obtain the scalar slices corresponding to the two inner products of x 1 × (y t-1 ) 2 , x 2 × (y t-1 ) 1 respectively. These two scalar slices and the corresponding slices obtained by the second party respectively form the sum and share form of x 1 × (y t-1 ) 2 , x 2 × (y t-1 ) 1 .

[0069] By extension, in the case where each j≠1, after two two-party secure multiplication computations for x 1 × (y t-1 ) j , x j × (y t -1 ) 1,. In this way, the first party performs a total of 2(n - 1) two-party secure multiplication calculations. After each two-party secure calculation, the first party can obtain an inner integral slice of x×y t-1 and gets a total of 2(n - 1) inner integral slices. The sum of these slices and the inner product of constitutes x×y t-1 The inner integral slice is the first slice of the first party. This first slice and the other slices determined by other data parties in a similar way constitute the sum sharing form of x×y t-1 .

[0070] Step 302, expand the first slice of the above inner product through an m-dimensional unit vector to obtain the first slice of the first reference vector.

[0071] Since the above inner product is in scalar form, in order to obtain the iterative vector superimposed on the current normalized vector, the m-dimensional unit vector I m can be used for expansion. The expansion result <x, y t-1 >I m is, for example, denoted as the first reference vector. According to the definition of the unit vector and the multiplication of a scalar and a vector, the numerical values of each dimension of the expanded first reference vector are all the above inner product. Considering the particularity of the expansion based on the unit vector, in the case where the above inner product constitutes the sum sharing form among n data parties, a single data party can use the local inner product-related slices and expand them locally with the m-dimensional unit vector I m to obtain the corresponding expanded vector as the corresponding slice of the first reference vector. For example, based on the product of the m-dimensional unit vector and the first slice of the above inner product, the first party expands the first slice of the inner product into the first slice of the first reference vector, such as denoted as [<x, y t-1 >I m 1 . The expanded vectors obtained by each data party locally constitute the sum sharing form of the first reference vector.

[0072] Step 303, determine the first difference vector between the first slice of the first vector x and the first slice of the first reference vector as the first slice of the amplitude vector.

[0073] Since in the iterative process, based on the first vector x for the current normalized vector y t-1 processing, each time a shift amount is superimposed on the current normalized vector y t-1 basis, and the shift amount of a single dimension is determined by the relationship between the value of this dimension and the values of all dimensions. The first reference vector describes the numerical characteristics of all current dimensions in the processing result of the first vector x for the current normalized vector y t-1 . Therefore, in the process of determining the shift amount of the current iteration period, by fusing the first vector x and the first reference vector <x, y​t-1 >I m The offset amplitude in each dimension can be determined.

[0074] For example, the fusion of the first vector x and the first reference vector can be performed by determining the difference vector. The fusion result is, for example: x - <x, y t-1 >I m . During the multi-party secure calculation process, in the sum-sharing form, each data party can determine the corresponding shard of the difference vector according to the corresponding shards of the minuend and the subtrahend respectively. For the first party, which holds the first shard of the first vector x and the first shard of the first reference vector, it can locally determine the first shard of the difference vector, such as: x 1 - <x, y t-1 >I m 1 .

[0075] The first shard of this difference vector can be used as the first shard of the offset vector. Optionally, considering that it is necessary to gradually add an offset to the current normalized vector through k iterations and gradually correct the current normalized vector to approximate the normalized value of softmax, the first shard of the difference vector can also be divided by the number of iterations k, that is, each dimension of the first shard of the difference vector is divided by the value k, so as to obtain the first shard of the offset vector.

[0076] In other embodiments, the offset amplitude can also be determined by other reasonable methods, so that the first shard of the amplitude vector can be obtained by the first party, which will not be elaborated here one by one.

[0077] After obtaining the offset vector, the n data parties can also determine the specific offset based on multi-party secure calculation.

[0078] According to a possible design, in order to determine the current offset on the basis of the current normalized vector, the amplitude vector describing the offset amplitude in each dimension can be multiplied by the current normalized vector y t-1 dimension by dimension. The first party can use the first shard of the amplitude vector to securely multiply with other data parties to determine the product of the amplitude vector and the current normalized vector y t-1 multiplied dimension by dimension, so as to locally obtain the first shard of this product. The first shard of this product and the other shards obtained by other data parties form the sum-sharing form of the product of the amplitude vector and the current normalized vector y t-1 multiplied dimension by dimension. Among them, multiplying dimension by dimension means that the first dimension of the amplitude vector is multiplied by the first dimension of the current normalized vector y t-1 , the second dimension of the amplitude vector is multiplied by the second dimension of the current normalized vector y t-1 , and so on. The obtained product is still an m-dimensional vector. ​

[0079] As an example, refer to Figure 3 the case where the difference vector obtained from the shown process is used as the magnitude vector. For convenience of description, the difference vector can be denoted as d, then d = x - <x, y t-1 >I m =(d1, d2,... d m ), and the product of the difference vector and the current normalized vector y t -1 can be: P = (d1, d2,... d m ) × y t-1 =(d1 × y1 t-1 , d2 × y2 t-1 ,... d m × y m t-1 ).

[0080] Since both d and y t-1 are in the form of composition and sharing by n data parties, a single dimension such as d i , y i t-1 is respectively in the form of composition and sharing by n data parties. Then there is:

[0081]

[0082] For a single dimension, the first party can perform 2(n - 1) two-party secure multiplication operations and obtain a shard of the corresponding dimension locally. This shard and the other shards of the other n - 1 data parties form a sum-sharing form of the offset of this dimension.

[0083] Optionally, it is also possible to refer to the way of vector inner product multiplication. When performing multiplication operations by dimension, replace Figure 4 the relevant data transmitted in with vectors. Different from the inner product described above, a0, a1, w0, and w1 are all in vector form. In the vector calculation form, the number of communications is reduced and the communication volume remains unchanged.

[0084] In some alternative embodiments, since k iterations are to be performed, the offset of a single iteration can be controlled by a preset number of iterations. For example, the above product is averaged by k as the offset of the current iteration, such as denoted as P / k, that is, (x - <x, y t-1 >I m )·y t-1 / k. It is also possible to divide the above product by the total number of iterations k as the offset. It should be noted that generally, when 1 / k has been taken when determining the offset amplitude, k may not be considered anymore when determining the offset based on the offset amplitude. When the iteration number k is not considered when determining the offset amplitude, when determining the offset based on the offset amplitude, each data party can divide the local shard by k, or multiply by 1 / k.

[0085] Step 203, correct the first shard of the current normalized vector y through the first shard of the above offset vector, and use the correction result as the current normalized vector y updated in the current update round. t-1 The first shard of, where the correction result is used as the first shard of the current normalized vector y updated in the current update round. t It can be understood that the first shard of the current normalized vector y updated by the first party t Combines with the other shards obtained by other data parties to form the sum-sharing form of the updated current normalized vector y. t

[0086] Since the correction process of the current normalized vector y t-1 Is a summation process. For the two addends in the sum-sharing form composed of n data parties, a single data party can calculate the local sum shard locally using the local addend shard as one shard of the correction result. Each data party can obtain a sum shard locally, and the sum shards form the sum-sharing form of the correction result (denoted as y t-1 ) of the current normalized vector y. t

[0087] In the above process, the basic idea of the process of securely determining the normalized vector of the m-dimensional first vector x stored in n data parties in sum-sharing manner based on the normalized exponential function softmax includes: setting the initial normalized vector as the mean normalized vector of the m-dimensional unit vectors, and performing iterative correction through multiple update rounds for the initial normalized vector to approximate the normalized result of the normalized exponential function. In a single update round, each data party determines the offset of the current update round based on secure multiplication, and in the iterative process, the calculation results of each item maintain the sum-sharing form. This way of updating through multiple rounds has a relatively high accuracy for the approximate result of softmax normalization. Thus, in the business processing process based on softmax (such as the prediction business processing process of consumer interested products using the prediction model based on softmax as described above), the accuracy of business processing is improved.

[0088] The following combines Figure 5 The experimental results shown to illustrate the accuracy of the technical solution provided in this specification. Figure 5 The table shown is for 10 randomly selected groups of 10 (m = 10)-dimensional standard normal distribution samples. The ASM method and the iterative method provided in this specification are respectively used (through y t = y t-1 +(x - <x, y t-1 >I m )·y t-1 ​​The mean squared error values obtained for 10 groups each, by comparing the normalization results obtained through iteration with / k (where k takes the value 16), with the exponential normalization results obtained by the softmax method. As Figure 5 shown, for each group of samples, the order of magnitude of the mean squared error value using the ASM method is 10 -3 or 10 -2 , while for the iterative method provided in this specification, the order of magnitude of the mean squared error value is 10 -5 、10 -6 、10 -7 . That is to say, compared with the ASM method, the iterative method provided in this specification has at least a 2 - 3 order of magnitude improvement in the accuracy represented by the mean squared error in the normalization result simulating softmax, and the accuracy is greatly improved.

[0089] According to an embodiment of another aspect, there is also provided a data normalization processing device for secure multi - party computation. This device can be set in any data party performing secure multi - party multiplication calculation, such as the first party. This device is applicable to the process of securely determining the normalization vector of the m - dimensional first vector x stored in n data parties in a sum - sharing manner based on the normalization exponential function softmax, and this process can be achieved through k update rounds of iterative correction for the initial normalization vector y 0 . Among them, the initial normalization vector can be the mean - normalized vector of the m - dimensional unit vectors, k is the preset total number of update rounds, and the first party among the n data parties holds the first shard of the first vector x. Figure 6 FIG. shows a data normalization processing device 600 for secure multi - party computation according to an embodiment. As Figure 6 shown, the device 600 can include an acquisition unit 601, an offset determination unit 603, and an update unit 604.

[0090] Among them, in the current t - th update round:

[0091] The acquisition unit 601 is configured to acquire the first shard of the current normalization vector y t-1 , where the first shard of the current normalization vector y t-1 and the other shards held by other data parties form the sum - sharing form of the current normalization vector y t-1 among the n data parties;

[0092] The offset determination unit 602 is configured to use the first shard of the first vector x, the first shard of the current normalization vector y t-1 , and the number of iterations k, and perform secure multiplication with other data parties based on the sum - sharing form to calculate the offset vector of the current update round, so as to obtain the first shard of the offset vector locally;

[0093] An update unit 603, configured to correct the current normalized vector y by a first shard of an offset vector, and use the correction result as the current normalized vector y updated in the current update round. t-1 The first shard of t-1 , and the updated current normalized vector y is obtained by using the correction result as the current normalized vector y updated in the current update round. t The first shard of t , and the updated current normalized vector y. t The first shard of t and other shards obtained by other data parties form the updated current normalized vector y in a sum sharing form. t in a sum sharing form.

[0094] It should be noted that Figure 6 the device 600 shown in Figure 6 corresponds to Figure 2 the method described in Figure 2 . The corresponding descriptions in the method embodiments of Figure 2 also apply to the device 600 and will not be repeated here. Figure 2 The corresponding descriptions in the method embodiments of Figure 2 also apply to the device 600 and will not be repeated here.

[0095] According to an embodiment of another aspect, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method described in combination with Figure 2 , Figure 3 and so on.

[0096] According to an embodiment of still another aspect, a computing device is further provided, including a memory and a processor. An executable code is stored in the memory. When the processor executes the executable code, the method described in combination with Figure 2 , Figure 3 and so on is implemented.

[0097] Those skilled in the art should be able to realize that in one or more of the above examples, the functions described in the embodiments of this specification can be implemented by hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0098] The specific embodiments described above further elaborate on the purpose, technical solutions, and beneficial effects of the technical concept of this specification. It should be understood that the above description is only the specific embodiments of the technical concept of this specification and is not used to limit the protection scope of the technical concept of this specification. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions in the embodiments of this specification should be included in the protection scope of the technical concept of this specification.

Claims

1. A data normalization processing method based on multi-party secure computation, applicable to the process of securely determining and sharing the normalization vector obtained by normalizing an m-dimensional first vector x stored in n data parties based on the normalization exponential function softmax. This process is achieved through k update rounds of iterative correction for the initial normalization vector y 0 and is implemented through k rounds of iterative correction of updates, where The initial normalized vector is the mean normalized vector of m-dimensional unit vectors, k is the preset total number of update rounds, and the first of the n data parties holds the first shard of the first vector x; The method is executed by the first party. In the current t-th update round, where t is an integer greater than 0, the method includes: Obtain the current normalized vector y t-1 's first shard, where the current normalized vector y t-1 's first shard and other shards held by other data parties form the current normalized vector y t-1 in the sum-sharing form among n data parties; Using the first shard of the first vector x, the first shard of the current normalized vector y t-1 , and the iteration number k, calculate the offset vector for the current update round with other data parties based on secure multiplication, so as to locally obtain the first shard of the offset vector, and the first shard of the offset vector and other shards obtained by other data parties form a sum-sharing form of the offset vector; Modify the first shard of the current normalized vector y through the first shard of the offset vector, and use the correction result as the current normalized vector y updated in the current update round. t-1 The first shard of t-1 is used to update the first shard of the current normalized vector y, and the corrected result is used as the current normalized vector y updated in the current update round. t The first shard of t , the updated current normalized vector y t The first shard of t and the other shards obtained by other data parties form the updated current normalized vector y t in a sum-sharing form.

2. The method according to claim 1, wherein When t = 1, the current normalized vector y t-1 is the initial normalized vector, and the first shard of the current normalized vector y t-1 is allocated to the first party by the semi-trusted service provider after randomly splitting the initial normalized vector as a total amount.

3. The method according to claim 1, wherein The first shard of the first vector x, the first shard of the current normalized vector y t-1 , and the iteration number k, calculating the offset vector for the current update round with other data parties based on secure multiplication includes: Using the first shard of the first vector x and the first shard of the current normalized vector y t-1 Based on secure multiplication with each of the other data parties, determine the magnitude vector describing the respective offset magnitudes corresponding to each dimension, thereby obtaining the first shard of the magnitude vector locally; Based on the first shard of each offset amplitude and the current normalized vector y t-1 's first shard, determine, based on secure multiplications corresponding to each dimension with other data parties, the respective offsets corresponding to each dimension of the offset vector in the t-th iteration update, so as to obtain the first shard of the offset vector locally.

4. The method according to claim 3, wherein, Using the first shard of the first vector x and the first shard of the current normalized vector y t-1 Based on secure multiplications with each of the other data parties, determining a magnitude vector that describes the respective offset magnitudes corresponding to each dimension, so that obtaining the first shard of the magnitude vector locally includes: Determine the inner product of the first vector x and the current normalized vector y with other data parties based on secure multiplication, so as to obtain the first shard of the inner product locally; t-1 ​ Using the m-dimensional unit vector to expand the first shard of the inner product into the first shard of the first reference vector; Determining the first difference vector between the first shard of the first vector x and the first shard of the first reference vector as the first shard of the amplitude vector.

5. The method according to claim 4, wherein, The using the m-dimensional unit vector to expand the first shard of the inner product into the first shard of the first reference vector includes: Based on the product of the first shard of the inner product and the m-dimensional unit vector, obtaining the first shard of the m-dimensional first reference vector with each dimension being the first shard of the inner product.

6. The method according to claim 4, wherein The inner product of the first vector x and the current normalized vector y determined based on secure multiplication with other data parties, so as to obtain the first shard of the inner product locally includes: t-1 ​ For the first shard of the first vector x and the first shard of the current normalized vector y t-1 respectively, based on the secure multiplication in vector form with other data parties, determine the n - 1 shard - inner products between the first shard of the first vector x and the other n - 1 shards of the current normalized vector y t-1 and the n - 1 shard - inner products between the first shard of the current normalized vector y t-1 and the other n - 1 shards of the first vector x, so as to locally obtain the first shards of the 2(n - 1) shard - inner products respectively; Adding the 2(n - 1) first shards corresponding to the 2(n - 1) shard inner products as the first shard of the inner product obtained locally.

7. The method according to claim 3, wherein Based on the first slice of the amplitude vector and the first slice of the current normalized vector y t-1 determine, based on secure multiplications with each of the other data parties, the respective offsets corresponding to the respective dimensions of the offset vector in the t-th iteration update, so as to locally obtain the first slice of the offset vector, including: For the current normalized vector y t-1 in a single dimension, based on secure multiplications with each of the other data parties, determine a first shard of the product of the values of the magnitude vector and the current normalized vector y t-1 in this single dimension; Averaging the first shard of the product of the values on this single dimension with k, and determining the first shard of the offset corresponding to this single dimension according to the average result; Obtaining the first shard of the offset vector describing each offset according to the first shards of the m offsets corresponding to the m dimensions respectively.

8. The method according to claim 1, wherein Correct the current normalized vector y by the first shard of the offset vector t-1 The first shard includes: Superimpose the first slice of the offset vector on the first slice of the current normalized vector y t-1 to correct the first slice of the current normalized vector y t-1 of the first slice.

9. A data normalization processing device based on multi-party secure computing is applicable to the process of securely determining and sharing the normalization vector of the m-dimensional first vector x stored in n data parties based on the normalization exponential function softmax. This process is achieved through k update rounds of iterative correction for the initial normalization vector y 0 and is implemented, where The initial normalized vector is the mean normalized vector of m-dimensional unit vectors, k is the preset total number of update rounds, and the first of the n data parties holds the first shard of the first vector x; The device is provided in the first party and includes an acquisition unit, an offset determination unit, and an update unit. In the current t-th update round: The obtaining unit is configured to obtain a first shard of the current normalized vector y t-1 where the first shard of the current normalized vector y t-1 and other shards held by other data parties form the current normalized vector y t-1 in a sum-sharing form among n data parties; The offset determination unit is configured to use the first shard of the first vector x, the first shard of the current normalized vector y t-1 and the iteration number k to calculate the offset vector of the current update round through secure multiplication in the form of sum and sharing with other data parties, so as to obtain the first shard of the offset vector locally; The update unit is configured to correct a first shard of the current normalized vector y by a first shard of the offset vector, and use the correction result as the current normalized vector y updated in the current update round. t-1 The first shard of t-1 is used as the first shard of the current normalized vector y updated in the current update round. t The first shard of t is the first shard of the updated current normalized vector y. t The first shard of t and other shards obtained by other data parties form a sum-sharing form of the updated current normalized vector y. t The sum-sharing form of t .

10. A computer-readable storage medium, on which a computer program is stored. When the computer program is executed in a computer, the computer is made to execute the method according to any one of claims 1-8.

11. A computing device, comprising a memory and a processor, characterized in that, An executable code is stored in the memory. When the processor executes the executable code, the method according to any one of claims 1-8 is implemented.

Images