Data processing system and method

Abstract

The present invention discloses a data processing system and method, which includes: a data splitting node, an intermediate node, and a merging node; the data splitting node is used to split data into two or more data slices, submit one data slice to the intermediate node configured with a first model, and submit the other data slices to the intermediate node configured with a second model, wherein the second model and the first model are neural network models with the same model structure but different constant bias terms; the intermediate node is used to run the model configured thereon to process the data slices to obtain slice-processed data; and the merging node is used to reconstruct and merge the slice-processed data output by the intermediate node to obtain data processing results. This system implements the per-use and non-depositability of user data and models, making it possible to use user data based on authorization.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data processing system and method. Background Art

[0002] Currently, the technology of using machine models to learn and train massive amounts of data is widely used. In machine learning scenarios, there may be multiple data holders and model holders. To promote the application of deep learning in the industrial world, there is a need for data sharing between data holders and model holders. To ensure the security and privacy of data during data sharing, existing technologies provide the following solutions:

[0003] The first approach involves breaking the model down into two parts: feature extraction and a neural network model. During the forward propagation phase of model training, the data owner passes the data through their local feature extraction network to obtain an output vector, which they then send to the model owner's neural network model. During the backward propagation phase of model training, the model owner passes the gradient to the data owner and requests their help in updating the network parameters of the previous part.

[0004] From the perspective of transfer learning, this approach doesn't allow the data holder to fully recover the data transferred from the local model, but it does prevent the model holder from using it for other purposes. Furthermore, the network structure of the previous model will be accessible to the data holder, which is unacceptable to the model holder.

[0005] The second approach involves encrypting the data. The data holder encrypts the training samples using homomorphic encryption technology and sends them to the model holder for model training. The trained model then uses the same encryption method on the running data and passes it into the network to obtain predictions. Homomorphic encryption differs from simple mapping encryption methods in that it preserves the linear relationships between continuous-valued features. However, this approach incurs high computational overhead due to training on encrypted data. Furthermore, homomorphic encryption requires that the encryption method used for prediction data be consistent with that used for model training. To ensure data security for the training samples, the encryption method cannot be disclosed.

[0006] In summary, the two existing solutions are usually only applicable to scenarios where the data holder and model holder are the same. They are not very universal and cannot be used as general solutions. Summary of the Invention

[0007] In view of the above problems, the present invention is proposed to provide a data processing system and method that overcome the above problems or at least partially solve the above problems.

[0008] An embodiment of the present invention provides a data processing system, comprising: a data splitting node, an intermediate node, and a merging node;

[0009] A data splitting node is used to split the data into two or more data slices, submit one data slice to an intermediate node set with a first model, and submit the other data slices to an intermediate node set with a second model, wherein the second model and the first model are neural network models with the same model structure but different constant bias terms;

[0010] The intermediate node is used to run the model set on it to process the data shards and obtain the shard processing data;

[0011] The merge node is used to reconstruct and merge the sharded processing data output by the intermediate nodes to obtain the data processing results.

[0012] In some optional embodiments, the data splitting node is used to split the data into two or more data shards, including:

[0013] The original vector of the data is split into multiple slice vectors with the same dimension as the original vector; the sum of the element values ​​at the same position in all the slice vectors is equal to the element value at the corresponding position in the original vector.

[0014] In some optional embodiments, the intermediate node is configured to run the model configured thereon to process the data shards to obtain shard-processed data, including:

[0015] Each intermediate node receives the data fragments sent by the data splitting node, uses the set first model or second model to perform polynomial expansion calculation on the data fragments, and calculates the polynomial variable y based on the pre-generated random number pair. i Encryption is performed based on the encrypted polynomial variable y i Get the shard processing data corresponding to the data shard.

[0016] In some optional embodiments, the data splitting node selects a corresponding data splitting method to split the data according to the data reconstruction method set by the merging node, and the intermediate node uses a corresponding model to process the data fragments.

[0017] In some optional embodiments, a first model is set in one intermediate node, and a second model that cannot be used alone is set in other intermediate nodes, including:

[0018] Perform polynomial approximate expansion on the original model function to obtain a first model, which is set in an intermediate node;

[0019] The constant terms in the polynomial approximate expansion result of the original model function are set to zero and / or the constant bias term in at least one non-constant term is set to zero to obtain a second model, which is respectively set in other intermediate nodes.

[0020] An embodiment of the present invention further provides a data processing method based on multi-party computing, including:

[0021] The data splitting node splits the data into two or more data slices, submits one data slice to the intermediate node configured with the first model, and submits the other data slices to the intermediate nodes configured with the second model; wherein the second model uses the same neural network model as the first model but has a different constant bias term;

[0022] The intermediate node runs the model configured on it to process the data shards, obtains the sharded processed data, and provides it to the merging node;

[0023] The merging node reconstructs and merges the sharded processing data output by each intermediate node to obtain the data processing result.

[0024] In some optional embodiments, the data splitting node splits the data into two or more data shards, including:

[0025] The original vector of the data is split into multiple slice vectors with the same dimension as the original vector; the sum of the element values ​​at the same position in all the slice vectors is equal to the element value at the corresponding position in the original vector.

[0026] In some optional embodiments, the intermediate node runs the model configured thereon to process the data shards to obtain shard-processed data, including:

[0027] Each intermediate node receives the data fragments sent by the data splitting node, uses the set first model or second model to perform polynomial expansion calculation on the data fragments, and calculates the polynomial variable y based on the pre-generated random number pair. i Encryption is performed based on the encrypted polynomial variable y i Get the shard processing data corresponding to the data shard.

[0028] In some optional embodiments, the data splitting node selects a corresponding data splitting method to split the data according to the data reconstruction method set by the merging node, and the intermediate node uses a corresponding model to process the data fragments.

[0029] In some optional embodiments, setting a first model in one intermediate node and setting a second model that cannot be used alone in other intermediate nodes includes:

[0030] Perform polynomial approximate expansion on the original model function to obtain a first model, which is set in an intermediate node;

[0031] The constant terms in the polynomial approximate expansion result of the original model function are set to zero and / or the constant bias term in at least one non-constant term is set to zero to obtain a second model, which is respectively set in other intermediate nodes.

[0032] The beneficial effects of the above technical solutions provided by the embodiments of the present invention include at least:

[0033] The data processing system provided by the embodiment of the present invention can ensure the dual security of data and models. On the one hand, the present application introduces multiple intermediate nodes, and by splitting the data into data shards, the data processing process is distributed to different intermediate nodes for execution, and the intermediate nodes are handed over to the data holder and the model holder for management respectively, thereby ensuring that the intermediate nodes cannot collude. Therefore, even if the data shard is running on the intermediate node managed by the model holder, due to the lack of the data shard on the intermediate node managed by the data holder, the data cannot be recovered and reused. This makes it impossible for the model holder to reuse and train multiple times with the data shard obtained by itself, nor can it use the obtained data shard to restore the original data, thereby ensuring the privacy of the data; on the other hand, in order to ensure the effective processing of data, the first model and the second model set by the intermediate node are neural network models with the same structure, but because the second model is a model with the constant term of the first model changed, when the second model is set on the intermediate node managed by the data holder, the data holder cannot obtain the real data of the model. Therefore, this system realizes the sharing of data and models while ensuring the security of data and models.

[0034] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0035] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0037] Figure 1 This is a schematic diagram of the structure of a data processing system based on multi-party computing in an embodiment of the present invention;

[0038] Figure 2 A schematic diagram illustrating the implementation principle of a data processing system based on multi-party computing in an embodiment of the present invention;

[0039] Figure 3 Flowchart of a data processing method based on multi-party computing in an embodiment of the present invention;

[0040] Figure 4This is an example diagram of secure multi-party computing in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0042] To address the potential for user data leakage when training user data through models, as is common in existing technologies, and to ensure the security and non-reusability of training sample data, embodiments of the present invention provide a data processing system and method based on multi-party computing. This system addresses data privacy protection through multi-party computing and data reconstruction, preventing user data from being reused after recovery. This system ensures that training sample data can be used on a per-time basis without being deposited, enabling authorized use of user data. This is a data and model use-and-non-deposit solution based on secure multi-party computing.

[0043] Secure Multi-Party Computation (SMC) is a collaborative computing solution that protects privacy among a group of mutually distrusting parties. SMC ensures the independence of inputs and the correctness of computations, while not leaking input values ​​to other participants.

[0044] An embodiment of the present invention provides a data processing system based on multi-party computing. The data and models implemented based on multi-party computing can be used on a per-use basis and cannot be deposited. The core is to utilize the multi-party computing key sharing method to disperse the data calculation process to multiple nodes for execution. As long as the multiple nodes are not completely in collusion, the data cannot be recovered or reused, thereby ensuring the privacy of the data. At the same time, the protection of the model is achieved by setting a real model and a dual model that cannot be used alone in multiple intermediate nodes.

[0045] In this system, the true model and the dual model can be obtained by training on sample data. The model training process is similar to the prediction process based on user data. However, during model training, after obtaining prediction results based on the training data, the objective function value is calculated using the predicted results and the true results. The gradient is then propagated back to the true model and each dual model. The following describes the prediction process based on user data as an example.

[0046] The data processing system based on multi-party computing provided by the embodiment of the present invention has the following structure: Figure 1 As shown, it includes data splitting node 1, intermediate node 2 and merging node 3;

[0047] Data splitting node 1 is used to split the data into two or more data slices, submit one data slice to the intermediate node set up with the first model, and submit the other data slices to the intermediate nodes set up with the second model; wherein the second model and the first model are neural network models with the same model structure but different constant bias terms;

[0048] Intermediate node 2 is used to run the model set on it to process the data shards and obtain shard-processed data;

[0049] The merging node 3 is used to reconstruct and merge the sharded processing data output by the intermediate nodes to obtain the data processing results.

[0050] In the following description, the first model is a true model and the second model is a dual model.

[0051] In the above system, a true model is set in one intermediate node 2, and a dual model is set in other intermediate nodes 2. The true model and the dual model use the same neural network model, wherein the constant bias term of the dual model is set to a value different from the constant bias term of the true model. Preferably, the constant bias term of the dual model is set to zero.

[0052] The selection of the true model and the dual model requires reference to the characteristics of the multi-party computing technology actually used. The specific implementation method may be, but is not limited to, the following combination: For example, when using SPDZ in multi-party computing technology, it can be considered that when SPDZ adds a constant, only one machine needs to add the constant. Therefore, when setting the true model and the dual model in the intermediate node in the above system, the dual model can optionally be obtained by setting the constant bias term of each layer of the true model to zero. The constant bias term (bias) is the intercept of the neuron function in the neural network, usually represented by the constant b. Since the dual model and the dual model use the same neural network model, but only its constant bias term is set to zero, the dual model cannot be used alone and will not produce true prediction results.

[0053] When setting up the model, optionally, the original model function can be approximated by a polynomial expansion to obtain a true model, which is set in an intermediate node; the constant terms in the polynomial approximate expansion result of the original model function are set to zero and / or the constant bias term in at least one non-constant term is set to zero to obtain a dual model, which is set in other intermediate nodes respectively.

[0054] In the above system, even if multiple machines need to deploy models for computation, only one machine needs to deploy the actual model, while the other machines can deploy dual models that meet certain requirements and cannot be used independently. This protects both the model and the data. Optionally, for security reasons, the model owner can maintain the model and deploy it on an intermediate node.

[0055] In this system, the data splitting node selects a corresponding data splitting method based on the data reconstruction method set by the merging node, and the intermediate nodes process the data slices using the corresponding model. When splitting the original data vector, it can be split into multiple slice vectors with the same dimension as the original vector; the sum of the element values ​​at the same position in all the slice vectors is equal to the element value at the corresponding position in the original vector.

[0056] The data processing system provided by the embodiment of the present invention can ensure the dual security of data and models. On the one hand, the present application introduces multiple intermediate nodes, and by splitting the data into data shards, the data processing process is distributed to different intermediate nodes for execution, and the intermediate nodes are handed over to the data holder and the model holder for management respectively, thereby ensuring that the intermediate nodes cannot collude. Therefore, even if the data shard is running on the intermediate node managed by the model holder, due to the lack of the data shard on the intermediate node managed by the data holder, the data cannot be recovered and reused. This makes it impossible for the model holder to use the data shard obtained by itself for reuse and multiple training, nor can it use the obtained data shard to restore the original data, thereby ensuring the privacy of the data; on the other hand, in order to ensure effective data processing, the true model and the dual model set by the intermediate node are neural network models with the same structure, but because the dual model is a model with the constant terms of the true model changed, when the dual model is set on the intermediate node managed by the data holder, the data holder cannot obtain the real data of the model. Therefore, this system realizes the sharing of data and models while ensuring the security of data and models.

[0057] See also Figure 2 The schematic diagram of the data processing system based on multi-party computing is shown. Taking data prediction as an example, when the model owner needs to make a data prediction, it requests data from the data owner. The data owner obtains the data to be provided to the model owner from the data stored on its own server and shards the data into multiple data shards. Figure 2 The shards shown in the figure are divided into multiple data shards such as shard 1 (share1), shard 2 (share2), ..., shard n (sharen), where shard 1 is provided to the real model, shard 1 is input into the real model, and the neural network calculates and outputs the shard processing data 1 (such as Figure 2The result 1), slice 2, ..., slice n, are provided to the corresponding dual model respectively, slice 2, ..., slice n are input into the dual model respectively, and the neural network calculates and outputs the slice processing data 2, ..., slice processing data n (such as Figure 2 Result 2, ..., result n in the shard processing data). Shard processing data 1, shard processing data 2, ..., and shard processing data n are merged to obtain the data processing result (such as the prediction result in the figure). Here, multiple data shards are passed through the neural network separately, and after each obtains the results, the results are merged. Multi-party computing key sharing technology supports linear operation of activation function f. For vectors a and b, f(share1+share2+...+sharen) = f(share1)+f(share2)+...+f(sharen). Therefore, the prediction result of the user data obtained after merging is the same as the prediction result obtained by directly predicting the user data using the true model, but different from the prediction result obtained by directly predicting the user data using the dual model.

[0058] When the data splitting node splits the data, it can split the original vector of the data into multiple slice vectors with the same dimension as the original vector; the sum of the element values ​​at the same position in all the slice vectors is equal to the element value at the corresponding position in the original vector.

[0059] This system can deploy the dual model on multiple, mutually trusted machines. If at least one of the owners of the real and dual models is not colluding, the original data cannot be restored. Furthermore, since each computation requires the cooperation of all machines, the model owner cannot use their own shards for multiple training runs. This system can be used for data transfer learning, a machine learning method that leverages existing knowledge to solve problems in different but related fields.

[0060] Based on the same inventive concept, the embodiment of the present invention also provides a data processing method based on multi-party computing, the process of which is as follows: Figure 3 As shown, the following steps are included:

[0061] Step S101: The data splitting node splits the data into two or more data shards.

[0062] The original vector of the data is split into multiple slice vectors with the same dimension as the original vector; the sum of the element values ​​at the same position in all the slice vectors is equal to the element value at the corresponding position in the original vector.

[0063] Optionally, the data splitting node may select a corresponding data splitting method to split the data according to the data reconstruction method set by the merging node.

[0064] Step S102: Submit one data shard to the intermediate node for setting the real model, and submit the other data shards to the intermediate nodes for setting the dual model.

[0065] Step S103: The intermediate node runs the model configured thereon to process the data shards, obtains shard-processed data, and provides the data to the merging node.

[0066] If the intermediate node is equipped with a real model, the received data slices are processed by the real model. If the intermediate node is equipped with a dual model, the received data slices are processed by the dual model. For example, the polynomial of the real model is ay 3 +by 2 +cy+d, where a, b, c, d are constants and x, y, z are variables, then the polynomial of the dual model can be set to ay 3 +by 2 +cy or ay 3 +by 2 +cy+k, k is a constant different from d, preferably k=0, that is, the polynomial is ay 3 +by 2 +cy.

[0067] The intermediate nodes can also encrypt the data. Each intermediate node receives the data fragments sent by the data splitting node, and uses the set real model or dual model to perform polynomial expansion calculation on the data fragments. Based on the pre-generated random number pairs, the polynomial variable y i Encryption is performed based on the encrypted polynomial variable y i Get the shard prediction data corresponding to the data shard.

[0068] Step S104: The merging node reconstructs and merges the sharded processing data output by each intermediate node to obtain a data processing result.

[0069] When the merge node only reconstructs and merges more than one data provided by the data splitting node, the merge node can merge the output sharded processing data. Simply, it can be directly added. For example: for data x, after data splitting, data shards (x1, x2, ...) are obtained, and after processing, the data shards (x1, x2, ...) are obtained to obtain sharded processing data (X1, X2, ...). X1, X2, ... in the sharded processing data are added to obtain the data processing result, that is, the reconstructed data z of x. At this time, z is equal to x.

[0070] When a merge node needs to provide more than one piece of data to a split node, it can reconstruct and merge the data using addition or multiplication. For example, if the split node provides data x and y, and the merge node reconstructs data z, then if z = x + y, then addition is used; if z = x * y, then multiplication is used.

[0071] When the addition method is used, for data x and data y, after data splitting, data fragments (x1, x2, ...) and data fragments (y1, y2, ...) are obtained, and these data fragments are sent to the intermediate nodes for multi-party computing, where x1 and y1 are sent to the first intermediate node, x2 and y2 are sent to the second intermediate node, ...; x1 and y1 are processed and calculated at the first intermediate node to obtain fragmented processing data z1, for example, the simplest one can be z1=x1+y1, of course, other algorithms can also be used, and x2 and y2 are calculated at the second intermediate node to obtain fragmented processing data z2, for example, z2=x2+y2, ...; z1, z2, ... are reconstructed and merged to obtain the data processing result z=z1+z2+..., at this time z=x+y.

[0072] When the multiplication method is used, for data x and data y, after data splitting, data fragments (x1, x2, ...) and data fragments (y1, y2, ...) are obtained, and these data fragments are sent to the intermediate nodes for multi-party calculation, where x1 and y1 are sent to the first intermediate node, x2 and y2 are sent to the second intermediate node, ...; x1 and y1 are calculated at the first intermediate node to obtain the fragmented processing data z1, and x2 and y2 are calculated at the second intermediate node to obtain the fragmented processing data z2, ..., and a suitable algorithm can be selected to calculate z1 and z2 based on x1, y1, x2, y2; z1, z2, ... are reconstructed and merged to obtain the data processing result z = z1 + z2 + ..., at this time z = x*y.

[0073] Here are some examples:

[0074] In the above method, when performing data processing calculations, if the data z received by the data receiving end is the sum of the data x and y sent by the sending end, that is, z = x + y, an addition method can be used, for example:

[0075] For data x = a, the data transmitter acts as a data splitting node and splits data a into data slices a0 and a1, where a0 can be a random number and a1 is the difference between a and this random number. For data y = b, the data splitting node splits data b into data slices b0 and b1, where b0 can be a random number and b1 is the difference between b and this random number. Data slices a0 and b0 are sent to an intermediate node, which processes the slices and sends the processed data c0 = a0 + b0 to a merging node. Data slices a1 and b1 are sent to another intermediate node, which then processes the slices and sends the processed data c1 = a1 + b1 to the receiving end. The receiving end, acting as a merging node, calculates data c0 + c1 based on c0 and c1. Since c0 + c1 = a0 + b0 + a1 + b1 = a + b, the final data z = x + y = a + b can be obtained without exposing data a and b.

[0076] When calculating the predicted data, if the data z received by the data receiving end is the product of the data x and y sent by the sending end, that is, z = x*y, a multiplication method can be used, for example:

[0077] For data x = a, the data sending end, acting as a data splitting node, splits data a into data fragments a0 and a1, where a0 can be a random number and a1 is the difference between a and the random number. For data y = b, the data splitting node splits data b into data fragments b0 and b1, where b0 can be a random number and b1 is the difference between b and the random number. Data fragments a0 and b0 are sent to the first intermediate node, and data fragments a1 and b1 are sent to the second intermediate node.

[0078] The semi-trusted server generates random numbers u0, u1, v0, and v1, and (u0+u1)(v0+v1)=(z0+z1), and provides u0, v0, and z0 to the first intermediate node. u1, v1, and z1 are provided to the second intermediate nodes.

[0079] The first intermediate node exchanges a0-u0 and b0-v0 to the second intermediate node, and the second intermediate node exchanges a1-u1 and b1-v1 to the first intermediate node. The first intermediate node and the second intermediate node each calculate e and f, where e=au, f=bv, u=u0+u1, v=v0+v1.

[0080] The first and second intermediate nodes calculate their respective slice prediction data according to their respective calculation rules. For example, the calculation rule set by the first intermediate node is such that c0 + c1 = a*b. Based on e and f, the slice processed data c0 = -e*f + a0*f + e*b0 + z0. The second intermediate node also uses the set calculation rule based on e and f to calculate the slice processed data c1 = a1*f + e*b1 + z1.

[0081] The processed data c0 and c1 are sent to the receiving end. The receiving end acts as a merging node for data merging and reconstruction, resulting in c0 + c1 = -e*f + a0*f + e*b0 + z0 + a1*f + e*b1 + z1 = -e*f + a*f + e*b + u*v = u*f + a*bu*b + u*v = u*b – u*v + a*bu*b + u*v = a*b. Therefore, without exposing data a and b, the final data z can be obtained as z = x*y = a*b.

[0082] Regarding the data processing method based on multi-party computing in the above embodiment, the specific process of executing the operation in each step has been described in detail in the relevant part of the system embodiment and will not be elaborated here.

[0083] The following describes the implementation process of data prediction using a specific example of predicting the likelihood of a mortgage loan based on a user's assets, income, and health during a bank's lending process.

[0084] Suppose, for example, that a bank is issuing loans and that a government agency (data holder) has reliable records of a user's assets, income, and health. These records are classified into three levels: low, medium, and high, with corresponding weights of 1, 2, and 3. The user applies for a loan based on this information but does not want the bank to obtain their raw data from the government. The bank (model holder) also does not want its credit assessment model exposed to government agencies. Therefore, the aforementioned multi-party computation-based data processing can be used to ensure that the user data recorded by the government and the assessment model held by the bank are usable and non-depositable.

[0085] Assume that the original evaluation model trained by the bank is the most basic single-layer logistic regression (LR) model (the principle of multi-layer neural network is similar), f(x) is as follows:

[0086] f(x)=sigmoid(wx+b)

[0087] Here, x is the user data vector, w is the weight vector, and b is the model's constant bias term. Both the weight vector and the constant bias term can be set arbitrarily, based on experience, or according to specific rules. For example, in the bank loan assessment model described above, w = (3, 2, 1) and b = -12 can be set.

[0088] When the evaluation model outputs an evaluation result (prediction result) f(x) ≥ 0.5, the loan is considered to be granted; otherwise, the loan is rejected. For two users, one of whom has high assets, income, and health, with corresponding weights of 3, and the other has low assets, income, and health, with corresponding weights of 1, the user data vectors are u = (3, 3, 3) and v = (1, 1, 1). If we ignore data and model privacy issues and directly input the data of both users into the real model for prediction, we will calculate the evaluation results f(u) = 0.9975 ≥ 0.5 and f(v) = 0.0025 < 0.5, indicating that user u can lend, but user v cannot.

[0089] When using the aforementioned secure multi-party computation method for prediction, the dual model f′(x) can be generated based on the true model f(x) by setting its constant bias term to 0. Since SPDZ only supports linear operations, the activation function (sigmoid) needs to be expanded approximately as a polynomial. The true model f(x) and the dual model f′(x) after the approximate expansion are as follows:

[0090]

[0091]

[0092] make but

[0093] f(x)≈0.5+1.20096*y+0.81562*y 3

[0094] Here, y is a polynomial variable, which can be called a local shard in the model.

[0095] The true model and the dual model are respectively set on multiple intermediate nodes. Taking three intermediate nodes as an example, the true model f(x) is deployed on the first intermediate node, which is owned by the model holder. A dual model f′(x) is deployed on each of the other two intermediate nodes. These two intermediate nodes are owned by a third party trusted by both parties. Taking the data of user u as an example, the user data u=(3,3,3) is divided into three data shards and submitted to the models of the three intermediate nodes respectively. The data and model can be computed using a non-precipitable secure multi-party computation process such as Figure 4As shown, the three data fragments split from the user data u=(3, 3, 3) are u1=(50, -16, 8), u2=(7, 4, 3), u3=(-54, 15, -8), and u=u1+u2+u3.

[0096] The polynomial variables of the three data slices are calculated as follows:

[0097]

[0098] See also Figure 4 As shown, according to the local shards y1, y2, y3, each time a multiplication operation is performed between shard data, such as for y 2 or y 3 The multiplication triple method can be used.

[0099] S201: Use a neutral node (which can be a server and will not collude with the nodes in secure multi-party computing) to generate random number pairs (a1, b1, c1), (a2, b2, c2), and (a3, b3, c3), and send them to three intermediate nodes respectively. The number pairs must meet the following conditions: (a1+a2+a3)*b1+b2+b3)=(c1+c2+c3).

[0100] Assign values ​​to the vector elements in the random number pairs as follows:

[0101] a1=1, a2=2, a3=0; b1=2, b2=9, b3=-4; c1=10, c2=9, c3=2

[0102] S202: The three intermediate nodes use the obtained random number pairs to calculate the polynomial variables y of the two multipliers. i and y i (Because this is a power operation, both are y i , exactly the same) are encrypted to obtain:

[0103] g i1 =y i -a i , h i1 =y i -b i

[0104] Each intermediate node uses the polynomial variable y i The random number pair is used to calculate the encrypted data of the two multipliers. Then the three nodes encrypt the first multiplier and obtain g respectively. 11 =13.25, g 21 =2,g 31=-17.5, the three nodes encrypt the second multiplier and obtain h 11 =12.25,h 21 =-5,h 31 =-13.5.

[0105] S203: Reconstruct the first multiplier g1 based on the first multiplier encrypted by the three nodes, and reconstruct the second multiplier h1 based on the second multiplier encrypted by the three nodes. Reconstruct g1 and h1 and send them to all nodes. At this time, the original y is not exposed.

[0106] g1=g 11 +g 21 +g 31 =-2.25, h1 = h 11 +h 21 +h 31 =-6.25

[0107] S204: The three nodes are respectively based on the formula Find p i , so that p = y 2 .

[0108] The p values ​​obtained for the three nodes are: p1=13.3125, p2=-23.75, and p3=11.

[0109] S205: Repeat steps S202-S204, and reuse the random number pair to find q=p*y=y 3

[0110] g i2 =p i -a i , h i2 =y i -b i

[0111] g i2 =p i -a i , h i2 =y i -b i

[0112] g 12 =12.3125, g 22 =-25.75, g 32 =11,h 12 =12.25,h 22 =-5,h 32 =-13.5

[0113] g2=g 12 +g 22+g 32 =-2.4375, h2 = h 12 +h 22 +h 32 =-6.25

[0114] q1=3.953125, q2=-20.359375, q3=16.828125

[0115] See also Figure 4 As shown, the three nodes calculate their own q1, q2, and q3 respectively.

[0116] S206: After the three nodes calculate their respective q1, q2, and q3, they output the prediction results f(u1), f′(u2), and f′(u3) of each shard through the true model or the dual model.

[0117] f(u1)=0.5+1.20096*14.25-0.81562*3.953125=14.3894321875

[0118] f′(u2)=1.20096*4-0.81562*(-20.359375)=21.9093534375

[0119] f′(u3)=1.20096*(-17.5)-0.81562*16.828125=-34.2421553125

[0120] S206: The data receiver (bank) merges the prediction results f(u1), f′(u2) and f′(u3) of each fragment to obtain the prediction result f(u) of the user data.

[0121] f(u1)+f′(u2)+f′(u3)=1.0566303125≈0.9975=f(u)

[0122] Then, we can judge whether to lend money to the user based on f(u). From the calculation results, we can see that the prediction results after shard aggregation are basically the same as the prediction results after directly inputting the user data vector u into the real model.

[0123] The above-described method and system in the embodiments of the present invention utilize multi-party computing technology and the concept of creating a dual model to achieve per-use data without sedimentation and per-use model without sedimentation. This solution simultaneously protects the privacy and security of both data and models, creating opportunities for collaboration between organizations that encourage data and model security. It also supports per-use data, reversing the current situation during model training where, once data is passed to the modeling party, its repeated use cannot be prevented, thus opening the door to a business model for per-use authorized data use.

[0124] Unless otherwise specifically stated, terms such as process, calculate, compute, determine, display, and the like may refer to the actions and / or processes of one or more processing or computing systems, or similar devices, that manipulate and convert data represented as physical (e.g., electronic) quantities within registers or memories of a processing system into other data similarly represented as physical quantities within the memories, registers, or other such information storage, transmission, or display devices of the processing system. Information and signals may be represented using any of a variety of different techniques and methods. For example, data, instructions, commands, information, signals, bits, symbols, and chips referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, light fields or particles, or any combination thereof.

[0125] It should be understood that the specific order or hierarchy of steps in the disclosed processes is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0126] In the foregoing detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that embodiments of the claimed subject matter require more features than are expressly recited in each claim. On the contrary, as reflected in the appended claims, the invention comprises less than all the features of any individual disclosed embodiment. The appended claims are therefore hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0127] Those skilled in the art will also appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments herein may be implemented as electronic hardware, computer software, or a combination thereof. In order to clearly illustrate the interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps described above are generally described around their functions. Whether such functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. A skilled person may implement the described functions in an adaptable manner for each specific application, but such implementation decisions should not be interpreted as departing from the scope of protection of this disclosure.

[0128] The steps of the methods or algorithms described in conjunction with the embodiments herein may be directly embodied as hardware, software modules executed by a processor, or a combination thereof. The software module may be located in a RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, register, hard disk, removable disk, CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and storage medium may be located in an ASIC. The ASIC may be located in a user terminal. Of course, the processor and storage medium may also be present in a user terminal as discrete components.

[0129] For software implementation, the techniques described in this application can be implemented using modules (e.g., procedures, functions, etc.) that perform the functions described in this application. These software codes can be stored in a memory unit and executed by a processor. The memory unit can be implemented within the processor or external to the processor. In the latter case, it is communicatively coupled to the processor via various means, which are well known in the art.

[0130] The foregoing description includes examples of one or more embodiments. Of course, it is not possible to describe all possible combinations of components or methods for the purposes of describing the above embodiments, but one of ordinary skill in the art will recognize that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to encompass all such changes, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent the term "comprising" is used in the specification or claims, the term is intended to be encompassed in a manner similar to the term "including," as explained in terms of "including," used as a transitional word in the claims. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or."

Claims

1. A data processing system, characterized in that: include: Data splitting nodes, intermediate nodes and merging nodes; A data splitting node is used to split the data into two or more data slices, submit one data slice to an intermediate node set with a first model, and submit the other data slices to an intermediate node set with a second model, wherein the first model and the second model are neural network models with the same model structure but different constant bias terms; The intermediate node is used to run the model set on it to process the data shards and obtain the shard processing data; The merge node is used to reconstruct and merge the sharded processing data output by the intermediate nodes to obtain the data processing results.

2. The system according to claim 1, wherein The data splitting node is used to split the data into two or more data shards, including: The original vector of the data is split into multiple slice vectors with the same dimension as the original vector; the sum of the element values ​​at the same position in all the slice vectors is equal to the element value at the corresponding position in the original vector.

3. The system according to claim 1, wherein The intermediate node is used to run the model set on it to process the data shards and obtain the shard processing data, including: Each intermediate node receives the data fragments sent by the data splitting node, uses the set first model or second model to perform polynomial expansion calculation on the data fragments, and calculates the polynomial variable y based on the pre-generated random number pair. i Encryption is performed based on the encrypted polynomial variable y i Get the shard processing data corresponding to the data shard.

4. The system according to claim 1, wherein: The data splitting node selects a corresponding data splitting method to split the data according to the data reconstruction method set by the merging node, and the intermediate node uses the corresponding model to process the data fragments.

5. The system according to any one of claims 1 to 4, characterized in that: The first model is set in one intermediate node, and the second model that cannot be used alone is set in other intermediate nodes, including: Perform polynomial approximate expansion on the original model function to obtain a first model, which is set in an intermediate node; The constant terms in the polynomial approximate expansion result of the original model function are set to zero and / or the constant bias term in at least one non-constant term is set to zero to obtain a second model, which is respectively set in other intermediate nodes.

6. A data processing method based on multi-party computing, characterized in that: include: The data splitting node splits the data into two or more data shards, submits one data shard to the intermediate node configured with the first model, and submits the other data shards to the intermediate nodes configured with the second model. The second model uses the same neural network model as the first model but with different constant bias terms. The intermediate node runs the model configured thereon to process the data shards, obtains shard-processed data, and provides it to the merging node. The merging node reconstructs and merges the sharded processing data output by each intermediate node to obtain the data processing result.

7. The method according to claim 6, wherein The data splitting node splits the data into two or more data shards, including: The original vector of the data is split into multiple slice vectors with the same dimension as the original vector; the sum of the element values ​​at the same position in all the slice vectors is equal to the element value at the corresponding position in the original vector.

8. The method according to claim 6, wherein The intermediate node runs the model configured on it to process the data shards and obtains the shard-processed data, including: Each intermediate node receives the data fragments sent by the data splitting node, uses the set first model or second model to perform polynomial expansion calculation on the data fragments, and calculates the polynomial variable y based on the pre-generated random number pair. i Encryption is performed based on the encrypted polynomial variable y i Get the shard processing data corresponding to the data shard.

9. The method according to claim 6, wherein The data splitting node selects a corresponding data splitting method to split the data according to the data reconstruction method set by the merging node, and the intermediate node uses the corresponding model to process the data fragments.

10. The method according to any one of claims 6 to 9, characterized in that: Setting a first model in an intermediate node and setting a second model that cannot be used alone in other intermediate nodes include: Perform polynomial approximate expansion on the original model function to obtain a first model, which is set in an intermediate node; The constant terms in the polynomial approximate expansion result of the original model function are set to zero and / or the constant bias term in at least one non-constant term is set to zero to obtain a second model, which is respectively set in other intermediate nodes.

Images