Longitudinal federated logistic regression method, device and system based on multi-party matrix multiplication

Abstract

The invention discloses a longitudinal federated logistic regression method, device and system based on multi-party matrix multiplication. The method comprises the following steps that each participant initializes a first model weight fragment of the participant and a second model weight fragment of an adjacent previous participant; each participant locally calculates a forward activation value fragment according to the first model weight fragment and a data set owned by the participant, and each participant cooperatively calculates a secret fragment of a forward activation value cross term by using a new matrix multiplication algorithm based on an erasable mask according to the second model weight fragment and the data set owned by the participant; each participant locally calculates a model gradient, the active party and the passive party cooperatively calculate a model gradient cross term and obtain a secret fragment of the model gradient cross term, and each participant locally updates a model weight fragment according to the secret fragment of the model gradient cross term. According to the method, data privacy can be effectively guaranteed for classification tasks or regression tasks, and meanwhile efficient model training and reasoning are achieved.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and more specifically, to a vertical federated logistic regression method, apparatus, and system based on multi-party matrix multiplication. Background Art

[0002] With the continuous attention to data privacy, how to build machine learning (ML) models on different data sources while protecting privacy has become increasingly popular. Vertical federated learning (VFL) considers the situation where different parties jointly train an ML model on partitioned features, which is suitable for many real-world cross-enterprise collaborations. Logistic regression (LR) is a classic algorithm in machine learning. Due to the simplicity of LR and its effectiveness in many binary classification tasks, the privacy-preserving model training or inference of logistic regression based on vertically distributed datasets has attracted wide attention.

[0003] Existing vertical logistic regression privacy protection solutions can be divided into two categories according to the processing method of private features. The first category uses secure multi-party computation (MPC) technologies, such as homomorphic encryption (HE) and secret sharing (SS), to achieve complete privacy guarantees. Usually, they use data outsourcing technologies widely used in database services to outsource the dataset to non-colluding servers for ML training or inference. To maintain privacy, all feature values are converted into HE or SS variables during outsourcing, so that the server cannot know the original values. However, this method is not suitable for many real-world datasets. For high-dimensional and sparse datasets, the outsourced features become completely dense because the non-zero feature indices are also private. This prevents us from performing sparse computations, resulting in performance risks when the sparsity is high. The second category follows the split learning paradigm, which does not outsource the original dataset and thus can handle sparse features well. Usually, each party maintains an underlying model in plaintext, which uses its own private features to extract forward activation values. The activation values of each party will be exchanged and input into the top-level model for prediction. However, the values generated in the underlying model exist in plaintext, which may lead to data leakage. This kind of data leakage is prohibited and even illegal in real-world applications.

[0004] In 2022, Fu, Fangcheng et al. proposed a secure and practical privacy-preserving vertical logistic regression scheme BlindFL based on HE and SS (the result "BlindFL: Vertical Federated Machine Learning without Peeking into Your Data." was published in the SIGMOD 2022 conference). BlindFL keeps the private datasets within each party without outsourcing and unifies the functions through an important component called "federated source layer". By doing so, BlindFL can support sparse input features while achieving security guarantees. As Figure 1 shown, the scheme implemented by BlindFL can be decomposed into two parts: forward propagation and backward propagation. During forward propagation, the federated source layer, as the basic building block, combines the features of passive party A and active party B (with the true label values), and its output is denoted as Z. Assume the datasets held by A and B are X A 、X B , that is, Z = X A W A +X B W B . The non-federated sub-module in party B, namely the top model, acts as a classifier / predictor. During backward propagation, B calculates the loss function based on the final predicted value and the true label, and then propagates it backward along the top model to obtain the partial derivative of the loss function with respect to Z Furthermore, the model gradient can be calculated through the chain rule Finally, the model weights of the source layer are updated using the model gradient.

[0005] Specifically, to prevent the leakage of the datasets held by A and B respectively and the label values of B, BlindFL prohibits both parties from having their own or the other party's underlying models, namely W A and W B . During forward propagation, passive party A is prohibited from revealing the forward propagation value Z. During backward propagation, A and B cannot reveal the model gradients of the other party, and A also cannot reveal its own model gradient. To achieve the above restrictions, BlindFL uses SS to secretly share W A and W B to A and B. A and B only hold the shards of W A and W B . At the same time, to calculate the forward propagation value Z, A and B need to homomorphically encrypt one of the shards and send it to the other party, and perform homomorphic scalar multiplication on the obtained ciphertext shard and the plaintext shard they hold, and convert the result into a secret sharing form. The above process requires multiple rounds of communication between A and B, and finally B can only obtain the forward propagation value Z. The backward propagation process is similar. After multiple rounds of communication between A and B, they can obtain the values for updating WA and W B Partitioned model gradients However, the BlindFL scheme only involves two parties and does not discuss the case of multiple parties (more than two parties). In addition, the scheme requires too many communication rounds. Especially when extended to multiple parties, pairwise communication is needed, and the time overhead caused by communication is huge.

[0006] In addition, the patent document with the publication number CN119323058A provides a privacy-preserving method and device for multi-party vertical logistic regression based on secret sharing, which uses the traditional Beaver triple method to solve the problems of multiplication of numbers and multi-party matrix multiplication. However, the transmission of Beaver triples is relatively troublesome, and it requires a trusted third party or methods such as oblivious transfer with a relatively high communication cost. Therefore, this method still has the technical problem of relatively high communication cost. Summary of the Invention

[0007] Aiming at the technical problem of relatively high communication cost existing in the existing privacy protection methods, the present invention provides a vertical federated logistic regression method, device and system based on multi-party matrix multiplication, which realizes vertical federated logistic regression based on a new multi-party matrix multiplication under a general privacy computing architecture. Compared with the multiplication of numbers and multi-party matrix multiplication realized by the traditional Beaver triple method, this method realizes the multiplication of numbers or matrices through one-round communication, greatly reducing the communication volume and improving the computing efficiency.

[0008] To achieve the above object, the first aspect of the present invention provides a vertical federated logistic regression method based on multi-party matrix multiplication, including:

[0009] Each party initializes the first model weight partition of its own model and the second model weight partition of the model of the adjacent previous party. The model weight partition includes the first model weight partition and the second model weight partition, and the sum of the first model weight partition and the second model weight partition is the model weight;

[0010] Each participating party calculates the shards of the forward activation value locally according to the first model weight shards and the dataset it owns. Each participating party collaboratively calculates the secret shards of the cross-term of the forward activation value using the new matrix multiplication algorithm based on the erasable mask according to the second model weight shards and the dataset it owns. Then, each participating party obtains the output value according to the dataset it owns, the model weight shards, and the secret shards of the cross-term of the forward activation value. The passive party among them sends the obtained output value to the active party, and the active party obtains the forward propagation value according to the output value of the passive party and its own output value. Among them, each participating party collaboratively calculates the cross-term of the forward activation value using the new matrix multiplication algorithm based on the erasable mask according to the second model weight shards and the dataset it owns, including: each participating party generates random numbers respectively, adds random masks to the dataset it owns based on the random numbers, the assistant and the participating parties jointly calculate the secret shards of the cross-term of the forward activation value based on matrix multiplication, and the sum of all shards of the forward activation value and the secret shards of the cross-term of the forward activation value constitutes the forward activation value;

[0011] Each participating party calculates the model gradient locally. The active party and the passive party collaboratively calculate the cross-term of the model gradient and obtain the secret shards of the cross-term of the model gradient. Each participating party locally updates the model weight shards according to the secret shards of the cross-term of the model gradient. Among them, the cross-term of the model gradient is a part of the model gradient that requires the collaborative calculation of the active party and the passive party.

[0012] In one implementation, each participating party initializes the first model weight shards of its own model and the second model weight shards of the model of the adjacent previous participating party, including:

[0013] Participating party P i Initializes U i and V i-1 , i = {1, 2, 3}, U i is the first model weight shard of participating party P i 's model, and V i-1 is the second model weight shard of the model of the adjacent previous participating party P i of participating party P i-1 . Specifically, assume that the adjacent previous participating party of participating party P1 is P3;

[0014] The model weight of each participating party is calculated by W = U + V, where U is the first model weight shard, V is the second model weight shard, and W is the model weight.

[0015] In one implementation, each participating party calculates the shards of the forward activation value locally according to the first model weight shards and the dataset it owns, including:

[0016] Participating party P i According to the obtained first model weight shard U iWith the owned dataset X i Calculate the local pre-computed forward activation value shard X i U i 。

[0017] In one implementation, each participant generates a random number, adds a random mask to the owned dataset based on the random number, and the helper and participants jointly calculate the secret shard of the forward activation value cross-term based on matrix multiplication, including:

[0018] Participants P1 and P2 use the same random number seed and random number generator to generate random number matrices A and B. The dimension of A is the same as that of X1, and the dimension of B is the same as that of V1;

[0019] Participant P1 calculates X1 - A and sends X1 - A to the helper P0. Participant P2 calculates V1 - B and sends V1 - B to the helper P0;

[0020] The helper P0 calculates (X1 - A)(V1 - B) = X1V1 - X1B - AV1 + AB and generates a random number vector R. The dimension of R is the same as that of X1V1;

[0021] The helper P0 calculates (X1 - A)(V1 - B) - R and sends (X1 - A)(V1 - B) - R to participant P1 and sends R to participant P2;

[0022] Participant P1 calculates [X1·V1] o =(X1 - A)(V1 - B) - R + X1B, and participant P2 calculates [X1·V1] o 、 are the secret shards of the forward activation value cross-term calculated for participants P1 and P2,

[0023]

[0024] In one implementation, each participant obtains the output value based on the owned dataset, the model weight shard, and the secret shard of the forward activation value cross-term. The passive party sends the obtained output value to the active party, and the active party obtains the forward propagation value based on the output value of the passive party and its own output value, including:

[0025] Each participant P i respectively calculates and records the calculation result as Z i ;

[0026] Parties P1 and P2 respectively send Z1 and Z2 to P3. P3 locally calculates Z = Z1 + Z2 + Z3 = X1W1 + X2W2 + X3W3. Using Z as the input of the top-level model can calculate the final prediction value of the model. Among them, Z is the forward propagation value, the top-level model is unique to P3, and does not require collaborative calculation by each party. P3 is the active party among the parties.

[0027] In one implementation, the active party and the passive party collaboratively calculate the model gradient cross-term and obtain the secret shards of the model gradient cross-term, including:

[0028] The passive party P1 and the active party P3 collaboratively calculate the model gradient cross-term using the new matrix multiplication algorithm based on the eliminable mask, and respectively obtain the corresponding secret shards of the model gradient cross-term and where represents the partial derivative of the loss function with respect to Z. Since is unique to P3 and Z is the forward propagation value;

[0029] The passive party P2 and the active party P3 collaboratively calculate the model gradient cross-term using the new matrix multiplication algorithm based on the eliminable mask, and respectively obtain the corresponding secret shards of the model gradient cross-term and

[0030] Based on the same inventive concept, the second aspect of the present invention provides a vertical federated logistic regression device based on multi-party matrix multiplication, which is applied to a privacy computing architecture including three or more parties. The parties include an active party and passive parties. Each party respectively has its own dataset. The assistant is responsible for assisting the secure multiplication calculation of each party during the training process. The device includes:

[0031] An initialization module. Each party initializes the first model weight shard of its own model and the second model weight shard of the adjacent previous party's model. The model weight shard includes the first model weight shard and the second model weight shard, and the sum of the first model weight shard and the second model weight shard is the model weight;

[0032] The forward propagation module is used for each participating party to calculate the shards of forward activation values locally according to the first model weight shards and the dataset it owns. Each participating party collaboratively calculates the secret shards of the cross terms of forward activation values using the new matrix multiplication algorithm based on an eliminable mask according to the second model weight shards and the dataset it owns. Then, each participating party obtains the output value according to the dataset it owns, the model weight shards, and the secret shards of the cross terms of forward activation values. Among them, the passive party sends the obtained output value to the active party, and the active party obtains the forward propagation value according to the output value of the passive party and its own output value. Among them, each participating party collaboratively calculates the secret shards of the cross terms of forward activation values using the new matrix multiplication algorithm based on an eliminable mask according to the second model weight shards and the dataset it owns, including: each participating party generates a random number respectively, adds a random mask to the dataset it owns based on the random number, and the helper and the participating parties jointly calculate the secret shards of the cross terms of forward activation values based on matrix multiplication. The sum of all shards of forward activation values and the secret shards of the cross terms of forward activation values constitutes the forward activation value;

[0033] The backward propagation module is used for each participating party to calculate the model gradient locally, the active party and the passive party collaboratively calculate the cross terms of the model gradient, and obtain the secret shards of the cross terms of the model gradient. Each participating party locally updates the model weight shards according to the secret shards of the cross terms of the model gradient. Among them, the cross terms of the model gradient are a part of the model gradient that requires the collaborative calculation of the active party and the passive party.

[0034] Based on the same inventive concept, the third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the longitudinal federated logistic regression method based on multi-party matrix multiplication described in the first aspect.

[0035] Based on the same inventive concept, the fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the longitudinal federated logistic regression method based on multi-party matrix multiplication described in the first aspect.

[0036] Based on the same inventive concept, the fifth aspect of the present invention provides a privacy computing system, including the longitudinal federated logistic regression device based on multi-party matrix multiplication described in the second aspect and a horizontal federated computing device. Among them, the horizontal federated computing device is used to calculate a check value by weighting and averaging the model parameters with random numbers to assist in detecting malicious tampering of the model parameters.

[0037] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0038] The present invention provides a vertical federated logistic regression method based on multi-party matrix multiplication. Each participating party has its corresponding dataset and does not need to outsource the original data, so it is applicable to high-dimensional and sparse datasets. At the same time, each participating party obtains the shards of the model weights based on the secret sharing technology, so that no party can master any underlying model. Therefore, it can resist model completion attacks or label information leakage attacks. In the forward propagation stage, each participating party collaboratively calculates the secret shards of the forward activation value cross terms according to the second model weight shards and the dataset it owns using a new matrix multiplication algorithm based on erasable masks, that is, a new online multiplication under the general privacy computing architecture is used to implement the multiplication of numbers or matrices through one round of communication. Compared with the traditional method using Beaver triples, only one round of communication is required, which greatly reduces the communication volume and improves the computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0040] Figure 1 It is a flowchart of the BlindFL vertical logistic regression privacy protection scheme;

[0041] Figure 2 The overall flowchart of the vertical federated logistic regression method based on multi-party matrix multiplication in the embodiments of the present invention;

[0042] Figure 3 It is a schematic diagram of the general privacy computing architecture;

[0043] Figure 4 It is a structural diagram of the vertical federated logistic regression device based on multi-party matrix multiplication in the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0045] Embodiment 1

[0046] This embodiment discloses a vertical federated logistic regression method based on multi-party matrix multiplication, which is applied to a privacy computing architecture including three or more participating parties. The participating parties include an active party and passive parties, and each participating party has its own data set. The facilitator is responsible for assisting in the secure multiplication calculation among all parties during the training process. Please refer to Figure 2 , and the method includes:

[0047] S1: Each participating party initializes the first model weight shard of its own model and the second model weight shard of the previous adjacent participating party's model. The model weight shard includes the first model weight shard and the second model weight shard, and the sum of the first model weight shard and the second model weight shard is the model weight;

[0048] S2: Each participating party locally calculates the forward activation value shard according to the first model weight shard and its own data set. Each participating party collaboratively calculates the secret shard of the forward activation value cross-term using a new matrix multiplication algorithm based on eliminable masks according to the second model weight shard and its own data set. Then each participating party obtains the output value according to its own data set, model weight shard, and the secret shard of the forward activation value cross-term. Among them, the passive party sends the obtained output value to the active party, and the active party obtains the forward propagation value according to the output value of the passive party and its own output value. Among them, each participating party collaboratively calculates the forward activation value cross-term using a new matrix multiplication algorithm based on eliminable masks according to the second model weight shard and its own data set, including: each participating party generates random numbers respectively, adds random masks to the data set it owns based on the random numbers, and the facilitator and the participating parties jointly calculate the secret shard of the forward activation value cross-term based on matrix multiplication. The sum of all forward activation value shards and the secret shard of the forward activation value cross-term constitutes the forward activation value;

[0049] S3: Each participating party locally calculates the model gradient. The active party and the passive parties collaboratively calculate the model gradient cross-term and obtain the secret shard of the model gradient cross-term. Each participating party locally updates the model weight shard according to the secret shard of the model gradient cross-term. Among them, the model gradient cross-term is a part of the model gradient that requires collaborative calculation by the active party and the passive parties.

[0050] As Figure 3 shown, it is an existing general privacy computing architecture, which includes an auxiliary server P0 and several participating party servers P i , where i = 1, 2,..n. Each participating party server is equipped with the same random number generator and the same initial seed.

[0051] For the classic algorithm of logistic regression, inspired by the BlindFL paper, the present invention proposes a privacy-preserving algorithm for multi-party vertical logistic regression based on secret sharing under a general privacy computing framework, where secure matrix multiplication is a basic component of the algorithm. The present invention proposes a new multi-party matrix multiplication algorithm. Assuming that all participating parties are honest but curious, the method of the present invention can be easily extended to three or more parties, achieving a lower communication round while ensuring security. The vertical federated logistic regression algorithm based on the new matrix multiplication is given below.

[0052] First, the algorithm process of the new matrix multiplication is given, and then, taking three participating parties as an example, a principled description of the privacy-preserving algorithm for multi-party vertical logistic regression based on secret sharing is given.

[0053] Use [*] to represent the state after the secret is shared. For example, [a] means that the secret a has been shared with the participants through the secret sharing function. For two-party secret sharing, denote the two shard values [a] o and That is

[0054] The new multiplication algorithm based on the eliminable mask is described as follows:

[0055] Suppose there are two participating parties, P1 and P2. The participating party P1 holds the secret value x, and the participating party P2 holds the secret value y. Now it is necessary for the two participating parties to calculate [x·y] without exposing their respective secret values, that is, the secret shard of the product of x and y.

[0056] (1) The participating party P1 generates random numbers a and b, calculates x - a, and sends x - a to the helper P0.

[0057] (2) The participating party P2 generates random numbers a and b, calculates y - b, and sends y - b to the helper P0. Since each participating party adds a random mask before sending the secret value, no secret leakage will occur.

[0058] (3) The helper P0 calculates (x - a)(y - b) = xy - xb - ay + ab, and generates a random number r. The helper P0 calculates (x - a)(y - b) - r, and sends (x - a)(y - b) - r to the participating party P1 and sends r to the participating party P2.

[0059] (4) The participating party P1 calculates [x·y] o = (x - a)(y - b) - r + xb.

[0060] (5) The participating party P2 calculates

[0061] Proof of correctness:

[0062]

[0063] The above method can be easily extended to matrix operations. By applying the random number generation and calculation process to matrix elements, secure multiplication calculation at the matrix level can be achieved.

[0064] Specifically, S1 is the initialization step. Each participant initializes the first model weight shard of its own model and the second model weight shard of the adjacent previous participant's model, so as to ensure that no participant can master any underlying model, that is, each party does not have the plaintext model weights W1, W2, and W3.

[0065] S2 is the forward propagation step. The goal is for the active party P3 to master the value of Z, where Z is the forward activation value of the underlying model. Among them, the forward activation value shards X1U1, X2U2, and X3U3 are locally calculated by P1, P2, and P3. X1V1, X2V2, and X3V3 belong to the cross terms and require the parties who master X i and V i to use a new matrix multiplication algorithm based on erasable masks to calculate collaboratively. The final result is that both parties can obtain the secret shards of the cross terms.

[0066] S3 is the backpropagation step. The goals are: P1 updates U1, V3; P2 updates U2, V1; P3 updates U3, V2. For participant P i , the model gradient in plaintext form can be calculated through . Because P3 obtained the value of Z during forward propagation, P3 can locally calculate where represents the partial derivative of the loss function with respect to Z. Since is unique to P3, similar to the method of calculating cross terms during forward propagation, P1 and P3 need to calculate the value of the cross term collaboratively and obtain its secret shard, and P2 and P3 need to calculate the value of the cross term collaboratively and obtain its secret shard. In addition, since P2 is the party that masters the model weight shard V1 of P1, P3 needs to send to P2.

[0067] In one implementation, S1 includes:

[0068] Participant P i initializes U i and V i-1 , i = {1, 2, 3}, U i is the first model weight shard of participant P i 's model, and V i-1 is the second model weight shard of the adjacent previous participant P i of participant P i-1The second model weight shard of the model. Specifically, assume that the previous participant adjacent to participant P1 is P3.

[0069] The model weights of each participant are calculated by W = U + V, where U is the first model weight shard, V is the second model weight shard, and W is the model weight.

[0070] Specifically, the first model weight shard U and the second model weight shard V are held by different participants, so neither of them knows what W is.

[0071] After initialization, the data mastered by each participant is as follows:

[0072] Participant P1: U1, V3, X1;

[0073] Participant P2: U2, V1, X2;

[0074] Participant P3: U3, V2, X3.

[0075] In one implementation, each participant calculates the forward activation value shard locally according to the first model weight shard and the dataset it owns, including:

[0076] Participant P i According to the obtained first model weight shard U i And the owned dataset X i Calculate the locally calculated forward activation value shard X i U i .

[0077] Goal: The active party P3 masters the value of Z. The value of Z is the forward activation value of the underlying model.

[0078] Z = X1W1 + X2W2 + X3W3 = X1(U1 + V1) + X2(U2 + V2) + X3(U3 + V3)

[0079] = X1U1 + X1V1 + X2U2 + X2V2 + X3U3 + X3V3

[0080] Among them, X1U1, X2U2, X3U3 can be calculated locally by P1, P2, P3. X1V1, X2V2, X3V3 are cross terms and require the two parties who master X i And V i To use the new matrix multiplication algorithm based on the erasable mask to calculate collaboratively, and the final result is that the two parties can obtain the secret shards of the cross terms.

[0081] Taking the cross term X1V1 as an example, X1 is owned by P1 and V1 is owned by P2, introduce the new matrix multiplication.

[0082] Each party generates a random number respectively, adds a random mask to the dataset it owns based on the random number, and the helper and the parties jointly calculate the secret shards of the cross - term of the forward activation value based on matrix multiplication, including:

[0083] Parties P1 and P2 use the same random number seed and random number generator to generate random number matrices A and B. The dimension of A is the same as that of X1, and the dimension of B is the same as that of V1;

[0084] Party P1 calculates X1 - A and sends X1 - A to the helper P0. Party P2 calculates V1 - B and sends V1 - B to the helper P0;

[0085] The helper P0 calculates (X1 - A)(V1 - B)=X1V1 - X1B - AV1 + AB, and generates a random number vector R. The dimension of R is the same as that of X1V1;

[0086] The helper P0 calculates (X1 - A)(V1 - B)-R and sends (X1 - A)(V1 - B)-R to Party P1, and sends R to Party P2;

[0087] Party P1 calculates [X1·V1] o =(X1 - A)(V1 - B)-R + X1B, and Party P2 calculates [X1·V1] o 、 which are the secret shards of the cross - term of the forward activation value calculated by Parties P1 and P2,

[0088] The following is the proof of correctness:

[0089]

[0090] Similarly, the shards of other cross - terms can be calculated according to the above method.

[0091] After the above local calculations, the data held by each party is as follows:

[0092] Party P1:

[0093] Party P2:

[0094] Party P3:

[0095] Each party P i respectively calculates and records the calculation result as Z i, then P1 and P2 respectively send Z1 and Z2 to P3, and P3 locally calculates Z = Z1 + Z2 + Z3 = X1W1 + X2W2 + X3W3. Taking Z as the input of the top-level model can calculate the final predicted value of the model. The top-level model is unique to P3 and does not require collaborative calculation by each participant.

[0096] The specific algorithm of the above process is as follows:

[0097] (1) P1 locally calculates P2 locally calculates P3 locally calculates

[0098] (2) P1 sends Z1 to P3, and P2 sends Z2 to P3.

[0099] P3 calculates Z = Z1 + Z2 + Z3.

[0100] In one implementation, each participant obtains the output value according to the dataset, model weight shard, and secret shard of the forward activation value cross-term it owns. The passive party sends the obtained output value to the active party, and the active party obtains the forward propagation value according to the output value of the passive party and its own output value, including:

[0101] Each participant P i respectively calculates and records the calculation result as Z i ;

[0102] Participants P1 and P2 respectively send Z1 and Z2 to P3. P3 locally calculates Z = Z1 + Z2 + Z3 = X1W1 + X2W2 + X3W3. Taking Z as the input of the top-level model can calculate the final predicted value of the model. Among them, Z is the forward propagation value, the top-level model is unique to P3, and does not require collaborative calculation by each participant. P3 is the active party among the participants.

[0103] In one implementation, the active party and the passive party collaboratively calculate the model gradient cross-term and obtain the secret shard of the model gradient cross-term, including:

[0104] The passive party P1 and the active party P3 collaboratively calculate the model gradient cross-term using the new matrix multiplication algorithm based on the erasable mask, and respectively obtain the corresponding secret shards of the model gradient cross-term and where represents the partial derivative of the loss function with respect to Z. Since is unique to P3 and Z is the forward propagation value;

[0105] The passive party P2 and the active party P3 cooperate to calculate the cross-term of the model gradient using a new matrix multiplication algorithm based on erasable masks, and respectively obtain the secret shards of the corresponding cross-term of the model gradient. and

[0106] Specifically, the goal of backpropagation is: P1 updates U1, V3, P2 updates U2, V1, and P3 updates U3, V2.

[0107] For the participant P i , the model gradient in plaintext form can be calculated by . Since P3 obtained the value of Z during forward propagation, P3 can locally calculate where represents the partial derivative of the loss function with respect to Z. Since is unique to P3, similar to the method of calculating the cross-term during forward propagation, P1 and P3 need to cooperate to calculate the value of the cross-term and obtain its secret shard, and P2 and P3 need to cooperate to calculate the value of the cross-term and obtain its secret shard. In addition, since P2 is the party that holds the shard V1 of the model weights of P1, P3 needs to send to P2. The specific algorithm for the above process is as follows:

[0108] (1) P3 locally calculates

[0109] (2) P1 and P3 respectively calculate and

[0110] (3) P2 and P3 respectively calculate and

[0111] (4) P3 sends to P2.

[0112] After the above local calculations and one round of communication, the data held by each participant during backpropagation is as follows:

[0113] Participant P1:

[0114] Participant P2:

[0115] Participant P3:

[0116] Assuming the learning rate is μ, each party locally updates the shards of the model weights:

[0117] P1:

[0118] P2:

[0119] P3:

[0120] Taking W1 as an example, in plaintext form, through update W1, and the proof of the correctness of the above update result is as follows:

[0121]

[0122] Since both linear regression and logistic regression are based on similar linear models, and their calculation processes are essentially similar, therefore, this privacy protection scheme can be applied not only to the logistic regression model, but also to the linear regression model. Whether for classification tasks or regression tasks, this scheme can effectively guarantee data privacy while achieving efficient model training and inference.

[0123] Embodiment 2

[0124] Based on the same inventive concept, this embodiment discloses a vertical federated logistic regression device based on multi-party matrix multiplication, which is applied to a privacy computing architecture including three or more participating parties. The participating parties include an active party and passive parties, and each participating party has its own dataset. The facilitator is responsible for assisting the secure multiplication calculation of all parties during the training process. Please refer to Figure 4 and the said device includes:

[0125] Initialization module 101, each participating party initializes the first model weight shard of its own model and the second model weight shard of the adjacent previous participating party's model. The model weight shard includes the first model weight shard and the second model weight shard, and the sum of the first model weight shard and the second model weight shard is the model weight;

[0126] The forward propagation module 102 is used for each participating party to locally calculate the shards of forward activation values according to the first model weight shards and the datasets they own. Each participating party collaboratively calculates the secret shards of the cross terms of forward activation values using the new matrix multiplication algorithm based on the erasable mask according to the second model weight shards and the datasets they own. Then, each participating party obtains the output values according to the datasets they own, the model weight shards, and the secret shards of the cross terms of forward activation values. Among them, the passive party sends the obtained output values to the active party, and the active party obtains the forward propagation values according to the output values of the passive party and its own output values. Among them, each participating party collaboratively calculates the secret shards of the cross terms of forward activation values using the new matrix multiplication algorithm based on the erasable mask according to the second model weight shards and the datasets they own, including: each participating party generates random numbers respectively, adds random masks to the datasets they own based on the random numbers, and the assistant and the participating parties jointly calculate the secret shards of the cross terms of forward activation values based on matrix multiplication. The sum of all shards of forward activation values and the secret shards of the cross terms of forward activation values constitutes the forward activation values;

[0127] The backpropagation module 103 is used for each participating party to locally calculate the model gradients. The active party and the passive party collaboratively calculate the cross terms of model gradients and obtain the secret shards of the cross terms of model gradients. Each participating party locally updates the model weight shards according to the secret shards of the cross terms of model gradients. Among them, the cross terms of model gradients are part of the model gradients that require collaborative calculation by the active party and the passive party.

[0128] Since the system introduced in the second embodiment of the present invention is the system adopted for implementing the vertical federated logistic regression method based on multi-party matrix multiplication in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, those skilled in the art can understand the specific structure and variations of this system, so it will not be elaborated here. Any system adopted by the method in the first embodiment of the present invention belongs to the scope protected by the present invention.

[0129] Embodiment Three

[0130] Based on the same inventive concept, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method described in Embodiment One.

[0131] Since the computer-readable storage medium introduced in the third embodiment of the present invention is the computer-readable storage medium adopted for implementing the vertical federated logistic regression method based on multi-party matrix multiplication in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, those skilled in the art can understand the specific structure and variations of this computer-readable storage medium, so it will not be elaborated here. Any computer-readable storage medium adopted by the method in the first embodiment of the present invention belongs to the scope protected by the present invention.

[0132] Embodiment Four

[0133] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method described in Embodiment 1 is implemented.

[0134] Since the computer device introduced in Embodiment 4 of the present invention is the computer device adopted for implementing the vertical federated logistic regression method based on multi-party matrix multiplication in Embodiment 1 of the present invention, based on the method described in Embodiment 1 of the present invention, those skilled in the art can understand the specific structure and variations of this computer device, so it will not be elaborated here. Any computer device adopted for the method of Embodiment 1 of the present invention falls within the scope protected by the present invention.

[0135] Embodiment 5

[0136] Based on the same inventive concept, the present invention also provides a privacy computing system, including the vertical federated logistic regression device based on multi-party matrix multiplication in Embodiment 2 and a horizontal federated computing device. Among them, the horizontal federated computing device is used to calculate a check value by weighting and averaging model parameters with random numbers to assist in detecting malicious tampering of model parameters.

[0137] Specifically, the horizontal federated computing device adopts a horizontal federated algorithm, which is different from the existing method of randomly selecting several parameters to participate in verification and also different from the scheme based on verifiable signatures. The present invention calculates the check value by weighting and averaging model parameters with random numbers to detect malicious tampering of model parameters by the auxiliary server.

[0138] In the specific implementation process, it is assumed that the number of participating parties is greater than 2. Each participating party server i runs a machine learning algorithm locally and masks its own model gradient with a random number matching its own number, that is: mg i, = g i, + m ij , where g i, represents the j-th gradient of the participating party server i, j = 1, 2,... k, and m ij represents the mask generated by the random number generator for this gradient. Then each participating party server designs a checksum using the random number mechanism to prevent the server from tampering with the model parameters. For example, each participating party server generates k random numbers (r1, r2,..., r k ) respectively and calculates the check value checksum i =(∑ j r j g i, ) / k, and then sends the masked gradient and the check value to the aggregation server together, r j gi, Denotes the value obtained by masking a gradient gij with r j (random number), and the verification value is the sum of all the masked gradients of the summation server i itself.

[0139] The aggregation server aggregates the masked gradients and verification values of all parties according to the aggregation rules of horizontal federation and returns them to each participating party server. Each participating party server subtracts the sum of the random masks of all participating parties from the aggregated value to obtain the plaintext aggregated gradient, and then recalculates the verification value in the same way according to the random numbers used when generating the verification value and compares it with the received verification value to detect whether the aggregation server has tampered with the gradient value. If the verification passes, the aggregated gradient value is used to update the local model parameters. It is difficult for a malicious auxiliary server to pass the verification check without knowing the random numbers (r1, r2,..., r k ). The scheme designed by the present invention is more efficient than the verifiable signature-based scheme in terms of the calculation and verification of the checksum, and can prevent undetected or exhaustive attacks.

[0140] Since the above scheme is a scheme under the semi-honest assumption. To meet the conditions required by the scheme, technical means are needed to make the participating parties abide by the protocol. Therefore, let the auxiliary server run in the TEE environment. Currently, the existing attack means against TEE cannot modify the code inside the TEE as expected, and the communication between each participating party and the auxiliary server party uses different secret channels to prevent data interception.

[0141] Generally speaking, the present invention is committed to solving the problems existing in the current schemes of vertical federated logistic regression and horizontal federation with a general privacy computing architecture. The proposed new scheme for multi-party vertical federated logistic regression and the new scheme for vertical federated linear regression do not outsource the original data, so they are applicable to high-dimensional and sparse data sets. At the same time, each party cannot master any underlying model, so it can resist model completion attacks or label information leakage attacks. The designed horizontal federation algorithm can detect any modification and is more efficient than the existing schemes.

[0142] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0143] The present invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each flow and / or block in the flowchart illustrations and / or block diagrams, and combinations of flows and / or blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing apparatus create means for implementing the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the functions specified in one or more blocks.

[0144] Although the preferred embodiments of the present invention have been described, additional changes and modifications can be made by those skilled in the art once they learn of the basic inventive concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art can make various changes and variations to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A vertical federated logistic regression method based on multi-party matrix multiplication, characterized in that, Applied to a privacy computing architecture involving three or more participating parties, where the participating parties include an active party and passive parties, and each participating party has its own dataset. A facilitator is responsible for assisting in the secure multiplication calculation among all parties during the training process. The method includes: Each participating party initializes the first model weight shard of its own model and the second model weight shard of the adjacent previous participating party's model. The model weight shard includes the first model weight shard and the second model weight shard, and the sum of the first model weight shard and the second model weight shard is the model weight. Each participating party locally calculates the forward activation value shard based on the first model weight shard and its own dataset. Each participating party collaboratively calculates the secret shard of the forward activation value cross-term using a new matrix multiplication algorithm based on an eliminable mask according to the second model weight shard and its own dataset. Then each participating party obtains the output value based on its own dataset, the model weight shard, and the secret shard of the forward activation value cross-term. Among them, the passive party sends the obtained output value to the active party, and the active party obtains the forward propagation value based on the output value of the passive party and its own output value. Among them, each participating party collaboratively calculates the secret shard of the forward activation value cross-term using a new matrix multiplication algorithm based on an eliminable mask according to the second model weight shard and its own dataset, including: each participating party generates a random number respectively, and adds a random mask to the dataset it owns based on the random number. The facilitator and the participating parties jointly calculate the secret shard of the forward activation value cross-term based on matrix multiplication. The sum of all forward activation value shards and the secret shard of the forward activation value cross-term constitutes the forward activation value. Each participating party locally calculates the model gradient. The active party and the passive party collaboratively calculate the model gradient cross-term and obtain the secret shard of the model gradient cross-term. Each participating party locally updates the model weight shard according to the secret shard of the model gradient cross-term. Among them, the model gradient cross-term is a part of the model gradient that requires collaborative calculation by the active party and the passive party.

2. The vertical federated logistic regression method based on multi-party matrix multiplication according to claim 1, characterized in that, Each participating party initializes the first model weight shard of its own model and the second model weight shard of the adjacent previous participating party's model, including: Participant P i Initialize U i and V i-1 where i = {1, 2, 3}, U i is the first model weight shard of the model for Participant P i and V i-1 is the second model weight shard of the model for the adjacent previous Participant P i of Participant P. The adjacent previous participant of Participant P1 is P3; i-1 ​ The model weight of each participating party is calculated by W = U + V, where U is the first model weight shard, V is the second model weight shard, and W is the model weight.

3. The vertical federated logistic regression method based on multi-party matrix multiplication according to claim 2, characterized in that, Each participating party locally calculates the forward activation value shard based on the first model weight shard and its own dataset, including: Participant P i According to the obtained first model weight shard U i And the owned dataset X i Calculate the local pre-computation forward activation value shard X i U i .

4. The vertical federated logistic regression method based on multi-party matrix multiplication according to claim 2, characterized in that, Each participating party generates a random number respectively, and adds a random mask to the dataset it owns based on the random number. The facilitator and the participating parties jointly calculate the secret shard of the forward activation value cross-term, including: Participating parties P1 and P2 use the same random number seed and random number generator to generate random number matrices A and B. The dimension of A is the same as that of X1, and the dimension of B is the same as that of V1. Participating party P1 calculates X1 - A and sends X1 - A to facilitator P0. Participating party P2 calculates V1 - B and sends V1 - B to facilitator P0. Facilitator P0 calculates (X1 - A)(V1 - B) = X1V1 - X1B - AV1 + AB and generates a random number vector R. The dimension of R is the same as that of X1V1. The assistant P0 calculates (X1 - A)(V1 - B) - R and sends (X1 - A)(V1 - B) - R to the participant P1, and sends R to the participant P2; Participant P1 calculates [X1·V1] o =(X1 - A)(V1 - B) - R + X1B, and Participant P2 calculates where is the secret share of the cross-term of the forward activation values calculated by Participants P1 and P2, 5. The vertical federated logistic regression method based on multi-party matrix multiplication according to claim 2, wherein Each participant obtains the output value based on its own dataset, the shards of the model weights, and the secret shards of the cross - terms of the forward activations. The passive party sends the obtained output value to the active party, and the active party obtains the forward propagation value based on the output value of the passive party and its own output value, including: Each participating party P i Calculate separately And record the calculation result as Z i ; The participants P1 and P2 respectively send Z1 and Z2 to P3. P3 locally calculates Z = Z1 + Z2 + Z3 = X1W1 + X2W2 + X3W3. Using Z as the input of the top - level model can calculate the final prediction value of the model. Here, Z is the forward propagation value, the top - level model is unique to P3 and does not require the cooperation of each participant in calculation. P3 is the active party among the participants.

6. The vertical federated logistic regression method based on multi-party matrix multiplication according to claim 1, characterized in that, The active party and the passive party cooperate to calculate the cross - terms of the model gradients and obtain the secret shards of the cross - terms of the model gradients, including: The passive party P1 and the active party P3 cooperate to calculate the gradient cross-term of the model using a new matrix multiplication algorithm based on an eliminable mask, and respectively obtain the secret shards of the corresponding gradient cross-term of the model. and where represents the partial derivative of the loss function with respect to z. Since is unique to P3, and Z is the forward propagation value; The passive party P2 and the active party P3 use a new matrix multiplication algorithm based on an eliminable mask to collaboratively calculate the gradient cross-term of the model, and respectively obtain the secret shards of the corresponding gradient cross-term of the model. and 7. A vertical federated logistic regression device based on multi-party matrix multiplication, characterized in that Applied to a privacy computing architecture with three or more participants. The participants include active parties and passive parties. Each participant has its own dataset, and the assistant is responsible for assisting the secure multiplication calculation among all parties during the training process. The device includes: An initialization module. Each participant initializes the first shard of the model weights of its own model and the second shard of the model weights of the adjacent previous participant's model. The shards of the model weights include the first shard of the model weights and the second shard of the model weights, and the sum of the first shard of the model weights and the second shard of the model weights is the model weight. A forward propagation module. It is used for each participant to locally calculate the shards of the forward activations according to the first shard of the model weights and its own dataset. Each participant cooperatively calculates the secret shards of the cross - terms of the forward activations using a new matrix multiplication algorithm based on an eliminable mask according to the second shard of the model weights and its own dataset. Then each participant obtains the output value according to its own dataset, the shards of the model weights, and the secret shards of the cross - terms of the forward activations. Among them, the passive party sends the obtained output value to the active party, and the active party obtains the forward propagation value according to the output value of the passive party and its own output value. Here, each participant cooperatively calculates the cross - terms of the forward activations using a new matrix multiplication algorithm based on an eliminable mask according to the second shard of the model weights and its own dataset, including: each participant generates its own random number and adds a random mask to the dataset it owns, and the assistant and the participant jointly calculate the secret shards of the cross - terms of the forward activations based on matrix multiplication. The sum of all shards of the forward activations and the secret shards of the cross - terms of the forward activations constitutes the forward activation value. A backward propagation module. It is used for each participant to locally calculate the model gradients. The active party and the passive party cooperate to calculate the cross - terms of the model gradients and obtain the secret shards of the cross - terms of the model gradients. Each participant locally updates the shards of the model weights according to the secret shards of the cross - terms of the model gradients. Here, the cross - terms of the model gradients are part of the model gradients that require the cooperation of the active party and the passive party in calculation.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the vertical federated logistic regression method based on multi-party matrix multiplication as described in any one of claims 1 to 6.

9. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the vertical federated logistic regression method based on multi-party matrix multiplication as described in any one of claims 1 to 6.

10. A privacy computing system, characterized in that, It includes the vertical federated logistic regression device based on multi-party matrix multiplication as described in claim 7 and a horizontal federated computing device, wherein the horizontal federated computing device is used to calculate a check value by means of weighted averaging of model parameters with random numbers to assist in detecting malicious tampering with the model parameters.

Images