Gradient boosting decision table-based privacy protection vertical federated learning method and system

Abstract

The application provides a privacy protection vertical federated learning method and system based on gradient boosting decision table, and the method is applied to a system comprising a plurality of participating terminals, each participating terminal holds part of feature data of each learning sample, and the categories of the feature data held by each participating terminal are different; in the learning method provided by the application, all intermediate data is encrypted in the whole learning process, each participating terminal can only obtain additive secret sharing shares of the data, and cannot obtain complete plaintext data. The application can realize privacy protection of privacy protection vertical federated learning data based on gradient boosting decision table.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information security, and particularly relates to a privacy protection vertical federated learning method and system based on gradient boosted decision tables. BACKGROUND

[0002] Federated learning (FL) as a kind of distributed machine learning greatly enhances the cooperation of multiple parties. FL allows the training data of participants to be saved locally and only shares the intermediate output of the training algorithm for aggregation. According to the distribution of data among participants, FL can be divided into two types: horizontal federated learning (HFL) and vertical federated learning (VFL). Among them, VFL is aimed at the scenario where each participant has the same sample but has different feature data. Participants of VFL hold datasets with the same row index (corresponding to the same set of instances) but different non-overlapping column indexes (corresponding to different features). In recent years, VFL has attracted more and more attention in the cooperation of different enterprises.

[0003] In existing VFL research, gradient boosting is a research hotspot because gradient boosting has been widely used in many fields such as web search ranking, online advertising and fraud detection. Gradient boosting trains a number of weak learners through gradient boosting technology to obtain a model with better prediction performance. Many works show that gradient boosted decision tables can handle many machine learning tasks and achieve better prediction efficiency than common decision trees.

[0004] All existing decision table learning and prediction algorithms consider centralized data and cannot directly support secure gradient boosting algorithms for decision tables under VFL, that is, in the prior art, in the process of VFL, each party may obtain the data held by other parties, and may also obtain the intermediate data in the learning process and thus obtain the parameters of the entire model. However, in practical applications, due to privacy protection considerations, enterprises may not be willing to share their own data with other enterprises, and at the same time, may not be willing to share their own model parameters with other enterprises. Therefore, the prior art cannot realize privacy protection of intermediate data of VFL based on gradient boosted decision tables. SUMMARY

[0005] The application provides a privacy protection vertical federated learning method and system based on gradient boosting decision table, which solves the problem that the privacy of intermediate data in vertical federated learning based on gradient boosting decision table cannot be protected in the prior art, and realizes privacy protection of vertical federated learning data based on gradient boosting decision table.

[0006] The application provides a privacy protection vertical federated learning method based on gradient boosting decision table, which is applied to a system comprising a plurality of participating terminals, each of which holds partial feature data of each learning sample, and the feature data held by each participating terminal is different in category; the method comprises:

[0007] A first target participating terminal in each participating terminal determines an indication vector corresponding to a first target layer based on first target feature data of each learning sample held locally and optimal tests corresponding to the first target layer in a target decision table, wherein the target decision table is a decision table to be learned in the current learning round, the target feature data corresponds to the first target layer, the first target participating terminal holds the first target feature data corresponding to the first target layer, and the indication vector is used to reflect that each node in the first target layer corresponds to a sample divided into a first sub-node or a second sub-node of the node, and the optimal test corresponding to each layer in the target decision table is used to determine the division result of each sample in the layer;

[0008] The first target participating terminal generates n first additive secret sharing shares corresponding to the indication vector, and sends n-1 first additive secret sharing shares to the remaining participating terminals in the system, respectively, wherein n is the number of participating terminals;

[0009] Each participating terminal updates a second additive secret sharing share and a third additive secret sharing share corresponding to each node in a second target layer held locally based on each first additive secret sharing share held locally, wherein the second target layer is a layer in which a sub-node of the first target layer in the target decision table is located, the second additive secret sharing share is an additive secret sharing share of a first-order gradient vector, and the third additive secret sharing share is an additive secret sharing share of a second-order gradient vector, when the i-th learning sample is divided into a target node, the i-th value in the first-order gradient vector corresponding to the target node is the current first-order gradient of the i-th learning sample, and the remaining values are 0, and the i-th value in the second-order gradient vector corresponding to the target node is the current second-order gradient of the i-th learning sample, and the remaining values are 0;

[0010] A second target participating terminal in each of the participating terminals calculates an evaluation score corresponding to each candidate test in a secret sharing domain based on the locally held second additive secret sharing share and the third additive secret sharing share, and determines an optimal test corresponding to the second target layer based on a secret sharing share of the evaluation score, wherein the second target participating terminal holds second target feature data corresponding to the second target layer;

[0011] After the optimal test of each layer of the target decision table is determined, the current learning round is completed, and each participating terminal obtains a fourth additive secret sharing share of a prediction result of each learning sample based on the learned decision table, and obtains the second additive secret sharing share and the third additive secret sharing share corresponding to a first node in a decision table learned in a next learning round based on the fourth additive secret sharing share, until the number of learned decision tables reaches a preset number, to complete the learning process.

[0012] The application further provides a privacy protection vertical federated learning system based on gradient boosting decision table, comprising a plurality of participating terminals, which cooperatively complete the privacy protection vertical federated learning method based on gradient boosting decision table.

[0013] The privacy protection vertical federated learning method based on gradient boosting decision table provided by the application maintains a corresponding gradient vector for each node in the decision table, each participating terminal holds an additive secret sharing share of the gradient vector, and a participating terminal holding complete feature data sends an additive secret sharing share of an indicator vector to each participating terminal, and each participating terminal updates the secret sharing share of the gradient vector of the next layer node through the indicator vector, so that the value corresponding to the sample not divided into the sub-node in the gradient vector of the node is 0, and the value corresponding to the sample divided into the sub-node is 1, so that the number of samples processed by each node remains unchanged, except for the terminal holding the complete feature data, all participating terminals cannot infer any information from the node splitting, including the optimal test threshold of the complete feature data and which sample feature data is less than or greater than the threshold, further, the method provided by the application further determines the additive secret sharing share of the threshold of the best test of each layer in the decision table based on additive secret sharing, and determines the additive secret sharing share of the prediction result of the learning sample, which is used for learning of the next round of decision table, and all intermediate data is encrypted during the whole learning process, each participating terminal can only obtain the additive secret sharing share of the data, and cannot obtain the complete plaintext data, therefore, the application can realize privacy protection of the privacy protection vertical federated learning data based on gradient boosting decision table. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0015] Figure 1 Schematic diagram of learning algorithm based on decision table for plaintext domain;

[0016] Figure 2 Schematic diagram of comparison of decision tree equivalent to conventional decision tree and decision table;

[0017] Figure 3 Flowchart of the privacy protection vertical federated learning method based on gradient boosting decision table provided by the present application;

[0018] Figure 4 Schematic diagram of the overall framework of the privacy protection vertical federated learning method based on gradient boosting decision table provided by the present application;

[0019] Figure 5 Schematic diagram of the process of secure node splitting in the privacy protection vertical federated learning method based on gradient boosting decision table provided by the present application;

[0020] Figure 6 Schematic diagram of the process of secure permutation in the privacy protection vertical federated learning method based on gradient boosting decision table provided by the present application;

[0021] Figure 7 Schematic diagram of the process of secure discretization in the privacy protection vertical federated learning method based on gradient boosting decision table provided by the present application;

[0022] Figure 8 Schematic diagram of the process of distributed decision table training in the privacy protection vertical federated learning method based on gradient boosting decision table provided by the present application;

[0023] Figure 9 Schematic diagram of the process of secure decision table prediction in the privacy protection vertical federated learning method based on gradient boosting decision table provided by the present application. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts belong to the scope of protection of the present invention.

[0025] Aiming at the problem that privacy protection cannot be achieved in the existing vertical federated learning technologies based on gradient boosting decision tables, the present invention provides a vertical federated learning method based on gradient boosting decision tables, which supports any number of participating parties to jointly train gradient boosting decision tables. At the same time, it allows these participating parties to store data locally and provides strong privacy protection for sensitive intermediate output data during the training process. To facilitate those skilled in the art to understand the technical solutions of the privacy-protected vertical federated learning method based on gradient boosting decision tables provided by the present invention, the technical background of the method provided by the present invention will be described first.

[0026] 1. Decision Table

[0027] Consider a data set containing N samples where x i =(x i1 ,..., x iJ ) is a J-tuple, and y i is the label of the i-th sample. The j-th element of x i is the value of feature X j . A D-dimensional decision table consists of D boolean tests and 2 D output values. The boolean test is in the form of X j <t, and if the j-th element in the given input tuple is less than the threshold t, the output is 1, otherwise the output is 0.

[0028] As Figure 2 shown, a D-dimensional decision table is equivalent to a D+1-layer full binary tree, where each internal node has a boolean test from the 0th layer (for the root node) to the D-1th layer; each edge is assigned the result of its parent node's test, and each leaf node in the Dth layer is associated with an output value. This equivalent tree is called an oblivious tree because all internal nodes at the same level share the same test, which is completely different from a conventional decision tree with different tests at the same level. More specifically, the test at the d-th layer of the oblivious tree can be expressed as F d <t d , where d ∈ [0, D-1], the split feature F d ∈ (X1,..., X J ), t dis the split threshold. The special structure of oblivious tree leads to different training and prediction methods compared with regular decision trees such as CART.

[0029] In the method provided by the invention, an oblivious tree is trained top-down. As Figure 1 shown, Figure 1 the training algorithm of oblivious tree is given in Algorithm 1, which outputs D tests and 2 D output values. The learning algorithm starts from the 0th layer and iteratively builds the oblivious tree layer by layer. Given a test X j < t, define This notation is also applied to the subset Let denote the set of subsets at the lth layer, where l e [0, D]. At the 0th layer, the training dataset is associated with the root node and satisfies (line 1). Once the optimal test F0< T0 Figure 1 for this layer is found by the method find_split( (line 4 in Algorithm 1), the dataset will be divided into two subsets according to this test. After that, is added to and a new layer is created (lines 6-7 in Algorithm 1). Figure 1

[0030] At the 1st layer, the training algorithm finds an optimal test which divides the dataset into two subsets according to F1< t1, respectively. Repeat the above steps until all D tests are learned. This way makes the tree structure completely symmetrical. At the lth layer, the decision table satisfies where each set in is associated with a node at this layer. When the Dth layer is reached, the training algorithm will calculate the output values for the leaf nodes. Finally, an oblivious tree consisting of D tests and 2 D output values can be learned.

[0031] The optimal test for each layer is found by evaluating the candidate tests with the find_split method. The evaluation of candidate tests can be done by different metrics. In this invention, the commonly used second-order approximation method is followed to evaluate the tests, because the decision table in this invention is trained for gradient boosting systems. In addition, the output values of the decision table can also be calculated according to the gradient boosting theory.

[0032] The decision table is significantly superior to regular decision trees in prediction speed. As Figure 2 ​As shown, each leaf node of the oblivious tree ( Figure 2 The nodes on the right side of the table correspond to a Boolean sequence, and the comparisons required for the D tests can be parallelized. In contrast, predictions in a regular decision tree are made by traversing the tree from the root node to the leaf nodes, meaning the direction of the prediction path after the current node depends on the test results of that node. Predictions in a decision table do not have this dependency; D Boolean comparisons can be performed simultaneously, resulting in a Boolean sequence. The prediction result is then quickly obtained by looking up the table using the Boolean sequence. For ease of description, the output Boolean sequence will be referred to as the prediction result in the following text.

[0033] 2. Gradient Improvement Decision Table

[0034] Gradient boosting systems are built upon boosting algorithms by training a set of weak learners (a decision table can be considered a weak learner). For a given dataset... The gradient boosting system sums the predictions of T weak learners to obtain the final prediction result for the i-th sample: Where f t This corresponds to the t-th weak learner. A given sample will be assigned to a leaf node of the decision table based on the test results in the decision table, and its final prediction result is calculated by summing the output values ​​associated with the corresponding leaf node.

[0035] The essence of the gradient boosting algorithm lies in how it trains weak learners sequentially. After training t-1 weak learners, it is necessary to train the t-th model f. t Train and add to minimize the following objective function:

[0036]

[0037] in It is a quadratically differentiable convex loss function, which is based on... As input, and output loss. Regularization term Ω(f) t Refer to the standard XGBoost settings. Next, we will use a second-order approximation to quickly approximate the objective function:

[0038]

[0039] in These are the first and second gradients of the i-th sample, respectively. Typically, for regression problems, the MSE is used as the loss function, and the gradients are calculated as follows: When the problem is classification, a common choice is logistic loss, and the gradient is calculated as follows: g i =p i -y i ,h i= p i x (1 - p i ), where For x e R, the sigmoid function is sigmoid(x) = 1 / (1 + e -x ). For a leaf node k associated with a subset , define I k = {i | (x i , y i ) e V k} as its sample set. This notation is also used to denote the sample set associated with an internal node, e.g., write node q as I q . Then, after removing the constant term, formula 1 can be rewritten as

[0040]

[0041] where w k is the output value associated with leaf node k, L is the number of leaf nodes in the tree, and λ, γ are hyperparameters that control regularization. When the tree stops growing, w k and the minimum loss of the current tree are computed as follows:

[0042]

[0043]

[0044] Formula 4 can be used as an impurity function to evaluate tests. Following the theory described above, to find the optimal test in a decision table, assume that b tests have been learned from level 0 to (b-1) of a D-dimensional decision table, then a best test needs to be found at level b. A candidate test X j < t will split the 2 b nodes at this level into 2 b+1 nodes. Among all the candidate tests, the optimal test is the one with the smallest score. The score is defined as:

[0045]

[0046] where,

[0047]

[0048] is the impurity of the node, are the sample sets associated with the left and right child nodes of the qth node after splitting, respectively.

[0049] 3. Additive secret sharing

[0050] In the present invention, additive secret sharing over a ring is used, where Q represents the number of bits used to represent numerical values. In such a secret sharing, a secret value additively split into n secret shares to satisfy [[x]] = <x> 1+ <x> 2+…+ <x> n mod2 Q And these n shares are held by n parties respectively. For simplicity, let [[x]] denote the additive secret sharing of x. The following introduces the basic operations related to additive secret sharing in the n-party scenario.

[0051] Sharing (Sharing): In order to share P l privately x of a party, P l n-1 random numbers {x m}, m∈[1, n], m≠l, and send x m to the corresponding P m . Then P l holds Each other participant P m∈[1,n],m≠l holds <x> m = x m , as a secret share of x. For brevity, we will omit the modulo operation notation in the following protocols.

[0052] Reconstruction: In order to reconstruct (Rec(·)) the shared value [[x]] on P l , each other participant P l will provide its secret share m∈[1,n],m≠l of x to P <x> m send to P l , then P l compute

[0053] Addition: For two secret shared values [[x]] and [[y]], to securely compute the addition ([[z]] = [[x]] + [[y]]), each participant P m∈[1,n] locally compute <z> m ＝ <x> m + <y> m Similarly, to compute a subtraction ( [[z]] = [[x]] - [[y]] ), each participant subtracts the share of y from the share of x.

[0054] Multiplication: To multiply a secret share value [[x]] by a constant c ( [[z]] = c * [[x]] ), each participant can simply multiply its local share of x by c. To multiply two secret share values [[x]] and [[y]], a multiplication triple can be used. In the offline phase, all parties get a secret share of a multiplication triple ( [[a]], [[b]], [[c]] ), where a, b are uniform random numbers in Zp and c = ab. The secret share triple is data independent and can be prepared and distributed offline by a third party, so we assume that triples are available for online secure computation between parties from now on. The secret share multiplication process is as follows: First, each party P computes m∈[1,n] locally <e> m ＝ <x> m - m and <f> m ＝ <y> m - m . After that, both parties run Rec([[x]]), Rec([[y]]). Next, P m computes <z> m = j x e x f + f x< / z> < / y> < / f> m +e× m + <c> m where j = 1 if m = 1, and j = 0. Finally, P m will be <z> m The secret share is the result of multiplication.

[0055] The method provided by this invention can be applied in the field of cloud computing to solve the problem of privacy protection when multiple stakeholders (e.g., business organizations) collaboratively train gradient boosting decision tables on vertically partitioned data. In such a scenario, a dataset containing N samples (each sample is associated with a feature vector and a label) is distributed across the terminals P1, P2, ..., P... of n stakeholders. n Vertically divided between them. Each participating terminal P m They each hold their own datasets J m P represents m The number of features possessed and satisfying represent The i-th sample in the dataset. In other words, the method provided by this invention is applied to a system comprising multiple participating terminals, each terminal holding partial feature data of each learning sample, and the categories of feature data held by each participating terminal are different. The label set represents the sample. Based on previous work on vertical federated learning, we consider two roles of participants: active participants (AP) and passive participants (PP). Specifically, there is one AP that holds a local dataset and a label set y; the remaining participants are PPs, each holding only one local dataset.

[0056] Throughout the entire secure training process of this invention, each participating terminal stores its feature data locally. The privacy protection technology in this invention learns the Boolean tests and output values ​​in the decision table in a way that avoids data leakage. During and after the entire training process, no participant can obtain the complete model, thus achieving data privacy protection. In this invention, all participating terminals know that the training yields the splitting feature of each test in the gradient boosting decision table and who possesses that feature, but only the participating terminal possessing that feature knows the test's splitting threshold. Formally, for the Boolean test F of the t-th decision table (t∈[1,T]) at the d-th layer (d∈[0,D-1]) in the ensemble... d <t d Splitting characteristics F d The feature type is disclosed to all participating terminals, but only one participating terminal holds the specific feature data, and the corresponding threshold t d Only those who possess F d The participating terminals hold the values. Furthermore, all output values ​​of the leaf nodes are stored in a secret shared manner among all participating terminals.

[0057] Based on the method provided in the application, privacy protection can be realized in a semi-honest adversary model. Specifically, it is assumed that each participating terminal follows the protocol specification, but may attempt to infer the privacy information of other participating terminals. At the same time, although two roles AP and PP are considered for the participating terminal, the AP is not assumed to be given additional trust, because federated learning aims to break the data silos between organizations, and the behavior of each organization strictly complies with privacy regulations, so the semi-honest adversary model is a reasonable setting. In addition, the application can also support ensuring that the local data and the learned partial model (Boolean test and output value of each decision table in the integrated model) of each participating terminal are not leaked to other participating terminals in the presence of a static adversary that can compromise up to n-1 participating terminals. In the application, general parameters unrelated to data, such as the dimension D and the number T of decision tables, are not hidden.

[0058] The following will be described in combination with< / z> < / c> Figures 3-9 The application provides a privacy protection vertical federated learning method based on gradient boosting decision table.

[0059] First, the overall framework of the method provided by the application is briefly introduced. The method provided by the application is applied to a system including a plurality of participating terminals. Each participating terminal holds part of the feature data of each learning sample, and the feature data held by each participating terminal is different in category, as shown in Figure 4 Each participating terminal inputs a vertically classified data set and a label set y, thereby outputting a distributed decision table set ε. At the beginning, the encrypted prediction result is initialized to a secret sharing [[O N ]] The AP assigns the secret sharing of its label set to other participating terminals. After that, T decision tables are securely established in T rounds. The method provided by the application supports secure training of a single (distributed) decision table in each round, which can be regarded as mainly composed of three components, mainly including (i) secure node splitting SecSplit, (ii) secure sigmoid function calculation SecSigmoid, and (iii) secure discretization SecDisc. The secure node splitting is to securely split a node at a certain level and split the sample set related to these nodes, without leaking the sample set after splitting. The secure sigmoid function calculation process is to input a secret sharing value and calculate the sigmoid function in the secret sharing domain. The secure discretization is to securely rearrange the secret sharing gradient according to the local permutation owned by each participant, and then group them into buckets. In this way, the method provided by the application allows participating terminals to securely train a distributed decision table in each round. After securely training the distributed decision table in a certain round, secure prediction needs to be performed, and the result will be aggregated into the previous prediction result for use in decision table training in the next round. For this purpose, the method provided by the application also provides a secure decision table-based prediction result acquisition method, which can realize input of local data and part of the model of each participant to generate a secret sharing prediction result without leaking their data and part of the model. It is worth noting that in addition to the training process, the prediction result acquisition method in the method provided by the application can also be used to support secure prediction of new data after the completion of the entire training process.

[0060] As shown in Figure 3 The method provided by the application comprises the following steps:

[0061] S100, a first target participating terminal in each participating terminal determines an indication vector corresponding to a first target layer based on first target feature data of each learning sample held locally and an optimal test corresponding to the first target layer in a target decision table, wherein the target decision table is a decision table to be learned in a current learning round, the target feature data corresponds to the first target layer, the first target participating terminal holds the first target feature data corresponding to the first target layer, and the indication vector is used to reflect that each node in the first target layer corresponds to a sample being divided into a first sub-node or a second sub-node of the node; and the optimal test corresponding to each layer in the target decision table is used to determine a division result of each sample in the layer.

[0062] S200, the first target participating terminal generates n first additive secret sharing shares corresponding to the indication vector, and sends n-1 first additive secret sharing shares to the remaining participating terminals in the system respectively, wherein n is the number of participating terminals.

[0063] S300, each participating terminal updates a second additive secret sharing share and a third additive secret sharing share corresponding to each node in a second target layer held locally based on each first additive secret sharing share held locally, wherein the second target layer is a layer in which a sub-node of the first target layer in the target decision table is located, the second additive secret sharing share is an additive secret sharing share of a first-order gradient vector, the third additive secret sharing share is an additive secret sharing share of a second-order gradient vector, when an ith learning sample is divided into a target node, an ith value in a first-order gradient vector corresponding to the target node is a current first-order gradient value of the ith learning sample, and the remaining values are 0, and an ith value in a second-order gradient vector corresponding to the target node is a current second-order gradient value of the ith learning sample, and the remaining values are 0.

[0064] As explained earlier regarding the plaintext learning process of decision tables, the Oblivious tree grows to a new layer by splitting each node in the current layer into two child nodes. In plaintext decision table learning, node splitting is performed by partitioning the samples associated with the node to be split. However, in vertical federated learning, since all participants hold the same sample space, the sample partitioning method must be disclosed to all participants. For example, given a test "height < 180", participants with "height" feature data need to tell other participants which samples are less than 180 and which are greater than 180. Otherwise, participants holding feature data for the next layer cannot perform further partitioning, which would reveal the "height" range of each sample, causing privacy risks. To avoid such leakage, the method provided in this invention uses a secure node splitting method (called SecSplit) to protect the privacy of feature data. Specifically, in the method provided in this invention, each node is associated with a first-order gradient vector and a second-order gradient vector, that is, the k-th node (k ∈ [0, 2) of the d-th layer (d ∈ [0, D-1]) of the oblivious tree. d-1 ]) and a first-order gradient vector [[g (k,d) ]] and a second-order gradient vector [[h (k ,d) Related, each vector contains N elements, in all participating terminals P m∈[1,n] They are secretly shared among them. If the i-th sample is assigned to this node, [[g (k,d) ]] and [[h (k,d) The i-th element in the gradient vector will be set to the first and second gradients of the i-th sample, respectively; otherwise, the i-th element will be set to [[0]]. Since the elements in the gradient vector that are not assigned to a node are set to 0, even if these elements participate in the calculation, they will not affect the accuracy of the final result.

[0065] like Figure 5 As shown, based on the setting that each node is associated with a first-order gradient vector and a second-order gradient vector, the process of safe node splitting is as follows:

[0066] The optimal test F is found in layer d (the first target layer). d <t d The participating terminal (i.e., the first target participating terminal) P l Generate the first indicator vector v locally. l Second indicator vector v r Then it is secretly shared with other participating terminals (represented as [[v]). l ]] and [[v r ]]). That is, the indicator vectors include a first indicator vector and a second indicator vector, the i-th value in the first indicator vector is 1 when the i-th learning sample is divided into the first sub-node of the belonging node, otherwise 0, the i-th value in the second indicator vector is 1 when the i-th learning sample is divided into the second sub-node of the belonging node, otherwise 0. Specifically, as shown in FIG. 3, in the decision tree corresponding to the decision table, each node is split into two nodes, which are the sub-nodes of the split node, so according to the optimal test of the first target layer, all learning samples will have two division cases: divided into the first sub-node (which can be the left sub-node) or the second sub-node (which can be the right sub-node). Figure 2

[0067] After receiving [[v l ]] and [[v r ]], for the k-th node of the second target layer, P m∈[1,n] The first-order and second-order gradient vectors of the left and right sub-nodes of the k-th node are updated. The update is realized by secure element multiplication between the indicator vector and the gradient vector. That is, each participating terminal updates the second additive secret sharing share and the third additive secret sharing share corresponding to each node in the second target layer based on the local holding of each first additive secret sharing share, including:

[0068] When the target node in the second target layer is the first sub-node, each participating terminal calculates the secret sharing share of the product of the first indicator vector and the first-order gradient vector corresponding to the target node based on the additive secret sharing protocol as the second additive secret sharing share corresponding to the updated target node, and calculates the secret sharing share of the product of the first indicator vector and the second-order gradient vector corresponding to the target node as the third additive secret sharing share corresponding to the updated target node.

[0069] When the target node in the second target layer is the second sub-node, each participating terminal calculates the secret sharing share of the product of the second indicator vector and the first-order gradient vector corresponding to the target node based on the additive secret sharing protocol as the second additive secret sharing share corresponding to the updated target node, and calculates the secret sharing share of the product of the second indicator vector and the second-order gradient vector corresponding to the target node as the third additive secret sharing share corresponding to the updated target node.

[0070] ​Thus, since the learning samples divided to the right child node have 0 in the corresponding element in the first indicator vector, that is, the value corresponding to the learning samples divided to the right child node in the first-order gradient vector corresponding to the left child node becomes 0, which is equivalent to excluding the learning samples from the sample set corresponding to the left child node. This design hides the sample set processed by each node, and the number of samples processed by each node remains unchanged as N, which means that except for the participating terminal holding the optimal test of the current layer, the remaining participating terminals cannot infer any information from the node splitting.

[0071] It can be understood that, when training the first decision table, since it does not have a previous prediction process, the first-order gradient and the second-order gradient of each learning sample cannot be obtained from the previous prediction result, at this time, the first-order gradient vector and the second-order gradient vector can be randomly initialized or directly set to 0.

[0072] Referring back to Figure 3 The method provided by the application further includes the steps of:

[0073] S400, the second target participating terminal in each participating terminal calculates the evaluation score corresponding to each candidate test in the secret sharing domain based on the second additive secret sharing share and the third additive secret sharing share held locally, and the second target participating terminal determines the optimal test corresponding to the second target layer based on the secret sharing share of the evaluation score, wherein the second target participating terminal holds the second target feature data corresponding to the second target layer.

[0074] A simple method for finding the optimal test is to use a greedy algorithm to enumerate each training sample to find the best test, but enumerating all training samples will generate a large amount of computational overhead, and will also generate expensive communication overhead in a distributed environment, reducing the efficiency of the system. The method provided by the application uses secure discretization learning (referred to as SecDisc) to enable the application to adapt to larger data sets.

[0075] Specifically, the second target participating terminal in each participating terminal calculates the evaluation score corresponding to each candidate test in the secret sharing domain based on the second additive secret sharing share and the third additive secret sharing share held locally, including:

[0076] The second target participating terminal sorts the second target feature data of each learning sample held locally to obtain a first sorting result, wherein the second target participating terminal holds the second target feature data corresponding to the second target layer;

[0077] The second target participating terminal updates the second additive secret sharing share and the third additive secret sharing share corresponding to each node in the second target layer based on the first sorting result and the remaining each participating terminal, to realize the reordering of the gradient values in the first-order gradient vector and the second-order gradient vector corresponding to each node in the second target layer, so that the sorting result of the gradient values in the first-order gradient vector and the second-order gradient vector is consistent with the first sorting result;

[0078] Each participating terminal performs bucketing on each gradient value in the first-order gradient vector and the second-order gradient vector held locally based on the preset number of buckets, to obtain secret sharing shares of each bucket, and each bucket includes a plurality of first-order gradient values and second-order gradient values arranged in order;

[0079] Each participating terminal determines the secret sharing share of the evaluation score corresponding to each candidate test based on the secret sharing share of each bucket held this time, wherein the threshold value of each candidate test is one of the first gradient value and the second gradient value corresponding to the second target feature data, or a value between the first gradient value and the second gradient value corresponding to the second target feature data, the first gradient value is the last first-order gradient value or second-order gradient value in the first bucket, the second gradient value is the first first-order gradient value or second-order gradient value in the second bucket, and the first bucket is the former one of the two adjacent buckets, and the second bucket is the latter one of the two adjacent buckets.

[0080] Discretization, also known as bucketing, is a commonly used method in large-scale machine learning. Specifically, discretization groups samples into a small number of buckets in order to allow model training to support larger datasets. Let B denote the number of buckets in discretization, where B << N, N is the number of samples. In gradient boosting, gradients are grouped into buckets, and the sum of gradients in each bucket needs to be computed during the training phase. Typically, for each feature, the gradients are first permuted by a permutation π, which is obtained by sorting the values of the feature. The permuted gradients are then divided into B buckets. Obviously, the bucketing operation can greatly reduce the cost of training. However, discretization is not easy in privacy-preserving machine learning. In the present invention, the training data is vertically partitioned, so the sorting process can be implemented locally to reduce the overhead. Specifically, in the present invention, the secret shared gradients are securely discretized into B buckets and stored in the secret shared vectors [[α k,d ]] and [[β k,d ]] respectively. There are B-1 intervals in the B buckets, and each interval corresponds to a candidate test. In this way, the present invention only needs to select the best test from the B-1 candidate tests for each feature, instead of enumerating N samples, thereby saving a large amount of computation and communication cost. In the present invention, a secret shared vector [[θ]] = [[0 B-1 ]] is initialized for each feature to store the scores of the B-1 candidate tests.

[0081] It can be understood that when the optimal test corresponding to the second target layer needs to be learned, if discretization learning is adopted, the second target participating terminal needs to sort the second target feature data held, and then sort the corresponding gradient vector, but the gradient vector is held by all participating terminals in a secret sharing manner, so other participating terminals also need to reorder the secret sharing shares of the gradient vector held this time. However, without disclosing π to other participating terminals, it is difficult for participating terminal P l to permute the secret sharing gradient vector by holding the permutation π. In order to solve this problem, the method provided by the present application adopts a secure permutation method, referred to as SecPerm, as shown in the following table. Figure 6 At the beginning of SecPerm, each participating terminal P m∈[1,n] holds a secret vector [[x]], P l holds a permutation π. At the end of SecPerm, participating terminal P m∈[1,n] holds a secret vector [[u]], where u = π(x). SecPerm ensures that, except for P l , other participating terminals cannot know the permutation π. The initialization in SecPerm is completed offline by a third party (referred to as a coordinator terminal). Specifically, the coordinator terminal first distributes the secret shares of π p (r) and r to all participants, and then sends π p to P l . After initialization, all participants cooperate to permute [[x]] in 2 rounds. In the first round, P l generates a permutation π s that satisfies π(·) = π s [π p (·)]. Then P l sends π s to all other participating terminals. After that, each participating terminal P m∈[1,n] locally computes <x> m - <y> m In the second round, x-r is revealed to P l Finally, the participant outputs [[u]] = [[π(x)]]. The correctness analysis of SecPerm is as follows:

[0082] u= 1+ 2+…+<u} l +…+ n

[0083] =π s (<π p (r)>1)+π s (<π p (r)>2)+…+π(x-r)+π s (<π p (r)>1)+…+π s (<π p (r)> n )

[0084] =π(x-r)+π s (π p (r))

[0085] =π(x-r)+π(r)

[0086] =π(x)

[0087] As Figure 7 shown, in the secure discretization process provided by the application, first, the SecPerm is used to perform secure permutation based on the permutation π held by the second target participant terminal, and then the gradient is divided into B buckets. At the beginning, the encrypted first-order gradient vector and second-order gradient vector [[g]] and [[h]] are securely permuted by SecPerm. Based on SecPerm, the second target participant terminal updates the second additive secret sharing shares and the third additive secret sharing shares corresponding to each node in the second target layer held locally based on the first sorting result and the remaining participant terminals, including:

[0088] The second additive secret sharing shares and the third additive secret sharing shares are taken as target sharing shares, and the target sharing shares are updated based on the following steps:

[0089] The auxiliary terminal generates a random vector and a random sorting rule, sorts the values in the random vector based on the random sorting rule to obtain an auxiliary sorting vector, sends the n fifth secret sharing shares and the n sixth secret sharing shares to each participant terminal, and sends the auxiliary sorting vector to the second target participant terminal. The fifth secret sharing share is an additive secret sharing share of the random vector, and the sixth secret sharing share is an additive secret sharing share of the auxiliary sorting vector;

[0090] The participant terminal locally calculates a first difference vector of the target sharing share and the fifth secret sharing share held locally, and the second target participant terminal receives the first difference vector sent by the remaining participant terminals, and locally obtains a second difference vector. The second difference vector is a difference vector between the plaintext vector corresponding to the target sharing share and the random vector.

[0091] The second target participant terminal generates an auxiliary sorting rule, and sends the auxiliary sorting rule to the remaining participant terminals, wherein the auxiliary sorting rule satisfies a condition that a random vector is obtained by reordering the auxiliary sorting vector according to the auxiliary sorting rule;

[0092] The participant terminals other than the second target participant terminal reorder the locally held sixth secret sharing shares according to the auxiliary sorting rule, and take the reordered sixth secret sharing shares as updated target sharing shares;

[0093] The second target participant terminal sorts the second difference value vector according to the sorting rule corresponding to the first sorting result, and takes the sorted second difference value vector as an updated target sharing share.

[0094] After that, for b∈[0,B-1], the encrypted group first and second order gradients of the bth bucket are calculated as follows:

[0095]

[0096] where M=N / B is the number of gradients in a bucket. For simplicity, it is assumed that N is divisible by B. Since the invalid gradients are set to [[0]], the sum of the encrypted gradients in a bucket is equal to the sum of the gradients in the plaintext.

[0097] The second target participant terminal determines the optimal test corresponding to the second target layer based on the secret sharing shares of the evaluation scores, including:

[0098] All participant terminals determine secret sharing shares of comparison results between the evaluation scores based on the secret sharing shares of the evaluation scores;

[0099] The second target participant terminal determines the serial number of the optimal test based on the secret sharing shares of the comparison results between the evaluation scores, and the second target participant terminal finds the first gradient value and the second gradient value in the first sorting result based on the serial number of the optimal test, and determines the threshold corresponding to the optimal test based on the first gradient value and the second gradient value, to determine the optimal test.

[0100] After safely dispersing the gradients into B buckets, B-1 candidate tests are evaluated to select the best test for the jth feature. The evaluation score corresponding to the cth candidate test is determined by the impurity calculated based on the gradient values in the first c buckets and the impurity calculated based on the gradient values in the last B-c buckets, c is a positive integer, and B is the number of buckets. Specifically, for the cth candidate test (c∈[1,B-1]), the first c buckets are aggregated to obtain [[G l ]] and [[H l ]] which is the total sum of gradients related to the left child node, and the remaining B-c buckets are aggregated to obtain [[G r ]] and [[H r ]] which is the sum of the gradients associated with the right child node. The impurities of the two children of each node are securely computed according to Equation 6, and then these impurities are aggregated together to produce the encrypted score of the cth candidate test according to Equation 5. Then, for the jth feature, there will be B-1 encrypted scores stored in [[0]].

[0101] There is a division in Equation 6, which is not naturally supported in the additive secret sharing domain. In the method provided by the present application, a secure division component Sdiv is proposed to transform the division computation into a numerical optimization problem. Specifically, given [[x]] and [[y]], the computation of The core obstacle is to compute the encrypted reciprocal of Therefore, first, the Newton-Raphson algorithm is used to approximate , following the previous work: The result will eventually converge to In the present application, the initial value is fixed as where Y is a large enough value. It should be noted that the approximate calculation only includes basic subtraction and multiplication operations, which are naturally supported in the secret sharing domain. Therefore, given [[y]], the encrypted reciprocal of can be securely computed. After securely computing the reciprocal , multiply by [[x]] to get

[0102] The second target participant terminal determines the optimal test corresponding to the second target layer based on the secret sharing shares of the evaluation scores, including:

[0103] All participant terminals determine the secret sharing shares of the comparison results between the evaluation scores based on the secret sharing shares of the evaluation scores;

[0104] The second target participant terminal determines the serial number of the optimal test based on the secret sharing shares of the comparison results between the evaluation scores, and the second target participant terminal finds the first gradient value and the second gradient value in the first sorting result based on the serial number of the optimal test, and determines the threshold value corresponding to the optimal test based on the first gradient value and the second gradient value to determine the optimal test.

[0105] After obtaining the scores of B-1, the best test with the smallest score needs to be selected, which requires a method to safely calculate the index of the minimum value in the secret sharing vector. To address this challenge, the present application introduces a component SecArgmin. Note that selecting the minimum value from a secret sharing vector requires a secure comparison operation, but it is not naturally supported in the secret sharing field. Therefore, given two secret sharing values [[A]] and [[B]], [[A-B]] is first decomposed into bits, and then these bits are input into a parallel prefix adder (PPA) to securely compute the encrypted most significant bit (MSB) of [[A-B]]. After that, the encrypted MSB is converted into an arithmetic secret sharing value, thereby obtaining the encrypted result of the secure comparison. That is, all participating terminals determine the secret sharing shares of the comparison results between the evaluation scores based on the secret sharing shares of the evaluation scores, including:

[0106] All participating terminals determine the comparison result of the first evaluation score and the second evaluation score based on the following steps:

[0107] All participating terminals obtain the additive secret sharing shares of the score difference based on the locally held additive secret sharing shares of the first evaluation score and the second evaluation score, the score difference being the difference between the first evaluation score and the second evaluation score;

[0108] All participating terminals convert the locally held additive secret sharing shares of the score difference into bit values, and obtain the additive secret sharing shares of the most significant bit of the score difference as the secret sharing shares of the comparison result of the first evaluation score and the second evaluation score through a prefix adder.

[0109] Based on the above-mentioned secure comparison method, SecArgmin inputs the encrypted vector [[θ]], and then outputs the bucket ID corresponding to the best test of the jth feature in the plaintext, which can also be referred to as the bucket number, bucket number q ∈ [0, B-2]. In the present application, all participating terminals can learn the generated bucket ID in the training stage, but only the participating terminal that owns the jth feature can obtain the threshold value of the best test of the jth feature. Since the value of the jth feature (denoted as ) is sorted in ascending order, the participating terminal that owns the jth feature can obtain the split threshold value by looking up the sorted value with index (q+1) × (N / B) (i.e., π j [D(j)]).

[0110] As shown in Figure 8 , in combination with the foregoing description, the training process of the decision table in the method provided by the present application can be summarized as: P m∈[1,n] Calculate the first and second order gradient vectors of the secret sharing (for the root node) (i.e.) Figure 8 The [g] and [h]] in lines 1-6. After calculating the gradient of the secret sharing, we initialize a secret vector. To store 2 D A secret shared output value. Then, a decision table is built layer by layer. (and...) Figure 1 Similar to the function of find_split, this invention safely selects the optimal test F at the d-th layer (d∈[0,D-1]). d <t d Specifically, the first step is to select the best test for each feature (i.e., ...). Figure 8 (Lines 10-25), then select the best test from the selected J candidate tests (i.e. Figure 8 (Lines 26-28). It's worth noting that the selection here is based on the plaintext field, but the computation in the selection is replaced by components of secure computation and design. For J candidate tests, initialize a secret-shared vector [[σ]] = [[0] J This is used to store the score of the best test for each feature. Furthermore, this invention also uses a publicly available vector γ = 0. J To record the bucket ID of the best test for each feature. To select the best test for all features, this invention allows participating terminals to record the best bucket number q and the encrypted minimum split score [θ] for the j-th feature at the j-th position of γ and [σ], respectively. q ]](Right now Figure 8 (Lines 23-24). To recap, for J features, γ and [[σ]] are used to store the bucket ID and split score of the best test for each feature. The J indices of γ and [[σ]] correspond to the J features respectively. After enumerating the J features, the J scores are stored in [[σ]], and SecArgmin needs to be called again on [[σ]]. The output is the split feature F of the best test. d All participating terminals know this. Then, F can be used. d Retrieve the optimal bucket ID (q) from γ to find the splitting feature. d ()( Figure 8 (Line 27). Afterwards, F possesses the splitting characteristic. d The participating terminals search for their sorting values. The index is (q) d +1)×(N / B) yields the splitting threshold t d After learning the d-th test F d <t d After that, possessing F d <t d The participating terminal collaborates with other participating terminals to securely split all nodes of layer d using SecSplit to create a new layer. Figure 8 (Line 29). The decision table in this invention is learned layer by layer in this manner. At level D, the output value of the leaf node is safely calculated according to Formula 3 (i.e., ... Figure 8 (Lines 31-34 in the table), where the split is computed securely using SDiv. Finally, SecTable outputs a result consisting of D tests and 2... D A distributed decision table consisting of secretly shared output values. Specifically, all participating terminals know the splitting feature F at level d. d Where d∈[0,D-1], but each split threshold t d Only for those with feature F d The participating terminals are visible.

[0111] like Figure 3 As shown, the method provided by the present invention further includes the following steps:

[0112] S500: After the optimal test for each layer of the target decision table is determined, the current learning round is completed. Based on the learned decision tables, each participating terminal obtains the fourth additive secret share of the prediction results of each learning sample, and based on the fourth additive secret share, obtains the second and third additive secret shares corresponding to the first node in the learned decision table in the next learning round, until the number of learned decision tables reaches a preset number, thus completing the learning process.

[0113] In this invention, each decision table in the set learned during the secure training phase is held in a distributed manner by participating terminals, with each terminal holding a portion of it. Under this setup, it is necessary to consider how to ensure that participating terminals can securely use the model for prediction. Simultaneously, the prediction data is also vertically split among the participating terminals in the vertical federated learning process, and therefore also needs to be protected. To prevent the leakage of model information and prediction data from each participating terminal, this invention proposes a secure prediction algorithm, SecInfer, as follows... Figure 9 As shown, SecInfer allows participating terminals to securely predict their local data using a distributed gradient boosting decision table set and generate secretly shared prediction results, while keeping the local data unavailable to other participating terminals throughout the prediction process.

[0114] The fourth additive secret share of the prediction results of each learning sample obtained by each participating terminal includes:

[0115] The participating terminal holding the feature data corresponding to the split layer of the target decision table generates the leaf index vector of the target learning sample in the split layer. The split layer is the layer with child nodes in the target decision table. The leaf index vector is used to represent the first or second child node of the node in the split layer to which the target learning sample is assigned. The number of elements in the leaf index vector is equal to the number of nodes in the layer with the most nodes in the learned decision table.

[0116] The participating terminal holding the feature data corresponding to the split layer encrypts the leaf index vector and sends the additive secret share of the leaf index vector to the other participating terminals.

[0117] All participating terminals obtain the additive secret share of the product of the leaf index vectors corresponding to the target learning sample in each split layer based on the additive secret share held locally, and use it as the secret share of the prediction result score corresponding to the target decision table.

[0118] All participating terminals aggregate the prediction scores corresponding to all learned decision tables based on additive secret sharing to obtain the fourth additive secret sharing share corresponding to the target learning sample.

[0119] The participating terminal holding the feature data corresponding to the split layer generates the leaf index vector of the target learning sample corresponding to the split layer, including:

[0120] Based on the feature data corresponding to the split layer held locally, the participating terminal determines whether the target learning sample is assigned to the first or second child node of the node in the split layer.

[0121] When the target learning sample is assigned to the first child node of a node in the split layer, the first element in each element group of the leaf index vector corresponding to the split layer is 1 and the second element is 0.

[0122] When the target learning sample is assigned to the second child node of a node in the split layer, the first element in each element group of the leaf index vector corresponding to the split layer is 0 and the second element is 1.

[0123] Among them, the leaf index vector corresponding to the split layer includes 2 d There are 10 element groups, each containing two elements: d is the layer number of the split layer in the target decision table, d∈[0,D-1], and D is the layer number of the target decision table.

[0124] Specifically, in order to safely partition the samples vertically... Make a prediction, having a test F at the d-th layer (d∈[0,D-1]). d <t d Participating terminal P l The eigenvalues of the F d ) of the sample are compared with the split threshold t d A leaf indicator vector (denoted as u d ) is generated locally. Then, the leaf indicator vector u d is secret shared to other participating terminals. Then, the prediction result can be securely computed by the secure element multiplication between the D encrypted leaf indicator vectors and the encrypted vector of the output values of the decision table [[w]].

[0125] It is noted that SecInfer can also be used in the secure training phase. That is, after the distributed decision table is generated in a round of secure training, SecInfer can be used to generate the encrypted prediction result for each training sample. The prediction result of this round will be aggregated into the previous prediction result to establish the gradient for the next round. In the whole secure training process, the aggregated prediction result of the decision table in the set is saved in the secret sharing domain. In the secure prediction process, the final aggregated prediction result of all T decision tables is disclosed to all participating terminals.

[0126] In the learning process, after obtaining the prediction result, the first-order gradient and the second-order gradient required in the next round of decision table learning are calculated based on the prediction result. In the process of calculating the first-order gradient and the second-order gradient, the sigmoid function may be used, which includes division and exponential operation. The secure division operation has been described in the foregoing. For the calculation of the exponential of a given secret sharing value [[x]] [[e x ]], the present application approximates ​It is noted that since the approximate calculation includes basic addition and multiplication operations, which are naturally supported in the secret sharing domain, given [[x]], the encrypted exponent [[e x ]] can be securely calculated. However, the approximate method requires 2 n times of chain multiplication, and thus requires 2 n rounds of online communication. The approximate method is inefficient in practice because the communication complexity increases exponentially. Therefore, the present application further reduces the exponential communication complexity to linear communication. More specifically, it is noted that the calculation in the approximation can be regarded as where Therefore, given [[a]], the present application first securely calculates [[a 2 ]] which only requires one round of communication. After that, the present application regards the output as [[y]] = [[a n ]] and then securely calculates [[y 2 ]] which also only requires one round of communication. Therefore, in this way, [[y n ]] can be securely calculated in log2 n rounds instead of 2

[0127] In summary, the privacy protection vertical federated learning method based on gradient boosting decision table provided by the present application maintains a corresponding gradient vector for each node in the decision table, each participating terminal holds an additive secret sharing share of the gradient vector, and a participating terminal holding complete feature data sends an indication vector to each participating terminal by generating an indication vector and sending an additive secret sharing share of the indication vector to each participating terminal, and each participating terminal updates the secret sharing share of the gradient vector of the next layer node through the indication vector, so that the value corresponding to the sample divided to the sub-node in the gradient vector corresponding to the sub-node of the node is 0, and the value corresponding to the sample divided to the sub-node is 1, which realizes that the number of samples processed by each node remains unchanged, and all participating terminals except the terminal holding the complete feature data cannot infer any information from the node splitting, including the optimal test threshold of the complete feature data and which sample feature data is less than or greater than the threshold, further, in the method provided by the present application, each participating terminal also determines the additive secret sharing share of the best test threshold of each layer in the decision table and the additive secret sharing share of the prediction result of the learning sample based on additive secret sharing, for learning of the next round of decision table, in the whole process of learning, all intermediate data are encrypted, and each participating terminal can only obtain the additive secret sharing share of the data and cannot obtain the complete plaintext data, therefore, the present application can realize privacy protection of the privacy protection vertical federated learning data based on gradient boosting decision table.

[0128] The gradient boosting decision table based privacy protection vertical federated learning system provided by the present application is described below, and the gradient boosting decision table based privacy protection vertical federated learning system described below can be correspondingly referred to the gradient boosting decision table based privacy protection vertical federated learning method described above.

[0129] The gradient boosting decision table based privacy protection vertical federated learning system provided by the present application includes a plurality of participating terminals, and the plurality of participating terminals cooperatively complete the gradient boosting decision table based privacy protection vertical federated learning method as described above.

[0130] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course, can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product in essence or in the form of a part shared with the prior art. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method described in each embodiment or some part of the embodiment.

[0131] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. < / y> < / x> < / x> < / e> < / y> < / x> < / z> < / x> < / x> < / x> < / x> < / x>

Claims

1. A privacy-preserving vertical federated learning method based on gradient boosting decision table, characterized in that, The method is applied to a system comprising a plurality of participating terminals, each of which holds partial feature data of each learning sample, and the feature data held by each of the participating terminals is different in category; the method comprises: a first target participating terminal in each of the participating terminals determines an indication vector corresponding to a first target layer based on first target feature data of each of the learning samples held locally and an optimal test corresponding to the first target layer in a target decision table, wherein the target decision table is a decision table to be learned in a current learning round, the target feature data corresponds to the first target layer, the first target participating terminal holds the first target feature data corresponding to the first target layer, and the indication vector is used to reflect that each node in the first target layer corresponds to a sample being divided into a first sub-node or a second sub-node of the node, and the optimal test corresponding to each layer in the target decision table is used to determine the division result of each sample in the layer; the first target participating terminal generates n first additive secret sharing shares corresponding to the indication vector, and sends n-1 first additive secret sharing shares to the remaining participating terminals in the system respectively, wherein n is the number of the participating terminals; each of the participating terminals updates a second additive secret sharing share and a third additive secret sharing share corresponding to each node in a second target layer held locally based on each of the first additive secret sharing shares held locally, wherein the second target layer is a layer in which a sub-node of the first target layer in the target decision table is located, the second additive secret sharing share is an additive secret sharing share of a first-order gradient vector, and the third additive secret sharing share is an additive secret sharing share of a second-order gradient vector, when an i-th learning sample is divided into a target node, an i-th value in the first-order gradient vector corresponding to the target node is a current first-order gradient value of the i-th learning sample, and the remaining values are 0, and an i-th value in the second-order gradient vector corresponding to the target node is a current second-order gradient value of the i-th learning sample, and the remaining values are 0; a second target participating terminal in each of the participating terminals calculates an evaluation score corresponding to each candidate test based on the second additive secret sharing share and the third additive secret sharing share held locally in a secret sharing domain, and determines an optimal test corresponding to the second target layer based on a secret sharing share of the evaluation score, wherein the second target participating terminal holds second target feature data corresponding to the second target layer. After the optimal test of each layer of the target decision table is determined, the current learning round is completed, each of the participating terminals obtains a fourth additive secret sharing share of a prediction result of each of the learning samples based on the learned decision table, and obtains the second additive secret sharing share and the third additive secret sharing share corresponding to a first node in a decision table learned in a next learning round based on the fourth additive secret sharing share until the number of the learned decision tables reaches a preset number, so as to complete the learning process.

2. The privacy-preserving vertical federated learning method based on gradient boosting decision table according to claim 1, characterized in that, The indication vector includes a first indication vector and a second indication vector, a value in the first indication vector is 1 when the ith learning sample is divided into the first sub-node of the belonging node, otherwise 0, a value in the second indication vector is 1 when the ith learning sample is divided into the second sub-node of the belonging node, otherwise 0; each of the participating terminals updates the second additive secret sharing share and the third additive secret sharing share corresponding to each node in the second target layer held locally based on each of the first additive secret sharing shares held locally, including: When the target node in the second target layer is the first sub-node, each of the participating terminals calculates a secret sharing share of a product of the first indication vector and the first-order gradient vector corresponding to the target node based on an additive secret sharing protocol as the second additive secret sharing share corresponding to the target node after the update, and calculates a secret sharing share of a product of the first indication vector and the second-order gradient vector corresponding to the target node as the third additive secret sharing share corresponding to the target node after the update; When the target node in the second target layer is the second sub-node, each of the participating terminals calculates a secret sharing share of a product of the second indication vector and the first-order gradient vector corresponding to the target node based on an additive secret sharing protocol as the second additive secret sharing share corresponding to the target node after the update, and calculates a secret sharing share of a product of the second indication vector and the second-order gradient vector corresponding to the target node as the third additive secret sharing share corresponding to the target node after the update.

3. The privacy-preserving vertical federated learning method based on gradient boosting decision table according to claim 1, characterized in that, A second target participating terminal in each of the participating terminals calculates an evaluation score corresponding to each candidate test in a secret sharing domain based on the second additive secret sharing share and the third additive secret sharing share held locally, including: The second target participating terminal sorts the second target feature data of each of the learning samples held locally to obtain a first sorting result, wherein the second target participating terminal holds the second target feature data corresponding to the second target layer. The second target participant terminal updates the second additive secret share and the third additive secret share corresponding to each node in the second target layer based on the first sorting result and the rest of the participant terminals, to reorder the gradient values in the first-order gradient vector and the second-order gradient vector corresponding to each node in the second target layer, so that the sorting result of the gradient values in the first-order gradient vector and the second-order gradient vector is consistent with the first sorting result. Each participant terminal performs bucketing on each gradient value in the first-order gradient vector and the second-order gradient vector based on a preset number of buckets, to obtain secret share of each bucket, each bucket including a plurality of first-order gradient values and second-order gradient values arranged in order. Each participant terminal determines the secret share of the evaluation score corresponding to each candidate test based on the secret share of each bucket held by the participant terminal, wherein the threshold value of each candidate test is one of the first gradient value and the second gradient value corresponding to the second target feature data, or a value between the first gradient value and the second gradient value corresponding to the second target feature data, the first gradient value being the last first-order gradient value or second-order gradient value in the first bucket, the second gradient value being the first first-order gradient value or second-order gradient value in the second bucket, the first bucket being the first of the two adjacent buckets, and the second bucket being the second of the two adjacent buckets.

4. The privacy-preserving vertical federated learning method based on gradient boosting decision table according to claim 3, characterized in that, The second target participant terminal updates the second additive secret share and the third additive secret share corresponding to each node in the second target layer based on the first sorting result and the rest of the participant terminals, including: The second additive secret share and the third additive secret share are taken as target shares respectively, and the target shares are updated based on the following steps: The auxiliary terminal generates a random vector and a random sorting rule, sorts the values in the random vector based on the random sorting rule to obtain an auxiliary sorting vector, sends n fifth secret shares and n sixth secret shares to each participant terminal respectively, and sends the auxiliary sorting vector to the second target participant terminal, the fifth secret share being an additive secret share of the random vector, and the sixth secret share being an additive secret share of the auxiliary sorting vector. The participant terminal locally calculates a first difference vector of the target share and the fifth secret share held locally, the second target participant terminal receives the first difference vector sent by the rest of the participant terminals, and locally obtains a second difference vector, the second difference vector being a difference vector between the plaintext vector corresponding to the target share and the random vector. The second target participant terminal generates an auxiliary sorting rule, and sends the auxiliary sorting rule to the remaining participant terminals, wherein the auxiliary sorting rule satisfies a condition that the random vector is obtained by reordering the auxiliary sorting vector according to the auxiliary sorting rule. The participant terminals other than the second target participant terminal reorder the sixth secret share locally held by the participant terminals according to the auxiliary sorting rule, and take the reordered sixth secret share as the updated target share. The second target participant terminal sorts the second difference vector according to the sorting rule corresponding to the first sorting result, and takes the sorted second difference vector as the updated target share.

5. The privacy-preserving vertical federated learning method based on gradient boosting decision table according to claim 3, characterized in that, The evaluation score corresponding to the cth candidate test is determined by impurity calculated based on total gradient values in the first c buckets and impurity calculated based on total gradient values in the last B-c buckets, c is a positive integer, and B is the number of buckets.

6. The privacy-preserving vertical federated learning method based on gradient boosting decision table according to claim 3, characterized in that, The second target participant terminal determines the optimal test corresponding to the second target layer based on the secret share of the evaluation score. All the participant terminals determine the secret share of the comparison result between each evaluation score based on the secret share of the evaluation score. The second target participant terminal determines the serial number of the optimal test based on the secret share of the comparison result between the evaluation scores, and the second target participant terminal finds the first gradient value and the second gradient value in the first sorting result based on the serial number of the optimal test, and determines the threshold value corresponding to the optimal test based on the first gradient value and the second gradient value, to determine the optimal test.

7. The privacy-preserving vertical federated learning method based on gradient boosting decision table according to claim 6, characterized in that, All the participant terminals determine the secret share of the comparison result between each evaluation score based on the secret share of the evaluation score. All the participant terminals determine the comparison result between the first evaluation score and the second evaluation score based on the following steps: All the participant terminals obtain the additive secret share of the score difference based on the additive secret shares of the first evaluation score and the second evaluation score locally held by the participant terminals, wherein the score difference is the difference between the first evaluation score and the second evaluation score. All the participant terminals convert the additive secret share of the score difference locally held by the participant terminals into a bit value, and obtain the highest bit of the additive secret share of the score difference as the secret share of the comparison result between the first evaluation score and the second evaluation score through a prefix adder.

8. The privacy-preserving vertical federated learning method based on gradient boosting decision table according to claim 1, characterized in that, The participant terminals obtain the fourth additive secret share of the prediction result of each learning sample, including: The participating terminal holding the feature data corresponding to the split layer of the target decision table generates a leaf index vector corresponding to the split layer of the target learning sample, wherein the split layer is a layer in the target decision table having a child node, the leaf index vector is used to indicate that the target learning sample is divided into a first child node or a second child node of a node in the split layer, and the number of elements in the leaf index vector is equal to the number of nodes in a layer in the learned decision table having the most nodes; The participating terminal holding the feature data corresponding to the split layer encrypts the leaf index vector and sends an additive secret sharing share of the leaf index vector to the remaining participating terminals; All the participating terminals obtain an additive secret sharing share of a product result of the leaf index vector corresponding to each split layer of the target learning sample based on the locally held additive secret sharing share, as a secret sharing share of a prediction result score corresponding to the target decision table; All the participating terminals aggregate all the prediction result scores corresponding to the learned decision tables based on additive secret sharing to obtain the fourth additive secret sharing share corresponding to the target learning sample.

9. The privacy-preserving vertical federated learning method based on gradient boosting decision table according to claim 1, characterized in that, The participating terminal holding the feature data corresponding to the split layer of the target decision table generates a leaf index vector corresponding to the split layer of the target learning sample, comprising: The participating terminal holding the feature data corresponding to the split layer determines, based on the locally held optimal test of the split layer and the feature data corresponding to the target learning sample, that the target learning sample is divided into a first child node or a second child node of a node in the split layer; When the target learning sample is divided into the first child node of the node in the split layer, a first element in each element group in the leaf index vector corresponding to the split layer is 1 and a second element is 0; When the target learning sample is divided into the second child node of the node in the split layer, the first element in each element group in the leaf index vector corresponding to the split layer is 0 and the second element is 1; Wherein, the split layer corresponds to the leaf index vector including 2 d element groups, each element group including two elements, d is the layer number of the split layer in the target decision table, d∈[0,D-1], and D is the number of layers of the target decision table.

10. A privacy-preserving vertical federated learning system based on gradient boosting decision table, characterized in that, The system comprises a plurality of participating terminals, and the plurality of participating terminals cooperatively complete the privacy protection vertical federated learning method based on the gradient boosting decision table according to any one of claims 1-9.

Images