Longitudinal federal lifting tree method for privacy protection

Abstract

The invention provides a longitudinal federated lifting tree method for privacy protection, and relates to the technical field of longitudinal federated learning. The method comprises the following steps: combining a secretly shared binary matrix and gradient information to obtain a shared matrix and secretly sharing the shared matrix; each training terminal performs row permutation on the sharing matrix based on the permutation triad and secretly shares a permutation result; the linear gain scores of the candidate split are calculated cooperatively and shared secretly, and the indication vector of the split corresponding to the maximum linear gain score is calculated; according to the candidate split number, selecting corresponding columns and indication vectors from the matrix to calculate sub-node samples and a permutation matrix of sub-nodes; and recursively executing a calculation splitting step, completing splitting from the root node to the leaf node, calculating the weight of the leaf node, and storing the weight into a preset set. Updating the binary permutation matrix of the root node of the next decision model based on the leaf node weight, and continuing calculation until all decision tree models are generated; the problem that an existing longitudinal federated gradient boosting tree protocol is low in efficiency can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of vertical federated learning, and in particular, to a privacy-preserving vertical federated boosting tree method. Background Art

[0002] The gradient boosting tree algorithm is a commonly used machine learning algorithm that sequentially constructs multiple decision trees and optimizes subsequent decision trees based on the errors of previous decision trees. It has achieved state-of-the-art performance in various machine learning tasks and has been dominant in applied machine learning competitions as well as real-world applications. With the increase in the number of data sources, the need for collaboration across multiple data owners has been growing to build more powerful gradient boosting tree models. However, due to reasons such as laws and regulations, privacy policies, or business competition, these data owners may be reluctant to share their data with each other. In addition, existing research has shown that transmitting intermediate results during model training still poses a risk of privacy leakage. Therefore, there is a need to develop a multi-party collaborative gradient boosting tree training method that meets privacy protection constraints, which allows these data owners to collaboratively train a gradient boosting tree model without disclosing input data and intermediate results.

[0003] Currently, existing privacy-preserving vertical federated gradient boosting tree protocols typically use secret sharing technology, which can complete collaborative computations of various basic operations on the premise of provable security.

[0004] However, in order to ensure the hiding of intermediate results, existing protocols often adopt oblivious data computation, making the computational and communication complexities too high. At the same time, a large number of non-linear operations are used during training, and their computations under secret sharing incur a large overhead, resulting in existing protocols being often limited in practical applications due to efficiency issues. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a privacy-preserving vertical federated boosting tree method to eliminate or improve one or more defects existing in the prior art. It can solve the problem of low efficiency of existing privacy-preserving vertical federated gradient boosting tree protocols.

[0006] One aspect of the present invention provides a privacy-preserving vertical federated boosting tree method, which is applied to a vertical federated learning training system constructed by at least two training terminals. The training terminals include an initiating terminal and participating terminals, and are used to jointly construct a preset number of decision tree models. The method includes the following steps:

[0007] The initiating terminal combines the local binary matrix, gradient information, and the binary matrix participating in the secret sharing of the terminal to obtain the sharing matrix of the root node of the first decision tree model and secretly shares it with the participating terminals; the binary matrix is obtained by converting the left subset of the candidate split of the sample features; the gradient vector is calculated based on the predicted value of the sample features, the true label, and a preset loss function;

[0008] Each training terminal secretly performs a row permutation on the sharing matrix based on the pre-shared permutation triple, obtains the permutation result and secretly shares and splices it to obtain the binary permutation matrix of the root node;

[0009] Taking the root node as the current node, based on the binary permutation matrix and the secretly shared addition and multiplication operators, collaboratively calculate the linear gain score array of the candidate split of the current node and secretly share it;

[0010] Through the secretly shared comparison operator, collaboratively calculate the indication vector of the best split corresponding to the maximum linear gain score in the linear gain score array and store it in the preset secret sharing set;

[0011] According to the number of candidate splits, select the corresponding columns from the binary permutation matrix and calculate with the indication vector, collaboratively calculate the sample of the child node of the current node, and calculate the binary permutation matrix of the child node based on the sample of the child node and the binary permutation matrix;

[0012] Taking the child node as the current node, execute the step of calculating the linear gain score array of the candidate split of the current node until the split from the root node to the leaf node is completed, and calculate the leaf node weight and store it in the preset secret sharing set;

[0013] When the number of decision tree models does not reach the preset number, based on the leaf node weight and the predicted value, update the binary permutation matrix of the root node to obtain the binary permutation matrix of the root node of the next decision tree model, take the root node of the next decision tree model as the current node, and execute the step of calculating the linear gain score array of the candidate split of the current node until the number of decision tree models reaches the preset number, and output the preset secret sharing set.

[0014] In some embodiments of the present invention, the number of candidate splits is M; based on the binary permutation matrix and the secretly shared addition and multiplication operators, collaboratively calculate the linear gain score array of the candidate split of the current node and secretly share it, including:

[0015] For the m-th candidate split among the M candidate splits, each training terminal collaboratively calculates the secretly shared second-order reciprocal sum of the current node based on the last column from the bottom in the binary permutation matrix and the addition operator; m is an integer that takes values from 1 to M in sequence;

[0016] Based on the m-th column in the binary permutation matrix, the penultimate column in the binary permutation matrix, and the multiplication operator, collaboratively calculate the sum of the first-order derivatives of the secret sharing of the left child node obtained from the m-th candidate split partition, and the sum of the first-order derivatives of the secret sharing of the right child node;

[0017] Based on the m-th column, the last column, and the multiplication operator, collaboratively calculate the sum of the second-order derivatives of the secret sharing of the left child node obtained from the m-th candidate split partition, and the sum of the second-order derivatives of the secret sharing of the right child node;

[0018] Based on the sum of the second-order reciprocals of the secret sharing of the current node, the sum of the first-order derivatives of the secret sharing of the left child node, the sum of the first-order derivatives of the secret sharing of the right child node, the sum of the second-order derivatives of the secret sharing of the left child node, the sum of the second-order derivatives of the secret sharing of the right child node, and the linear gain fraction algorithm, collaboratively calculate the linear gain fraction of the m-th candidate split;

[0019] After traversing M candidate splits, obtain the linear gain fraction array;

[0020] The linear gain fraction algorithm is represented by the following formula:

[0021] <S * > A =(<G m,L > A ) 2 ·(2 <h> A -<H m,L > A ) 2 +(<G m,R > A ) 2

[0022] ·(2 <h> A -<H m,R > A ) 2

[0023] In the formula, <G m,L > A represents the sum of the first-order derivatives of the secret shares of the left child node; <h> A Denote the second-order derivative sum of the secret sharing of the current node; <H m,L > A Denote the second-order derivative sum of the secret sharing of the left child node; <G m,R > A Denote the first-order derivative sum of the secret sharing of the right child node; <H m,R > A Denote the second-order derivative sum of the secret sharing of the right child node.

[0024] In some embodiments of the present invention, the permutation triple is represented as (<π> C , A , <π(U)> A ), where <π> C represents the permutation operation of secret sharing based on cycle decomposition. A Denotes a secret sharing matrix, and each training terminal holds a part of the secret sharing matrix; <π(U)> A Denotes the permutation matrix obtained after the secret sharing matrix undergoes a permutation operation;

[0025] Based on the pre-shared permutation triples, each training terminal respectively performs a secret row permutation on the sharing matrix, obtains the permutation result and secretly shares and splices it to obtain the binary permutation matrix of the root node, including:

[0026] Each training terminal, based on the secretly shared addition operator, collaboratively adds the sharing matrix and the secret sharing matrix to obtain the matrix after adding the mask;

[0027] After restoring the matrix after adding the mask to the plaintext form, perform a row permutation on the pre-allocated partial rows through a permutation operation, and set the other rows to 0 to obtain the local permutation result and secretly share it;

[0028] After splicing all the local permutation results, add them to the secret sharing matrix through the addition operator to obtain the binary permutation matrix.

[0029] In some embodiments of the present invention, before each training terminal respectively performs a secret row permutation on the sharing matrix based on the pre-shared permutation triples, obtains the permutation result and secretly shares and splices it to obtain the binary permutation matrix of the root node, it further includes:

[0030] The model initiator allocates the rows responsible for permutation to each training terminal;

[0031] Each training terminal generates a local permutation locally to obtain the permutation operation <π> C ;

[0032] Each training terminal collaboratively generates the secret sharing matrix A ;

[0033] Cyclically use the permutation operations corresponding to each training terminal on the secret sharing matrix A Perform row permutation. Use threshold fully homomorphic encryption to obtain the encrypted form of the secret sharing matrix. Each training terminal locally performs its own permutation operation on the encrypted form of the secret sharing matrix to perform row permutation on the encrypted form of the secret sharing matrix, and convert the permuted matrix into the secret sharing form; after completing the cyclic permutation, obtain the permutation matrix <π(U)> A .

[0034] In some embodiments of the present invention, the rows responsible for permutation are assigned to each training terminal by the model initiator, which means that the initiator terminal assigns row indices according to the number of each training terminal;

[0035] Each training terminal locally generates a local permutation to obtain the permutation operation <π> C , including: each training terminal randomly shuffles the order of the received row indices, and determines the mapping relationship from the original order to the new order as the permutation operation.

[0036] In some embodiments of the present invention, through the comparison operator of secret sharing, collaboratively calculate the indication vector corresponding to the best split of the maximum linear gain score in the linear gain score array, and store it in the preset secret sharing set, including:

[0037] Take the linear gain score array as the forward iteration score array of the current forward iteration round. When the number of linear gain scores in the forward iteration score array is even, evenly divide the forward iteration score array into a forward iteration left subset and a forward iteration right subset from the middle position;

[0038] Each training terminal compares the linear gain scores at the corresponding positions in the forward iteration left subset and the forward iteration right subset one by one according to the comparison operator, and stores the larger linear gain score in the pre-secretly shared gain score vector;

[0039] Based on the gain score vector, the forward iteration left subset and the forward iteration right subset, calculate a new score array as the forward iteration score array of the next forward iteration round. When the number of linear gain scores in the forward iteration score array is even, perform the step of evenly dividing the forward iteration score array into a forward iteration left subset and a forward iteration right subset from the middle position until the number of linear gain scores in the new score array is 1;

[0040] When the number of linear gain scores in the new score array is 1, take the new score array as the backward iteration score array, and perform element-wise multiplication with the gain score vector to obtain the left subset gain score;

[0041] Based on the backward iteration score array and the left subset gain score, calculate the right subset gain score;

[0042] Concatenate the left subset gain scores and the right subset gain scores to obtain the reverse iteration score array for the next reverse iteration round, and perform the step of performing an element-wise product with the gain score vector to obtain the left subset gain scores; until all iteration rounds are completed, obtain the indication vector; the number of forward iterations is the same as the number of reverse iterations.

[0043] In some embodiments of the present invention, in the case where the number of elements in the reverse iteration score array is greater than the number of elements in the gain score vector, the method further includes:

[0044] Strip the extra elements from the reverse iteration score array and splice them into the reverse iteration score array for the next reverse iteration round.

[0045] In some embodiments of the present invention, in the case where the number of linear gain scores in the forward iteration score array is even, the method further includes:

[0046] Take out the last item in the forward iteration score array and put it into a temporary variable;

[0047] Before the next forward iteration round is executed, take out the last item from the temporary variable and add it to the forward iteration score array for the next forward iteration round.

[0048] In some embodiments of the present invention, for any leaf node, calculating the leaf node weight includes:

[0049] Take out the last two columns from the binary permutation matrix corresponding to the leaf node;

[0050] Based on the last two columns, the sample corresponding to the leaf node, and a preset weight algorithm, calculate the weight corresponding to the leaf node; the preset weight algorithm is expressed as:

[0051]

[0052] In the formula, represents the sample of the leaf node; <w t,d > A The weight of the leaf node; represents the penultimate column; represents the last column.

[0053] In some embodiments of the present invention, before combining the local binary matrix, gradient information, and the binary matrix participating in the terminal secret sharing by the initiating terminal to obtain the sharing matrix of the root node of the first decision tree model and secretly sharing it with the participating terminals, it further includes:

[0054] Each training terminal determines the candidate splits of the sample features it holds, and divides its sample features into a left partition subset and a right partition subset based on each candidate split;

[0055] Convert the left-swipe subset to obtain a binary matrix and secretly share it with the model initiator;

[0056] The model initiator predicts the sample features based on a preset decision tree model to obtain prediction values;

[0057] Input the prediction values and sample features into a preset loss function, calculate the first-order gradient and second-order gradient of each sample feature, and combine the first-order gradient and second-order gradient into two gradient vectors to obtain gradient information.

[0058] The privacy-preserving vertical federated boosting tree method of the present invention can solve the problem of low efficiency of the existing privacy-preserving vertical federated gradient boosting tree protocol; before model training, anonymization of sample identifiers is achieved through random row permutation, that is, the training terminal cannot know which samples are actually included in the sample set of each node, thereby allowing breaking the "computational obliviousness" that must be satisfied by the existing protocol, significantly reducing the computational and communication complexity during training; at the same time, a linear gain score is provided, which, compared with the original gain score of the gradient boosting tree algorithm, does not introduce division operations, and the division operation is quite time-consuming to implement under secure multi-party computation, and can shorten the calculation time, thereby further improving the training efficiency.

[0059] In addition, during the process of optimal split selection, through forward iteration and backward iteration, the number of uses of computationally expensive comparison operations is reduced to the minimum, which can reduce the computational complexity and further improve the model training efficiency.

[0060] The additional advantages, objects, and features of the present invention will be partially described below, and will become partially apparent to those of ordinary skill in the art after studying the following text, or can be learned from the practice of the present invention. The objects and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in the specification and the drawings.

[0061] Those skilled in the art will understand that the objects and advantages that can be achieved by the present invention are not limited to the above specifically described, and the above and other objects that the present invention can achieve will be more clearly understood according to the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The drawings described herein are used to provide a further understanding of the present invention, form a part of this application, and do not limit the present invention. In the drawings:

[0063] Figure 1 It is a flowchart of the privacy-preserving vertical federated boosting tree method provided by an embodiment of the present invention.

[0064] Figure 2 Schematic diagram of the conversion of candidate splits and binary matrices provided by an embodiment of the present invention.

[0065] Figure 3 Schematic diagram of the secret row permutation of each training terminal provided by an embodiment of the present invention.

[0066] Figure 4 Schematic diagram of forward iteration and backward iteration provided by an embodiment of the present invention.

[0067] Figure 5 Comparison chart of the model accuracy provided by an embodiment of the present invention. Detailed implementation manners

[0068] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in combination with the implementation manners and the drawings. Herein, the illustrative implementation manners of the present invention and the descriptions thereof are used to explain the present invention, but do not limit the present invention.

[0069] Herein, it should also be noted that in order to avoid obscuring the present invention due to unnecessary details, only the structures and / or processing steps closely related to the solution of the present invention are shown in the drawings, while other details less related to the present invention are omitted.

[0070] It should be emphasized that the term "including / containing" when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0071] Herein, it should also be noted that if not otherwise specified, the term "connection" in this article can not only refer to a direct connection, but also represent an indirect connection with an intermediate.

[0072] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0073] The privacy-preserving vertical federated boosting tree method provided by the present application will be introduced in detail below.

[0074] In some embodiments of the present invention, the privacy-preserving vertical federated boosting tree method provided by the present application is applied to a vertical federated learning training system constructed by at least two training terminals, wherein the training terminals include an initiating terminal and participating terminals, and are used to jointly construct a preset number of decision tree models; the initiating terminal includes devices such as mobile phones, tablets, personal computers or servers, etc.; the device type of the initiating terminal is not limited in this embodiment; the number of participating terminals is at least one, including devices such as mobile phones, tablets, personal computers or servers, etc., and the device type and the number of devices of the participating terminals are not limited in this embodiment.

[0075] This embodiment provides a vertical federated boosting tree method for privacy protection, as Figure 1 shown, the method at least includes steps S101 to S105:

[0076] Step S101, combine the local binary matrix, gradient information, and the binary matrix for secret sharing among participating terminals by the initiating terminal to obtain the sharing matrix of the root node of the first decision tree model and secretly share it with the participating terminals.

[0077] Among them, the binary matrix is obtained by converting the left split subset of the candidate split of the sample features; the gradient vector is calculated based on the predicted value of the sample features, the true label, and a preset loss function.

[0078] In some embodiments of the present invention, each training terminal (including the initiating terminal and the participating terminals) holds its own sample features and generates candidate splits for the sample features it holds. According to each candidate split, the sample features are divided into a left split subset and a right split subset.

[0079] For example: Referring to Figure 2 , taking the sample data including age, salary, and deposit as an example; for the age sample, the candidate splits include {30, 40}, and the left split subset includes the features where the age is less than or equal to 30 and the features where the age is less than or equal to 40; for the salary sample, the candidate splits include {4000, 7000}, and the left split subset includes the features where the salary is less than or equal to 4000 and the features where the salary is less than 7000; for the deposit sample, the candidate splits include {30000, 60000}, and the left split subset includes the features where it is less than or equal to 30000 and the features where it is less than or equal to 60000.

[0080] Referring to Figure 2 , represent the left split subset as a binary vector, and then combine it into a binary matrix. After the participating terminal generates the binary matrix, it cryptographically shares the binary matrix with the initiating terminal.

[0081] The initiating terminal holds the true label corresponding to the sample features, makes a prediction on the sample features through a preset initial decision tree model to obtain a predicted value, denoted as and calculates the first-order gradient and second-order gradient of each sample based on the true label, the initial predicted value, and a preset loss function, and then combines them into two gradient vectors and Concatenate the gradient vectors, the binary matrix of the initiating terminal, and the binary matrix secretly shared by the participating terminals into a secret sharing matrix, denoted as and secretly share the secret sharing matrix with the participating terminals.

[0082] Among them, the initial decision tree model can be a regression tree model or a classification tree model, and this embodiment does not limit the type of the initial decision tree model; the preset loss function includes a mean square error loss function or a logistic regression loss function, etc., and this embodiment does not limit the type of the preset loss function.

[0083] Specifically, before combining the local binary matrix, gradient information and the binary matrix participating in the terminal secret sharing by the initiating terminal to obtain the sharing matrix of the root node of the first decision tree model and secretly sharing it with the participating terminals, it further includes: each training terminal determines the candidate splits of the sample features it holds, and divides the sample features into a left subset and a right subset based on each candidate split; converts the left subset to obtain a binary matrix and secretly shares it with the model initiating end; the model initiating end predicts the sample features based on the preset decision tree model to obtain prediction values; inputs the prediction values and the sample features into the preset loss function, calculates the first-order gradient and second-order gradient of each sample feature, and combines the first-order gradient and the second-order gradient into two gradient vectors to obtain gradient information.

[0084] Step S102, each training terminal secretly performs a row permutation on the sharing matrix respectively based on the pre-shared permutation triple, obtains the permutation result and secretly shares and splices it to obtain the binary permutation matrix of the root node.

[0085] In some embodiments of the present invention, in order to effectively improve the privacy and security during the calculation process, when each training terminal performs the calculation, it needs to secretly perform a row permutation on the sharing matrix through secret sharing and the given permutation triple, ensuring that no single terminal can obtain the complete data or calculation result, thereby preventing single-point leakage.

[0086] The permutation operation makes it impossible for an attacker to infer the specific data even if they try to access the intermediate results, ensuring the security of the calculation. At the same time, this collaborative calculation method ensures that the correct result can be finally restored and privacy protection is maintained by sharing and splicing local results.

[0087] The unpredictability of the permutation triple further enhances the protection of the data structure, making it impossible for an attacker to easily infer the data content. Overall, this method ensures the correctness of the calculation and improves the security, preventing data leakage through distributed computing and encryption means.

[0088] In some embodiments of the present invention, the permutation triple is represented as (<π> C , A , <π(U)> A )。

[0089] Among them, <π> C represents the permutation operation of secret sharing based on cycle decomposition, A Denotes a secret sharing matrix, and each training terminal holds a part of the secret sharing matrix; <π(U)> A Denotes the permutation matrix obtained after the secret sharing matrix undergoes a permutation operation.

[0090] In some embodiments of the present invention, first, the initiating terminal assigns the rows to be permuted to each terminal, and each terminal generates a local permutation locally, thus obtaining <π> C ; Second, all training terminals cooperate to generate a secret sharing matrix A ; then, locally permute <π> of each training terminal is used cyclically C for the secret sharing matrix A Perform row permutation, that is, obtain the encrypted form of matrix U using threshold fully homomorphic encryption And each training terminal locally uses the local permutation <π> C For Perform row permutation; then, convert the permuted matrix into a secret sharing form; finally, after completing the cyclic permutation, <π(U)> is obtained A .

[0091] Specifically, before each training terminal respectively performs secret row permutation on the shared matrix based on the pre-shared permutation triple, obtains the permutation result and secret shares and splices them to obtain the binary permutation matrix of the root node, it also includes: the model initiator allocates the rows for which each training terminal is responsible for permutation; each training terminal locally generates a local permutation to obtain the permutation operation <π> C ; each training terminal collaboratively generates a secret sharing matrix A ; Recursively use the permutation operations corresponding to each training terminal on the secret sharing matrix A Perform row permutation, and use threshold fully homomorphic encryption to obtain the encrypted form of the secret sharing matrix. Each training terminal locally performs its own permutation operation on the encrypted form of the secret sharing matrix to perform row permutation on it, and converts the permuted matrix into a secret sharing form; after completing the cyclic permutation, the permutation matrix <π(U)> is obtained. A .

[0092] Among them, the rows responsible for permutation assigned to each training terminal by the model initiator refer to the row indices assigned by the initiating terminal according to the number of each training terminal.

[0093] For example: taking the number of row indices as q and the number of training terminals as 3 as an example, randomly divide the q row indices into 3 parts. The number of divided row index parts corresponds one-to-one with the number of training terminals. Assign the first part of row indices to the first training terminal; assign the second part of row indices to the second training terminal; assign the third part of row indices to the third training terminal.

[0094] In some embodiments of the present invention, generating a local permutation means randomly shuffling the order of the received row indices, and the mapping relationship from the original order to the new order is a local permutation.

[0095] Specifically, each training terminal locally generates a local permutation to obtain the permutation operation <π> C , including: each training terminal randomly shuffles the order of the received row indices, and determines the mapping relationship from the original order to the new order as the permutation operation.

[0096] Through a given permutation triple of secret sharing (<π> C , A , <π(U)> A ), such as Figure 3 shown, all training terminals first use secret - shared addition calculation and restore it to the plain - text form to obtain Next, each training terminal performs row permutation on the pre - assigned partial rows, sets other rows to 0, and secretly shares the local permutation result; finally, each training terminal concatenates all local permutation results together to obtain and compare with A Perform the addition operation for secret sharing to obtain the permutation result of

[0097] Specifically, each training terminal, based on the pre-shared permutation triples, respectively performs secret row permutations on the sharing matrix, obtains the permutation result and secretly shares and splices it to obtain the binary permutation matrix of the root node, including: each training terminal, based on the secret-sharing addition operator, collaboratively adds the sharing matrix and the secret-sharing matrix to obtain the matrix after adding the mask; after restoring the matrix after adding the mask to the plaintext form, perform row permutations on some pre-allocated rows through the permutation operation, set the other rows to 0, obtain the local permutation result and secretly share it; after splicing all the local permutation results, add them to the secret-sharing matrix through the addition operator to obtain the binary permutation matrix.

[0098] It should be noted that in some embodiments of the present invention, all local permutations are the cycle decomposition of the global permutation, that is, each training terminal is only responsible for permuting a part of the samples, and there is no intersection in the responsible ranges of each training terminal.

[0099] Step S103: Take the root node as the current node, and collaboratively calculate the linear gain score array of the candidate split of the current node based on the binary permutation matrix and the secretly shared addition and multiplication operators, and secretly share it.

[0100] In some embodiments of the present invention, for any decision tree model t, iterative splitting is performed from the root node to the intermediate node and then to the leaf node of the decision tree model. For each node, each training terminal collaboratively calculates the linear gain scores of all candidate splits. The number of candidate splits corresponding to each node is M; where M is an integer greater than 1.

[0101] Taking the d-th node in the t-th decision tree model as an example, there are M candidate splits for the d-th node. For the m-th candidate split, first, it is necessary to first calculate the second-order derivative sum of the secret sharing of the node based on the binary permutation matrix corresponding to the node and the secretly shared addition operator, which can be expressed by the following formula:

[0102]

[0103] In the formula, <h> A Represents the sum of the second-order derivatives of the secret sharing of the current node; Represents the last column from the bottom in the binary permutation matrix; n d Represents the number of sample features of the current node.

[0104] Next, based on the m-th column in the binary permutation matrix, the second-to-last column in the binary permutation matrix, and the multiplication operator of the secret sharing, calculate the sum of the first-order derivatives of the secret sharing of the left child node of the m-th candidate split of the current node, and the sum of the first-order derivatives of the secret sharing of the right child node. Among them, the sum of the first-order derivatives of the secret sharing of the left child node of the m-th candidate split of the current node can be expressed by the following formula:

[0105]

[0106] In the formula, <G m,L > A Represents the sum of the first-order derivatives of the secret sharing of the left child node of the m-th candidate split; <I m,L > A Represents the m-th column in the binary permutation matrix; Represents the second-to-last column in the binary permutation matrix.

[0107] The sum of the first-order derivatives of the secret sharing of the right child node of the m-th candidate split of the current node can be expressed by the following formula:

[0108]

[0109] In the formula, <G m,R > A Represents the sum of the first-order derivatives of the secret sharing of the right child node of the m-th candidate split; <I m,L > A Represents the m-th column in the binary permutation matrix; Represents the second-to-last column in the binary permutation matrix; n d Represents the number of sample features of the current node.

[0110] Next, based on the m-th column in the binary permutation matrix, the last column from the bottom in the binary permutation matrix, and the multiplication operator, jointly calculate the sum of the second-order derivatives of the secret sharing of the left child node obtained by the m-th candidate split, and the sum of the second-order derivatives of the secret sharing of the right child node; among them, the sum of the second-order derivatives of the secret sharing of the left child node can be expressed by the following formula:

[0111]

[0112] In the formula, <H m,L > A Represents the sum of the second-order derivatives of the secret sharing of the left child node; <I m,L > A represents the m-th column in the binary permutation matrix; represents the last column in the binary permutation matrix.

[0113] The sum of the second-order derivatives of the secret sharing of the right child node can be expressed by the following formula:

[0114]

[0115] In the formula, <H m,R > A represents the sum of the second-order derivatives of the secret sharing of the right child node; <I m,L > A represents the m-th column in the binary permutation matrix; represents the last column in the binary permutation matrix; n d represents the number of sample features of the current node.

[0116] Finally, based on the above calculation results and the pre-set linear gain fraction algorithm, the linear gain fraction corresponding to the m-th candidate split is calculated. After traversing all M candidate splits, a linear gain fraction array is obtained. Among them, the linear gain fraction algorithm is expressed by the following formula:

[0117] <S * > A =(<G m,L > A ) 2 ·(2 <h> A -<H m,L > A ) 2 +(<G m,R > A ) 2

[0118] ·(2 <h> A -<H m,R > A ) 2

[0119] wherein, <G m,L > A represents the sum of the first-order derivatives of the secret sharing of the left child node; <h> A Denote the second - order derivative sum of the secret sharing of the current node; <H m,L > A Denote the first - order derivative sum of the secret sharing of the left child node; <G m,R > A Denote the first - order derivative sum of the secret sharing of the right child node; <H m,R > A Denote the second - order derivative sum of the secret sharing of the right child node.

[0120] Specifically, based on the binary permutation matrix and the addition and multiplication operators of secret sharing, collaboratively calculate the linear gain score array of the candidate splits of the current node and perform secret sharing, including: for the m - th candidate split among the M candidate splits, collaboratively calculate the second - order derivative sum of the secret sharing of the current node through each training terminal based on the last column of the binary permutation matrix and the addition operator; m is an integer that takes values from 1 to M in sequence; based on the m - th column of the binary permutation matrix, the second - last column of the binary permutation matrix and the multiplication operator, collaboratively calculate the first - order derivative sum of the secret sharing of the left child node and the first - order derivative sum of the secret sharing of the right child node obtained by the m - th candidate split; based on the m - th column, the last column and the multiplication operator, collaboratively calculate the second - order derivative sum of the secret sharing of the left child node and the second - order derivative sum of the secret sharing of the right child node obtained by the m - th candidate split; based on the second - order derivative sum of the secret sharing of the current node, the first - order derivative sum of the secret sharing of the left child node, the first - order derivative sum of the secret sharing of the right child node, the second - order derivative sum of the secret sharing of the left child node, the second - order derivative sum of the secret sharing of the right child node, and the linear gain score algorithm, collaboratively calculate the linear gain score of the m - th candidate split; after traversing the M candidate splits, obtain the linear gain score array.

[0121] Step S104, through the comparison operator of secret sharing, collaboratively calculate the indication vector of the best split corresponding to the maximum linear gain score in the linear gain score array, and store it in the preset secret sharing set.

[0122] In some embodiments of the present invention, for any decision tree model t, each training terminal will collaboratively create a preset secret sharing set <Θ t > A , as a container for storing information related to the decision tree model.

[0123] Among them, the information related to the decision tree model includes the indication vector of the best split of the node.

[0124] In some embodiments of the present invention, in order to screen for candidate splits with the largest linear gain fraction without exposing the screening process and results, two iterations need to be performed, namely a forward iteration based on a secret-sharing comparison operator and a backward iteration based on the results of the forward iteration, as Figure 4 shown.

[0125] In the forward iteration, the linear gain fraction array is divided into two subsets: the left subset and the right subset Specifically, if has an odd size, the last item is taken out and placed in a temporary variable χ, and added to the next-round linear gain fraction array before the next round of execution.

[0126] Compare the two subsets element by element and generate a comparison result, which is stored in the pre-secret-shared binary vector Then, all parties collaboratively calculate the following expression:

[0127]

[0128] to obtain the linear gain fraction array of the next-round forward iteration In the formula, represents the left subset in the forward iteration; represents the secret-shared binary vector storing the comparison result; represents the right subset in the forward iteration.

[0129] When the linear gain fraction array has only one item left, the forward iteration stops and the backward iteration is performed. The backward iteration is to reverse-derive the optimal split position from the results of the forward iteration, and the number of iteration rounds is the same as that of the forward iteration.

[0130] At the beginning of the first round of backward iteration, the linear gain fraction array and the secret-shared binary vector obtained in the last round of the forward iteration are already obtained. Represent the linear gain fraction array obtained in the last round as <c i+1 > A , as the result of the previous round of iteration. The key step of the backward iteration is to calculate the gain fractions (left and right subsets) after splitting. By element-wise multiplication <c i+1 > A and to obtain <c i,L > A , that is, the gain fraction of the left subset, which can be expressed by the following formula:

[0131]

[0132] where <c i+1 > A represents the result of the previous iteration; represents the binary vector of the secret sharing, storing the comparison result.

[0133] Removing the left subset gain score from the total gain score to obtain the right subset gain score can be expressed by the following formula:

[0134] <c i,R > A = <c i+1 > A - <c i,L > A

[0135] where <c i+1 > A represents the result of the previous iteration; <c i,L > A represents the left subset gain score.

[0136] Concatenating <c i,R > A and <c i,L > A to obtain <c i > A , which is the indication vector <I for the position of the largest term in. When reaching the last round, the output m > A . <c1> A is the indication vector of the optimal splitting position.

[0137] Specifically, through the comparison operator of secret sharing, the indication vector corresponding to the optimal splitting of the maximum linear gain fraction in the linear gain fraction array is collaboratively calculated and stored in the preset secret sharing set, including: taking the linear gain fraction array as the forward iteration fraction array of the current forward iteration round. When the number of linear gain fractions in the forward iteration fraction array is even, the forward iteration fraction array is evenly divided into a forward iteration left subset and a forward iteration right subset from the middle position; through each training terminal, the linear gain fractions at the corresponding positions in the forward iteration left subset and the forward iteration right subset are compared one by one according to the comparison operator, and the larger linear gain fraction is stored in the pre-secretly shared gain fraction vector; based on the gain fraction vector, the forward iteration left subset and the forward iteration right subset, a new fraction array is calculated as the forward iteration fraction array of the next forward iteration round, and the step of evenly dividing the forward iteration fraction array into a forward iteration left subset and a forward iteration right subset from the middle position when the number of linear gain fractions in the forward iteration fraction array is even is executed until the number of linear gain fractions in the new fraction array is 1; when the number of linear gain fractions in the new fraction array is 1, taking the new fraction array as the backward iteration fraction array, and performing an element-wise product with the gain fraction vector to obtain the left subset gain fraction; based on the backward iteration fraction array and the left subset gain fraction, the right subset gain fraction is calculated; the left subset gain fraction and the right subset gain fraction are concatenated to obtain the backward iteration fraction array of the next backward iteration round, and the step of performing an element-wise product with the gain fraction vector to obtain the left subset gain fraction is executed; until all iteration rounds are completed, the indication vector is obtained; the number of forward iterations is the same as the number of backward iterations.

[0138] In addition, when the number of elements in the backward iteration fraction array is greater than the number of elements in the gain fraction vector, it further includes: stripping the extra elements from the backward iteration fraction array and splicing them into the backward iteration fraction array of the next backward iteration round.

[0139] When the number of linear gain fractions in the forward iteration fraction array is even, it further includes: taking out the last item in the forward iteration fraction array and putting it into a temporary variable; before the next forward iteration round is executed, taking out the last item from the temporary variable and adding it to the forward iteration fraction array of the next forward iteration round.

[0140] Step S105, according to the candidate splitting number, select the corresponding columns from the binary permutation matrix to calculate with the indication vector, collaboratively calculate the child node samples of the current node, and calculate the binary permutation matrix of the child node based on the child node samples and the binary permutation matrix.

[0141] Among them, according to the number of candidate splits, the corresponding columns are selected from the binary permutation matrix and calculated with the indicator vector, and the child node samples of the current node are calculated collaboratively, which can be expressed by the following formula:

[0142]

[0143] In the formula, represents the binary permutation matrix of the current node (the d-th node in the t decision tree models); M represents the number of candidate splits; <I m > A represents the indicator vector.

[0144] Disclose <I s > A to obtain <I s . All the samples with values of 1 in <I s are divided into the sample set of the left child node of the current node, and the remaining samples are divided into the sample set of the right child node.

[0145] Based on the child node samples and the binary permutation matrix, the binary permutation matrices of the child nodes are calculated, including obtaining the binary permutation matrix of the left child node and the binary permutation matrix of the right child node. Among them, the binary permutation matrix of the left child node is expressed as:

[0146]

[0147] In the formula, represents the binary permutation matrix of the current node (the d-th node in the t decision tree models); represents the sample set of the left child node of the current node.

[0148] The binary permutation matrix of the right child node is expressed as:

[0149]

[0150] In the formula, represents the binary permutation matrix of the current node (the d-th node in the t decision tree models); represents the sample set of the right child node of the current node.

[0151] Step S106: Take the child node as the current node, and execute the step of calculating the linear gain score array of the candidate splits of the current node until the split from the root node to the leaf node is completed, and calculate the leaf node weights and store them in the preset secret sharing set.

[0152] After all non-leaf nodes are split, the weights of each leaf node need to be calculated.

[0153] Specifically, for any leaf node, calculating the leaf node weight includes: taking out the last two columns from the binary permutation matrix corresponding to the leaf node; calculating the weight corresponding to the leaf node based on the last two columns, the sample corresponding to the leaf node, and a preset weight algorithm; the preset weight algorithm is expressed as:

[0154]

[0155] In the formula, represents the sample of the leaf node; <w t,d > A represents the weight of the leaf node; represents the penultimate column of the binary permutation matrix; represents the last column of the binary permutation matrix.

[0156] Step S107, when the number of decision tree models has not reached the preset number, based on the leaf node weight and the prediction value, update the binary permutation matrix of the root node to obtain the binary permutation matrix of the root node of the next decision tree model, take the root node of the next decision tree model as the current node, and execute the step of calculating the linear gain score array of the candidate split of the current node until the number of decision tree models reaches the preset number, and output the preset secret sharing set.

[0157] Among them, updating the binary permutation matrix of the root node based on the leaf node weight and the prediction value to obtain the binary permutation matrix of the root node of the next decision tree model includes: adding the leaf node weight and the prediction value by the initiating terminal to obtain the prediction value corresponding to the next decision tree model; calculating the gradient information of the next decision tree model based on the prediction value of the next decision tree model, the true label, and a preset loss function; taking the gradient information of the next decision tree model as the last two columns of the binary permutation matrix of the first decision tree model to obtain the binary permutation matrix of the root node of a decision tree model.

[0158] Among them, adding the leaf node weight and the prediction value to obtain the prediction value corresponding to the next decision tree model can be expressed by the following formula:

[0159]

[0160] In the formula, <w t,d > A represents the weight of the leaf node; represents the prediction value.

[0161] After all decision tree models are constructed, output the preset secret sharing set {<Θ0> A ,…,<Θ T > A}, which contains the best split information of each intermediate node and the weight information of each leaf node.

[0162] In summary, the vertical federated boosted tree method for privacy protection provided in this embodiment can solve the problem of low efficiency of the existing vertical federated gradient boosted tree protocol for privacy protection; before model training, anonymization of sample identifiers is achieved through random row permutation, that is, the training terminal cannot know which samples are exactly included in the sample set of each node, thereby allowing the "computational obliviousness" that must be satisfied by the existing protocol to be broken, significantly reducing the computational and communication complexity during training; at the same time, a linear gain score is provided, which, compared with the original gain score of the gradient boosted tree algorithm, does not introduce division operations, and the division operation is quite time-consuming to implement under secure multi-party computation, and can shorten the computation time, thereby further improving the training efficiency.

[0163] In addition, during the process of optimal split selection, through forward iteration and backward iteration, the number of uses of costly comparison operations is reduced to the minimum, which can reduce the computational complexity and further improve the model training efficiency.

[0164] In some embodiments of the present invention, experiments are conducted on the training efficiency and prediction accuracy. The information of the datasets used is shown in Table 1:

[0165] Dataset Feature dimension Number of samples Task Adult 104 45222 Binary classification Bank 51 45211 Binary classification Credit - card 91 30000 Binary classification Super 784 60000 Regression

[0166] Table 1

[0167] The list of parameters involved is shown in Table 2:

[0168]

[0169] Table 2

[0170] There are four comparison methods involved:

[0171] 1) Training method of non-private vertical federated gradient tree (non-private): This method collaboratively constructs a gradient boosted tree without considering privacy protection;

[0172] 2) Squirrel: This method is designed for two parties and achieves efficiency improvement by applying secret sharing, fully homomorphic encryption, and oblivious transfer to appropriate operators;

[0173] 3) Random permutation XGB (CRP-XGB): This method is designed for two parties and reduces the overhead generated by secret sharing by introducing random permutation;

[0174] 4) Pivot: This method is designed for multiple parties and combines secret sharing and threshold semi-homomorphic encryption.

[0175] Table 3 shows the comparison results of the training efficiency of the privacy-preserving vertical federated boosting tree method (Ours in Table 3) provided by this application with the comparative methods under different parameter settings.

[0176] Specifically, the privacy-preserving vertical federated boosting tree method provided by this application is split into initialization and training time, and the training dataset used is artificially generated.

[0177]

[0178] Table 3

[0179] It can be observed from Table 3 that under any setting, the method provided by this application can achieve better training efficiency than Squirrel, CRP-XGB, and Pivot. This is because these comparative methods introduce a large amount of data-agnostic computations to protect the sample sets of each node in the tree, resulting in the computational complexity of each node being related to the total number of samples. In contrast, the method provided by this application avoids using data-agnostic computations by performing a secure random permutation before training, thus minimizing the computational complexity of each node to be related to the number of samples available for that node. Moreover, the method provided by this application reduces the number of computationally expensive non-linear operators during the model training process by introducing linear gain scores and a synchronous best split selection method.

[0180] The method provided by this application uses linear gain scores instead of the gain scores used in the original algorithm of gradient boosting trees. To verify whether the method provided by this application will lose model accuracy due to this, the model accuracies of the method provided by this application and non-private under different parameter settings were compared on a real dataset to fully demonstrate the utility differences between linear gain scores and the original gain scores.

[0181] Specifically, the model accuracy metrics adopted are AUC (Area Under the ROC Curve) and MSE (Mean Squared Error). Figure 5 The comparison results are shown. From Figure 5 It can be observed that under any parameter setting, the method provided by this application can achieve a model accuracy close to that of non-private. This shows that the utility of linear gain scores is consistent with that of the original gain scores.

[0182] Another aspect of the present invention provides a privacy - protected vertical federated boosting tree device, including a processor, a memory, and computer programs / instructions stored on the memory. The processor is configured to execute the computer programs / instructions, and when the computer programs / instructions are executed, the device implements the steps of the aforementioned privacy - protected vertical federated boosting tree method. The present application also provides a computer - readable storage medium, in which a program is stored, and the program is loaded and executed by the processor to implement the privacy - protected vertical federated boosting tree method of the above - mentioned method embodiment.

[0183] The present application also provides a computer program product, including computer programs / instructions, and when the computer programs / instructions are executed by the processor, the privacy - protected vertical federated boosting tree method of the above - mentioned method embodiment is implemented.

[0184] The technical features of the above - described embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above - described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0185] Obviously, the above - described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, those of ordinary skill in the art can make other different forms of changes or modifications without creative efforts, and all of them should fall within the scope of protection of the present application. < / h> < / h> < / h> < / h> < / h> < / h> < / h>

Claims

1. A privacy-preserving vertical federated boosting tree method, characterized in that: Applied to a vertical federated learning training system constructed by at least two training terminals, the training terminals include an initiating terminal and a participating terminal, and are used to jointly construct a preset number of decision tree models; the method includes the following steps: The initiating terminal combines the local binary matrix, gradient information and the binary matrix secretly shared by the participating terminals to obtain the sharing matrix of the root node of the first decision tree model and secretly shares it with the participating terminals; the binary matrix is ​​obtained by converting the left partition subset of the candidate split of the sample feature; the gradient vector is calculated based on the predicted value of the sample feature, the true label and the preset loss function; By each training terminal performing secret row permutation on the shared matrix based on the pre-shared permutation triples, the permutation results are obtained and secretly shared and concatenated to obtain the binary permutation matrix of the root node; Taking the root node as the current node, based on the binary permutation matrix and the addition and multiplication operators of secret sharing, collaboratively calculating the linear gain score group of the candidate splits of the current node and sharing them secretly; By using a secret sharing comparison operator, collaboratively calculating an indicator vector of an optimal split corresponding to the maximum linear gain score in the linear gain score group, and storing it in a preset secret sharing set; According to the number of candidate splits, corresponding columns are selected from the binary permutation matrix and calculated with the indicator vector, and child node samples of the current node are obtained by collaborative calculation, and a binary permutation matrix of the child node is obtained based on the child node samples and the binary permutation matrix; Taking the child node as the current node, executing the step of calculating the linear gain score array of the candidate split of the current node until the split from the root node to the leaf node is completed, and calculating the leaf node weight, and storing it in the preset secret sharing set; When the number of decision tree models does not reach the preset number, based on the leaf node weights and the predicted values, the binary permutation matrix of the root node is updated to obtain the binary permutation matrix of the root node of the next decision tree model, and the root node of the next decision tree model is used as the current node to execute the step of calculating the linear gain score group of the candidate split of the current node until the number of decision tree models reaches the preset number, and the preset secret sharing set is output.

2. The method according to claim 1, characterized in that The number of candidate splits is M; the addition and multiplication operators based on the binary permutation matrix and secret sharing collaboratively calculate the linear gain score group of the candidate splits of the current node and share them secretly, including: For the mth candidate split among the M candidate splits, the training terminals collaboratively calculate the second-order inverse sum of the secret sharing of the current node based on the last column in the binary permutation matrix and the addition operator; m is an integer ranging from 1 to M in sequence; Based on the mth column in the binary permutation matrix, the penultimate column in the binary permutation matrix and the multiplication operator, collaboratively calculate the sum of the first-order derivatives of the secret sharing of the left child node obtained by the mth candidate split partition and the sum of the first-order derivatives of the secret sharing of the right child node; Based on the mth column, the penultimate column and the multiplication operator, collaboratively calculate the sum of the second-order derivatives of the secret sharing of the left child node obtained by the mth candidate split partition and the sum of the second-order derivatives of the secret sharing of the right child node; Based on the second-order inverse sum of the secret sharing of the current node, the first-order derivative sum of the secret sharing of the left child node, the first-order derivative sum of the secret sharing of the right child node, the second-order derivative sum of the secret sharing of the left child node, the second-order derivative of the secret sharing of the right child node, and the linear gain score algorithm, collaboratively calculate the linear gain score of the m-th candidate split; After traversing the M candidate splits, obtaining the linear gain score group; The linear gain fraction algorithm is expressed by the following equation: <S * > A =(<G m，L > A ) 2 ·(2 <h> A -<H m，L > A ) 2 +(<G m，R > A ) 2 ·(2 <h> A -<H m，R > A ) 2 < / h> < / h> In the formula, <G m，L > A represents the sum of the first-order derivatives of the secret sharing of the left child node; <h> A represents the second-order inverse sum of the secret sharing of the current node; <H m，L > A represents the sum of the second-order derivatives of the secret sharing of the left child node; <G m，R > A represents the sum of the first-order derivatives of the secret sharing of the right child node; <H m，R > A represents the sum of the second-order derivatives of the secret sharing of the right child node.< / h> 3. The method according to claim 1, characterized in that The permutation triple is represented by (<π> C , A , <π(U)> A ), wherein said <π> C represents the permutation operation of secret sharing based on round-robin decomposition, A represents a secret sharing matrix, each training terminal holds a part of the secret sharing matrix; the <π(U)> A represents a permutation matrix obtained after the secret sharing matrix is ​​subjected to the permutation operation; The shared matrix is ​​secret row permuted by each training terminal based on the pre-shared permutation triples, and the permutation results are obtained and secretly shared and concatenated to obtain the binary permutation matrix of the root node, including: By means of the respective training terminals, the sharing matrix and the secret sharing matrix are added collaboratively by an addition operator based on secret sharing to obtain a masked matrix; After restoring the masked matrix to a plain text form, performing row permutation on some pre-allocated rows through the permutation operation, setting other rows to 0, obtaining a partial permutation result and sharing it secretly; After all the local permutation results are concatenated, they are added to the secret sharing matrix through the addition operator to obtain the binary permutation matrix.

4. The method according to claim 3, characterized in that Before obtaining the binary permutation matrix of the root node by performing secret row permutations on the shared matrix based on the pre-shared permutation triples by the training terminals, obtaining permutation results and secretly sharing and splicing them, the method further includes: The model initiator allocates the rows that each training terminal is responsible for replacing through the model initiator; A local permutation is generated locally by each training terminal to obtain the permutation operation <π> C ; The secret sharing matrix is ​​generated by the various training terminals in a collaborative manner A ； Cyclic use of the permutation operations corresponding to the training terminals to the secret sharing matrix A Perform row permutation, use threshold fully homomorphic encryption to obtain the encrypted form of the secret sharing matrix, and use each training terminal to perform row permutation on the encrypted form of the secret sharing matrix locally using its own permutation operation, and convert the permuted matrix into a secret sharing form; after completing the cyclic permutation, obtain the permutation matrix <π(U)> A .

5. The method according to claim 4, characterized in that The allocating, by the model initiating terminal, rows that each training terminal is responsible for replacing refers to allocating, by the initiating terminal, row indexes according to the number of each training terminal; The local permutation is generated locally by each training terminal to obtain the permutation operation <π> c , including: each training terminal randomly disrupts the order of received row indexes, and determines the mapping relationship from the original order to the new order as the permutation operation.

6. The method according to claim 1, characterized in that The best split indicator vector corresponding to the maximum linear gain score in the linear gain score group is collaboratively calculated through a secret sharing comparison operator, and stored in a preset secret sharing set, including: Using the linear gain score group as the forward iteration score group of the current forward iteration round, and when the number of linear gain scores in the forward iteration score group is even, equally dividing the forward iteration score group from the middle position into a forward iteration left subset and a forward iteration right subset; By means of the respective training terminals, the linear gain scores of the corresponding positions in the forward iteration left subset and the forward iteration right subset are compared one by one according to the comparison operator, and the larger linear gain score is stored in the gain score vector shared in secret in advance; Based on the gain score vector, the forward iteration left subset and the forward iteration right subset, a new score group is calculated as the forward iteration score group of the next forward iteration round, and when the number of linear gain scores in the forward iteration score group is even, the forward iteration score group is equally divided from the middle position into a forward iteration left subset and a forward iteration right subset until the number of linear gain scores in the new score group is 1; When the number of linear gain scores in the new score array is 1, the new score array is used as a reverse iterative score array, and element-by-element multiplication is performed with the gain score vector to obtain the number of left subset gain shares; Based on the reverse iteration score group and the left subset gain number, a right subset gain score is calculated; The left subset gain shares and the right subset gain scores are concatenated to obtain a reverse iteration score array for the next reverse iteration round, and the step of obtaining the left subset gain shares by element-by-element multiplication with the gain score vector is performed; until all iteration rounds are completed to obtain the indicator vector; the number of forward iterations is the same as the number of reverse iterations.

7. The method according to claim 6, characterized in that In the case where the number of elements in the reverse iteration score array is greater than the number of elements in the gain score vector, the method further includes: The redundant elements are stripped out from the reverse iteration score array and added to the reverse iteration score array of the next reverse iteration round.

8. The method according to claim 6, characterized in that In the case where the number of linear gain scores in the forward iteration score group is even, the method further includes: Take out the last item in the forward iteration score array and put it into a temporary variable; Before the next forward iteration round is executed, the last item is taken out from the temporary variable and added to the forward iteration score group of the next forward iteration round.

9. The method according to claim 1, characterized in that: For any leaf node, calculating the leaf node weight includes: Take the last two columns from the binary permutation matrix corresponding to the leaf node; Based on the last two columns, the samples corresponding to the leaf nodes and the preset weight algorithm, the weight corresponding to the leaf nodes is calculated; the preset weight algorithm is expressed as: In the formula, is a sample representing the leaf node; <w t，d > A The weight of the leaf node; Represents the second to last column; Indicates the last column.

10. The method according to claim 1, characterized in that Before the initiating terminal combines the local binary matrix, gradient information and the binary matrix secretly shared by the participating terminals to obtain the sharing matrix of the root node of the first decision tree model and secretly shares it with the participating terminals, the method further includes: Determine, by means of each training terminal, a candidate split of the sample features held by each training terminal, and divide the sample features of each training terminal into the left partition subset and the right partition subset based on each candidate split; The left partitioned subset is transformed to obtain the binary matrix and the binary matrix is ​​secretly shared with the model initiator; The model initiator predicts the sample features based on a preset decision tree model to obtain a predicted value; The predicted value and the sample feature are input into the preset loss function, the first-order gradient and the second-order gradient of each sample feature are calculated, and the first-order gradient and the second-order gradient are combined into two gradient vectors to obtain the gradient information.

Images