Privately-protected federated boosting trees

Abstract

The application provides a privacy-protected longitudinal federated boosting tree method and relates to the technical field of longitudinal federated learning; the method comprises the following steps: combining secret sharing binary matrix and gradient information to obtain a shared matrix and secret sharing; each training terminal performs row permutation on the shared matrix based on a permutation triple and secret shares the permutation result; linear gain scores of candidate splits are calculated and secret shared, and an indication vector corresponding to the maximum linear gain score is calculated; according to the number of candidate splits, corresponding columns are selected from the matrix, and the indication vector is used to calculate the sample and the permutation matrix of the child node; the splitting step is recursively executed to complete the splitting from the root node to the leaf node, the weight of the leaf node is calculated and stored in a preset set. The binary permutation matrix of the root node of the next decision model is updated based on the weight of the leaf node, and the calculation is continued until all decision tree models are generated; the method can solve the problem of low efficiency of the existing longitudinal federated gradient boosting tree protocol.

Description

Technical Field

[0001] This invention relates to the field of vertical federated learning technology, and more particularly to a privacy-preserving vertical federated boosting tree method. Background Technology

[0002] Gradient boosting tree (GPRS) is a commonly used machine learning algorithm that sequentially builds multiple decision trees, optimizing subsequent trees based on the errors of previous ones. It has achieved state-of-the-art performance on various machine learning tasks and has consistently dominated applied machine learning competitions and real-world applications. With the increasing number of data sources, the need for collaboration across multiple data owners to build more robust GPRS models is growing. However, these data owners may be unwilling to share their data due to legal regulations, privacy policies, or business competition. Furthermore, existing research indicates that transmitting intermediate results during model training still carries the risk of privacy breaches. Therefore, there is a need to develop a multi-party collaborative GPRS training method that satisfies privacy constraints, allowing these data owners to collaboratively train GPRS models without disclosing input data and intermediate results.

[0003] Currently, existing privacy-preserving vertical federated gradient boosting tree protocols typically use secret-sharing technology, which can perform collaborative computation of various basic operations while achieving provably secure results.

[0004] However, in order to ensure the concealment of intermediate results, existing protocols often employ unintentional data computation, which makes the computation and communication complexity too high. At the same time, a large number of nonlinear operations are used during training, which incur significant overhead when computed under secret sharing. As a result, existing protocols are often limited in practical applications due to efficiency issues. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a privacy-preserving vertical federated boosting tree method to eliminate or improve one or more defects existing in the prior art. It can solve the problem of low efficiency in existing privacy-preserving vertical federated gradient boosting tree protocols.

[0006] One aspect of the present invention provides a privacy-preserving vertical federated boosting tree method, applied to a vertical federated learning training system constructed by at least two training terminals, wherein the training terminals include an initiating terminal and participating terminals, for jointly constructing a preset number of decision tree models; the method includes the following steps:

[0007] The initiating terminal combines the local binary matrix, gradient information, and the binary matrix secretly shared by the participating terminals to obtain the sharing matrix of the first decision tree model root node, which is then secretly shared with the participating terminals. The binary matrix is ​​obtained by transforming the left partition subset of the candidate split of the sample features. The gradient vector is calculated based on the predicted value of the sample features, the true label, and the preset loss function.

[0008] By using each training terminal to perform secret row permutations on the shared matrix based on the pre-shared permutation triples, the permutation results are obtained and secretly shared and concatenated to obtain the binary permutation matrix of the root node.

[0009] Using the root node as the current node, based on the binary permutation matrix and the secretly shared addition and multiplication operators, we collaboratively compute the linear gain subarray of the candidate split of the current node and secretly share it.

[0010] By using the comparison operator of secret sharing, the indicator vector of the best split corresponding to the maximum linear gain fraction in the linear gain fraction array is calculated collaboratively and stored in the preset secret sharing set;

[0011] Based on the number of candidate splits, select the corresponding columns from the binary permutation matrix and the indicator vector to calculate the child node samples of the current node through collaborative calculation, and calculate the binary permutation matrix of the child node based on the child node samples and the binary permutation matrix.

[0012] Take the child node as the current node, perform the step of calculating the linear gain subarray of the candidate split of the current node, until the split from the root node to the leaf node is completed, and calculate the weight of the leaf node and store it in the preset secret sharing set.

[0013] If the number of decision tree models has not reached the preset number, the binary permutation matrix of the root node is updated based on the weights and predicted values ​​of the leaf nodes to obtain the binary permutation matrix of the root node of the next decision tree model. The root node of the next decision tree model is taken as the current node, and the step of calculating the linear gain subarray of the candidate split of the current node is executed until the number of decision tree models reaches the preset number, and the preset secret sharing set is output.

[0014] In some embodiments of the present invention, the number of candidate splits is M; based on a binary permutation matrix and secret-sharing addition and multiplication operators, the linear gain subarray of the candidate splits of the current node is collaboratively calculated and secretly shared, including:

[0015] For the m-th candidate split among M candidate splits, the second-order reciprocal sum of the secret sharing of the current node is obtained by each training terminal collaboratively computing based on the penultimate column of the binary permutation matrix and the addition operator; m is an integer ranging from 1 to M.

[0016] Based on the m-th column of the binary permutation matrix, the penultimate column of the binary permutation matrix, and the multiplication operator, the first derivative sum of the secret sharing of the left child node obtained by the m-th candidate split is obtained through collaborative computation, as well as the first derivative sum of the secret sharing of the right child node.

[0017] Based on the m-th column, the last column, and the multiplication operator, the sum of the second derivatives of the secret sharing of the left child node obtained by the m-th candidate split partition and the sum of the second derivatives of the secret sharing of the right child node are obtained by collaborative computation.

[0018] Based on the second derivative sum of the secret sharing of the current node, the first derivative sum of the secret sharing of the left child node, the first derivative sum of the secret sharing of the right child node, the second derivative sum of the secret sharing of the left child node, the second derivative sum of the secret sharing of the right child node, and the linear gain score algorithm, the linear gain score of the m-th candidate split is obtained through collaborative computation.

[0019] After traversing M candidate splits, a linear gain split array is obtained;

[0020] The linear gain fractional algorithm is expressed by the following formula:

[0021] * > A =( <G m,L > A ) 2 ·(2 <h> A -<H m,L > A ) 2 +(<G m,R > A ) 2

[0022] ·(2 <h> A - <H m,R > A ) 2

[0023] In the formula, <G m,L > A This represents the sum of the first derivatives of the secret sharing of the left child node; <h> A This represents the second-order reciprocal sum of the secret sharing of the current node; <H m,L > A This represents the sum of the second derivatives of the secret sharing of the left child node; <G m,R > A Let represent the sum of the first derivatives of the secret sharing of the right child node; <H m,R > A Let represent the sum of the second derivatives of the secret sharing of the right child node.

[0024] In some embodiments of the present invention, the permutation triplet is represented as (<π> C , A ,<π(U)> A ), where <π> C This represents a permutation operation based on secret sharing in a round-robin decomposition. A Represents a secret sharing matrix, with each training terminal holding a portion of the secret sharing matrix; <π(U)> A This represents the permutation matrix obtained after the secret sharing matrix has undergone a permutation operation;

[0025] Each training terminal performs secret row permutations on the shared matrix based on pre-shared permutation triples, obtains the permutation results, and then concatenates them using secret sharing to obtain the binary permutation matrix of the root node, including:

[0026] Through various training terminals, the sharing matrix and the secret sharing matrix are added collaboratively using the addition operator based on secret sharing to obtain the matrix after adding the mask;

[0027] After restoring the masked matrix to plaintext, a row permutation operation is performed on a pre-allocated portion of the rows, while setting the other rows to 0, thus obtaining the local permutation result, which is then secretly shared.

[0028] After concatenating all the local permutation results, add them to the secret sharing matrix using the addition operator to obtain the binary permutation matrix.

[0029] In some embodiments of the present invention, before each training terminal performs secret row permutations on the shared matrix based on pre-shared permutation triples, obtains the permutation results, and secretly shares and concatenates them to obtain the binary permutation matrix of the root node, the method further includes:

[0030] The model initiator assigns rows to be replaced by each training terminal;

[0031] Each training terminal generates a local permutation locally, resulting in the permutation operation <π>. C ;

[0032] A secret sharing matrix is ​​generated collaboratively by various training terminals. A ;

[0033] The secret sharing matrix is ​​then repeatedly subjected to the permutation operations corresponding to each training terminal. A Row permutations are performed, and threshold fully homomorphic encryption is used to obtain the encrypted form of the secret sharing matrix. Each training terminal then performs its own permutation operation locally on the encrypted form of the secret sharing matrix, converting the permuted matrix into the secret sharing form. After completing the cyclic permutations, the permutation matrix <π(U)> is obtained. A .

[0034] In some embodiments of the present invention, allocating the rows to be replaced by each training terminal through the model initiator means allocating row indexes by the initiator according to the number of each training terminal.

[0035] Each training terminal generates a local permutation locally, resulting in the permutation operation <π>. C This includes: randomly shuffling the order of the received row indices at each training terminal, and determining the mapping relationship from the original order to the new order as a permutation operation.

[0036] In some embodiments of the present invention, a comparison operator of secret sharing is used to collaboratively calculate the indicator vector of the optimal split corresponding to the maximum linear gain fraction in the linear gain fraction array, and store it in a preset secret sharing set, including:

[0037] The linear gain fraction is used as the forward iteration fraction for the current forward iteration round. If the number of linear gain fractions in the forward iteration fraction is even, the forward iteration fraction is divided equally from the middle position into the left and right subsets of the forward iteration.

[0038] Through each training terminal, the linear gain scores at corresponding positions in the left and right subsets of the forward iteration are compared one by one according to the comparison operator, and the larger linear gain score is stored in the pre-secretly shared gain score vector.

[0039] Based on the gain fraction vector, the left subset of the forward iteration, and the right subset of the forward iteration, a new subarray is calculated as the forward iteration subarray for the next forward iteration round. If the number of linear gain fractions in the forward iteration subarray is even, the forward iteration subarray is divided equally from the middle position into the left subset of the forward iteration and the right subset of the forward iteration until the number of linear gain fractions in the new subarray is 1.

[0040] If there is only one linear gain fraction in the new subarray, the new subarray is used as the reverse iterative subarray, and the element-wise product with the gain fraction vector is performed to obtain the gain fraction of the left subset.

[0041] The gain score of the right subset is calculated based on the reverse iterative subarray and the gain score of the left subset;

[0042] The gain fractions of the left subset and the gain fractions of the right subset are concatenated to obtain the reverse iteration fraction array for the next reverse iteration round. The step of performing element-wise multiplication with the gain fraction vector is then performed to obtain the gain fractions of the left subset. This process continues until all iteration rounds are completed, resulting in an indicator vector. The number of forward iterations is the same as the number of reverse iterations.

[0043] In some embodiments of the present invention, when the number of elements in the reverse iteration subarray is greater than the number of elements in the gain fraction vector, the method further includes:

[0044] The redundant elements will be stripped from the reverse iteration subarray and appended to the reverse iteration subarray of the next reverse iteration round.

[0045] In some embodiments of the present invention, when the number of linear gain fractions in the forward iterative subarray is even, the method further includes:

[0046] Take the last item from the forward iteration array and put it into a temporary variable;

[0047] Before the next forward iteration round is executed, the last item is taken out from the temporary variable and added to the forward iteration subarray of the next forward iteration round.

[0048] In some embodiments of the present invention, calculating the weight of any leaf node includes:

[0049] Take the last two columns from the binary permutation matrix corresponding to the leaf nodes;

[0050] Based on the last two columns, the samples corresponding to the leaf nodes, and a preset weighting algorithm, the weights corresponding to the leaf nodes are calculated; the preset weighting algorithm is expressed as:

[0051]

[0052] In the formula, This represents a sample representing a leaf node; <w t,d > A The weight of the leaf nodes; This refers to the second to last column; This indicates the last column.

[0053] In some embodiments of the present invention, before the initiating terminal combines its local binary matrix, gradient information, and the binary matrix secretly shared by the participating terminals to obtain the sharing matrix of the first decision tree model root node and secretly shares it with the participating terminals, the method further includes:

[0054] Each training terminal determines its own candidate splits for the sample features it holds, and divides its own sample features into a left partition subset and a right partition subset based on each candidate split.

[0055] The left subset is transformed to obtain a binary matrix, which is then secretly shared with the model initiator.

[0056] The model initiator predicts sample features based on a pre-defined decision tree model to obtain predicted values;

[0057] The predicted value and sample features are input into the preset loss function to calculate the first-order gradient and second-order gradient of each sample feature. The first-order gradient and second-order gradient are then combined into two gradient vectors to obtain the gradient information.

[0058] The privacy-preserving vertical federated boosting tree method of this invention can solve the problem of low efficiency in existing privacy-preserving vertical federated gradient boosting tree protocols. Before model training, sample identifiers are anonymized by performing random row permutations, meaning that the training terminal cannot know which samples are included in the sample set of each node. This allows breaking the "computational inadvertence" requirement that must be met by existing protocols, significantly reducing the computational and communication complexity during training. At the same time, a linear gain score is provided, which, compared to the original gain score of the gradient boosting tree algorithm, does not introduce division operations. Division operations are quite time-consuming in secure multi-party computation, thus shortening the computation time and further improving training efficiency.

[0059] In addition, during the optimal split selection process, the number of comparison operations with high overhead is reduced to a minimum through forward and backward iterations, which can reduce computational complexity and further improve model training efficiency.

[0060] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0061] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0062] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0063] Figure 1 A flowchart of a privacy-preserving vertical federated boosting tree method provided in an embodiment of the present invention.

[0064] Figure 2 This is a schematic diagram illustrating the candidate splitting and binary matrix transformation provided in an embodiment of the present invention.

[0065] Figure 3 This is a schematic diagram illustrating the principle of secret row permutation for each training terminal according to an embodiment of the present invention.

[0066] Figure 4 This is a schematic diagram illustrating the forward and reverse iterations provided in an embodiment of the present invention.

[0067] Figure 5 This is a comparison chart of the accuracy of a model provided in an embodiment of the present invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0069] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0070] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0071] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0072] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0073] The following section provides a detailed description of the privacy-preserving vertical federated boosting tree method provided in this application.

[0074] In some embodiments of the present invention, the privacy-preserving vertical federated boosting tree method provided in this application is applied to a vertical federated learning training system constructed by at least two training terminals, wherein the training terminals include an initiating terminal and participating terminals, used to jointly construct a preset number of decision tree models; the initiating terminal includes devices such as mobile phones, tablets, personal computers or servers; this embodiment does not limit the device type of the initiating terminal; the number of participating terminals is at least one, including devices such as mobile phones, tablets, personal computers or servers, this embodiment does not limit the device type and number of participating terminals.

[0075] This embodiment provides a privacy-preserving vertical federated boosting tree method, such as... Figure 1 As shown, the method includes at least steps S101 to S105:

[0076] Step S101: The initiating terminal combines the local binary matrix, gradient information, and the binary matrix secretly shared by the participating terminals to obtain the sharing matrix of the first decision tree model root node and secretly shares it with the participating terminals.

[0077] The binary matrix is ​​obtained by transforming the left partition subset of the candidate split of the sample features; the gradient vector is calculated based on the predicted value of the sample features, the true label and the preset loss function.

[0078] In some embodiments of the present invention, each training terminal (including initiating terminal and participating terminal) holds its own sample features and generates candidate splits for its own sample features, and divides the sample features into a left partition subset and a right partition subset according to each candidate split.

[0079] For example: Reference Figure 2 Taking sample data including age, salary, and savings as an example; for age samples, the candidate split includes {30, 40}, and the left subset includes features of age less than or equal to 30 and features of age less than or equal to 40; for salary samples, the candidate split includes {4000, 7000}, and the left subset includes features of salary less than or equal to 4000 and features of salary less than or equal to but less than 7000; for savings samples, the candidate split includes {30000, 60000}, and the left subset includes features of 30000 and features of 60000 and less than or equal to 30000.

[0080] refer to Figure 2 The left subset is represented as a binary vector, which is then combined into a binary matrix. After generating the binary matrix, the participating terminal ciphertextually shares this binary matrix with the initiating terminal.

[0081] The initiating terminal holds the true label corresponding to the sample features, and uses a preset initial decision tree model to predict the sample features to obtain the predicted value, represented as follows: Based on the true labels, initial predicted values, and a preset loss function, the first and second gradients of each sample are calculated and then combined into two gradient vectors. and The gradient vector, the binary matrix of the initiating terminal, and the binary matrix of the participating terminals' secret sharing are concatenated into a single secret sharing matrix, represented as follows: The secret sharing matrix is ​​then secretly shared with the participating terminals.

[0082] The initial decision tree model can be a regression tree model or a classification tree model. This embodiment does not limit the type of the initial decision tree model. The preset loss function includes the mean squared error loss function or the logistic regression loss function, etc. This embodiment does not limit the type of the preset loss function.

[0083] Specifically, before the initiating terminal combines its local binary matrix, gradient information, and the binary matrix secretly shared by the participating terminals to obtain the sharing matrix of the first decision tree model root node and secretly shares it with the participating terminals, the process includes: determining the candidate splits of the sample features held by each training terminal, and dividing the sample features of each terminal into a left partition subset and a right partition subset based on each candidate split; transforming the left partition subset to obtain a binary matrix and secretly sharing it with the model initiating terminal; the model initiating terminal predicts the sample features based on the preset decision tree model to obtain the predicted values; inputting the predicted values ​​and sample features into a preset loss function to calculate the first-order gradient and second-order gradient of each sample feature, and combining the first-order gradient and second-order gradient into two gradient vectors to obtain the gradient information.

[0084] Step S102: Based on the pre-shared permutation triples, each training terminal performs secret row permutation on the shared matrix to obtain the permutation results, which are then secretly shared and spliced ​​to obtain the binary permutation matrix of the root node.

[0085] In some embodiments of the present invention, in order to effectively improve the privacy and security of the computation process, each training terminal needs to perform secret row permutation on the shared matrix by secret sharing and given permutation triples when performing computation, so that no single terminal can obtain complete data or computation results, thereby preventing single point leakage.

[0086] The permutation operation ensures that even if an attacker attempts to access intermediate results, they cannot deduce the specific data, thus guaranteeing computational security. Simultaneously, this collaborative computation method, by sharing and piecing together partial results, ensures that the correct result can ultimately be recovered while maintaining privacy.

[0087] The unpredictability of the permutation triples further enhances the protection of the data structure, making it difficult for attackers to deduce the data content. Overall, this method, through distributed computing and encryption, ensures both the correctness of the computation and improves security, preventing data leakage.

[0088] In some embodiments of the present invention, the permutation triplet is represented as (<π> C , A ,<π(U)> A ).

[0089] Among them, <π> C This represents a permutation operation based on secret sharing in a round-robin decomposition. A Represents a secret sharing matrix, with each training terminal holding a portion of the secret sharing matrix; <π(U)> A This represents the permutation matrix obtained after the secret sharing matrix undergoes a permutation operation.

[0090] In some embodiments of the present invention, firstly, the initiating terminal allocates the rows to be replaced by each terminal, and each terminal generates a local replacement locally, thus obtaining <π>. C Secondly, all training terminals collaboratively generate a secret sharing matrix. A Then, the local permutation <π> of each training terminal is used cyclically. C Secret Sharing Matrix A Perform row permutations, i.e., use threshold homomorphic encryption to obtain the encrypted form of matrix U. And each training terminal uses local permutation <π> locally. C right Perform row permutations; then, convert the permuted matrix into a secret-sharing form; finally, after completing the cyclic permutations, you obtain <π(U)>. A .

[0091] Specifically, before obtaining the binary permutation matrix of the root node by performing secret row permutations on the shared matrix based on pre-shared permutation triples at each training terminal, and then secretly sharing and concatenating the permutation results, the process also includes: allocating the rows to be permuted by each training terminal through the model initiator; and generating a local permutation locally on each training terminal to obtain the permutation operation <π>. C Secret sharing matrix is ​​generated collaboratively by various training terminals. A The secret sharing matrix is ​​then repeatedly processed using the permutation operations corresponding to each training terminal. A Row permutations are performed, and threshold fully homomorphic encryption is used to obtain the encrypted form of the secret sharing matrix. Each training terminal then performs its own permutation operation locally on the encrypted form of the secret sharing matrix, converting the permuted matrix into the secret sharing form. After completing the cyclic permutations, the permutation matrix <π(U)> is obtained. A .

[0092] In this context, allocating the rows to be replaced by each training terminal through the model initiator means allocating row indexes according to the number of each training terminal through the initiator.

[0093] For example, if the number of row indices is q and the number of training terminals is 3, the q row indices are randomly divided into 3 parts, and the number of row indices corresponds one-to-one with the number of training terminals. The first part of the row index is assigned to the first training terminal; the second part of the row index is assigned to the second training terminal; and the third part of the row index is assigned to the third training terminal.

[0094] In some embodiments of the present invention, generating a local permutation means randomly shuffling the order of the received row indices, and the mapping relationship from the original order to the new order is a local permutation.

[0095] Specifically, a local permutation is generated locally on each training terminal to obtain the permutation operation <π>. C This includes: randomly shuffling the order of the received row indices at each training terminal, and determining the mapping relationship from the original order to the new order as a permutation operation.

[0096] Given a secret shared permutation triple (<π>) C , A ,<π(U)> A ),like Figure 3 As shown, all training terminals first utilize secret-shared addition calculations. And restore it to plaintext form to obtain Next, each training terminal performs row permutations on a pre-allocated subset of rows, sets other rows to 0, and secretly shares the local permutation results; finally, each training terminal concatenates all local permutation results together to obtain... and A Performing an addition operation to secretly share results in... The result of the permutation

[0097] Specifically, each training terminal performs secret row permutations on the shared matrix based on pre-shared permutation triples, obtains the permutation results, and concatenates them in secret sharing to obtain the binary permutation matrix of the root node. This includes: each training terminal collaboratively adds the shared matrix and the secret shared matrix using the addition operator based on secret sharing to obtain a masked matrix; after restoring the masked matrix to plaintext form, performing row permutations on some pre-assigned rows using permutation operations, setting other rows to 0, obtaining local permutation results, and secretly sharing them; concatenating all local permutation results, and adding them to the secret shared matrix using the addition operator to obtain the binary permutation matrix.

[0098] It is worth noting that in some embodiments of the present invention, all local permutations are a rotational decomposition of global permutations, that is, each training terminal is only responsible for permuting a portion of the samples, and the scope of responsibility of each training terminal does not overlap.

[0099] Step S103: Using the root node as the current node, based on the binary permutation matrix and the secretly shared addition and multiplication operators, collaboratively calculate the linear gain subarray of the candidate split of the current node and secretly share it.

[0100] In some embodiments of the present invention, for any decision tree model t, iterative splitting is performed from the root node to intermediate nodes and then to leaf nodes. For each node, the number of linear gain components for all candidate splits is collaboratively calculated by each training terminal. The number of candidate splits corresponding to each node is M; where M is an integer greater than 1.

[0101] Taking the d-th node in the t-th decision tree model as an example, the d-th node has M candidate splits. For the m-th candidate split, firstly, it is necessary to base the split on the binary permutation matrix corresponding to the node. The sum of the second derivatives of the secret share of a node, calculated using the addition operator with the secret share, can be expressed as follows:

[0102]

[0103] In the formula, <h> A Let represent the sum of the second derivatives of the secret sharing of the current node; n represents the last column of a binary permutation matrix; d This indicates the number of sample features for the current node.

[0104] Next, based on the m-th column of the binary permutation matrix, the penultimate column of the binary permutation matrix, and the multiplication operator for secret sharing, the sum of the first derivatives of the secret sharing of the left child node and the sum of the first derivatives of the secret sharing of the right child node of the m-th candidate split of the current node are calculated. The sum of the first derivatives of the secret sharing of the left child node of the m-th candidate split of the current node can be expressed by the following formula:

[0105]

[0106] In the formula, <G m,L > A Let represent the sum of the first derivatives of the secret sharing of the left child node of the m-th candidate split; m,L > A This represents the m-th column in a binary permutation matrix; This represents the second-to-last column in a binary permutation matrix.

[0107] The sum of the first derivatives of the secret sharing of the right child node of the m-th candidate split of the current node can be expressed by the following formula:

[0108]

[0109] In the formula, <G m,R > A Let represent the sum of the first derivatives of the secret sharing of the right child node of the m-th candidate split; m,L > A This represents the m-th column in a binary permutation matrix; n represents the second-to-last column in a binary permutation matrix; d This indicates the number of sample features for the current node.

[0110] Next, based on the m-th column of the binary permutation matrix, the penultimate column of the binary permutation matrix, and the multiplication operator, the sum of the second derivatives of the secret sharing of the left child node obtained from the m-th candidate split is calculated collaboratively, as well as the sum of the second derivatives of the secret sharing of the right child node; wherein, the sum of the second derivatives of the secret sharing of the left child node can be expressed by the following formula:

[0111]

[0112] In the formula, <H m,L > A This represents the sum of the second derivatives of the secret sharing of the left child node; <I​​ m,L > A This represents the m-th column in a binary permutation matrix; This represents the last column of a binary permutation matrix.

[0113] The sum of the second derivatives of the secret sharing of the right child node can be expressed by the following formula:

[0114]

[0115] In the formula, <H m,R > A Let represent the sum of the second derivatives of the secret sharing of the right child node; m,L > A This represents the m-th column in a binary permutation matrix; n represents the last column of a binary permutation matrix; d This indicates the number of sample features for the current node.

[0116] Finally, based on the above calculation results and the pre-set linear gain fraction algorithm, the number of linear gain fractions corresponding to the m-th candidate split is calculated. After traversing all M candidate splits, the number of linear gain fractions is obtained. The linear gain fraction algorithm is expressed by the following formula:

[0117] * > A =( <G m,L > A ) 2 ·(2 <h> A -<H m,L > A ) 2 +(<G m,R > A ) 2

[0118] ·(2 <h> A - <H m,R > A ) 2

[0119] In the formula, <G m,L > A This represents the sum of the first derivatives of the secret sharing of the left child node; <h> A This represents the second-order reciprocal sum of the secret sharing of the current node; <H m,L > A This represents the sum of the second derivatives of the secret sharing of the left child node; <G m,R > A Let represent the sum of the first derivatives of the secret sharing of the right child node; <H m,R > A Let represent the sum of the second derivatives of the secret sharing of the right child node.

[0120] Specifically, based on the binary permutation matrix and the addition and multiplication operators for secret sharing, the linear gain distribution of the candidate splits of the current node is collaboratively calculated and secretly shared. This includes: for the m-th candidate split among M candidate splits, the second-order reciprocal sum of the secret sharing of the current node is collaboratively calculated by each training terminal based on the penultimate column of the binary permutation matrix and the addition operator; m is an integer ranging from 1 to M; based on the m-th column and the penultimate column of the binary permutation matrix and the multiplication operator, the first-order derivative sum of the secret sharing of the left child node obtained by the m-th candidate split and the secret sharing of the right child node are collaboratively calculated. The first derivative of the secret share is calculated; based on the m-th column, the penultimate column, and the multiplication operator, the second derivative of the secret share of the left child node and the second derivative of the secret share of the right child node obtained by the m-th candidate split are calculated collaboratively; based on the second derivative of the secret share of the current node, the first derivative of the secret share of the left child node, the first derivative of the secret share of the right child node, the second derivative of the secret share of the left child node, the second derivative of the secret share of the right child node, and the second derivative of the secret share of the right child node, and the linear gain fraction algorithm, the linear gain fraction of the m-th candidate split is calculated collaboratively; after traversing M candidate splits, the linear gain fraction array is obtained.

[0121] Step S104: Using the comparison operator of secret sharing, collaboratively calculate the indicator vector of the best split corresponding to the maximum linear gain fraction in the linear gain fraction array, and store it in the preset secret sharing set.

[0122] In some embodiments of the present invention, for any decision tree model t, each training terminal will collaboratively create a preset secret sharing set <Θ>. t > A It serves as a container for storing information related to the decision tree model.

[0123] Among them, the relevant information of the decision tree model includes the indicator vector of the optimal split of the node.

[0124] In some embodiments of the present invention, in order to screen out candidate splits with the maximum linear gain score without exposing the screening process and results, two iterations are required: a forward iteration based on the comparison operator of secret sharing, and a backward iteration based on the result of the forward iteration, such as... Figure 4 As shown.

[0125] In the forward iteration, the linear gain subarray... Divided into two subsets: left subset and right subset In particular, if If the value is odd, the last item is taken out and placed into a temporary variable χ, and then added to the linear gain subarray of the next round before the next round of execution. middle.

[0126] Compare each of the two subsets element by element and generate the comparison results, which are then stored in a pre-secretly shared binary vector. Then, all participants collaboratively computed the following expression:

[0127]

[0128] This yields the linear gain subarray for the next round of forward iteration. In the formula, This represents the left subset in the forward iteration; A binary vector representing the secret sharing, storing the comparison results; This represents the right subset in the forward iteration.

[0129] When linear gain is divided into groups When only one item remains, the forward iteration stops and the reverse iteration begins. The reverse iteration derives the optimal splitting position from the results of the forward iteration, and its iteration rounds are the same as those of the forward iteration.

[0130] At the start of the first round of reverse iteration, the linear gain subarray and the secret-shared binary vector obtained from the last round of forward iteration have already been obtained. The linear gain array obtained in the last round is represented as follows: <c i+1 > A This is the result of the previous iteration. The key step in the reverse iteration is to calculate the gain fraction (left and right subsets) after splitting. This is done through element-wise multiplication. <c i+1 > A and get <c i,L > A The gain fraction of the left subset can be expressed by the following formula:

[0131]

[0132] In the formula, <c i+1 > A This represents the result of the previous iteration; A binary vector representing the secret sharing, storing the comparison results.

[0133] The gain score of the right subset is obtained by removing the gain score of the left subset from the total gain score, which can be expressed as follows:

[0134] <c i,R > A = <c i+1 > A - <c i,L > A

[0135] In the formula, <c i+1 > A This represents the result of the previous iteration; <c i,L > A This represents the gain fraction of the left subset.

[0136] Will <c i,R > A and <c i,L > A By piecing them together, we obtain <c i > A ,for Indicator vector of the location of the maximum term m > A When the final round is reached, the output is... <c1> A This is the indicator vector for the optimal splitting position.

[0137] Specifically, through a secretly shared comparison operator, the indicator vector corresponding to the optimal split in the linear gain subarray corresponding to the maximum linear gain score is collaboratively calculated and stored in a pre-defined secretly shared set. This includes: using the linear gain subarray as the forward iteration subarray for the current forward iteration round; if the number of linear gain scores in the forward iteration subarray is even, dividing the forward iteration subarray equally from the middle position into a forward iteration left subset and a forward iteration right subset; through each training terminal, comparing the linear gain scores at corresponding positions in the forward iteration left subset and the forward iteration right subset one by one according to the comparison operator, and storing the larger linear gain score in the pre-defined secretly shared gain score vector; based on the gain score vector, the forward iteration left subset, and the forward iteration right subset, calculating a new subarray as the forward iteration subarray for the next forward iteration round. If the number of linear gain fractions in the forward iteration subarray is even, the forward iteration subarray is divided equally from the middle position into a forward iteration left subset and a forward iteration right subset until the number of linear gain fractions in the new subarray is 1. If the number of linear gain fractions in the new subarray is 1, the new subarray is used as the reverse iteration subarray, and the left subset gain fraction is obtained by element-wise multiplication with the gain fraction vector. Based on the reverse iteration subarray and the left subset gain fraction, the right subset gain fraction is calculated. The left subset gain fraction and the right subset gain fraction are concatenated to obtain the reverse iteration subarray for the next reverse iteration round, and the left subset gain fraction is obtained by element-wise multiplication with the gain fraction vector. This process continues until all iteration rounds are completed and an indicator vector is obtained. The number of forward iterations is the same as the number of reverse iterations.

[0138] In addition, if the number of elements in the reverse iteration subarray is greater than the number of elements in the gain fraction vector, the method also includes: removing the excess elements from the reverse iteration subarray and concatenating them into the reverse iteration subarray of the next reverse iteration round.

[0139] If the number of linear gain fractions in the forward iteration subarray is even, the method further includes: taking the last item in the forward iteration subarray and putting it into a temporary variable; before the next forward iteration round is executed, taking the last item from the temporary variable and adding it to the forward iteration subarray of the next forward iteration round.

[0140] Step S105: Based on the number of candidate splits, select the corresponding columns from the binary permutation matrix and calculate the corresponding indicator vectors to obtain the child node samples of the current node through collaborative calculation, and calculate the binary permutation matrix of the child nodes based on the child node samples and the binary permutation matrix.

[0141] The process involves selecting the corresponding columns from the binary permutation matrix and calculating the indicator vector based on the number of candidate splits. The resulting child node samples of the current node are obtained through collaborative computation, and can be expressed as follows:

[0142]

[0143] In the formula, represents the binary permutation matrix of the current node (d nodes in the t decision tree model); M represents the number of candidate splits; m > A Indicates the indicator vector.

[0144] public s > A Get I s I s The set of samples corresponding to all values ​​of 1 is assigned to the left child node of the current node. The remaining samples are divided into the sample set of the right child node. middle.

[0145] The binary permutation matrix of each child node is calculated based on the child node samples and the binary permutation matrix, including the binary permutation matrix of the left child node and the binary permutation matrix of the right child node. The binary permutation matrix of the left child node is expressed as follows:

[0146]

[0147] In the formula, Represents the binary permutation matrix of the current node (d nodes in the t decision tree model); This represents the set of samples of the left child nodes of the current node.

[0148] The binary permutation matrix of the right child node is represented as:

[0149]

[0150] In the formula, Represents the binary permutation matrix of the current node (d nodes in the t decision tree model); This represents the set of samples of the right child nodes of the current node.

[0151] Step S106: Take the child node as the current node, and perform the step of calculating the linear gain subarray of the candidate split of the current node until the split from the root node to the leaf node is completed, and calculate the weight of the leaf node and store it in the preset secret sharing set.

[0152] After splitting all non-leaf nodes, the weight of each leaf node needs to be calculated.

[0153] ​​Specifically, for any leaf node, calculating the leaf node weight includes: extracting the last two columns from the binary permutation matrix corresponding to the leaf node; calculating the weight corresponding to the leaf node based on the last two columns, the sample corresponding to the leaf node, and a preset weighting algorithm; the preset weighting algorithm is expressed as:

[0154]

[0155] In the formula, The sample representing the leaf node; <w t,d > A Indicates the weight of the leaf node; This represents the second-to-last column of the binary permutation matrix; This represents the last column of a binary permutation matrix.

[0156] Step S107: If the number of decision tree models has not reached the preset number, based on the weights of the leaf nodes and the predicted values, update the binary permutation matrix of the root node to obtain the binary permutation matrix of the root node of the next decision tree model. Take the root node of the next decision tree model as the current node and execute the step of calculating the linear gain subarray of the candidate split of the current node until the number of decision tree models reaches the preset number, and output the preset secret sharing set.

[0157] Specifically, based on the leaf node weights and predicted values, the binary permutation matrix of the root node is updated to obtain the binary permutation matrix of the root node of the next decision tree model. This includes: adding the leaf node weights and predicted values ​​through the initiating terminal to obtain the predicted value corresponding to the next decision tree model; calculating the gradient information of the next decision tree model based on the predicted value, true label, and preset loss function of the next decision tree model; and using the gradient information of the next decision tree model as the last two columns of the binary permutation matrix of the first decision tree model to obtain the binary permutation matrix of the root node of a decision tree model.

[0158] The sum of the leaf node weights and the predicted values ​​yields the predicted value for the next decision tree model, which can be expressed by the following formula:

[0159]

[0160] In the formula, <w t,d > A Indicates the weight of the leaf node; This represents the predicted value.

[0161] After all decision tree models are built, the preset secret sharing set {<Θ0> is output. A ,…,<Θ T > A This includes the optimal splitting information for each intermediate node and the weight information for each leaf node.

[0162] In summary, the privacy-preserving vertical federated boosting tree method provided in this embodiment can solve the problem of low efficiency in existing privacy-preserving vertical federated gradient boosting tree protocols. Before model training, random row permutations are performed to anonymize sample identifiers, meaning the training terminal cannot know which samples are included in the sample set of each node. This allows breaking the "computational inadvertence" requirement of existing protocols, significantly reducing computational and communication complexity during training. Furthermore, a linear gain score is provided, which, compared to the original gain score of the gradient boosting tree algorithm, does not introduce division operations. Division operations are quite time-consuming in secure multi-party computation, thus shortening computation time and further improving training efficiency.

[0163] In addition, during the optimal split selection process, the number of comparison operations with high overhead is reduced to a minimum through forward and backward iterations, which can reduce computational complexity and further improve model training efficiency.

[0164] In some embodiments of the present invention, experiments were conducted on training efficiency and prediction accuracy. The dataset information used is shown in Table 1:

[0165] Dataset Feature dimension Sample size Task Adult 104 45222 Two categories Bank 51 45211 Two categories Credit card 91 30000 Two categories Super 784 60000 return

[0166] Table 1

[0167] The list of parameters involved is shown in Table 2:

[0168]

[0169] Table 2

[0170] There are four comparison methods involved:

[0171] 1) Non-private training method for longitudinal federated gradient trees: This method collaboratively constructs gradient boosting trees without considering privacy protection;

[0172] 2) Squirrel: This method is designed for two parties and achieves efficiency improvements by applying secret sharing, fully homomorphic encryption, and unintentional transmission to appropriate operators;

[0173] 3) Random Permutation XGB (CRP-XGB): This method is set up for both parties and reduces the overhead of secret sharing by introducing random permutation;

[0174] 4) Pivot: This method is multi-party set up and combines secret sharing and threshold semi-homomorphic encryption.

[0175] Table 3 shows the comparison results of the training efficiency of the privacy-preserving longitudinal federated boosting tree method (Ours in Table 3) provided in this application with the comparative methods under different parameter settings.

[0176] In particular, the privacy-preserving vertical federated boosting tree method provided in this application is split into initialization and training times, and the training dataset used is artificially generated.

[0177]

[0178] Table 3

[0179] As can be observed from Table 3, the method provided in this application achieves better training efficiency than Squirrel, CRP-XGB, and Pivot under any setting. This is because these comparative methods introduce a large amount of data-independent computation to protect the sample set of each node in the tree, resulting in the computational complexity of each node being related to the total number of samples. In contrast, the method provided in this application avoids the use of data-independent computation by performing safe random permutations before training, thereby minimizing the computational complexity of each node to be related to the number of samples available for that node. Moreover, the method provided in this application reduces the number of costly nonlinear operators during model training by introducing linear gain scoring and a synchronous optimal split selection method.

[0180] The method provided in this application uses a linear gain score, rather than the gain score used in the original gradient boosting tree algorithm. To verify whether the method provided in this application will result in a loss of model accuracy, the accuracy of the method provided in this application and the non-private model under different parameter settings were compared on a real dataset to fully demonstrate the difference in effectiveness between the linear gain score and the original gain score.

[0181] Specifically, the model accuracy metrics used are AUC (Area Under the ROC Curve) and MSE (Mean Squared Error). Figure 5 The comparison results are presented. (By...) Figure 5 It can be observed that, under any parameter settings, the method provided in this application can achieve model accuracy close to that of the non-private model. This indicates that the utility of the linear gain fraction is consistent with that of the original gain fraction.

[0182] Another aspect of the present invention provides a privacy-preserving vertical federated boosting tree apparatus, including a processor, a memory, and a computer program / instructions stored in the memory. The processor executes the computer program / instructions, and when the computer program / instructions are executed, the apparatus implements the steps of the aforementioned privacy-preserving vertical federated boosting tree method. This application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the privacy-preserving vertical federated boosting tree method of the above-described method embodiments.

[0183] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the privacy-preserving vertical federated boosting tree method of the above-described method embodiments.

[0184] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0185] Obviously, the embodiments described above are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, those skilled in the art can make other variations or modifications without creative effort, and all such variations or modifications should fall within the scope of protection of this application. ​< / h> < / h> < / h> ​​< / h> < / h> < / h> < / h> ​

Claims

1. A privacy-preserving vertical federated boosting tree method, characterized in that, The method is applied to a longitudinal federated learning training system constructed by at least two training terminals, wherein the training terminals include an initiating terminal and participating terminals, for jointly constructing a preset number of decision tree models; the method includes the following steps: The initiating terminal combines the local binary matrix, gradient information, and the binary matrix secretly shared by the participating terminal to obtain the sharing matrix of the first decision tree model root node, which is then secretly shared with the participating terminal. The binary matrix is ​​obtained by transforming the left partition subset of the candidate split of the sample features. The gradient information is calculated based on the predicted value of the sample features, the true label, and the preset loss function. Each training terminal performs secret row permutations on the shared matrix based on the pre-shared permutation triples, obtains the permutation results, and secretly shares and splices them together to obtain the binary permutation matrix of the root node. Using the root node as the current node, based on the binary permutation matrix and the secretly shared addition and multiplication operators, collaboratively calculate the linear gain subarray of the candidate split of the current node and secretly share it; By using a secret-sharing comparison operator, the indicator vector of the best split corresponding to the maximum linear gain fraction in the linear gain subarray is collaboratively calculated and stored in a preset secret-sharing set; Based on the number of candidate splits, the corresponding columns are selected from the binary permutation matrix and calculated together with the indicator vector to obtain the child node samples of the current node, and the binary permutation matrix of the child node is calculated based on the child node samples and the binary permutation matrix. The child node is taken as the current node, and the step of calculating the linear gain subarray of the candidate split of the current node is performed until the split from the root node to the leaf node is completed, and the weight of the leaf node is calculated and stored in the preset secret sharing set. If the number of decision tree models has not reached the preset number, the binary permutation matrix of the root node is updated based on the leaf node weights and the predicted values ​​to obtain the binary permutation matrix of the root node of the next decision tree model. The root node of the next decision tree model is taken as the current node, and the step of calculating the linear gain subarray of the candidate split of the current node is executed until the number of decision tree models reaches the preset number, and the preset secret sharing set is output.

2. The method according to claim 1, characterized in that, The number of candidate splits is M; the step of collaboratively calculating and secretly sharing the linear gain subarray of the candidate splits of the current node based on the binary permutation matrix and the secretly shared addition and multiplication operators includes: For the m-th candidate split among the M candidate splits, the second-order reciprocal sum of the secret sharing of the current node is obtained by the collaborative calculation of each training terminal based on the penultimate column of the binary permutation matrix and the addition operator; where m is an integer ranging from 1 to M. Based on the m-th column of the binary permutation matrix, the penultimate column of the binary permutation matrix, and the multiplication operator, the secret shared first derivative sum of the left child nodes obtained by the m-th candidate split, and the secret shared first derivative sum of the right child nodes are calculated collaboratively. Based on the m-th column, the penultimate column, and the multiplication operator, the sum of the second derivatives of the secret sharing of the left child node obtained by the m-th candidate split partition, and the sum of the second derivatives of the secret sharing of the right child node are calculated collaboratively. Based on the sum of the second derivatives of the secret sharing of the current node, the sum of the first derivatives of the secret sharing of the left child node, the sum of the first derivatives of the secret sharing of the right child node, the sum of the second derivatives of the secret sharing of the left child node, the sum of the second derivatives of the secret sharing of the right child node, and the linear gain score algorithm, the linear gain score of the m-th candidate split is calculated collaboratively. After traversing the M candidate splits, the linear gain split array is obtained; The linear gain fractional algorithm is expressed by the following formula: In the formula, the The sum of the first derivatives of the secret sharing of the left child node; This represents the second-order reciprocal sum of the secret sharing of the current node; Let represent the sum of the second derivatives of the secret sharing of the left child node; This represents the sum of the first derivatives of the secret sharing of the right child node; Let represent the sum of the second derivatives of the secret sharing of the right child node.

3. The method according to claim 1, characterized in that, The permutation triplet is represented as follows: , wherein This represents a permutation operation based on secret sharing in a round-robin decomposition, the... This represents a secret sharing matrix, and each training terminal holds a portion of the secret sharing matrix; This represents the permutation matrix obtained after the permutation operation on the secret sharing matrix; Each training terminal performs secret row permutations on the shared matrix based on pre-shared permutation triples, obtains the permutation results, and concatenates them in secret sharing to obtain the binary permutation matrix of the root node, including: Through the various training terminals, the sharing matrix and the secret sharing matrix are added collaboratively using the addition operator based on secret sharing to obtain the matrix after adding the mask; After restoring the masked matrix to plaintext form, the pre-allocated rows are permuted using the permutation operation, while other rows are set to 0, to obtain the local permutation result, which is then secretly shared. After concatenating all the local permutation results, the binary permutation matrix is ​​obtained by adding it to the secret sharing matrix using the addition operator.

4. The method according to claim 3, characterized in that, Before obtaining the binary permutation matrix of the root node by performing secret row permutations on the shared matrix based on the pre-shared permutation triples by each training terminal, obtaining the permutation results and concatenating them in secret sharing, the process further includes: The row to be replaced is assigned to each training terminal by the initiating terminal; The permutation operation is obtained by generating a local permutation locally through each training terminal. ; The secret sharing matrix is ​​generated collaboratively by the various training terminals. ; The secret sharing matrix is ​​cyclically processed using the permutation operations corresponding to each training terminal. Row permutations are performed, and threshold fully homomorphic encryption is used to obtain the encrypted form of the secret sharing matrix. Each training terminal then performs its own permutation operation locally on the encrypted form of the secret sharing matrix, converting the permuted matrix into the secret sharing form. After completing the cyclic permutations, the permutation matrix is ​​obtained. .

5. The method according to claim 4, characterized in that, The phrase "allocating the rows to be replaced by each training terminal through the initiating terminal" means allocating row indices through the initiating terminal according to the number of each training terminal. The permutation operation is obtained by generating a local permutation locally through each training terminal. This includes: randomly shuffling the order of the received row indices at each training terminal, and determining the mapping relationship from the original order to the new order as the permutation operation.

6. The method according to claim 1, characterized in that, Using a secret-shared comparison operator, the indicator vector for the optimal split corresponding to the maximum linear gain fraction in the linear gain fraction array is collaboratively calculated and stored in a preset secret-shared set, including: The linear gain fraction array is used as the forward iteration fraction array for the current forward iteration round. If the number of linear gain fractions in the forward iteration fraction array is even, the forward iteration fraction array is equally divided into a forward iteration left subset and a forward iteration right subset from the middle position. Through each training terminal, the linear gain scores at corresponding positions in the left and right subsets of the forward iteration are compared one by one according to the comparison operator, and the linear gain score with the larger value at the corresponding position is stored in the pre-secretly shared gain score vector. Based on the gain score vector, the left subset of the forward iteration, and the right subset of the forward iteration, a new score array is calculated as the forward iteration score array for the next forward iteration round. If the number of linear gain scores in the forward iteration score array is even, the forward iteration score array is divided equally from the middle position into the left subset of the forward iteration and the right subset of the forward iteration, until the number of linear gain scores in the new score array is 1. When the number of linear gain fractions in the new subarray is 1, the new subarray is used as the reverse iterative subarray, and the element-wise product with the gain fraction vector is performed to obtain the number of gain fractions in the left subset; Based on the reverse iterative subarray and the gain fraction of the left subset, the gain fraction of the right subset is calculated; The gain fractions of the left subset and the gain fractions of the right subset are concatenated to obtain the reverse iteration fraction array for the next reverse iteration round. The step of performing element-wise multiplication with the gain fraction vector to obtain the gain fractions of the left subset is then performed until all iteration rounds are completed, and the indicator vector is obtained. The number of forward iterations is the same as the number of reverse iterations.

7. The method according to claim 6, characterized in that, If the number of elements in the reverse iterative subarray is greater than the number of elements in the gain fraction vector, the method further includes: The redundant elements will be stripped from the reverse iteration subarray and appended to the reverse iteration subarray of the next reverse iteration round.

8. The method according to claim 6, characterized in that, If the number of linear gain fractions in the forward iterative subarray is even, the method further includes: Take the last item from the forward iterative subarray and put it into a temporary variable; Before the next forward iteration round is executed, the last item is taken out from the temporary variable and added to the forward iteration subarray of the next forward iteration round.

9. The method according to claim 1, characterized in that, For any leaf node, the calculation of the leaf node weight includes: Take the last two columns from the binary permutation matrix corresponding to the leaf nodes; Based on the last two columns, the samples corresponding to the leaf nodes, and a preset weighting algorithm, the weights corresponding to the leaf nodes are calculated; the preset weighting algorithm is expressed as follows: In the formula, This represents a sample representing the leaf node; The weight of the leaf node; This refers to the second to last column; the... This indicates the last column.

10. The method according to claim 1, characterized in that, Before combining the local binary matrix, gradient information, and the binary matrix secretly shared by the participating terminal through the initiating terminal to obtain the sharing matrix of the first decision tree model root node and secretly sharing it with the participating terminal, the method further includes: Each training terminal determines its own candidate splits for the sample features it holds, and divides its own sample features into the left partition subset and the right partition subset based on each candidate split. The left subset is transformed to obtain the binary matrix, which is then secretly shared with the initiating terminal. The initiating terminal predicts the sample features based on a preset decision tree model to obtain a predicted value; The predicted value and the sample features are input into the preset loss function to calculate the first-order gradient and the second-order gradient of each sample feature. The first-order gradient and the second-order gradient are then combined into two gradient vectors to obtain the gradient information.

Images