Longitudinal federal random forest model training method, system and device based on Stackelberg game incentive mechanism and medium

Abstract

The invention discloses a longitudinal federal random forest model training method, system and device based on a Stackelberg game incentive mechanism and a medium, and belongs to the field of cyberspace security, and the model training method completes training of a random forest model through four stages of sample sampling, feature sampling, model training and model fusion. A passive party payment strategy is defined through an active party payment upper limit, then benefits of a decision-making tree model are defined, finally an objective function of an active party is calculated, excitation of participants in a federated learning task is achieved through an excitation mechanism of the two-stage Stackelberg game, and the active party can complete the federated learning task by setting a reasonable excitation strategy and objective. The passive party is stimulated to participate and pay data, so that the overall federal learning effect and the model performance are improved; the system, the equipment and the medium are used for implementing the method. The method is high in safety and fairness, and guarantees the performance of a global model.

Description

Technical Field

[0001] The present invention belongs to the technical field of cyberspace security, and particularly relates to a vertical federated random forest model training method, system, device and medium based on a Stackelberg game incentive mechanism. Background Art

[0002] Vertical Federated Learning (VFL) is a privacy-preserving distributed machine learning method for solving the model training problem in the case where multiple parties have data in different feature spaces. Different from Horizontal Federated Learning (HFL), each party in vertical federated learning has similar sample objects, but different data features. This method is applicable to scenarios with more sample overlaps and fewer data feature overlaps. In vertical federated learning, the parties have different feature sets. For example, one party may have the medical records of patients, while another party has the financial data of patients. Due to considerations of data privacy and security, the parties cannot directly exchange the original data.

[0003] For vertical federated learning where data features are distributed among different parties, the current training methods are mainly divided into two categories: one is the vertical federated training method of various basic machine learning models, and the other is the vertical federated neural network training method based on embedding technology. The random forest model in the basic machine learning model is mainly trained by the decision tree model through ensemble learning. At present, there are solutions for vertical federated learning of tree models, but there are still privacy interaction problems in the vertical federated learning solution of its extended random forest model. At the same time, due to the vertical distribution of data features in vertical federated learning, multi-party collaborative training faces many challenges in privacy and accuracy, further testing the collaboration ability of all parties in the collaborative training process.

[0004] Guo Yanqing et al. disclosed a federated decision tree algorithm for privacy security (Guo Yanqing, Wang Xinlei, Fu Haiyan, etc. Federated Decision Tree Algorithm for Privacy Security [J]. Chinese Journal of Computers, 2021, vol. 40, no. 10, pp. 2090-2103.). This solution proposed a decision tree model training algorithm in the vertical federated learning scenario, which meets the privacy training requirements between parties under the vertical data distribution characteristics, but the decision tree model has the problem of overfitting.

[0005] Ge Ning et al. published "Failure Prediction in Production Line Based on Federated Learning: An Empirical Study" (Ge, N., Li, G., Zhang, L. et al. Failure Prediction in Production Line Based on Federated Learning: An Empirical Study[J]. J Intell Manuf, 2022, vol. 33, pp. 2277–2294.). The empirical study on failure prediction of production lines using federated learning proposed in this solution improved the training process of the vertical federated random forest model by introducing feature selection and pruning steps, ensuring that the final model only selects the most contributing features. The research results show that the improved algorithm can obtain results comparable to those of centralized methods in manufacturing failure prediction. However, in the process of constructing the vertical federated random forest model described in this method, it is necessary to broadcast the aligned data IDs between participants in plaintext, which still reveals the local data privacy of the participants and does not meet the privacy attributes of vertical federated learning. Summary of the Invention

[0006] In order to overcome the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a training method, system, device and medium for a vertical federated random forest model based on the Stackelberg game incentive mechanism. By combining the confusion Bloom filter, secret sharing and oblivious transfer protocol, the privacy interaction model training process between participants in vertical federated learning is realized, and the equilibrium solution objectives of the active and passive participants are solved through the utility definition of the active and passive participants in the Stackelberg game modeling, guiding the model training process of the active and passive participants, and finally improving the cooperation enthusiasm of the participants. It has the characteristics of privacy interaction attributes of vertical federated learning, fairness of participant cooperation, and improvement of participant cooperation enthusiasm and global model performance.

[0007] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0008] A training method for a vertical federated random forest model includes a sample sampling stage, a feature sampling stage, a base model training stage and a model fusion stage;

[0009] In the sample sampling stage: before the training of each decision tree starts, the active party AU generates a sample subset S by randomly sampling and aligning the index set, then generates a Bloom filter BF based on the sample subset S, and generates a sub-secret share2 through the (n, n) secret sharing protocol and transmits it to all passive parties PB, PB = {PU i |i ∈ [1, n]}, and each passive party PUi Parse the obtained sub-secret share2 to determine the sample subset S used in this training.

[0010] In the feature sampling stage: In the sample subset S generated in the sample sampling stage, the active party AU and the passive party PU i respectively generate their own subsets of the feature space for the current tree model training through random sampling

[0011] In the base model training stage: Use the Gini index as the evaluation metric for the optimal splitting point, and based on the subset of the feature space generated in the feature sampling stage train a decision tree as the base model in the random forest model and record the feature importance data generated during the base model training;

[0012] In the model fusion stage: Repeat the sample sampling stage, the feature sampling stage, and the base model training stage, and based on the feature importance data generated during the base model training and the incentive mechanism of the Stackelberg game, train T decision tree base models to form a random forest model;

[0013] Traverse the decision tree base models in the random forest model to obtain the prediction results of the decision tree base models, and then perform weighted voting on the prediction results of the decision tree base models to obtain the prediction result of the random forest model, which is the final classification prediction result.

[0014] The sample sampling stage specifically includes the following steps:

[0015] Step 1.1: The active party AU randomly samples to obtain the sample subset S, and then creates a Bloom filter BF, specifically as follows: First, initialize the Bloom filter BF to a vector of all 0s with a size of q, select k independent hash functions {h1, h2,..., h k}, each hash function maps the input value to [1, q], and then insert each sample index s (s ∈ S) obtained by sampling into the Bloom filter BF. For each sample index s, there is:

[0016] BF[h1(s)] = 1, BF[h2(s)] = 1,..., BF[h k (s)] = 1 (1);

[0017] Step 1.2: The active party AU performs secret sharing on the q bits of the Bloom filter BF created in Step 1.1, generates a random number sub-secret share1[i] bit by bit, and calculates the shared sub-secret share2[i] of the random number sub-secret share1[i] as:

[0018] share2[i] = BF[i] - share1[i] mod p (2)

[0019] where p is the modulus;

[0020] Step 1.3: The active party AU sends the random vector share1 composed of the random number sub - secrets share1[i] generated in Step 1.2 and the shared vector share2 composed of the shared secrets share2[i] calculated in Step 1.2 to all passive parties PB through a two - of - one oblivious transfer protocol, specifically as follows: The active party AU generates two q - bit random vectors Rand1 and Rand2; The passive party PU B calculates its own sample set X B of the sample x i of the hash values h1(i), h2(i), …, h k (i) of the index i, sets the elements at these k positions to 1, and the rest to 0 to generate a selection vector τ;

[0021] Then, run the two - of - one oblivious transfer protocol. When τ[i] = 1, share j '[i] = share j [i]; when τ[i] = 0, share j '[i] = Rand j [i], where j = 1, 2;

[0022] Step 1.4: The passive party PU B receives the random vector share1' and the shared vector share2' after the transfer protocol in Step 1.3, and calculates the verification value:

[0023] BF'[h j (i)] = share1'[h j (i)] + share2'[h j (i)] mod p, j ∈ [1, k] (3)

[0024] If the k - element bits BF'[h i (i)] = 1 (j ∈ [1, k]) of the sample x j , then it is considered that the sample x i ∈S, and thus the sample subset S selected by the active party AU is recovered.

[0025] The specific steps of the basic model training stage are as follows:

[0026] The decision tree is constructed using the Gini index, specifically: The label set of the data held by the active party AU is C, and the probability that the sample data belongs to class c is p c , where c ∈ C, then the Gini index of the probability distribution is defined as:

[0027]

[0028] Select the feature splitting point with the maximum gain as the optimal splitting point through formula (5), and find the optimal splitting point of the vertical federated decision tree model through private interaction. For a certain leaf node of the decision tree, the Gini index Gini(p) represents the impurity of the sample data assigned to this leaf node. The lower the Gini index Gini(p), the lower the probability that the labels of two randomly selected samples in the sample set of this leaf node are different. And the gain of this leaf node represents the change in the Gini index after splitting at this leaf node, that is:

[0029] gain = Gini(p) - ΣGini(p i ) (5)

[0030] where p i represents the sample probability distribution of the child nodes after the leaf node is split. Then the gain of a certain splitting point of the binary decision tree is:

[0031]

[0032] where Gini l (p l ) and Gini r (p r ) represent the Gini indices of the left and right child nodes respectively; |D| represents the number of samples at the splitting point, |D l | and |D r | represent the number of samples of the left and right child nodes respectively. When the gain gain is greater than the splitting threshold Δ, continue to split until all nodes cannot be split, and the construction of the decision tree base model is completed.

[0033] Find the optimal splitting point of the vertical federated decision tree model through private interaction. The process is as follows:

[0034] Step 3.1: The active party AU constructs the active party federated histogram in the current state where |A l | represents the number of samples in which the feature A belongs to the label l, and t represents the number of bins of the attribute A;

[0035] The passive party PU i First constructs the passive party collaborative histogram in the current state represents dividing the attribute B of the passive party PU i into T bins, where B l represents the index set of c categories included in the l-th bin; then the passive party PU iGenerate a garbled Bloom filter GBF corresponding to the collaborative histogram using the (n,n) secret sharing protocol. The garbled Bloom filter GBF stores the index IDs of the aligned data samples and provides the ability to perform private set intersection.

[0036] Step 3.2: The active party AU and the passive party PU i Use the oblivious transfer protocol to perform private set intersection on the information in the garbled Bloom filter GBF generated in Step 3.1. The active party AU obtains the garbled Bloom filter GBF' that only contains the intersection of both parties, while the passive party PU i does not obtain any other information.

[0037] Step 3.3: The active party AU constructs a complete federated histogram based on the private set intersection result in Step 3.2 And calculate the optimal splitting point sp according to the Gini index, and the optimal splitting feature Attr of the optimal splitting point sp sp The cumulative result of the Gini index is used as the feature importance Σ of the participating party P i , that is, as the feature importance Σ of the active party AU and the passive party PU i , that is, as the feature importance Σ of the active party AU and the passive party PU i (i = [1,n]), in the t-th decision tree base model, the feature importance Σ of the participating party P i (i = [1,n+1]), in the t-th decision tree base model, the feature importance Σ of the participating party P i is calculated as shown in formula (7): i The calculation method is as shown in formula (7):

[0038]

[0039] where θ i,j represents the importance of the feature X of the participating party P i , and is calculated using the Mean Decrease Impurity (MDI) method. The importance of the feature is measured by statistically calculating the decrease in impurity when the node splits, that is, the Gini index gain of the entire base model f i,j with respect to this feature, which is expressed as formula (8): t The calculation method is as shown in formula (8):

[0040]

[0041] Step 3.4: When the optimal splitting feature Attr sp obtained in Step 3.3 is in the feature space X of the active party n+1 , that is, Attr sp ∈X n+1, the active party AU constructs a confused Bloom filter GBF_R for the new right node, records the aligned sample IDs in the right node, and transmits them to the passive party PB through a two-choice oblivious transfer protocol. The passive party PB determines the set of sample IDs of the current new node. When the optimal splitting feature Attr calculated in step 3.3 sp in the feature space X of the passive party i is, that is, Attr sp ∈X i , i ∈ [i, n], the active party AU sends the ID of the corresponding attribute confused Bloom filter GBF, and the passive party PB constructs the splitting point of the decision tree according to the result, and constructs the confused Bloom filter GBF_R of the new right node according to the splitting situation, records the aligned sample IDs in the right node, and transmits them to the active party ΔU through a two-choice oblivious transfer protocol. The active party AU determines the set of sample IDs of the current new node;

[0042] Step 3.5: Through steps 3.1 to 3.4, find the optimal splitting point round by round until all leaf nodes meet the conditions for stopping splitting. So far, the training of a decision tree base model is completed.

[0043] In the classification prediction of the model fusion stage, for the T decision trees constructed, the prediction result of each tree is f i (x), then the final prediction result is:

[0044]

[0045] where Ι is the indicator function, c is the candidate category, and α i is the weight of the voting results of each decision tree.

[0046] The incentive mechanism specifically includes the following steps:

[0047] First, define the payment ceiling B of the active party AU, and on this basis, define the feature payment strategy of the passive party PB; for any passive party PU i , the active party AU pays the passive party PU i the reward B i is expressed as formula (10):

[0048]

[0049] where Σ i is the feature importance, representing the contribution of the passive party PU i to the model training, which is calculated by formula (7) in the base model training stage of vertical random forest modeling;

[0050] When constructing the t-th decision tree, the passive party PU iThe revenue is the reward paid by the active party AU for the feature budget contributed by its training. As shown in formula (11):

[0051]

[0052] The passive party PU i The cost is a linear function related to its feature importance Σ i That is, formula (12), where l i Represents the unit cost value of the passive party PU i ;

[0053]

[0054] Combining formula (11) and formula (12), the utility function of the passive party PU i Is expressed as formula (13): That is:

[0055]

[0056] Finally, the objective function of the passive party PU i Is expressed as formula (14):

[0057]

[0058] Secondly, the benefit of the decision tree model CT (t) Is expressed as formula (15):

[0059]

[0060] Among them, CT (t) Represents the t-th decision tree base model, n represents the number of passive parties, X i Represents the feature set of the passive party PD i In particular, Represents the importance of the j-th feature of the passive party PU i ; Represents the participating party P i In the base classifier CT (t) The sum of the feature importances, ω is the weight coefficient, and

[0061] When the budget of the active party AU in the t-th decision tree base classifier CT (t) Is R (t) The utility function of the active party is expressed as shown in formula (16):

[0062]

[0063] Among them, α represents the model performance weight, which stipulates the upper limit of the system performance output, and ω determines the speed of the marginal diminishing of the model performance;

[0064] The objective function of the final active party is formula (17):

[0065]

[0066] Based on the Stackelberg game-based incentive mechanism, the active party AU is regarded as the leader, and all passive parties PB are regarded as followers. The training process of the vertical federated random forest model is divided into two stages, namely the master-slave dynamic game between the active party AU and the passive parties PB and the non-cooperative game between the passive parties PB, and finally an equilibrium is formed. First, solve the non-cooperative game between the passive parties PB to determine the feature importance target of each passive party PU i Then, turn to the master-slave dynamic game stage between the active party AU and the passive parties PB, and find the optimal incentive reward by maximizing the benefit of the active party AU;

[0067] In the master-slave dynamic game stage between the active party AU and the passive parties PB, the active party AU first determines its own incentive reward parameter; then, under this constraint, each passive party PU i determines an optimal feature importance target to maximize its benefit; the optimal feature importance target of the passive party satisfies formula (18): satisfies formula (18):

[0068]

[0069] Among them,

[0070] As the leader, after observing the optimal feature importance target of each passive party PU i in the non-cooperative game stage between the passive parties PB, the active party AU determines an optimal incentive reward for the master-slave dynamic game stage between the active party AU and the passive parties PB, which satisfies formula (19):

[0071]

[0072] A random forest training system, including:

[0073] Sample sampling module: Before the training of each decision tree starts, the active party AU generates a sample subset S by randomly sampling and aligning the index set, then generates a Bloom filter BF based on the sample subset S, and generates a sub-secret share2 through the (n,n) secret sharing protocol and transmits it to all passive parties PB. Each passive party PU i parses the obtained sub-secret share2 to determine the sample subset S used in this training;

[0074] Feature Sampling Module: In the sample subset S generated by the Sample Sampling Module, the active party AU and the passive party PU i respectively generate their respective subsets of the feature space for the current tree model training through random sampling

[0075] Base Model Training Module: Using the Gini index as the evaluation metric for the optimal split point, based on the subset of the feature space generated by the Feature Sampling Module train a decision tree as the base model in the random forest model and record the feature importance data generated during the base model training;

[0076] Model Fusion Module: Repeat the Sample Sampling Module, the Feature Sampling Module, and the Base Model Training Module, and based on the feature importance data generated during the base model training and the incentive mechanism of the Stackelberg game, train T decision tree base models to form a random forest model;

[0077] Traverse the decision tree base models in the random forest model to obtain the prediction results of the decision tree base models, and then perform weighted voting on the prediction results of the decision tree base models to obtain the prediction results of the random forest model, which are the final classification prediction results.

[0078] A vertical federated random forest training device, comprising:

[0079] Memory: Used to store a computer program for implementing a vertical federated random forest model training method;

[0080] Processor: Used to implement a vertical federated random forest model training method when executing the computer program.

[0081] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a vertical federated random forest model training method are implemented.

[0082] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0083] 1. The vertical federated random forest modeling method disclosed by the present invention can effectively alleviate the overfitting problem of the vertical federated decision tree algorithm by establishing a random forest model. At the same time, by using the Bloom filter, secret sharing, and oblivious transfer protocol, the data privacy of the participating parties in the training interaction process is maintained, meeting the privacy requirements of vertical federated learning.

[0084] 2. The incentive mechanism of the present invention is based on the vertical federated random forest modeling method, adopts the Stackelberg game model, improves the utility levels of the active and passive participants, effectively enhances the enthusiasm of each participant in participating in federated learning, encourages resource sharing and cooperation among the participants, optimizes resource allocation, thereby giving play to the advantages of the data of all parties, and thus improving the fairness of the entire system.

[0085] 3. The incentive mechanism of the present invention is based on the vertical federated random forest modeling method, and guides the model training process of the participants through negotiation and game, thereby improving the performance of the global model.

[0086] In summary, the present invention has high security and fairness, and improves the performance of the global model. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 is the flowchart of the vertical federated random forest modeling method of the present invention.

[0088] Figure 2 is the basic framework diagram of the adaptive feature incentive mechanism of the vertical federated random forest of the present invention.

[0089] Figure 3 is the utility graph of the passive party in the simulation experiment of the present invention.

[0090] Figure 4 is the utility graph of the active party in the simulation experiment of the present invention.

[0091] Figure 5 is the comparison graph of the utility of the passive party in the simulation experiment of the present invention.

[0092] Fig. 6(a) is the comparison graph of the utility of the active party under the α parameter.

[0093] Fig. 6(b) is the comparison graph of the utility of the active party under the ω parameter. DETAILED DESCRIPTION OF THE INVENTION

[0094] The present invention will be described in detail below with reference to the accompanying drawings.

[0095] The notations used in the present invention are shown in Table 1.

[0096] Table 1 Notation Explanation

[0097]

[0098]

[0099] As Figure 1 shown, a vertical federated random forest model training method proposed by the present invention mainly includes two types of entities, namely the active party AU and the passive party PB. The active party AU is responsible for providing the training data including the feature part and the label part in vertical federated learning where X n+1 is the feature data, Y is the label, n represents the number of passive parties, and d n+1 represents the feature dimension, and the active party assumes the role of coordinator and is responsible for calculating the optimal splitting points of all participating party nodes. The passive party PU i provides the training data required for vertical federated learning where m represents the number of samples and d i represents the feature dimension.

[0100] A method for training a vertical federated random forest model, including a sample sampling stage, a feature sampling stage, a base model training stage, and a model fusion stage;

[0101] In the sample sampling stage: Before the training of each decision tree starts, the active party AU generates a sample subset S by randomly sampling and aligning the index set, then generates a Bloom filter BF based on the sample subset S, and generates a sub-secret share2 through the (n,n) secret sharing protocol and transmits it to all passive parties PB, where PB = {PU i | i ∈ [1,n]}, and each passive party PU i parses the obtained sub-secret share2 to determine the sample subset S used in this training;

[0102] In the feature sampling stage: In the sample subset S generated in the sample sampling stage, the active party AU and the passive party PU i respectively generate their own feature space subsets for the training of the current tree model through random sampling

[0103] In the base model training stage: The Gini index is used as the evaluation index for the optimal splitting point, and based on the feature space subset generated in the feature sampling stage train a decision tree as the base model in the random forest model and record the feature importance data generated during the base model training;

[0104] In the model fusion stage: Repeat the sample sampling stage, the feature sampling stage, and the base model training stage, and based on the feature importance data generated during the base model training and the incentive mechanism of the Stackelberg game, train T decision tree base models to form a random forest model;

[0105] Traverse the decision tree base models in the random forest model to obtain the prediction results of the decision tree base models, and then perform weighted voting on the prediction results of the decision tree base models to obtain the prediction result of the random forest model, which is the final classification prediction result.

[0106] The sample sampling stage specifically includes the following steps:

[0107] Step 1.1: The active party AU randomly samples to obtain a sample subset S, and then creates a Bloom filter BF as follows: First, initialize the Bloom filter BF as a vector of all 0s with size q, select k independent hash functions {h1, h2, …, h k}, each hash function maps the input value to [1, q], and then inserts each sampled sample index s (s ∈ S) into the Bloom filter BF. For each sample index s, there is:

[0108] BF[h1(s)] = 1, BF[h2(s)] = 1, … BF[h k (s)] = 1 (1);

[0109] Step 1.2: The active party AU secret - shares the q bits of the Bloom filter BF created in Step 1.1, generates a random sub - secret share1[i] bit - by - bit, and calculates the shared sub - secret share2[i] of the random sub - secret share1[i] as:

[0110] share2[i] = BF[i] - share1[i] mod p (2)

[0111] where p is the modulus;

[0112] Step 1.3: The active party AU sends the random vector share1 composed of the random sub - secrets share1[i] generated in Step 1.2 and the shared vector share2 composed of the shared secrets share2[i] calculated in Step 1.2 to all passive parties PB through a two - out - of - one oblivious transfer protocol as follows: The active party AU generates two q - bit random vectors Rand1 and Rand2; The passive party PU B calculates the hash values h1(i), h2(i), …, h B of the index i of the sample x i in its own sample set X k , sets the elements at these k positions to 1, and the rest to 0 to generate a selection vector τ;

[0113] Then, run the two - out - of - one oblivious transfer protocol. When τ[i] = 1, share j '[i] = share j [i]; When τ[i] = 0, share j '[i] = Rand j [i], where j = 1, 2;

[0114] Step 1.4: The passive party PU BReceive the random vector share1' and the shared vector share2' after the step 1.3 transmission protocol, and calculate the verification value:

[0115] BF'[h j (i)] = share1'[h j (i)] + share2'[h j (i)] mod p, j ∈ [1, k] (3)

[0116] If the k element bits of the sample x i BF'[h j (i)] = 1 (j ∈ [1, k]), then the sample x i ∈ S, and thus recover the sample subset S selected by the active party AU.

[0117] The basic model training stage specifically includes the following steps:

[0118] The decision tree is constructed using the Gini index, specifically: The label set of the data held by the active party AU is C, and the probability that the sample data belongs to class c is p c , where c ∈ C, then the Gini index of the probability distribution is defined as:

[0119]

[0120] Select the feature splitting point with the largest gain as the optimal splitting point through Equation (5), and find the optimal splitting point of the vertical federated decision tree model through private interaction. For a certain leaf node of the decision tree, the Gini index Gini(p) represents the impurity of the sample data assigned to this leaf node. The lower the Gini index Gini(p), the lower the probability that the labels of two samples randomly drawn from the sample set of this leaf node are different. And the gain of this leaf node represents the change in the Gini index after splitting at this leaf node, that is:

[0121] gain = Gini(p) - ΣGini(p i ) (5)

[0122] where p i represents the sample probability distribution of the child nodes after the leaf node is split, then the gain of a certain splitting point of the binary decision tree is:

[0123]

[0124] where Gini l (p l ) and Gini r (p r ) respectively represent the Gini indices of the left and right child nodes; |D| represents the number of samples at the splitting point, |Dl |and|D r |respectively represent the number of samples of the left and right child nodes; when the gain is greater than the splitting threshold Δ, continue to split until all nodes cannot be split, and the construction of the decision tree base model is completed.

[0125] Find the optimal splitting point of the vertical federated decision tree model through private interaction, and the process is as follows:

[0126] Step 3.1: The active party AU constructs the active party federated histogram in the current state where |A l | represents the number of samples in which the feature A belongs to the label l, and t represents the number of bins of the attribute A;

[0127] The passive party PU i First constructs the passive party collaborative histogram in the current state represents dividing the attribute B of the passive party PU i into T bins, where B l represents the index set of c categories contained in the l-th bin; then the passive party PU i uses the (n,n) secret sharing protocol to generate the confused Bloom filter GBF corresponding to the collaborative histogram. The confused Bloom filter GBF stores the index IDs of the aligned data samples and provides the ability of private set intersection;

[0128] Step 3.2: The active party AU and the passive party PU i Adopt the one-out-of-two oblivious transfer protocol to perform private set intersection on the information in the confused Bloom filter GBF generated in Step 3.1. The active party AU obtains the confused Bloom filter GBF' that only contains the intersection of both parties, while the passive party PU i does not obtain any other information;

[0129] Step 3.3: The active party AU constructs a complete federated histogram according to the private set intersection result in Step 3.2 And calculates the optimal splitting point sp according to the Gini index. The cumulative Gini index result of the optimal splitting feature Attr sp is used as the feature importance Σ of the participating party P i , that is, as the feature importance Σ of the active party AU and the passive party PU i (i = [1,n]), and in the t-th decision tree base model, the feature importance Σ of the participating party P i (i = [1,n+1]) is calculated as shown in formula (7): i (i = [1,n+1]), in the t-th decision tree base model, the feature importance Σ of the participating party P i is calculated as shown in formula (7): i The calculation method is as shown in formula (7):

[0130]

[0131] Among them, θ i,j represents the importance of feature X i of participant P i,j which is calculated by the Mean Decrease Impurity (MDI) method. The importance of the feature is measured by statistically counting the decrease in impurity when the node splits, that is, the Gini index gain of this feature in the entire base model f t is expressed by formula (8):

[0132]

[0133] Step 3.4: When the optimal splitting feature Attr sp is in the active party's feature space X n+1 , that is, Attr sp ∈X n+1 , the active party AU constructs a new right-node's confused Bloom filter GBFR, records the aligned sample IDs in the right node, and transmits them to the passive party PB through the oblivious transfer protocol of choosing one from two. The passive party PB determines the sample ID set of the current new node. When the optimal splitting feature Attr sp is in the passive party's feature space X i , that is, Attr sp ∈X i , i ∈ [i, n], the active party AU sends the id of the corresponding attribute confused Bloom filter GBF, and the passive party PB constructs the splitting point of the decision tree according to the result, and constructs a new right-node's confused Bloom filter GBFR according to the splitting situation, records the aligned sample IDs in the right node, and transmits them to the active party AU through the oblivious transfer protocol of choosing one from two. The active party AU determines the sample ID set of the current new node;

[0134] Step 3.5: Through Steps 3.1 to 3.4, the optimal splitting point is searched for round by round until all leaf nodes meet the conditions for stopping splitting. Thus, the training of a decision tree base model is completed.

[0135] In the classification prediction of the model fusion stage, for the T decision trees constructed, the prediction result of each tree is f i (x), then the final prediction result is:

[0136]

[0137] Among them, Ι is the indicator function, c is the candidate category, and α i is the weight of the voting results of each decision tree.

[0138] The above steps implement the training and prediction process of the vertical federated random forest, ensuring the privacy protection of the participating parties' data and the performance improvement of the model training results during the model training process.

[0139] In the process of constructing the vertical federated random forest model, multiple participating parties need to train the model in the form of multi-role cooperation. When all parties are honest, trustworthy, and actively participate, the effect of cooperative training can be effectively guaranteed. However, in practical applications, there are often trust issues among the participating parties, resulting in their reservation during the cooperative training process and not fully devoting themselves to training. This situation requires the design of an effective incentive mechanism to encourage all parties to actively participate in cooperative training, so as to achieve the training effect under the data aggregation mode in traditional machine learning as much as possible while protecting the privacy of the local private data of all parties.

[0140] The incentive mechanism specifically includes the following steps:

[0141] First, define the payment ceiling B of the active party AU, and on this basis, define the feature payment strategy of the passive party PB; for any passive party PU i , the active party AU pays the passive party PU i 's reward B i is expressed as formula (10):

[0142]

[0143] where, Σ i is the feature importance, representing the model training contribution provided by the passive party PU i , which is calculated by formula (7) in the basic model training stage of vertical random forest modeling;

[0144] When constructing the t-th decision tree, the benefit of the passive party PU i is the reward paid by the active party AU for the feature budget of its training contribution as shown in formula (11):

[0145]

[0146] The cost of the passive party PU i is a linear function related to its feature importance Σ i , that is, formula (12), where l i represents the unit cost value of the passive party PU i ;

[0147]

[0148] Combining formula (11) and formula (12), the utility function of the passive party PU i Expressed as Equation (13):

[0149]

[0150] The final passive party PU i The objective function is expressed as Equation (14):

[0151]

[0152] Secondly, the benefit of the decision tree model CT (t) is expressed as Equation (15):

[0153]

[0154] where CT (t) represents the t-th decision tree base model, n represents the number of passive parties, and X i represents the feature set of the passive party PD i In particular, represents the importance of the j-th feature of the passive party PU i , represents the sum of the feature importances of the participating party P i in the base classifier CT (t) . ω is the weight coefficient, and

[0155] When the budget of the active party AU in the t-th decision tree base classifier CT (t) is R (t) , the utility function of the active party is expressed as shown in Equation (16):

[0156]

[0157] where α represents the model performance weight, which is the upper limit of the system performance output by convention, and ω determines the rate of marginal decrease in model performance;

[0158] The objective function of the final active party is Equation (17):

[0159]

[0160] Such as Figure 2As shown, in order to motivate the passive party PU to participate in the federated learning system, the present invention designs an incentive mechanism based on Stackelberg game, regarding the active party AU as the leader and all passive parties PB as the followers. The training process of the vertical federated random forest model is divided into two stages, namely the master-slave dynamic game between the active party AU and the passive parties PB and the non-cooperative game among the passive parties PB, and finally an equilibrium is formed. For simplicity, the cost expenditures (such as communication costs and computing overheads) of the participating party P in the system are ignored. Since these costs are constants in the present invention, deleting them will not affect the effectiveness of the present invention.

[0161] Next, the above two-stage Stackelberg game is solved to calculate the optimal equilibrium solution. First, the non-cooperative game among the passive parties PU is solved to determine the feature importance target of each passive party PU i and then, turning to the master-slave dynamic game stage between the active party AU and the passive parties PB, the best incentive reward is found by maximizing the benefit of the active party AU;

[0162] In the master-slave dynamic game stage between the active party AU and the passive parties PB, the active party AU first determines its own incentive reward parameter; then, under this constraint, each passive party PU i determines an optimal feature importance target to maximize its benefit; according to the reward calculation method of formula (10), the reward obtained by each passive party PU i increases as its feature importance increases; therefore, each passive party PU i increases its benefit by setting a larger feature importance target. However, when all passive parties PB try to increase their feature importance targets, a non-cooperative game will be formed. In this regard, it is necessary to ensure that the optimal feature importance targets of all passive parties PB generated in the second stage of the Stackelberg game form a Nash equilibrium, that is, no participating party can obtain additional benefits by unilaterally changing its feature importance target. In this way, we can ensure that the optimal feature importance target is the optimal solution of the entire game. In this regard, the optimal feature importance target of the passive party satisfies equation (18): Satisfies equation (18):

[0163]

[0164] where,

[0165] As the leader, after observing the optimal feature importance target of each passive party PU in the non-cooperative game stage among the passive parties PB, the active party AU determines an optimal incentive reward for the master-slave dynamic game stage between the active party AU and the passive parties PB, which satisfies equation (19): i Satisfies equation (19):

[0166]

[0167] Among them, the calculation and derivation processes of formulas (18) and (19) are as follows:

[0168] Theorem 1: For any passive party in the game process Its optimal feature importance target Satisfies formula (20).

[0169]

[0170] Proof 1:

[0171] First, calculate the partial derivative of the benefit function of the passive party with respect to the feature importance target :

[0172]

[0173] Let It can be obtained that:

[0174]

[0175] Repeat the above process for all passive parties and add formula (22), and it can be obtained that:

[0176]

[0177] Among them,

[0178] When , obviously holds. Therefore, from formula (23), it can be obtained that:

[0179]

[0180] Substitute formula (24) into formula (22) to get:

[0181]

[0182] Subtract formula (25) from formula (24) to get:

[0183]

[0184] At this time, the optimal benefit of the passive party is:

[0185]

[0186] Theorem 2: For the active party in the game process, its optimal incentive reward (R(t) ) * Satisfies formula (28).

[0187]

[0188] Proof 2:

[0189] Calculate the partial derivative of the benefit function of the active party with respect to the incentive reward R according to formula (16) (t) to obtain:

[0190]

[0191] Let The optimal incentive reward of the active party can be obtained as formula (30):

[0192]

[0193] To verify that formula (28) is the optimal solution of formula (17), calculate the second-order partial derivative of the benefit function of the active party with respect to the incentive reward R (t) to obtain:

[0194]

[0195] Therefore, formula (28) is the optimal solution of formula (17), that is, formula (28) holds. At this time, the benefit of the active party AU is as shown in formula (32), and this benefit is not less than 0;

[0196]

[0197] A random forest training system, comprising:

[0198] Sample sampling module: Before the training of each decision tree starts, the active party AU generates a sample subset S by randomly sampling and aligning the index set, and then generates a Bloom filter BF based on the sample subset S, and generates a sub-secret share2 through the (n, n) secret sharing protocol and transmits it to all passive parties PB, and each passive party PU i parses the obtained sub-secret sgare2 to determine the sample subset S used in this training, so as to implement the sample sampling stage of a vertical federated random forest model training method;

[0199] Feature sampling module: In the sample subset S generated by the sample sampling module, the active party AU and the passive party PU i respectively generate their own feature space subsets for the current tree model training to implement the feature sampling stage of a vertical federated random forest model training method;

[0200] Base model training module: Using the Gini index as the evaluation metric for the optimal splitting point, based on the subset of the feature space generated by the feature sampling module Train a decision tree as the base model in the random forest model, used to implement the base model training stage of a vertical federated random forest model training method, and record the feature importance data generated during the base model training;

[0201] Model fusion module: Repeat the sample sampling module, feature sampling module, and base model training module, and based on the feature importance data generated during the base model training and the incentive mechanism of the Stackelberg game, train T decision tree base models to form a random forest model;

[0202] Traverse the decision tree base models in the random forest model to obtain the prediction results of the decision tree base models, and then perform weighted voting on the prediction results of the decision tree base models to obtain the prediction results of the random forest model, which is the final classification prediction result, used to implement the model fusion stage of a vertical federated random forest model training method.

[0203] A vertical federated random forest training device, including:

[0204] Memory: Used to store the computer program for implementing a vertical federated random forest model training method;

[0205] Processor: Used to implement a vertical federated random forest model training method when executing the computer program.

[0206] A computer-readable storage medium, the computer-readable storage medium stores a computer program, and the computer program implements the steps of a vertical federated random forest model training method when executed by a processor.

[0207] Simulation experiment

[0208] The following part verifies the effectiveness of the present invention through simulation experiments. The simulation experiment settings include 1 active party AU and n passive parties PU, and the performance of the proposed incentive mechanism is analyzed from the aspects of utility function settings and utility effects. Use BreastCancer and UCI_HAR as benchmark datasets, and divide the datasets into a training set and a validation set according to a ratio of 8:2. At the same time, the number of features owned by the active and passive parties AU is divided according to a ratio of 1:1.

[0209] (1) Rationality experiment

[0210] To verify the rationality of the passive party's utility function design, when the passive party's feature cost l i is fixed, explore the influence of the passive party's selection of different feature importance budgets Σ i on the passive party's utility. To simplify the experiment, uniformly set the cost parameter l of the passive party PUi = 0.1. During the non - cooperative game process among passive PUs, the feature selection of other passive PUs will affect the utility of the current passive PU. To eliminate the influence of other passive PUs, the feature importance budget of other passive PUs is fixed as Σ j =(Σ j ) * , j ∈ [1, n], j ≠ i.

[0211] The influence of feature importance on the passive - party utility is as Figure 3 shown. It can be seen that as the feature importance Σ i of the passive party increases, its utility also increases. When the passive party selects the optimal feature importance Σ i =(Σ i ) * , the utility of the passive party reaches the maximum value. When Σ i >(Σ i ) * , the further increase in feature importance instead leads to a decrease in the passive - party utility. This is because although the increase in feature importance can improve the model performance in the short term, this improvement also has a bottleneck. At the same time, due to the increase in feature importance, the model shows over - fitting phenomenon, resulting in a decline in model performance. Therefore, after the feature importance exceeds the maximum value, the improvement in model performance brought by the increase in feature importance cannot balance the cost increase caused by the passive - party feature expenditure, thus reducing its utility.

[0212] To verify the rationality of the active - party utility function design, under the condition that the feature importance Σ j of the passive party is certain, consider the influence of the active party's selection of different feature payment upper limits R (t) on the active - party utility. In the experiment of active - party utility analysis, set the active - party utility parameters α = 50, ω = 0.1. Since the passive - party feature selection will also affect the active - party utility, in this experiment, it is assumed that the feature cost parameters of all passive parties are l i = 0.1. When the active party gives the feature payment upper limit R (t) , the passive party selects its optimal feature target by optimizing its own utility, that is, Σ j =(Σ j ) * .

[0213] The change of the active - party utility with the feature payment upper limit R (t) is as Figure 4As shown. It can be observed from the figure that as the upper limit of the active party's feature payment increases, the utility of the active party rises rapidly accordingly. After reaching the maximum value, the utility of the active party decreases slowly as the upper limit of the feature payment increases. This is because the increase in the upper limit of the active party's feature payment can motivate the passive party to provide better features, thereby improving the model performance and increasing the active party's revenue. However, an excessively high upper limit of the feature payment will significantly increase the cost of the active party. At the same time, there is also a marginal effect on the improvement of the model performance by the importance of the passive party's features, that is, the improvement of the model performance gradually tends to saturation and the performance improvement is no longer obvious. As a result, the revenue growth is slow in the later stage while the cost continues to rise, and the utility of the active party finally shows a downward trend.

[0214] Comprehensively Figure 3 and Figure 4 It can be known that the utility functions of both the active party and the passive party are strictly concave functions within their value spaces, and there is a unique optimal solution. This result indicates that the design of the utility function of the present invention is reasonable.

[0215] (2) Effectiveness experiment

[0216] To evaluate the passive party's utility of the feature incentive method proposed by the present invention, the random method, and the static method under different numbers n of passive parties, experiments were conducted using the UCI_HAR dataset. The experimental settings are as follows: The model revenue coefficients are set as α = 100, ω = 0.8 respectively, and the upper limit of the active party's feature payment is B = 2. Assume that the cost of the passive party is fixed at l i = 0.1. When the passive party number parameter n = 2, 4, 9, the passive party's utility of the feature pricing incentive method, the random pricing method, and the static pricing method in this paper is as Figure 5 shown.

[0217] From Figure 5 it can be seen that under the same settings, the passive party's utility of the feature adaptive pricing method proposed in this paper is higher than that of the random pricing method and the static pricing method. And as the number of passive parties decreases, the passive party's utility of the method in this paper increases. This is because the random pricing method and the static pricing method randomly or fixedly select feature targets, and the gap between the selected feature budget and the optimal feature budget is relatively large, and the corresponding passive party's utility is relatively low. In contrast, the feature adaptive pricing method makes feature payment decisions based on the equilibrium solution of feature pricing and the budget capabilities of all parties learned in the early stage. This method can more accurately approach the optimal feature budget, thereby improving the passive party's utility.

[0218] For the active party, we respectively evaluate the active party's utility of the method in this paper, the random method, and the static method in the experimental environment of the BreastCancer dataset. The cost of the passive party is defined as l i= 0.1, and the parameter ω = 100. When the parameter α = 20, 50, 100, the active party utilities of characteristic pricing, random pricing, and static pricing are shown in Fig. 6(a). When the parameter α = 100 and the parameter ω = 0.2, 2, 20, the active party utilities of each method are shown in Fig. 6(b). It can be seen that regardless of the different settings of the parameter α or ω, the active party utility under the characteristic adaptive pricing mechanism is better than that of the random pricing method and the static pricing method. Similarly, under the random pricing and static pricing methods, the decision of the active party is fixed and will not be adjusted with the change of the passive party's information. This method cannot flexibly optimize the characteristic payment ceiling according to the actual situation, resulting in the active party utility being lower than that of the characteristic adaptive pricing method. In contrast, the characteristic adaptive pricing method can dynamically adjust the decision according to the change of the information of the participating parties, ensuring that the active party can obtain higher utility under the changeable parameter settings. This flexibility makes the characteristic adaptive pricing method always show better utility.

Claims

1. A method for training a vertical federated random forest model, characterized in that It includes sample sampling stage, feature sampling stage, base model training stage and model fusion stage; The sample sampling phase: Before the training of each decision tree begins, the active party AU generates a sample subset S by randomly sampling and aligning the index set. Then, a Bloom filter BF is generated based on the sample subset S, and sub-secrets share2 are generated through the (n,n) secret sharing protocol and transmitted to all passive parties PB, where PB = {PU i |i ∈ [1,n]}. Each passive party PU i parses the obtained sub-secret share2 to determine the sample subset S used in this training; The feature sampling stage: In the sample subset S generated in the sample sampling stage, the active party AU and the passive party PU i respectively generate subsets of the feature space for the current tree model training through random sampling The base model training stage: Using the Gini index as the evaluation metric for the optimal splitting point, based on the subset of the feature space generated in the feature sampling stage train a decision tree as the base model in the random forest model and record the feature importance data generated during the base model training; The model fusion stage: repeating the sample sampling stage, the feature sampling stage, and the base model training stage, and based on the feature importance data generated in the base model training and the incentive mechanism of the Stackelberg game, training T decision tree base models to form a random forest model; The decision tree base model in the random forest model is traversed to obtain the prediction result of the decision tree base model, and then the prediction result of the decision tree base model is weighted voted to obtain the prediction result of the random forest model, which is the final classification prediction result.

2. The method according to claim 1, wherein The sample sampling stage specifically includes the following steps: Step 1.1: The active party AU randomly samples to obtain a sample subset S, and then creates a Bloom filter BF as follows: First, initialize the Bloom filter BF to a vector of all 0s with size q, and select k independent hash functions {h1, h2, …, h k}, where each hash function maps the input value to [1, q]. Then, insert each sampled sample index s (s ∈ S) into the Bloom filter BF. For each sample index s, we have: BF[h1(s)] = 1, BF[h2(s)] = 1, … BF[h k (s)] = 1 (1); Step 1.2: The active party AU performs secret sharing on the q bits of the Bloom filter BF created in step 1.1, generates random number sub-secret share1[i] bit by bit, and calculates the shared sub-secret share2[i] of the random number sub-secret share1[i] as follows: share2[i]=BF[i]-share1[i]mod p (2) Where p is the modulus; Step 1.3: The active party AU sends the random vector share1 composed of the random number sub-secrets share1[i] generated in Step 1.2 and the shared vector share2 composed of the shared secrets share2[i] calculated in Step 1.2 to all passive parties PB through a two-choice oblivious transfer protocol, specifically as follows: The active party AU generates two q-bit random vectors Rand1 and Rand2; the passive party PU B calculates its own sample set X B for the samples x i in it, the hash values h1(i), h2(i), …, h k (i) of the index i of, set the elements at these k positions to 1 and the rest to 0 to generate a selection vector τ; Then, run the oblivious transfer protocol with a choice between two options. When τ[i] = 1, share j '[i] = share j [i]; when τ[i] = 0, share j '[i] = Rand j [i], where j = 1, 2; Step 1.4: Passive Party PU B Receive the random vector share1' and the shared vector share2' after the transmission protocol in Step 1.3, and calculate the verification value: BF'[h j (i)] = share1'[h j (i)] + share2'[h j (i)] mod p, j ∈ [1, k] (3) If the sample x i has k element bits BF'[h j (i)] = 1 (j ∈ [1, k]), then the sample x i is considered to be in S, and thus the sample subset S selected by the active party AU is recovered.

3. The method according to claim 1, wherein The base model training phase specifically includes the following steps: The decision tree is constructed using the Gini index, specifically as follows: The set of labels of the data held by the active party AU is C, and the probability that the sample data belongs to class c is p c , where c ∈ C, then the Gini index of the probability distribution is defined as: The feature splitting point with the largest gain is selected as the optimal splitting point through formula (5), and the optimal splitting point of the vertical federated decision tree model is found through privacy interaction. For a leaf node of the decision tree, the Gini index Gini(p) represents the impurity of the sample data assigned to the leaf node. The lower the Gini index Gini(p), the lower the probability of randomly selecting two samples with different labels in the sample set of the leaf node. The gain of the leaf node represents the change of the Gini index after the split at this leaf node, that is: gain = Gini(p) - ΣGini(p i ) (5) where p i represents the sample probability distribution of the child nodes after the splitting of the leaf nodes, and the gain of a certain splitting point of the binary decision tree is: Among them, Gini l (p l ) and Gini r (p r ) represent the Gini indices of the left and right child nodes respectively; |D| represents the number of samples at the split point, |D l | and |D r | represent the number of samples of the left and right child nodes respectively; when the gain gain is greater than the split threshold Δ, continue to split until all nodes cannot be split, and the construction of the decision tree base model is completed.

4. The method according to claim 3, wherein The optimal split point of the vertical federated decision tree model is found through privacy interaction. The process is as follows: Step 3.1: The initiating party AU constructs the initiating party's federated histogram in the current state where |A l | represents the number of samples in which the feature A belongs to the label l, and t represents the number of bins of the attribute A; Passive Party PU i First, construct the collaborative histogram of the passive party in the current state It means that the attributes B of the passive party PU i are divided into T buckets, where B l represents the index set of c categories contained in the l-th bucket; then the passive party PU i uses the (n,n) secret sharing protocol to generate the obfuscated Bloom filter GBF corresponding to the collaborative histogram. The obfuscated Bloom filter GBF stores the index IDs of the aligned data samples and provides the ability to perform private set intersection Step 3.2: Active party AU and passive party PU i Use a one-out-of-two oblivious transfer protocol to perform private set intersection on the information in the obfuscated Bloom filter GBF generated in Step 3.

1. The active party AU obtains an obfuscated Bloom filter GBF' that only contains the intersection of the two parties, while the passive party PU i does not obtain any other information; Step 3.3: The active party AU constructs a complete federated histogram based on the privacy set intersection result in Step 3.2 And calculates the optimal splitting point sp according to the Gini index, and the optimal splitting feature Attr of the optimal splitting point sp sp The cumulative result of the Gini index is used as the feature importance ∑ of the participating party P i That is, as the feature importance ∑ of the active party AU and the passive party PU i (i = [1, n]) i (i = [1, n + 1]), in the t-th decision tree base model, the feature importance ∑ of the participating party P i The calculation method is shown in formula (7): i The feature importance ∑ of i is calculated as shown in formula (7): Among them, θ i,j represents the importance of feature X i of participant P i,j which is calculated by the Mean Decrease Impurity (MDI) method. The importance of the feature is measured by statistically counting the decrease in impurity when the node splits, that is, the Gini index gain of the entire base model f t for this feature, expressed as formula (8): Step 3.4: When the optimal splitting feature Attr calculated in Step 3.3 sp is in the active party's feature space X n+1 , i.e., Attr sp ∈X n+1 , the active party AU constructs a confusion Bloom filter GBFR for the new right node, records the aligned sample IDs in the right node, and transmits them to the passive party PB through the oblivious transfer protocol. The passive party PB determines the sample ID set of the current new node. When the optimal splitting feature Attr calculated in Step 3.3 sp is in the passive party's feature space X i , i.e., Attr sp ∈X i , i ∈ [i, n], the active party AU sends the id of the corresponding attribute confusion Bloom filter GBF. The passive party PB constructs the splitting point of the decision tree according to the result, and constructs a confusion Bloom filter GBFR for the new right node according to the splitting situation, records the aligned sample IDs in the right node, and transmits them to the active party AU through the oblivious transfer protocol. The active party AU determines the sample ID set of the current new node; Step 3.5: Through steps 3.1 to 3.4, find the optimal split point round by round until all leaf nodes meet the conditions for stopping splitting. At this point, the training of a decision tree-based model is completed.

5. The method according to claim 1, wherein In the classification prediction of the model fusion stage, for the T decision trees constructed, the prediction result of each tree is f i (x), then the final prediction result is: where Ι is the indicator function, c is the candidate class, and α i is the weight of the voting results of each decision tree.

6. The method according to claim 1, wherein The incentive mechanism specifically includes the following steps: First, define the payment ceiling B of the active party AU, and on this basis, define the characteristic payment strategy of the passive party PB; for any passive party PU i , the active party AU pays the passive party PU i a reward B i which is expressed as formula (10): Among them, ∑ i is the feature importance, representing the model training contribution provided by the passive party PU i and is calculated by formula (7) in the basic model training stage of vertical random forest modeling; When constructing the t-th decision tree, the benefit of the passive party PU i is the reward paid by the active party AU for the feature budget contributed by the passive party PU to its training As shown in formula (11): Passive party PU i The cost is a linear function related to its feature importance ∑ i , that is, formula (12), where l i represents the unit cost value of the passive party PU i ; Combining Equation (11) and Equation (12), the utility function of the passive party PU i is expressed as Equation (13): Final passive party PU i The objective function is expressed as Equation (14): Second, the benefit of the decision tree model CT (t) is expressed as Equation (15): Among them, CT (t) represents the t-th decision tree base model, n represents the number of passive parties, and X i represents the feature set of the passive party PU i . In particular, represents the importance of the j-th feature of the passive party PU i . represents the sum of the feature importances of the participating party P i in the base classifier CT (t) . ω is the weight coefficient, and The budget of the active party AU for the t-th decision tree base classifier CT (t) is R (t) When this is the case, the utility function of the active party is expressed as shown in formula (16): Among them, α represents the model performance weight, which stipulates the upper limit of system performance output, and ω determines the speed of marginal decline of model performance; The final objective function of the active party is formula (17): Based on the Stackelberg game-based incentive mechanism, the active party AU is regarded as the leader, and all passive parties PB are regarded as followers. The training process of the vertical federated random forest model is divided into two stages, namely the master-slave dynamic game between the active party AU and the passive parties PB and the non-cooperative game among the passive parties PB, and finally an equilibrium is formed. First, solve the non-cooperative game among the passive parties PB to determine the feature importance objective of each passive party PU i , and then turn to the master-slave dynamic game stage between the active party AU and the passive parties PB to find the optimal incentive reward by maximizing the benefit of the active party AU; In the master-slave dynamic game stage between the active party AU and the passive party PB, the active party AU first determines its incentive reward parameter; then, under this constraint, each passive party PU i determines an optimal feature importance target to maximize its benefit; the passive party 's optimal feature importance target satisfies equation (18): Among them, As the leader, the active party AU determines an optimal incentive reward for the principal-agent dynamic game stage between the active party AU and the passive party PB after observing the optimal feature importance target of each passive party PU during the non-cooperative game stage between the passive parties PB. i After that, it satisfies equation (19):

7. A random forest training system based on the method according to any one of claims 1 to 6, characterized in that include: Sample Sampling Module: Before the training of each decision tree starts, the active party AU generates a sample subset S by randomly sampling and aligning the index set. Then, based on the sample subset S, a Bloom filter BF is generated, and sub-secrets share2 are generated through the (n,n) secret sharing protocol and passed to all passive parties PB. Each passive party PU i parses the obtained sub-secret share2 to determine the sample subset S used in this training; Feature Sampling Module: In the sample subset S generated by the Sample Sampling Module, the active party AU and the passive party PU i respectively generate their own subsets of the feature space for the current tree model training through random sampling Base model training module: Using the Gini index as the evaluation metric for the optimal splitting point, based on the feature space subset generated by the feature sampling module Train a decision tree as the base model in the random forest model and record the feature importance data generated during the training of the base model; Model fusion module: Repeat the sample sampling module, feature sampling module, and base model training module, and train T decision tree base models based on the feature importance data generated in the base model training and the incentive mechanism of the Stackelberg game to form a random forest model; The decision tree base model in the random forest model is traversed to obtain the prediction result of the decision tree base model, and then the prediction result of the decision tree base model is weighted voted to obtain the prediction result of the random forest model, which is the final classification prediction result.

8. A vertical federated random forest training device, characterized in that, include: Memory: used to store a computer program for implementing the method according to any one of claims 1 to 6; Processor: used to implement the method as claimed in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images