A third-party federated gradient boosting decision tree model training method

Abstract

The invention discloses a third-party federated gradient boosting decision tree model training method. It includes the following steps: synchronous initialization of the training initiator and training participants; synchronous sampling of d sample data sets by the training initiator and training participants; binning of each feature data in their respective sample data sets by the training initiator and training participants , Record binning information and store bit slices; the training initiator calculates the first-order gradient sum and second-order gradient sum corresponding to each binning of each feature data of each sample data set, and the training initiator, training participants The party calculates the first-order gradient sum and the second-order gradient sum corresponding to each sub-bin of each feature data in the training participant's sample data set according to the safe multiplication protocol; the training initiator searches for the optimal split point, and synchronizes the results to the training participants side; repeat the above steps until the termination condition is met. The invention protects data security, reduces storage space, and greatly compresses communication traffic.

Description

technical field [0001] The invention relates to the technical field of gradient boosting decision tree model training, in particular to a third-party federated gradient boosting decision tree model training method. Background technique [0002] The federated gradient boosting decision tree model can solve both classification problems and regression problems, and has good interpretability, so it is widely used in the field of federated learning, especially in the field of bank risk control. The federated gradient boosting decision tree model is a very practical tree model. In the federated gradient boosting decision tree model, each participant calculates the first and second derivatives of the decision tree based on local data, and decides the best In this process, the first and second derivatives of different participants need to be added. Additive homomorphic encryption can be used to protect the data privacy of each participant from being leaked to the tree model during t...

AI Technical Summary

Problems solved by technology

The first batch of performance evaluation data of the privacy computing institute directly under the Ministry of Industry and Information Technology shows that the average time-consuming of federal tree modeling with 900 features and 400,000 samples in the industry is 2 hours, 23 minutes and 47 seconds, which is difficult to meet the needs of the industry;

[0005] 2. There are third-party assistants to participate in training and synchronize model parameter distribution, but it is difficult to find a trusted third party for actual commercial implementation, and there is a risk of data leakage;

[0006] 3. The existing feature value storage efficiency is low, and a data set with 900 features and 400,000 samples needs to occupy 3.9G space

If the intermediate results of federated gradient boosting decision tree model training are stored on the local disk, a federated gradient boosting decision tree model training will consume more than 10G of space

Images