Random Forest Training via Grouped Worker Node Sampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing random forest training methods face inefficiencies when handling massive datasets due to the need to scan complete data sets for sampling and calculate Gini coefficients, leading to high computational complexity and long iterative update times.
Innovation Solution
The method involves dividing worker nodes into groups, where each group randomly samples and distributes partial data to train decision trees, reducing data read volume and using importance sorting to calculate Gini coefficients, thereby lowering computational complexity from O(2n-1) to O(n).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If each worker node samples subset from full set of sample data, then random forest training can be performed in parallel, but data read volume increases and training efficiency decreases
Solution Approach 1:
The patent divides worker nodes into multiple groups, where each group independently samples from the full dataset. This segmentation allows parallel processing while reducing the sampling burden on individual nodes, as each group only needs to read a portion of the data rather than every worker node reading the complete dataset.
Solution Approach 2:
The patent performs preliminary sampling by having worker node groups read and sample data in advance before the actual decision tree training begins. This preliminary action separates the data reading phase from the training phase, allowing optimization of data access patterns and reducing the time impact on iterative updates.
2Measurement precision
If exhaustive method is used to calculate Gini coefficient for all feature combinations, then decision tree training accuracy is improved, but computational complexity becomes exponential
Solution Approach 1:
The patent extracts and calculates only the necessary Gini coefficients for the actual splitting decisions made during tree construction, rather than computing all possible feature combination Gini coefficients. This selective extraction reduces computational complexity from exponential to linear while maintaining the accuracy needed for optimal splits.
Solution Approach 2:
The patent applies partial action by calculating Gini coefficients only for features and splits that are actually used in the decision tree, rather than performing exhaustive calculations for all possible (2n-1-1) combinations. This partial computation approach achieves sufficient precision for training accuracy without the exponential cost of complete enumeration.
Data Source
AI summary
Methods and apparatuses for training model based on random forest are provided. The method includes: dividing worker nodes into one or more groups; performing random sampling, by worker nodes in each group, in the preset sample data to obtain the target sample data; and training, by the worker nodes in each group, one or more decision tree objects using the target sample data. Example embodiments of the present disclosure do not need to scan the complete sample data for once, thereby greatly reducing the amount of data to be read, the time cost, and further the iterative update time of the model. The efficiency of training is improved.