Decision Tree Tensor Subsets for Faster Ensemble Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for accelerating decision tree inferences, particularly in ensemble models like random forest and gradient boosting, are computationally costly due to the time and resource-intensive nature of processing large numbers of input records and decision trees.
Innovation Solution
The method decomposes tensor operations into complementary subsets based on leaf node statistics, ranking them by likelihood of use, and iteratively processes input records through these subsets, leveraging hardware accelerators for efficient computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional tensor operations are used to process all input records through all decision trees, then complete inference results are obtained, but computational time and resource consumption increase significantly
Solution Approach 1:
The patent segments the tensor operations into multiple subsets based on leaf node statistics. Each subset corresponds to a portion of the decision trees and processes a subset of input records. This segmentation allows parallel processing and early termination for records that have been successfully classified, reducing overall computational time while maintaining inference completeness.
Solution Approach 2:
The patent performs preliminary sorting of tensor operation subsets based on leaf node statistics before processing input records. By pre-ordering the subsets according to their likelihood of containing the correct inference, the system can quickly resolve most records in early iterations, avoiding the need to process all subsets for all records and thus reducing computational time.
2Reliability
If all tensor operations are executed to ensure accurate inference results, then inference accuracy is maintained, but computational resource consumption increases
Solution Approach 1:
The patent applies partial action by processing only the necessary subset of tensor operations for each input record. Based on leaf node statistics, the system determines the minimum number of tensor operation subsets required to achieve accurate inference for each record, avoiding unnecessary computations and reducing resource consumption while maintaining accuracy.
Solution Approach 2:
By dividing tensor operations into manageable subsets and processing them in ordered iterations, the system can terminate processing for individual records once sufficient accuracy is achieved, rather than executing all tensor operations. This segmented approach reduces overall computational resource consumption while preserving inference accuracy.
3Productivity
If tensor operations are decomposed into multiple subsets and processed iteratively, then computational efficiency improves, but algorithm complexity increases
Solution Approach 1:
The patent performs preliminary sorting and organization of tensor operation subsets based on leaf node statistics before the iterative processing begins. This pre-processing step establishes a clear execution order that simplifies the iterative algorithm, making it more efficient despite the decomposition into multiple subsets. The preliminary action reduces the complexity of managing multiple subsets during execution.
4Speed
If leaf node statistics are used to rank tensor operation subsets, then processing order is optimized for speed, but additional preprocessing steps are required
Solution Approach 1:
The patent performs the ranking of tensor operation subsets based on leaf node statistics as a preliminary action before the main inference process. Although this adds a preprocessing step, it enables significantly faster processing during the actual inference by ensuring that the most promising subsets are evaluated first, reducing the average number of iterations needed and optimizing processing speed.
Data Source
AI summary
A tensor representation of a machine learning inferences to be performed is built by forming complementary tensor subsets that respectively correspond to complementary subsets of one or more leaf nodes of one or more decision trees based on statistics of the one or more leaf nodes of the one or more decision trees and data capturing attributes of one or more split nodes of the one or more decision trees and the one or more leaf nodes of the decision trees. The complementary tensor subsets are ranked such that a first tensor subset and a second tensor subset of the complementary tensor subsets correspond to a first leaf node subset and a second leaf node subset of the complementary subsets of the one or more leaf nodes.