Dynamic Decision Tree Inference Through Two-Phase Memory Reconfiguration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for accelerating tree-based inferences in machine learning, such as random forest and gradient boosting, are computationally intensive and resource-consuming, particularly when input data properties fluctuate over time, impacting performance.
Innovation Solution
A method and system that dynamically optimize decision tree inference operations by intermittently monitoring access characteristics during active phases and reconfiguring the underlying tree representation during idle phases, using a two-phase cycle approach to update a reference data structure based on statistical characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If decision tree inference operations are executed repeatedly during active periods with online monitoring of access characteristics, then the optimization can adapt to fluctuating data properties, but the monitoring process impacts the performance of the inferences
Solution Approach 1:
The system implements periodic action by alternating between active periods for inference execution and idle periods for data structure reconfiguration. During active periods, decision trees are executed repeatedly with monitoring of accessed nodes. During subsequent idle periods, the reference data structure is reconfigured based on statistical characteristics collected during the active period. This periodic alternation allows adaptation to data fluctuations while ensuring inference performance is maintained during active periods.
2Reliability
If the reference data structure is reconfigured during idle periods based on statistical characteristics, then the memory representation is optimized for future inferences, but the reconfiguration process consumes computational resources and time
Solution Approach 1:
The system applies preliminary action by collecting statistical characteristics of accessed nodes during active periods and using these characteristics to preconfigure the reference data structure during idle periods before the next inference workload arrives. This allows the system to be prepared in advance for upcoming inference operations, optimizing memory representation proactively rather than reactively, thereby reducing latency when the next active period begins.
3Measurement precision
If multiple decision trees are walked in parallel for random forest and gradient boosting methods, then ensemble predictions are achieved, but the computations are time and resource consuming
Solution Approach 1:
The system merges multiple decision tree evaluations by reconfiguring the reference data structure to optimize memory representation for ensemble methods. Statistical characteristics from monitoring accessed nodes across multiple trees during active periods inform the reconfiguration during idle periods, creating an optimized data structure that reduces memory access overhead when walking multiple trees in parallel, thereby improving computation efficiency while maintaining prediction accuracy.
Data Source
AI summary
A method of dynamically optimizing decision tree inference is provided. The method, which is performed at the computerized system, repeatedly executes one or more decision trees for inference purposes and repeatedly performs an optimization procedure according to two-phase cycles. Each cycle includes two alternating phases, i.e., a first phase followed by a second phase. The decision trees are executed based on a reference data structure, whereby attributes of nodes of the decision trees are repeatedly accessed from the reference data structure during the first phase of each of the cycles. First, the accessed attributes are monitored during the first phase of each cycle, which leads to update statistical characteristics of the nodes. Second, a substitute data structure is configured during the second phase of each cycle based on the updated statistical characteristics. Third, the reference data structure is updated in accordance with the substitute data structure.