Decision Tree Inference With Mixed Precision Threshold Uncertainty
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models using decision trees face accuracy issues when deployed on systems with lower arithmetic precision due to precision loss during conversion, leading to uncertainty in decision rules.
Innovation Solution
Utilize historical statistical data collected at higher precision to assist decision-making through majority voting, and optionally update this data during inference, defining upper threshold values in lower precision to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If decision tree models trained with high precision (e.g., single-precision floating-point) are deployed on systems with lower precision (e.g., half-precision floating-point), then computational efficiency and resource utilization improve, but measurement precision and reliability of decision rules deteriorate due to precision loss during conversion
Solution Approach 1:
The patent applies preliminary action by collecting historical statistical data during the training phase (before deployment) to establish baseline decision-making patterns. This pre-collected data is then used during inference on lower-precision systems to resolve uncertainties that arise from precision loss, thereby maintaining decision accuracy without requiring high-precision computations during deployment.
Solution Approach 2:
The patent introduces historical statistical data as an intermediary element between the high-precision training phase and low-precision inference phase. This intermediary data structure acts as a bridge, carrying decision-making knowledge from the high-precision environment to the low-precision environment, enabling accurate decisions without directly transferring the actual precision requirements.
2Device complexity
If threshold values are converted from high precision to low precision, then device complexity and computational overhead decrease, but uncertainty in decision boundaries increases
Solution Approach 1:
The patent implements feedback by continuously updating the historical statistical data with actual inference results from the lower-precision system. This feedback loop allows the system to learn from real-world performance and adjust its decision-making strategies, thereby compensating for the loss of precision in threshold values and maintaining reliable decision boundaries over time.
Solution Approach 2:
The patent applies parameter changes by dynamically adjusting the use of historical statistical data based on the specific inference context. When precision loss is detected or anticipated, the system leverages the pre-collected statistical parameters to adjust decision-making, effectively changing the operational parameters to maintain reliability despite fixed low-precision threshold values.
3Measurement precision
If majority voting based on historical statistical data is used to resolve precision-related uncertainties, then decision accuracy improves, but computational time and processing overhead increase
Solution Approach 1:
The patent applies partial action by using majority voting selectively rather than universally. The system determines when precision-related uncertainties actually occur and applies the historical statistical data and majority voting mechanism only in those specific cases, rather than applying it to every decision. This reduces the overall computational overhead while still maintaining accuracy where needed.
Data Source
AI summary
A method, system, and computer program product perform machine-learning inferences with a tree-based model. The tree-based model includes a decision tree that was trained on a first system, which is configured to perform computations with a first arithmetic precision. The inferences are performed with the tree-based model on a second system, which is configured to perform computations with an arithmetic precision that is lower than the first arithmetic precision. Performing the inference includes determining that an input feature value is equal to a threshold value of a corresponding node and, in response, using a majority voting to select a left or right path of the decision tree. The majority voting is based on historical statistical data that includes tree-path statistics.