Boosting Tree Model Reduction via Subtree Importance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Tree-based machine learning models, such as boosting tree models, face challenges in handling sudden increases in network traffic and hardware changes, leading to reduced accuracy and efficiency when reduced using pruning, quantization, or slicing selection methods.
Innovation Solution
A method that determines subtree importance based on feature importance information to extract continuous tree subsets from a boosting tree model, resulting in a reduced model with maintained inference accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If pruning is performed on the boosting tree model to reduce its size, then the model size is reduced, but the model cannot adapt to unexpected changes in practical application fields
Solution Approach 1:
The patent segments the boosting tree model into multiple subtrees and evaluates their individual importances. By dividing the model into manageable segments (subtrees), the system can selectively retain or remove specific segments based on their contribution to model performance, enabling adaptive size control without compromising overall model adaptability.
Solution Approach 2:
The patent introduces dynamic subtree importance evaluation that can adapt to different application scenarios. The importance metrics are calculated based on actual model performance data, allowing the system to dynamically adjust which subtrees to retain based on changing practical requirements, thus maintaining adaptability while reducing size.
2Quantity of substance
If quantization is performed on the boosting tree model to reduce its size, then the model size is reduced, but the inference accuracy deteriorates
Solution Approach 1:
The patent extracts and evaluates the contribution of individual subtrees to the overall model performance. By identifying and removing only the least important subtrees rather than quantizing all model parameters, the system achieves size reduction while preserving the precision of the remaining critical components.
Solution Approach 2:
The patent applies different treatment to different parts of the model based on their local importance. Critical subtrees that contribute significantly to accuracy are retained in full precision, while less important subtrees are removed or simplified. This local differentiation maintains overall inference accuracy while achieving compression.
3Quantity of substance
If slicing selection is performed on the boosting tree model to reduce its size, then the model size is reduced, but obtaining a tree subset with acceptable accuracy is time-consuming
Solution Approach 1:
The patent performs preliminary evaluation of subtree importances using efficient metrics (feature importance, usage frequency, contribution to accuracy) before final model reduction. This preliminary assessment allows the system to quickly identify candidate subtrees for retention without exhaustive evaluation of all possible subsets, significantly reducing computation time.
Solution Approach 2:
The patent employs automated algorithms that self-evaluate subtree importances based on intrinsic model characteristics and performance data. The system serves itself by automatically identifying and selecting important subtrees without requiring extensive manual evaluation or exhaustive search, thereby reducing the time investment required for model reduction.
4Speed
If the boosting tree model is reduced to handle sudden network traffic increases, then processing speed improves, but the risk of malicious threats increases
Solution Approach 1:
The patent incorporates feedback mechanisms where the importance of each subtree is evaluated based on its actual performance in handling different types of inputs including potential threats. The model continuously monitors which subtrees are most effective at identifying and handling malicious traffic patterns, and prioritizes retaining those subtrees during reduction, thereby maintaining security effectiveness while improving processing speed.
Data Source
AI summary
The disclosure provides a method for tree-based machine learning model reduction and an electronic device using the same. A boosting tree model including multiple subtrees is obtained. Subtree importance of each of the subtrees is determined according to feature importance information respectively corresponding to multiple model features of the boosting tree model. At least one continuous tree subset is extracted from the subtrees according to the subtree importance of each of the subtrees. The at least one continuous tree subset includes at least one of the subtrees. At least one reduced boosting tree model of the boosting tree model is obtained according to the at least one continuous tree subset.