Parallel Boosting Model Training with One-Level Decision Trees
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprises face challenges in choosing, applying, and interpreting predictive models due to the complexity and 'black box' nature of existing predictive modeling technologies, which hinders effective classification and prediction.
Innovation Solution
A system and method that train multiple boosting models in parallel using a first dataset, building one-level binary decision trees, calculating impurity, and determining an optimal split node, allowing for the transformation of models into linear formats to enhance interpretability and reduce computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional predictive models are used, then classification and prediction can be performed, but the models become complex and difficult to interpret ('black box' nature)
Solution Approach 1:
The patent segments the predictive modeling process into distinct components: data preprocessing, model training, model selection, and interpretation. It divides the model structure into interpretable elements such as decision trees and regression models that can be individually analyzed, replacing monolithic complex models with modular, explainable components.
Solution Approach 2:
The patent introduces intermediary components including validation datasets, performance metrics, and interpretation tools that mediate between the complex modeling process and the user. These intermediaries provide bridges for understanding model behavior without requiring direct engagement with the underlying complexity.
2Reliability
If multiple predictive models are trained to improve accuracy, then predictive performance increases, but computational resources and processing time increase
Solution Approach 1:
The patent applies partial action by training multiple models with different complexities and selecting only the necessary ones based on performance metrics. It uses validation datasets to evaluate models partially, stopping training when sufficient accuracy is achieved, thereby avoiding excessive computational resources.
Solution Approach 2:
The patent changes parameters such as model complexity, training data size, and validation criteria to optimize the balance between accuracy and computational resources. By adjusting these parameters, the system achieves satisfactory predictive performance without proportionally increasing computational expenditure.
3Reliability
If complex algorithms are used to find optimal split nodes, then model accuracy improves, but algorithm complexity increases from O(n2) to O(n)
Solution Approach 1:
The patent changes the algorithmic parameters and computational approach to reduce complexity. It employs optimized search methods and heuristics that achieve satisfactory accuracy without requiring exhaustive O(n2) comparisons, thereby reducing computational complexity while maintaining predictive performance.
Data Source
AI summary
In one embodiment, a system for enhancing predictive modeling includes an interface operable to receive a first dataset. The system may also include a processor communicatively coupled to the interface that is operable to generate a holdout dataset based on the first dataset. The processor may also train each of a plurality of boosting models in parallel using the first dataset, wherein for each of a number of iterations, training comprises: building a one-level binary decision tree to train a split-node variable; calculating an impurity of the split-node variable; and calculating an optimal split node, wherein the optimal split node is the split-node variable with a lowest impurity between the plurality of boosting models. The system may then determine a final model based on one of the plurality of boosting models that provides the lowest error rate when applied to the holdout dataset.