Ensemble Model Auto-Selection for Category-Specific Prediction Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models require manual intervention by data scientists for feature selection and retraining whenever new feature sets are required, which is labor-intensive and inefficient, especially in ensemble approaches where different data categories behave differently.
Innovation Solution
An automated method and system for model auto-selection using an ensemble of machine learning models, which trains candidate models per data category and determines an optimal ensemble by calculating median predictions across combinations, reducing the need for manual tuning and retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual feature selection and model retraining is performed by data scientists, then model accuracy can be maintained, but labor intensity and time consumption increase significantly
Solution Approach 1:
The system enables automated model selection and retraining through self-service mechanisms. The automated model selection module continuously evaluates multiple candidate models and automatically selects the best-performing model without requiring manual data scientist intervention, thus maintaining model accuracy while eliminating time-consuming manual processes.
Solution Approach 2:
The system dynamically changes model parameters and configurations by training multiple candidate models with different parameter sets. The automated selection process evaluates these varying parameters against performance metrics and automatically adjusts the deployed model's parameters to optimize accuracy while reducing manual intervention time.
2Measurement precision
If ensemble approaches are used to handle different data categories, then prediction accuracy improves, but computational resources and system complexity increase
Solution Approach 1:
The system segments the ensemble approach by creating separate candidate model groups for different data categories. Each category receives tailored model candidates trained on category-specific data, allowing accurate predictions per category while managing overall system complexity through modular organization of model groups.
Solution Approach 2:
The system implements dynamic model selection within the ensemble framework. Instead of using a fixed ensemble configuration, the automated selection module dynamically determines which models from the candidate pool should be combined for each prediction task, adapting the ensemble composition to the specific characteristics of the input data and reducing unnecessary computational overhead.
3Measurement precision
If multiple candidate models are trained for each data category, then model selection accuracy improves, but computational resources required increase
Solution Approach 1:
The system applies partial action by training a limited number of candidate models per data category rather than exhaustively training all possible model configurations. The automated selection process evaluates these partial candidates against performance thresholds, achieving sufficient model selection accuracy while avoiding the excessive computational resources required for complete model space exploration.
Data Source
AI summary
A system and method for model auto-selection for a prediction using an ensemble of machine learning models. The method includes: receiving historical data, the historical data including previous outcomes of a plurality of events associated with a plurality of data categories; training candidate machine learning models with the historical data, each candidate machine learning model trained using a respective one of the data categories; and determining an ensemble of machine learning models by determining a median prediction for combinations of candidate machine learning models and determining the combination that has the median prediction that is closest to at least one of the previous outcomes.