Abridged Ensemble Model Serving with Deviation Thresholds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer implemented models, particularly machine learning models, require increasing compute resources to maintain accuracy, posing a challenge in reducing resource usage while preserving performance.
Innovation Solution
The method involves generating an abridged model from a machine learning model, which uses fewer compute resources by selecting a subset of component models from an ensemble model. The abridged model processes inputs in real-time, switching to the original ensemble model when the standard deviation of component scores exceeds a deviation threshold.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If an ensemble model is used to maintain accuracy, then model accuracy is improved, but compute resource usage increases
Solution Approach 1:
The ensemble model is segmented into multiple component models that can be independently evaluated. The system divides the full ensemble into subsets and processes them separately, allowing selective execution based on input characteristics and deviation thresholds, thereby reducing overall compute resource usage while maintaining accuracy for critical cases.
Solution Approach 2:
The system dynamically adjusts which models are executed based on real-time conditions. By calculating deviation thresholds and comparing component model outputs, the system adaptively determines whether to use the full ensemble or a subset, optimizing compute resource usage according to the specific input rather than always executing all models.
2Use of energy by moving object
If compute resources are reduced by using an abridged model, then resource efficiency is improved, but model accuracy may deteriorate
Solution Approach 1:
The system implements feedback mechanisms by calculating deviation thresholds from component model outputs and using this information to determine whether the abridged model's prediction is sufficient. When deviation is within acceptable bounds, the abridged model's output is used; when deviation exceeds the threshold, the full ensemble is invoked, ensuring accuracy is maintained when needed.
Solution Approach 2:
The system changes the parameter of model selection based on deviation threshold calculations. By dynamically adjusting which model subset is executed (abridged vs. full ensemble) based on the calculated deviation of component scores, the system optimizes the balance between compute resource usage and prediction accuracy for each input.
Data Source
AI summary
A method implements efficient real time serving of ensemble models. The method includes receiving an input and processing the input with an abridged model to generate a set of component scores and an abridged score. The method further includes processing the set of component scores with a deviation threshold to select one of the abridged score and an ensemble score as an output and presenting the output.