Neural Network Cascade Accuracy via Conditional Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The accuracy increase from adding one or more machine learning models to an ensemble comes with an increase in computational cost, and for many inputs, the accuracy increase is minimal compared to the additional computational cost.
Innovation Solution
The implementation of a 'cascade' of machine learning models, where additional models are conditionally executed only when the benefits of such computation are likely to provide sufficiently increased output accuracy, by determining a correctness metric for the output of the first model and comparing it to a threshold.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If additional machine learning models are added to an ensemble, then model accuracy is improved, but computational cost increases
Solution Approach 1:
The patent implements a dynamic model execution strategy where the system adaptively determines whether to execute additional models based on a correctness metric from the first model. The correctness metric threshold is dynamically adjusted to balance accuracy and computational cost, allowing the system to transition between executing only the first model or executing both first and second models based on input characteristics.
Solution Approach 2:
The patent changes the parameter of model execution by introducing a correctness metric threshold parameter. By adjusting this threshold parameter, the system can control the trade-off between accuracy and computational cost, dynamically selecting whether to proceed to additional model execution based on the first model's output quality.
2Measurement precision
If additional machine learning models are added to an ensemble, then model accuracy is improved, but the complexity of the system increases
Solution Approach 1:
The patent segments the model execution process into distinct phases: first model execution, correctness metric evaluation, threshold comparison, and conditional second model execution. This segmentation allows the system to manage complexity by breaking down the ensemble process into modular, independently controllable steps with clear decision points.
Solution Approach 2:
The patent introduces a correctness metric as an intermediary between the first model output and the decision to execute the second model. This intermediary serves as a gatekeeper that simplifies the complexity management by providing a clear, quantifiable criterion for determining whether additional model execution is necessary, rather than always executing all models.
3Measurement precision
If additional machine learning models are added to an ensemble, then output accuracy is improved, but execution time increases
Solution Approach 1:
The patent implements partial action by executing additional models only when necessary, rather than always executing the full ensemble. The correctness metric threshold determines whether partial execution (first model only) or full execution (both models) is appropriate, reducing execution time for inputs where the first model already provides sufficient accuracy.
Solution Approach 2:
The patent introduces periodic decision points in the execution flow where the system evaluates the correctness metric and determines whether to proceed to the next model. This periodic evaluation creates an interruptible execution pattern that can terminate early when accuracy requirements are met, reducing overall execution time compared to sequential execution of all models.
Data Source
AI summary
A combination of two or more trained machine learning models can exhibit a combined accuracy greater than the accuracy of any one of the constituent models. However, this increase accuracy comes at additional computational cost. Cascades of machine learning models are provided herein that result in increased model accuracy and/or reduced model compute cost. These benefits are obtained by conditionally executing one or more of the models of the cascade based on the estimated correctness of already-executed models. The estimated correctness can be obtained as an additional output of the already-executed model(s) or could be determined as an entropy, maximum class probability, maximum class logit, or other function of the output(s) of the already-executed model(s). The expected computational cost of executing the model cascade is reduced by only executing the downstream model(s) when the upstream model(s) has resulted in an output whose accuracy is suspect.