Sequential Model Inference for QoS-Constrained ML
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning models often fail to provide high inference results while meeting user-defined Quality of Service (QoS) constraints, particularly in terms of latency, precision, and recall, in applications like video surveillance and IoT environments.
Innovation Solution
The system processes input data through multiple machine learning models, where the first inference result is compared to QoS parameters, and if it does not meet the requirements, the data is further evaluated by a second model to generate a second inference result, allowing for improved decision control and QoS management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single machine learning model is used to provide detailed inference results with high confidence, then measurement precision is improved, but loss of time increases due to relatively long computation time
Solution Approach 1:
The system segments the inference process into multiple stages by employing an ensemble of machine learning models with different complexity levels. Simple models provide quick preliminary assessments while complex models handle cases requiring higher precision, thereby reducing overall computation time while maintaining inference quality when needed.
Solution Approach 2:
The system dynamically selects which machine learning models to invoke based on real-time requirements. The selection process adjusts the complexity and number of models used according to the specific inference task, balancing between computation time and result quality on a per-request basis rather than using a fixed approach.
2Measurement precision
If multiple machine learning models are used to improve inference quality and meet QoS parameters, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The ensemble of models is segmented into different complexity tiers, allowing the system to invoke only the necessary number and type of models for each inference task. This segmentation reduces the effective complexity by avoiding the invocation of all possible models for every request.
Solution Approach 2:
The system changes operational parameters such as the number of models invoked, the confidence threshold, and the selection criteria based on QoS requirements. By dynamically adjusting these parameters, the system manages complexity while maintaining the ability to achieve high precision when needed.
3Productivity
If a single machine learning model is used to meet processing speed requirements, then productivity is improved, but measurement precision deteriorates due to inability to provide high confidence detailed results
Solution Approach 1:
The system dynamically adapts the inference process by adjusting the number and complexity of models based on real-time QoS feedback. When processing speed is critical, simpler models are used; when confidence and detail are paramount, more complex models are invoked, thereby balancing productivity and precision dynamically.
Solution Approach 2:
The system changes operational parameters such as confidence thresholds, model selection criteria, and the number of models invoked based on QoS requirements. This allows the system to optimize for either productivity or precision depending on the specific operational context and user needs.
Data Source
AI summary
Embodiments for processing data with multiple machine learning models are provided. Input data is received. The input data is caused to be evaluated by a first machine learning model to generate a first inference result. The first inference result is compared to at least one quality of service (QoS) parameter. Based on the comparison of the first inference result to the at least one QoS parameter, the input data is caused to be evaluated by a second machine learning model to generate a second inference result.