Dynamic Machine Learning Model Selection Orchestrator
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for processing machine learning model queries in distributed computing environments face issues with latency and accuracy performance, leading to undesirable delays and unsuitable results when models are located far from the querying application.
Innovation Solution
A system and method that dynamically select machine learning models based on latency and accuracy performance characteristics, using an orchestrator to collect and distribute performance metrics across multiple computing environments, and applying K-means clustering to identify suitable models for query handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning models are deployed in distributed computing environments, then model availability and processing capacity are improved, but query latency increases when models are located far from the querying application
Solution Approach 1:
The system dynamically selects machine learning models based on real-time performance characteristics (latency and accuracy) rather than using static model deployment. The model selection process adapts to current system conditions, choosing models that minimize latency while maintaining accuracy requirements, thus resolving the contradiction between distributed availability and query response time.
2Measurement precision
If multiple candidate machine learning models are examined for each query, then query accuracy is improved, but processing time and system complexity increase
Solution Approach 1:
The system performs preliminary examination and ranking of candidate models based on their performance characteristics before actually executing the query. By pre-evaluating models and maintaining a ranked list based on latency and accuracy metrics, the system can quickly select the optimal model without examining all candidates in detail during query execution, thus maintaining high accuracy while reducing processing time.
Solution Approach 2:
The system changes the parameters used for model selection based on query requirements and current system state. Instead of using fixed selection criteria, the system dynamically adjusts selection parameters (such as latency thresholds and accuracy requirements) to balance between examination thoroughness and processing time, allowing high accuracy when time permits and faster selection when time is constrained.
Data Source
AI summary
Methods, computer program products, and systems are presented. The method computer program products, and systems can include, for instance: obtaining service request data by a service application; generating query data for query of one or more machine learning model in dependence on the service request data; examining model data of a plurality of candidate machine learning models; selecting at least one model from the candidate machine learning models in dependence on the examining model data of the plurality of candidate machine learning models, wherein the at least one model defines a selected at least one model; and sending the query data to the selected at least one model for return of responsive prediction data.