AI Inferencing Orchestration Under Time and Cost Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI-models face challenges in maintaining accuracy and efficiency under time and cost constraints due to model drift caused by changes in customer bases and service offerings, requiring continuous training and processing of multiple models, which is time-consuming and costly.
Innovation Solution
A system and method for orchestrating the use of multiple AI-models by dividing datasets among inferencing engines to meet time or cost constraints, involving training and selecting AI-models based on specific requirements such as instance number, monetary value, processing cost, and time, and rerouting unrecognized instances to subsequent engines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple AI-models are continuously trained to handle model drift, then accuracy is maintained, but processing time and cost increase
Solution Approach 1:
The patent segments the AI processing system into multiple specialized inferencing engines, each trained on different subsets of data or for different scenarios. This allows the system to maintain accuracy across diverse situations without requiring a single large model to be continuously retrained, thereby reducing the time and computational resources needed for maintenance while preserving reliability.
Solution Approach 2:
The patent implements preliminary action by pre-training multiple inferencing engines on various data subsets and scenarios before deployment. This preparation allows the system to handle model drift and diverse situations without requiring continuous retraining of a single model, thus maintaining accuracy while reducing ongoing processing time and costs.
2Reliability
If multiple AI-models are used to process different situations, then accuracy is improved, but processing cost increases
Solution Approach 1:
The patent divides the processing workload across multiple specialized inferencing engines, each optimized for specific scenarios or data subsets. This segmentation allows the system to achieve high accuracy for different situations without the prohibitive cost of running multiple full-scale models simultaneously, as each engine is more efficient and targeted.
Solution Approach 2:
The patent applies parameter changes by training inferencing engines on different subsets of data with varying parameters and characteristics. This allows each engine to be highly accurate for its specific domain while being computationally efficient, thereby improving overall recognition accuracy without proportionally increasing processing costs.
3Productivity
If processing time is reduced to meet constraints, then service responsiveness is improved, but accuracy may deteriorate
Solution Approach 1:
The patent segments the inference task across multiple specialized engines that can process different portions of data in parallel. This segmentation enables the system to meet time constraints by distributing the workload, while each engine maintains high accuracy for its specific domain, thus preserving overall inference accuracy while improving processing speed.
Solution Approach 2:
The patent implements preliminary action by pre-training inferencing engines on comprehensive data subsets during deployment. This preparation allows the engines to make accurate inferences quickly without requiring extensive real-time processing, thereby achieving both high productivity and maintained reliability under time constraints.
Data Source
AI summary
As described herein, a system, method, and computer program are provided for processing a dataset. In one embodiment, an AI-model to inference the dataset is selected. A plurality of inferencing engines each using the AI-model are initiated, where a number of the inferencing engines that are initiated is calculated according to a predetermined time constraint. The dataset is divided between the plurality of inferencing engines.