AI Inferencing Orchestration Under Time and Cost Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI-models face challenges in maintaining accuracy and efficiency under time and cost constraints due to model drift caused by changes in customer bases and service offerings, requiring continuous training and processing of multiple models, which is time-consuming and costly.
Innovation Solution
A system and method for orchestrating the use of multiple AI-models by dividing datasets and selecting inferencing engines to meet time or cost constraints, utilizing a predetermined number of engines and training models to handle unrecognized instances by subsequent engines, optimizing processing time and cost.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple AI-models are used to process different situations, then accuracy is improved, but processing time and cost increase
Solution Approach 1:
The patent divides the dataset into multiple partitions and assigns different AI-models to process specific partitions based on their expertise. This segmentation allows parallel processing of different data subsets, reducing overall processing time while maintaining high accuracy through specialized model assignment.
Solution Approach 2:
The system implements a sequential inferencing approach where multiple models process the same dataset in sequence, with each model attempting to recognize instances. Instances recognized by earlier models are excluded from subsequent model processing, creating a partial action strategy that reduces redundant computation while ensuring comprehensive recognition coverage.
2Measurement precision
If multiple AI-models are used to process different situations, then accuracy is improved, but processing cost increases
Solution Approach 1:
The patent segments the dataset and assigns different AI-models to specific partitions, allowing cost-effective processing by matching model complexity with data requirements. Simpler models process straightforward cases while more complex models handle challenging instances, optimizing the cost-accuracy tradeoff.
Solution Approach 2:
The sequential processing framework with instance exclusion implements partial action by having later models only process instances missed by earlier models. This reduces redundant computation costs while maintaining comprehensive accuracy through the collective effort of multiple specialized models.
3Measurement precision
If continuous training of AI-models is performed to address model drift, then accuracy is maintained, but processing time and cost increase
Solution Approach 1:
The system performs preliminary clustering of the dataset into distinct partitions before processing, organizing data by characteristics that align with model specializations. This preliminary action enables more efficient subsequent processing by pre-grouping instances that can be handled by specific models, reducing the need for continuous retraining and optimization.
Data Source
AI summary
As described herein, a system, method, and computer program are provided for processing a dataset. In one embodiment, an AI-model to inference the dataset is selected. A plurality of inferencing engines each using the AI-model are initiated, where a number of the inferencing engines that are initiated is calculated according to a predetermined time constraint. The dataset is divided between the plurality of inferencing engines.