AI Inferencing Orchestration Under Time-Limited Processing Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI-models face challenges in maintaining accuracy and efficiency under time and cost constraints due to model drift and the need for continuous training and processing of large datasets, which can be resource-intensive.
Innovation Solution
A system and method for orchestrating the use of multiple AI-models by dividing datasets and selecting inferencing engines to meet time and cost constraints, utilizing a predetermined number of engines and training models to handle different labeled datasets, and rerouting unrecognized instances to subsequent engines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If continuous training of AI-models is performed to maintain accuracy under model drift, then model effectiveness is improved, but processing time and cost increase
Solution Approach 1:
The patent segments the dataset into multiple subsets and trains separate AI-models on each subset. This allows the system to process different data portions in parallel, reducing overall processing time while maintaining model accuracy through specialized training on segmented data.
Solution Approach 2:
The patent applies partial action by selectively training models only on specific data subsets rather than continuously retraining all models on complete datasets. This reduces unnecessary processing while maintaining effectiveness for the specific inference tasks at hand.
2Adaptability or versatility
If multiple AI-models are used to process different situations, then model versatility is improved, but processing cost increases
Solution Approach 1:
The patent divides the dataset into multiple segments and trains separate AI-models on each segment. This segmentation enables versatile processing of different data types while controlling costs by allocating computational resources efficiently across multiple specialized models rather than using a single comprehensive model.
Solution Approach 2:
The system achieves universality by creating multiple AI-models that can collectively handle diverse situations and data types. Each model is trained on specific subsets, and together they provide versatile processing capabilities across different scenarios.
3Productivity
If dataset is divided and processed by multiple inferencing engines in parallel, then processing speed is improved, but system complexity increases
Solution Approach 1:
The patent segments the dataset and processes segments in parallel using multiple inferencing engines. This segmentation approach improves processing speed by utilizing parallel computation while managing system complexity through structured data division and coordinated engine operation.
Data Source
AI summary
As described herein, a system, method, and computer program are provided for processing a dataset. In one embodiment, an AI-model to inference the dataset is selected. A plurality of inferencing engines each using the AI-model are initiated, where a number of the inferencing engines that are initiated is calculated according to a predetermined time constraint. The dataset is divided between the plurality of inferencing engines.