Edge Foundation Model Cascading for Accurate Local AI Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI systems face challenges when training datasets cannot describe the complete scenario, with discriminative AI systems being limited and generative AI systems prone to hallucinations and requiring large numbers of parameters.
Innovation Solution
A hybrid distributed inference system combining discriminative and generative AI systems, with localized and centralized models, where discriminative AI systems are used at edge sites for initial classification, and generative AI systems are employed if necessary, leveraging retrieval augmented generation (RAG) data and knowledge acquirers to transform data for training, and utilizing a digital twin system for data ranking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If generative AI systems are used to improve generalization ability, then the ability to handle data not in training set is improved, but the number of parameters required increases and hallucinations occur
Solution Approach 1:
The system segments AI functionality into two parts: a small generative AI model deployed locally at edge sites for initial inference, and a larger centralized generative AI model for complex queries. This segmentation allows the edge to operate with minimal parameters while maintaining access to powerful centralized processing when needed.
Solution Approach 2:
A hybrid distributed inference system acts as an intermediary between local edge devices and centralized cloud infrastructure. This intermediary coordinates inference requests, managing when to use local small models versus when to escalate to centralized large models, thereby optimizing parameter usage across the system.
2Measurement precision
If discriminative AI systems are used for classification, then classification accuracy within limited state spaces is improved, but the ability to handle scenarios beyond training data deteriorates
Solution Approach 1:
The system merges discriminative AI systems (for accurate classification within known states) and generative AI systems (for handling novel scenarios) into a hybrid architecture. This combination allows the system to leverage the strengths of both approaches: discriminative models provide precise classification for familiar patterns while generative models handle out-of-distribution data.
3Measurement precision
If centralized large foundation systems are used, then inference accuracy is improved, but data transmission requirements and computational resource usage increase
Solution Approach 1:
The inference workload is segmented between edge devices and centralized cloud systems. Simple, low-risk queries are processed locally at edge sites with minimal data transmission, while only complex or high-risk queries are transmitted to the centralized system. This segmentation dramatically reduces data transmission requirements and computational resource usage at the edge.
Solution Approach 2:
The system applies partial centralized processing by using small local models for most inference tasks and only invoking large centralized models when necessary. This partial approach to centralized computing minimizes resource consumption while maintaining accuracy for critical decisions.
4Productivity
If small foundation models are deployed at edge sites, then computational resource efficiency is improved, but model accuracy and generalization capability are reduced
Solution Approach 1:
The hybrid distributed inference system serves as an intermediary layer between small edge models and large centralized models. When the small model encounters uncertain or complex queries, the intermediary system escalates these to the centralized large model for resolution, thereby compensating for the limited accuracy of small models while maintaining overall system efficiency.
Data Source
AI summary
The disclosure is directed to a local small foundation system which leverages a generative Artificial Intelligence (AI) model to produce classification results. The local small foundation system includes less than 10 billion parameters and is located on an edge site. The local small foundation system is customized towards the edge site. The local small foundation system produces classification results based on a query by making an inference with the generative AI model. An output guardrail module determines if the classification results can be resolved to an existing class of a plurality of existing classes. The output guardrail module also provides instructions on how to implement classification results at the endpoint management system. The output guardrail module also determines if a centralized large foundation system needs to be consulted and what to send to the centralized large foundation system.