Local Small Foundation Models With Symbolic Reduction at the Edge
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Discriminative AI systems struggle when training datasets cannot describe the complete scenario, while generative AI systems face issues like hallucinations and high parameter requirements.
Innovation Solution
A hybrid distributed inference system combining discriminative and generative AI models, with localized small foundation systems and centralized large foundation systems, utilizing retrieval augmented generation (RAG) and symbolic reasoning to optimize classification and reduce resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If generative AI systems are used to generalize to data not in training set, then adaptability is improved, but device complexity and resource requirements increase
Solution Approach 1:
The system segments AI processing into multiple levels: local small foundation models at edge sites handle routine classification, while centralized large foundation models handle complex cases. This segmentation allows generative AI capabilities to be distributed, reducing the parameter burden on any single device while maintaining overall adaptability.
Solution Approach 2:
Different parts of the system use different model sizes appropriate to their function. Local edge sites use small foundation models (e.g., 7B parameters) for quick local classification, while centralized systems use large foundation models (e.g., 70B+ parameters) for complex reasoning. This local quality optimization reduces overall device complexity while preserving adaptability where needed.
2Speed
If local small foundation systems are used at edge sites, then response speed is improved, but measurement precision may be reduced
Solution Approach 1:
The system introduces an intermediary mechanism where local small foundation models first process queries locally for fast response, and when confidence thresholds are not met, the case is escalated to centralized large foundation models for higher precision. This intermediary escalation process maintains both speed and accuracy.
Solution Approach 2:
Local small foundation models perform partial classification action on all queries, handling simple cases completely and filtering complex cases for centralized processing. This partial action approach ensures fast responses for most cases while maintaining high precision for difficult cases through selective escalation.
3Measurement precision
If centralized large foundation systems are used for all cases, then classification accuracy is improved, but energy consumption and data transmission increase
Solution Approach 1:
The system extracts and processes simple classification cases locally at edge sites using small foundation models, removing these routine cases from the centralized processing load. Only complex cases requiring high precision are extracted and sent to centralized large foundation models, reducing overall energy consumption and data transmission.
Solution Approach 2:
The system dynamically changes the parameter scale of the AI model being used based on case complexity. Simple cases use small foundation models with fewer parameters locally, while complex cases escalate to centralized large foundation models with more parameters. This parameter adaptation reduces energy consumption by matching model size to task requirements.
4Productivity
If hybrid distributed inference system is implemented, then resource efficiency is improved, but device complexity increases
Solution Approach 1:
The system implements a universal architecture where both local and centralized components can function independently or in combination. The same small foundation model architecture can serve multiple edge sites, and the centralized large foundation model serves as a backup and knowledge repository for all sites. This multi-functionality improves resource efficiency while managing complexity through standardized interfaces.
Data Source
AI summary
The disclosure includes a symbolic reasoning engine. The symbolic reasoning engine allows for the distribution of relevant variables based on a centralized large foundation system to be used to refine a local small foundation model in a composable way. The local small foundation system is one of a plurality of local small foundation systems and each local small foundation system is refined in a way unique to the specific local small foundation system. This allows for the local small foundation systems to take advantage of relevant portions of the centralized large foundation system while remaining more focused and optimized the edge site the local small foundation system is located. The sharing of the variables also allows for a tokenization and semantic compression manager to send tokens between the local small foundation system and the centralized large foundation system instead of the complete data leading to a reduction of data transmitted.