Federated Enterprise LLM Ensembles for Context-Aware Answers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI platforms rely on a single large language model (LLM) trained with a large dataset, which may not produce optimal results and do not consider user-specific factors in generating responses, leading to answers that may not be understood by the user.
Innovation Solution
Generate contextually distinguished enterprise language models (ELLMs) based on enterprise data, employee access levels, and question context, and combine them with public LLMs to provide tailored responses using an ensemble model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single large language model is used to answer queries, then the system complexity is low, but the answer quality and user understanding are insufficient
Solution Approach 1:
The patent combines multiple LLMs into an ensemble system where each model processes the same query independently, and their responses are aggregated through voting or averaging mechanisms. This merging of multiple models improves answer reliability while managing system complexity through modular architecture.
Solution Approach 2:
The system segments the single LLM functionality into multiple specialized LLMs that can be selectively applied based on query type, user role, or topic domain. This segmentation allows the system to maintain lower complexity for simple queries while providing enhanced reliability for complex queries through targeted model ensembles.
2Measurement precision
If enterprise-specific data is used to train LLMs, then the relevance and accuracy of answers improve, but the data processing and model training complexity increase
Solution Approach 1:
The system performs preliminary actions by pre-processing and curating enterprise data before model training, organizing it into structured formats with metadata tagging. This preliminary preparation reduces the complexity of ongoing data processing while maintaining high answer accuracy through pre-validated training datasets.
Solution Approach 2:
The system implements self-service mechanisms where the LLMs automatically adapt to enterprise data characteristics and user preferences over time through continuous learning from interaction patterns, reducing the need for manual data processing intervention while maintaining high accuracy.
3Adaptability or versatility
If multiple LLMs are combined in an ensemble, then the answer quality and user-specific tailoring improve, but the computational resources and time required increase
Solution Approach 1:
The system applies partial action by selectively engaging only the necessary subset of LLMs based on query complexity and user role, rather than always deploying the full ensemble. This approach maintains user-specific tailoring capability while reducing response time for straightforward queries that require fewer model evaluations.
4Reliability
If LLMs are trained with classified enterprise data at different access levels, then the security and relevance of responses improve, but the model generation and management complexity increase
Solution Approach 1:
The system applies local quality by creating specialized LLM variants trained on specific classified data subsets corresponding to different security clearance levels and departmental domains. Each LLM possesses localized knowledge appropriate to its training data classification, enabling secure and relevant responses while managing complexity through clear model versioning and access control mappings.
Data Source
AI summary
Systems and methods for using enterprise data to train and generate enterprise related large language models (ELLMs) and selectively combining the generated ELLMs and publicly available LLMs to generate a customized LLM for automatically answering queries are disclosed. Enterprise data is obtained, classified, and curated to generate training data that is used to generate a plurality of contextually distinguished ELLMs. The ELLMs differ based on their relation to various enterprise functions, departments, and level of authorized access to enterprise confidential data. When a question is received from a user, both the identity of the user and context of the question are used to select one or more ELLMs and LLMs for processing the question. The processing involves sequential selection of a plurality of ELLMs to process the question, blend the results from the processing, and use the results as input into an ensemble model to obtain a final answer.