Multi-Level H-LLM Routing for Scalable AI Supercomputing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI generative models and large language models (LLMs) require significant supercomputing efforts to improve response times, accuracies, and reduce computational load, limiting their scalability and expandability.
Innovation Solution
A multi-level generative AI system utilizing derived requests, multiple h-LLMs with varying levels of accuracy, and a distributed architecture to efficiently process and assign tasks, incorporating techniques like bagging, boosting, and merging/fusing h-LLMs to enhance precision and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional machine learning algorithms are used for content analysis, then computational efficiency is maintained, but the ability to generate new content matching associated or preceding/future content is limited
Solution Approach 1:
The system segments the generative AI workload into multiple levels: LLM routing layer, h-LLM generation layer, and local execution layer. This segmentation allows different types of processing to be distributed appropriately, reducing the computational load on centralized supercomputing resources while maintaining generative capabilities through local h-LLM execution.
Solution Approach 2:
The patent introduces an LLM router as an intermediary that directs requests to appropriate h-LLM models based on task requirements. This intermediary layer optimizes resource allocation by matching computational tasks with suitable model capabilities, reducing unnecessary computational overhead while maintaining generative output quality.
2Measurement precision
If large language models with billions of parameters are deployed to improve response accuracy, then content generation quality improves, but scalability and expandability are limited due to significant supercomputing requirements
Solution Approach 1:
The system implements local quality by deploying specialized h-LLM models at edge locations and local servers rather than requiring all processing to occur on centralized supercomputing infrastructure. Each local h-LLM is optimized for specific tasks, providing high accuracy for local operations without requiring access to massive centralized computing resources.
Solution Approach 2:
The patent transitions from a single-dimension centralized supercomputing architecture to a multi-dimensional distributed architecture spanning cloud, edge, and local devices. This dimensional expansion allows the system to achieve high response accuracy through local h-LLMs while improving scalability by distributing computational requirements across multiple dimensions of the infrastructure.
3Reliability
If centralized supercomputing resources are used to process generative AI requests, then model accuracy can be maintained, but response times increase and system scalability is reduced
Solution Approach 1:
The system performs preliminary action by pre-loading and caching frequently used h-LLM models at local edges and servers. This allows the system to quickly serve common requests from local cached models without requiring real-time access to centralized supercomputing resources, thereby reducing response times while maintaining model accuracy for cached operations.
Solution Approach 2:
The patent implements a dynamic architecture where the system automatically routes requests based on real-time conditions: simple requests are handled by local h-LLMs for fast response, while complex requests requiring higher accuracy are dynamically routed to centralized LLM resources. This dynamic routing optimizes both response time and model accuracy based on request characteristics.
Data Source
AI summary
Systems and methods of answering queries using one or more h-LLMs including receiving user prompts via an API at a query layer from a user interface, determining an analysis mode of the user prompts to be a real-time mode, transmitting a query prompt including the user prompts from the query layer to a real-time layer responsive to determining the analysis mode being a real-time mode, the real-time layer including an h-LLM and being capable of generating a result in a response period on the order of seconds, receiving a real-time response from the real-time layer responsive to the query prompt, and transmitting the real-time response to the user interface via the API.