Multi-Level AI Supercomputer Design with Broker-Based LLM Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI generative models and large language models (LLMs) require significant supercomputing efforts to improve response times, accuracy, and reduce computational load, limiting their scalability and expandability.
Innovation Solution
A system and method for multi-level generative AI and LLMs that utilize derived requests, multiple h-LLMs with varying levels of accuracy, and a distributed architecture, including a local database and service mesh, to optimize workload assignment and result weighting, enhancing computational efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional machine learning algorithms are used for content analysis, then computational efficiency is maintained, but the ability to generate new content matching associated or preceding/future content is limited
Solution Approach 1:
The system segments the monolithic LLM into multiple specialized h-LLMs, each trained on specific datasets for particular tasks. This segmentation allows the system to use only the necessary specialized models for each request rather than running a full-scale LLM, reducing computational load while maintaining generative capabilities.
Solution Approach 2:
The system dynamically selects and weights multiple h-LLMs based on the specific characteristics of each input request. The AI broker evaluates derived requests and determines the optimal combination of h-LLMs to use, adjusting the system's behavior in real-time to balance generative capability with computational efficiency.
2Measurement precision
If a single large LLM is used to handle all tasks, then model accuracy is maintained, but response times increase and scalability is limited
Solution Approach 1:
The system divides the single large LLM into multiple smaller h-LLMs, each specialized for specific tasks. This segmentation enables parallel processing of different aspects of a request across multiple models, significantly reducing response time while maintaining overall accuracy through the aggregation of specialized results.
Solution Approach 2:
The system changes the parameter of model specialization by training different h-LLMs on specific datasets for particular tasks rather than using a general-purpose large LLM. This parameter change allows each model to be more efficient at its specific task, improving both speed and accuracy for targeted functions.
3Productivity
If multiple specialized h-LLMs are deployed to reduce computational load, then scalability is improved, but system complexity increases
Solution Approach 1:
The AI broker serves as a universal coordinating component that manages multiple specialized h-LLMs. It provides a unified interface for handling diverse requests, evaluating derived requests, selecting appropriate models, and aggregating results. This multi-functional broker simplifies the overall system architecture by centralizing management functions.
Solution Approach 2:
The AI broker acts as an intermediary between the user and the multiple h-LLMs. It receives input, generates derived requests, selects appropriate h-LLMs, and combines their outputs. This intermediary layer shields users from the complexity of the multi-model system while enabling efficient utilization of multiple specialized models.
4Measurement precision
If LLMs are trained on vast amounts of data to capture complex language patterns, then language understanding accuracy is improved, but training time and computational resources increase
Solution Approach 1:
Instead of training one large LLM on all possible language data, the system segments the training process into multiple h-LLMs, each trained on specific subsets of data for particular tasks. This segmentation dramatically reduces the training time for each model while collectively achieving comprehensive language understanding across multiple domains.
Solution Approach 2:
The system performs preliminary training of multiple specialized h-LLMs on specific datasets before deployment. This preliminary action allows each model to be pre-adapted to its specific task domain, reducing the need for extensive fine-tuning at deployment and enabling faster response times while maintaining high accuracy.
Data Source
AI summary
A system for answering queries using one or more families of large language models (h-LLMs) including a processor, a communication device, one or more h-LLMs implemented as microservices in cloud container environments and being accessible via a cloud service API, and software that is executable to operate an input broker having a broker API operable to receive a user prompt from a user interface, generate a plurality of derived prompts, transmit the plurality of derived prompts to one or more h-LLMs via the cloud service API, operate an output broker operable to receive a plurality of h-LLM results, process the plurality of LLM results at the output broker to generate a result, and transmit the result to the user interface via the broker API.