Custom-Domain LLM Controller Using Preloaded Context Pools
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Public pre-trained large language models (LLMs) like ChatGPT lack domain-specific knowledge, requiring substantial computing resources and time for training, and their responses are often incorrect without additional input.
Innovation Solution
A custom-domain controller for LLMs that maintains a database of domain-specific text, prepares multiple LLM instances, and uses user corrections to enhance context, reducing the need for retraining by leveraging the LLM's internal learning capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If public pre-trained LLMs are used, then general text generation capability is provided, but domain-specific accuracy deteriorates
Solution Approach 1:
The system prepares multiple LLM instances in advance with different domain-specific context texts loaded into their memory before user interaction. This preliminary action allows the LLMs to have domain knowledge ready when needed, improving response correctness without requiring retraining.
Solution Approach 2:
A controller acts as an intermediary between the user and the LLM instances. The controller selects appropriate domain-specific context texts and injects them into the LLM memory based on user input, enabling the LLM to access domain knowledge without being retrained for each domain.
2Adaptability or versatility
If LLMs are retrained for specific domains, then domain-specific knowledge is improved, but computing resource consumption increases
Solution Approach 1:
The system extracts only the necessary domain-specific knowledge from training data and stores it as context text in a database. Instead of retraining the entire LLM, the relevant knowledge is extracted and injected into the LLM memory when needed, significantly reducing computing resource consumption.
Solution Approach 2:
Instead of retraining the LLM for each domain, the system creates copies of the base LLM instance and loads different domain-specific context texts into their memory. This copying approach allows multiple domain specializations without the high computational cost of retraining.
3Adaptability or versatility
If LLMs are retrained for specific domains, then domain expertise is improved, but training time increases
Solution Approach 1:
Domain-specific context texts are prepared and stored in advance in a database. When a user needs domain expertise, the controller quickly retrieves and injects the appropriate context text into the LLM memory, eliminating the need for time-consuming retraining processes.
Solution Approach 2:
The controller serves as an intermediary that rapidly matches user queries with appropriate domain context texts from the database and injects them into the LLM. This mediation process takes minimal time compared to retraining, enabling quick adaptation to different domains.
4Reliability
If domain-specific text is provided to LLM, then response accuracy is improved, but system complexity increases
Solution Approach 1:
The controller acts as an intermediary that manages the complexity of providing domain-specific text to the LLM. It automatically selects appropriate context texts based on user input and injects them into the LLM memory, shielding the user from the underlying system complexity while maintaining high response accuracy.
Solution Approach 2:
The system is designed to automatically manage domain-specific text provision without requiring user intervention. The controller self-services by detecting when domain knowledge is needed, retrieving appropriate context texts from the database, and injecting them into the LLM memory, thereby improving response accuracy while keeping the interface simple.
Data Source
AI summary
A database of text associated with different domains is maintained. Large language models (LLMs) are prepared for use in the different domains by providing the associated text to an instance of an LLM. Thus, using multiple instances of the same pre-trained LLM, domain-specific LLMs are generated. The text provided to the LLM instance may be selected based on an account identifier of the user accessing the LLM, the tenant accessing the LLM, a user selection of a domain, or any suitable combination thereof. A pool of prepared LLM instances may be generated before the access request is received. If a response provided by an LLM instance in a domain to a prompt was rejected by a user and additional information was received during the session to improve the response of the LLM instance, the additional information may be to the text used to prepare future LLM instances for the domain.