Domain-Specific Vector Embedding Index for LLM Query Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Large Language Models (LLM) face challenges in providing optimal results for domain-specific queries due to a lack of specialized knowledge and context understanding, and they struggle with integrating new information and maintaining relevance and accuracy over time.
Innovation Solution
The proposed method, Retrieval Augmentation Generation (RAG), enhances LLM capabilities by classifying queries into specific domains, retrieving domain-specific vector embeddings, and using these embeddings to prompt the LLM, thereby grounding responses in relevant index results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If LLM is used for domain-specific queries, then general language understanding is improved, but domain-specific accuracy deteriorates
Solution Approach 1:
The patent introduces a retrieval system as an intermediary between the user query and the LLM. This intermediary retrieves domain-specific information from external sources and provides it to the LLM, enabling the model to answer domain-specific queries accurately without requiring domain-specific fine-tuning. The retrieval component acts as a mediator that bridges the gap between general language understanding and specialized knowledge.
Solution Approach 2:
The system segments the knowledge base into multiple domains, each with its own vector index. When a query is received, the system first classifies the query into a specific domain, then retrieves information only from that domain's index. This segmentation allows the LLM to access relevant domain-specific information without being overwhelmed by unrelated data, improving both accuracy and efficiency.
2Reliability
If LLM is fine-tuned with added information, then knowledge is improved, but computational cost deteriorates
Solution Approach 1:
The patent pre-processes and stores information in vector embeddings during an offline phase, organizing it into domain-specific indexes. This preliminary action allows the system to quickly retrieve relevant information during online query processing without performing expensive computations in real-time. The heavy lifting of organizing knowledge is done beforehand, reducing the computational burden during actual use.
Solution Approach 2:
The system extracts only the necessary domain-specific information from the knowledge base and provides it to the LLM through retrieval augmentation. Instead of fine-tuning the entire model with all available information, the system extracts and provides only the relevant snippets needed to answer the specific query, significantly reducing the computational resources required.
3Stability of the object's composition
If LLM processes all queries uniformly, then consistency is improved, but efficiency deteriorates
Solution Approach 1:
The patent introduces dynamic query classification that adapts the retrieval process based on the specific domain and type of each query. Instead of a static uniform processing approach, the system dynamically determines which domain index to search and how many top results to retrieve, optimizing the balance between consistency and efficiency for each individual query while maintaining overall system coherence.
Data Source
AI summary
Aspects of the invention provide a method, computer system, and computer program product for retrieval augmented generation. In one aspect, the method includes receiving a query. The method further includes classifying the query to a first domain within a multitude of domains. The method additionally includes retrieving an index of domain-specific vector embeddings corresponding to the domains. The method further includes prompting a large language model (LLM) with the query and the domain-specific vector embeddings. The method also includes receiving a query response from the LLM as grounded with the most relevant index results. The method further includes forwarding the query response.