RAG Query Configuration for Low-Hallucination Domain Answers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) trained on publicly available documents struggle with domain-specific questions and exhibit hallucinations due to irrelevant or conflicting retrieved documents, leading to unreliable responses.
Innovation Solution
An enhanced Retrieval-Augmented Generation (RAG) architecture that includes a query augmentor to gather sufficient details through follow-up questions, a query configurator to categorize queries and assign document weights, and a document identifier to select the best document combinations for accurate responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If LLMs are trained on publicly available documents, then they can answer general questions, but they struggle with domain-specific questions and produce hallucinations
Solution Approach 1:
The system performs preliminary actions by retrieving relevant documents from external sources before the LLM generates its response. The query configurator prepares optimized search queries, and the document identifier pre-selects relevant documents based on query categories, ensuring domain-specific knowledge is available before generation occurs.
Solution Approach 2:
The patent introduces an intermediary retrieval-augmented generation architecture between the user query and the LLM. This intermediary system includes a query configurator, document identifier, and document retriever that mediate between the general-purpose LLM and domain-specific knowledge sources, filtering and preparing relevant information before presentation to the model.
2Loss of information
If RAG retrieves multiple documents to answer queries, then it provides more information, but irrelevant or conflicting documents cause hallucinations
Solution Approach 1:
The system applies local quality by treating different query categories with different document selection strategies. The query configurator identifies specific query categories (e.g., product information, troubleshooting, billing) and the document identifier selects documents optimized for each category, ensuring that each document retrieved is highly relevant to the specific type of question being asked.
Solution Approach 2:
The patent changes parameters by dynamically adjusting query configurations based on query categories. The query configurator modifies search parameters, document weights, and retrieval strategies according to the identified query type, optimizing the retrieval process for each specific domain and reducing the likelihood of retrieving irrelevant or conflicting information.
3Adaptability or versatility
If the LLM is given more context documents, then it can answer more complex questions, but the absence of relevant documents leads to hallucinations
Solution Approach 1:
The system implements dynamics by making the document retrieval process adaptive and flexible. The query configurator dynamically adjusts search strategies based on query complexity and category, and the document identifier flexibly selects from multiple document sources depending on the specific needs of each query, allowing the system to handle varying levels of question complexity while maintaining reliability.
Data Source
AI summary
A query is received from a client device. A subset of documents relevant to the query is determined in part by determining an optimal configuration for the query. The subset of documents is inputted in a context window for a response generator and the query is inputted as a prompt for a query response. The query response received from the response generator is outputted to the client device.