Retrieval-Augmented Question Answering to Reduce LLM Hallucinations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) like ChatGPT often provide incorrect answers due to 'hallucinations' when prompted with queries unrelated to their internal knowledge sources, limiting the effectiveness of open domain question-answering systems in contact centers.
Innovation Solution
A system that uses enhanced retrieval-augmented generation, involving pre-processing user queries, retrieving relevant data from a knowledge base using keyword and semantic indices, and prompting a large language model to generate answers based on this data, with confidence checks and filters to ensure accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a large language model is prompted to answer queries using its world knowledge, then it can provide quick responses, but it results in hallucinations and incorrect answers
Solution Approach 1:
The system performs preliminary retrieval of relevant information from the knowledge base before prompting the LLM. The query is processed to identify and retrieve pertinent documents or data segments, which are then provided as context to the LLM. This preliminary action ensures the model has accurate reference material before generating answers, reducing hallucinations while maintaining response speed.
2Reliability
If retrieval-augmented generation is used to improve answer accuracy, then hallucinations are reduced, but system complexity increases
Solution Approach 1:
The system segments the knowledge base into organized collections or groups of related documents. The retrieval process is divided into distinct steps: query processing, candidate document selection, and context preparation. This segmentation makes the complex retrieval-augmented generation system more manageable and maintainable while preserving answer accuracy.
3Reliability
If relevant data is retrieved and verified with confidence checks, then answer reliability improves, but processing time increases
Solution Approach 1:
The system retrieves more candidate documents than strictly necessary and applies confidence scoring to evaluate their relevance. Documents below a confidence threshold are discarded, and only high-confidence results are used for answer generation. This partial verification approach maintains reliability while avoiding excessive processing time by not verifying every retrieved document.
Data Source
AI summary
A method of question answering using enhanced retrieval-augmented generation according to an embodiment includes receiving, by a computing system, a user query, pre-processing, by the computing system, the user query to determine whether the user query is associated with malicious intent, retrieving, by the computing system, relevant data from a knowledge base by using a keyword index and a semantic index in response to determining that the user query is not associated with malicious intent, prompting, by the computing system, a large language model to generate an answer to the user query based on only the relevant data retrieved from the knowledge base, and receiving, by the computing system, the answer to the user query from the large language model in response to the prompt.