Retrieval-Augmented Q&A for Cited Answers From Document Corpora
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) face challenges in handling complex data types like tables and visuals, incorporating human feedback, and providing recommendations beyond answering questions, with issues in storage and memorization of facts leading to inaccuracies and lack of citation for outputs.
Innovation Solution
A retrieval-based augmentation method segments documents into latent semantic embeddings, allowing incorporation of complex data types and user feedback, and generates responses with embedded document segments and citations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If LLMs store facts in their neural network weights, then they can perform tasks using stored knowledge, but the quality and accuracy of stored facts cannot be verified and citations cannot be provided
Solution Approach 1:
The system segments knowledge storage into two separate components: (1) LLM weights for task performance and (2) external document corpus for factual storage and citation. This segmentation allows verification of fact quality through the retrieval component while maintaining the LLM's task execution capabilities, resolving the contradiction between reliability and citability.
Solution Approach 2:
The system introduces an intermediary retrieval mechanism that acts as a bridge between the LLM and external knowledge sources. This intermediary retrieves relevant document segments and provides citations, allowing the LLM to use external facts without storing them in its weights, thus maintaining both accuracy and citability.
2Adaptability or versatility
If LLMs are trained on finite text corpora, then they can answer questions from training data, but they cannot answer questions about documents not present in training data or produced after the information cut-off date
Solution Approach 1:
The system performs preliminary indexing and embedding of documents in the external corpus before they are needed for query processing. This preliminary action enables the system to quickly retrieve and provide accurate answers about new documents without requiring retraining of the LLM, thus improving adaptability while maintaining reliability.
3Reliability
If LLMs are trained or fine-tuned on new documents, then they can incorporate updated information, but the training process is expensive and time-consuming
Solution Approach 1:
The system uses an intermediary retrieval mechanism that accesses updated information from an external document corpus without requiring LLM fine-tuning. When documents are updated, only the embedding index needs to be refreshed, not the entire LLM, thus incorporating updated information efficiently while maintaining reliability.
4Reliability
If retrieval-based augmentation includes related document content as raw text in the prompt, then the LLM can answer questions using retrieved information, but it becomes difficult to incorporate complex data types like tables, figures, and videos
Solution Approach 1:
The system extracts and separates different data types from documents, converting complex data types (tables, figures, videos) into text descriptions or structured formats that can be included in the prompt. This extraction process enables the LLM to access and reason about complex data while maintaining the ability to answer questions using retrieved information.
Data Source
AI summary
Large language models (LLMs) are versatile in responding to user questions on a wide variety of topics. However, LLMs suffer from several drawbacks, such as hallucinations, incomplete information, and inability to cite original sources of information. Disclosed herein are systems and methods for using an LLM in a restricted manner to respond to queries regarding document corpora, e.g., documents related to a set of products, such that the impact of these drawbacks is minimized. Information retrieval is coupled with LLMs to build a question and answer (Q&A) system on the text corpora. Complex retrieved information, incorporating human feedback, and recommendations in the Q&A system are provided.