LLM Answer Retrieval With Context Pages for Hallucination Checks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) like GPT and PaLM2 can generate improper or hallucinated answers, leading to users accepting untrue information.
Innovation Solution
An information retrieval system that utilizes a document database associating text groups with characteristic vectors and page numbers, retrieves context information using a characteristic vector of a question, generates a prompt incorporating the question and context, and outputs answers with associated page numbers for validation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a large language model is used to generate answers to questions, then the answer can be generated in a natural language, but the model may produce hallucinated or improper answers that users may mistakenly accept as correct
Solution Approach 1:
The patent introduces context information as an intermediary element between the user's question and the LLM's answer. The context information retrieving unit fetches relevant documents from a database, and the prompt generating unit incorporates this context into the prompt sent to the LLM. This intermediary context acts as a mediator that guides the LLM to generate more accurate and reliable answers while maintaining natural language processing capabilities.
2Reliability
If context information and page numbers are added to the answer output, then users can verify the validity of the answer, but the output structure becomes more complex
Solution Approach 1:
The patent segments the answer output into distinct components: the LLM-generated answer, the context information, and the page numbers. This segmentation allows each component to serve its specific purpose - the answer provides the response, the context information provides the source material for verification, and the page numbers enable precise location in the original documents. This structured segmentation improves verifiability while keeping the output organizationally clear.
Data Source
AI summary
This information retrieval system provides an answer corresponding to a question using a large language model. A context information retrieving unit retrieves a document database with a characteristic vector of the question and thereby acquires as context information a text group that a similarity level between the characteristic vector and a characteristic vector of the text group satisfies a predetermined condition. In the document database, character vectors and page numbers of text groups obtained by dividing a document are registered. A prompt generating unit generates a prompt that includes the question and the context information. An answer acquiring unit acquires an answer corresponding to the prompt using a large language model. An answer outputting unit outputs as an answer corresponding to the question the context information and a page number associated with the context information together with the answer corresponding to the prompt.