LLM Answer Basis Extraction for Faster Document Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large-scale language models (LLMs) generate answers based on referenced documents, but it is time-consuming for users to search through large volumes of text to verify the correctness of these answers, especially when documents are lengthy.
Innovation Solution
A computer system that includes a processor, storage, and a network interface, connected to a large-scale language model and a database, which performs answer generation and basis extraction processes to identify the referenced parts of documents for generating answers, allowing users to verify the correctness of LLM-generated responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the LLM generates answers by referencing large documents, then the accuracy and reliability of answers improve, but the time required for users to verify answers increases significantly
Solution Approach 1:
The patent extracts the basis part (reference sentences) from the large document that the LLM used to generate the answer. Instead of requiring users to search through the entire document, the system automatically identifies and presents only the specific sentences that form the basis of the answer, enabling quick verification without manual document search
Solution Approach 2:
The patent introduces a basis extraction module as an intermediary between the LLM and the user. This module automatically analyzes the document referenced by the LLM, identifies the relevant basis parts, and presents them to the user. The intermediary handles the time-consuming search task automatically, resolving the contradiction between answer reliability and verification time
2Quantity of substance
If the document contains many pages or characters, then more information is available for accurate answers, but searching for the basis sentence becomes considerably time-consuming
Solution Approach 1:
The system enables self-service by automatically performing the basis extraction task. The basis extraction module autonomously analyzes the referenced document, identifies the basis sentences, and presents them to the user without requiring manual search effort. The system serves itself by handling the time-consuming information retrieval task automatically
Solution Approach 2:
The patent replaces the mechanical manual search process with an automated computational system. Instead of users manually searching through large documents, the basis extraction module uses natural language processing and text analysis algorithms to automatically identify and extract the relevant basis sentences, substituting mechanical human effort with automated computational processing
Data Source
AI summary
When an LLM (Large Language Model) generates an answer to a question, it also presents the external information it referenced. The system includes an answer generation process using the LLM along with a basis extraction process to identify the specific external information sources referenced in generating the answer. The answer generation process creates an answer instruction, prompting the LLM to consider actions needed to obtain the answer. It then generates an answer sentence that includes the result of this reasoning, along with either the answer itself or information about the required actions. Additionally, the system executes these actions, records the execution results, metadata about the referenced external information, and the reasoning outcomes, forming a comprehensive history. The basis extraction process generates basis information to pinpoint relevant reference parts of the external information, based on multiple reference points. This setup enhances transparency and traceability of the information used by the LLM.