Long-Document QA Evidence Retrieval Using Structured LLM Section Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional long document question answering (LDQA) systems face challenges due to transformer-based models' token limits, leading to inefficient processing and high computational costs, especially when dealing with long documents, and existing retrieve-then-read techniques rely on supervised fine-tuning with poor generalization on out-of-distribution data.
Innovation Solution
Utilizing large language models (LLMs) to process a condensed document representation based on structure, such as headings and summaries, to identify relevant sections, followed by fine-grained evidence retrieval, reducing the amount of text processed while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If transformer-based pretrained language models process entire long documents to answer questions, then answer accuracy can be maintained, but processing time and computational costs increase significantly
Solution Approach 1:
The patent divides the long document into multiple sections based on structural elements (headings, subheadings, paragraphs). The system processes these sections separately rather than treating the document as a single block, enabling efficient identification of relevant portions without processing the entire document.
Solution Approach 2:
The system extracts and identifies only the relevant sections and passages from the long document that are necessary to answer the given question. By taking out only the essential portions rather than processing the whole document, the system maintains answer accuracy while significantly reducing processing time and computational resources.
2Productivity
If transformer-based models process long documents within token limits by chunking, then processing becomes feasible, but the system loses overall document context and requires sophisticated reasoning to connect dispersed information
Solution Approach 1:
The system performs preliminary analysis by examining the document structure (headings, subheadings, section titles) before processing the full content. This preliminary action creates a structural framework that preserves document context and organization, allowing the model to understand the overall document layout without immediately processing all tokens.
Solution Approach 2:
The patent introduces a structural dimension by utilizing the document's hierarchical structure (sections, subsections, paragraphs) as an additional layer of information. This structural dimension allows the system to navigate and understand document context without being constrained by token limits, effectively adding a new way to access and process information.
3Reliability
If supervised fine-tuning is used for evidence selection in retrieve-then-read methods, then initial performance can be achieved, but generalization on out-of-distribution data remains poor
Solution Approach 1:
The system employs a self-service mechanism where the model uses its own internal reasoning capabilities to identify relevant sections based on the question and document structure, rather than relying on externally trained fine-tuned parameters. This self-service approach allows the model to adapt to different documents and questions without requiring supervised fine-tuning, improving generalization while maintaining performance.
Data Source
AI summary
Embodiments are disclosed for long document question answering using large language models. The method may include receiving a question for a document. A representation of the document may then be obtained. A large language model (LLM) is used to identify one or more sections of the document that are relevant to the question using the representation of the document. A document question answering model determines an answer to the question using the one or more sections of the document.