Layered Language Models for Long-Document Question Answering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) face limitations in processing large documents due to capacity, capability, latency, and result quality issues, leading to suboptimal question answering performance.
Innovation Solution
A layered approach using multiple LLMs for different tasks, including outline generation, metadata extraction, and natural language response synthesis, to provide structured data for improved accuracy and latency in document question answering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single large language model is used to process the entire document, then the model has sufficient capacity to understand the document, but the processing latency increases and resource usage increases
Solution Approach 1:
The patent divides the document processing task into multiple segments handled by different LLMs. One LLM generates an outline of the document, another extracts metadata, and a third generates the final answer. This segmentation allows each model to process only a portion of the document, reducing overall processing latency while maintaining understanding accuracy through coordinated multi-model processing.
Solution Approach 2:
The patent applies preliminary action by having one LLM generate an outline and another extract metadata before the final answer-generation LLM processes the query. These preliminary processing steps organize the document content in advance, enabling the final model to work with structured information rather than raw text, thereby reducing the time needed for the critical answer generation while preserving comprehension quality.
2Productivity
If a smaller, faster LLM is used to reduce latency, then the processing speed increases, but the result quality and accuracy decrease
Solution Approach 1:
The patent merges the capabilities of multiple LLMs with different strengths. One LLM is optimized for speed and generates an outline, another focuses on metadata extraction, and a third generates the final answer. By combining these specialized models, the system achieves both high processing speed and high result quality, as each model contributes its specific capability to the overall task.
Solution Approach 2:
The patent creates a multi-functional system where different LLMs perform different functions (outline generation, metadata extraction, answer generation) rather than relying on a single universal model. This allows the system to leverage the strengths of each model for its specific function, achieving both speed and quality that would be difficult to obtain from a single model optimized for one purpose.
3Measurement precision
If the entire document is processed to ensure comprehensive understanding, then the answer accuracy improves, but the resource usage and processing time increase
Solution Approach 1:
The patent extracts only the essential elements of the document (outline and metadata) using specialized LLMs before the final answer-generation model processes the query. This extraction approach allows the system to capture the most important information without processing every detail of the entire document, thereby reducing computational resource usage while maintaining answer accuracy through focused processing of key content.
4Productivity
If multiple LLMs are used for different tasks to improve accuracy and reduce latency, then the system performance improves, but the device complexity increases
Solution Approach 1:
The patent uses an intermediary controller that coordinates between multiple LLMs, managing the flow of information and task distribution. This intermediary layer simplifies the overall system architecture by providing a centralized coordination point, making the complex multi-model system more manageable while still achieving improved performance through specialized task assignment.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a set of large language models to determine a natural language response to a query. One of the methods includes receiving a query related to a document. The document is submitted to a first model along with a prompt to generate an outline of the document. The document is submitted to a second model along with a prompt to generate metadata of the document. At least a portion of the query, document metadata, and the document outline are submitted to a third model with a prompt to generate a natural language response to the query. A selected sentence from the natural language response is correlated to a document sentence. The natural language response is provided to the user with an indication that the selected sentence from the natural language response is correlated to the document sentence.