Large Document Query Segmentation for LLM Context Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems struggle to efficiently extract information from large sets of documents, such as those spanning over 200 pages, due to capacity limitations of machine learning models like large language models.
Innovation Solution
A system and method that segments documents into smaller units, creates semantic vectors, determines relevant subsets based on comparisons, and uses these segments as context for queries in large language models to provide information extraction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the entire large document set is provided as context to the machine learning model, then complete information extraction is achieved, but the model capacity is exceeded
Solution Approach 1:
The patent segments the large document set into multiple smaller chunks or blocks. Each chunk is processed separately by the machine learning model, which has limited context capacity. The segments are then combined and aggregated to form the complete extracted information, resolving the contradiction between providing complete information and staying within model capacity limits.
2Quantity of substance
If multiple segments are processed separately and combined, then model capacity is respected, but processing complexity increases
Solution Approach 1:
The patent merges multiple processed segments by combining their extracted information into a unified result. This aggregation step consolidates the outputs from individual segment processing, managing the complexity of handling multiple separate processing operations while achieving complete information extraction.
3Quantity of substance
If the document is divided into segments, then it fits within model capacity, but the number of processing steps increases
Solution Approach 1:
The patent divides the large document into manageable segments that fit within the machine learning model's context capacity. This segmentation enables parallel or sequential processing of smaller units, reducing the time required for each individual processing step while maintaining complete information extraction through aggregation of all segments.
Data Source
AI summary
Systems and methods for using a machine learning model for a set of one or more documents are disclosed. Exemplary implementations may: create a set of document segments from the set of one or more documents; create a set of semantic vectors; create a query vector that semantically represents a query from a user; determine a subset of the set of semantic vectors based on at least two different comparisons involving the query vector; create a combination of the individual document segments that are associated with the subset of the set of semantic vectors; provide a prompt to the machine learning model, using the created combination of the individual document segments as context; present replies from the machine learning model, and/or perform other steps.