Large Language Model Context Selection for Large Documents
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems struggle to efficiently extract information from large sets of documents exceeding the capacity of machine learning models, such as large language models, due to limitations in processing and context handling.
Innovation Solution
A system and method that segments documents into subsets, creates semantic vectors, determines relevant document segments based on comparisons, and uses these segments as context for large language models to provide information extraction, enabling efficient processing of large document sets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the entire large document set is provided as context to the machine learning model, then complete information extraction is achieved, but the model capacity is exceeded and processing becomes infeasible
Solution Approach 1:
The system divides the large document set into multiple smaller chunks or segments. Each chunk is processed individually by the machine learning model, which has limited context capacity. The results from multiple chunks are then aggregated to provide complete information extraction, resolving the contradiction between model capacity and document size.
Solution Approach 2:
The system extracts only the most relevant information from each document chunk before processing, rather than feeding the entire document. This extraction step reduces the input size to fit within model capacity while preserving the essential information needed for accurate extraction.
2Speed
If document segments are selected based on single comparison method, then processing speed is improved, but selection accuracy deteriorates
Solution Approach 1:
The system combines multiple comparison methods (e.g., semantic similarity, keyword matching, vector cosine similarity) to evaluate and select relevant document segments. By merging multiple evaluation criteria, the system achieves both efficient processing and accurate identification of relevant segments, resolving the contradiction between speed and accuracy.
Data Source
AI summary
Systems and methods for using a machine learning model for a set of one or more documents are disclosed. Exemplary implementations may: create a set of document segments from the set of one or more documents; create a set of semantic vectors; create a query vector that semantically represents a query from a user; determine a subset of the set of semantic vectors based on at least two different comparisons involving the query vector; create a combination of the individual document segments that are associated with the subset of the set of semantic vectors; provide a prompt to the machine learning model, using the created combination of the individual document segments as context; present replies from the machine learning model, and/or perform other steps.