Permissioned LLM Document Search Under Prompt Size Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) are limited by size constraints and lack data security awareness, leading to inefficiencies and potential unauthorized access to sensitive information during document searches.
Innovation Solution
A document search system that generates prompts for LLMs based on relevant document portions, respecting user permissions and context, allowing for secure and efficient searching of large document sets without size constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire large set of documents is included in the LLM prompt, then the search comprehensiveness is improved, but the prompt size limit is exceeded and processing efficiency deteriorates
Solution Approach 1:
The patent divides the large document corpus into multiple smaller chunks or segments that can be individually processed by the LLM. Each chunk is within the prompt size limit, allowing the system to process comprehensive document sets by breaking them into manageable pieces that maintain both completeness and feasibility.
Solution Approach 2:
The patent introduces a new dimension of processing by implementing a multi-stage pipeline that transforms the problem from a single large prompt into multiple smaller prompts processed in sequence. This dimensional transformation allows comprehensive search across large corpora while respecting LLM prompt size constraints through iterative processing.
2Measurement precision
If all documents are provided to the LLM, then the accuracy of responses is improved, but the risk of unauthorized access to sensitive information increases
Solution Approach 1:
The patent introduces an intermediary component that acts as a gateway between the document corpus and the LLM. This intermediary filters and selectively provides only relevant, non-sensitive portions of documents to the LLM, maintaining response accuracy while preventing unauthorized access to confidential information through controlled information flow.
Solution Approach 2:
The patent applies different quality levels to different portions of the document corpus based on sensitivity and relevance. Sensitive or confidential sections are either excluded or redacted, while non-sensitive relevant sections are provided in full, creating a differentiated information provision strategy that maintains accuracy for permissible content while protecting sensitive data.
3Productivity
If document portions are selected based on relevance, then the processing efficiency is improved, but the system complexity increases
Solution Approach 1:
The patent performs preliminary actions by pre-processing the document corpus to identify and tag relevant portions before LLM processing. This advance preparation includes relevance scoring, filtering, and organization of documents, which enables efficient selective processing without requiring complex real-time analysis during the main search operation.
Data Source
AI summary
Computer-implemented systems and methods are disclosed, including systems and methods utilizing language models for searching a large corpus of data. A computer-implemented method may include: receiving a first user input comprising a natural language query; vectorizing the first user input into a query vector; executing, using the query vector, a similarity search in a document search model to identify one or more similar document portions, where the document search model includes a plurality of vectors corresponding to a plurality of portions of a set of documents; generating a first prompt for a large language model (“LLM”), the first prompt including at least the first user input, and the one or more similar document portions; transmitting the first prompt to the LLM; receiving a first output from the LLM in response to the first prompt; and providing, via a user interface, the first output from the LLM.