RAG Query Response Using Multi-Scale Document Chunk Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) struggle to provide accurate responses to queries related to information not included in their training dataset, necessitating the use of retrieval augmented generation (RAG) to retrieve relevant information from external knowledge bases.
Innovation Solution
An enhanced LLM-based RAG system that subdivides documents into chunks, uses an information retrieval system to identify relevant chunks, ranks them using a re-ranker LLM, and generates responses based on a subset of these chunks, leveraging query modification to enhance query relevance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If documents are subdivided into small chunks, then retrieval precision is improved, but information completeness deteriorates
Solution Approach 1:
The patent applies segmentation by dividing documents into multiple chunk sizes (first plurality of smaller chunks and second plurality of larger chunks) and creating separate search indexes for each. This allows the system to retrieve from appropriately-sized chunks based on query requirements, balancing precision and completeness.
Solution Approach 2:
The system dynamically selects which search index to query based on the specific query characteristics. The processor can adaptively choose to search smaller chunks for precise queries or larger chunks for queries requiring broader context, making the retrieval process flexible and context-aware.
2Loss of information
If documents are subdivided into large chunks, then information completeness is improved, but retrieval precision deteriorates
Solution Approach 1:
The patent creates multiple segmented versions of the same document corpus with different chunk sizes. Each segmentation serves a different retrieval need, allowing the system to optimize for either precision or completeness depending on the query type.
Solution Approach 2:
The system changes the parameter of chunk size by maintaining multiple search indexes with different granularities. This allows dynamic adjustment of the retrieval unit size to match the information needs of different queries.
3Measurement precision
If multiple search indexes are generated, then retrieval accuracy is improved, but system complexity increases
Solution Approach 1:
The multiple search indexes serve universal purposes - each type of index (small chunks, large chunks) can handle different query scenarios. This multi-functionality allows a single retrieval system to address both precision-oriented and completeness-oriented queries without requiring separate specialized systems.
Solution Approach 2:
The search indexes are pre-generated and stored before actual queries are processed. This preliminary action of indexing during document ingestion time allows for fast retrieval during query time without repeatedly processing the raw documents, reducing the computational complexity of the retrieval operation itself.
4Measurement precision
If all chunks are retrieved and ranked, then response accuracy is improved, but processing time increases
Solution Approach 1:
The system retrieves a partial set of chunks from the search index - specifically, it retrieves chunks from the first search index that are relevant to the query, then uses the second search index to filter and select only the most relevant subset. This partial action approach avoids processing all chunks while still maintaining high accuracy.
Solution Approach 2:
The system extracts and retrieves only the relevant subset of chunks needed for answering the specific query, rather than processing the entire corpus. The multi-index structure enables efficient extraction of relevant information without unnecessary processing of irrelevant chunks.
Data Source
AI summary
Systems and methods for retrieving information from a corpus of documents that is relevant to a query. The method comprises: generating a first plurality of chunks by subdividing each document in the corpus of documents into one or more chunks of a first size; generating a second plurality of chunks by subdividing each document in the corpus of documents into one or more chunks of a second, larger, size; using an information retrieval system to identify, from the second plurality of chunks, a set of chunks of the second size that are relevant to a query; and using the information retrieval system to identify, from a subset of chunks of the first plurality of chunks, a set of chunks of the first size that are relevant to the query. The subset is based on the set of chunks of the second size that are relevant to the query.