Weighted Query Embeddings for Accurate RAG Chunk Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative AI models like LLMs face challenges in providing accurate and relevant answers due to limited prompt sizes and ineffective chunking methodologies in Retrieval Augmented Generation (RAG), leading to incorrect or irrelevant results.
Innovation Solution
A method that involves extracting topics from queries and documents, generating weighted embeddings, and identifying candidate chunks for prompts to enhance semantic matching, using techniques like TF-IDF and BERT for improved chunk indexing and similarity search.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If RAG methodology is used to break documents into chunks for retrieval, then the prompt size limitation is addressed, but the accuracy and relevance of answers deteriorates due to poor chunking and mismatch between query and chunks
Solution Approach 1:
The patent segments documents into chunks and further segments queries into multiple topics with weightings. Each chunk is associated with multiple topic weightings that represent the semantic content distribution. This segmentation allows the system to handle large documents while improving retrieval accuracy by matching query topics to chunk topics with appropriate weightings, resolving the contradiction between handling large documents and maintaining answer accuracy.
Solution Approach 2:
The patent applies local quality by assigning different weightings to different topics within a query and to different chunks based on their semantic content. Instead of treating all query terms equally or all chunks uniformly, the system identifies the most relevant topics and chunks through topic extraction and weighting mechanisms, ensuring that only the most relevant information is used for generation, thereby improving answer reliability.
2Ease of manufacture
If common indexing and retrieval techniques are used to match queries to content chunks, then the retrieval process is simple, but the match quality deteriorates leading to incorrect content chunks being selected
Solution Approach 1:
The patent changes the parameters of query and chunk representations from simple text to weighted topic embeddings. Queries are transformed into multiple topics with weightings, and chunks are associated with multiple topic weightings. This parameter transformation enables more precise semantic matching while maintaining computational efficiency through embedding vectors, resolving the contradiction between retrieval simplicity and match quality.
3Productivity
If the prompt size limit is enforced by LLMs, then the model can process information efficiently, but the ability to search large documents deteriorates
Solution Approach 1:
The patent extracts the most relevant information from large documents by segmenting them into chunks and associating each chunk with multiple topic weightings. Only the top relevant chunks are selected for the prompt based on topic matching, effectively extracting only the necessary information needed for the query. This allows the system to work within prompt size limits while maintaining the ability to search through large document corpora.
Data Source
AI summary
A method, computer program product, and computing system for processing a query using a generative artificial intelligence (AI) model. A topic of the query is extracted. A weighting for the topic of the query is generated. A weighted query topic embedding for the topic of the query is generated. A candidate chunk is identified from a plurality of chunks of a target document by determining a similarity between the weighted query topic embedding and a plurality of chunk embeddings for the plurality of chunks. A prompt is generated using the query and the candidate chunk. The prompt is provided to the generative AI model.