Embedding Granularity Control for Adaptive RAG Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current large language model (LLM) systems are inflexible due to fixed granularity in embedding context addition and vector similarity metrics, limiting their ability to control retrieval augmented generation (RAG) mechanisms.
Innovation Solution
The method involves dividing a document corpus into multiple fragments based on different threshold sizes for various LLMs, computing corresponding embeddings, and aggregating responses from these fragments to enhance natural language understanding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a fixed degree of granularity is used for embedding context addition, then the system operation is simplified, but the adaptability of the RAG mechanism is reduced
Solution Approach 1:
The patent implements dynamic granularity control by allowing the system to adjust the fragmentation level of document embeddings based on query requirements. Different granularity levels (e.g., sentence-level, paragraph-level, document-level) can be selected dynamically, enabling the RAG mechanism to adapt between fine-grained and coarse-grained context retrieval as needed, thus resolving the contradiction between operational simplicity and adaptability.
Solution Approach 2:
The system changes the granularity parameter of embeddings dynamically. By adjusting the fragmentation size parameter (from fine-grained sentences to coarse-grained documents), the system can optimize retrieval precision for different query types while maintaining a unified RAG framework, thereby achieving both ease of operation and high adaptability.
2Device complexity
If a fixed metric is used for vector similarity matching, then the system complexity is reduced, but the measurement precision of context relevance is limited
Solution Approach 1:
The patent enables dynamic selection and adjustment of similarity metrics (e.g., cosine similarity, Euclidean distance, Manhattan distance) based on the characteristics of the query and document corpus. This allows the system to optimize measurement precision for different scenarios without requiring a completely complex system architecture, as the metric selection can be implemented as a configurable parameter.
3Adaptability or versatility
If multiple sets of embeddings are computed for different LLMs, then the adaptability to different models is improved, but the computational resources required increase
Solution Approach 1:
The patent segments the document corpus into multiple embedding sets at different granularity levels (e.g., document-level embeddings, paragraph-level embeddings, sentence-level embeddings). This segmentation allows different LLMs to selectively use appropriate embedding sets based on their token limits and performance requirements, reducing the need to compute all possible embedding sets and thereby lowering computational resource consumption while maintaining model-specific adaptability.
Data Source
AI summary
In one embodiment, a method for managing embeddings and text for enhanced natural language understanding includes dividing, by a process, a corpus of one or more documents into a first plurality of fragments based on a first threshold size for a first large language model and computing, by the process, a first set of embeddings using the first large language model to analyze the first plurality of fragments. The method further includes dividing, by the process, the corpus of one or more documents into a second plurality of fragments based on a second threshold size for a second large language model and computing, by the process, a second set of embeddings using the second large language model to analyze the second plurality of fragments.