RAG Query Response Using Synthetic Chunk Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) struggle to provide accurate responses to queries related to information outside their training dataset, particularly when dealing with enterprise-specific or internal knowledge bases, necessitating a more efficient and automated response generation method.
Innovation Solution
A retrieval augmented generation (RAG) system that utilizes a synthetic generation LLM to generate synthetic information from document chunks, an embedding model to create vectors, and an information retrieval system to identify relevant chunks, followed by a re-ranker LLM to rank and select a subset of chunks, and a generation LLM to generate a response.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If LLMs are used to generate responses to queries, then automation and efficiency are improved, but accuracy deteriorates when queries relate to information outside the training dataset
Solution Approach 1:
An information retrieval system acts as an intermediary between the query and the LLM. The system first retrieves relevant information from external knowledge bases or document corpora, then provides this retrieved information to the LLM along with the original query. This intermediary step ensures the LLM has access to accurate, up-to-date information while maintaining automated response generation.
Solution Approach 2:
The system performs preliminary information retrieval before the LLM generates the response. By pre-fetching relevant documents or knowledge snippets that match the query intent, the system prepares accurate source material in advance, allowing the LLM to focus on synthesizing the final response rather than searching for information.
2Reliability
If RAG is used to retrieve information from external knowledge bases, then response accuracy is improved, but system complexity increases
Solution Approach 1:
The system segments the complex RAG architecture into distinct functional modules: a query processing component that breaks down user queries into searchable terms, a retrieval component that fetches relevant documents, and a generation component that synthesizes answers. This modular segmentation makes the overall system more manageable and easier to implement despite its complexity.
3Measurement precision
If manual searching of internal documents is performed, then information accuracy is improved, but time consumption increases
Solution Approach 1:
The system performs self-service by automatically retrieving and processing relevant information from internal document corpora without human intervention. The LLM independently searches, retrieves, and synthesizes information from the knowledge base, eliminating the need for manual document searching while maintaining high information accuracy through its advanced retrieval and comprehension capabilities.
Data Source
AI summary
Systems and methods for retrieving information from a corpus of documents that is relevant to a query. The method comprising: subdividing each document in the corpus of documents into one or more chunks; using a synthetic generation LLM to generate at least one piece of synthetic information related to each chunk; generating, using an embedding model a plurality of vectors for each chunk, the plurality of vectors for a chunk comprising a vector generated from the chunk, and a vector generated from each of the at least one piece of synthetic information related to that chunk; and using an information retrieval system to identify, from the plurality of vectors for each chunk, a set of chunks that are relevant to a query.