RAG Query Response Pipeline With Synthetic Chunk Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) struggle to provide accurate responses to queries related to information outside their training dataset, such as enterprise-specific knowledge, necessitating a more efficient and automated method for generating responses using retrieval augmented generation (RAG) systems.
Innovation Solution
A system utilizing a synthetic generation LLM to generate synthetic information, an embedding model to create vectors, an information retrieval system to identify relevant chunks, a re-ranker LLM to rank these chunks via chain-of-thought prompting, and a generation LLM to generate responses based on a selected subset of chunks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If LLMs are used to generate responses to queries, then response generation capability is improved, but accuracy deteriorates when queries relate to information outside training data
Solution Approach 1:
The patent introduces an intermediary RAG system that mediates between the LLM and the corpus of documents. The RAG system retrieves relevant documents from the corpus using the query, then feeds both the query and retrieved documents to the LLM for generating accurate responses. This intermediary layer enables the LLM to access enterprise-specific knowledge without retraining.
Solution Approach 2:
The patent performs preliminary retrieval of relevant documents from the corpus before the LLM generates its response. By pre-fetching and preparing relevant information based on the query, the system ensures that the LLM has access to necessary context, improving response accuracy for enterprise-specific topics without requiring the LLM to be retrained on this data.
2Reliability
If manual search through documents is performed, then information accuracy is improved, but time consumption increases
Solution Approach 1:
The patent replaces the manual mechanical search process with an automated RAG system that uses query-based retrieval mechanisms. Instead of manually scanning documents, the system automatically retrieves relevant information from the corpus using the query, significantly reducing time while maintaining accuracy.
Solution Approach 2:
The RAG system performs self-service by automatically retrieving and processing relevant information from the corpus without human intervention. The system independently queries the corpus, retrieves appropriate documents, and prepares them for response generation, eliminating the need for manual document search.
3Reliability
If RAG system is implemented, then response accuracy for external knowledge is improved, but system complexity increases
Solution Approach 1:
The patent segments the response generation system into distinct functional modules: a retrieval module that queries the corpus, a processing module that prepares retrieved information, and a generation module that produces responses using the LLM. This segmentation makes the complex RAG system more manageable and easier to implement by dividing functionality into separate, well-defined components.
Data Source
AI summary
Systems and methods for generating a response to a query based on a corpus of documents. The method comprising: subdividing each document into one or more chunks; using an LLM to generate synthetic information related to each chunk; generating, using an embedding model, a plurality of vectors for each chunk, the plurality of vectors for a chunk comprising a vector generated from the chunk, and a vector generated from the related synthetic information; using an information retrieval system to identify, from the plurality of vectors for each chunk, a set of chunks that are relevant to a query; using an LLM to rank the set of chunks based on their relevance to the query via chain-of-thought prompting; selecting a subset of chunks from the set of chunks based on the ranking; using an LLM to generate a response to the query based on the subset of chunks.