RAG Retrieval Pipeline with Query Rewriting and Chunk Reranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) struggle with knowledge-intensive tasks that require up-to-date information beyond their initial training data, leading to inconsistent responses.
Innovation Solution
A retrieval system pipeline in a RAG architecture that optimizes searching and ranking phases using query rewriting algorithms and a reranking model with a decoder-only model to generate embeddings, enhancing the accuracy and relevance of responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If LLMs are used for knowledge-intensive tasks, then they can understand and generate content, but they fail to provide accurate and up-to-date information outside their training data
Solution Approach 1:
The patent introduces a retrieval system as an intermediary between the user query and the LLM. This retrieval system searches external knowledge bases and training documents to fetch relevant, up-to-date information before passing it to the LLM. This mediator component enables the LLM to access current information outside its training data while maintaining its content generation capabilities, thus resolving the contradiction between versatility and reliability.
2Reliability
If information is retrieved from external sources, then LLM responses become more accurate and up-to-date, but the system complexity increases
Solution Approach 1:
The patent segments the system into distinct functional modules: a retrieval system with query processing and document search capabilities, an embedding model for generating vector representations, and the LLM for content generation. This segmentation allows each component to specialize in its function, making the overall complex system more manageable and maintainable while achieving improved response accuracy through external information retrieval.
3Reliability
If a retrieval system is implemented, then information accuracy improves, but the processing time and computational resources increase
Solution Approach 1:
The patent implements preliminary action by pre-processing queries into embedded representations and pre-searching the knowledge base before the LLM generates responses. The retrieval system performs document fetching and embedding generation in advance, so that when the LLM receives the query, the relevant information is already prepared and ready, reducing the overall response time while maintaining high information accuracy.
Data Source
AI summary
In some embodiments, a system transforms an initial user query into a first rewritten query using a first query rewriting algorithm, executes a search of a data repository using the first rewritten query to generate a set of results, executes a chunking process on the set of results to generate chunks of data, transforms the initial user query into a second rewritten query using a second query rewriting algorithm, generates corresponding embeddings for the second rewritten query and the chunks of data using a reranking model, selects a subset of the chunks of data based on a comparison of the embeddings for the chunks of data and the embedding for the initial user query, generates a prompt based on the initial user query and the subset of the chunks of data, submits the prompt to a Large Language Model (LLM) to generate a response to the initial user query.