Hybrid RAG Query Routing for SQL and Vector Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing retrieval-augmented generation (RAG) systems struggle with handling structured and semi-structured data effectively, as similarity search-based retrieval approaches are not effective for Structured Query Language (SQL) tables, leading to inefficiencies and excessive workloads.
Innovation Solution
A RAG architecture that generates and executes both vector-based and SQL queries on different data repositories based on an initial user query, merging results, and selectively stores structured data in an SQL database to avoid excessive workload.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If similarity search-based retrieval is used for SQL tables, then unstructured data can be retrieved effectively, but structured data retrieval becomes inefficient and creates excessive workload
Solution Approach 1:
The patent segments the data retrieval process by creating separate retrieval pathways: one for unstructured data using vector similarity search and another for structured data using SQL queries. The system divides incoming queries and routes them to appropriate data repositories based on data type, avoiding the inefficiency of applying similarity search to structured SQL tables.
Solution Approach 2:
The patent changes the retrieval parameter based on data type: using vector similarity metrics for unstructured data and SQL query conditions for structured data. This parameter adaptation allows the system to optimize retrieval efficiency for each data type while minimizing unnecessary computational workload.
2Adaptability or versatility
If all data is processed through vectorization and similarity search, then unified retrieval approach is achieved, but processing time and computational resources increase significantly
Solution Approach 1:
The patent implements a dynamic retrieval strategy where the system adaptively selects the appropriate retrieval method based on the query type and data repository characteristics. Rather than uniformly applying vectorization to all data, the system dynamically chooses between similarity search and SQL queries, reducing processing time while maintaining versatility.
Solution Approach 2:
The patent performs preliminary classification of queries and data to determine the appropriate retrieval method before execution. By pre-identifying which data repositories contain structured versus unstructured data, the system can select the optimal retrieval approach in advance, avoiding unnecessary vectorization and reducing overall processing time.
3Ease of operation
If structured data is converted to unstructured format for retrieval, then simpler retrieval processes are used, but data structure information is lost and query accuracy decreases
Solution Approach 1:
Instead of converting structured data to unstructured format, the patent inverts the approach by keeping structured data in its native SQL format and applying SQL queries for retrieval. This inversion preserves the structural information and query accuracy while maintaining operational simplicity through automated query generation.
Data Source
AI summary
In some embodiments, a system generates a first query to be executed on a first data repository and a second query to be executed on a second data repository based on an initial user query using a first LLM, executes the first query on the first data repository to generate a first set of results, executes the second query on the second data repository to generate a second set of results, merges the first and second sets of results using a second LLM to form a merged set of results, selects a subset of the merged set of results based on a comparison of the merged set of results to the initial user query, generates a prompt based on the initial user query and the subset of the merged set of results, and submits the prompt to a third LLM to generate a response to the initial user query.