User Questions Repository for Natural Language to SQL Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) face challenges in directly using natural language questions for structured data querying due to differences in structure, syntax, and semantics between natural language and structured query languages like SQL.
Innovation Solution
A system and method that utilize a user questions repository (UQR) to convert natural language questions into structured queries by prompting a large language model (LLM) with pre-generated questions and metadata, employing techniques like retrieval augmented generation (RAG) for optimized outputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If LLMs are trained on general Internet data to provide natural language responses, then the LLMs can converse fluently with users, but the LLMs are insufficient to retrieve and provide accurate answers from structured data databases
Solution Approach 1:
The system segments the data retrieval process into two distinct components: (1) a semantic search component that handles natural language queries using LLMs, and (2) a structured query component that generates precise SQL queries from matched semantic results. This segmentation allows each component to specialize in its strength while compensating for the other's weaknesses.
Solution Approach 2:
The system introduces an intermediary component that acts as a bridge between natural language queries and structured database queries. This intermediary matches semantic search results with pre-generated SQL queries, ensuring accurate data retrieval while maintaining natural language interaction. The intermediary translates the intent from natural language to structured query format without losing accuracy.
2Loss of information
If the service provider shares all data view and table definitions with the LLM to enable complete data querying, then the LLM has access to all necessary information, but the LLM may not be capable of performing optimal data joins when there are many columns, views, and tables
Solution Approach 1:
The system extracts only the necessary data context from the complete database schema by matching semantic search results with relevant pre-generated SQL queries. Instead of providing all table and view definitions to the LLM, the system extracts and provides only those specific queries that are relevant to the user's intent, reducing complexity while maintaining completeness.
Solution Approach 2:
The system performs preliminary actions by pre-generating SQL queries for various data views and tables before runtime. These pre-generated queries are stored and matched against semantic search results during execution. This preliminary preparation reduces the real-time complexity of data join operations by having the heavy lifting done in advance.
3Adaptability or versatility
If the LLM is trained to be general-purpose for various tasks, then the LLM can handle diverse user questions, but the LLM cannot infer which specific views or tables from the entire set are most relevant for the question at hand
Solution Approach 1:
The system implements feedback by matching semantic search results with pre-generated SQL queries and using this match information to identify the most relevant data sources. The feedback loop continuously refines the identification of relevant views and tables by comparing semantic intent with actual query patterns, improving precision while maintaining versatility.
Solution Approach 2:
The system introduces an intermediary matching mechanism that connects semantic search results with pre-generated SQL queries. This intermediary component analyzes the match between user intent and available queries to precisely identify which views and tables are most relevant, bridging the gap between general-purpose question handling and specific data source identification.
4Productivity
If semantic searching is used for LLMs to search text content, then the searching works well for text data, but the LLMs face significant challenges when users ask questions against structured data stored in tables and columns
Solution Approach 1:
The system segments the search process into two specialized paths: semantic search for text content and structured query matching for database tables. Each path is optimized for its data type, with semantic search handling natural language understanding and structured queries handling precise data retrieval, resolving the contradiction between text search efficiency and structured data accuracy.
Solution Approach 2:
The system introduces an intermediary component that bridges semantic text search and structured database querying. This intermediary matches semantic search results with pre-generated SQL queries, allowing the benefits of both approaches: the natural language understanding of semantic search and the precision of structured queries for table and column data.
Data Source
AI summary
A data query system and methods are provided that are configured to intelligently generate structured data queries from natural language questions using large language models (LLMs). The system includes a processor and a computer readable medium operably coupled thereto, the computer readable medium comprising a plurality of instructions stored in association therewith that are accessible to, and executable by, the processor, to perform operations which include receiving a natural language question for structured data, converting the natural language question to embeddings, matching the embeddings to pre-generated questions from a user questions repository (UQR), determining an accuracy of the matching meets or exceeds a threshold similarity, determining, using an LLM and metadata corresponding to the pre-generated questions from the UQR, a structured data query for querying for the structured data, and querying the structured database system using the structured data query.