Natural Language Query Planning for Heterogeneous Data Sources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for querying data from heterogeneous data sources require strong technical expertise due to complex data sources, inconsistent naming schemes, and similar table/column names, making it difficult for users to derive information effectively.
Innovation Solution
An analysis system uses a machine learning-based language model to generate database queries from natural language questions, determining actual data assets and fields, and executes these queries across multiple heterogeneous data sources while maintaining metadata privacy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional querying techniques are used to access heterogeneous data sources, then data retrieval capability is maintained, but user accessibility deteriorates due to requiring strong technical expertise
Solution Approach 1:
The patent introduces a natural language processing interface as an intermediary between users and heterogeneous data sources. This mediator translates user-friendly natural language queries into the complex queries required by different data source systems, eliminating the need for users to directly interact with complex query languages and system-specific protocols.
Solution Approach 2:
The patent creates a universal natural language interface that can handle multiple types of data sources (relational databases, file systems, cloud storage) through a single unified system. This universal interface supports various query types and data formats, allowing one system to serve multiple functions across diverse data sources without requiring separate interfaces for each system.
2Ease of operation
If expert knowledge is required to query data sources, then query accuracy is maintained, but ease of operation deteriorates
Solution Approach 1:
The system employs automated machine learning models and natural language processing algorithms that perform query generation, optimization, and execution autonomously. The system self-adjusts to different data sources and query patterns, learning from interactions to improve accuracy without requiring expert user intervention while maintaining reliable query results.
3Ease of operation
If natural language processing is used to generate queries, then ease of operation improves, but query accuracy deteriorates due to generated data asset names not corresponding to actual assets
Solution Approach 1:
The system implements feedback mechanisms where the generated queries are validated against actual data source schemas and metadata. When discrepancies are detected between generated data asset names and actual assets, the system receives feedback and automatically corrects the mappings, ensuring that natural language queries accurately correspond to real data assets while maintaining ease of operation.
Data Source
AI summary
An analysis system connects to a set of data sources and perform natural language questions based on the data sources. The analysis system connects with the data sources and retrieves metadata describing data assets stored in each data source. The analysis system generates an execution plan for the natural language question. The analysis system finds data assets that match the received question based on the metadata. The analysis system ranks the data assets and presents the ranked data assets to users for allowing users to modify the execution plan. The analysis system may use execution plans of previously stored questions for executing new questions. The analysis system supports selective preprocessing of data to increase the data quality.