Query Engine Fetching External Data via SQL Interface
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database management systems (DBMS) are limited in their ability to fetch data from multiple unrelated external sources in near real-time, particularly when the data is in structured or unstructured forms, and require pre-existing data storage mechanisms, which restricts their flexibility and accessibility.
Innovation Solution
A system and method that uses a SQL interface to execute external code, allowing it to fetch structured or unstructured data from external databases or embedded applications in near real-time, independent of the DBMS, using a query engine and protocol server that parses and formats data without storage, enabling data access from various sources through standard protocols like HTTP, HTTPS, or gRPC.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in a database before querying, then data availability is improved, but system flexibility and adaptability to new data sources deteriorate
Solution Approach 1:
The patent introduces a query engine as an intermediary layer between SQL queries and external data sources. This mediator executes external code, invokes remote procedures, and processes data streams without requiring data to be pre-stored in a database, thereby maintaining data availability while enabling flexible access to diverse external sources
Solution Approach 2:
The query engine is designed to universally handle multiple data source types (databases, external codes, streaming sources, embedded applications) through a unified SQL interface. This multi-functional approach allows the system to adapt to various data sources without requiring source-specific customization, resolving the contradiction between reliable data access and flexibility
2Productivity
If proprietary DBMS mechanisms are used for data fetching, then data retrieval capability is improved, but system complexity and dependency on specific DBMS deteriorate
Solution Approach 1:
The patent extracts the data retrieval functionality from proprietary DBMS mechanisms and places it in a universal query engine. This separation allows the query engine to access data from multiple sources using standard protocols (HTTP, gRPC, etc.) without relying on DBMS-specific languages like PL/SQL or OCI, thereby reducing dependency while maintaining retrieval capability
Solution Approach 2:
The patent replaces proprietary mechanical DBMS mechanisms with a standardized query execution model. Instead of using DBMS-specific procedural languages, the system uses a universal query interface that can invoke external code and remote procedures through standard communication protocols, simplifying the system architecture and reducing vendor lock-in
3Ease of operation
If data is retrieved and stored in a streaming source, then data accessibility is improved, but real-time processing capability deteriorates
Solution Approach 1:
The query engine maintains continuous connection to streaming data sources and executes queries against live data streams without requiring data to be pre-stored. This continuous action enables near real-time data access while maintaining ease of operation through the unified SQL interface, eliminating the trade-off between accessibility and real-time processing
Data Source
AI summary
A system and method for fetching data using an SQL query from the execution of an external code that may be in any language and executed in any environment. The system includes a query engine that can receive a query from a SQL interface and parses the query to the desired format. The system also includes a protocol server that can trigger a code executor to execute an external code based on the query and transmits the data returned by the external code to the query engine. The query engine can process the data to an SQL understandable format and present the processed data on the SQL interface.