External Table Operators for Secure Database Query Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database systems face challenges in processing database queries that involve external data, requiring preprocessing and additional storage, and often need access permissions from external systems, which can be a security concern for sensitive information.
Innovation Solution
A database system processes queries by generating an execution plan with external table operators, establishing a data stream connection with external systems, and interacting using credentials like host addresses and passwords, allowing for secure and efficient processing of external data without requiring upfront data import or storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is extracted from external system and stored as tables within database system, then data processing capability is improved, but additional storage is required and preprocessing time is increased
Solution Approach 1:
The patent extracts only the necessary data from external systems on-demand through external tables, rather than importing entire datasets. The external table mechanism allows the database system to pull specific data portions when needed, eliminating the need to store complete external datasets locally while maintaining processing capability.
Solution Approach 2:
The patent performs preliminary registration of external tables with the database system, establishing metadata and connection information in advance. This preliminary setup enables efficient on-demand data extraction without requiring actual data import beforehand, reducing both storage requirements and preprocessing time while maintaining processing readiness.
2Adaptability or versatility
If database system creates connection with external system, then data access capability is improved, but security risk increases due to access permissions
Solution Approach 1:
The patent introduces external tables as an intermediary layer between the database system and external systems. This intermediary mechanism allows data access without requiring direct connection permissions to the external system. The external table acts as a virtual representation that mediates data retrieval, reducing security risks while maintaining access capability.
Solution Approach 2:
The patent creates virtual copies of external data structures within the database system through external tables. Instead of establishing direct connections to external systems, the database system works with local metadata representations of external tables, enabling data access through copied structural information rather than direct system access permissions.
3Productivity
If external table operators are used to process external data, then processing efficiency is improved, but system complexity increases
Solution Approach 1:
The patent implements external table operators that provide universal functionality for handling various external data sources through a unified interface. These operators can scan, insert, modify, and delete data from different external systems using the same mechanism, reducing system complexity by eliminating the need for separate handling logic for each external data source while maintaining high processing efficiency.
Data Source
AI summary
A database system processes database queries that process external tables representing data obtained from an external systems. The database system generates an execution plan to execute the database query. The database system sends a request for data of the external table to an external system. The request provides connection information to the external system. The external system sends a request to connect to the database system and provides data of the external table as a data stream. If the database system reaches an operator that processes the data of the external table, the database system waits for data from the external system. The database system processes the database query using the data of the data stream. The database system provides the result of the database query to a requestor, for example, a client device.