Transparent RDBMS Offload via Polymorphic UDF Binding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current relational database management systems (RDBMS) face challenges in offloading relational tables to external engines, as existing solutions require extensive changes to the query optimizer and SQL, and do not support structured query language (SQL) for creating or referencing relational tables in offload engines, leading to poor performance and architectural disruptions.
Innovation Solution
The approach involves delegating data access from an RDBMS to an external offload engine by reformulating tasks using standard SQL operators, allowing user-defined functions (UDFs) and types to provide metadata for offloaded tables, enabling transparent acceleration and workload distribution without invasive changes to the RDBMS, and utilizing polymorphic UDFs for reuse and dynamic datatype discovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If relational tables are offloaded to external engines, then performance and scalability are improved, but extensive changes to query optimizer and SQL are required
Solution Approach 1:
The patent introduces an intermediary layer that translates standard SQL operations into operations suitable for external offload engines. This intermediary mechanism allows relational tables to be offloaded without requiring extensive changes to the core query optimizer or SQL language, thus improving performance while avoiding excessive system complexity
Solution Approach 2:
The patent segments the database system into core RDBMS functions and offloadable operations. By dividing the system architecture into manageable segments where specific table operations can be independently offloaded to external engines, the system achieves performance improvement without requiring complete architectural restructuring
2Adaptability or versatility
If SQL is extended to support relational tables in offload engines, then functionality is improved, but compatibility with existing RDBMS implementations is lost
Solution Approach 1:
The patent implements a universal interface layer that maintains compatibility with existing RDBMS SQL implementations while enabling extended functionality for offloaded relational tables. This universal approach allows the system to work with both traditional in-memory tables and offloaded tables using the same SQL interface, preserving compatibility while enhancing adaptability
3Ease of operation
If catalog materialization is required before query execution, then object resolution is simplified, but memory demand and startup time increase
Solution Approach 1:
The patent implements dynamic catalog materialization where metadata is materialized on-demand rather than statically before query execution. This dynamic approach allows the system to maintain simplified object resolution while reducing memory consumption by only materializing catalog entries when actually needed for query processing
4Ease of operation
If data is materialized in RDBMS before offloading, then data access is simplified, but network traffic and memory demand increase
Solution Approach 1:
The patent performs preliminary filtering and preparation of data access operations within the RDBMS before offloading to external engines. This preliminary action simplifies the data access process by pre-processing queries and only transferring necessary data over the network, thereby reducing network traffic while maintaining ease of operation
Data Source
AI summary
Approaches herein transparently delegate data access from a relational database management system (RDBMS) onto an offload engine (OE). The RDBMS receives a database statement referencing a user defined function (UDF). In an execution plan, the RDBMS replaces the UDF reference with an invocation of a relational operator in the OE. Execution invokes the relational operator in the OE to obtain a result based on data in the OE. Thus, the UDF is bound to the OE, and almost all of the RDBMS avoids specially handling the UDF. The UDF may be a table function that offloads a relational table for processing. User defined objects such as functions and types provide metadata about the table. Multiple tables can be offloaded and processed together, such that some or all offloaded tables are not materialized in the RDBMS. Offloaded tables may participate in standard relational algebra such as in a database statement.