Query Execution Engine Decouples Application and Database Layers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database management systems face challenges in optimizing performance when handling complex queries and high transaction volumes, as pushing complex operations to the application layer increases processing demands, while pushing them to the database layer can reduce response times.
Innovation Solution
A query execution engine decouples the application layer from the database layer, atomizing query operations into primitive calls such as dictionary lookup, cardinality estimation, search, and materialization, which are executed by a data management engine, allowing for parallelization and optimized execution across different database types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If complex query operations are pushed to the application layer, then processing flexibility and adaptability are improved, but processing time and system response time increase
Solution Approach 1:
The patent segments complex query operations into atomic primitive calls (e.g., dictionary lookup, cardinality estimation, search, materialization). Each primitive call represents a discrete, executable unit that can be independently processed. This segmentation enables the system to maintain flexibility in query formulation while achieving efficient execution through standardized atomic operations at the database layer.
Solution Approach 2:
The patent introduces a query execution engine as an intermediary layer between the application layer and the database layer. This execution engine translates high-level query operations into primitive calls and manages their execution, thereby decoupling the application logic from database-specific implementation details. The intermediary enables flexible query processing while maintaining fast response times through optimized execution at the database layer.
2Reliability
If complex operations are handled by specialized execution engines for different database types, then compatibility and reliability are improved, but device complexity and system architecture complexity increase
Solution Approach 1:
The patent implements a universal query execution engine that can handle multiple database types (column-oriented, row-oriented, distributed, in-memory) through a common interface. The execution engine translates various database-specific operations into standardized primitive calls, eliminating the need for separate specialized execution engines for each database type. This universal approach maintains reliability across different database types while significantly reducing system complexity.
Solution Approach 2:
The patent changes the operational parameters of the execution engine by defining a standardized set of primitive calls with specific parameters (e.g., lookup_key, estimate_table_size, search_columns). By standardizing these parameters across different database types, the system achieves compatibility without requiring multiple specialized engines, thereby reducing complexity while maintaining reliability.
3Productivity
If more primitive calls are executed in parallel, then query processing speed and productivity are improved, but coordination overhead and system resource consumption increase
Solution Approach 1:
The patent implements dynamic parallel execution where the query execution engine determines the execution order and parallelization strategy based on the specific query characteristics and available resources. Independent primitive calls are executed in parallel when possible, while dependent calls are executed sequentially. This dynamic approach maximizes processing speed without unnecessarily consuming system resources, as the system adapts its resource usage to the actual query requirements.
Data Source
AI summary
A system for generating a query plan is provided. In some example embodiments, the system performs operations comprising: determining, at a query execution engine, a first primitive call for implementing a query operation on data at a database; performing the query operation by at least sending, to a data management engine coupled to the database, the first primitive call for execution by the data management engine; and determining, based at least on a result of the first primitive call, a result of the query operation. Related methods and articles of manufacture, including computer program products, are also described.