Dynamic Top Operator for Database Query Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database management systems face challenges in optimizing performance when handling complex and voluminous queries, leading to resource consumption and inefficiencies in data processing and transfer between database servers and application servers.
Innovation Solution
The implementation of a calculation engine that executes query scenarios with dynamic top operators, allowing for dynamic partitioning and ordering of attributes, and the use of calculation scenarios that model complex queries as a directed acyclic graph, optimizing data processing and reducing the need for large data transfers by executing operations within the database server.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If complex multidimensional analysis queries are executed on the underlying database, then analytical capabilities are improved, but query execution time and resource consumption increase
Solution Approach 1:
The patent segments complex analytical queries into multiple calculation nodes organized in a directed acyclic graph (DAG). Each node represents a specific calculation operation (e.g., aggregation, filtering, sorting) that can be executed independently and in parallel. This segmentation allows the database system to process complex queries by breaking them down into manageable, optimizable units, thereby reducing overall execution time while maintaining full analytical capabilities.
Solution Approach 2:
The patent implements preliminary action by pre-compiling calculation scenarios into executable calculation plans before query execution. The calculation engine analyzes the DAG structure, determines optimal execution strategies, and prepares execution plans in advance. This pre-processing allows the system to quickly execute complex analytical queries without performing full optimization at runtime, significantly reducing query execution time while preserving analytical versatility.
2Adaptability or versatility
If the database handles high volume transactions and complex queries, then system functionality is improved, but performance optimization becomes more difficult
Solution Approach 1:
The patent creates a universal calculation engine that can handle both high-volume transactions and complex analytical queries through a unified DAG-based architecture. The same calculation engine and execution framework process different query types by interpreting different calculation scenarios, eliminating the need for separate optimization mechanisms for different query workloads. This universal approach simplifies performance optimization while maintaining diverse system functionality.
Solution Approach 2:
The patent utilizes parameter changes by allowing dynamic configuration of calculation node properties and execution parameters within the DAG framework. Different query types can adjust parameters such as partitioning strategies, sorting methods, and aggregation approaches without changing the fundamental architecture. This flexibility enables performance optimization for various workload types while maintaining a single, manageable system complexity level.
3Ease of operation
If data is transferred between database server and application server for processing, then processing flexibility is improved, but data transfer overhead and resource consumption increase
Solution Approach 1:
The patent introduces a calculation engine as an intermediary component that resides within the database server architecture. This calculation engine executes calculation scenarios directly on the database server, acting as a mediator between the database storage layer and the application layer. By performing complex calculations within the database server rather than transferring all data to the application server, the system reduces data transfer overhead and resource consumption while maintaining processing flexibility through the programmable DAG framework.
Data Source
AI summary
A method, a system, and a computer program product for executing a query. A query associated with a calculation scenario defining a data flow model that includes one or more calculation nodes is received. Each calculation node corresponds to an operation being performed on one or more database tables stored at a database. The calculation nodes include one or more nodes specifying a window function operation. The window function operation including one or more first attributes and one or more second attributes. A calculation engine executes the calculation scenario by performing, using at least one of the first and second attributes, the window function operation on the database tables stored at the database. Based on the execution of the calculation scenario, a result data set is generated and provided by the database server to the application server.