Ranking Filter Operation for Database Calculation Scenarios
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex calculations in database systems require executing numerous queries, consuming significant resources and leading to inefficient data processing and transfer between database servers and application servers.
Innovation Solution
A cloud-based analytics engine performs calculation scenarios with a ranking filter operation, aggregating and filtering rows in database partitions to generate results efficiently, and pushes down filter attributes to minimize intermediate results, allowing for parallel execution and optimized data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If complex calculations are performed by executing numerous queries in traditional database systems, then the required computations can be completed, but resource consumption increases significantly and data processing efficiency decreases
Solution Approach 1:
The patent merges multiple separate queries into a single calculation scenario that can be executed in parallel. The calculation engine consolidates numerous individual query operations into one unified execution unit, processing multiple database partitions simultaneously to reduce both resource consumption and improve processing efficiency.
Solution Approach 2:
The patent segments the calculation scenario into multiple independent calculation nodes that can be executed in parallel across different database partitions. Each node processes a specific portion of the data independently, allowing concurrent execution and reducing overall processing time while optimizing resource utilization.
2Productivity
If numerous queries are executed to perform complex calculations, then the required data processing can be completed, but the time required for execution increases
Solution Approach 1:
The calculation scenario is divided into multiple calculation nodes that can be executed in parallel across different database partitions. This segmentation enables concurrent processing of data subsets, significantly reducing total execution time compared to sequential query processing.
Solution Approach 2:
The system performs preliminary actions by pre-defining the calculation scenario structure and optimization strategies before query execution. The calculation engine prepares the execution plan in advance, identifying parallel processing opportunities and optimizing data access patterns, which reduces actual query execution time.
3Reliability
If filter operations are applied at higher levels of the calculation hierarchy, then more comprehensive filtering can be performed, but the quantity of intermediate results increases
Solution Approach 1:
Filtering operations are segmented and applied at multiple levels of the calculation hierarchy. Local filters are applied at individual calculation nodes to reduce intermediate result volumes, while global filters ensure comprehensive filtering accuracy. This multi-level segmentation allows filtering to be performed efficiently without generating excessive intermediate data.
Solution Approach 2:
Different filtering strategies are applied locally at different calculation nodes based on their specific requirements. Each node applies filters appropriate to its local data context, optimizing the balance between filtering accuracy and intermediate result volume reduction for that particular segment of the calculation.
Data Source
AI summary
A calculation engine of a database management system is described. The calculation engine may receive a query associated with a calculation scenario that defines a data flow model. The data flow model may include one or more calculation nodes, each of which corresponding to an operation performed on one or more database tables stored at a database. The one or more calculation nodes may include at least one calculation node corresponding to a ranking filter operation. The calculation engine may execute the query including by executing the calculation scenario. The executing of the calculation scenario may include performing the ranking filter operation to generate a result corresponding to at least a portion of rows included in a first partition of a database table stored at the database. Related systems, methods, and articles of manufacture are provided.