High-Dimensional Query Aggregation with Sub-Grid Solve Order
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing query evaluation systems face challenges in scalability and efficiency due to combinatorial expansion in hierarchical aggregation calculations, leading to issues like over-specification, over-precomputation, over-recomputation, over-computation, and repeat aggregation, especially in large and complex multi-dimensional databases.
Innovation Solution
The system employs a solve order with dynamic programming to partition input data into sub-grids, process these sub-grids in parallel, and evaluate queries efficiently by minimizing redundant calculations and scans, using a technique that determines dependencies and hierarchies to optimize query execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If hierarchical aggregation calculations are performed in multi-dimensional databases, then data analysis capability is improved, but computational complexity and processing time increase due to combinatorial expansion
Solution Approach 1:
The patent divides the hierarchical aggregation calculation into discrete levels (leaf level, intermediate levels, root level) and processes them in a structured sequence. Each level is handled independently with specific algorithms, preventing combinatorial explosion by breaking down the complex calculation into manageable segments rather than processing the entire hierarchy at once.
Solution Approach 2:
The system performs preliminary actions by pre-computing and storing aggregation results at intermediate levels of the hierarchy. This allows subsequent queries to leverage previously calculated data rather than re-computing from scratch, reducing computational complexity while maintaining comprehensive analysis capability.
2Adaptability or versatility
If query evaluation is performed on large dimensional spaces, then analytical depth is improved, but processing time and computational resources increase
Solution Approach 1:
The patent implements dynamic query evaluation that adapts to the specific dimensions and data distribution of each query. The system dynamically determines the optimal evaluation path through the hierarchy, skipping unnecessary levels or using pre-computed aggregates when appropriate, thereby maintaining deep analytical capability while reducing processing time for large dimensional spaces.
Solution Approach 2:
The system performs preliminary aggregation computations at intermediate hierarchy levels and stores these results for reuse. When queries require analysis of large dimensional spaces, the system can leverage these pre-computed intermediate results rather than performing complete re-aggregation, significantly reducing processing time while preserving analytical depth.
3Reliability
If redundant calculations and scans are performed, then data completeness is ensured, but computational efficiency decreases
Solution Approach 1:
The patent implements feedback mechanisms that track which data has been computed and stored at each hierarchy level. When evaluating queries, the system uses this feedback information to determine whether to perform complete recalculations or reuse existing results, ensuring data completeness where needed while avoiding redundant computations that would reduce computational efficiency.
Solution Approach 2:
The system discards redundant calculation attempts by using indexed references to pre-computed aggregate data. Instead of repeatedly calculating the same hierarchy aggregates, the system recovers and reuses previously computed results through efficient data structures, maintaining data completeness while dramatically improving computational efficiency.
Data Source
AI summary
A system, method, and device for evaluating a query is provided. The method includes (i) receiving the query to be evaluated against a data structure, (ii) determining input data corresponding to a subset of the data structure that is to be used in evaluating the query, (iii) determining a solve order for evaluating the query against the input data, (iv) partitioning the input data into a plurality of sub-grids based at least in part on the solve order, (v) processing the plurality of sub-grids to compute corresponding parts of the query, and (vi) evaluating the query based at least in part on the processing of the plurality of sub-grids to obtain a query result.