Compact Query Plan for Mega Query Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Operational Business Intelligence (BI) reporting faces challenges in optimizing mega queries due to their complexity, particularly in reducing the cost of query execution in terms of time and space, as existing methods such as dynamic programming and randomized algorithms do not scale well for large queries and fail to minimize data processing.
Innovation Solution
The creation of a compact query plan (CQP) that transforms the original query plan into a simplified tree structure, reducing the height of the query tree and grouping operations into macro-nodes, along with optimization algorithms that compute maximal sub-queries and semi-join reductions to minimize data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If dynamic programming or randomized algorithms are used to optimize mega queries, then query optimization can be performed, but the methods do not scale well for large queries and fail to minimize data processing
Solution Approach 1:
The patent segments the query plan into a compact representation with abstract nodes that group multiple operations together. Instead of processing each operation individually in traditional dynamic programming approaches, the invention creates a hierarchical structure where abstract nodes represent sets of operations with equivalent results, enabling efficient optimization of mega queries without exponential complexity increase
Solution Approach 2:
The invention changes the representation parameters of the query plan from a detailed operation-by-operation structure to a compact form using abstract nodes. This parameter transformation allows the query plan to be represented with significantly fewer elements while preserving semantic equivalence, making optimization feasible for mega queries
2Productivity
If traditional query plans are used, then query execution can be performed, but the complexity of query optimization increases significantly for mega queries
Solution Approach 1:
The patent merges multiple operations into abstract nodes that represent equivalent computational results. By combining operations that can be reordered without changing results into single abstract nodes, the invention reduces the number of optimization considerations from thousands of individual operations to a manageable set of abstract nodes
Solution Approach 2:
The abstract nodes serve multiple functions: they represent operational results, enable reordering optimization, and reduce complexity simultaneously. This multi-functionality allows a single representation structure to address multiple optimization challenges that would traditionally require separate approaches
3Loss of information
If mega queries are executed to retrieve comprehensive data, then complete information can be obtained, but time and space costs increase significantly
Solution Approach 1:
The invention performs preliminary optimization by creating the compact query plan representation before execution. By pre-computing the abstract node structure and identifying reorderable operations, the system prepares an optimized execution path that minimizes data processing time and space requirements while maintaining result completeness
Data Source
AI summary
Apparatus, systems, and methods may operate to receive an original query plan, to transform the original query plan into an equivalent executable compact query plan, and to store the compact query plan on a machine readable device. Further activities may include computing maximal source sub-queries associated with the compact query plan, and computing semi-join reductions of the maximal source sub-queries to provide an executable derivative query plan, which may also be stored on a machine readable device. Additional apparatus, systems, and methods are disclosed.