Database Partition Pruning via Dependency Graph
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database pruning techniques are limited as they require table identification at compile time, leading to inefficient query performance when table access is unknown until runtime, resulting in unnecessary data scanning and increased processing time.
Innovation Solution
The creation of a dependency graph during compile time, which represents potential partition candidates and their dependencies, allows for dynamic pruning of partitions at runtime using forward and backward propagation rules, enabling efficient data retrieval even when table identifiers are unknown until execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional partition pruning techniques are used at compile time, then query optimization is achieved for known tables, but query performance deteriorates when table access is unknown until runtime
Solution Approach 1:
The patent pre-generates dependency graphs at compile time that capture all possible partition access paths and dependencies. These graphs are prepared in advance but not executed until runtime, when they enable rapid pruning decisions without the overhead of dynamic analysis, thus resolving the contradiction between compile-time optimization and runtime flexibility
Solution Approach 2:
The dependency graph serves as an intermediary data structure that bridges compile-time query analysis and runtime execution. It pre-computes and stores partition dependencies in a format that enables efficient runtime pruning, acting as a mediator between static optimization and dynamic table access patterns
2Reliability
If all partitions are scanned when table identifier is unknown, then complete data retrieval is ensured, but data retrieval time increases significantly
Solution Approach 1:
The patent applies partial action by pruning only the partitions that are provably unnecessary based on the dependency graph and available runtime information, rather than scanning all partitions. This partial pruning maintains reliability for queries where pruning is safe while dramatically reducing retrieval time
3Productivity
If partition pruning is performed at compile time with known tables, then processing efficiency is improved, but adaptability to runtime parameters is lost
Solution Approach 1:
The patent transforms the static compile-time pruning approach into a dynamic system by using pre-generated dependency graphs that can be evaluated at runtime with actual parameter values. This allows the pruning decision to adapt to runtime parameters while maintaining the efficiency of pre-computed dependency information
Data Source
AI summary
Provided is a system and method for pruning partitions from a database access operation based on a dependency graph. In one example, the method may include generating a dependency graph for a partition-wise operation, the dependency graph comprising nodes representing partition candidates and links between the nodes identifying dependencies of the partition candidates, receiving, at runtime, a database query comprising a partition identifier, identifying a partition candidate that can be excluded from processing the database query based on the partition identifier, pruning a second partition candidate based on a dependency in the dependency graph between the excluded partition candidate and the second partition candidate, and performing a database access for the database query based on the pruning.