Graph-Based Database Partitioning for Reducing Data Redundancy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database partitioning methods, particularly tree-based schemes, face issues such as data redundancy, performance degradation due to data dependency, and inability to represent cyclic relationships, leading to inefficient query processing and increased redundancy.
Innovation Solution
A graph-based partitioning method that uses cost-based analysis and hash-partitioning to eliminate data dependency between tables, representing cyclic relationships and reducing redundancy through a multi-graph model and bitmap vectors for tuple-level information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If tree-based database partitioning method is used to improve query processing by exploiting referential constraints, then join operations can be processed without network communication, but data redundancy increases significantly due to tuple-level duplication across partitions
Solution Approach 1:
The patent changes the partitioning parameter from parent-child relationship-based (tree structure) to hash-based partitioning using partitioning columns. This allows tuples to be distributed across partitions based on hash values of specific columns rather than following parent-child relationships, thereby eliminating tuple-level duplication while maintaining query processing efficiency through partition pruning capabilities
Solution Approach 2:
Instead of copying tuples across multiple partitions based on parent-child relationships (which causes redundancy), the patent uses a single copy of each tuple stored in one partition determined by hash function, eliminating the need for multiple copies while still enabling efficient query processing through selective partition access
2Reliability
If tree-based partitioning method references all partitions of parent table to partition child table, then referential constraints are satisfied, but data loading performance degrades significantly due to data dependency
Solution Approach 1:
The patent extracts the data dependency relationship between parent and child tables by using hash-based partitioning on partitioning columns. This allows child table partitions to be created independently without referencing parent table partitions, eliminating the sequential dependency that degrades data loading performance while still maintaining referential constraint satisfaction through the partitioning scheme
Solution Approach 2:
The patent segments the database into independent partitions based on hash values of partitioning columns rather than organizing them in a hierarchical tree structure. This segmentation allows each partition to be processed independently during data loading, eliminating the need to reference other partitions and thereby improving data loading performance while maintaining data integrity
3Productivity
If tree-based partitioning scheme is used to reduce network communication for join operations, then distributed query processing efficiency improves, but the scheme cannot represent cyclic relationships between tables
Solution Approach 1:
The patent creates a universal partitioning scheme based on hash functions that can handle both acyclic and cyclic relationships between tables. Unlike tree-based schemes that require a hierarchical structure, the hash-based partitioning can represent any relationship pattern including cycles, making the system more adaptable while maintaining distributed query processing efficiency through partition pruning
Data Source
AI summary
A relational database storage method according to one embodiment generates a join graph including tables and edges based on either a database schema including referential constraints among database tables or a query workload containing a set of join predicates, classifies the type of tables based on the cost of a join operation between the tables included in the join graph, classifies the type of edges in the join graph on the basis of the type of the tables connected to each other by the edge, adds a virtual edge between two tables satisfying the indirect join condition, and generates a database partitioning scheme for the join graph based on the set of tables in the join graph and their type, the set of edges in the join graph and their type, and virtual edges.