Relational Database Data Redistribution via Multi-Column Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing relational database systems face challenges in redistributing data evenly across multiple slices, leading to inefficient query performance and resource utilization due to uneven data distribution, particularly in scenarios like the 'locked_index' representation where most rows hash to the same slice, causing disproportionate load and performance issues.
Innovation Solution
The system dynamically monitors database statistics to detect redistribution conditions and adjusts the distribution function by including additional columns, such as the 'mid' column, to redistribute data more evenly across slices, thereby improving data balance and query performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is distributed across slices using a single-column hash function, then data retrieval speed is improved through parallel processing, but data distribution becomes uneven causing disproportionate load on certain slices
Solution Approach 1:
The patent transitions from single-column hash distribution to multi-column composite hash distribution, adding dimensional complexity to the distribution function. By incorporating multiple columns (e.g., uid, mid, locked) into the hash computation, the system achieves more uniform data spread across slices while maintaining parallel query performance
Solution Approach 2:
The system dynamically changes the distribution function parameters based on monitored data characteristics. When skew is detected in single-column distribution, the patent modifies the hash function to use composite keys with multiple columns, thereby adjusting the distribution parameters to achieve better balance without sacrificing retrieval speed
2Manufacturing precision
If data is redistributed dynamically based on monitored statistics, then data distribution evenness is improved, but system complexity increases due to monitoring and redistribution mechanisms
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors data distribution statistics and automatically triggers redistribution when skew thresholds are exceeded. This closed-loop control balances data distribution without requiring complex manual intervention, as the system self-regulates based on monitored conditions
Solution Approach 2:
The redistribution mechanism operates autonomously by monitoring its own data distribution state and initiating rebalancing operations when needed. The system serves itself by detecting skew conditions and executing redistribution without external control, reducing the need for complex external management infrastructure
3Ease of operation
If all rows are kept on a single node, then data access simplicity is maintained, but scalability and reliability are reduced
Solution Approach 1:
The patent divides the database into multiple slices distributed across different nodes, with each slice containing a portion of the data. This segmentation enables parallel access to different data portions while maintaining simple access patterns within each slice, and provides reliability through distribution across multiple failure domains
Solution Approach 2:
The distributed slice architecture serves multiple functions simultaneously: it enables parallel query processing for improved performance, provides fault tolerance through distribution, and maintains simple access patterns within each slice. The same slice structure supports both scalability and ease of operation
Data Source
AI summary
Systems and methods for redistributing data in a relational database are disclosed. In one embodiment, the database includes a plurality of rows of data distributed across a plurality of slices of a table in the database. The database system is configured to distribute the rows of data across the slices according to a first function based on one or more columns of the table. The database system monitors at least one database statistic indicative of variation in a distribution of the rows of data across the slices and detects a redistribution condition based on the at least one monitored database statistic. The database system is further configured to respond to the detected redistribution condition by redistributing the rows of data across the slices according to a second function based on a different number of columns than the first function.