Distributed Database Rebalancing Without Downtime
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed shared-nothing databases face challenges in adding storage and processing capacity without downtime while maintaining serializable isolation during data rebalancing.
Innovation Solution
A method for transparently rebalancing data by calculating a new consistent hash ring, generating an index for data transfer, executing serialized transactions, and rerouting in-flight transactions to maintain serializable isolation and update the hash ring, allowing for the addition of new logical partitions without stopping database operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data rebalancing is performed to add new partitions to the database system, then database capacity is increased, but database downtime occurs during the rebalancing process
Solution Approach 1:
The patent applies preliminary action by pre-calculating the desired consistent hash ring configuration before actual data migration begins. The system determines the target hash ring with new partitions included, pre-identifies which data ranges need to be migrated, and prepares the migration plan in advance. This allows the rebalancing process to proceed efficiently once initiated, minimizing the actual downtime period while achieving capacity expansion.
2Adaptability or versatility
If data is moved between partitions during rebalancing, then new partitions are activated, but serializable isolation is compromised
Solution Approach 1:
The patent applies segmentation by dividing the data migration process into discrete, manageable ranges. Each data range is identified and migrated as a separate unit between partitions. The system segments the hash ring into specific ranges that need rebalancing and processes them individually while maintaining the overall serializable isolation guarantee. This allows fine-grained control over the migration process without compromising transaction isolation.
Solution Approach 2:
The patent uses an intermediary mechanism in the form of a coordinator that manages the rebalancing process. The coordinator oversees data movement between partitions, ensures proper synchronization, and maintains the hash ring state. This intermediary layer coordinates the complex interactions during rebalancing, ensuring that serializable isolation is preserved even as data is being moved between partitions.
3Measurement precision
If the consistent hash ring is updated frequently during rebalancing, then data location accuracy is improved, but system complexity increases
Solution Approach 1:
The patent applies periodic action by updating the consistent hash ring at specific, predetermined intervals or milestones during the rebalancing process rather than continuously. The system performs hash ring updates periodically after completing certain data migration milestones, which maintains data location accuracy while avoiding the complexity of continuous updates. This periodic approach balances precision requirements with system simplicity.
Data Source
AI summary
A distributed shared-nothing database provides serializable isolation for transactions and includes a mechanism for adding storage and processing capacity to the database without stopping the database from processing transactions.