Decoupled Database Sharding with Copy-on-Write Volume Cloning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database scaling methods face inefficiencies and downtime due to slow rebalancing processes, particularly when incorporating new shards, as they involve costly data transmission and additional load on existing nodes, leading to capacity shortages and potential downtime.
Innovation Solution
Implement a decoupled storage and compute architecture with a copy-on-write snapshot mechanism, utilizing a lightweight clone of database volumes and a separate storage tier to manage data durability and consistency, allowing for rapid scaling without immediate data copying and minimizing impact on the foreground workload.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional sharding with rebalancing is used to scale databases, then data can be distributed across multiple nodes, but the scaling process becomes slow due to mismatches between storage capacity and data transmission rates
Solution Approach 1:
The patent divides the database system into separate compute nodes and storage nodes. Compute nodes handle query processing while storage nodes manage data persistence. This segmentation allows independent scaling of storage capacity without requiring proportional data transmission across the entire system, enabling faster scaling by isolating the heavy lifting to dedicated storage infrastructure.
Solution Approach 2:
The patent introduces a buffer layer or message queue system between compute nodes and storage nodes. This intermediary decouples the data transmission process, allowing compute nodes to continue processing queries while data is asynchronously transferred to new storage nodes. The buffer absorbs the mismatch between storage capacity and transmission rates, enabling faster scaling without blocking query operations.
2Adaptability or versatility
If rebalancing is performed to add new shards, then data distribution can be optimized, but the source database nodes experience increased load during the copying process
Solution Approach 1:
By separating compute and storage functions across different node types, the patent ensures that when new storage nodes are added, only the storage layer experiences the scaling load. Compute nodes continue serving queries from existing storage nodes without being burdened by data copying operations, maintaining system availability and preventing source nodes from becoming overloaded.
Solution Approach 2:
The system maintains continuous query processing capability during scaling operations by allowing compute nodes to continue accessing data from existing storage nodes while new storage nodes are being populated. The decoupled architecture enables data migration to proceed in the background without interrupting the useful action of query serving, ensuring high availability throughout the scaling process.
3Adaptability or versatility
If data is copied from existing shards to new shards, then new nodes can be incorporated into the cluster, but the process is slow due to storage capacity and transmission rate mismatches
Solution Approach 1:
The patent segments the data transmission task by having dedicated storage nodes handle data persistence while compute nodes handle query processing. When incorporating new nodes, the system can leverage the combined storage capacity of all storage nodes for parallel data transfer, rather than being limited by the transmission capacity of individual compute nodes. This segmentation improves overall data transmission efficiency during node incorporation.
Solution Approach 2:
Storage nodes are designed to serve multiple functions: they persist data for multiple compute nodes, handle data migration during scaling, and provide backup capabilities. This multi-functionality allows the storage layer to efficiently handle data incorporation into new shards by utilizing the aggregate storage and network capacity of the entire storage subsystem, improving productivity during scaling operations.
Data Source
AI summary
Techniques for fast online scaling of a database via a split architecture including decoupled storage and compute tiers in a database service are described. A cluster of database (DB) nodes is scaled to add a new DB node. The scaling includes determining a split for data of a first volume managed by an existing DB node. A second DB node is obtained, and the first volume is cloned according to a lightweight copy technique to yield a second volume for use by the second DB node. After the cloning, a set of database modifications are applied to the second volume based on modifications caused by database traffic received by the first DB node, involving the volume, during the cloning of the first volume. Each DB node may drop the portion of the volume that it does not need according to the split.