Asynchronous Block Volume Replication Using Snapshot Delta Checkpoints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cloud-based data replication methods, such as periodic backups, result in significant metadata overhead, increased computational demands, and prolonged recovery times due to synchronous data transfer, leading to high RPO and RTO, which can be exacerbated by system failures and elevated latency.
Innovation Solution
Implementing asynchronous cross-region block volume replication by generating snapshots and deltas, allowing incremental data transfer and reducing the impact on input-output operations, thereby minimizing data loss and recovery time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous data transfer is used for data replication, then data consistency is improved, but recovery time objective (RTO) worsens due to prolonged transfer times
Solution Approach 1:
The patent segments the block volume into multiple partitions and creates snapshots at different logical times for each partition. This allows incremental data transfer where only changed partitions (deltas) need to be transmitted, significantly reducing recovery time while maintaining data consistency through the checkpoint mechanism that tracks which partitions have been replicated.
Solution Approach 2:
The system performs preliminary actions by creating snapshots at different logical times before a failure occurs. These snapshots are prepared in advance and stored, so when a failure happens, the system can immediately begin transferring the most recent snapshot and its deltas without waiting for a synchronous transfer to complete, thus reducing RTO while maintaining consistency.
2Reliability
If periodic backups are performed, then data loss risk is reduced, but metadata overhead and computational demands increase significantly
Solution Approach 1:
The patent extracts only the changed portions (deltas) of data between snapshots rather than transferring entire backup sets periodically. By identifying and transferring only the modified partitions, the system reduces metadata overhead and computational demands while still providing comprehensive data protection and minimizing data loss risk.
Solution Approach 2:
The system changes the parameter of data transfer from periodic full backups to incremental delta transfers based on logical time snapshots. This parameter change reduces the volume of data transferred and the associated metadata overhead, while the checkpoint mechanism ensures complete data protection by tracking which partitions have been successfully replicated.
3Reliability
If cross-region replication is implemented, then disaster recovery capability is improved, but latency during failures is exacerbated
Solution Approach 1:
By segmenting the block volume into partitions with individual snapshots and deltas, the system enables parallel transfer of multiple partition changes. This segmentation allows cross-region replication to proceed more efficiently with reduced latency, as transfers can be optimized and prioritized without waiting for entire volume synchronization.
Solution Approach 2:
The system performs preliminary actions by pre-positioning snapshot data and deltas in the destination region before failures occur. When a failure happens, the replication process can immediately resume from the last checkpoint without initiating a full data transfer, significantly reducing latency during disaster recovery operations while maintaining robust cross-region capability.
4Manufacturing precision
If synchronous transfer methods are used, then data accuracy is improved, but productivity during normal operations deteriorates due to I/O suspension
Solution Approach 1:
The patent implements periodic snapshot creation at different logical times rather than continuous synchronous transfer. This periodic action allows normal I/O operations to continue uninterrupted between snapshots, while data accuracy is maintained by capturing the state of partitions at specific logical time points. The asynchronous delta transfer further ensures that snapshot creation does not block I/O operations.
Solution Approach 2:
The system performs preliminary snapshot capture at logical time points without suspending I/O operations. By using copy-on-write or similar techniques to create snapshots in the background, the system maintains data accuracy for replication purposes while allowing normal productivity to continue. The deltas are then transferred asynchronously, further decoupling the accuracy requirement from I/O performance.
Data Source
AI summary
Techniques are provided for cross-region replication of block volume data. The techniques include a method wherein a computer system implements operations including creating a first snapshot of a block volume at a first geographic region and at a first logical time, the block volume including a plurality of partitions, transmitting first snapshot data to an object storage system at a second geographic region, creating a second snapshot of the block volume at the first geographic region and at a second logical time, generating a plurality of deltas, transmitting a plurality of delta data sets corresponding to the plurality of deltas to the object storage system at the second geographic region, generating a checkpoint at least in part by aggregating object metadata associated with the plurality of deltas and the first snapshot, receiving a restore request to generate a restore volume, and generating the restore volume from the checkpoint.