Snapshot-Based Volume Replication with Dirty List Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data replication systems face inefficiencies in transferring large volumes of data, as initial snapshot transfers can take significant time, and maintaining short-lived snapshots is challenging, leading to larger differences between successive snapshots.
Innovation Solution
The method involves generating snapshots and dirty lists to synchronize replica volumes with production volumes, transferring only portions of differences, and refreshing snapshots to minimize data transfer time, using electronic hardware circuitry and computer-executable instructions to manage the process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire production volume is transferred to the replica site in one snapshot, then the initial data synchronization is complete, but the data transfer time becomes significant and unacceptably long
Solution Approach 1:
The patent divides the production volume into multiple sections and transfers data in incremental snapshots rather than as a single large transfer. Each snapshot captures only the portions of the volume that have changed since the previous snapshot, breaking down the large transfer task into manageable segments that can be transmitted more quickly and efficiently
Solution Approach 2:
The system transfers only the necessary portions of data (portions that have changed) rather than transferring the entire volume repeatedly. By identifying and transferring only the dirty portions of the volume, the system avoids unnecessary data transmission while ensuring complete synchronization
2Duration of action of stationary object
If snapshots are maintained for a long duration to ensure data availability, then data recovery options are preserved, but the differences between successive snapshots become larger increasing transfer time
Solution Approach 1:
The system performs preliminary identification of changed portions (dirty lists) before initiating data transfer. By pre-processing the volume to identify what has changed and organizing this information in advance, the system can quickly transfer only the necessary portions without waiting for full snapshot comparisons, reducing overall transfer time while maintaining snapshot retention
Solution Approach 2:
The system uses dirty lists that track changes between snapshots as feedback mechanisms. This feedback allows the system to intelligently determine what portions of the volume need to be transferred in each incremental snapshot, optimizing the transfer process by focusing only on changed areas rather than transferring entire volumes repeatedly
Data Source
AI summary
In one aspect, a method to synchronize a replica volume with a production volume includes providing a first snapshot of the production volume and a first dirty list that includes differences between the first snapshot and the replica volume; sending only a portion of the differences between the first snapshot and the replica volume to the replica site associated with a section of the production volume, generating a second snapshot of the production volume, adding differences between the first snapshot and the second snapshot to a second dirty list, erasing the first snapshot of the production volume and renaming the second snapshot to the first snapshot.