Marker Delta Set for Point-in-Time State Capture in Asynchronous Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Delta set based asynchronous remote replication methods struggle to capture an exact point-in-time state of a local copy at a remote site, especially when external events occur independently of delta cycles, leading to potential data loss and incompatibility with applications relying on external event-based snapshots.
Innovation Solution
Implementing a marker delta cycle that ends the current delta cycle, inserts a marker delta set on the local site after an external event, and monitors for its receipt at the remote site to ensure the remote copy's consistency with the local copy's state at the time of the event, allowing asynchronous replication to continue without disrupting performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If delta set based asynchronous remote replication is used to aggregate writes over time, then bandwidth efficiency and application performance are improved, but the ability to capture exact point-in-time state at external events is lost
Solution Approach 1:
The system performs preliminary actions by inserting a marker delta set at the beginning of a delta cycle when an external event is detected. This marker serves as a pre-prepared reference point that enables later identification of the exact point-in-time state corresponding to the external event, resolving the contradiction between aggregated replication efficiency and precise state capture capability
Solution Approach 2:
A marker delta set is introduced as an intermediary element within the delta set replication stream. This marker acts as a special reference point that mediates between the aggregated replication process and the requirement for exact point-in-time state capture, allowing the system to maintain bandwidth efficiency while providing precise state identification at external events
2Reliability
If synchronous remote replication is used to ensure data consistency, then data accuracy is improved, but application performance deteriorates due to transfer time
Solution Approach 1:
The system uses periodic delta cycles to transfer aggregated writes asynchronously, improving application performance by eliminating per-write synchronization delays. Additionally, it periodically inserts marker delta sets at external events to maintain the ability to capture consistent state points, thus achieving both high performance and reliability
3Productivity
If asynchronous write ordered replication is used to improve application performance, then response time is improved, but data loss risk increases during disasters
Solution Approach 1:
The system performs preliminary actions by queuing writes locally and flushing them to remote storage before allowing application completion. This preliminary flushing ensures data safety while maintaining asynchronous operation for performance, and the marker delta sets provide additional safety by marking consistent state points for disaster recovery
Data Source
AI summary
A storage system executes a delta set asynchronous remote replication protocol, wherein writes to a local copy are aggregated into delta sets during successive delta cycles. The delta sets are sent to a remote storage system to be written to a remote copy. Independent of the delta cycles, the local storage system receives a signal representing an external event. In response to receiving the signal, all writes just prior to and in accordance with the external event are posted to the local copy, the current delta set is ended, and a marker delta set is inserted in the delta set stream. The delta sets received at the remote storage system are monitored for the marker delta set. When the marker delta set is received, it is indicated that the remote copy is consistent with the local copy just prior to the external event at the local storage system.