Replication Log for Asynchronous Remote Data Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face high processing costs and inefficiencies in remote replication due to the expensive snapshot difference technique and cache misses when accessing data for replication, particularly in asynchronous remote replication scenarios with high Recovery Point Objectives (RPO) and minimal performance impact.
Innovation Solution
The implementation of a replication log on the source system that tracks write data flushed but not yet replicated, allowing efficient access without mapping information, and using checkpoint or barrier records for snapshot semantics without constructing metadata structures, enabling efficient asynchronous replication with minimal performance impact.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the snapshot difference technique is used for remote replication, then data consistency is maintained, but processing overhead and system complexity increase significantly
Solution Approach 1:
The patent extracts the replication metadata from the complex snapshot difference technique and stores it separately in a replication log. This log contains only the essential information (logical addresses, data lengths, checksums) needed for replication, eliminating the need to manage full snapshot metadata structures and their associated complexity while maintaining data consistency.
Solution Approach 2:
Instead of creating full snapshots and computing differences between them, the patent uses a simplified copying approach where write operations are directly logged in the replication log. This log serves as a lightweight copy of the essential replication information, avoiding the overhead of snapshot creation and difference computation while ensuring data consistency.
2Speed
If data is accessed from cache for replication, then replication speed improves, but cache misses occur when data has been flushed, increasing access time
Solution Approach 1:
The patent performs preliminary actions by logging write operations to the replication log immediately when they occur, before the data is flushed from the cache. This ensures that the replication log is updated in advance, so when replication needs to occur, the necessary information is already available without requiring cache access or causing delays.
3Productivity
If asynchronous replication is implemented with high RPO, then performance impact is minimized, but data loss risk increases
Solution Approach 1:
The patent implements a feedback mechanism using checksums in the replication log. Each write operation is recorded with its checksum, and during replication, the destination system can verify data integrity by comparing checksums. This feedback loop ensures that even in asynchronous replication with potential data loss scenarios, the system can detect and handle errors, maintaining reliability while preserving performance.
Data Source
AI summary
Data replication techniques can include receiving, at a source system, a write directed to a source logical device configured for asynchronous remote replication to a destination system; performing processing that flushes a transaction log entry for the write; and performing replication processing that uses a replication queue including a replication queue entry corresponding to the write that stores the first content to a logical address. The processing can create a replication log entry in a replication log for the write responsive to determining that the write is directed to the source logical device configured for asynchronous remote replication and that the first content has not been replicated. Responsive to the first content not being in cache, the first content can be retrieved using the reference to a storage location storing the first content. The reference can be obtained from the replication log entry or the replication queue entry.