In-Memory Cache Snapshot Management for Asynchronous Replication RPO Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Asynchronous replication in storage systems faces challenges in minimizing the recover point objective (RPO) time while maintaining performance, as existing methods take significant time to execute snapshot creation, deletion, and copy-diff operations, leading to potential data loss during recovery.
Innovation Solution
The method involves maintaining dirty data in an in-memory cache for replication snapshots, executing difference operations within the cache, and flushing data to the base volume, which reduces the need for Mapper overhead and allows for more frequent snapshot creation and deletion, thereby lowering RPO times without impacting performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If asynchronous replication is executed frequently to reduce RPO time, then data protection improves, but system performance deteriorates due to latency spikes from snapshot operations
Solution Approach 1:
The system performs snapshot creation and deletion operations in advance within the in-memory cache before they are needed for replication. By pre-positioning snapshot data in the cache, the system eliminates the need for time-critical operations during replication execution, thus avoiding performance degradation while maintaining frequent replication capability for data protection
Solution Approach 2:
The in-memory cache acts as an intermediary layer between the base volume and the replication process. Snapshot operations are performed within this intermediary cache memory space, isolating them from the base volume operations. This intermediary structure allows frequent snapshot operations to occur without directly impacting the performance of the base storage system
2Speed
If snapshot creation and deletion operations are executed quickly to reduce RPO, then replication speed improves, but operation complexity increases due to frequent snapshot management
Solution Approach 1:
The system merges multiple snapshot operations into a unified in-memory cache structure. Instead of managing separate snapshot operations on the base volume, the cache consolidates snapshot data from multiple time points, allowing replication to efficiently access and compare snapshots without managing individual snapshot creation and deletion complexity on the base volume
Solution Approach 2:
The system creates copies of snapshot data in the in-memory cache rather than performing operations directly on the base volume. This copying approach allows frequent snapshot operations to be performed on the cache copies, speeding up replication while the base volume remains unaffected, thus reducing the operational complexity of frequent snapshot management
3Loss of time
If in-memory cache is used to maintain dirty data for snapshots, then RPO time reduces, but memory resource consumption increases
Solution Approach 1:
The system changes the state parameter of dirty data by maintaining it in the in-memory cache rather than immediately writing to the base volume. This parameter change in data location (from volume to cache) enables faster snapshot operations and reduced RPO time, as the cache provides rapid access to recent writes for snapshot creation and replication operations
Data Source
AI summary
A method, computer program product, and computer system for receiving, by a computing device, a snapshot create operation of a volume to create a first snapshot. Existing dirty data of the volume for the first snapshot may be flushed from an in-memory cache. New writes to the volume for the first snapshot may be maintained in the in-memory cache as dirty. A snapshot create operation to the volume may be received to create a second snapshot. The new writes to the volume for the first snapshot may be combined as part of the second snapshot.