Cascaded Replication Using Snapshot Boundaries for Data Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cascaded replication, there is a need to avoid conflicts between replication snapshots taken at different storage systems by ensuring that only changes occurring before the last synchronized snapshot are replicated, particularly for destination FSEs that are part of multiple replication streams, to maintain data consistency and prevent data loss during recovery.
Innovation Solution
The primary storage system takes a snapshot of CR FSEs at the start of each replication cycle and replicates changes only for source FSEs since the previous CR snapshot, while for destination FSEs, changes are replicated up to the last synchronized destination snapshot before the current CR cycle, avoiding updates made after the last sealed snapshot.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If changes are replicated continuously in cascaded replication, then data consistency is improved, but conflicts between snapshots from different storage systems occur
Solution Approach 1:
The system performs preliminary actions by tracking and recording snapshot information from multiple storage systems in advance. Before replication occurs, the system identifies the most recent snapshot timestamp and uses this pre-determined reference point to control what changes are replicated, preventing snapshot conflicts before they can occur.
Solution Approach 2:
The system implements feedback mechanisms by continuously monitoring snapshot timestamps from different storage systems and using this information to regulate replication. The replication process receives feedback about snapshot states and adjusts its behavior accordingly, only replicating changes that occur before the identified snapshot point to maintain consistency.
2Quantity of substance
If all changes are replicated in cascaded replication, then data completeness is improved, but data loss during recovery increases
Solution Approach 1:
The system performs preliminary identification of the safe replication boundary by tracking snapshot timestamps before replication occurs. This preliminary action determines exactly which changes can be safely replicated without risking data loss during recovery, ensuring both completeness and safety.
Solution Approach 2:
The system changes the replication parameter from replicating all changes to replicating only changes that occur before a specific snapshot timestamp. This parameter change controls the replication scope to prevent data loss while maintaining data completeness within the safe boundary.
3Adaptability or versatility
If replication includes destination FSEs from multiple replication streams, then replication versatility is improved, but snapshot synchronization conflicts increase
Solution Approach 1:
The system performs preliminary analysis of multiple replication streams to identify the most recent snapshot timestamp across all streams before replication. This preliminary action establishes a common reference point that all destination FSEs can use for synchronization, enabling versatile multi-stream replication while maintaining reliability.
Solution Approach 2:
The system creates equipotentiality by using a unified snapshot timestamp reference point for all destination FSEs from different replication streams. This equalizes the synchronization baseline across all streams, allowing them to be replicated consistently without conflicts despite their different origins.
Data Source
AI summary
A method for asynchronous replication, the method includes initiating a current replication cycle of multiple files system entities (FSEs) that include primary FESs originated from a primary storage system (PSS) and external FSEs that originated from one or more external storage systems (ESSs), each external storage system differs from the primary storage system and differs from a secondary storage system. The initiating is followed by sending primary content and external content from the primary storage system to the secondary storage system. The primary content is indicative of differences in the primary FSEs between a previous replication cycle point in time (PIT) and a current replication cycle PIT. For each external FSE, an external content is indicative of differences in the external FSEs between (a) a last external FSE snapshot that preceded the previous replication cycle PIT and was generate by an external FSE snapshot process, and (b) a last external FSE snapshot that preceded the current replication cycle PIT and was generate by the external FSE snapshot process. The external FSE snapshot process and a process for generating the current replication cycle are not mutually synchronized.