Op Log Based Fast Recovery of Bidirectional Synchronous Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing dual copy data storage systems with bidirectional synchronous replication face challenges in fast recovery and metafile management, particularly in symmetric active/active configurations where ensuring unique sequence numbers for operations and efficiently purging metafiles are complex issues.
Innovation Solution
The system designates one storage site as a global sequence number generator, maintains counters for operation sequence numbers and cumulative response sequence numbers on both sites, and employs an Op log journaling mechanism for inflight tracking replay and reconciliation to ensure fast resynchronization and metafile purging.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If bidirectional synchronous replication is implemented in a cross-site distributed storage system, then data consistency and availability are improved, but system complexity and metafile management difficulty increase
Solution Approach 1:
The system segments the replication state management by introducing separate Op log files at each storage site, tracking inflight operations independently. This segmentation allows each site to manage its own replication state without requiring full system coordination for every operation, reducing overall system complexity while maintaining data consistency.
Solution Approach 2:
The patent introduces an intermediary mechanism using Op log files that act as mediators between the primary and secondary storage sites. These logs track the sequence numbers and states of operations, enabling coordinated replication without direct complex interaction between the storage sites, thus simplifying the system architecture.
2Measurement precision
If operation sequence numbers are tracked for each storage site, then replication accuracy is improved, but metafile overhead and storage requirements increase
Solution Approach 1:
Each storage site maintains its own local Op log file with sequence numbers specific to that site's operations. This local quality approach allows precise tracking of replication state at each site without requiring redundant sequence number tracking across both sites, reducing metafile overhead while maintaining replication accuracy.
Solution Approach 2:
The system discards old Op log files that are no longer needed for replication tracking and recovers storage space by purging completed or expired logs. This mechanism prevents unbounded growth of metafiles while maintaining the necessary history for accurate replication state management.
3Productivity
If inflight operations are tracked and reconciled, then recovery speed is improved, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary action by continuously tracking inflight operations in Op log files at both storage sites during normal operation. This preparation allows the reconciliation process to proceed quickly by simply comparing existing logs rather than tracking operations in real-time during recovery, thus improving recovery speed without significant processing time penalty.
Solution Approach 2:
During recovery, the system skips redundant reconciliation steps by using the pre-tracked Op log information. Instead of re-evaluating every operation from scratch, the system rushes through the reconciliation process by directly comparing sequence numbers and states recorded in the logs, significantly reducing processing time while maintaining accuracy.
Data Source
AI summary
In one embodiment, a computer-implemented method comprises establishing bi-directional synchronous replication between one or more storage objects of a primary storage site and one or more storage objects of a secondary storage site with each storage site having read/write access while maintaining zero recovery point objective (RPO) and Zero recovery time objective (RTO), initiating a resynchronization process due to a loss of the bi-directional synchronous replication between the one or more storage objects and the one or more storage objects, and performing the resynchronization process based on using inflight tracking replay and reconciliation between a first Op log metafile of the primary storage site and a second Op log metafile of the secondary storage site.