Op Log Resynchronization Across Storage Clusters After Sync Failures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cross-site distributed storage systems face challenges in maintaining consistency and synchronicity between primary and secondary storage clusters due to transient or persistent connectivity issues, leading to disruptions and operational inefficiencies.
Innovation Solution
Implement persistent inflight tracking and reconciliation of operations (Ops) using Op logs across primary and secondary storage clusters, allowing for resynchronization and maintaining synchronous replication, thereby ensuring zero recovery point objective (RPO) protection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous replication is implemented between primary and secondary storage clusters, then data consistency and reliability are improved, but I/O latency increases due to the need to wait for replication completion
Solution Approach 1:
The patent applies preliminary action by pre-establishing and maintaining Op logs on both primary and secondary storage clusters before failures occur. These logs continuously track inflight operations and their states, enabling rapid resynchronization without requiring full re-replication after disruptions. The Op logs are updated in advance during normal operation, so when a failure occurs, the system already has the necessary operational history to quickly recover consistency.
2Reliability
If persistent inflight tracking is implemented using Op logs, then resynchronization capability is improved, but device complexity increases due to additional logging and tracking mechanisms
Solution Approach 1:
The patent applies universality by designing Op logs that serve multiple critical functions: tracking inflight operations during normal operation, enabling failure detection, facilitating resynchronization after disruptions, and maintaining replication consistency. Rather than implementing separate mechanisms for each function, the Op log structure is designed to handle all these purposes with a single unified system, reducing overall complexity despite the advanced capabilities provided.
3Reliability
If I/O operations are temporarily disallowed during failure, then data consistency is protected, but productivity decreases due to reduced availability
Solution Approach 1:
The patent applies continuity of useful action by using Op logs to track and manage I/O operations during failure conditions. Instead of completely blocking operations, the system continues to log and monitor inflight operations, enabling them to be processed and synchronized after the failure is resolved. The Op logs maintain the state of operations that were interrupted, allowing the system to resume service with minimal disruption and maintain data consistency without complete unavailability.
Data Source
AI summary
In one embodiment, a method comprises maintaining state information regarding a data synchronous replication status for a storage object of a primary storage cluster and a replicated storage object of a secondary storage cluster. The method includes temporarily disallowing input/output (I/O) operations for the storage object when the storage object of the primary storage cluster has a failure, which causes an internal state as out of sync for the storage object while maintaining an external state as in sync for external entities. The method performs persistent inflight tracking and reconciliation of I/O operations with a first Op log of the primary storage cluster and a second Op log of the secondary storage cluster and performs a resynchronization between the storage object and the replicated storage object based on the persistent inflight tracking and reconciliation of I/O operations.


