Clustered Storage Resynchronization via Inflight and Dirty Region Logs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Clustered storage systems face synchronization challenges due to failures, leading to unsynchronized data between primary and secondary clusters, which can result in data loss and service disruptions.
Innovation Solution
The implementation of synchronization logs, such as inflight tracker logs and dirty region logs, to track and manage data synchronization across clusters, ensuring that data is consistently written and committed on both primary and secondary clusters, and utilizing a resync engine to perform resynchronization operations when failures occur.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is written in parallel to both primary and secondary clusters, then write performance is improved, but synchronization reliability deteriorates when failures occur
Solution Approach 1:
The system performs preliminary actions by writing data to both primary and secondary clusters in parallel before the operation is complete, then uses synchronization logs to track and resolve any discrepancies that arise, ensuring eventual consistency while maintaining high write performance
Solution Approach 2:
Synchronization logs provide feedback mechanisms that monitor the state of data across clusters and trigger resynchronization operations when inconsistencies are detected, thereby maintaining reliability without sacrificing the parallel write performance benefit
2Reliability
If resynchronization operations are performed after failures, then data integrity is improved, but system downtime increases
Solution Approach 1:
Synchronization logs continuously track the state of data across clusters in advance, so when a failure occurs, the system can immediately resume synchronization from the logged state rather than performing a complete resynchronization, thereby reducing downtime while maintaining data integrity
Solution Approach 2:
The system skips unnecessary resynchronization steps by using synchronization logs to identify only the specific data portions that need to be resynchronized, rather than resynchronizing entire datasets, thus minimizing system downtime while ensuring data integrity
3Measurement precision
If synchronization logs are implemented to track data changes, then synchronization accuracy is improved, but device complexity increases
Solution Approach 1:
Synchronization logs are segmented into specific data structures (inflight tracker logs and dirty region logs) that track different aspects of data synchronization independently, improving accuracy while managing complexity through modular organization of tracking mechanisms
Solution Approach 2:
Synchronization logs act as intermediary structures between primary and secondary clusters, providing precise tracking of data changes without requiring direct complex coordination between clusters, thereby improving synchronization accuracy while isolating complexity into manageable log structures
Data Source
AI summary
Various embodiments are generally directed an apparatus and method for receiving information to write on a clustered system comprising at least a first cluster and a second cluster, determining that a failure event has occurred on the clustered system creating unsynchronized information, the unsynchronized information comprising at least one of inflight information and dirty region information, and performing a resynchronization operation to synchronize the unsynchronized information on the first cluster and the second cluster based on log information in at least one of an inflight tracker log for the inflight information and a dirty region log for the dirty region information.