Active-Active Storage Failback Using Log Marker Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In active-active storage systems, the failback process often results in data logs of secondary storage nodes being out of sync with primary nodes, leading to incomplete data recovery and extended system suspend times during failback operations due to limited direct access to secondary nodes' persistent RAM.
Innovation Solution
Implementing a method where a primary storage node places a marker in the secondary node's data log and synchronizes both logs by moving head and tail pointers to allocate and reclaim space, ensuring data entries are copied and flushed in a coordinated manner, thus determining when the logs are up-to-date and synchronized.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the surviving node provides data to the failback node's persistent RAM during failover, then data synchronization is improved, but the failback node's inability to directly access its own persistent RAM prevents it from staying up-to-date
Solution Approach 1:
The patent uses the surviving node as an intermediary to transfer data log entries to the failback node's persistent RAM. Since the failback node cannot directly access its own persistent RAM during failover, the surviving node acts as a mediator that receives data entries from hosts and forwards them to the failback node, ensuring the failback node's data log remains synchronized without requiring direct access capabilities.
Solution Approach 2:
The patent implements a copying mechanism where data log entries are duplicated from the surviving node to the failback node's persistent RAM. The failback node receives copies of data entries that were written to the surviving node during the failover period, ensuring that both nodes maintain identical data logs without requiring the failback node to directly write to its own persistent storage.
2Productivity
If the failback node is immediately restored to service after failover, then system availability is improved, but data log synchronization is incomplete causing data loss
Solution Approach 1:
The patent implements preliminary actions by establishing a marker in the failback node's data log before failover begins. This marker indicates the position up to which data has been successfully transferred. During the failback process, the system can resume from this marker position, ensuring that all data entries are properly synchronized before the failback node returns to service, preventing any data loss while maintaining high availability.
Solution Approach 2:
The patent uses feedback mechanisms where the surviving node monitors the synchronization status of the failback node's data log and adjusts data transfer accordingly. The system continuously checks whether the failback node has received and processed all data entries up to the current position, providing feedback to ensure complete synchronization before allowing the failback node to resume operations, thus preventing information loss.
3Reliability
If data logs are fully synchronized during failback, then data completeness is improved, but system suspend time is extended
Solution Approach 1:
The patent performs preliminary synchronization actions during the failover period by continuously writing data entries to both the surviving node and the failback node's persistent RAM. The marker is positioned in advance to indicate the current synchronization point. This preliminary action ensures that when failback occurs, minimal additional synchronization time is needed, reducing system suspend time while maintaining data completeness.
Solution Approach 2:
The patent implements a skipping mechanism where the failback node can quickly resume processing from the marker position rather than reprocessing all data entries from the beginning. This allows the system to rush through the synchronization process by only handling the remaining unsynchronized entries, significantly reducing the time the system remains suspended during failback while ensuring complete data synchronization.
Data Source
AI summary
Techniques for providing hiccup-less failback and journal recovery. The techniques include determining a head position of a data log of a primary node and placing a marker at a corresponding position in a data log of a secondary node. In response to an IO request, the head of the primary log is moved to allocate space for writing a data entry, and the secondary node is directed to move a head of the secondary log to allocate space for writing the data entry. In response to a data entry being flushed from the primary node, a tail of the primary log is moved to reclaim allocated space for a previous data entry, and the secondary node is directed to move a tail of the secondary log to reclaim previously allocated space. In response to a tail of the secondary log coinciding with the marker position, content of the logs is synchronized.