Synchronous Replicated Dataset Batching for Snapshot I/O Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-site distributed storage systems experience significant I/O latency spikes during common snapshot creation due to the processing of synchronous replicated datasets, which can disrupt business operations and exceed network protocol timeouts.
Innovation Solution
Optimizations include forming batches of synchronous replicated datasets, flushing snapshots in aggregate consistency points, utilizing multithreading, and maintaining file system consistency to minimize I/O latency during snapshot creation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous replication is performed for each storage object individually during common snapshot creation, then data consistency between primary and secondary sites is maintained, but I/O latency increases significantly and processing time is extended
Solution Approach 1:
The patent combines multiple synchronous replication operations into a single batched consistency point operation. Instead of processing each storage object's replication independently, the system groups multiple storage objects into batches and processes them together through a unified consistency point mechanism, reducing the total number of operations and minimizing I/O latency while maintaining data consistency across primary and secondary sites
Solution Approach 2:
The patent segments the common snapshot creation process into distinct phases: a fencing phase where I/O operations are paused and a consistency point is established, and a snapshot creation phase where snapshots are taken. This segmentation allows the system to maintain strong consistency guarantees while reducing overall processing time by concentrating synchronization operations into a controlled time window
2Reliability
If multiple consistency points are created for each storage object during snapshot operation, then data consistency is ensured, but processing overhead increases and system performance deteriorates
Solution Approach 1:
The patent merges multiple individual consistency point operations into a single aggregate consistency point that covers multiple storage objects. By combining these operations, the system reduces processing overhead and improves performance while still ensuring data consistency across all storage objects in the batch through the unified consistency point mechanism
3Reliability
If I/O operations are paused for extended periods during common snapshot creation, then consistency is maintained, but business operations are disrupted and resumption time is increased
Solution Approach 1:
The patent performs preliminary actions by establishing the consistency point and fencing I/O operations only for the minimum necessary time to ensure consistency. The system prepares the consistency point in advance and then quickly proceeds with snapshot creation, minimizing the duration of I/O pausing while still maintaining strong consistency guarantees
Solution Approach 2:
The patent segments the snapshot creation process into a brief fencing phase followed by a snapshot creation phase. This segmentation ensures that I/O operations are paused only during the essential consistency establishment period, not throughout the entire snapshot process, thereby maintaining business operation continuity while ensuring consistency
Data Source
AI summary
Multi-site distributed storage systems and computer-implemented methods are described for improving a resumption time of input/output (I/O) operations during a common snapshot procedure for storage objects. A computer-implemented method includes initiating a snap create handler operation for a storage object of a batch of storage objects having a plurality of replicated datasets with each replicated dataset having a synchronous replication relationship between at least one storage object of the first storage node and at least one replicated storage object of the second storage node, determining whether a consistency point is currently in progress or not, and providing a hint to accelerate a currently in progress consistency point when the consistency point is currently in progress.