Asynchronous Replication Queue for Crash-Consistent Object Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face challenges in ensuring crash consistency and proper sequencing of replication tasks in large distributed systems, particularly when network conditions are unfavorable, leading to inconsistencies and unsatisfactory latency.
Innovation Solution
A transaction protocol generates and persistently stores replication management metadata as part of an atomic transaction, using a file system directory structure to maintain the state of replication tasks, and employs object metadata as a single source of truth for sequencing, ensuring crash consistency and eventual progress of replication tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If asynchronous replication is implemented in large distributed systems, then replication latency is reduced and system performance is improved, but crash consistency and proper sequencing of replication tasks cannot be guaranteed under unfavorable network conditions
Solution Approach 1:
The system performs preliminary actions by creating replication markers and storing them in a queue before actual replication occurs. These markers serve as advance preparation for crash recovery, allowing the system to resume replication from the correct sequence after a crash without losing consistency.
Solution Approach 2:
The patent introduces replication markers as intermediary objects that mediate between the source storage node and destination storage node. These markers track replication state and enable crash consistency by providing a reliable mechanism to resume replication tasks in the correct sequence after network failures or crashes.
2Loss of time
If replication tasks are processed asynchronously in the background, then original storage operation latency is improved, but inconsistencies may occur due to disconnects between storage operations and replication notifications
Solution Approach 1:
The system implements feedback mechanisms where the source storage node monitors the replication queue and processes replication markers. This feedback loop ensures that replication tasks are properly tracked and completed, preventing information loss even when storage operations proceed asynchronously in the background.
Solution Approach 2:
The patent replaces traditional mechanical notification systems with an event-driven architecture using replication markers and queues. This substitution eliminates the need for direct notification mechanisms between storage operations and replication processes, allowing asynchronous processing without consistency loss.
3Reliability
If synchronous replication is used to ensure crash consistency, then data consistency is maintained, but storage operation latency becomes unacceptable in large distributed systems
Solution Approach 1:
The patent segments the replication process into distinct phases: marker creation, queue storage, and background processing. This segmentation allows the storage operation to complete quickly without waiting for replication, while still maintaining crash consistency through the structured multi-stage approach.
Solution Approach 2:
The system dynamically adjusts replication behavior based on system state. The source storage node dynamically processes replication markers from the queue, allowing flexible adaptation to network conditions and system load while maintaining consistency guarantees without fixed synchronous timing.
Data Source
AI summary
Systems and methods are provided for durable asynchronous replication in a distributed storage system. A storage system may commit a transaction comprising saving a data object to a persistent object storage location and saving a replication marker to a first persistent storage directory for pending replication tasks, wherein the replication marker comprises replication data for replicating the data object to a remote storage system, and wherein the first persistent storage directory comprises a replication queue; determine, based on a presence of the replication marker in the first persistent storage directory, to dequeue the replication marker from the replication queue; move the replication marker to a second persistent storage directory for in-process replication tasks; attempt replication of the data object to the remote storage system; and move the replication marker to a third persistent storage directory.