Decoupled Reliability Groups for Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional multi-phase commit protocols in distributed data storage systems introduce latency and resource demands due to lockstep communication among nodes, limiting progress and complicating error scenarios, especially when updating data and checksum elements independently.
Innovation Solution
Decoupling update processes between data zones and checksum zones by using version counters and contribution vectors, allowing independent updates and maintaining relative state tracking for error recovery, reducing communication and enabling nodes to operate at their own pace.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multi-phase commit protocols are used to maintain synchronization between data zones and checksum zones, then data reliability is improved, but communication overhead and latency increase
Solution Approach 1:
The patent segments the reliability group into independent data zones and checksum zones that can operate autonomously. Each zone maintains its own update state and can progress independently, eliminating the need for coordinated multi-phase commit protocols while preserving data reliability through version tracking mechanisms.
Solution Approach 2:
The patent implements preliminary version counter increments and contribution vector updates before actual data transfers complete. This allows nodes to be preemptively notified of updates and prepare for recovery operations in advance, reducing the latency associated with waiting for synchronized commit states.
2Stability of the object's composition
If multi-phase commit protocols are used to ensure synchronized updates, then data consistency is improved, but node progress is limited by the slowest node
Solution Approach 1:
The patent introduces dynamic version counters and contribution vectors that automatically track the relative update states of different nodes. This dynamic tracking allows each node to progress at its own speed while maintaining consistency information, eliminating the bottleneck effect of the slowest node in lockstep protocols.
Solution Approach 2:
The patent implements feedback mechanisms where nodes continuously report their update states through version counters and contribution vectors. This feedback allows the system to monitor and track consistency without requiring synchronous coordination, enabling independent node progression while maintaining overall data consistency.
3Reliability
If lockstep communication is used among nodes, then error recovery reliability is improved, but communication overhead and resource demands increase
Solution Approach 1:
The patent extracts the synchronization coordination function from the data update process itself. By separating version tracking (through contribution vectors) from actual data transfers, the system eliminates the need for repeated lockstep communication rounds while maintaining the ability to recover from errors, significantly reducing communication overhead.
4Stability of the object's composition
If multi-phase commit protocols are used to coordinate updates, then atomicity of updates is improved, but system complexity increases
Solution Approach 1:
The patent enables each node to self-manage its update state through local version counters and contribution vectors. Nodes independently track their own progress and can autonomously participate in recovery operations without requiring complex coordinated protocols, simplifying the system while maintaining atomicity guarantees through the version tracking mechanism.
Data Source
AI summary
Methods and apparatuses for updating members of a data storage reliability group are provided. In one exemplary method, a reliability group includes a data zone in a first storage node and a checksum zone in a second data storage node. The method includes updating a version counter associated with the data zone in response to destaging a data object from a staging area of the data zone to a store area of the data zone without synchronizing the destaging with the state of the checksum zone. The method further includes transmitting, from the data zone to the checksum zone, an update message indicating completion of the destaging of the data object, wherein the update message includes a current value of the version counter.