Locality Based Quorums for Distributed Data Durability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed data stores employing a single-master replication model face challenges in ensuring data durability and integrity when the master data store fails, as the slave data stores need to assume the master role and maintain the last successful updates committed to the system.
Innovation Solution
Implementing a locality-based durability and failover quorum mechanism, where a new master data store is elected using a consensus algorithm, ensuring that updates are synchronized with a smaller number of slave nodes, facilitating faster and fault-tolerant failover without loss of client updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single-master replication model is used, then data write performance is improved, but system reliability deteriorates when the master fails
Solution Approach 1:
The system performs preliminary actions by replicating data to multiple slave nodes before the master failover occurs. The locality-based durability quorum ensures that data is pre-synchronized to a sufficient number of nodes in different locations, so when master failure happens, the new master can immediately serve requests without data loss.
Solution Approach 2:
The patent applies local quality by introducing locality-based quorums where different geographical locations have different replication requirements. Each location can configure its own durability level, allowing local optimization where critical locations have higher replication factors while less critical locations use lower factors, thus balancing reliability and performance.
2Reliability
If data is replicated to all slave nodes, then data durability is improved, but failover time increases
Solution Approach 1:
The system applies partial action by replicating data to only the necessary number of slave nodes defined by the durability quorum parameter, rather than all slave nodes. This allows the system to achieve sufficient durability with fewer replications, reducing the time required for failover while maintaining data integrity.
Solution Approach 2:
The patent segments the replication process by dividing slave nodes into different locality groups. Each group can be independently configured with different quorum requirements, allowing parallel replication to multiple segments simultaneously, which reduces overall failover time while maintaining durability guarantees.
3Speed
If a locality-based quorum mechanism is implemented, then failover speed is improved, but system complexity increases
Solution Approach 1:
The system uses parameter changes by introducing configurable quorum parameters (durability and availability quorums) that can be adjusted without changing the underlying system architecture. This allows fast failover to be achieved by simply modifying numerical parameters rather than restructuring the entire system, thus managing complexity.
Solution Approach 2:
The locality-based quorum mechanism enables self-service by allowing each location to independently configure its own durability and availability requirements. The system automatically selects the appropriate quorum based on these configurations without requiring complex centralized coordination, reducing overall system complexity.
Data Source
AI summary
Disclosed are various embodiments for distributing data items within a plurality of nodes. A data item that is subject to a data item update request is updated from a master node to a plurality of slave notes. The update of the data item is determined to be locality-based durable based at least in part on acknowledgements received from the slave nodes. Upon detection that the master node has failed, a new master candidate is determined via an election among the plurality of slave nodes.