Leaderless Consistency Protocol for Distributed Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional distributed computing systems with a cluster leader suffer from performance bottlenecks and downtime due to leader election processes and two-phase commit protocols, leading to complexity and reduced scalability and availability.
Innovation Solution
A leaderless cluster system where any node can initiate a write operation, using a sequence number validation mechanism to ensure data consistency across nodes, allowing for simultaneous writes and a supermajority vote to confirm successful operations, eliminating the need for a centralized leader and reducing downtime.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a centralized leader is used to coordinate write operations, then consistency can be maintained, but performance degrades due to the leader becoming a bottleneck
Solution Approach 1:
The patent removes the centralized leader role from the system entirely. Instead of extracting the leader's coordination function to a separate component, it distributes the write operation capabilities to all nodes equally, allowing any node to initiate writes without needing leader approval or coordination.
Solution Approach 2:
The patent segments the write operation process into independent phases that can be executed concurrently by multiple nodes. Rather than having a single leader sequence all operations, each node can independently propose, validate, and commit writes using sequence numbers and version vectors to maintain consistency without centralized coordination.
2Reliability
If a leader election process is implemented to handle leader failure, then system availability is maintained, but downtime occurs during the election process
Solution Approach 1:
The patent completely removes the leader election mechanism from the system. By eliminating the leader role entirely, there is no need for election processes, and the system maintains continuous operation without any election-related downtime or availability interruptions.
3Reliability
If a two-phase commit protocol is used to ensure consistency, then data integrity is maintained, but system complexity increases
Solution Approach 1:
The patent changes the fundamental parameters of the commit protocol by replacing the two-phase approach with a continuous phase system using sequence numbers and version vectors. Instead of distinct prepare and commit phases with complex coordination, nodes use monotonically increasing sequence numbers and vector clocks to automatically determine commit order and maintain consistency with simpler logic.
4Reliability
If majority votes are required for each write operation, then consistency is ensured, but write performance decreases due to coordination overhead
Solution Approach 1:
The patent applies preliminary action by having nodes pre-establish sequence numbers and version vectors before write operations occur. These pre-computed parameters allow nodes to quickly determine commit validity without real-time voting or coordination, enabling faster writes while maintaining consistency through the pre-set ordering information.
Data Source
Figure 1
Figure 2~3
Figure 4a
AI summary
Systems and methods are provided for processing requests, received from clients and peer nodes, in a leaderless cluster of nodes. A node can initiate a read or a write operation in the cluster. The node can solicit votes from its peer nodes in order to confirm that the requested operation should be performed. A threshold number of positive responses, such as a supermajority of the nodes in the cluster, can be required in order to confirm the requested operation.