Topology-Aware Consensus Protocol for Distributed Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing consensus protocols in distributed applications face scalability issues due to reliance on centralized coordinators, leading to increased latency and limited performance as the number of participants grows, especially in wide-area deployments with restricted network link capacities.
Innovation Solution
A leaderless, parallel, and topology-aware consensus protocol that groups nodes by physical proximity, achieving consensus in multiple rounds, with each round aggregating and ordering write requests across progressively larger groups, allowing all nodes to agree on the order of write requests without a centralized leader.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a centralized coordinator is used to service client requests and replicate state changes, then the system achieves consensus, but the processing load and network traffic concentrate at the coordinator, causing increased latency and limited scalability
Solution Approach 1:
The patent divides the centralized coordinator role into multiple peer nodes that all participate in consensus. Instead of one coordinator handling all requests, the system segments the work across N nodes, where each node can service client requests and participate in replication, distributing the processing load and eliminating the bottleneck at a single coordinator.
Solution Approach 2:
The patent merges the roles of coordinator and participant into a single peer-to-peer model where all nodes have equal status. Each node simultaneously services client requests, maintains state changes, and participates in consensus voting, combining functions that were previously separated between coordinator and followers.
2Reliability
If the number of participants in the consensus protocol increases to increase fault tolerance, then the system becomes more reliable, but the performance degrades due to increased coordination overhead
Solution Approach 1:
The patent segments the consensus process into multiple parallel phases: proposal generation, voting, and commitment. Each phase can be executed independently and in parallel across nodes, reducing the coordination overhead that would otherwise scale quadratically with the number of participants.
Solution Approach 2:
The patent changes the parameter of consensus rounds from sequential to parallel execution. Multiple voting rounds can occur simultaneously for different proposals or different partitions of data, allowing the system to maintain fault tolerance with N nodes while preventing performance degradation through parallelized consensus operations.
3Productivity
If existing consensus protocols move from a single centralized coordinator to a set of coordinators, then the processing load is spread out, but message dissemination remains non-parallel and unaware of network topology, limiting scalability in wide-area deployments
Solution Approach 1:
The patent applies local quality by having nodes communicate preferentially with geographically or topologically proximate nodes. The consensus protocol is aware of network topology and routes messages through intermediate nodes based on their physical location, reducing the complexity of message dissemination in wide-area deployments by exploiting the structure of the network.
Solution Approach 2:
The patent adds a spatial dimension to the consensus protocol by organizing nodes into regions or datacenters and allowing consensus to proceed independently within each region before global coordination. This transforms the flat, single-layer consensus into a multi-layered approach that reduces communication complexity by processing locally first and then coordinating globally.
Data Source
AI summary
Methods are provided for achieving consensus among an order in which write requests are received by various ones of a plurality of nodes in a distributed system using a shared data structure. The plurality of nodes are organized into groups of nodes and successively larger groupings of groups, based on physical proximity. A consensus protocol is used to achieve consensus among groups of nodes, and then among the groupings of groups of nodes in a logical tree structure up to a root level virtual node.