Distributed Cluster Scaling with Learner Node Transitions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for scaling distributed systems by adding nodes often result in significant business disruptions and decreased availability due to the need to take the entire system offline, leading to potential data inconsistencies and loss of high availability.
Innovation Solution
The integration of learner nodes that do not participate in leader elections, allowing for synchronized log replication and controlled transition to follower nodes, ensuring high availability and preventing split-brain scenarios during cluster scaling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional scaling methods are used to add nodes to the distributed system, then cluster capacity is improved, but system availability deteriorates due to requiring the entire system to be taken offline
Solution Approach 1:
The patent segments the node integration process into distinct phases: initial cluster formation with voter nodes, followed by sequential addition of learner nodes that do not participate in quorum decisions, and finally transition to follower status. This segmentation allows scaling operations to proceed without taking the entire system offline, as learner nodes can be added independently of the voting quorum mechanism.
Solution Approach 2:
The patent implements preliminary actions by establishing learner nodes in a non-voting state before they are fully integrated into the cluster. These learner nodes synchronize logs and prepare for future follower roles without disrupting the existing quorum composition, enabling proactive capacity expansion while maintaining system availability.
2Productivity
If nodes are added to expand cluster capacity, then productivity is improved, but system complexity increases due to managing node transitions and preventing split-brain scenarios
Solution Approach 1:
The patent introduces learner nodes as intermediary entities between the initial voter nodes and future follower nodes. These learner nodes serve as a buffer that simplifies the transition process by providing a controlled intermediate state, reducing the complexity of direct node addition and preventing split-brain scenarios through their non-voting status.
Solution Approach 2:
The patent utilizes parameter changes by transitioning nodes through different operational states (learner → follower) with distinct characteristics. Learner nodes have modified parameters (non-voting, log-synchronizing only) that differ from voters and followers, allowing systematic management of node transitions and reducing overall system complexity.
3Manufacturing precision
If learner nodes are configured to synchronize logs before transitioning to follower nodes, then data integrity is improved, but time required for scaling increases
Solution Approach 1:
The patent maintains continuity of useful action by allowing learner nodes to synchronize logs concurrently with ongoing cluster operations rather than requiring sequential processing. The log synchronization process continues in the background without interrupting voter node operations or requiring system shutdowns, thereby preserving data integrity while minimizing scaling time.
Data Source
AI summary
A first cluster of nodes is deployed in at least one data center, the first cluster of nodes comprising: a leader node in communication with a client device and at least one follower node in communication with the leader node. One or more additional nodes are established and configured as a set of learner nodes associated with the first cluster of nodes. A predetermined number of learner nodes from the set of learner nodes are configured as a predetermined number of follower nodes in a second cluster of nodes. The leader node and the at least one follower node are transitioned from the first cluster of nodes to the second cluster of nodes. At least one learner node from the set of learner nodes is configured as a follower node in the second cluster of nodes.