Active-Standby Failover via Tentative Active Mode
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network configurations face challenges in managing split-brain conditions where both active and standby nodes mistakenly become active, leading to unnecessary role reversals and potential service disruptions, especially when heartbeats are missed or BFD sessions fail, causing both nodes to attempt to provide services independently.
Innovation Solution
Introducing a method that differentiates between tentative and confirmed active modes, where nodes can transition to tentative active without peer communication and require handshaking to confirm active status, ensuring only one node remains active and the other transitions to standby, regardless of initial ranks or failure scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the standby node declares itself active when it does not receive a heartbeat from the active node, then service availability is maintained, but split-brain condition occurs where both nodes become active
Solution Approach 1:
The active mode is segmented into two distinct states: tentative active mode and confirmed active mode. This segmentation allows the standby node to transition to tentative active mode when heartbeat is lost, while requiring additional handshaking communication to become confirmed active, thereby preventing split-brain conditions where both nodes operate independently as active nodes.
Solution Approach 2:
The node performs preliminary action by transitioning to tentative active mode before fully becoming active. This preliminary state allows the node to prepare for taking over services while still requiring verification through handshaking with the peer node, ensuring that the transition is coordinated and prevents conflicting active states.
2Ease of operation
If the higher rank node takes over as active node after recovery, then role assignment follows configured priority, but unnecessary role reversal occurs disrupting service continuity
Solution Approach 1:
The system implements dynamic role assignment where the node that was previously active retains priority to remain active after recovery, rather than strictly following static rank configuration. The handshaking protocol dynamically determines the final role assignment based on current state, preventing unnecessary role reversals and service disruptions while still allowing rank-based assignment when appropriate.
3Adaptability or versatility
If a third entity is used to break ties in active-standby selection, then tie-breaking capability is provided, but message loss and incorrect state information may occur
Solution Approach 1:
The tie-breaking functionality is extracted from a separate third entity and integrated directly into the peer-to-peer handshaking protocol between the two nodes. This eliminates the need for external message passing through a management or control plane, thereby preventing message loss and incorrect state information while maintaining the ability to break ties through direct node communication.
4Reliability
If the standby node transitions directly to active mode upon heartbeat failure, then service continuity is maintained, but unnecessary role changes occur when the active node recovers
Solution Approach 1:
The active mode is divided into tentative active and confirmed active states, adding a intermediate step that prevents premature role changes. This segmentation maintains service continuity by allowing the standby node to quickly transition to tentative active mode while preventing unnecessary role reversals by requiring handshaking confirmation before becoming confirmed active.
Data Source
AI summary
A method of determining the state of each peer node in a pair of first and second peer nodes is provided. The first node is configured as a confirmed active node and the second node is configured as a standby node. The second node transitions into a tentative active mode when a predetermined number of heartbeat messages are lost between the pair nodes while the first and second nodes remain functional. The second node receives a first message from the first node after the heartbeat messages are established between the peer nodes. The first message indicates that the first node is in the confirmed active mode. In response to the first message, the second node transitions from the tentative active mode into the standby mode and sends a second message from the second node to the first node indicating that the second node is in the standby mode.