Edge Node Split-Brain Recovery via Management Plane
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In virtual networking, split-brain scenarios occur when edge nodes fail to communicate properly, leading to network disruptions and IP address overlaps, as both nodes declare themselves active, causing high availability issues.
Innovation Solution
A method is employed by the management and control system to dynamically recover from split-brain conditions by specifying which edge node should be active or standby, using a correction timer and majority cluster status to ensure proper node states, and optionally increasing HA interfaces for better communication and detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If both edge nodes declare themselves active in a split-brain scenario, then network service availability is improved, but network stability deteriorates due to IP address overlaps and configuration conflicts
Solution Approach 1:
The patent introduces a management plane as an intermediary between the two edge nodes. When split-brain occurs, the management plane receives state information from both nodes, determines which node should be active based on predefined criteria, and sends corrective commands to the appropriate node. This intermediary coordination prevents both nodes from remaining active simultaneously, resolving the contradiction between service availability and network stability.
2Stability of the object's composition
If edge nodes are rebooted to resolve split-brain failure, then network stability is restored, but service interruption time increases
Solution Approach 1:
The patent implements preliminary action by having the management plane detect split-brain conditions and send corrective commands to edge nodes before the failure fully propagates through the network. The management plane monitors node states continuously and intervenes proactively by commanding the appropriate node to transition to standby state, preventing the need for complete reboots and reducing service interruption time while restoring stability.
3Stability of the object's composition
If the management plane actively resolves split-brain failure by specifying node states, then network stability is improved, but control system complexity increases
Solution Approach 1:
The patent applies self-service by enabling edge nodes to autonomously report their state to the management plane and automatically execute corrective commands received from the management plane. The edge nodes themselves participate in resolving the split-brain condition by transitioning to standby state when commanded, rather than requiring complex external intervention. This reduces control system complexity while maintaining network stability.
Data Source
AI summary
Some embodiments provide a method for employing the management and control system of a network to dynamically recover from a split-brain condition in the edge nodes of the network. The method of some embodiments takes a corrective action to automatically recover from a split-brain failure occurred at a pair of high availability (HA) edge nodes of the network. The HA edge nodes include an active machine and a standby machine. The active edge node actively passes through the network traffic (e.g., north-south traffic for a logical network), while the standby edge node is synchronized and ready to transition to the active state, should a failure occur. Both HA nodes share the same configuration settings and only one is active until a path, link, or system failure occurs. The active edge node also provides stateful services (e.g., stateful firewall, load balancing, etc.) to the data compute nodes of the network.