Two-Channel High-Availability Node Failure Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Detecting failures in high availability (HA) node clusters is inefficient and prone to false detections due to aggressive timers in Bidirectional Forwarding Detection (BFD) channels, leading to unnecessary failovers even when nodes are still healthy.
Innovation Solution
Implementing a two-channel-based HA system that uses both underlay control channels and management control channels to monitor BFD control packets, allowing local control planes to determine node failures more accurately through diagnostic codes, thereby reducing false detections and improving reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If aggressive timers are used in BFD channels to detect node failures quickly, then failure detection speed is improved, but false detection rate increases due to congestion-induced packet loss
Solution Approach 1:
The patent introduces a separate management control channel as an intermediary communication path between HA nodes, distinct from the underlay control channel that carries BFD traffic. This intermediary channel provides a dedicated path for management communications, isolating them from congestion on user traffic channels and enabling reliable failure detection without false positives from congestion-induced packet loss.
Solution Approach 2:
The patent segments the communication infrastructure into two distinct channels: an underlay control channel for BFD and user traffic, and a separate management control channel for HA management communications. This segmentation allows each channel to be optimized independently, with the management channel providing reliable failure detection while the underlay channel handles high-volume user traffic without interfering with detection accuracy.
2Device complexity
If BFD control packets are monitored over a single channel shared with user traffic, then device complexity is reduced, but measurement precision deteriorates due to inability to distinguish congestion from actual failures
Solution Approach 1:
The patent adds another dimension to the communication infrastructure by introducing a separate management control channel alongside the underlay control channel. This dimensional addition provides an independent observation path for monitoring node health, enabling the system to cross-validate failure detections and distinguish between congestion-induced packet loss and actual node failures, thereby improving measurement precision without significantly increasing operational complexity.
3Ease of operation
If a single channel is used for both underlay control and management communications, then ease of operation is improved, but reliability deteriorates due to single point of failure
Solution Approach 1:
The patent applies local quality by assigning different functional characteristics to different channels: the underlay control channel is optimized for BFD control packets and user traffic, while the separate management control channel is optimized for HA management communications. This functional differentiation ensures that management communications are not affected by congestion on the underlay channel, providing localized reliability improvement for critical failure detection functions.
Data Source
AI summary
A method for providing two-channel-based high-availability in a cluster of nodes is disclosed. In an embodiment, a method comprises: initiating, by a local control plane executing on a first node, a first state for an underlay control channel and a second state for a management control channel; detecting a bidirectional forwarding detection (BFD) control packet from a second node; determining whether the BFD control packet has been received from the underlay control channel; in response to determining that the BFD control packet was received from the underlay control channel: parsing the BFD control packet to extract a first diagnostic code; updating the first state with the first diagnostic code; determining whether both the first state and the second state indicate a need to switch services configured on the second node; in response to the determining, initiating a switchover of services configured on the second node.