Cluster Failure Detection via Proactive Status Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional high availability and disaster recovery solutions rely on a reactive inquiry mechanism to detect cluster failures, which increases delay and requires multiple message exchanges, making it inefficient in quickly distinguishing between network outages and cluster failures during disasters.
Innovation Solution
Implementing a proactive Status Update protocol that maintains a connected cluster list across clusters, allowing for rapid detection of connectivity changes and eliminating the need for inquiry responses, thereby reducing message exchanges and detection time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a reactive inquiry mechanism is used to detect cluster failures, then the system can determine cluster health status, but the detection time increases and multiple message exchanges are required
Solution Approach 1:
The patent implements a proactive status update mechanism where clusters continuously maintain and broadcast their connectivity status to all other clusters in the system. This preliminary action of continuously updating status information eliminates the need for reactive inquiry mechanisms, allowing any cluster to immediately determine failure conditions by checking their own status records without waiting for response messages.
2Reliability
If a reactive inquiry mechanism is used, then cluster failure detection is possible, but the number of message exchanges increases significantly
Solution Approach 1:
Each cluster maintains its own connectivity status information and automatically determines whether a failure has occurred by examining its own status records. This self-service approach eliminates the need for complex multi-party message exchange protocols, as each cluster independently monitors its own health state and notifies others of any changes, significantly reducing communication overhead.
3Loss of information
If conventional inquiry mechanisms are used, then the system can detect network outages, but distinguishing between network outages and cluster failures becomes complex
Solution Approach 1:
The patent segments the connectivity status information into distinct fields that separately track direct connectivity status and indirect connectivity status. This segmentation allows each cluster to independently determine whether a failure is direct (unreachable) or indirect (network outage by examining alternative paths), simplifying the distinction logic through structured data organization rather than complex query protocols.
Data Source
AI summary
A method and system for fast failure detection in a distributed computer system. The method includes executing a distributed computer system having a plurality of clusters comprising at least a first cluster, a second cluster and the third cluster, and initializing failure detection by creating a connected cluster list in each of the plurality of clusters, wherein for each one of the plurality of clusters, a respective connected cluster list describes others of the plurality of clusters said each one is communicatively connected with. A status update message is sent upon changes in connectivity between the plurality of clusters, and generating an updated connected cluster list in each of the plurality of clusters in accordance with the status update message. The method then determines whether the change in connectivity results from a cluster failure by examining the updated connected cluster list in each of the plurality of clusters.