Distributed Failover Control Using Multi-Director Health Consensus
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional failover systems require technical expertise, are inflexible, inconvenient, and costly, often leading to redundant failovers and increased reaction times, especially in complex systems like banking or financial systems, and lack compatibility between applications.
Innovation Solution
A distributed director and probe system that monitors application availability, updates status in a data store, and triggers failover processes based on consecutive unavailability determinations by multiple director systems, allowing user-configurable failover management for multiple applications through a unified interface.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If conventional failover systems perform immediate failover upon detecting unhealthiness, then reaction time is improved, but reliability deteriorates due to redundant failovers caused by incomplete data or temporary issues
Solution Approach 1:
The system performs preliminary actions by requiring multiple consecutive health checks before triggering failover. The director system checks application health at regular intervals and only initiates failover after a predetermined number of consecutive unsuccessful checks, preventing premature failovers due to temporary issues or incomplete data.
Solution Approach 2:
The system implements feedback mechanisms where the director system continuously monitors application health status and adjusts its decisions based on repeated observations. The probe system provides feedback about application availability, and the director system uses this feedback to determine when failover is truly necessary, avoiding redundant failovers.
2Reliability
If conventional failover systems require human operator intervention, then reliability is improved by avoiding premature failovers, but reaction time deteriorates dramatically
Solution Approach 1:
The system enables self-service failover management where the director system automatically monitors application health, makes decisions about failover based on consecutive health check results, and executes failover without requiring human operator intervention. This maintains reliability through automated decision-making while preserving fast reaction time.
Solution Approach 2:
The system changes the parameters of failover decision-making by introducing a threshold-based approach (number of consecutive unsuccessful checks) that automatically determines when failover is necessary. This parameter-based decision logic eliminates the need for human operators while maintaining reliable failover decisions and fast response times.
3Adaptability or versatility
If conventional failover systems are configured individually for each application, then adaptability is improved for specific application needs, but device complexity and cost increase significantly
Solution Approach 1:
The system achieves universality by creating a single failover management platform that can serve multiple applications simultaneously. The director system is designed to monitor and manage failover for various applications through a unified interface, eliminating the need for separate failover configurations for each application while maintaining application-specific adaptability.
Solution Approach 2:
The system applies segmentation by dividing the failover management function into independent components: probe systems that monitor individual applications, a central director system that makes decisions, and a failover system that executes actions. This modular segmentation allows the system to handle multiple applications with different requirements without increasing overall complexity.
4Ease of operation
If systems use simple failover mechanisms, then ease of operation is improved, but reliability deteriorates for complex systems like banking or financial systems requiring high availability
Solution Approach 1:
The system introduces dynamics by implementing adaptive failover logic that adjusts its behavior based on the number of consecutive health check failures. The director system dynamically decides when to trigger failover based on the pattern of health status observations, providing sophisticated reliability for complex systems while maintaining ease of operation through automated decision-making.
Solution Approach 2:
The system ensures continuity of useful action by continuously monitoring application health through repeated health checks at regular intervals. This continuous monitoring enables the system to distinguish between temporary issues and genuine failures, maintaining high availability for complex systems while keeping the operation simple through automated continuous observation.
Data Source
AI summary
Systems and methods for application management are disclosed. The system may include a probe system and one or more director systems, each comprising at least one memory and one or more processors configured to execute instructions. The instructions may include monitoring an availability of an application; updating a status associated with the availability of the application in a first data store; polling the first data store in intervals to retrieve the status associated with the availability of the application; upon retrieving at least a consecutive predetermined number of statuses associated with the application being unavailable, determining the application is unavailable; determining whether at least one other director system of the one or more director systems has determined the application is unavailable; and upon determining the at least one other director system of the one or more director systems has determined the application is unavailable, triggering a failover process.