Master Node Failover via Centralized Priority Database
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently resolving master node failures within node clusters, as current fault tolerance mechanisms consume significant resources and may not accurately determine a new master node without polling and reconciling recommendations from individual nodes, impacting service delivery.
Innovation Solution
A centralized database is used to store status information about nodes, allowing for efficient identification and designation of a new master node based on priority order, reducing resource consumption and improving failover efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a centralized database is used to store status information and determine new master nodes, then resource consumption is reduced and failover efficiency is improved, but the system requires additional infrastructure (centralized database) and complexity in managing node registration and status updates
Solution Approach 1:
A centralized database is introduced as an intermediary component to manage node status information and master node determination. The database acts as a mediator between nodes, storing registration information and priority orders, thereby simplifying the failover process by providing a single source of truth for master node selection without requiring complex peer-to-peer communication protocols
Solution Approach 2:
Nodes register themselves with the centralized database in advance, storing their status information and priority orders before any failure occurs. This preliminary action ensures that when a master node fails, the system can immediately query the database to determine the next appropriate master node based on pre-established priority orders, eliminating the need for time-consuming discovery protocols
2Reliability
If polling and reconciliation from individual nodes is used to determine a new master node, then the system can accurately identify a suitable replacement, but significant system resources are consumed during the failure resolution process
Solution Approach 1:
The determination logic for selecting a new master node is extracted from the individual nodes and centralized in the database. Instead of nodes polling each other and reconciling recommendations, the system extracts the authority for master node selection to the centralized database, which stores pre-computed priority orders and can immediately determine the appropriate successor without resource-intensive communication loops
Solution Approach 2:
The centralized database maintains up-to-date status information about all nodes and their priorities, enabling it to autonomously determine the next appropriate master node when a failure occurs. This self-service capability eliminates the need for other nodes to poll and reconcile, as the database independently possesses all necessary information to make the determination
Data Source
AI summary
Fault tolerance techniques for a plurality of nodes executing application thread groups include executing at least a portion of a first application thread group based on a delegation by a first node, wherein the first node delegates an execution of the first application thread group amongst the plurality of nodes and has a highest priority indicated by an ordered priority of the plurality of nodes. A failure of the first node can be identified based on the first node failing to respond to a message sent to it. A second node can then be identified as having a next highest priority indicated by the ordered priority such that the second node can delegate an execution of a second application thread group amongst the plurality of nodes.