Cluster Node Failover via SCSI Persistent Reservations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cluster computing systems face challenges in ensuring application availability and reliability due to node or network failures, often resulting in unnecessary failovers and loss of compute resources, especially in scenarios where majority nodes become unavailable.
Innovation Solution
A cluster computer system that determines node unavailability and forms a sub-cluster to retain ownership of applications, using SCSI 3 Persistent Reservations to control access to storage volumes, allowing the system to maintain application availability without external arbitration and prevent unnecessary failovers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If external arbitration resources are used to determine node availability and manage failover, then application availability can be maintained during node failures, but system complexity increases and external resources are required
Solution Approach 1:
The patent uses SCSI Persistent Reservation as an intermediary mechanism to manage storage access rights between nodes. Instead of requiring external arbitration resources, the system uses the storage device's built-in reservation capability to automatically mediate access conflicts, thereby maintaining application availability while reducing system complexity
Solution Approach 2:
The system enables nodes to self-manage failover operations using local information and SCSI Persistent Reservation protocols. Each node can independently determine its status and initiate failover without requiring external arbitration, making the system self-sufficient and reducing complexity
2Reliability
If majority nodes are brought down during network splits, then split-brain scenarios are prevented, but compute resources are lost and applications may not run even when nodes are available
Solution Approach 1:
The patent implements dynamic node eligibility determination based on real-time cluster state and SCSI reservation status. Instead of statically bringing down majority nodes during network splits, the system dynamically adjusts which nodes can access storage and run applications, allowing compute resources to remain productive while maintaining consistency through the reservation mechanism
Solution Approach 2:
The system changes the parameter of node eligibility from a static majority-vote requirement to a dynamic state based on SCSI Persistent Reservation status. This allows nodes to remain active and productive even during network splits, as long as they can successfully acquire or maintain storage reservations, thereby maintaining both consistency and productivity
3Reliability
If nodes are marked as ineligible or fenced during failures, then application availability is maintained, but unnecessary failovers occur and compute resources are lost
Solution Approach 1:
The patent establishes SCSI Persistent Reservations in advance on storage devices before failures occur. These pre-established reservations serve as predetermined access rights that automatically validate node eligibility during failures, eliminating the need for reactive node fencing or marking as ineligible, thereby preventing unnecessary failovers and preserving compute resources
4Reliability
If external resources are used for node fencing and arbitration, then failover can be managed, but the system depends on external resources and complexity increases
Solution Approach 1:
The patent uses the storage device's SCSI Persistent Reservation capability as an intermediary that provides failover management functions without requiring external resources. The storage device itself mediates access conflicts between nodes, enabling the system to manage failovers independently while maintaining reliability
Solution Approach 2:
The system uses its own storage infrastructure to provide arbitration and fencing functions through SCSI Persistent Reservations. Nodes can independently determine their eligibility to access storage and run applications without external resources, making the system self-sufficient and adaptable to various deployment environments
Data Source
AI summary
Some examples described herein relate to a cluster computer system. In an example, a first node in a cluster computer system may determine that a second node has become unavailable. The first node may form a first sub-cluster with a plurality of nodes that are communicatively reachable in the cluster computer system. The first node may retain ownership over a first application running on the first node, to the exclusion of remaining nodes in the first sub-cluster, wherein retaining ownership over the first application comprises controlling access to a storage volume related to the first application. The first node may attempt to obtain ownership over a second application previously running on the second node before the second node became unavailable. In response to a successful attempt the first node may obtain ownership over the second application, to the exclusion of remaining nodes in the first sub-cluster.