CLAM Protocol Dynamic HA Partner Reconfiguration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in maintaining high availability when a node fails, as the HA partner node becomes a single point of failure until the failed node is operational, and existing solutions like data relocation or aggregate relocation are either infeasible due to large storage container sizes or require user intervention.
Innovation Solution
A distributed control protocol, known as the CLAM protocol, dynamically establishes and rebalances high availability partner relationships among nodes in a cluster by leveraging the Cluster Liveliness and Availability Manager (CLAM) utility to mirror NVRAM information and alter HA partner relationships, allowing seamless addition or removal of nodes without user intervention and maintaining data availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is mirrored to HA partner node over high speed connection, then data availability is guaranteed, but HA partner node becomes single point of failure when local node fails
Solution Approach 1:
The patent segments the HA relationship into multiple independent pairs within a cluster. Instead of a single HA pair where the partner becomes a SPOF, the system creates multiple HA pairs (e.g., Node1-HA1, Node2-HA2, Node3-HA3) where each node has its own dedicated HA partner. This segmentation distributes the availability burden and eliminates the SPOF condition.
Solution Approach 2:
The patent transitions from a one-dimensional single HA pair model to a multi-dimensional cluster model with multiple HA pairs. By adding the dimension of multiple independent HA relationships, the system provides alternative paths for data availability and prevents any single node from becoming a bottleneck or SPOF.
2Reliability
If data is physically relocated to another node in different HA pair, then dependency on SPOF is reduced, but operation becomes infeasible due to large storage container sizes
Solution Approach 1:
The patent uses copying (mirroring) of data containers to HA partner nodes instead of physical relocation. When a node fails, its data container is already copied to the HA partner, allowing immediate takeover without the need to physically move large amounts of data. This copying approach maintains reliability while avoiding the feasibility issues of data relocation.
3Reliability
If aggregate relocation approach is used, then physical data movement is avoided, but user intervention is required and only relocated aggregate has HA support
Solution Approach 1:
The patent implements self-service automation where the system automatically establishes HA pairs, mirrors data containers, and manages failover without requiring user intervention. The HA relationships are dynamically created and managed by the system itself, eliminating the need for users to manually identify aggregates or configure HA pairs, while ensuring all containers have HA support.
4Ease of manufacture
If HA pair configuration is determined at pre-setup phase, then initial configuration is simplified, but configuration cannot be changed after setup
Solution Approach 1:
The patent transforms the static HA pair configuration into a dynamic model where HA relationships can be automatically established, modified, and rebalanced based on cluster conditions. The system dynamically creates HA pairs when nodes are added and automatically rebalances data containers when nodes fail or are removed, providing configuration flexibility while maintaining operational simplicity.
5Reliability
If multiple nodes are added to cluster, then HA support scales, but additional node reboots are required with existing solutions
Solution Approach 1:
The patent performs preliminary actions by pre-establishing HA pairs and pre-mirring data containers before failures occur. When nodes are added to the cluster, the system proactively creates HA relationships and replicates data in advance, so that when failover is needed, the HA partner is already prepared and no reboot is required. This preliminary preparation eliminates the time loss associated with node reboots during failover events.
Data Source
AI summary
A distributed control protocol dynamically establishes high availability (HA) partner relationships for nodes in a cluster. A HA partner relationship may be established by copying (mirroring) information maintained in a non-volatile random access memory (NVRAM) of a node over a HA interconnect to the NVRAM of a partner node in the cluster. The distributed control protocol leverages a Cluster Liveliness and Availability Manager (CLAM) utility of a storage operating system executing on the nodes to rebalance NVRAM mirroring and alter HA partner relationships of the nodes in the cluster. The CLAM utility is configured to maintain various cluster related issues, such as CLAM quorum events, addition or subtraction of a node in the cluster and other changes in configuration of the cluster. Notably, the CLAM utility is an event based manager that implements the control protocol to keep the nodes informed of any cluster changes through event generation and propagation.