System and method for establishing bi-directional failover in a two node cluster

Abstract

A system and method for permitting bi-directional failover in two node clusters utilizing quorum-based data replication. In response to detecting an error in its partner the surviving node establishes itself as the primary of the cluster and sets a first persistent state in its local unit. A temporary epsilon value for quorum voting purposes is then assigned to the surviving node, which causes it to be in quorum. A second persistent state is stored in the local unit and the surviving node comes online as a result of being in quorum.

Description

FIELD OF THE INVENTION[0001]The present invention relates to clustered systems and, in particular to establishing bi-directional failover in a two node clustered system.BACKGROUND OF THE INVENTION[0002]A storage system is a computer that provides storage service relating to the organization of information on storage devices, such as disks. The storage system may be deployed within a network attached storage (NAS) environment and, as such, may be embodied as a file server. The file server or filer includes a storage operating system that implements a file system to logically organize the information as a hierarchical structure of directories and files on the disks. Each “on-disk” file may be implemented as a set of data structures, e.g., disk blocks, configured to store information. A directory, on the other hand, may be implemented as a specially formatted file in which information about other files and directories are stored.[0003]A filer may be further configured to operate accord...

AI Technical Summary

Benefits of technology

[0017]If the node is primary, and if the other node does not hold the HA-activity lock at the D-blade, then the node acquires the HA-activity lock by marking an entry in the RDBHASP data structure designating itself as the HA-active node in the cluster; this involves recording its node (site) ID in RDBHASP. It then persistently stores the HA_PREACTIVE state in its local unit, in effect caching a local indication of the locked ability to be active (i.e., the HA-activity lock). By caching this persistent state information locally, the node may continue in HA active mode even if it or the D-blade suffers an error condition and/or is reinitialized, or if the primary moves from one D-blade to another. Note that this process is repeated independently for each of the data replication rings.

[0018]Once it has transitioned to the HA_PREACTIVE state, the surviving node disables the quorum voting mechanism, and waits an appropriate amount of time to ensure that quorum voting has ceased. The surviving node then assigns a temporary epsilon value to itself, which causes the node to enter the quorum via the (1+e)/2 requirement. The temporary epsilon is an implementation trick that allows easy reuse of all the standard configuration-epsilon based quorum design and code. Once the temporary epsilon value has been set, the node transitions to a HA_ACTIVE state and then comes online as t

Problems solved by technology

A partition occurs when connectivity is lost to one set of nodes as a result of a power failure or other failures to the cluster.

In this case, it will not be able to form quorum, and will not be able to perform updates.

A noted disadvantage of such quorum-based data replication systems is the inability to meet quorum requirements, thereby preventing any update (write) operations from occurring.

This can occur as a result of communication failures, problems with health of individual node(s), or a combination of these problems.

In clusters that utilize these services, the inability to form quorum may prevent an administrator from modifying the manageme

Images