Distributed Failover in Shared Storage Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional storage area network (SAN) and network-attached storage (NAS) architectures are not designed to handle large multi-petabyte capacity levels efficiently, leading to challenges in data storage and protection, particularly in ensuring seamless failover in case of node failures.
Innovation Solution
Implementing a distributed failover system in shared storage clusters where each data node has a backup node assigned across multiple nodes, allowing for decentralized and rapid failover operations, with a cluster controller managing active and backup node assignments and using heartbeat protocols to monitor node health and initiate failover.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional SAN/NAS architectures are used, then data storage capacity is limited, but system complexity and cost increase when scaling to multi-petabyte levels
Solution Approach 1:
The storage system is divided into multiple independent data node devices, each capable of autonomous operation and hosting multiple NAS servers. This segmentation allows the system to scale capacity by simply adding more nodes without increasing overall system complexity, as each node operates independently with its own resources and backup relationships.
2Reliability
If centralized failover management is implemented, then coordination is simplified, but failover speed and system availability decrease
Solution Approach 1:
Each data node device autonomously monitors its own health status and independently initiates failover operations when failures are detected. The system implements self-service failover where nodes automatically detect failures through heartbeat monitoring, select appropriate backup nodes, and transfer services without requiring centralized coordination, thereby achieving both high reliability and fast failover speed.
3Ease of operation
If backup nodes are concentrated on single devices, then failover coordination is simplified, but the burden on individual nodes increases and single points of failure are created
Solution Approach 1:
Backup responsibilities are segmented and distributed across multiple data node devices rather than concentrated on single nodes. Each node serves as a backup for specific NAS servers on other nodes, creating a distributed backup architecture that eliminates single points of failure and balances the failover burden across the entire system.
Solution Approach 2:
Each data node device is designed with multi-functionality, serving both as an active node hosting NAS servers and as a backup node for other nodes. This universal design allows any node to assume backup responsibilities, distributing the failover burden evenly across all nodes and preventing any single node from becoming a bottleneck or single point of failure.
4Productivity
If rapid failover is implemented, then service continuity is improved, but data integrity and system stability may be compromised
Solution Approach 1:
The system performs preliminary actions by pre-configuring backup node relationships and maintaining standby capabilities before failures occur. When failures are detected through heartbeat monitoring, the system can rapidly activate pre-established backup relationships, achieving fast failover while maintaining data integrity because the backup nodes are already prepared and synchronized with the failed nodes.
Data Source
AI summary
Described herein is a system that provides data service failover in shared storage clusters. An example shared storage cluster can include multiple data node devices equipped to access to a shared storage location. Each of the data node devices can host multiple network attached storage (NAS) servers. For each NAS server at a data node device, a different data node device in the shared storage cluster can be designated as a backup node. The designated backup nodes for NAS servers at any one data node device in the shared storage cluster can be distributed across multiple other data node devices in the shared storage cluster, thereby sharing the burden of data node device failure across multiple other data node devices.