Storage Controller Cluster Failover Detection via Expander Accessibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face challenges in quickly and efficiently implementing failover for storage logical volumes due to difficulties in distinguishing between split brain and controller removal scenarios, leading to potential storage downtime.
Innovation Solution
The system employs a method where each storage controller in a cluster can detect device removal events and differentiate between split brain and controller removal modes by assessing accessibility to expanders via communication links, enabling rapid failover by taking ownership of logical volumes in case of controller failure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system uses traditional failover detection methods, then the system can detect controller failures, but the failover time is too long to meet timing requirements
Solution Approach 1:
The system performs preliminary actions by having standby storage controllers continuously monitor and maintain readiness to take over logical volumes before actual failures occur. The standby controllers are pre-configured with the necessary information and connections to assume control immediately when a failure is detected, eliminating the need for time-consuming discovery and configuration processes during actual failover events.
Solution Approach 2:
The system implements feedback mechanisms where storage controllers continuously exchange status information and connectivity data with each other and with the management module. This real-time feedback allows the system to detect controller removals or failures immediately and trigger the failover process without delay, ensuring that the standby controller can take over the logical volume promptly.
2Measurement precision
If the system implements comprehensive monitoring of all controllers, then the system can distinguish between split brain and controller removal scenarios, but the system complexity increases
Solution Approach 1:
The system introduces a management module as an intermediary that centralizes the monitoring and analysis functions. Instead of each controller independently monitoring all others (which would create complex peer-to-peer monitoring), the management module receives connectivity information from controllers, analyzes the data to determine failure scenarios, and coordinates the failover process. This intermediary approach simplifies the overall system architecture while maintaining precise failure detection capabilities.
Solution Approach 2:
The management module serves multiple functions: it receives connectivity information from controllers, analyzes failure scenarios, coordinates failover processes, and maintains system state information. By consolidating these diverse functions into a single multi-functional component, the system avoids the complexity of implementing separate specialized systems for each function while achieving comprehensive monitoring and precise failure scenario differentiation.
Data Source
AI summary
Aspects of the disclosure pertain to a system and method for detecting server removal from a cluster to enable fast failover of storage (e.g., logical volumes). A method of operation of a storage controller of a cluster is disclosed. The method includes receiving a signal. The method further includes, based upon the received signal, determining that communicative connection between a second storage controller of the cluster and the first storage controller of cluster is unable to be established. The method further includes determining whether communicative connection between the first storage controller and expanders of first and second enclosure services manager modules of the cluster is able to be established. The method further includes, when it is determined that communicative connection between the first storage controller and the expanders of the first and second enclosure services manager modules of the cluster is able to be established, performing a failover process.