Storage Failover for Co-located Compute and Storage Owners
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In high availability and fault-tolerant application environments, the separation of compute and storage owners across sites during failover events leads to increased latency and decreased throughput, resulting in degraded performance.
Innovation Solution
Monitoring input/output (I/O) and Persistent Reservation (PR) activity patterns to detect changes indicative of compute and storage owners being located at different sites, and automatically triggering a storage failover to ensure co-location for improved performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If compute and storage owners are separated across different sites during failover, then fault tolerance is improved, but latency increases and throughput decreases
Solution Approach 1:
The system continuously monitors I/O activity patterns and Persistent Reservation changes to detect when compute and storage owners are separated across sites. When degradation is detected, the system automatically triggers failover to restore co-location, creating a closed-loop feedback mechanism that maintains optimal performance while ensuring fault tolerance.
Solution Approach 2:
The system proactively monitors for patterns indicating impending separation of compute and storage owners before full failover occurs. By detecting early signs of degradation through I/O pattern analysis and PR changes, the system can initiate corrective failover actions to restore co-location and prevent sustained performance degradation.
2Speed
If compute and storage owners are co-located at the same site, then I/O performance is improved, but fault tolerance capability is reduced
Solution Approach 1:
The system dynamically adjusts the co-location state of compute and storage owners based on real-time monitoring. Through continuous observation of I/O patterns and Persistent Reservation changes, the system can transition between co-located and separated states as needed, optimizing performance when possible while ensuring fault tolerance when required.
Solution Approach 2:
The system changes the spatial parameter relationship between compute and storage owners based on operational conditions. By monitoring I/O activity and PR state changes, the system can alter the co-location status to match current workload requirements and failure risks, balancing performance and resilience needs.
3Speed
If automatic failover is implemented to restore co-location, then performance degradation is reduced, but system complexity increases
Solution Approach 1:
The system performs self-diagnosis and self-correction by automatically monitoring its own I/O patterns and Persistent Reservation state to detect compute-storage separation. The failover mechanism is self-triggered based on monitored conditions, eliminating the need for external intervention or complex manual control systems while maintaining performance optimization.
Solution Approach 2:
The monitoring system provides continuous feedback on I/O activity patterns and PR changes, enabling automatic detection of performance degradation conditions. This feedback loop triggers appropriate failover actions without requiring complex external control systems, simplifying overall architecture while maintaining intelligent autonomous operation.
Data Source
AI summary
A technique monitors input/output (I/O) and storage ownership takeover activity patterns to detect degraded performance of a highly available and fault tolerant application executing in a multi-site environment. Multiple instances of the application execute in different containers or pods running on virtual machines (VMs) of a compute layer within a containerized (e.g., Kubernetes) clustering configuration that extends across clusters of the sites. A storage layer of the clusters provides shared storage to the pods running on the VMs across the multiple sites. One of the sites is configured as an active storage site configured to receive and service I/O requests from the compute layer. A single instance of the application is active at a time and configured as a “compute owner” of the shared storage to issue the I/O requests to the shared storage. The compute owner and active storage site may not be co-located on the same site, leading to excessive I/O and storage ownership takeover activity patterns indicative of degraded performance. Upon detecting such patterns, the technique automatically triggers a storage failover to ensure that compute owner and active storage site are co-located at the same site.