Stalled Storage Device Management in Clustered Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storage systems face performance degradation due to stalled storage devices, which can cause global slowdowns and unreliable access to data, especially in clustered systems where a single slow device can cascade issues across the system.
Innovation Solution
Implement a method to detect stalled storage devices, transition them to a stalled state, and redirect access requests to healthy devices, using diagnostic tests and state management modules to evaluate and manage the operational state of storage nodes, ensuring data availability and system performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a storage device is stalled or responding slowly, then data access requests to that device will experience delays, but continuing to serve requests to the stalled device can cause global slowdowns and system performance degradation
Solution Approach 1:
The patent extracts the stalled storage device from the active service pool by transitioning it to a stalled state. This removes the problematic device from the set of devices serving requests, allowing the system to continue operating with remaining healthy devices while the stalled device is isolated and managed separately.
Solution Approach 2:
The patent introduces a stalled state as an intermediary condition between normal operation and complete failure. This intermediate state allows the system to detect and respond to slow devices without treating them as completely failed devices, enabling selective request routing and gradual recovery attempts.
2Reliability
If the system continuously monitors and manages storage device states, then data access reliability improves, but system complexity increases
Solution Approach 1:
The patent applies local quality by managing different states for different storage devices individually. Each device can be in a different state (normal, stalled, failed) and is managed according to its specific condition, rather than applying uniform system-wide controls. This allows targeted response to problematic devices without affecting the entire system.
Solution Approach 2:
The patent implements dynamic state management where the system can transition devices between different states based on real-time performance monitoring. The stalled state can be transitioned to normal operation if the device recovers, or to failed state if it doesn't recover, allowing flexible adaptation to changing device conditions.
Data Source
AI summary
Embodiments relate to systems and methods for managing stalled storage devices of a storage system. In one embodiment, a method for managing access to storage devices includes determining that a first storage device, which stores a first resource, is stalled and transitioning the first storage device to a stalled state. The method also includes receiving an access request for at least a portion of the first resource while the first storage device is in the stalled state and attempting to provide access to a representation of the portion of the first resource from at least a second storage device that is not in a stalled state. In another embodiment, a method of managing access requests by a thread for a resource stored on a storage device includes initializing a thread access level for an access request by a thread for the resource. The method also includes determining whether the storage device, which has a device access level, is accessible based at least in part on the thread access level and the device access level and selecting a thread operation based at least in part on the determination of whether the storage device is accessible. The thread operation may be selected from attempting the thread access request if the device is accessible and determining whether to restart the thread access request if the device is not accessible.