Distributed Storage IO Resiliency via Path Remapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face challenges in ensuring the resiliency of IO operations during network interruptions, where failed operations may be delayed and overwrite subsequent successful operations, leading to data inconsistencies.
Innovation Solution
The method involves resenting failed IO operations on a different path and initiating a storage volume remapping upon completion, updating the generation number, and discarding older operations to ensure data integrity and prevent overwrites.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If failed IO operations are resent on a different network path, then network resiliency is improved, but data consistency deteriorates due to delayed operations overwriting successful ones
Solution Approach 1:
The system performs preliminary actions by sending a notification to terminate delayed IO operations before they can overwrite successful ones. This advance termination prevents data inconsistency while maintaining the ability to resend failed operations on alternative paths.
Solution Approach 2:
A notification mechanism acts as an intermediary between the resending process and the storage system. This intermediary communicates the need to terminate delayed operations, coordinating between multiple paths and preventing harmful overwrites without requiring complex system-wide changes.
2Reliability
If multiple paths are used for IO operations, then network resiliency is improved, but system complexity increases due to path management and remapping
Solution Approach 1:
The system changes parameters by updating generation numbers associated with storage volumes and IO operations. This parameter-based approach simplifies multi-path management by providing a straightforward mechanism to identify and terminate delayed operations without complex path tracking or state management.
Solution Approach 2:
The notification mechanism serves as an intermediary that abstracts the complexity of multi-path management. By communicating termination requests through this intermediary, the system avoids the need for complex path selection, tracking, and coordination logic in the storage system itself.
3Stability of the object's composition
If delayed IO operations are terminated, then data consistency is improved, but loss of time occurs due to discarding operations
Solution Approach 1:
The system discards delayed IO operations that pose a risk of overwriting successful ones, and recovers by ensuring data integrity through the notification mechanism. This selective discarding prevents data inconsistency while the generation number tracking ensures no valid data is lost, as successful operations are already committed.
Solution Approach 2:
The potential harm of delayed operations overwriting successful ones is converted into a benefit by using the termination notification as a coordination mechanism. This approach transforms a problematic side effect into a useful signal for maintaining data consistency across multiple network paths.
Data Source
AI summary
Techniques are provided for increasing resiliency of IO operations to network interruptions. One method comprises, in response to a failure of a given IO operation on a first path between at least one initiator of a host device and at least one storage target of a storage volume of a distributed storage system, resending the given IO operation on a second path between the at least one initiator and the at least one storage target; and, in response to a completion of the given IO operation on a given one of the first path and the second path, initiating a remapping of the storage volume. The remapping of the storage volume may comprise unmapping the storage volume and mapping the storage volume. One or more IO operations having an older generation number than the generation number of the storage volume may be discarded.