Slice-Level Quorum Handling for Distributed Storage Writes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, write operations are not completed until a quorum of replicas is available, leading to inefficiencies and potential data loss if the quorum is not met, while read operations can continue unaffected.
Innovation Solution
A distributed storage system that suppresses acknowledging write command completion until a quorum of replicas is available, allowing read operations to proceed independently, with retry mechanisms and exponential backoff for write commands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If write operations wait for quorum of replicas to be available, then data reliability is improved, but operation time and productivity deteriorate
Solution Approach 1:
The storage volume is divided into slices, and each slice is further divided into replicas distributed across multiple storage devices. This segmentation allows the system to independently manage quorum requirements for each slice, enabling selective availability and independent operation of read/write commands for different slices without blocking the entire volume.
Solution Approach 2:
The system dynamically adjusts quorum requirements based on the operational state of replicas. When a quorum is not available for a particular slice, the system can still process read commands for other slices or retry write commands for the affected slice after a delay, making the system adaptable to varying availability conditions.
2Reliability
If write operations block until quorum is available, then data consistency is improved, but productivity deteriorates
Solution Approach 1:
By segmenting the storage volume into slices with independent quorum requirements, the system can process read operations on unaffected slices while retrying write operations on slices with unavailable quorums, thereby maintaining overall productivity while ensuring consistency where possible.
Solution Approach 2:
The system continues to process read commands for slices where quorum is available, and retries write commands for slices where quorum becomes available. This continuous processing approach maintains productivity by keeping operations flowing rather than blocking entirely when partial availability exists.
3Reliability
If quorum verification is performed for all slices, then data robustness is improved, but system complexity increases
Solution Approach 1:
The quorum verification process is segmented by slice, allowing the system to independently determine quorum availability for each slice rather than treating the entire volume as a single unit. This reduces overall complexity by enabling parallel and independent verification across multiple slices.
Solution Approach 2:
Each slice has its own quorum requirements and availability status, allowing the system to apply different quorum verification strategies to different slices based on their specific needs and availability conditions, rather than applying a uniform complex verification process to the entire volume.
Data Source
AI summary
A distributed storage system executes write commands with respect to a storage volume mounted to an operating context. The storage volume is composed of a plurality of slices, each of which includes a plurality replicas. If a quorum of replicas are not available to execute a write command referencing a slice, acknowledgment of the write command is suppressed. However, read commands continue to be executed with respect to the slice and other slices of the storage volume. Likewise, write commands for other slices also continue to be executed. A storage manager maintains a state of each replica of each slice and manages restarting or reallocation of replicas that become unavailable.