Block Storage Durability State Reduction for Network Partitions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Block-based storage systems face performance degradation and latency issues due to increased I/O demands, which can lead to a loss of virtualized computing performance when the capacity to service requests is exceeded, and existing solutions for managing redundancy and replication are disrupted by network partitions or system failures.
Innovation Solution
Implementing a method to reduce the durability state of data volumes in block-based storage systems by allowing operations to continue with reduced replication, and efficiently replicating data by marking and updating only modified chunks, enabling replication to resume when network conditions improve.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If replication is maintained at full durability state during network disruptions, then data durability is preserved, but system performance degrades and operations are blocked
Solution Approach 1:
The patent applies dynamics by making the durability state adjustable rather than fixed. The system can dynamically transition between full durability state (normal operation) and reduced durability state (during disruptions), allowing replication to be temporarily lowered when network conditions deteriorate, thereby maintaining performance while preserving durability when needed
2Reliability
If complete data replication is performed during recovery, then durability state is restored, but network traffic increases and recovery time extends
Solution Approach 1:
The patent segments data into chunks and tracks which chunks have been replicated and which need replication. During recovery, instead of replicating the entire data volume, the system only replicates the modified chunks that changed during reduced durability state, significantly reducing network traffic and recovery time while still achieving full durability state restoration
Solution Approach 2:
The system performs preliminary tracking of modified chunks during reduced durability state operations. By maintaining metadata about which data chunks were modified, the system prepares the information needed for efficient selective replication during recovery, avoiding the need to re-transmit unchanged data
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A block-based storage system may implement reducing durability state for a data volume. A determination may be made that storage node replicating write requests for a data volume is unavailable. In response, subsequent write requests may be processed according to a reduced durability state for the data volume such that replication for the data volume may be disabled for the storage node. Write requests may then be completed at a fewer number of storage nodes prior to acknowledging the write request as complete. Durability state for the data volume may be increase in various embodiments. A storage node may be identified and replication operations may be performed to synchronize the current data volume at the storage node with a replica of the data volume maintained at the identified storage node.