Storage Snap Set Scheduling for I/O Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Synchronous replication in storage systems causes I/O latency hiccups due to the need to suspend and drain I/O operations for snapshot creation, and existing time-based snapshot policies do not adapt to workload or provide recovery time guarantees.
Innovation Solution
Implementing a method that sets minimum and maximum snap set creation intervals and a recovery time threshold, monitoring data changes and throughput statistics to dynamically schedule snap set creation and replication, ensuring optimal recovery snap sets are created without excessive system load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If snap sets are created frequently to ensure data recovery, then recovery time is reduced, but I/O latency increases due to suspending and draining I/O operations
Solution Approach 1:
The snap set creation policy dynamically adjusts the snap set creation interval based on monitored workload characteristics and data change rates. The system transitions from static time-based intervals to adaptive intervals that respond to actual system conditions, creating snap sets more frequently when data changes rapidly and less frequently when the system is idle, thereby optimizing both recovery time and I/O latency
Solution Approach 2:
The system changes the parameter of snap set creation interval from a fixed value to a dynamically adjusted value based on monitored conditions. By monitoring data change rates, throughput statistics, and workload characteristics, the system adjusts the snap set creation interval parameter in real-time to balance recovery requirements with I/O performance
2Device complexity
If snap sets are created at fixed time intervals, then scheduling is simple, but the policy does not adapt to workload variations or provide recovery time guarantees
Solution Approach 1:
The system implements feedback mechanisms by monitoring data change rates, throughput statistics, and workload characteristics between snap set creations. This feedback is used to dynamically adjust the snap set creation interval, allowing the system to adapt to varying workload conditions while maintaining simple policy management through automated adjustments
Data Source
AI summary
In one aspect, data replication processes include setting a minimum snap set creation interval, a maximum snap set creation interval, and a recovery time threshold (RTT). An aspect further includes creating snap sets during a synchronous replication process based on the snap set creation intervals and the RTT. The snap set creation further includes monitoring an amount of data changes since the last snap set creation, monitoring throughput statistics between a source and target system, and estimating an amount of time to replicate the data changes to the target system based on the data changes and throughput statistics. Upon determining the time to replicate the data changes reaches the RTT and the time since the last snap set creation passes the minimum interval, the snap set creation includes creating a next snap set, storing the next snap set at the source, and replicating the next snap set at the target.