Remote Copy Lag Monitoring for RPO-Based Transfer Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in managing remote copy operations, particularly in asynchronous remote copy scenarios, where data transfer delays and bandwidth limitations can lead to data gaps and reduced availability, especially in wide-area disasters, resulting in potential service contract violations and increased costs.
Innovation Solution
A method for calculating lag time in remote copy operations by periodically inserting time markers in the journal, setting policies based on lag time limits, and controlling copy transfer rates to manage bandwidth and prioritize data copying based on recovery point objectives (RPO) to prevent data loss and ensure compliance with service level agreements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If asynchronous remote copy is used for wide-area disaster protection, then disaster resilience is improved, but data transfer speed decreases due to longer communication delays
Solution Approach 1:
The system dynamically adjusts the copy transfer rate based on real-time lag time measurements. When lag time exceeds the RPO threshold, the system automatically reduces the transfer rate to prevent data gaps, and can resume normal operation when lag time is within acceptable limits. This dynamic adjustment resolves the contradiction by adapting transfer speed to actual network conditions and disaster protection requirements.
Solution Approach 2:
The system changes the transfer rate parameter based on lag time conditions and RPO policies. By monitoring lag time and comparing it against configured RPO thresholds, the system adjusts the copy transfer rate parameter to maintain data consistency while adapting to wide-area network conditions, thus achieving both disaster resilience and acceptable transfer performance.
2Reliability
If copy transfer rate is increased to reduce lag time, then data consistency is improved, but network bandwidth consumption increases
Solution Approach 1:
The system dynamically adjusts the copy transfer rate based on real-time lag time measurements. When lag time is within acceptable RPO limits, the system maintains a balanced transfer rate that conserves bandwidth. When lag time exceeds thresholds, the system increases the transfer rate only to the extent necessary to restore data consistency, thus optimizing bandwidth usage while maintaining data consistency.
Solution Approach 2:
The system continuously monitors lag time and uses this feedback to adjust the copy transfer rate. By establishing a feedback loop that measures actual data consistency (lag time) and adjusts transfer parameters accordingly, the system achieves data consistency only when necessary, minimizing unnecessary bandwidth consumption while maintaining reliability.
3Reliability
If ample storage network resources are allocated to prevent data gaps, then data availability is improved, but system cost increases significantly
Solution Approach 1:
The system dynamically adjusts resource allocation through automated lag time monitoring and transfer rate control. Instead of allocating ample resources continuously, the system only increases resource usage (transfer rate) when lag time exceeds RPO thresholds, and reduces resource usage when data consistency is maintained. This dynamic resource management achieves data availability while minimizing system costs.
Solution Approach 2:
The system performs self-monitoring and self-adjustment of transfer rates based on lag time conditions. By automatically detecting data gaps and adjusting copy operations without external intervention, the system maintains data availability through intelligent resource management rather than requiring permanently over-provisioned resources, thus reducing system costs.
4Reliability
If strict system design with ample resources is used to secure transfer bandwidth, then service level agreement compliance is improved, but implementation cost increases
Solution Approach 1:
The system dynamically adjusts copy transfer rates based on real-time lag time measurements against RPO thresholds. Instead of implementing strict design with permanently secured bandwidth, the system adaptively manages transfer rates to meet RPO compliance only when necessary, reducing implementation costs while maintaining SLA compliance through intelligent, condition-based resource allocation.
Data Source
AI summary
Systems and methods can involve, for an execution of a remote copy operation from a primary storage system to a secondary storage system, calculating lag time from a current time and latest copy time received during execution of the remote copy operation. The calculation of the lag time for the execution of the remote copy operation can be conducted by either the primary storage system or the secondary storage system.