Replication Link Smoothing via Historical Data Rate Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The amount of data to be transmitted during each replication cycle in content-based storage systems is unknown, leading to inefficient use of system resources like network bandwidth and processing cycles, as existing systems lack a method to effectively manage data transmission within a user-defined recovery point objective (RPO) period.
Innovation Solution
A method that determines the number of chunks written during a replication cycle, estimates the host write rate, and adjusts the transmission rate based on the recovery point objective (RPO) period to throttle data transmissions, ensuring data is transmitted within the defined window, thereby improving system performance by smoothing network and processing loads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data transmission is performed at maximum speed during replication cycles, then replication speed is improved, but system resource consumption (network bandwidth and processing cycles) increases and may interfere with other processes
Solution Approach 1:
The patent implements dynamic rate adjustment by continuously monitoring the actual data generation rate during each replication cycle and adjusting the transmission rate accordingly. The system transitions from static maximum-speed transmission to dynamic adaptive transmission, where the replication speed is modulated in real-time based on actual data changes, thereby improving resource efficiency while maintaining replication effectiveness.
Solution Approach 2:
The system changes the transmission rate parameter based on monitored data generation rates. By calculating the actual amount of data that changed during the replication cycle and adjusting the transmission rate parameter dynamically, the system optimizes the balance between replication speed and resource consumption, preventing excessive resource usage while ensuring timely replication.
2Use of energy by moving object
If data transmission is throttled to reduce resource consumption, then system performance is improved, but replication may not complete within the user-defined RPO period
Solution Approach 1:
The patent implements a feedback mechanism where the actual data generation rate is measured during each replication cycle, and this measured rate is fed back to control the transmission rate. This closed-loop control ensures that the transmission rate adapts to actual data changes, preventing both excessive resource consumption and premature replication completion, thereby ensuring replication finishes within the RPO period while optimizing resource usage.
Solution Approach 2:
The system performs preliminary monitoring of data generation rates during the replication cycle before finalizing the transmission rate. By提前 (in advance) assessing the actual data changes and preparing the appropriate transmission rate based on this assessment, the system ensures that replication can complete within the RPO period while avoiding excessive resource consumption during transmission.
3Reliability
If replication transmission rate is increased to meet RPO requirements, then replication reliability is improved, but network bandwidth and processing resources are overwhelmed
Solution Approach 1:
The system dynamically adjusts the transmission rate based on actual data generation rates monitored during each replication cycle. This dynamic adjustment ensures that the transmission rate is increased only when necessary to meet RPO requirements, rather than consistently operating at maximum speed, thereby maintaining replication reliability while preventing network bandwidth and processing resource exhaustion.
4Reliability
If unknown amount of data is transmitted during each replication cycle, then data completeness is ensured, but transmission efficiency decreases due to lack of rate management
Solution Approach 1:
The patent implements feedback-based rate management where the actual amount of data transmitted is monitored and compared against the data generation rate. This feedback mechanism enables efficient rate management by adjusting transmission based on actual data changes, ensuring data completeness while improving transmission efficiency through informed rate control rather than blind maximum-speed transmission.
Data Source
AI summary
A computer program product, system, and method for determining a number of chunks written to a storage volume during a first replication cycle; determining a host write rate based on the number of chunks written to the storage volume during the first replication cycle; estimating a number of chunks written during a second replication cycle; determining a target rate based on the estimated number of chunks written during the second replication cycle; and transmitting chunks written during the second replication cycle at the target rate.