Dynamic Resource Allocation for Data Copy RPO Compliance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face challenges in efficiently meeting Recovery Point Objective (RPO) goals due to unexpected large amounts of data pending to be copied between storage systems, often resulting in data loss during outages or failures, as conventional systems fail to reflect short-term utilization changes and resource allocations.
Innovation Solution
A computing device determines the amount of pending data and ingest speed, estimates the time to reach the RPO goal by considering copy speed and resource allocation, and takes actions such as recommending or automatically deploying additional resources to meet the RPO goal, potentially using cloud resources or repurposing existing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If more resources are assigned to data copying, then the RPO goal is more likely to be achieved, but the cost and resource consumption increase
Solution Approach 1:
The system dynamically adjusts resource allocation based on real-time conditions. The management computing device monitors pending data amounts, copy speeds, and ingest speeds, then automatically modifies resource assignments to meet RPO goals without over-provisioning. This dynamic adaptation allows the system to allocate more resources when needed (e.g., during outages) and fewer resources during normal operation.
Solution Approach 2:
The system implements continuous feedback loops where the management computing device monitors actual copy performance against RPO requirements. Based on this feedback, the system adjusts resource allocation decisions. The feedback mechanism ensures that resource assignments are data-driven and responsive to actual system conditions rather than static or overly conservative allocations.
2Quantity of substance
If fewer resources are assigned to data copying, then the cost is reduced, but the RPO goal may not be achieved due to slow copy speed
Solution Approach 1:
The system performs preliminary assessment of pending data amounts, copy speeds, and ingest speeds before making resource allocation decisions. By evaluating these parameters in advance, the management computing device can determine the minimum resources needed to meet RPO goals, avoiding both over-provisioning and under-provisioning. This preliminary analysis enables optimized resource assignment that is just sufficient for RPO compliance.
3Device complexity
If resources are allocated based on long-term utilization statistics, then planning is simplified, but short-term changes and outages are not reflected
Solution Approach 1:
The system transitions from static, long-term-based resource planning to dynamic, real-time resource allocation. The management computing device continuously monitors current conditions including pending data amounts, copy speeds, and ingest speeds, and adjusts resource assignments accordingly. This dynamic approach allows the system to respond to short-term changes, outages, and workload spikes without requiring complex long-term planning adjustments.
Data Source
AI summary
In some examples, a computing device may determine an amount of pending data to copy over a network from a first storage system to a second storage system. Further, the computing device may determine an ingest speed based on a quantity of data received by the first storage system and a copy speed associated with one or more first computing resources associated with the first storage system. The computing device may determine an estimated time to copy at least a portion of the pending data to the second storage system to meet a data copy requirement. For instance, the estimated time may be based at least in part on the copy speed, the amount of pending data, and the ingest speed. In addition, at least one action may be performed based on the estimated time.