Dynamic Read Stream Scaling for Backup Restore Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Backup software on data storage platforms like DellEMC DataDomain struggles to support lower RTO and better SLA for large-scale simultaneous restores due to resource constraints, such as limited CPU and memory, which restricts the number of parallel read streams and slows down data integrity scanning.
Innovation Solution
A method to dynamically adjust the number of read streams by spawning on-demand compute instances associated with lightweight data movers, based on available resources and SLA requirements, to support restore operations and maximize restore throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of parallel read streams is increased to support large-scale simultaneous restores, then restore throughput and data recovery speed are improved, but resource constraints (CPU and memory) are exceeded
Solution Approach 1:
The system dynamically adjusts the number of read streams based on available resources and restore priorities. Instead of using a fixed number of read streams, the system monitors resource availability and automatically scales the number of parallel read streams up or down to match current workload demands and resource capacity, thereby maximizing restore throughput without consistently exceeding resource limits
Solution Approach 2:
The system changes operational parameters (number of read streams) based on changing conditions. By monitoring resource utilization metrics such as CPU and memory availability, the system adjusts the read stream parameter in real-time to optimize restore performance while preventing resource exhaustion
2Loss of time
If more read streams are allocated to meet SLA requirements, then data recovery time is reduced, but the complexity of resource management increases
Solution Approach 1:
The system implements self-service resource management through automated monitoring and adjustment mechanisms. The resource management module continuously monitors system state and automatically adjusts read stream allocation without requiring manual intervention, thereby reducing data recovery time while managing complexity through automation rather than manual processes
3Productivity
If the number of compute instances is increased to handle more read requests, then restore operation performance is improved, but resource consumption increases
Solution Approach 1:
The system dynamically provisions compute instances based on actual restore workload and resource availability. Instead of maintaining a large fixed pool of compute instances, the system scales compute resources up during high-demand restore operations and scales down during lower-demand periods, thereby improving restore operation performance when needed while reducing compute resource consumption during normal operations
Data Source
AI summary
One example method includes receiving read requests from a backup application, determining whether or not adequate compute instances are available to service the read requests, when adequate compute instances are not available to service the read requests, spawning additional compute instances so that the read requests can be serviced, servicing the read requests using the additional compute instances, and after the read requests have been serviced, spinning down any compute instances that are no longer needed.