Backup Session Reservation for Proxy Resource Starvation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing backup systems inefficiently utilize resources, leading to wasted computing resources and incomplete backup sessions due to resource constraints on proxy hosts.
Innovation Solution
Implementing a reservation mechanism to optimize backup session creation based on available proxy infrastructure resources, dividing assets into slices, and executing each slice in separate parallel sessions within containers, while continuously feeding new slices to occupied containers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple backup sessions are created without resource checking, then backup throughput is increased, but proxy hosts become overloaded and backup sessions fail to complete
Solution Approach 1:
The system performs preliminary resource availability checks before creating backup sessions. The resource availability manager queries proxy hosts to determine current resource capacity, and this information is used to calculate an optimal number of parallel backup sessions that can be successfully completed without overloading the infrastructure.
Solution Approach 2:
The system implements continuous feedback loops where the resource availability manager monitors proxy host performance and resource utilization in real-time. Based on this feedback, the system dynamically adjusts the number of backup sessions created and dispatched to proxy hosts, ensuring optimal throughput while maintaining reliable completion rates.
2Use of energy by moving object
If backup sessions are created based on maximum capacity, then resource utilization is maximized, but resource starvation occurs and backup sessions are abandoned
Solution Approach 1:
The system dynamically adjusts resource allocation based on real-time conditions. The resource availability manager continuously monitors proxy host capacity and adjusts the number of backup sessions accordingly. This dynamic approach allows the system to maximize resource utilization during periods of high availability while preventing resource starvation when capacity is constrained, ensuring backup sessions can complete reliably.
3Speed
If parallel backup sessions are increased, then backup speed is improved, but proxy host resources are depleted and sessions fail
Solution Approach 1:
The system changes the parameter of parallel session count based on measured proxy host capacity. Instead of using a fixed or maximum parallel session count, the resource availability manager calculates an optimal parallel session parameter that achieves high backup speed while maintaining sufficient resources for reliable completion, effectively optimizing the speed-throughput tradeoff.
Data Source
AI summary
A method for performing a backup operation, the method comprising receiving a backup request for an asset, partitioning, based on available capacity in a proxy infrastructure, a plurality of slices of the asset into a plurality of backup sessions, wherein each backup session comprises a separate portion of the slices, wherein the plurality of slices are associated with a queue, sending a first reservation request to the proxy infrastructure, wherein the first reservation request specifies the plurality of backup sessions, making a first determination that a first portion of the plurality of backup sessions was successfully reserved and that a second portion of the plurality of backup sessions was unsuccessfully reserved, sending a request to initiate the first portion of the plurality of backup sessions on the proxy infrastructure, and sending the second portion of the plurality of backup sessions to the queue based on the first determination.