Per-Slice Backup Session Orchestration via Resource Reservation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing backup systems inefficiently utilize computing resources, leading to wasted resources and failed backup sessions due to insufficient processing capacity on proxy hosts.
Innovation Solution
Implementing a reservation mechanism and dividing assets into slices to optimize the creation and processing of backup sessions on proxy infrastructure, ensuring only processable sessions are created and maintaining optimal workload management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If backup sessions are created without reservation mechanism, then backup operations can be initiated freely, but computing resources are wasted and backup sessions fail due to insufficient processing capacity
Solution Approach 1:
The system performs preliminary reservation of computing resources before initiating backup sessions. The reservation mechanism checks available processing capacity on proxy hosts and only creates backup sessions when sufficient resources are guaranteed, preventing resource exhaustion and backup failures while maintaining reliable resource allocation
Solution Approach 2:
The backup system divides assets into slices and creates separate backup sessions for each slice, which are then distributed to multiple proxy hosts. This segmentation allows parallel processing and improves resource utilization efficiency, resolving the contradiction between reliability and complexity
2Productivity
If multiple backup sessions are created simultaneously, then backup throughput increases, but proxy hosts become overloaded and processing capacity is insufficient
Solution Approach 1:
The system dynamically adjusts the number of concurrent backup sessions based on real-time availability of computing resources on proxy hosts. The reservation mechanism continuously monitors processing capacity and creates only as many backup sessions as the infrastructure can support, maintaining optimal throughput without overloading any single proxy host
Solution Approach 2:
By dividing assets into slices and creating individual backup sessions for each slice, the system can distribute workload across multiple proxy hosts. This segmentation enables parallel processing that increases throughput while preventing any single proxy host from becoming overloaded
3Loss of energy
If backup sessions wait for completion before creating new sessions, then resource allocation is simplified, but computing resources remain idle and wasted
Solution Approach 1:
The reservation mechanism performs preliminary assessment of resource availability before creating backup sessions, allowing the system to proactively allocate resources rather than reactively waiting for session completion. This enables continuous creation of new backup sessions as resources become available, eliminating idle time and resource wastage
Solution Approach 2:
The system maintains continuous creation of backup sessions by using the reservation mechanism to ensure resources are ready before each session starts. Once a session completes, resources are immediately reallocated to new sessions without interruption, ensuring continuous productive operation and eliminating idle computing resources
Data Source
AI summary
A method for performing a backup operation, the method comprising receiving a backup operation request for an asset, identifying a queue comprising the plurality of slices, wherein each slice references a separate portion of the asset, sending a first backup request to a proxy manager to instantiate a container for each of a plurality of backup sessions, wherein each backup session corresponds to a slice of the plurality of slices, receiving, from the proxy manager, a notification that one of the number of backup sessions is complete and a corresponding container has been torn down, making a second determination that there is an additional slice on a second queue associated with a second asset, and sending, based on the second determination, a backup request to the proxy manager to instantiate a new container for the additional slice associated with the second asset.