Universal Backup Host Adaptive Throttling for Metadata Fetching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Frequent backup of production systems in ERP and CRM systems takes a prohibitive amount of time and burdens the production system with resource competition.
Innovation Solution
Implementing adaptive throttling in a universal backup host that adjusts the thread pool size based on latency and request counts to optimize metadata fetching and backup processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If frequent backup of production systems is performed, then backup frequency is improved, but backup time and resource burden increase prohibitively
Solution Approach 1:
The patent implements adaptive throttling that dynamically adjusts the thread pool size based on real-time system conditions. The backup host monitors latency metrics and request counts from the production system, and automatically scales the number of concurrent backup threads up or down. This dynamic adjustment allows frequent backups to be performed while adapting the resource consumption to current system capacity, preventing prohibitive backup times and excessive resource burden.
2Productivity
If frequent backup of production systems is performed, then backup frequency is improved, but resource competition on the production system increases
Solution Approach 1:
The patent implements a feedback mechanism where the backup host continuously monitors performance metrics from the production system, including latency and request counts. These metrics are fed back to the adaptive throttling controller, which adjusts the thread pool size accordingly. When the production system shows signs of resource competition or high latency, the feedback loop triggers a reduction in backup thread count, thereby reducing resource competition while maintaining frequent backup capability.
3Productivity
If adaptive throttling is implemented, then resource utilization efficiency is improved, but system complexity increases
Solution Approach 1:
The patent implements adaptive throttling by changing the parameter of thread pool size based on monitored system conditions. Rather than implementing a completely new complex system, the solution focuses on dynamically adjusting this single critical parameter (number of concurrent threads) based on latency and request count metrics. This approach improves resource utilization efficiency while keeping the added system complexity relatively low, as it primarily involves monitoring existing metrics and adjusting a configurable parameter.
Data Source
AI summary
Systems and methods to throttle a universal backup host are described. The system executes a job, at a backup host, to back up a file set from a source host including fetching metadata from the source host. The system identifies a first operation set from operation sets, the operation set including a first operation. The system communicates, in parallel, requests for metadata items, over a network, to the source host, receives responses, and processes the responses by utilizing threads from a thread pool. The system generates latencies, counts the number of requests, and stores the latencies and number of requests in samples. The system aggregates the samples responsive to a timeout. The system resizes the thread pool based on the aggregating. Finally, the system backs up the file set from the source host based on the metadata.