Proxy Pool Load Balancer for Data Backup Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data backup systems using a single proxy can slow down the backup process, necessitating a more efficient method to distribute and manage data backup across multiple proxies while minimizing impact on productivity and cost.
Innovation Solution
A data backup system utilizing a proxy pool with a load balancer that distributes data sets across multiple proxies based on a weighted average load calculation, including factors like file size, compressibility, and encryption, to optimize workload distribution and parallel processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single proxy is used to back-up all files, then the system complexity is low, but the backup speed and productivity deteriorate
Solution Approach 1:
The patent divides the backup system into multiple proxies, each handling specific subsets of files. The load balancer segments the overall backup task and distributes it across multiple proxies, enabling parallel processing and improving backup speed while maintaining manageable system complexity through modular architecture.
Solution Approach 2:
The patent combines multiple proxies working in parallel under a unified load balancer coordination system. By merging the capabilities of multiple proxies while centralizing task distribution, the system achieves higher productivity without proportionally increasing complexity, as the load balancer provides unified management.
2Productivity
If multiple proxies are used to speed up backup, then backup productivity improves, but workload distribution unevenness and system complexity increase
Solution Approach 1:
The load balancer implements feedback mechanisms by monitoring the workload and performance of each proxy in real-time. Based on this feedback, the load balancer dynamically adjusts task distribution to ensure balanced workload allocation across proxies, preventing any single proxy from becoming a bottleneck while maintaining high backup productivity.
Solution Approach 2:
The system employs dynamic workload distribution where the load balancer continuously adapts task allocation based on current proxy states. This dynamic adjustment ensures that workload distribution remains balanced even as system conditions change, maintaining ease of operation while maximizing backup speed through efficient parallel processing.
3Loss of time
If multiple proxies are used for parallel processing, then backup time is reduced, but the complexity of managing and coordinating proxies increases
Solution Approach 1:
The load balancer serves multiple functions simultaneously: it distributes tasks to proxies, monitors their performance, balances workloads, and coordinates parallel processing. This multi-functionality reduces the need for separate specialized components, thereby reducing overall coordination complexity while enabling effective parallel processing to minimize backup time.
Solution Approach 2:
The load balancer acts as an intermediary between the file system and multiple proxies, centralizing the coordination complexity in a single component. This intermediary manages all communications and task distributions, simplifying the overall system architecture while enabling parallel processing across multiple proxies to reduce backup time.
Data Source
AI summary
A data back-up system configured to back-up one or more data sets from one or more devices to a data back-up server by using a proxy pool is presented. The data back-up system includes a load-balancer configured to distribute the one or more data sets across the proxy pool. The load balancer includes a data receiver configured to receive types of files, number of files, and total size of each file in the one or more data sets; a load estimator configured to estimate a weighted average load of each data set based on the number of files, the total size of each file, a compressibility factor for each file type, and an encryption factor for each file type; and a load distributor configured to distribute the one or more data sets as a plurality of workloads across the proxy pool.