Network Informed Policy Creator for Dynamic Backup Window Sizing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current backup software treats all stages of data backup as a single process, leading to unpredictable load distribution and potential backup failures due to overloaded primary and secondary storage systems, as they do not dynamically adjust to available bandwidth over time.
Innovation Solution
A network informed policy creator (IPC) that connects with network devices to monitor data flow, gather historical data, and recommend optimal transfer times and paths based on available bandwidth, decoupling backup stages and dynamically sizing backup windows for improved bandwidth utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If backup software applies a single backup policy on all stages as one process, then the backup process is simple to manage, but the primary and secondary storage systems become overloaded and unpredictable
Solution Approach 1:
The backup process is divided into multiple discrete stages (snapshot creation, data transfer, verification) that can be independently managed and optimized. Each stage has its own policy controls, allowing granular management of storage system loads while maintaining overall backup reliability.
2Productivity
If fixed blocks of time are used for backup windows, then backups can be ordered based on data size, but some bandwidth remains under-utilized
Solution Approach 1:
Backup window sizes are dynamically adjusted based on real-time network bandwidth availability and historical patterns. The system monitors actual bandwidth conditions and resizes backup windows accordingly, maximizing bandwidth utilization while preventing storage system overload.
Solution Approach 2:
The system continuously monitors network bandwidth conditions and feedback from storage systems, using this information to adjust backup scheduling and window sizes. This closed-loop control optimizes bandwidth utilization while preventing system overload.
3Measurement precision
If network tools only monitor latency and packet drops, then quality metrics are simple to measure, but transfer timing and path optimization are insufficient
Solution Approach 1:
The system monitors multiple network parameters beyond basic quality metrics, including available bandwidth, transfer rates, and historical network conditions. These expanded parameters enable optimal transfer timing and path selection, significantly improving data transfer efficiency.
Data Source
AI summary
Optimizing backups for a data protection system by determining a size of a dataset to be saved in each backup operation and an available bandwidth in each transfer window of transfer windows for data transfers between a source and destination. An order of the backup operations is defined based on first ordering the backups based on decreasing dataset size and second ordering the transfer windows in order of decreasing bandwidth, and matching the backups to the transfer windows in accordance with these orderings. A dynamic window sizing process determines an initial change between the minimum and maximum bandwidth utilization over a period of time and then iteratively split and consolidate the time blocks until optimal utilization over the time period is reached.