Parallel Backup Task Segmentation for Data Volumes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data backup methods are inefficient for large volumes of data, leading to increased backup and restore times, as well as difficulties in allocating storage space, resulting in potential data loss and business disruptions.
Innovation Solution
A method that divides data into sections and assigns multiple backup tasks to operate in parallel, adjusting based on available memory, fragmentation levels, and tape mounts, to expedite the backup process and optimize storage allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single backup task is used to copy the entire volume, then the backup process is simple to manage, but the backup time and restore time increase significantly as volume size expands
Solution Approach 1:
The volume is divided into multiple sections, and multiple backup tasks are created to back up different sections in parallel. This segmentation allows the backup process to complete faster while maintaining manageability through the coordination framework provided by the system.
Solution Approach 2:
The system transitions from a single-threaded sequential backup approach to a multi-threaded parallel backup approach by creating multiple backup tasks that operate simultaneously on different sections of the volume, effectively adding a temporal dimension to the backup process.
2Ease of manufacture
If a single backup task is used, then storage allocation is straightforward, but finding and allocating enough storage space becomes difficult as backup file size increases
Solution Approach 1:
The backup process is segmented into multiple tasks that create multiple smaller backup files instead of one large backup file. This makes storage allocation easier as the system can distribute these smaller files across multiple storage locations, and individual files are more manageable in size.
3Loss of time
If multiple backup tasks operate in parallel, then backup time is reduced, but system resource management becomes more complex
Solution Approach 1:
The system implements feedback mechanisms where the backup manager monitors the status of multiple backup tasks and adjusts resource allocation dynamically. This feedback loop allows parallel tasks to be coordinated efficiently without requiring complex manual management, as the system automatically responds to task completion and resource availability.
Solution Approach 2:
The backup system is designed to self-manage the coordination of multiple parallel tasks through automated task creation, monitoring, and completion handling. The system serves itself by automatically dividing the volume into sections, creating appropriate backup tasks, and managing their execution without requiring complex external coordination.
4Device complexity
If larger backup files are created, then fewer storage locations are needed, but the time to restore data increases
Solution Approach 1:
The backup data is segmented into multiple smaller files corresponding to different volume sections. During restoration, these smaller files can be processed in parallel, significantly reducing restore time compared to reading from a single large backup file, while the system manages the distribution across storage locations through automated task coordination.
Data Source
AI summary
A method for backing up data is disclosed. In one embodiment, such a method includes identifying a volume of data to back up, and determining a number of backup tasks that can operate in parallel to back up data in the volume. The number of backup tasks may be based on an amount of memory available, a fragmentation level of a target storage area, a number of tape mounts that are available, or the like. The method then divides the volume into a number of sections corresponding to the number of backup tasks. Each section is associated with a particular backup task. The method then initiates the backup tasks to back up their corresponding sections in parallel. In certain embodiments, each backup task generates a data set storing backup data from its corresponding section. A corresponding system and computer program product are also disclosed.