Serverless Backup Job Splitting and Parallel Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing backup systems face challenges such as data conflicts and inefficiencies when multiple host devices share a storage device, particularly in master/slave systems, where a single master server's failure requires extensive reconfiguration, and backup operations heavily utilize server resources, necessitating minimal load times.
Innovation Solution
A system and method that allow users to input a master job into a job queue, which is split into smaller jobs and scheduled for execution, utilizing available storage devices efficiently, with the ability to manage storage devices and perform backups without significant server resource utilization, enabling serverless backups and efficient data transfer through multiple streaming and extended copy commands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a master/slave system is used to manage backup storage access, then access control and data consistency are improved, but system reliability deteriorates when the master server fails requiring extensive reconfiguration
Solution Approach 1:
The system enables automatic master server election and failover without requiring manual reconfiguration. When the master server fails, the system automatically selects a new master from the slave servers, allowing the backup system to continue operating with minimal intervention.
Solution Approach 2:
Slave servers maintain their operational status and configuration information even before the master server fails. This preliminary preparation ensures that when failover is needed, the system can immediately transition to a new master without requiring reconfiguration of slave servers.
2Productivity
If traditional backup operations are performed, then data backup is achieved, but server resource utilization increases causing application performance degradation
Solution Approach 1:
The backup operation is divided into multiple parallel streams that can execute simultaneously. Each stream handles a portion of the backup data independently, allowing the backup process to achieve high throughput while distributing the resource burden across multiple streams rather than overwhelming a single server.
Solution Approach 2:
The system introduces parallelism as an additional dimension to the backup process. Instead of sequential backup operations that consume server resources linearly, the system executes multiple backup streams in parallel, effectively reducing the time window during which server resources are heavily utilized.
3Productivity
If multiple storage devices are used for backup, then backup capacity and throughput are improved, but device management complexity increases
Solution Approach 1:
The system implements a unified management interface that handles multiple storage devices through a single controller. This universal approach allows the system to manage diverse storage hardware (disks, tapes, optical media) using the same software framework and control mechanisms, reducing management complexity despite having multiple devices.
Solution Approach 2:
The system merges the management functions for multiple storage devices into a single integrated controller. Rather than requiring separate management systems for each device, the unified controller coordinates all storage devices, simplifying administration while enabling parallel backup operations across the entire storage array.
Data Source
AI summary
A system for backing up and restoring information, includes at least one computer system including information to be backed up and restored, a storage device for receiving at least part of the information to be backed up and for storing and backing up the information. A controller includes a scheduling system for allowing a user to input into a job queue, a master job indicating one or more portions of the information of the at least one computer system to be backed up or restored, and a job control system that splits the master job into a plurality of smaller jobs and inputs the plurality of smaller jobs into the job queue.