Dynamic Backup System Scaling for Distributed Storage

Solution Overview

Problem

Traditional backup systems perform poorly when backing up large data sets stored on distributed scale-out storage systems, often taking too long and requiring expensive hardware.

Innovation Solution

The system dynamically provisions backup systems based on the size of the backup job, divides the job into smaller tasks, and performs these tasks in parallel, combining the results to create a unified backup that reflects the unified storage view of the distributed scale-out data system.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If traditional backup systems are used to back up large data sets on distributed scale-out storage systems, then backup functionality is provided, but backup time becomes excessively long and hardware requirements become excessively high

Engineering Contradiction:

Improvebackup speedVSAvoidbackup time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The backup job is divided into multiple smaller backup tasks based on the underlying storage architecture of the distributed scale-out data system. Each backup task corresponds to a specific storage node or data subset, allowing parallel execution across multiple backup systems. This segmentation enables the backup process to scale with the data size while maintaining manageable task granularity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system dynamically provisions backup systems based on the size of the backup job and the scope of data to be backed up. The number of backup systems deployed is determined by analyzing the backup job scope and storage architecture, allowing the system to adapt resource allocation to actual workload requirements rather than using fixed hardware configurations.

Inventive Principle:

Principle #15Dynamics

2Productivity

If traditional backup systems are used for distributed scale-out storage systems, then backup is performed, but expensive hardware is required

Engineering Contradiction:

Improvebackup performanceVSAvoidhardware requirements

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system dynamically provisions backup systems based on the size of the backup job and the scope of data to be backed up. The number of backup systems deployed is determined by analyzing the backup job scope and storage architecture, allowing the system to adapt resource allocation to actual workload requirements rather than using fixed hardware configurations.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes the parameter of backup system provisioning from static to dynamic based on job scope. By determining the number of backup systems to deploy based on the backup job's scope and the underlying storage architecture, the system optimizes hardware resource utilization without requiring over-provisioned expensive hardware for all scenarios.

Inventive Principle:

Principle #35Parameter changes

3Ease of operation

If backup is performed on unified storage view without considering underlying architecture, then simplified operation is achieved, but backup efficiency decreases

Engineering Contradiction:

Improvebackup operation simplicityVSAvoidbackup efficiency

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The backup job is divided into multiple smaller backup tasks based on the underlying storage architecture of the distributed scale-out data system. Each backup task corresponds to a specific storage node or data subset, allowing parallel execution across multiple backup systems. This segmentation enables the backup process to scale with the data size while maintaining manageable task granularity.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS10191817B2Systems and methods for backing up large distributed scale-out data systems

Publication Date: 2019.01.29 COHESITY INC

AI summary

A computer-implemented method for backing up large distributed scale-out data systems may include (1) identifying a backup job to be performed on a distributed scale-out storage system that presents a unified storage view, (2) determining, in response to the backup job applying to distributed scale-out storage and based on a scope of the backup job, a number of backup systems to deploy for performing the backup job, (3) deploying a plurality of backup systems creating a plurality of backups covering the plurality of data objects by (i) assigning, to each backup system, a subset of data objects within the backup job and (ii) backing up, by the backup system, the subset of data objects assigned to the backup system, and (4) combining the plurality of backups into a unified backup that represents the unified storage view. Various other methods, systems, and computer-readable media are also disclosed.