Deduplication Scheduling for Backup Coordination

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Deduplication operations often fail to complete before scheduled backup operations, resulting in duplicate data being locked in backup volumes, which are read-only and cannot be deduplicated, leading to inefficiencies in storage space usage and compliance with service level agreements.

Innovation Solution

A system that coordinates deduplication and backup operations by scheduling deduplication to complete immediately prior to the start of each backup, using a deduplication engine to determine the required time interval based on data modifications since the last deduplication, ensuring no duplicate data is stored in backups.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If deduplication operations are scheduled at regular intervals, then the scheduling is simple and predictable, but the deduplication operations may not complete prior to the start of the next scheduled backup, resulting in duplicate data being locked in the backup

Engineering Contradiction:

Improvescheduling simplicityVSAvoiddeduplication completion before backup

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The system dynamically adjusts the scheduling of deduplication operations based on real-time conditions. Instead of fixed regular intervals, the scheduling engine determines when to initiate deduplication by checking if the storage volume is currently being backed up and whether deduplication has already been performed since the last backup. This dynamic approach ensures deduplication completes before each backup while adapting to varying data modification rates and backup schedules.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system implements feedback mechanisms where the scheduling engine continuously monitors the state of the storage volume, including whether it is currently being backed up and whether deduplication has been performed since the last backup. This feedback loop allows the system to make informed decisions about when to schedule deduplication operations, ensuring they complete before backups while avoiding redundant operations.

Inventive Principle:

Principle #23Feedback

2Loss of substance

If the storage volume is deduplicated immediately prior to creating the backup, then duplicate data is removed from the backup volume, but the backup operation may be delayed if service level agreements stipulate that scheduled backups not be delayed

Engineering Contradiction:

Improveduplicate data in backupVSAvoidbackup operation delay

Core Design Contradiction:

Loss of substanceVSLoss of time

Solution Approach 1:

The system performs deduplication as a preliminary action before backup operations by strategically scheduling deduplication to complete immediately prior to each scheduled backup. The scheduling engine calculates the optimal start time for deduplication based on the backup schedule and the time required to complete deduplication, ensuring duplicate data is eliminated before the backup begins without delaying the backup operation itself.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system takes preliminary anti-action by preventing duplicate data from being locked in backups through proactive scheduling. The scheduling engine proactively initiates deduplication operations at calculated optimal times that ensure completion before each backup, thereby preemptively preventing the problem of duplicate data locking rather than reacting to it after it occurs.

Inventive Principle:

Principle #9Preliminary anti-action

3Reliability

If the time interval for deduplication operations is increased to ensure completion before backup, then duplicate data removal is ensured, but the storage space efficiency is reduced due to longer intervals between deduplication operations

Engineering Contradiction:

Improvededuplication completion before backupVSAvoidstorage space efficiency

Core Design Contradiction:

ReliabilityVSLoss of substance

Solution Approach 1:

The system changes the parameter of deduplication scheduling from fixed regular intervals to dynamic intervals based on backup schedules and data modification rates. The scheduling engine adjusts the timing and frequency of deduplication operations to complete immediately before each backup, optimizing both the reliability of duplicate data removal and storage space efficiency by performing deduplication only when necessary rather than at fixed intervals.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS8095756B1System and method for coordinating deduplication operations and backup operations of a storage volume

Publication Date: 2012.01.10 NETAPP INC

AI summary

A system and method are described for coordinating deduplication operations and backup operations of a storage volume. In one embodiment, a request is identified to eliminate duplicate data from a storage volume. The storage volume may be scheduled to be backed up on a periodic basis. The duration of time required to complete an operation to eliminate duplicate data from the storage volume may be determined. The determination may be based on the amount of data modified on the storage volume since the last operation to eliminate duplicate data from the storage volume. The determined duration of time may be used to schedule the start time of the operation to eliminate duplicate data such that the operation is completed prior to the next scheduled backup of the storage volume. The operation may be initiated at the scheduled start time, ensuring the operation is completed prior to the next scheduled backup.