Client-Side Deduplication Overwrite Ratio Control

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Data protection operations face challenges due to increasing data sizes and complexities in computing environments, requiring improved methods to manage bandwidth, deduplication, encryption, and backup types, with existing solutions inefficiently determining when to perform deduplication, impacting performance and resource usage.

Innovation Solution

The system determines an overwrite ratio based on the amount of data written to a volume and its impacted extents, using a change log and bitmap to enable or disable client-side deduplication, optimizing data transfer and resource usage by approximating the overwrite ratio to decide when to perform deduplication.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If client-side deduplication is performed, then data transfer efficiency is improved, but system resource consumption increases

Engineering Contradiction:

Improvedata transfer efficiencyVSAvoidsystem resource consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The system dynamically changes the deduplication parameter (enabled/disabled) based on the overwrite ratio. When the overwrite ratio exceeds a threshold, deduplication is disabled to reduce resource consumption; when it is below the threshold, deduplication is enabled to improve data transfer efficiency. This parameter change resolves the contradiction by adapting the system state to workload characteristics.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The system transitions from a static deduplication configuration to a dynamic one that automatically adjusts based on real-time workload conditions (overwrite ratio). This dynamic adaptation allows the system to optimize between data transfer efficiency and resource consumption depending on the current backup scenario.

Inventive Principle:

Principle #15Dynamics

2Quantity of substance

If deduplication is always enabled, then backup compression is improved, but processing overhead increases

Engineering Contradiction:

Improvebackup compressionVSAvoidprocessing overhead

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The system uses the overwrite ratio as a parameter to determine when to enable or disable deduplication. By changing this operational parameter based on workload characteristics, the system achieves better backup compression when beneficial while avoiding unnecessary processing overhead when deduplication would be ineffective.

Inventive Principle:

Principle #35Parameter changes

3Speed

If deduplication is disabled, then system performance is maintained, but data transfer efficiency decreases

Engineering Contradiction:

Improvesystem performanceVSAvoiddata transfer efficiency

Core Design Contradiction:

SpeedVSProductivity

Solution Approach 1:

The system dynamically adjusts the deduplication parameter based on the overwrite ratio threshold. When the overwrite ratio is high (indicating poor deduplication potential), the system disables deduplication to maintain system performance. When the ratio is low (indicating good deduplication potential), it enables deduplication to improve data transfer efficiency, thus resolving the contradiction between speed and productivity.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11709800B2Optimized client-side deduplication

Publication Date: 2023.07.25 EMC IP HLDG CO LLC

AI summary

One example method includes optimizing client-side deduplication. When backing up a client, an overwrite ratio is determined based on a size of actual changes made to a volume and a size indicated by changes in a change log. Client-side deduplication is enabled or disabled based on a value of the overwrite ratio.