Snapshot Pre-Deduplication for Faster Writes via Hot-Extent Forecasting

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing snapshot activation methods in computing systems lead to increased storage system response time due to the CPU-intensive process of generating metadata for redirect on write operations and the resource consumption of copy on write operations, especially in environments with frequent snapshot activations.

Innovation Solution

Implement a time series-based prediction model, such as ARIMA, to forecast which extents of a source device are likely to be 'hot' and pre-deduplicate only those extents on a snapshot, allowing write IO operations to be implemented as redirect on write operations rather than copy on write, thereby optimizing CPU usage and reducing response time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If snapshot activation uses traditional copy on write operations, then data integrity is maintained, but storage system response time increases and CPU resources are consumed

Engineering Contradiction:

Improvedata integrityVSAvoidstorage system response time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-deduplicating snapshot tracks before write operations occur. The system uses time series prediction to identify likely hot extents and pre-establishes deduplication relationships for those extents, so that when write operations occur, the system can immediately use redirect on write instead of copy on write, reducing response time while maintaining data integrity

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent segments the source device into multiple extents and further segments each extent into tracks. This hierarchical segmentation allows the system to apply deduplication selectively to specific tracks in predicted hot extents rather than processing all tracks uniformly, optimizing the balance between CPU resource consumption and response time

Inventive Principle:

Principle #1Segmentation

2Loss of time

If snapshot activation uses redirect on write operations, then storage system response time is reduced, but CPU resource consumption increases due to metadata generation

Engineering Contradiction:

Improvestorage system response timeVSAvoidCPU resource consumption

Core Design Contradiction:

Loss of timeVSUse of energy by moving object

Solution Approach 1:

The patent applies local quality by making different parts of the storage system have different properties. Specifically, it identifies and pre-deduplicates only the hot extents that are likely to receive write operations, while leaving cold extents in their original state. This selective approach reduces CPU resource consumption compared to processing all extents, while still providing the response time benefits of redirect on write for the critical hot regions

Inventive Principle:

Principle #3Local quality

3Loss of time

If all extents are pre-deduplicated on snapshot, then all write operations can use redirect on write, but CPU resources are excessively consumed and response time is not significantly improved

Engineering Contradiction:

Improvestorage system response timeVSAvoidCPU resource consumption

Core Design Contradiction:

Loss of timeVSUse of energy by moving object

Solution Approach 1:

The patent applies partial action by performing deduplication on only a subset of extents rather than all extents. It uses time series prediction to identify the most likely hot extents and pre-deduplicates only those, avoiding the excessive CPU resource consumption of processing all extents while still achieving significant response time improvement for the critical write operations

Inventive Principle:

Principle #16Partial or excessive action

4Productivity

If traditional snapshot activation is used, then all tracks are processed uniformly, but response time increases due to processing cold tracks that will not receive write operations

Engineering Contradiction:

Improvewrite IO processing efficiencyVSAvoidstorage system response time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent uses time series prediction to perform preliminary identification of hot extents before write operations occur. By predicting which extents are likely to be accessed and pre-deduplicating only those, the system avoids the waste of processing cold tracks that will not receive write operations, thereby improving write IO processing efficiency without sacrificing response time for actual writes

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12380065B1Time series forecasting for pre-deduplication of snapshots

Publication Date: 2025.08.05 DELL PROD LP

AI summary

For a source device with an activated snapshot, time series-based prediction of source device write IO operations is implemented, on a per-extent basis, to predict a subset of the source device extents that are likely to be hot (receive write IO operations) during an upcoming time window. Snapshot tracks corresponding to tracks of the predicted hot extents are pre-deduplicated, to accelerate write IO operations on the source device. In instances where the time series-based prediction correctly predicts write IO operations on tracks of an extent, and the tracks of the extent of the source device are pre-deduplicated on the snapshot, it is possible to implement the write IO operations as a redirect on write operation, without first replicating the original track of source data for use by the snapshot. Write IO operations on tracks that are not pre-deduplicated are implemented as copy on write operations.