Deduplication Storage Image Healing via Segment Appending

Solution Overview

Problem

Traditional deduplication systems face challenges in efficiently healing corrupted data segments, as they often rely on imperfect storage solutions, leading to widespread impact when a single data segment becomes corrupted, and existing methods either replace corrupted data segments, which can result in re-corruption, or write non-corrupted segments to new containers, causing metadata modifications and increased costs.

Innovation Solution

A computer-implemented method that detects corrupted data segments using algorithms like checksum, hash, or digital fingerprint, checks for matching digital fingerprints during backup operations, and appends non-corrupted data segments to the same container with a greater offset, rather than replacing the corrupted ones, thereby avoiding re-corruption and metadata modifications.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If corrupted data segments are replaced in traditional deduplication systems, then data integrity is improved, but the risk of re-corruption increases and storage costs increase due to writing to new containers

Engineering Contradiction:

Improvedata integrityVSAvoidre-corruption risk

Core Design Contradiction:

ReliabilityVSObject-affected harmful factors

Solution Approach 1:

The system performs preliminary verification by computing checksums or hash values of data segments before and after the corruption event. This preliminary action enables the system to identify corrupted segments through fingerprint mismatch and subsequently retrieve the correct versions from backup images, preventing re-corruption by verifying data integrity before restoration.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of replacing corrupted data segments by writing to new containers, the system copies the correct data segment versions from backup images directly into the existing container at the original offsets. This copying approach maintains the original container structure, avoids metadata modifications, and eliminates the need for new storage allocations.

Inventive Principle:

Principle #26Copying

2Reliability

If corrupted data segments are replaced by writing to new containers, then data recovery is achieved, but metadata modification costs and storage costs increase

Engineering Contradiction:

Improvedata recoveryVSAvoidstorage cost

Core Design Contradiction:

ReliabilityVSLoss of substance

Solution Approach 1:

The system performs preliminary identification of corrupted data segments by comparing current fingerprints with previously stored fingerprints before attempting recovery. This preliminary action enables targeted recovery only of corrupted segments rather than entire containers, minimizing storage operations and costs.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system copies correct data segment versions from backup images directly into the existing container at the original offsets, rather than writing to new containers. This approach maintains the original container structure, avoids metadata modifications, and eliminates the need for new storage allocations, thereby reducing storage costs.

Inventive Principle:

Principle #26Copying

3Reliability

If corrupted data segments are replaced by writing to new containers, then data availability is improved, but metadata modification complexity and operational costs increase

Engineering Contradiction:

Improvedata availabilityVSAvoidmetadata modification complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system performs preliminary verification by computing checksums or hash values of data segments before and after the corruption event. This preliminary action enables the system to identify corrupted segments through fingerprint mismatch and subsequently retrieve the correct versions from backup images, preventing re-corruption by verifying data integrity before restoration.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of replacing corrupted data segments by writing to new containers, the system copies the correct data segment versions from backup images directly into the existing container at the original offsets. This copying approach maintains the original container structure, avoids metadata modifications, and eliminates the need for new storage allocations.

Inventive Principle:

Principle #26Copying

Data Source

PatentEP3519965B1Systems and methods for healing images in deduplication storage

Publication Date: 2023.05.03 VERITAS TECHNOLOGIES LLC

Figure 1
Figure 2
Figure 3

AI summary

The disclosed computer-implemented method for healing images in deduplication storage may include (1) detecting that a data segment stored within deduplication storage is corrupted by executing an algorithm on the data segment to generate a numerical identifier for the data segment and determining that the numerical identifier does not match a previously stored numerical identifier for the data segment, (2) checking, during a backup operation, whether a digital fingerprint of a data segment of an image being backed up matches a digital fingerprint of the corrupted data segment that was taken prior to corruption and that is already stored within the deduplication storage, and (3) appending the data segment from the image being backed up to a container that already contains the corrupted data segment rather than replacing the corrupted data segment. Various other methods, systems, and computer-readable media are also disclosed.