Similarity-Based Deduplication Repair Strings for Corrupt Snapshots
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deduplication storage systems face challenges in efficiently repairing snapshots that include corrupt data strings, leading to potential data loss and reduced system performance when relying on older snapshots that are less similar to the source item due to accumulated changes.
Innovation Solution
A deduplication storage system controller identifies corrupt strings by matching fingerprints, selects a comparison window, and compares it to candidate manifests to find non-corrupt strings for repair, minimizing data loss by replacing references to corrupt strings with references to non-corrupt strings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If older snapshots are used for repair, then more time has passed allowing corruption to be detected, but the snapshots are less similar to the source item due to accumulated changes leading to greater data loss
Solution Approach 1:
The system pre-calculates and stores similarity scores between snapshots and source items before repair is needed. When corruption is detected, the system can immediately query pre-computed similarity data to identify the most similar non-corrupt snapshot, avoiding the need to calculate similarities from scratch and enabling rapid selection of the best repair candidate that minimizes data loss.
Solution Approach 2:
The patent replaces the traditional sequential approach of checking snapshots chronologically with a similarity-based selection mechanism. Instead of mechanically iterating through snapshots in time order, the system uses fingerprint comparison and similarity scoring to directly identify the most appropriate repair candidate, substituting time-based selection with quality-based selection.
2Manufacturing precision
If comprehensive snapshot comparison is performed to find the best repair candidate, then repair accuracy improves, but system performance decreases due to increased processing time
Solution Approach 1:
The system implements a tiered comparison approach where it first performs a rapid preliminary filter using key fingerprint characteristics to eliminate obviously unsuitable candidates. Then it performs detailed comparison only on a reduced set of promising candidates. This partial action approach achieves high repair accuracy without requiring exhaustive comparison of all snapshots, thus maintaining system performance.
Solution Approach 2:
The system pre-computes and stores fingerprint signatures and similarity metrics for all snapshots before repair operations are needed. When a repair is required, these pre-computed data structures enable rapid identification of candidate snapshots without performing full comparisons at repair time, thereby maintaining both high accuracy and system performance.
Data Source
AI summary
Example implementations relate to deduplication operations in a storage system. An example includes selecting a comparison window in a first manifest of a deduplication storage system, where the comparison window comprises multiple data units and includes a corrupt string. The example also includes identifying multiple manifests, determining match scores for the manifests based on matching against the comparison window, and identifying multiple repair strings based on the match scores. The example also includes recording the corrupt string and the multiple repair strings in a first entry of a repair string data structure, and repairing the corrupt string in the first manifest using at least one of the multiple repair strings.