Deduplication Filesystem Metadata Verification for Missing Segments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data backup systems struggle to identify and recover files impacted by data loss due to the difficulty in identifying and listing missing segments, making it challenging to restore data from backup copies.
Innovation Solution
A method and apparatus that utilize a fingerprint hashtable to verify metadata segments in a deduplicated filesystem, enabling the identification of missing segments and impacted files through a breadth-first scan, thereby facilitating their recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional data backup systems are used to store backup copies, then data protection is provided, but it becomes difficult or impossible to identify and list missing segments and impacted files
Solution Approach 1:
The patent introduces a fingerprint index as an intermediary data structure that maps data segment fingerprints to their storage locations. This intermediary enables efficient identification of missing segments by allowing the system to quickly check whether fingerprints exist in the backup without scanning entire backup volumes, thus resolving the contradiction between maintaining data protection and enabling easy detection of missing segments
Solution Approach 2:
The system implements a feedback mechanism where the fingerprint index continuously tracks the presence and location of data segments. When verification is performed, the system compares expected fingerprints against actual backup contents, and the feedback from this comparison identifies missing segments. This feedback loop enables automatic detection and reporting of data loss without manual intervention
2Measurement precision
If all backup copies are scanned to identify missing segments, then complete identification of impacted files is achieved, but system performance and productivity deteriorate
Solution Approach 1:
The patent pre-computes and stores fingerprints of all data segments in the fingerprint index before verification is needed. This preliminary action organizes segment information in an easily queryable structure, so that during verification, the system only needs to check specific fingerprints rather than scanning entire backup volumes. This resolves the contradiction by enabling complete identification of missing segments while maintaining high productivity
Solution Approach 2:
The verification process is segmented into discrete fingerprint checks rather than a monolithic scan of all backup data. The fingerprint index divides the backup verification task into individual segment lookups, allowing the system to efficiently identify only the missing segments without processing entire backup volumes, thus maintaining both precision and productivity
Data Source
AI summary
One example method includes finding an Ln segment in a namespace, and adding the Ln segment to a fingerprint hashtable, scanning a container set and locating Ln in the container set, looking for Ln in the fingerprint hashtable, when Ln is found in the fingerprint hashtable, deleting Ln from the fingerprint hashtable, otherwise, retaining Ln in the fingerprint hashtable, when Ln is found in in the fingerprint hashtable, adding all fingerprints, pointed to by Ln, to the fingerprint hashtable, and when all levels of a tree that includes Ln have been checked, identifying as missing, any fingerprints still remaining in the fingerprint hashtable.