SSD Cache Index for Deduplicated Storage Data Location
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deduplicated storage systems face inefficiencies in locating data during backup and recovery operations due to the need to scan entire volumes for duplicate data segments, which wastes resources and reduces system performance.
Innovation Solution
A solid-state drive (SSD) index is created using a hash table that stores fingerprint/container associations, allowing for quick identification of data segments by hashing fingerprints to identify hash buckets and storing fingerprint/container identifier values, with mechanisms for merging in-memory indexes into persistent storage and scaling SSD buckets to maintain efficient lookup times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire volume is scanned to locate duplicate data segments, then data location completeness is ensured, but system performance deteriorates and resource usage increases
Solution Approach 1:
The patent applies preliminary action by creating and maintaining an index of data segment fingerprints before recovery operations are needed. This index is built during normal storage operations, allowing the system to quickly locate data segments during recovery without scanning the entire volume. The index is updated in advance with fingerprint-container identifier mappings, enabling efficient lookup when needed.
Solution Approach 2:
The patent creates a simplified copy of the storage volume's data segment information in the form of an index structure. This index contains fingerprint-to-container-identifier mappings that replicate the essential location information without duplicating the actual data. The index serves as a lightweight reference that enables fast data location without accessing the entire volume.
2Measurement precision
If the entire volume is scanned to locate data segments, then accurate data identification is achieved, but time consumption increases
Solution Approach 1:
The system performs preliminary actions by pre-computing and storing fingerprint-to-location mappings in an index during data ingestion. When data location is needed, the system simply queries the pre-built index rather than scanning the volume, dramatically reducing lookup time while maintaining accurate identification through fingerprint matching.
Solution Approach 2:
The patent segments the data location task into two parts: (1) maintaining a compact index structure that stores only the essential fingerprint-to-location mappings, and (2) using this segmented index for rapid lookup. This segmentation separates the heavy computation of fingerprint analysis from the lightweight query operation, improving both speed and accuracy.
3Reliability
If computational resources are used to scan the entire volume for duplicates, then complete duplicate detection is achieved, but resource efficiency decreases
Solution Approach 1:
The patent extracts only the essential information needed for duplicate detection and location—the fingerprint-to-container-identifier mappings—and stores this extracted information in a compact index. This extraction eliminates the need to process and analyze the entire volume during duplicate detection, significantly reducing computational resource usage while maintaining complete duplicate detection capability.
Solution Approach 2:
The system performs the computationally intensive fingerprint analysis and duplicate detection in advance, storing the results in the index. This preliminary action shifts the computational burden from the data location operation to the data ingestion operation, making subsequent duplicate detection and recovery operations resource-efficient while maintaining complete detection accuracy.
Data Source
AI summary
A method, article of manufacture, and apparatus for locating data inside a deduplicated storage system is discussed. An index on a solid state device may contain a finger print to container identifier mapping. The container identifier may identify a logical container storing data related to the fingerprint. This data may therefore be located using the index given the fingerprint.