Scalable SSD Index for Deduplicated Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deduplicated storage systems face inefficiencies in locating data during backup and recovery operations due to the need to scan entire volumes for duplicate data segments, which wastes resources and reduces system performance.
Innovation Solution
A scalable SSD index is created using a hash table that stores fingerprint/container associations, allowing for quick identification of data segments by hashing fingerprints to identify hash buckets and storing fingerprint/container identifier values, with mechanisms for merging and scaling the index to maintain efficient lookup times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire volume is scanned to locate duplicate data segments, then data location completeness is ensured, but system performance deteriorates and resource consumption increases
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing fingerprint information for all data segments in an index structure before actual data location operations. When a recovery operation needs to locate data, the system simply queries the pre-built index rather than scanning the entire volume, thus ensuring complete data location while maintaining high performance.
2Measurement precision
If the index size increases to accommodate more data segments, then data location accuracy improves, but storage resource consumption increases
Solution Approach 1:
The patent applies local quality by storing only essential fingerprint information in the index rather than complete data segment copies. The index contains compact fingerprint identifiers that uniquely identify data segments, providing accurate location information while consuming minimal storage resources. Full data segments are stored only in their original locations, not duplicated in the index.
3Speed
If the SSD index is scaled to handle larger datasets, then data retrieval speed is maintained, but system complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the large-scale index into multiple smaller SSD buckets that can be managed independently. Each bucket handles a portion of the fingerprint data, and the system scales by adding more buckets rather than increasing the complexity of individual buckets. This modular approach maintains fast retrieval speeds while simplifying the management of large datasets.
Data Source
AI summary
A method, article of manufacture, and apparatus for scaling an index on a solid state drive in a deduplicated storage system is discussed. The index may be scaled as index hash buckets approach capacity, or as new solid state drives are added to the system. The hash buckets may be mapped to HDD hash buckets, which are stored on a hard drive.