Deduplication Metadata Search Across Heterogeneous Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Performing an exhaustive search for a target data item across multiple storage systems with different deduplication schemes consumes significant computing and network resources, reducing the performance of the storage environment.
Innovation Solution
A computing device applies the hashing schemes of accessible storage systems to generate fingerprints, matches these against deduplication metadata, and generates a summary report listing user-visible locations of the target data item without accessing the deduplicated data, thus optimizing the search process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If an exhaustive search is performed across multiple storage systems with different deduplication schemes, then the completeness of data location results is improved, but the consumption of computing and network resources increases
Solution Approach 1:
The patent extracts only the necessary information (fingerprints and metadata) from the storage systems rather than performing exhaustive searches of actual data contents. The controller generates fingerprints for the target data item and compares them against metadata stored in each deduplication storage system, extracting location information without accessing the deduplicated data itself. This significantly reduces computing and network resource consumption while maintaining complete data location results.
Solution Approach 2:
The patent introduces fingerprints as an intermediary element between the target data item and the storage systems. Instead of directly searching for the target data item across multiple storage systems (which would consume significant resources), the controller generates fingerprints and uses them as mediators to query metadata. This intermediary approach enables complete data location with minimal resource consumption, as fingerprint comparison is computationally efficient compared to full data search.
2Measurement precision
If an exhaustive search is performed across multiple storage systems, then the completeness of data location results is improved, but the performance of the storage environment deteriorates
Solution Approach 1:
The patent extracts only metadata and fingerprint information from the storage systems, avoiding access to the actual deduplicated data. The controller queries each storage system using generated fingerprints and retrieves only location information from metadata, leaving the deduplicated data untouched. This extraction approach ensures complete data location results while maintaining storage environment performance, as no actual data transfer or processing occurs.
Solution Approach 2:
The patent uses fingerprint copies as surrogates for the actual target data item. Instead of searching for and potentially transferring the actual data across multiple storage systems, the controller creates and compares fingerprint representations. These fingerprint copies enable complete data location without impacting storage environment performance, as fingerprint comparison is a lightweight operation that does not involve moving or processing the actual data.
3Adaptability or versatility
If different hashing schemes are applied to locate data across heterogeneous storage systems, then the compatibility of the search process is improved, but the complexity of the searching operation increases
Solution Approach 1:
The patent implements a universal fingerprint generation mechanism that can handle multiple hashing schemes. The controller is designed to generate fingerprints using the same hashing scheme as each target storage system, making the search process compatible with heterogeneous storage systems. This multi-functionality approach maintains compatibility across different systems while managing complexity through a standardized fingerprint comparison process that works uniformly across all storage systems regardless of their specific hashing schemes.
Data Source
AI summary
Example implementations relate to deduplication operations in a storage system. An example implementation includes receiving a target data item to be located in a storage environment, determining deduplication storage systems included in the storage environment, and determining hashing schemes used by the plurality of deduplication storage systems, respectively. The example implementation also includes generating fingerprints by applying, to the target data item, respective hashing schemes of each of the deduplication storage systems. The example implementation further includes identifying potential storage locations of the target data item based on the fingerprints, and generating a location report based on the identified potential storage locations.