Fingerprint-Based Storage Selection for Cross-Device Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems struggle to minimize data duplication across diverse storage devices due to the inability to compare incoming data with stored data across different types of destination storage devices, leading to inefficient data transfer and increased storage costs.
Innovation Solution
A management system determines a destination storage device by obtaining fingerprint information from a source storage device, comparing it with candidate storage devices, and selecting the one with the lowest estimated data storage requirement, using a storage consumption estimation module to optimize data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional deduplication methods are used to reduce data transfer, then data transfer efficiency is improved for similar storage devices, but the ability to minimize duplication across different types of destination storage devices deteriorates
Solution Approach 1:
The patent performs deduplication analysis before actual data transfer by comparing fingerprint information of source data with fingerprint information stored in the fingerprint database of candidate destination devices. This preliminary comparison identifies the destination device with the highest data match rate, minimizing the amount of data to be transferred. The fingerprint information is obtained in advance and stored, enabling pre-transfer optimization across different storage device types.
Solution Approach 2:
The patent introduces fingerprint information as an intermediary element that mediates between source storage devices and diverse destination storage devices. Instead of directly comparing raw data across different storage types, the system uses fingerprint information (hash values or metadata) as a universal intermediary representation. This intermediary enables efficient comparison and deduplication identification across heterogeneous storage devices without requiring direct data access or uniform data formats.
2Loss of substance
If fingerprint information is obtained from source storage device and compared with candidate storage devices, then storage capacity costs are reduced, but computational complexity increases
Solution Approach 1:
The patent extracts only the essential fingerprint information (hash values or metadata) from the source storage device and candidate destination devices, rather than processing or comparing entire data sets. This extraction of key identifying features enables efficient comparison with minimal computational overhead. The fingerprint information serves as a compact representation that captures the essential characteristics needed for deduplication identification without the complexity of full data analysis.
Data Source
AI summary
Systems and methods described herein comprise a management system that, in response to obtaining from a volume of a source storage system, a fingerprint information, provide that information to candidate storage systems, which return estimates indicative of an amount of data storage each of the candidate storage systems would require to store the data. The management system then selects, among the candidate storage systems, one or more destination storage systems to transfer the data from the volume to the selected destination storage systems.