Hash-Based Data Deduplication with Pointer-Count Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies due to the storage of multiple copies of single files, leading to wasted storage space, and current deduplication methodologies are computationally expensive due to the use of cryptography like hash calculations.
Innovation Solution
A method involving maintaining pointer quantity counts for virtual mapping groups associated with duplicative data portions and using hash values to manage deduplication, where new data is stored only if the hash value is unique, and existing data is linked if the hash value already exists, with incremental pointer counts for shared data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If deduplication methodologies use cryptography (hash calculations) to identify and eliminate redundant files, then storage space is reduced, but computational cost increases
Solution Approach 1:
The system pre-calculates and stores hash values for data portions in a hash repository during the deduplication process. When the same data portion is encountered again, the pre-stored hash value is retrieved and compared instead of performing a new hash calculation, thereby reducing computational overhead while maintaining deduplication effectiveness
Solution Approach 2:
The invention creates a hash repository that stores copies of hash values for data portions that have been previously processed. This copy mechanism allows the system to quickly compare incoming data against stored hashes without re-computing, significantly reducing the computational energy required for deduplication operations
2Reliability
If multiple copies of single files are stored on the storage system, then data accessibility is improved, but storage space efficiency deteriorates
Solution Approach 1:
The system introduces virtual mapping groups as intermediaries between the physical storage and the data access requests. These mapping groups track multiple pointers to the same data portion, allowing the system to maintain data accessibility through multiple access paths while storing only one physical copy of the data, thus improving storage efficiency without compromising reliability
Solution Approach 2:
The invention creates virtual copies (pointers) that reference the same physical data portion multiple times. Instead of storing multiple physical copies of files, the system stores multiple virtual references to the single physical copy, enabling data to be accessed through multiple paths while consuming minimal additional storage space
Data Source
AI summary
A method, computer program product, and computing system for maintaining a pointer quantity count for each of a plurality of virtual mapping groups, wherein each virtual mapping group is associated with a duplicative data portion within a storage platform and each pointer quantity count defines the number of pointers included within the corresponding virtual mapping group; and obtaining a hash value for a specific duplicative data portion associated with a specific virtual mapping group if a specific pointer quantity count for the specific virtual mapping group is reduced to zero.