Storage Data Compression Using Similar Fingerprint Grouping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data reduction technologies, such as deduplication, have a poor data reduction effect due to low probabilities of identical data blocks, leading to inefficient use of storage space and high costs in big data, cloud computing, and artificial intelligence environments.
Innovation Solution
A method that determines whether deduplication can be performed on a target data block, and if not, obtains a similar fingerprint to group the data block with others for similar compression based on a reference block, combining deduplication and compression to improve reduction rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If deduplication technology is used to improve storage space utilization, then storage efficiency is improved, but data reduction effect is poor due to low probability of identical data blocks
Solution Approach 1:
The patent changes the parameter from exact fingerprint matching to similar fingerprint matching. Instead of requiring identical data blocks for deduplication, the system calculates similar fingerprints that can match data blocks with slight variations, thereby significantly improving the data reduction effect while maintaining storage space utilization benefits
Solution Approach 2:
The patent segments the fingerprint matching process into multiple levels: first performing fast exact match deduplication, then performing similar match compression on remaining data blocks. This segmentation allows the system to handle different types of data redundancy efficiently, improving overall data reduction while maintaining storage efficiency
2Productivity
If similar compression is performed on all data blocks, then data reduction rate is improved, but processing complexity increases
Solution Approach 1:
The patent applies partial action by performing similar compression only on data blocks that fail exact match deduplication. Instead of applying compression to all data blocks, the system selectively applies it to the portion that needs it, thereby improving data reduction rate while avoiding unnecessary processing complexity on already deduplicated blocks
Solution Approach 2:
The patent performs preliminary exact match deduplication before performing similar compression. This preliminary action filters out easily compressible data blocks, so that the more complex similar compression process is only applied to remaining blocks, reducing overall processing complexity while maintaining high data reduction rates
Data Source
AI summary
In a method for compressing data, a storage system determines whether deduplication can be performed on a target data block. When deduplication cannot be performed on the target data block, the storage system obtaining a similar fingerprint of the target data block, and determines, based on the similar fingerprint, a combined data block group to which the target data block belongs. The storage system then performs similar compression on the target data block based on a reference block in the combined data block group.