Data Block Grouping by Similarity for Higher Compression Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods struggle to distinguish the similarity degree between data blocks with the same fingerprint, leading to low compression efficiency.
Innovation Solution
Group data blocks with the same fingerprint into compression groups based on their similarity degree, using specific conditions to ensure high similarity, and perform delta compression on these groups.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If similar fingerprint matching is used to find reference data blocks, then data blocks with same fingerprint can be identified, but similarity degree between data blocks cannot be distinguished leading to low compression efficiency
Solution Approach 1:
The patent changes the parameter used for similarity measurement from binary fingerprint matching to multi-dimensional feature vector comparison. By representing data blocks as feature vectors with multiple dimensions (e.g., statistical features, frequency features) and calculating similarity based on the degree of overlap in these dimensions, the system can distinguish different levels of similarity rather than treating all matching fingerprints as equally similar.
2Productivity
If data blocks are compressed individually based on fingerprint matching, then compression can be performed, but compression efficiency is low due to inability to group highly similar blocks
Solution Approach 1:
The patent merges multiple highly similar data blocks into compression groups based on their feature vector similarity. By clustering data blocks that share significant feature dimensions into groups and performing coordinated compression on these groups, the system achieves better compression ratios than individual compression while managing complexity through systematic grouping criteria.
Solution Approach 2:
The patent segments the compression process into distinct phases: feature extraction, similarity calculation, group formation, and compression execution. This segmentation allows the system to manage the complexity of multi-dimensional similarity assessment by breaking it down into manageable steps, where each phase handles a specific aspect of the compression task.
Data Source
AI summary
A data compression method includes providing one or more data block sets, where each data block set includes a plurality of data blocks with a same fingerprint; providing one or more compression groups based on a similarity degree between a plurality of data blocks included in each data block set, where one compression group includes a plurality of similar data blocks, the plurality of similar data blocks are a subset of data blocks included in the one or more data block sets, and a similarity degree between the plurality of similar data blocks meets a specific similarity condition; and compressing the plurality of similar data blocks included in the compression group.