Combined Data Block Compression to Cut Delay and Storage Cost
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data deduplication and compression technologies suffer from redundant compression and decompression procedures, leading to increased delays and resource consumption, which complicates the data processing and increases storage costs.
Innovation Solution
Implement a data processing method that combines multiple data blocks for compression and decompression using a combine compression/decompression approach, optimizing the coding partition to enhance efficiency and reduce resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If hierarchical data reduction technology is used to perform multiple compression procedures (delta compression and deep compression), then the reduction ratio is improved, but the compression and decompression delay increases
Solution Approach 1:
The patent combines multiple compression procedures (delta compression and deep compression) into a unified hierarchical data reduction process. By merging these operations and optimizing their execution sequence, the system achieves high reduction ratios while reducing the overall compression and decompression delay compared to performing separate compression procedures.
2Loss of substance
If multiple compression procedures are performed on clustered similar data, then the reduction ratio is improved, but the resource consumption increases
Solution Approach 1:
The patent performs preliminary clustering of similar data using sampled feature values before applying compression procedures. By pre-organizing data into clusters of similar items, the subsequent compression operations work on more homogeneous data sets, improving reduction ratios while reducing the computational resources needed compared to compressing unorganized data.
3Loss of substance
If data is clustered by sampling feature values, then the reduction ratio is improved, but the device complexity increases
Solution Approach 1:
The patent uses sampling of feature values rather than analyzing complete data sets for clustering purposes. By examining only selected features and a portion of data characteristics, the system achieves effective clustering and improved reduction ratios while keeping the clustering mechanism relatively simple and avoiding excessive computational complexity.
Data Source
AI summary
A data processing method includes obtaining a plurality of to-be-compressed data blocks; combining the plurality of to-be-compressed data blocks; and compressing the plurality of combined to-be-compressed data blocks to obtain a data set with combine compression.