Selective Entropy Computation for Adaptive Data Chunk Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression methods in data storage systems are inefficient as they often perform unnecessary entropy computation, which is CPU-intensive, and fail to dynamically adjust compression based on the compressibility ratio of data chunks, leading to suboptimal storage utilization.
Innovation Solution
A method that determines whether to activate or deactivate entropy computation for data chunks based on a compressibility ratio and cost ratio, performing entropy computation only when beneficial, and compressing data chunks if they achieve a threshold level of data reduction, thereby optimizing CPU usage and storage efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If entropy computation is performed for all data chunks, then compression efficiency is improved, but CPU overhead increases
Solution Approach 1:
The patent implements dynamic activation and deactivation of entropy computation based on real-time compressibility ratio monitoring. The system adjusts the entropy computation status (active/inactive) dynamically according to whether the current data chunk compressibility ratio exceeds the threshold, making the computation process adaptive rather than static.
Solution Approach 2:
The patent changes the parameter of entropy computation from a fixed state to a variable state controlled by the compressibility ratio threshold. When the compressibility ratio exceeds the threshold, entropy computation is deactivated; when it falls below the threshold, entropy computation is activated, thus optimizing CPU usage based on data characteristics.
2Use of energy by moving object
If entropy computation is deactivated, then CPU overhead is reduced, but storage efficiency deteriorates
Solution Approach 1:
The patent employs feedback mechanisms by continuously monitoring the compressibility ratio of data chunks and using this information to control the activation status of entropy computation. The system calculates the compressibility ratio, compares it with the threshold, and adjusts entropy computation accordingly, creating a closed-loop control system that balances CPU overhead and storage efficiency.
Solution Approach 2:
The system dynamically switches between two operational modes: entropy computation active mode and entropy computation inactive mode. This dynamic switching is triggered by the compressibility ratio threshold, allowing the system to optimize performance based on actual data characteristics rather than operating in a fixed mode.
3Productivity
If compression is performed on all data chunks, then storage efficiency is improved, but computational cost increases
Solution Approach 1:
The patent changes the compression processing parameter from unconditional execution to conditional execution based on the compressibility ratio threshold. Data chunks are compressed only when their compressibility ratio indicates potential benefit, avoiding unnecessary compression computation for data that would not achieve meaningful compression.
Solution Approach 2:
The patent applies partial compression action by selectively compressing only those data chunks that meet the compressibility ratio criteria. Instead of compressing all data chunks uniformly, the system performs compression partially - only when the compressibility ratio threshold is satisfied, thus reducing unnecessary computational cost.
Data Source
AI summary
Techniques for data processing may include: receiving a data chunk of the data set; determining, in accordance with criteria including a compressibility ratio for the data set and a cost ratio of compression computation cost and entropy computation cost, whether to activate or deactivate entropy computation for the data set, wherein the compressibility ratio is ratio of a number of compressible data chunks of the data set and a number of uncompressible data chunks of the data set; and responsive to determining to activate entropy computation for the data set, performing first processing comprising: determining an entropy value for the data chunk; and determining, in accordance with the entropy value for the data chunk, whether to compress the data chunk.