Storage Compression Control via Ratio Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data compression in storage systems is resource-intensive and can result in poor compression ratios for certain data entities, such as already compressed data, leading to inefficient use of computing resources and performance issues.
Innovation Solution
A method that estimates the compression ratio of uncompressed storage entities by monitoring access requests and using a lightweight compression algorithm to calculate an average compression ratio without altering the stored data, allowing for selective and efficient compression based on predicted savings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If compression is applied to storage entities, then storage space usage is reduced, but computing resources are consumed excessively
Solution Approach 1:
The system applies compression selectively to only those storage entities that meet specific criteria (compression ratio threshold, data characteristics) rather than compressing all data. This partial action approach ensures computing resources are invested only where compression will be effective, avoiding waste on already-compressed or incompressible data
Solution Approach 2:
The system changes the parameter of compression application from universal to conditional based on data characteristics analysis. By evaluating compression ratios and data types, the system dynamically adjusts which storage entities undergo compression, optimizing the balance between storage savings and computational cost
2Volume of stationary object
If compression is applied to all storage entities, then storage cost is reduced, but system performance deteriorates
Solution Approach 1:
The system applies different compression treatments to different storage entities based on their local characteristics. Storage entities with favorable compression characteristics receive compression, while those with poor characteristics (already compressed video, incompressible data) are left uncompressed, creating a locally optimized system that balances storage savings with performance
3Volume of stationary object
If compression is applied to storage entities with poor compressibility, then storage space is reduced minimally, but computing resources are wasted
Solution Approach 1:
The system uses feedback from compression ratio evaluation to determine whether to apply compression. By first assessing the compression ratio of storage entities and comparing it against thresholds, the system receives feedback on potential effectiveness before committing computational resources, avoiding waste on poor candidates
Solution Approach 2:
The system performs preliminary evaluation of data characteristics and compression ratios before applying full compression. This preliminary action identifies suitable candidates in advance, ensuring that subsequent compression operations are applied only to storage entities where they will be effective
Data Source
AI summary
A method for controlling compression based on compression ratio, the method includes monitoring access requests related to a storage entity of a storage system; wherein the access requests are associated with multiple data units; evaluating a compression ratio related to the storage entity; wherein the evaluating comprises applying a first compression process on at least a subset of the multiple data units to provide first compressed data units; serving the access requests without storing, in a permanent storage layer of the storage system, the first compressed data units; and reacting, by the storage system, to the compression ratio.