Storage Record Compression with Delayed High-Ratio Recompression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods in storage subsystems often consume excessive CPU resources, leading to performance issues due to high CPU utilization during high-ratio compression, and conventional two-staged compression methods are CPU intensive and require double decompression processes.
Innovation Solution
Implementing a high data compression system that uses low-ratio compression initially with minimal CPU load and adds record headers for selective re-compression with a high-ratio method in the background, reducing CPU usage and improving storage capacity by compressing data records on a record basis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If high-ratio compression method is used initially, then storage capacity is improved, but CPU usage increases excessively
Solution Approach 1:
The system applies low-ratio compression immediately upon data arrival to reduce storage requirements, then schedules high-ratio compression for background processing when CPU resources are available. This preliminary action ensures storage capacity improvement without immediate CPU overload.
Solution Approach 2:
The compression process is divided into two distinct stages: initial low-ratio compression applied to all incoming data, and subsequent high-ratio compression applied selectively to specific records. This segmentation allows the system to improve storage capacity progressively without concentrating all compression demands on the CPU at once.
2Quantity of substance
If conventional two-staged compression is used, then compression ratio is improved, but CPU intensity and decompression complexity increase
Solution Approach 1:
The system applies different compression ratios to different data records based on their characteristics and storage needs. Not all records undergo high-ratio compression - only those that benefit most from it. This local differentiation improves overall compression efficiency without requiring complex double decompression for all data.
Solution Approach 2:
Instead of applying high-ratio compression to all data (excessive action), the system applies it partially only to records where the compression ratio improvement justifies the additional processing. This selective approach avoids the complexity of universal two-staged compression while still achieving improved compression ratios where needed.
3Use of energy by moving object
If low-ratio compression is used initially, then CPU usage is reduced, but storage capacity improvement is limited
Solution Approach 1:
Low-ratio compression is applied immediately upon data arrival to achieve quick storage reduction with minimal CPU impact. This preliminary compression provides immediate storage capacity improvement, while the system retains the option to apply additional high-ratio compression later when resources permit.
Solution Approach 2:
The compression strategy is segmented into mandatory low-ratio compression for all records and optional high-ratio compression for selected records. This ensures a baseline storage capacity improvement is always achieved, with the possibility of further optimization through selective high-ratio compression applied in the background.
Data Source
AI summary
An embodiment of the invention may include a method, computer program product and system for saving data received from a host computing device to a storage system. The storage system includes at least one processor and at least one storage. An embodiment may include storing the received data to the storage on a record basis. A record includes a record header including information indicative of an implemented compression method of the record. An embodiment may include monitoring a processing load of the at least one processor. In response to the processing load being less than a predetermined level, an embodiment may include further compressing the record utilizing a high-ratio compression method based on the record requiring further compression. An embodiment may include updating the record header information to reflect details of the utilized a high-ratio compression method. An embodiment may include storing the further compressed record to the storage.