Queue Occupancy Based Data Compression in Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression techniques in computing systems face challenges due to metadata overheads, which can offset bandwidth benefits and increase memory traffic, especially in high-capacity memory systems, where storing metadata becomes impractical and requires additional memory requests.
Innovation Solution
Implementing a method to enable data compression in computing systems based on queue occupancy, where each storage block is split, and compression is activated only when the queue occupancy exceeds a dynamic threshold, allowing for partial or complete data fetching from divided storage blocks accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data compression is always activated, then bandwidth efficiency is improved, but metadata overhead increases and memory traffic increases
Solution Approach 1:
The system dynamically activates or deactivates compression based on queue occupancy thresholds. When the queue occupancy exceeds a dynamic threshold, compression is activated; when it falls below the threshold, compression is deactivated. This dynamic adaptation allows the system to optimize bandwidth efficiency only when beneficial, avoiding the metadata overhead penalties of always-on compression.
Solution Approach 2:
The system changes the compression parameter (enabled/disabled) based on the queue occupancy parameter. By monitoring queue occupancy and comparing it against dynamic thresholds, the system adjusts the compression state to match current workload conditions, thereby improving bandwidth efficiency during high occupancy while avoiding metadata overhead during low occupancy periods.
2Productivity
If compression is activated for all storage blocks, then data transfer efficiency is improved, but device complexity increases due to split storage blocks and threshold management
Solution Approach 1:
The storage device is divided into multiple storage blocks, each independently managed with its own compression state. This segmentation allows the system to apply compression selectively to individual blocks based on their specific queue occupancy conditions, rather than forcing a uniform compression policy across the entire device, thus managing complexity through modular organization.
Solution Approach 2:
The system implements feedback mechanisms by continuously monitoring queue occupancy for each storage block and using this information to dynamically adjust compression states. The dynamic thresholds provide feedback boundaries that automatically trigger compression activation or deactivation, reducing the need for complex manual management while maintaining optimal data transfer efficiency.
3Adaptability or versatility
If compression metadata is stored in memory, then compression functionality is enabled, but additional memory requests are required increasing memory traffic
Solution Approach 1:
The system extracts and eliminates the need for traditional compression metadata storage in memory. Instead of storing compression metadata that would require additional memory requests, the system uses queue occupancy information already present in the memory controller to dynamically control compression, thereby enabling compression capability without increasing memory traffic.
4Productivity
If compression is always enabled, then bandwidth utilization is improved, but uncompressible data increases memory traffic overhead
Solution Approach 1:
The system dynamically adjusts compression based on queue occupancy thresholds rather than always enabling it. When queue occupancy indicates that compression would be beneficial (exceeding the dynamic threshold), compression is activated to improve bandwidth utilization. When occupancy is low or data is likely uncompressible, compression is deactivated to avoid increasing memory traffic overhead.
Data Source
AI summary
Various embodiments are provided for enabling data compression in a computing system by a processor. Each storage block of a storage device associated with a queue may be split. Compression of data may be activated upon data occupancy within a queue exceeding a dynamic threshold. In one aspect, only a partial amount of the data is fetched, back to back, from a divided storage block in the storage block according to the queue based upon the data occupancy within the queue exceeding the dynamic threshold. A complete amount of the data may be fetched from the divided storage block in a storage block according to the queue upon the data occupancy within the queue being less than dynamic threshold.