In-Memory Compression with Non-Volatile Capacity Buffer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing demand for memory in computer systems, despite decreasing memory costs per unit, leads to higher overall system costs due to the need for larger memory capacities, and existing memory pooling methods struggle with unpredictable storage capacity when data compression is applied.
Innovation Solution
Implementing an appliance with both volatile and non-volatile storage, using low-overhead compression algorithms to maximize storage capacity, and employing additional non-volatile storage as a backup to ensure advertised memory capacity is met, even with varying data compressibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data compression is applied in memory pooling, then storage capacity is increased, but storage capacity becomes unpredictable
Solution Approach 1:
The patent allocates a portion of non-volatile storage as a buffer or cushion before the volatile storage capacity is fully utilized. This buffer zone ensures that even when compression ratios vary, the system can guarantee a minimum advertised capacity by having pre-allocated backup space in non-volatile storage that can be used if compression does not achieve expected compression levels.
Solution Approach 2:
The system dynamically adjusts the compression ratio parameter based on data characteristics and storage availability. By monitoring actual compression performance and adjusting the target compression ratio, the system can maintain predictable effective capacity while still achieving compression benefits. The hybrid storage architecture allows parameter changes between volatile and non-volatile storage allocation.
2Quantity of substance
If volatile storage capacity is reduced, then cost is reduced, but storage capacity for uncompressed data is insufficient
Solution Approach 1:
Non-volatile storage acts as an intermediary between the reduced volatile storage and the applications requiring memory. When volatile storage is full, the system can offload data to non-volatile storage, which serves as a buffer or overflow area. This intermediary layer allows the system to advertise higher capacity than the volatile storage alone provides, while maintaining reliability through the backup non-volatile storage.
Solution Approach 2:
The patent segments the memory pool into volatile and non-volatile portions with distinct roles. Volatile storage handles high-speed access for active data, while non-volatile storage provides capacity extension and backup. This segmentation allows each storage type to be optimized for its specific function while working together to provide the overall advertised capacity.
3Quantity of substance
If compression algorithms with higher compression ratios are used, then storage capacity is increased, but performance degradation increases
Solution Approach 1:
Instead of applying heavy compression to all data, the system uses moderate compression that achieves sufficient capacity increase without excessive processing overhead. The compression level is calibrated to provide just enough capacity extension while maintaining acceptable performance, avoiding the diminishing returns of aggressive compression algorithms.
Solution Approach 2:
The system applies different compression strategies to different data regions or types based on their access patterns and compressibility characteristics. Frequently accessed data may use lighter compression or be kept in uncompressed form in volatile storage, while less frequently accessed data can use more aggressive compression. This local differentiation optimizes the balance between capacity and performance for each data segment.
Data Source
AI summary
An appliance is described. A write engine may process a write request from an application to write a first data into a memory. The write request may including the first data and an address. A compression engine may compress the first data to produce a first compressed data. A write module may store the first compressed data in the memory. The first data may be a first part of a page that may further include a second data as a second part. The first compressed data may be a first part of a compressed page that may further include a second compressed data as a second part.