Nearstore Data Compression for Buffer Cache Capacity Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage systems face limitations in handling large quantities of data due to memory constraints, where increasing physical memory is costly and may not be feasible, especially when addressable memory is limited or physical slots are unavailable.
Innovation Solution
Implementing a compression module in a storage server that compresses data blocks before storing them in system memory, allowing for increased data processing capacity without the need for additional physical memory modules, by using a buffer cache and nonvolatile memory to store and flush compressed data to physical storage devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If additional memory modules are added to increase memory capacity, then the data storage system can handle larger quantities of data, but the cost increases significantly
Solution Approach 1:
The patent applies compression algorithms to change the state of data from uncompressed to compressed form, reducing the space required in memory by a factor of 2-4 times while maintaining the same logical capacity, thereby avoiding the need to purchase additional expensive memory modules
Solution Approach 2:
The system creates compressed copies of data blocks in memory rather than storing full-size copies, allowing the same physical memory space to hold more data blocks, effectively increasing memory capacity without additional hardware
2Quantity of substance
If physical memory slots are limited or addressable memory is constrained, then the maximum memory capacity is limited, but the data storage system needs to handle larger quantities of data
Solution Approach 1:
By applying compression algorithms, the system changes the density parameter of stored data, allowing more data blocks to fit within the fixed physical memory constraints, effectively bypassing the limitation without requiring additional hardware slots or addressable memory space
3Speed
If uncompressed data is stored in buffer cache, then data access is fast, but the buffer cache capacity is limited by physical memory size
Solution Approach 1:
The system changes the compression state parameter of data blocks in the buffer cache, allowing compressed data to be stored in the same physical space, thereby increasing the logical buffer cache capacity to hold more data blocks while maintaining fast access speeds through selective decompression
Data Source
AI summary
A network adapter receives a request to store a data block. The data block is sent from the network adapter to a compression module. The compression module generates a compressed data block from the data block. The compressed data block or a reference to the compressed data block is stored in a buffer cache. The compressed data block is stored in nonvolatile memory. It is determined that the compressed data block should be flushed a storage device. In response to determining that the compressed data block should be flushed to the storage device, the compressed data block is flushed from the nonvolatile memory to the storage device.