Nearstore Data Compression Groups for Memory Capacity Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage systems face limitations in handling large quantities of data due to limited memory, making it costly and impractical to increase memory capacity, and existing solutions do not effectively enhance performance without significant hardware upgrades.
Innovation Solution
A storage server compresses data blocks into compression groups, storing them in non-volatile memory and flushing them to physical storage devices at consistency points, thereby increasing data processing capacity without the need for additional physical memory modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If additional memory modules are physically included to increase memory capacity, then the ability to handle larger quantities of data is improved, but the cost increases significantly
Solution Approach 1:
The patent applies compression algorithms to change the state of data from uncompressed to compressed form, reducing the space required in memory by a factor of 2-4 times. This parameter change allows the same physical memory to effectively handle larger quantities of data without adding hardware
Solution Approach 2:
The patent creates compressed copies of data that occupy less space in memory while preserving the original information. These compressed representations allow more data to be cached in available memory, improving performance without additional hardware cost
2Productivity
If memory capacity is increased to handle larger data quantities, then data processing performance is improved, but hardware complexity increases
Solution Approach 1:
By changing the compression parameter of data stored in memory, the system effectively increases memory capacity without adding physical components. The same memory hardware can hold more data when compression is applied, avoiding hardware complexity increases
3Quantity of substance
If physical memory slots are limited, then memory expansion is constrained, but data handling capacity remains insufficient
Solution Approach 1:
The patent overcomes physical memory slot limitations by changing the information density parameter through compression. Instead of expanding hardware, the system increases effective capacity by storing compressed data, making the system adaptable to fixed hardware constraints
Data Source
AI summary
A storage server is configured to receive a request to store a data block from a client. The request to store the data block is serviced by the storage server by compressing the data block into a compression group, which includes a number of compressed data blocks. The storage server stores the compression group in a non-volatile memory and flushes the compression group from the non-volatile memory to a physical storage device in response to reaching a consistency point. By compressing data to be stored in system memory of a storage server, the amount of data that can be processed during a given time period by a data storage system is increased. Furthermore, an increase in performance can be achieved at a lower cost, since the cost of additional physical system memory modules can be avoided.