Cache Memory Data Block Deduplication via Access Popularity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deduplication methods in storage systems lead to increased read latencies due to indiscriminate deduplication of data blocks on disk devices, as they often require a threshold number of sequential blocks before deduplication, and this approach is not effectively applied in cache memory, resulting in missed storage savings and prolonged read times.
Innovation Solution
Implement a system that deduplicates data blocks in cache memory based on their popularity, where sets of redundant blocks with a high number of accesses are identified and further deduplicated, and their deduplicated versions are stored, reducing storage space and read latencies by retaining popular data blocks in cache memory for longer periods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If indiscriminate deduplication is applied to all data blocks on storage devices, then storage space is saved, but read latency increases due to the threshold requirement and disk seek operations
Solution Approach 1:
The invention segments the storage system into two distinct layers: cache memory and disk storage. Deduplication is applied selectively to popular data blocks in the cache memory layer, while non-popular blocks remain in their original form on disk. This segmentation allows deduplication benefits to be realized for frequently accessed data without incurring the performance penalty of deduplicating all data on disk.
Solution Approach 2:
The invention applies different quality treatments to different portions of data based on their access characteristics. Popular data blocks that meet the deduplication threshold are deduplicated and stored in cache memory with optimized access patterns, while non-popular blocks are left undeduplicated on disk. This local quality approach ensures that deduplication is applied only where it provides net benefit.
2Reliability
If deduplication threshold requires a certain number of sequential blocks, then false deduplication is reduced, but read latency increases due to sequential access requirements
Solution Approach 1:
The invention dynamically adjusts the deduplication threshold based on data block popularity rather than using a fixed sequential block requirement. Data blocks are tracked for their access patterns, and the threshold for deduplication is adapted according to how frequently each block is accessed. This dynamic approach allows popular blocks to be deduplicated even if they don't meet strict sequential thresholds, while maintaining reliability for less popular data.
3Speed
If cache memory is used to store deduplicated data blocks, then read speed is improved, but storage capacity is reduced due to the smaller size of cache memory
Solution Approach 1:
The invention changes the parameter of deduplication application from a fixed threshold-based approach to a popularity-based approach. By tracking access patterns and identifying popular data blocks, the system dynamically determines which blocks are candidates for cache storage. This parameter change allows the system to optimize cache utilization by storing only the most frequently accessed deduplicated blocks, thereby maximizing read speed improvements while minimizing the impact on effective storage capacity.
Data Source
AI summary
Techniques for deduplicating data in cache memory include determining that a first data block stored in the cache memory matches a second data block stored in the cache memory. It is further determined that a number of accesses associated with at least one of the first data block or the second data block is equal to or greater than a threshold number of accesses. In response to determining that the number of accesses is equal to or greater than the threshold number of accesses, the first data block is deduplicated in the cache memory.