DRAM Deduplication Hash Table Virtual Buckets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data deduplication methods in memory devices increase read latency and memory bandwidth, and do not effectively address the need for fine-grain deduplication in volatile memory like DRAM, which limits the capacity and lifespan of memory devices.
Innovation Solution
A deduplication memory module with a hash table memory system that includes physical and virtual buckets, an address lookup table memory, and a buffer memory, which performs deduplication algorithms to store and retrieve data blocks efficiently, allowing for adaptive and non-adaptive deduplication strategies based on application patterns and memory controller information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional in-memory deduplication technology is used, then data redundancy is reduced, but read latency increases and memory bandwidth is reduced
Solution Approach 1:
The patent segments the deduplication process into multiple independent hash tables, each handling a portion of the data space. This allows parallel processing of deduplication operations across different hash tables, reducing the sequential read latency while maintaining effective deduplication of data blocks.
Solution Approach 2:
The patent introduces virtual buckets as an additional dimension beyond physical buckets, creating a multi-level hash table structure. Virtual buckets map to physical buckets and enable more efficient data organization and retrieval, reducing read latency by optimizing the search space dimensionality.
2Quantity of substance
If data deduplication is implemented in volatile memory, then memory capacity is increased, but device complexity increases
Solution Approach 1:
The patent divides the deduplication system into multiple hash tables with physical and virtual buckets, segmenting the data management function. This modular approach increases memory capacity through efficient data organization while keeping individual components manageable and reducing overall system complexity.
Solution Approach 2:
The patent uses virtual buckets as copies or representations of physical buckets, creating an abstraction layer. This copying mechanism allows the system to manage data blocks efficiently in volatile memory, increasing effective capacity while the virtual layer simplifies the interface between data blocks and physical storage.
3Reliability
If write requests for duplicated blocks are processed, then data consistency is maintained, but write endurance is reduced
Solution Approach 1:
The patent uses hash table entries as copies that point to actual data blocks. When a write request comes in, the system checks the hash table for duplicate entries rather than writing to physical memory directly. This copying approach maintains data consistency through reference tracking while avoiding redundant physical writes, thereby extending write endurance.
Solution Approach 2:
The hash table acts as an intermediary between write requests and physical memory. It mediates the write operation by first checking for duplicates and only initiating physical writes when necessary. This intermediary layer maintains data consistency through proper reference management while significantly reducing the number of physical write operations, thus protecting write endurance.
Data Source
AI summary
A deduplication memory module, which is configured to internally perform memory deduplication, includes a hash table memory for storing multiple blocks of data in a hash table array including hash tables, each of the hash tables including physical buckets and a plurality of virtual buckets each including some of the physical buckets, each of the physical buckets including ways, an address lookup table memory (ALUTM) including a plurality of pointers indicating a location of each of the stored blocks of data in a corresponding one of the physical buckets, and a buffer memory for storing unique blocks of data not stored in the hash table memory when the hash table array is full, a processor, and memory, wherein the memory has stored thereon instructions that, when executed by the processor, cause the memory module to exchange data with an external system.