Deduplication Hash Cylinder for Virtual Memory Expansion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional random-access memory (RAM) is limited in data storage capacity due to its physical design, and future data center applications require high capacity and low latency memory solutions to handle complex data-intensive tasks.
Innovation Solution
The implementation of deduplication algorithms and a combined data structure that includes a hash table, reference counter table, and signature table, allowing for context addressing and efficient data storage beyond physical memory limits by using hash functions to index and manage data within a hash cylinder structure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If physical RAM capacity is increased, then memory storage capacity is improved, but device cost and physical size increase proportionally
Solution Approach 1:
The patent creates virtual copies of physical memory lines through hash table entries. Each physical memory line can be referenced by multiple logical addresses via hash collisions, effectively creating multiple virtual instances from a single physical resource. This allows the system to present a larger virtual memory capacity without proportionally increasing physical RAM.
Solution Approach 2:
The patent makes physical memory lines serve multiple functions simultaneously by allowing them to be accessed through multiple logical addresses. A single physical memory line can store data that is referenced by multiple logical addresses in the hash table, enabling one physical resource to fulfill multiple memory access requests.
2Quantity of substance
If deduplication algorithms are implemented, then memory capacity utilization is improved, but processing overhead increases
Solution Approach 1:
The patent performs deduplication hashing in advance when data is written to memory. The hash function is applied to incoming data blocks before they are stored, and the resulting hash values are used to populate the hash table. This preliminary hashing action enables fast lookup and deduplication during read operations without requiring complex real-time processing.
Solution Approach 2:
The patent replaces complex comparison-based deduplication mechanisms with a simpler hash-based lookup system. Instead of comparing data blocks directly to identify duplicates, the system uses hash functions to map data to specific locations in the hash table, where duplicates naturally collide and can be identified through simple equality checks on hash values.
3Speed
If hash table structure is used, then data retrieval speed is improved, but memory overhead for data structures increases
Solution Approach 1:
The patent organizes the hash table into discrete buckets, each containing a limited number of entries. This local organization allows the system to manage memory usage in manageable units and enables selective caching of frequently accessed buckets. Not all hash table entries need to be kept in fast memory simultaneously, only those that are actively needed.
Solution Approach 2:
The patent divides the hash table into multiple buckets, where each bucket can be independently managed and cached. This segmentation allows the system to load only the necessary buckets into fast memory, reducing the overall memory overhead while maintaining fast retrieval speeds for accessed data. Less frequently accessed buckets can reside in slower memory or be loaded on demand.
Data Source
AI summary
A method of retrieving data stored in a memory associated with a dedupe module is provided. The method includes: identifying a logical address of the data; identifying a physical line ID of the data in accordance with the logical address by looking up at least a portion of the logical address in a translation table; locating a respective physical line, the respective physical line corresponding to the PLID; and retrieving the data from the respective physical line, the retrieving including copying a respective hash cylinder to the read cache, the respective hash cylinder including: a respective hash bucket, the respective hash bucket including the respective physical line; and a respective reference counter bucket, the respective reference counter bucket including a respective reference counter associated with the respective physical line.