Nonvolatile Memory De-duplication via Hash Key Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As storage capacities increase, existing nonvolatile memory devices face inefficiencies in utilizing storage space due to duplication and similarity of data, leading to overlapping data and excessive resource usage in managing de-duplication tables.
Innovation Solution
A nonvolatile memory device with a memory controller that generates hash keys for write requested data, compares them with reference hash keys, performs XOR operations and compression, and manages a de-duplication table to prevent data overlap, while minimizing the size of the de-duplication table by erasing less frequently referenced data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data de-duplication is performed to prevent data overlap, then storage space efficiency is improved, but the size of de-duplication table increases
Solution Approach 1:
The patent divides the de-duplication table into a hash manage table and a de-duplication matching table. The hash manage table stores only hash key information for reference data, while the de-duplication matching table stores mapping relationships between logical addresses and physical addresses. This segmentation reduces the overall table size while maintaining de-duplication functionality.
Solution Approach 2:
The patent extracts only the essential hash key information from the full reference data and stores it in the hash manage table. The complete reference data remains in the nonvolatile memory, and only the hash keys are used for de-duplication comparisons, reducing the memory footprint of the management structure.
2Device complexity
If de-duplication table size is reduced to minimize resource usage, then storage efficiency is improved, but data overlap prevention capability deteriorates
Solution Approach 1:
The patent introduces a hash function as an intermediary that transforms the original data into a compact hash key representation. This hash key serves as a mediator between the full reference data and the de-duplication matching table, enabling efficient comparisons without storing complete data copies, thus maintaining reliability while reducing table size.
Solution Approach 2:
The patent creates a simplified copy of the reference data structure - specifically, only the hash key information is copied into the hash manage table, while the full reference data remains in the nonvolatile memory. This selective copying maintains the essential de-duplication capability without the memory overhead of storing complete data copies.
3Measurement precision
If hash key generation and comparison operations are performed, then data similarity determination is improved, but processing time increases
Solution Approach 1:
The patent pre-calculates and stores hash keys for all reference data in the nonvolatile memory before actual write operations occur. When a new write request arrives, the system only needs to generate a hash key for the new data and compare it with pre-computed reference hash keys, significantly reducing processing time compared to computing hashes on-the-fly during write operations.
Data Source
AI summary
A storage device is provided which includes a nonvolatile memory device configured to store a plurality of reference data, a memory configured to store a hash manage table used to manage a plurality of reference hash keys of each of the plurality of reference data, a hash key generator configured to generate a plurality of hash keys based on write requested data, and a memory controller configured to compare the plurality of hash keys and reference hash keys of each reference data to determine whether to store the write requested data in the nonvolatile memory device. The memory controller selects one of the plurality of reference data according to a similarity between the plurality of hash keys and the plurality of reference hash keys of each reference data and stores the write requested data and the selected reference data in the nonvolatile memory device to refer to each other.