IMS Memory Deduplication for Efficient Data Storage Writes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data de-duplication methods suffer from inefficient storage space usage and increased data swapping, leading to reduced efficiency.
Innovation Solution
A storage system and method that utilizes an in-memory search (IMS) memory device to compare user data features with stored hash values, determining whether to write data based on the comparison results, thereby optimizing storage space usage and reducing data swapping.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data de-duplication is enabled and hash values are stored in a hash table, then data duplication can be detected, but storage space usage efficiency decreases as non-duplicated data grows
Solution Approach 1:
The patent extracts the hash value comparison function from the main storage system and implements it in a dedicated IMS (Instant Memory Search) memory device. This separates the de-duplication detection mechanism from the primary storage, allowing the hash table to be minimized while maintaining de-duplication capability. The IMS device handles the comparison operations externally, enabling the main storage to focus on data retention rather than comparison operations.
Solution Approach 2:
The IMS memory device acts as an intermediary between the data controller and the storage system. It receives data blocks from the controller, performs hash value comparisons using its dedicated search capability, and returns comparison results without requiring the main storage system to maintain a large hash table. This intermediary structure resolves the contradiction by enabling de-duplication detection while minimizing storage space requirements.
2Reliability
If hash values are frequently swapped between storage space and memory, then data de-duplication can be performed, but data de-duplication efficiency decreases due to increased data swapping
Solution Approach 1:
The patent merges the hash value storage and comparison functions into a single IMS memory device that is closely coupled with the data controller. This integration eliminates the need for frequent data swapping between separate storage and memory components. The IMS device maintains hash values in its internal memory structure and performs comparisons in-place, significantly reducing the overhead associated with data swapping and improving de-duplication efficiency.
Solution Approach 2:
The patent replaces the traditional mechanical data swapping process with an electronic search mechanism in the IMS memory device. Instead of physically moving hash values between storage and memory through swapping operations, the IMS device uses its specialized architecture to perform instant memory search and comparison operations electronically. This substitution eliminates the performance penalty associated with data swapping while maintaining de-duplication functionality.
Data Source
AI summary
A data storage method includes: generating a first user data feature of a user data stored in a random access memory by a data controller; sending the first user data feature of the user data by the data controller to an IMS memory device; comparing the first user data feature with a plurality of second user data features from a plurality of characteristic values stored in the IMS memory device to generate a comparison result, the plurality of characteristic values including the plurality of second user data features and a plurality of physical block addresses corresponding to data stored in a storage device; sending the comparison result to the data controller by the IMS memory device; and based on the comparison result, determining whether to write the user data into the storage device by the data controller.