Compressed Memory Deduplication for Swap I/O Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The issue of duplicate storage and inefficient use of memory and input/output resources arises when pages are repeatedly swapped in and out of a host CPU's memory, leading to duplicate storage problems.
Innovation Solution
Implementing a deduplication mechanism using a controller to manage a deduplication table that identifies and deletes identical compressed data in volatile memory, along with a mechanism to move cold data to nonvolatile memory, optimizing memory usage and reducing duplicates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If pages are swapped out to storage to maximize memory utilization, then memory capacity is improved, but duplicate storage occurs leading to inefficient use of storage and I/O resources
Solution Approach 1:
The controller performs preliminary compression on data pages before swapping them out to storage. By compressing the data in advance, the system reduces the amount of data stored, preventing duplicate storage of uncompressed data and improving storage efficiency while maintaining memory utilization benefits
Solution Approach 2:
The system creates hash value copies of data pages and stores these lightweight representations in a hash table. Instead of storing complete duplicate data pages, only hash values are copied and compared, enabling efficient duplicate detection with minimal storage overhead and reduced I/O operations
2Adaptability or versatility
If the same page is loaded into memory multiple times and then swapped out, then memory flexibility is improved, but duplicate storage problems occur
Solution Approach 1:
The controller implements a feedback mechanism using hash value comparison to detect duplicate data pages before swapping them out. When a hash match is found, the system provides feedback to prevent swapping the duplicate, thereby maintaining memory flexibility while avoiding storage efficiency losses from redundant data
Solution Approach 2:
The system changes the parameter of data representation by compressing data pages and using hash values instead of complete data copies for comparison. This parameter transformation enables efficient duplicate detection and prevents storage of redundant compressed data, resolving the contradiction between memory flexibility and storage efficiency
Data Source
AI summary
An electronic device includes a host processor, a volatile memory configured to store a plurality of compressed data, and a controller configured to be connected to the host processor and the volatile memory, wherein the controller is configured to generate a deduplication table that stores hash values associated with respective ones of the plurality of compressed data, identify, based on the deduplication table, first compressed data and second compressed data among the plurality of compressed data that correspond to each other, and delete one of the first compressed data and the second compressed data from the volatile memory.