Storage Device Cache Management for Deduplication Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage devices with deduplication functions face challenges in maintaining retrieval performance due to the need for reserved storage capacity and difficulties in handling repeated block data, leading to decreased performance and inefficient cache management.
Innovation Solution
A storage system that employs a content-addressable architecture with a cache management policy based on metadata, where block data is stored and retrieved in a specific order using content addresses, and the cache is controlled to optimize retention and eviction of block data based on retrieval turns, reducing unnecessary storage and improving access efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If deduplication is performed by storing new data at the end of storage area, then storage capacity is improved, but retrieval performance deteriorates due to dispersed block data requiring huge number of disk operations
Solution Approach 1:
The storage device is divided into multiple storage areas (first storage area for deduplicated data, second storage area for sequential data). This segmentation allows different data to be stored in different locations, enabling both efficient deduplication and fast retrieval by accessing the appropriate storage area based on data characteristics.
Solution Approach 2:
A cache memory is introduced as an intermediary between the host device and storage media. The cache memory stores frequently accessed block data, acting as a buffer that reduces the need for direct disk access and improves retrieval performance while maintaining the deduplication benefits.
2Productivity
If both first storage area for deduplicated data and second storage area for sequential data are reserved, then retrieval performance is improved, but storage capacity consumption increases
Solution Approach 1:
The system dynamically manages the storage areas by allowing the first storage area to be overwritten when full, and by managing cache memory entries based on access patterns. This dynamic behavior enables the system to maintain performance while optimizing capacity usage without requiring permanent reservation of both storage areas.
Solution Approach 2:
The system discards old cache entries and allows overwriting of the first storage area when capacity is exhausted. By selectively removing least recently used data and recovering storage space, the system maintains retrieval performance for active data while reducing overall storage consumption.
3Productivity
If block data is stored sequentially in storage area, then retrieval performance is improved, but duplicated storage of same content cannot be eliminated
Solution Approach 1:
The system segments data into two types: deduplicated data stored in the first storage area and sequential data stored in the second storage area. This segmentation allows the system to eliminate duplicates where possible while maintaining sequential access patterns for performance-critical data in the appropriate storage area.
Solution Approach 2:
The cache memory creates copies of frequently accessed block data, allowing multiple references to the same data without physically storing multiple copies in the main storage area. This copying mechanism enables efficient retrieval while reducing the need for duplicated storage of the same content.
Data Source
Figure 1
Figure 2
Figure 3~5
AI summary
A storage device has: a data storage part storing deduplicated block data;a temporary data storage part temporarily storing block data acquired from the data storage part; a data retrieval control part retrieving the block data stored by the data storage part, storing the block data into the temporary data storage part, and retrieving the block data from the temporary data storage part; and a temporary data control part controlling the storage state of the block data stored by the temporary data storage part.The storage device also has a retrieval turn information storage part storing retrieval turn information which is information about a turn to be retrieve of the block data. The data retrieval control part causes the temporary data storage part to store the block data acquired from the data storage part on the basis of the retrieval turn information acquired from the retrieval turn information storage part, and the temporary data control part controls the storage state of the block data in the temporary data storage part on the basis of the retrieval turn information.