Storage Deduplication Metadata Suppression for Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The processing time for deduplication in storage apparatuses becomes longer when comparing data byte by byte, leading to increased management information size, which hampers access performance, especially when using hash values for determination.
Innovation Solution
A storage apparatus performs deduplication on write data and records metadata only under certain conditions, suppressing the size of management information required for the deduplication process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If hash values are used for deduplication determination, then processing speed is improved, but management information size increases
Solution Approach 1:
The patent extracts only the necessary portion of data for deduplication determination by using hash values of fixed length (e.g., 128 bits) instead of storing or comparing entire data sets. This extraction approach maintains fast comparison speed while limiting the growth of management information size, as the hash table stores only these fixed-length hash values rather than complete data copies.
2Reliability
If management information size increases, then deduplication accuracy is improved, but access performance deteriorates
Solution Approach 1:
The patent changes the parameter of hash value length to an optimal fixed value (e.g., 128 bits) that provides sufficient deduplication accuracy while preventing management information from occupying excessive cache memory. This parameter optimization ensures that the hash table remains compact enough for fast access while maintaining reliable duplication detection capability.
3Measurement precision
If more management information is stored in cache memory, then deduplication precision is improved, but cache memory consumption increases
Solution Approach 1:
The patent uses hash values as copies or representations of the actual data for deduplication purposes, rather than storing complete data copies in the cache. This copying approach allows the system to maintain deduplication precision by comparing hash values (which uniquely represent data content) while consuming minimal cache memory, as the hash table occupies only a small fraction of available cache space.
Data Source
AI summary
A storage apparatus according to an aspect of the present invention is configured to perform a deduplication process on write data from a host. For the deduplication process, the storage apparatus calculates a feature value of write data, and records, in a management region, metadata that is a set of the feature value of the write data and information on a storage position of the write data. However, to prevent the amount of metadata stored in the management region from increasing, if write data meets a predetermined condition, the storage apparatus does not perform the deduplication process, and suppresses creation of metadata of the write data.