Storage Deduplication with Segmented Placement for Smaller Fingerprint Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deduplication technologies face challenges due to the large size of fingerprint tables, which consume significant storage space and impact performance.
Innovation Solution
A method for deduplication in storage systems that involves storing duplicate data blocks in higher-performance storage areas only after multiple occurrences, using a fingerprint table to manage mappings efficiently, and optimizing storage resource allocation based on performance tiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If deduplication operation is performed on all duplicate data blocks, then storage space is saved, but the fingerprint table becomes large and performance deteriorates
Solution Approach 1:
The patent segments the storage system into two distinct storage areas: a first storage area for storing unique data blocks and a second storage area for storing duplicate data blocks. This segmentation allows the system to maintain separate management strategies for unique and duplicate data, reducing the burden on the fingerprint table while preserving storage space efficiency.
Solution Approach 2:
The patent applies different quality characteristics to different storage areas by assigning the first storage area for unique data and the second storage area for duplicate data. This local differentiation enables optimized performance for duplicate data operations while maintaining comprehensive deduplication functionality.
2Reliability
If fingerprint table is queried for every write operation, then data integrity is ensured, but processing time increases
Solution Approach 1:
The patent performs preliminary classification of data blocks into unique and duplicate categories before final storage. By pre-determining whether a data block is a duplicate through efficient fingerprint comparison, the system avoids unnecessary full fingerprint table queries for every write operation, reducing processing time while maintaining data integrity.
3Device complexity
If all duplicate data blocks are stored in the same storage area, then management is simplified, but access performance decreases
Solution Approach 1:
The patent divides the storage system into multiple storage areas with distinct purposes: the first storage area stores unique data blocks while the second storage area stores duplicate data blocks. This segmentation enables optimized access patterns for different data types, improving overall access performance while maintaining manageable complexity through clear separation of concerns.
Data Source
Figure 1~2
Figure 3
Figure 4~8
AI summary
A solution to deduplication in a storage system is disclosed herein. In this solution, a controller stores, at a first storage address, a data block that is written for the first time, and inserts, in a fingerprint table, a first fingerprint of first to-be-written data that is written for the first time, but does not establish a mapping relationship between the first fingerprint and the first storage address. When receiving subsequently written second to-be-written data, the controller allocates a second storage address to the duplicate second to-be-written data, stores the second to-be-written data at the second storage address, and establishes a mapping relationship between the first fingerprint and the second storage address in the fingerprint table. This reduces an amount of data in the fingerprint table.