Master-Slave Deduplication for Solid State Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deduplication techniques are not optimized for faster performance, especially with newer storage media like solid state storage, which requires efficient data management to extend drive lifespan and improve network performance.
Innovation Solution
A master-slave deduplication process that uses address mapping and fingerprinting to identify and eliminate redundant data, with the address mapping process acting as the master and fingerprinting process as the slave, ensuring parallel operation to avoid race conditions and optimize resource usage, particularly suitable for solid state storage systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional sequential deduplication processing is used, then data deduplication is achieved, but processing speed is slow and does not scale well with newer storage media
Solution Approach 1:
The patent divides the deduplication process into two independent segments: address mapping process and fingerprinting process. These segments operate in parallel without interfering with each other, significantly improving processing speed while maintaining manageable complexity through clear functional separation.
Solution Approach 2:
The patent introduces a master-slave relationship dimension to the processing architecture. The address mapping process acts as master while the fingerprinting process acts as slave, enabling coordinated parallel execution that improves throughput without requiring complex synchronization mechanisms.
2Productivity
If multiple processes operate simultaneously for parallel processing, then processing speed improves, but race conditions and data conflicts may occur
Solution Approach 1:
By segmenting the deduplication process into independent address mapping and fingerprinting processes, the patent eliminates race conditions. Each process operates on its own data structures (address table and fingerprint table respectively), ensuring data consistency while achieving parallel processing.
Solution Approach 2:
The patent uses a queue as an intermediary mechanism between the master address mapping process and slave fingerprinting process. This queue facilitates coordinated data exchange while maintaining process independence and preventing direct conflicts between simultaneous operations.
3Quantity of substance
If duplicate data is stored in traditional systems, then data redundancy is handled simply, but storage efficiency is low and network bandwidth is wasted
Solution Approach 1:
The patent segments data identification into two independent tables: address table for logical-to-physical mapping and fingerprint table for data similarity detection. This segmentation enables efficient duplicate detection and storage management by processing address mapping and fingerprinting operations independently and in parallel.
Solution Approach 2:
Instead of storing actual duplicate data, the patent creates references (pointers) to the original data block. The address table stores mappings between logical addresses and physical block addresses, allowing multiple logical references to point to a single physical storage location, thereby eliminating redundancy.
4Adaptability or versatility
If simple address mapping is used, then implementation is straightforward, but it cannot handle solid state storage specific requirements like wear leveling
Solution Approach 1:
The patent segments the address mapping process to separate logical address mapping from physical block allocation. This allows the address mapping process to handle SS-specific requirements like wear leveling and garbage collection independently, while maintaining a simple and straightforward implementation for standard deduplication functionality.
Data Source
AI summary
A write instruction includes a logical address and write data to be stored. An address mapping master is used to determine if the logical address is stored in an address table. A deduplication state is selected based at least in part on whether the logical address is stored in the address table and whether a fingerprint is stored in a fingerprint table. The fingerprint is generated using the write data. A fingerprinting slave is used to determine if the fingerprint is stored in the fingerprint table, where the address mapper and the fingerprinter are configured to run in parallel and the address mapper is the master to the fingerprinter's slave.