Deduplication Metadata Copying Without Data Block I/O
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data deduplication systems face inefficiencies in copying data due to the need to read and write duplicate blocks, which consumes time and storage system bandwidth.
Innovation Solution
The method involves traversing deduplication metadata to update reference counts instead of reading and writing duplicate blocks, creating new logical block pointers and incrementing reference counts in the deduplication table to efficiently copy data within the system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data blocks are read and written to copy data in a deduplication system, then data copying can be performed, but time and storage system bandwidth are consumed
Solution Approach 1:
The patent applies the copying principle by creating metadata copies (fingerprints and block pointers) instead of copying actual data blocks. When data needs to be copied in a deduplication system, the system generates new metadata entries that reference the same physical data blocks, thereby achieving data copying without moving the actual data and eliminating I/O operations.
Solution Approach 2:
The patent replaces the mechanical data reading and writing process with a computational metadata manipulation process. Instead of physically reading data blocks from storage and writing them to new locations, the system substitutes this mechanical I/O operation with computational operations on metadata structures (creating fingerprints, generating block pointers, updating reference counts), which is significantly faster and requires no data movement.
2Productivity
If data blocks are read and written to copy data in a deduplication system, then data copying can be performed, but storage system internal bandwidth is consumed
Solution Approach 1:
The patent applies the copying principle by creating metadata copies (fingerprints and block pointers) instead of copying actual data blocks. When data needs to be copied in a deduplication system, the system generates new metadata entries that reference the same physical data blocks, thereby achieving data copying without moving the actual data and eliminating I/O operations.
Solution Approach 2:
The patent replaces the mechanical data reading and writing process with a computational metadata manipulation process. Instead of physically reading data blocks from storage and writing them to new locations, the system substitutes this mechanical I/O operation with computational operations on metadata structures (creating fingerprints, generating block pointers, updating reference counts), which is significantly faster and requires no data movement.
3Productivity
If traditional copying methods are used in a deduplication system, then data can be copied, but the system cannot efficiently track where duplicates are stored
Solution Approach 1:
The patent introduces metadata as an intermediary between the physical data blocks and the logical copy operations. The metadata layer (comprising fingerprints and block pointers) serves as a mediator that tracks all references to data blocks without requiring actual data movement. This intermediary layer enables efficient tracking of duplicate locations while maintaining the deduplication structure.
Solution Approach 2:
The patent segments the data storage system into two distinct layers: the physical data block layer and the metadata layer. By separating these functions, the system can manage data copies through metadata operations independent of actual data movement, enabling efficient tracking of duplicate locations without interfering with the physical storage structure.
Data Source
AI summary
A method for copying data efficiently within a deduplicating storage system eliminates the need to read or write the data per se within the storage system. The copying is accomplished by creating duplicates of the metadata block pointers only. The result is a process that creates and arbitrary number of copies using minimal time and bandwidth.