Storage Deduplication via Tree Structure Merging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data deduplication in storage systems incurs significant overhead, impacting performance due to the need for identifying and removing duplicate data across multiple locations, which existing techniques fail to minimize effectively.
Innovation Solution
Implementing a method using a storage control system that generates content signatures for data blocks, merges entries in a tree data structure, and initiates deduplication when matching keys are found, leveraging a persistent deduplication database with a log-structured merge (LSM) or B-epsilon tree structure to efficiently identify and eliminate duplicates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data deduplication is implemented to reduce storage data amount, then storage efficiency is improved, but system performance deteriorates due to overhead
Solution Approach 1:
The patent segments the deduplication database into multiple elements organized as a tree data structure, where each element contains a portion of the deduplication entries. This segmentation allows parallel processing during merges and reduces the overhead impact on system performance while maintaining effective data deduplication across the storage system.
Solution Approach 2:
The patent performs preliminary actions by pre-computing content signatures for data blocks and organizing them into a tree-structured deduplication database before actual deduplication operations. The tree structure is prepared with elements that can be efficiently merged, allowing deduplication to occur naturally during merge operations without requiring additional real-time processing overhead.
2Quantity of substance
If deduplication operations are performed to identify and remove duplicate data, then storage capacity is optimized, but processing overhead increases
Solution Approach 1:
The patent combines multiple elements of the tree data structure into fewer elements through merge operations. During these merges, duplicate data is automatically identified when matching keys are found across different elements. This merging approach efficiently reduces processing time by handling multiple deduplication tasks in a single operation rather than processing each data block individually.
Solution Approach 2:
The deduplication process serves itself by utilizing the tree structure's natural merge operations to identify duplicates. The system automatically detects matching content signatures during routine tree maintenance and merge operations, eliminating the need for separate, resource-intensive deduplication scans. The tree structure self-organizes to facilitate efficient duplicate detection during normal database operations.
3Measurement precision
If a persistent deduplication database is maintained to track data blocks, then deduplication accuracy is improved, but system complexity increases
Solution Approach 1:
The patent introduces a hierarchical dimension to the deduplication database by organizing entries into a tree structure with multiple levels and elements. This dimensional organization maintains precise tracking of data block content signatures while managing complexity through hierarchical abstraction. The tree structure allows efficient navigation and merging operations that would be infeasible in a flat database structure.
Solution Approach 2:
The patent changes the structural parameters of the deduplication database from a traditional flat structure to a tree-based hierarchical structure. This parameter change enables the database to maintain high deduplication accuracy through content signature matching while managing complexity through the tree's inherent organization. The tree structure's parameters (elements, levels, merge operations) provide scalable management of the deduplication information.
Data Source
AI summary
A storage control system receives a first data block to be written to a primary storage, and generates a content signature for the first data block. The storage control system adds a first entry for the first data block into a persistent deduplication database. The first entry comprises a key which comprises the content signature for the first data block. The persistent deduplication database comprises a tree data structure which comprises elements that are configured to store entries for data blocks. The storage control system merges the entries of at least two elements of the tree data structure to generate a set of merged entries which comprises the first entry for the first data block, and a second entry for a second data block, and commences a deduplication process in response to determining that the first entry and the second entry in the set of merged entries have matching keys.