Dynamic Comparison Block Management for Storage Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies due to the storage of multiple redundant data copies, outdated reference blocks, and challenges in managing storage space in Flash memories, leading to increased costs, latency, and reduced memory capacity.
Innovation Solution
A storage system with a non-volatile cache memory device and a storage controller that dynamically adapts comparison blocks using a data deduplication technique, manages the lifecycle of comparison blocks through global and local garbage collection, and optimizes storage resources by selecting active and passive sets of comparison blocks based on scores and events.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple copies of the same data are stored in the storage network, then data loss prevention is improved, but storage resource utilization deteriorates
Solution Approach 1:
The patent uses reference blocks as simplified copies that point to original data blocks, rather than storing full duplicate copies. This allows the system to maintain data redundancy for loss prevention while consuming minimal storage resources, as reference blocks occupy negligible space compared to actual data blocks.
Solution Approach 2:
The system dynamically manages reference blocks by deleting outdated ones when their referenced data blocks are no longer needed or have been superseded. This allows storage resources to be continuously recovered and reallocated, maintaining reliability while optimizing storage utilization over time.
2Productivity
If exact match algorithms are used to identify redundant data, then storage efficiency is improved, but measurement precision deteriorates
Solution Approach 1:
The patent changes the matching parameter from requiring exact matches to accepting matches based on similarity thresholds. This allows the system to identify substantially similar data blocks even with minor variations, improving detection precision while maintaining storage efficiency through deduplication.
3Measurement precision
If reference blocks are continuously updated to represent current data, then data accuracy is improved, but device complexity increases
Solution Approach 1:
The system implements dynamic reference block management where reference blocks are automatically updated, added, or deleted based on changing data patterns and usage. This dynamic approach maintains accurate data representation without requiring complex manual management, as the system adapts automatically to data lifecycle changes.
Solution Approach 2:
The system uses feedback mechanisms to monitor data block validity and reference block usefulness, automatically adjusting the reference block set based on this feedback. This maintains data representation accuracy while simplifying management through automated feedback-driven updates rather than complex manual processes.
4Stability of the object's composition
If outdated reference blocks are retained in limited storage space, then system stability is improved, but storage capacity deteriorates
Solution Approach 1:
The system implements automated deletion of outdated reference blocks when they no longer reference valid or actively used data blocks. This frees up storage capacity in the comparison block library while maintaining system stability by keeping reference blocks that are still relevant, achieving a balance between stability and capacity through selective discarding.
Data Source
AI summary
Various aspects for managing data blocks in a storage system are provided. For instance, a method may include storing, in a buffer memory device, a comparison block library, selecting a first set of comparison blocks in the comparison block library to create an active set of comparison blocks, and utilizing the active set in conjunction with a data deduplication technique. A method may further include determining an occurrence of a predetermined event in the data deduplication technique, selecting a second set of comparison blocks in the comparison block library to create a new active set of comparison blocks in response to the predetermined event, replacing the active set with the new active set, and utilizing the new active set in conjunction with the data deduplication technique.