Weak References for Online Block Deduplication Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Identifying and handling redundant data stored by a file system is resource-intensive and may introduce additional complexity to read and write operations, with existing deduplication techniques potentially causing data corruption if not properly implemented.
Innovation Solution
The use of a deduplication hash table with weak references to associate block hashes and block addresses, where a weak reference is generated on a block when added to the table, and is broken upon modification, ensuring data integrity by using intact references for deduplication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If deduplication techniques are implemented to identify and handle redundant data, then data storage efficiency is improved, but system complexity and resource consumption increase
Solution Approach 1:
The patent introduces a weak reference mechanism as an intermediary between the deduplication hash table and the actual data blocks. This weak reference acts as a lightweight mediator that tracks block integrity without requiring complex locking or synchronization protocols, thereby reducing system complexity while maintaining deduplication effectiveness
Solution Approach 2:
The patent extracts the integrity verification function from the main deduplication process by using weak references that automatically break when blocks are modified. This separation allows the deduplication system to operate independently without burdening it with complex integrity checking logic, reducing overall system complexity
2Quantity of substance
If deduplication techniques are implemented to identify and handle redundant data, then data storage efficiency is improved, but processor and disk utilization increase
Solution Approach 1:
The patent applies partial action by using weak references that only track integrity for blocks that have been deduplicated. Not all blocks require weak references—only those in the deduplication hash table. This selective approach reduces processor and disk utilization compared to comprehensive integrity tracking schemes
Solution Approach 2:
The weak reference is implemented as a lightweight, disposable data structure that requires minimal processor and disk resources. When a block is modified, the weak reference is simply broken or removed rather than requiring complex update operations, reducing overall resource consumption
3Reliability
If weak references are used to track block integrity for deduplication, then data corruption is prevented, but additional overhead is introduced
Solution Approach 1:
The weak reference mechanism is self-service in that it automatically breaks when the underlying block is modified. This eliminates the need for complex active monitoring or manual integrity checking, reducing overhead while maintaining reliable data corruption prevention
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Aspects of the present disclosure relate to weak references of allocated logical clusters. In examples, a deduplication hash table stores associations between block hashes and block addresses, such that a block can be identified and used in place of another block storing the same data. A weak reference may be generated on a block, such that an intact weak reference indicates that the block is unchanged. When a block is identified that has a hash matching that of an existing block in the hash table, it may be determined whether a weak reference on the existing block is intact. If there is not an intact weak reference on the existing block, the existing block may have changed, such that it is not used for deduplication. By contrast, if the weak reference is intact, the existing block has not changed, such that it may be used for deduplication accordingly.