Virtual Pointer Deduplication for Storage Fragmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data deduplication techniques cause performance degradation due to read and write splits, resulting from the introduction of fragments when redirecting pointers, which breaks continuous data storage areas.
Innovation Solution
Implement a method that sets a virtual pointer for to-be-deduplicated data blocks and delays reclaiming storage space until available capacity is insufficient, using a dual-pointer structure to maintain access efficiency and reduce fragmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If pointer redirection is performed for deduplicated data blocks, then storage space is reclaimed, but read/write performance degrades due to read redirect and write split
Solution Approach 1:
The patent introduces a virtual pointer as an intermediary layer between the file system and physical data blocks. This virtual pointer maintains the original continuous address mapping, allowing read/write operations to proceed without performance degradation, while the underlying physical storage benefits from deduplication and space reclamation.
Solution Approach 2:
The patent separates the address mapping function into two independent components: physical pointers that manage actual storage locations and enable deduplication, and virtual pointers that maintain continuous address spaces for performance. This segmentation allows each component to optimize for its specific function without interfering with the other.
2Quantity of substance
If continuous data storage areas are broken into pieces during deduplication, then duplicate data is removed, but fragments are introduced causing write split
Solution Approach 1:
The patent resolves the fragmentation problem by operating in two different address spaces: the physical dimension where deduplication occurs and breaks continuity, and the virtual dimension where continuous addressing is maintained. This dimensional separation allows deduplication to function effectively while preserving the appearance of continuous storage to applications.
3Productivity
If virtual pointers are maintained for all to-be-deduplicated data blocks, then direct access performance is improved, but storage capacity is wasted
Solution Approach 1:
The patent implements a dynamic virtual pointer management system where virtual pointers are created for to-be-deduplicated blocks but can be cancelled and storage reclaimed when capacity is needed. This dynamic approach allows the system to adapt between performance optimization and space utilization based on current storage conditions.
Solution Approach 2:
The system changes the availability parameter of virtual pointers based on storage capacity conditions. When capacity is sufficient, virtual pointers remain active for performance; when capacity is constrained, virtual pointers are cancelled and storage is reclaimed, thus adapting the system behavior to current resource availability.
Data Source
AI summary
The present disclosure provides a method, apparatus and system for data deduplication comprising setting, for a to-be-deduplicated data block, a pointer pointed to the to-be-deduplicated data block as a pointer pointed to a corresponding to-be-reserved data block, and setting a virtual pointer pointed to the to-be-deduplicated data block, cancelling, in response to an available storage capacity reaching a predetermined value, virtual pointers of respective to-be-deduplicated data blocks to be reclaimed, and reclaiming storage spaces of the respective to-be-deduplicated data blocks to be reclaimed, wherein when the virtual pointer pointed to the to-be-deduplicated data block is available, the to-be-deduplicated data block is directly accessed through the virtual pointer.