Predefined Block Pointers for Storage Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deduplication processes in storage systems face inefficiencies due to 'hot spots' and disk fragmentation caused by redundant allocation and deallocation of duplicate data blocks, particularly in virtualization environments where zero-filled blocks are misidentified as duplicates, leading to unnecessary processing and storage overhead.
Innovation Solution
Implement a method for specially allocating data blocks by pre-allocating 'special' data on disk, such as zero-filled blocks, and assigning special pointers, allowing these blocks to be accessed from memory instead of disk, thereby avoiding redundant writes and reads, and reducing fragmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If deduplication process fingerprints and compares all data blocks including zero-filled blocks, then storage space is reduced by eliminating duplicates, but processing time and overhead increase unnecessarily
Solution Approach 1:
The patent segments data blocks into two categories: regular data blocks that require fingerprinting and comparison, and special data blocks (zero-filled blocks) that are identified by their content characteristics. This segmentation allows the system to apply different processing rules to different types of blocks, avoiding unnecessary fingerprinting of zero-filled blocks while still achieving deduplication for regular data blocks.
Solution Approach 2:
The patent implements preliminary identification of zero-filled blocks before the fingerprinting process. By detecting that a block contains only zeros upfront, the system can skip the computationally expensive fingerprinting and comparison steps, performing the deduplication action in advance by directly referencing the special data block pointer without full verification.
2Quantity of substance
If single instance of zero-filled block is shared by multiple virtual disk image files, then storage efficiency improves, but hot spots occur on host disk due to frequent access
Solution Approach 1:
The patent introduces an intermediary layer (the storage system's special data block management) between the virtual disk image files and the physical host disk. Instead of directly sharing the same physical block instance across multiple files (which causes hot spots), the system creates special data block pointers that reference zero-filled content, acting as intermediaries that eliminate the need for frequent physical disk access while maintaining the logical sharing of zero-filled data.
Solution Approach 2:
The patent creates copies of zero-filled data in the form of special data block pointers rather than physical block instances. Each virtual disk image file contains references to zero-filled content through these special pointers, allowing multiple files to logically share zero-filled data without requiring physical duplication or shared access to the same disk blocks, thereby eliminating hot spots.
3Quantity of substance
If deduplication process deallocates and replaces duplicate blocks with pointers, then storage consumption is reduced, but disk fragmentation occurs
Solution Approach 1:
The patent treats zero-filled data blocks as disposable special cases that don't require permanent physical storage. Instead of allocating and deallocating physical disk blocks for zero-filled content, the system uses inexpensive special data block pointers that reference zero-filled content in memory or pre-allocated space. This approach eliminates the allocation/deallocation cycle that causes fragmentation while still achieving the storage savings of deduplication.
4Object-generated harmful factors
If maximum number of shared block references is predefined (e.g., 255), then hot spots are reduced by distributing references, but device complexity increases
Solution Approach 1:
The patent applies local quality by treating zero-filled blocks differently from regular data blocks. Instead of applying a uniform reference distribution mechanism across all data blocks (which would increase complexity), the system identifies zero-filled blocks by their local content characteristic and applies a specialized handling rule specifically to them. This localized approach eliminates hot spots for zero-filled data without imposing the complexity of reference distribution mechanisms on the entire storage system.
Data Source
AI summary
A method and system for eliminating the redundant allocation and deallocation of special data on disk, wherein the redundant allocation and deallocation of special data on disk is eliminated by providing an innovate technique for specially allocating special data of a storage system. Specially allocated data is data that is pre-allocated on disk and stored in memory of the storage system. “Special data” may include any pre-decided data, one or more portions of data that exceed a pre-defined sharing threshold, and/or one or more portions of data that have been identified by a user as special. For example, in some embodiments, a zero-filled data block is specially allocated by a storage system. As another example, in some embodiments, a data block whose contents correspond to a particular type document header is specially allocated.