File System Deduplication via Block Pointer Redirection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional file systems face inefficiencies in managing and deduplicating data blocks, leading to excessive storage requirements due to the lack of effective methods for identifying and managing duplicate data blocks across files.
Innovation Solution
The method involves adding a deduplication identifier to each data block pointer, detecting duplicates, determining a master copy, and redirecting pointers to share data blocks, utilizing a lookup table to track deduplicated blocks and manage them efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional deduplication methods are used to reduce storage space, then the amount of physical storage required is reduced, but the ability to efficiently identify and manage duplicate data blocks across files is insufficient
Solution Approach 1:
The patent segments data into fixed-size blocks and assigns unique identifiers to each block. By dividing files into discrete blocks rather than treating files as whole units, the system can efficiently identify and deduplicate individual blocks across multiple files, improving both storage reduction and deduplication efficiency
Solution Approach 2:
The patent creates a mapping structure that copies reference information about data blocks without duplicating the actual data content. The inode structure maintains references to data blocks, allowing multiple inodes to point to the same data block, enabling efficient deduplication while maintaining file access capabilities
2Quantity of substance
If multiple files share the same data blocks through deduplication, then storage requirements are minimized, but the complexity of managing and tracking duplicate blocks increases
Solution Approach 1:
The patent introduces an intermediary mapping structure (inode with block pointers) that mediates between files and actual data blocks. This intermediary layer tracks which files reference which blocks without requiring direct management of the block sharing relationships, reducing the complexity of managing deduplicated data
Solution Approach 2:
The patent adds a new dimension of organization by creating a two-level hierarchy: files organized into inodes, and inodes organized with pointers to data blocks. This dimensional separation allows efficient tracking of block references without increasing the complexity of the underlying storage management
Data Source
AI summary
A method for deduplicating and managing data blocks within a file system includes adding a deduplication identifier to each pointer pointing to a data block to indicate whether the data block is deduplicated, detecting duplicate data blocks, determining whether one of the duplicate data blocks has been deduplicated, when detected, determining that one duplicate data block is a master copy when it is determined that one duplicate data block has been deduplicated, selecting one of the duplicate data blocks to be a master copy when it is determined that the duplicate data blocks have not been deduplicated, and setting the deduplication identifier of the selected duplicate data block to indicate deduplication, and determining that the other duplicate data block is a new duplicate data block and setting the deduplication identifier of the other duplicate data block to indicate deduplication and directing the respective pointer to the master copy.