File Deletion Optimization via Indirect Block Reference Counting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for deleting snapshot copies of files in storage systems are inefficient, requiring extensive traversal of file system hierarchies and resulting in high CPU usage and resource consumption due to the need to update and flush metadata transactions across large file systems.
Innovation Solution
The technique optimizes file deletion and truncation by identifying sibling indirect blocks that share data blocks with the target file, allowing distributed reference counts to be returned to these blocks instead of updating per-block metadata for each shared data block, thereby reducing the number of metadata transactions and I/O operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional methods are used to delete snapshot copies of files, then data integrity is maintained through comprehensive metadata updates, but CPU usage and resource consumption increase significantly due to extensive traversal of file system hierarchies
Solution Approach 1:
The patent segments the file system hierarchy into distinct components (indirect blocks, data blocks, metadata structures) and processes only the relevant segments during deletion operations. Instead of traversing the entire file system hierarchy, the method identifies and updates only the specific indirect blocks and data blocks associated with the snapshot copy being deleted, thereby maintaining data integrity while reducing CPU usage.
Solution Approach 2:
The patent extracts the essential metadata update operations from the comprehensive file system traversal process. By identifying and extracting only the critical metadata that needs to be updated (specifically the indirect blocks and data blocks associated with the snapshot), the method eliminates unnecessary traversal and processing steps, reducing resource consumption while preserving data integrity.
2Reliability
If comprehensive metadata updates are performed during file deletion, then data consistency is ensured, but the number of I/O operations and time consumption increase
Solution Approach 1:
The patent applies local quality by updating metadata with the specific level of detail required for each component. Instead of performing uniform comprehensive updates across the entire file system, the method updates only the local metadata (indirect blocks and data blocks) that is directly relevant to the snapshot deletion, ensuring data consistency while minimizing I/O operations and time consumption.
Solution Approach 2:
The patent performs preliminary identification of the metadata that needs to be updated before executing the deletion operation. By pre-identifying the specific indirect blocks and data blocks associated with the snapshot copy, the method prepares the necessary metadata updates in advance, avoiding unnecessary I/O operations and reducing time consumption while maintaining data consistency.
3Measurement precision
If per-block metadata is updated for each shared data block, then reference count accuracy is maintained, but the complexity of the deletion process increases
Solution Approach 1:
The patent applies asymmetry by treating shared data blocks differently from non-shared data blocks during deletion. Instead of uniformly updating per-block metadata for all data blocks, the method identifies shared data blocks and updates their reference counts through indirect block metadata, while handling non-shared blocks more simply. This asymmetric approach maintains reference count accuracy while reducing deletion process complexity.
Solution Approach 2:
The patent introduces indirect blocks as intermediaries between the deletion operation and the data blocks. Instead of directly updating per-block metadata for each shared data block, the method updates the reference counts through the indirect block metadata, which acts as a mediator. This approach maintains reference count accuracy while simplifying the deletion process by reducing the number of direct metadata updates required.
Data Source
AI summary
A method is used in managing file deletions in storage systems. A request to delete a portion of a file of a file system is received. A first indirect block is associated with the portion of the file and includes a set of file system blocks. A second indirect block associated with a replica of the file is identified. The first and second indirect blocks share a subset of the set of file system blocks. The replica of the file represents a state of the file at a particular prior point in time. Metadata of the second indirect block is updated indicating that the subset of the set of file system blocks are no longer shared between the first and second indirect blocks.