File System Bulk Sub-Transaction for Efficient Deletion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional file deletion processes in file systems are inefficient, particularly for large files, as they require traversing the entire file system hierarchy, generating a high number of metadata transactions, and consuming significant system resources, leading to prolonged deletion times and increased I/O operations.
Innovation Solution
The approach involves creating a bulk sub-transaction that stores multiple sub-transactions for deleting data blocks, updating intermediate indirect data blocks only when all data blocks pointed to by a leaf indirect block are freed, and using multiple threads to process leaf indirect blocks concurrently, thereby reducing the number of metadata transactions and I/O operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional file deletion processes traverse the entire file system hierarchy to delete each data block individually, then data integrity is ensured, but deletion time and I/O operations increase significantly
Solution Approach 1:
The patent segments the file deletion process into two distinct phases: (1) creating a bulk sub-transaction that records all deletion operations in memory without immediate I/O, and (2) committing the bulk transaction atomically to the journal. This segmentation allows individual data blocks to be marked for deletion efficiently while ensuring integrity through the bulk commit mechanism, resolving the contradiction between fast deletion and data integrity.
Solution Approach 2:
The patent performs preliminary action by creating the bulk sub-transaction in memory that pre-reports all deletion operations before actually committing them to the journal. This preliminary transaction creation phase allows the system to prepare the entire deletion sequence without immediate I/O overhead, then commit atomically to ensure integrity, thus reducing deletion time while maintaining reliability.
2Loss of information
If conventional processes generate a metadata transaction for each data block deletion, then detailed transaction logging is achieved, but the number of metadata transactions and system resource consumption increase
Solution Approach 1:
The patent merges multiple individual metadata transactions into a single bulk sub-transaction that contains references to all data blocks being deleted. Instead of creating separate journal entries for each data block, the bulk transaction consolidates all deletion operations into one atomic unit that is logged once in the journal, significantly reducing the quantity of metadata transactions while preserving complete transaction logging detail.
Solution Approach 2:
The patent creates a copy of the deletion transaction information in the bulk sub-transaction structure that references multiple data blocks without creating separate journal entries for each. The bulk transaction contains a consolidated record that copies the essential deletion information for all blocks, reducing the number of metadata transactions written to the journal while maintaining complete logging capability.
3Stability of the object's composition
If intermediate indirect data blocks are updated frequently during file deletion, then real-time file system consistency is maintained, but I/O operations and system resource consumption increase
Solution Approach 1:
The patent extracts the intermediate indirect block update operations from the critical deletion path and defers them until the bulk transaction is committed. Instead of updating intermediate blocks immediately after each data block deletion, the system extracts these updates and batches them with the bulk commit operation, reducing I/O operations and system resource consumption during the deletion process while maintaining file system consistency through the atomic commit.
Solution Approach 2:
The patent performs preliminary marking of data blocks as deleted in the bulk sub-transaction without immediately updating intermediate indirect blocks. The intermediate block updates are deferred as a preliminary action that occurs only at commit time, reducing resource consumption during the deletion phase while ensuring consistency is restored through the subsequent bulk commit operation.
Data Source
AI summary
A method is used in managing files of file systems. A group transaction record for an indirect data block of a file of a file system is created. The indirect data block of the file includes a set of data blocks of the file. A set of transaction entries is added to the group transaction record. Each transaction entry of the set of transaction entries includes information regarding an operation performed on a data block of the set of data blocks. The group transaction record is stored in a journal. The journal includes metadata transactions upon a plurality of files of a plurality of file systems.