Global Sync List for File System Metadata Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional file systems face inefficiencies in managing metadata updates, leading to metadata transaction log full conditions due to random flushing orders and unused space, causing performance issues and potential data inconsistencies.
Innovation Solution
Implementing a global sync list to maintain references to dirty metadata in the order they appear on the file system transaction log, allowing for efficient flushing and prioritization, thereby improving I/O performance and space utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If metadata updates are flushed in random order, then individual file system operations can be completed independently, but the transaction log fills up quickly and creates unused space
Solution Approach 1:
The patent applies preliminary action by maintaining a global sync list that pre-organizes dirty metadata references in the order they appear on the transaction log before flushing occurs. This allows the system to flush metadata sequentially in the correct order without needing to process the entire transaction log each time, preventing the transaction log from filling up while maintaining efficient space utilization.
2Speed
If metadata is cached in memory to improve performance, then I/O operations are faster, but data consistency may be compromised if the cache is not properly synchronized
Solution Approach 1:
The patent implements feedback by continuously tracking dirty metadata in the global sync list and using this information to determine when and what to flush from cache to disk. The system monitors cache state and provides feedback to the flush operation, ensuring that cached metadata is synchronized with the transaction log in the correct order, thereby maintaining data consistency while preserving performance benefits.
3Adaptability or versatility
If multiple file systems are managed separately with individual sync processes, then each file system can be optimized independently, but the overall system complexity increases
Solution Approach 1:
The patent merges the sync management of multiple file systems into a single global sync list that handles dirty metadata from all file systems collectively. Instead of maintaining separate sync processes for each file system, the global sync list consolidates tracking and flushing operations, reducing system complexity while still allowing individual file systems to be managed independently when needed.
Data Source
AI summary
A method is used in managing updating of metadata of file systems. A reference to a change to metadata of a file of a file system is stored in a global sync list configured to store changes to metadata of a plurality of files of a plurality of file systems. The change to the metadata of the file is stored in a journal upon receiving an I/O request for the file of the file system. The journal includes metadata transactions upon the plurality of files of the plurality of the file system. The change to the metadata of the file of the file system is applied to metadata of the file system organized on a storage device using the reference to the change stored in the global sync list.