File System Metadata Isolation Using Upgrade Locks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file systems face issues with concurrency in metadata updates during concurrent read/write transactions, leading to race conditions and potential inaccuracy due to the inability to isolate metadata changes using range locks.
Innovation Solution
The implementation of an upgrade lock and a Read-Copy-Update (RCU) stat data structure to atomically manage metadata updates, ensuring that changes are privately held until transaction commit, and using a shared pointer to maintain multiple versions of metadata, thereby avoiding race conditions and ensuring data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If byte-level range locks are used to achieve concurrency, then read/write operations with non-overlapping ranges can execute concurrently, but metadata updates cannot be isolated resulting in race conditions
Solution Approach 1:
The patent segments the locking mechanism into two distinct types: byte-level range locks for data regions and a unified upgrade lock for metadata. This segmentation allows concurrent data operations while ensuring exclusive metadata access, resolving the race condition problem without sacrificing concurrency.
Solution Approach 2:
The upgrade lock acts as an intermediary mechanism that mediates between concurrent data operations and metadata updates. By requiring all metadata access to go through this intermediary lock, the system ensures isolation of metadata changes while allowing the underlying data operations to proceed concurrently.
2Productivity
If metadata is updated prior to transaction commit, then file stat changes are reflected early, but other processes may read corrupted or untruthful data
Solution Approach 1:
The upgrade lock is acquired preliminarily before any metadata updates occur during transaction execution. This preliminary locking action ensures that no other process can read or write metadata until the transaction commits or rolls back, preventing corrupted data reads while still allowing early metadata preparation.
Solution Approach 2:
The patent creates a private copy of the file stat data structure that is isolated from the shared metadata. Updates are made to this private copy during transaction execution, and only upon successful commit are changes atomically merged back to the shared metadata, ensuring other processes never read intermediate or corrupted states.
3Reliability
If upgrade lock is obtained for file stat data, then metadata updates are isolated atomically, but access time for multiple processes may be blocked
Solution Approach 1:
The upgrade lock is applied locally and selectively only to metadata access operations, not to data operations. This localized locking approach ensures that metadata updates are isolated atomically while allowing multiple processes to concurrently access and modify data regions without acquiring the upgrade lock, minimizing blocking time.
Data Source
AI summary
The disclosure provides for isolation of concurrent read and write transactions on the same file, thereby enabling higher file system throughput relative to serial-only transactions. Race conditions and lock contentions in multi-writer scenarios are avoided in file stat (metadata) updates by the use of an aggregator to merge updates of committed transactions to maintain file stat truth, and an upgrade lock that enforces atomicity of file stat access, even while still permitting multiple processes to concurrently read from and/or write to the file data. The disclosure is applicable to generic file systems, whether native or virtualized, and may be used, for example, to speed access to database files that require prolonged input/output (I/O) transaction time periods.