Non-exclusive Lock Mechanism for Deduplication Metadata Pages
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face performance issues due to contention when updating reference counts for deduplicated data, particularly across multiple nodes, leading to queuing and blocking of deduplication transactions as they compete for exclusive locks, which impacts system performance.
Innovation Solution
The system employs a non-exclusive lock mechanism for metadata pages with reference counts, allowing multiple concurrent transactions to update the reference counts without exclusive access, and uses a spinlock for cached copies to ensure data integrity, reducing contention and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If an exclusive lock is used for metadata pages with reference counts, then data integrity is ensured, but lock contention increases and system performance deteriorates
Solution Approach 1:
The patent divides the locking mechanism into two independent parts: a non-exclusive lock for the metadata page (allowing concurrent reads) and a separate spinlock for the reference count (ensuring atomic updates). This segmentation allows multiple threads to access the metadata page simultaneously without blocking each other, while still maintaining data integrity through the coordinated locking strategy.
Solution Approach 2:
The patent introduces a spinlock as an intermediary mechanism between the non-exclusive lock and the reference count update. The spinlock acts as a mediator that ensures atomicity of reference count modifications while allowing the non-exclusive lock to remain held by multiple threads, thus resolving the contradiction between concurrency and data integrity.
2Productivity
If multiple threads concurrently access metadata pages, then deduplication processing speed improves, but lock contention and queuing increase
Solution Approach 1:
The patent segments the locking requirements by applying a non-exclusive lock to the metadata page structure (enabling concurrent access) and a separate spinlock to the reference count (ensuring atomic updates). This allows multiple threads to process deduplication operations simultaneously without queuing for a single exclusive lock, significantly reducing lock contention time.
Solution Approach 2:
The patent transitions from a static exclusive lock model to a dynamic locking model where the non-exclusive lock can be held by multiple threads simultaneously, and the spinlock is dynamically acquired only when reference count updates are needed. This dynamic approach optimizes the balance between concurrency and data integrity.
Data Source
AI summary
A request to perform a first operation in a system that stores deduplicated data can be received. The system can include a data block stored at multiple logical address each referencing the data block. A reference count can be associated with the data block and can denote a number of logical addresses referencing the data block. Processing can be performed to service the request and perform the first operation, wherein the processing can include: acquiring a non-exclusive lock for a page that includes the reference count of the data block; storing, in a metadata log while holding the non-exclusive lock on the page, an entry to decrement the reference count of the data block; and releasing the non-exclusive lock on the page.