Consistent Hash for Distributed Lock State Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large-scale distributed file systems, node additions or removals lead to significant resource changes, necessitating lock redistribution, which results in substantial backplane communication and system downtime during the lock reconstruction process, impacting performance.
Innovation Solution
Implementing a consistent hash to reassess and maintain coordinator nodes for locks, minimizing communication and downtime by retaining existing locking status wherever possible, and using an API lock to manage new operations during group changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If locks are redistributed when a node is added or removed from the cluster, then the distributed file system can adapt to group changes, but substantial backplane communication is required among all initiator nodes to reassert locks
Solution Approach 1:
The patent segments the lock management by identifying a subset of locks that need redistribution based on the consistent hash of the group change, rather than requiring all initiator nodes to reassert all locks. This segmentation reduces the communication volume from cluster-wide to only the necessary subset of locks affected by the group change.
Solution Approach 2:
The patent applies local quality by having only the affected initiator nodes (those whose locks hash to the changed group) participate in the reassertion process, while other nodes maintain their existing lock states. This localized approach minimizes backplane communication to only the necessary nodes and locks.
2Adaptability or versatility
If locks are redistributed when a node is added or removed from the cluster, then the distributed file system can adapt to group changes, but the entire system is locked down during the reconstruction process
Solution Approach 1:
The patent segments the lock reconstruction process to only affect the subset of locks that need redistribution, allowing other locks and operations to continue normally. This prevents system-wide lockouts while still completing the necessary group change adaptation.
Solution Approach 2:
The patent performs partial action by only reasserting the necessary subset of locks that are affected by the group change, rather than requiring all locks to be reasserted. This partial approach maintains system availability while completing the required adaptation.
3Productivity
If coordinator nodes are reassigned after a group change, then lock distribution can be optimized, but minimal knowledge exists about which initiator nodes have locks asserted
Solution Approach 1:
The patent uses feedback by having initiator nodes reassert their lock information to the new coordinator nodes, allowing the coordinators to learn about the current lock state. This feedback mechanism resolves the information loss problem while enabling optimized lock distribution through consistent hashing.
Data Source
AI summary
Implementations are provided herein for an efficient reconstruction of distributed lock states during a group change within a distributed file system. When a node leaves or joins the cluster, a consistent hash can be used in reassigning the coordinator node to all the distributed locks throughout the file system. Through the use of the consistent hash, wherever possible, nodes that exist both prior to and after the group change can maintain their coordinator status for the locks they manage. If the node is aware they are maintaining coordinator status for the locks it manages, it can retain the locking status previously associated with the locks it was coordinating prior to the group change, thus reducing necessary backplane communication from a coordinator change, and increasing the efficiency for the group change process.