RPC-Less Locking via RDMA CAW for Active-Active Storage Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Synchronizing lock states between storage nodes in high-availability clusters using RPC messaging is problematic due to increased latency caused by waiting periods associated with RPC messaging, which contributes to higher IO latency in the cluster.
Innovation Solution
Implementing an RPC-less locking mechanism based on RDMA CAW, where locks are associated with logical block addresses and maintained in an ALS object array on memory visible to storage nodes, allowing for atomic compare-and-write operations over an RDMA channel to synchronize lock states between nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If RPC messaging is used to synchronize lock states between storage nodes, then lock coherency is guaranteed, but IO latency increases due to multiple waiting periods
Solution Approach 1:
The patent extracts the locking mechanism from the RPC messaging framework and implements it directly using RDMA compare-and-write operations on shared memory. This removes the RPC messaging layer that causes four distinct waiting periods, thereby reducing IO latency while maintaining lock coherency through atomic memory operations visible to all storage nodes.
Solution Approach 2:
The patent introduces shared memory as an intermediary between storage nodes for lock synchronization. Instead of using RPC messages to communicate lock state, nodes directly read and write lock state to shared memory using RDMA operations, eliminating the need for RPC request-reply cycles and their associated waiting periods.
2Adaptability or versatility
If RPC messaging is used for lock synchronization, then distributed lock management is achieved, but the number of waiting periods increases to four cycles
Solution Approach 1:
The patent replaces the mechanical RPC messaging system with a direct memory access system. Instead of sending RPC messages that require polling and waiting for reply cycles, storage nodes perform direct compare-and-write operations on shared memory locations, reducing lock acquisition from multiple RPC cycles to a single atomic memory operation.
3Reliability
If RPC messaging is used to synchronize locks, then lock state coherence is maintained, but overall system throughput decreases
Solution Approach 1:
The patent segments the lock synchronization process into direct atomic memory operations rather than monolithic RPC message exchanges. By using RDMA compare-and-write operations that atomically update lock state in shared memory, the system achieves lock coherence without the overhead of complete RPC request-reply cycles, thereby improving system throughput.
Data Source
AI summary
Techniques for providing an RPC-less locking mechanism based on RDMA CAW for a storage cluster with an active-active architecture. The techniques include, in response to receipt at a storage node of a lock request for accessing an LBA of a storage object, synchronizing, between the storage node and a remote node, a state of a lock for the LBA. The synchronizing of the lock state includes locating an ALS object having a field containing the LBA, updating a field of the ALS object to include the storage node, and setting a field of the ALS object to a type of lock specified in the lock request. If the storage node locates the ALS object in an array maintained within visible memory on the remote node, then the storage node updates/sets corresponding fields of the ALS object by performing atomic CAW operations supported by an RDMA channel between the respective nodes.