Libfabric Atomic Shared Memory Access Without Cluster Locks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cluster synchronization methods in distributed systems, such as high-performance computing environments, face challenges with increased latency and scalability issues due to the use of distributed lock managers and shared message queues, which can create bottlenecks and limit performance as the number of nodes grows.
Innovation Solution
The implementation of libfabric atomics-based lockless cluster-wide shared memory access API allows nodes to access shared data without acquiring locks, using compare-and-swap operations to ensure coherence, thereby reducing latency and contention across multiple nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If distributed lock managers and shared message queues are used for cluster synchronization, then data coherence is maintained, but latency increases and scalability is limited due to bottlenecks
Solution Approach 1:
The patent extracts the locking mechanism from the synchronization process by implementing lockless atomic operations. Nodes directly perform atomic read-modify-write operations on shared memory locations without acquiring locks, eliminating the bottleneck caused by distributed lock managers while maintaining data coherence through hardware-supported atomicity.
Solution Approach 2:
The patent introduces shared memory regions as intermediaries between nodes, where data is stored in distributed memory locations that can be atomically accessed. This intermediary layer enables direct node-to-node communication through the shared memory fabric, bypassing the need for message queue intermediaries and reducing latency.
2Reliability
If distributed lock managers and shared message queues are used for cluster synchronization, then data coherence is maintained, but scalability is limited due to bottlenecks
Solution Approach 1:
The patent segments shared data into multiple distributed memory regions across different nodes. Each node owns specific memory segments and can independently manage atomic operations on its local segments, allowing parallel access patterns that scale with the number of nodes while maintaining coherence through the atomic operation semantics.
Solution Approach 2:
The patent enables nodes to perform self-service synchronization by directly executing atomic operations on shared memory locations without external coordination. Each node independently manages its own access to shared data through hardware-supported atomic instructions, eliminating the need for centralized lock management and enabling linear scalability.
3Reliability
If locks are acquired for accessing shared data, then data coherence is ensured, but additional latency and contention are introduced
Solution Approach 1:
The patent replaces the mechanical locking system with hardware-supported atomic memory operations. Instead of software-based lock acquisition, release, and contention handling, the system uses CPU and fabric-supported atomic read-modify-write instructions that provide lockless synchronization, reducing both complexity and latency.
Data Source
AI summary
The system allocates, in a distributed system comprising a plurality of nodes, a plurality of portions of memory which comprise shared remote memory content. The system registers the allocated portions with an operating system to be accessed via RDMA. The system accesses, by a first node, the allocated portions to obtain a local copy. The system performs an atomic operation on one or more bits of the shared remote memory content via libfabric atomic application programming interface calls, by one or more of: updating the one or more bits based on a new value and an offset; retrieving, from the shared remote memory content based on the offset, a current value of the one or more bits prior to the updating; and performing an action on the shared remote memory content based on a comparison of the retrieved current value with an expected value in the local copy.