Receiver-Managed Offset Counter for PGAS One-Sided Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In high performance computing systems with a partitioned global address space, one-sided memory access is hindered by synchronization delays and additional network round trips due to bi-directional exchanges between sender and receiver nodes, affecting performance.
Innovation Solution
Implementing a receiver-managed mechanism where the receiver maintains a buffer address offset counter, updated based on the size of incoming data, to calculate the destination address and eliminate the need for bi-directional setup messages, thereby reducing communication operations and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If bi-directional exchanges are used between sender and receiver nodes to maintain buffer address offsets, then correct memory access coordination is achieved, but communication latency and network round trips increase
Solution Approach 1:
The receiver node autonomously manages buffer address offsets using a local counter that is updated based on the size of incoming data. This self-service mechanism eliminates the need for the sender to query or coordinate offset information, allowing the receiver to independently determine where to place incoming data in its buffer.
Solution Approach 2:
The receiver pre-allocates buffer space and maintains a counter indicating the next available offset position. This preliminary setup allows the receiver to immediately write incoming data at the correct offset without requiring prior coordination or address calculation exchanges with the sender.
2Reliability
If bi-directional setup messages are exchanged for one-sided memory access, then correct buffer address offsets are maintained, but device complexity and communication overhead increase
Solution Approach 1:
The receiver node autonomously manages buffer address offsets using a local counter that is updated based on the size of incoming data. This self-service mechanism eliminates the need for the sender to query or coordinate offset information, allowing the receiver to independently determine where to place incoming data in its buffer.
Solution Approach 2:
The offset management functionality is extracted from the communication protocol and implemented as a separate counter mechanism on the receiver side. This separation removes the need for offset-related message exchanges, simplifying the communication protocol while maintaining correct buffer management.
3Reliability
If traditional sender-managed offset coordination is used, then memory access correctness is maintained, but productivity and performance of parallel operations decrease
Solution Approach 1:
The receiver node autonomously manages buffer address offsets using a local counter that is updated based on the size of incoming data. This self-service mechanism eliminates the need for the sender to query or coordinate offset information, allowing the receiver to independently determine where to place incoming data in its buffer.
Solution Approach 2:
The receiver pre-allocates buffer space and maintains a counter indicating the next available offset position. This preliminary setup allows the receiver to immediately write incoming data at the correct offset without requiring prior coordination or address calculation exchanges with the sender.
Data Source
AI summary
Generally, this disclosure provides systems, devices, methods and computer readable media for improved coordination between sender and receiver nodes in a one-sided memory access to a PGAS in a distributed computing environment. The system may include a transceiver module configured to receive a message over a network, the message comprising a data portion and a data size indicator and an offset handler module configured to calculate a destination address from a base address of a memory buffer and an offset counter. The transceiver module may further be configured to write the data portion to the memory buffer at the destination address; and the offset handler module may further be configured to update the offset counter based on the data size indicator.