Remote Memory Access via Host Fabric Interface
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current parallel computing applications using the partitioned global address space (PGAS) programming model face inefficiencies in memory access, particularly with one-sided communication, as they require each sender process to store communication parameters for every receiver process, leading to increased memory usage and limitations in scalability and hardware acceleration support.
Innovation Solution
A system that allows a sender computing node to transmit one-sided memory access messages without storing communication parameters for each receiver node, using a host fabric interface to perform memory operations directly on the receiver node, which reduces memory requirements and supports hardware acceleration, enabling scalable remote memory access across multiple nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If each sender process stores communication parameters for every receiver process, then memory access operations can be performed, but memory usage increases linearly with the total number of processes
Solution Approach 1:
The patent extracts the communication parameters from the sender process memory and relocates them to the receiver process memory. Each receiver process now stores its own communication parameters (segment base addresses, segment sizes, etc.) locally, eliminating the need for sender processes to maintain large tables of receiver parameters. This extraction resolves the linear memory growth problem by shifting the storage burden to receivers.
Solution Approach 2:
The patent introduces a new intermediary mechanism where the receiver process acts as the mediator for its own memory segment management. Instead of senders directly managing receiver parameters, receivers self-register their memory segments and manage their own parameter storage. This intermediary role of the receiver process eliminates the need for centralized parameter storage at senders.
2Device complexity
If multiple memory segments are layered on a single-segment PGAS with symmetric addresses, then memory organization is simplified, but each process must reserve an address space large enough for the largest buffer and commodity OS do not support symmetric addresses across multiple nodes
Solution Approach 1:
The patent embraces asymmetry by allowing each receiver process to have its own unique memory segments with different base addresses and sizes, rather than requiring symmetric address spaces. Each process can allocate memory segments according to its specific needs, and the system handles the asymmetric addressing through the new parameter storage mechanism where receivers store their own segment information.
Solution Approach 2:
The patent divides the global address space into multiple independent memory segments, each owned by a specific receiver process. Instead of a single large address space that all processes must reserve, the system segments memory into smaller, process-specific units. This segmentation allows processes to allocate only the memory they need rather than reserving large address spaces for potential future use.
3Extent of automation
If sender processes store communication parameters for all receivers, then one-sided communication can be implemented, but scalability is limited due to increased memory requirements
Solution Approach 1:
The patent implements self-service by enabling receiver processes to manage their own communication parameters autonomously. Each receiver process automatically registers its memory segments, stores its own parameters locally, and manages its segment information without requiring other processes to store or access its parameters. This self-service approach eliminates the scalability bottleneck of centralized parameter storage.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Technologies for one-side remote memory access communication include multiple computing nodes in communication over a network. A receiver computing node receives a message from a sender node and extracts a segment identifier from the message. The receiver computing node determines, based on the segment identifier, a segment start address associated with a partitioned global address space (PGAS) segment of its local memory. The receiver computing node may index a segment table stored in the local memory or in a host fabric interface. The receiver computing node determines a local destination address within the PGAS segment based on the segment start address and an offset included in the message. The receiver computing node performs a remote memory access operation at the local destination address. The receiver computing node may perform those operations in hardware by the host fabric interface of the receiver computing node. Other embodiments are described and claimed.