Symmetric Heap Allocation for RDMA in PGAS Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In Partitioned Global Address Space (PGAS) systems, network read and write operations are significantly slower than local operations, and maintaining data structure location information is resource-intensive and performance-degrading due to the need for address translation, which prevents the use of remote direct memory access (RDMA).
Innovation Solution
Implementing a symmetric heap with isomorphic symmetric partitions, where each process has a symmetric partition starting at the same virtual memory address, and designating one process as an allocator to manage memory allocation and fragmentation across partitions, enabling efficient RDMA by eliminating the need for address transactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a list of virtual base addresses is maintained for each distributed data structure in each partition, then data location tracking is achieved, but storage cost and initialization time increase significantly
Solution Approach 1:
The patent merges the address tracking functionality into a single centralized location (the allocator process) rather than maintaining separate tracking structures in each partition. The allocator maintains a global view of all distributed data structures and their locations across partitions, eliminating the need for redundant address lists in each partition and dramatically reducing storage requirements.
Solution Approach 2:
The allocator process serves multiple functions: it manages memory allocation across all partitions, maintains the global state of distributed data structures, and provides address translation services. This multi-functional approach consolidates what would otherwise require separate mechanisms in each partition, reducing overall system complexity and storage needs.
2Ease of operation
If address translation is performed by the target process, then data access is enabled, but target process performance is reduced due to interruptions
Solution Approach 1:
The patent introduces the allocator as an intermediary between the requesting process and the target process. When a distributed data structure needs to be accessed, the allocator translates the handle to the actual memory address and returns it to the requesting process. This eliminates the need for the target process to be interrupted for address translation, as the translation is performed by the allocator using its maintained global state.
Solution Approach 2:
The allocator performs address translation in advance by maintaining the mapping between handles and actual memory addresses in its global state. This preliminary action of pre-computing and storing address mappings allows subsequent data access operations to proceed without interrupting the target process, as the translation work has already been done.
3Speed
If RDMA is used for accelerated network operations, then data transfer speed is improved, but it cannot be used in PGAS systems that use SVD because the remote virtual memory address is not known by the requesting process
Solution Approach 1:
The system uses a handle-based reference system where processes reference distributed data structures by handle rather than by direct memory address. The allocator maintains the mapping between handles and actual memory locations. When RDMA is needed, the allocator receives the handle, translates it to the actual remote memory address using its global state, and then enables RDMA operations with the resolved address. This feedback mechanism allows RDMA to be used while maintaining the simplified handle-based interface.
Data Source
AI summary
Allocating distributed data structures and managing allocation of a symmetric heap can include defining, using a processor, the symmetric heap. The symmetric heap includes a symmetric partition for each process of a partitioned global address space (PGAS) system. Each symmetric partition of the symmetric heap begins at a same starting virtual memory address and has a same global symmetric break. One process of a plurality of processes of the PGAS system is configured as an allocator process that controls allocation of blocks of memory for each symmetric partition of the symmetric heap. Using the processor executing the allocator process, isomorphic fragmentation among the symmetric partitions of the symmetric heap is maintained.