Fabric-Attached Shared Memory for HPC Messaging Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large HPC clusters typically lack shared memory architectures, leading to performance bottlenecks and complexity in implementing shared memory access, which is difficult to achieve with conventional network stacks and message passing systems.
Innovation Solution
Implement a fabric-attached memory (FAM) system that provides a global shared memory accessible via a high-speed, low-latency network, using a FAM API and MPI-compatible protocols to enable efficient access and management of memory windows across HPC compute nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional network stacks and message passing systems are used to implement shared memory access in HPC clusters, then distributed memory architecture is maintained, but performance bottlenecks and implementation complexity increase
Solution Approach 1:
The patent introduces a fabric-attached memory (FAM) system as an intermediary component that sits between the distributed compute nodes and the network fabric. This FAM system provides a shared memory interface that appears local to processes while actually residing in network-attached memory, thereby maintaining distributed memory architecture while enabling shared memory access patterns without the performance bottlenecks of conventional network stacks
Solution Approach 2:
The patent segments the memory system into local compute node memory and fabric-attached memory, allowing processes to access both local and shared memory through a unified interface. This segmentation enables shared memory functionality to be implemented without requiring full centralization, maintaining the benefits of distributed memory while adding shared access capabilities
2Adaptability or versatility
If shared memory is implemented using conventional network stacks, then memory sharing across nodes is achieved, but data throughput is limited and implementation becomes complex
Solution Approach 1:
The FAM system acts as an intermediary that abstracts the complexity of network-based memory sharing from applications. It provides a simplified interface that appears similar to local memory access, handling the complex network communication, memory management, and synchronization operations transparently, thereby reducing implementation complexity while enabling versatile memory sharing
Solution Approach 2:
The fabric-attached memory system provides multiple functions through a single interface: it enables shared memory access, maintains distributed memory architecture, provides high-bandwidth data transfer, and supports both local and remote memory access patterns. This multi-functionality achieves versatile memory sharing without requiring multiple separate systems or complex implementation details
3Speed
If local memory at each compute node is used, then fast access is achieved, but memory size is constrained and data sharing becomes difficult
Solution Approach 1:
The patent extends the memory address space from a single-node dimension to a multi-node dimension by introducing fabric-attached memory. This allows processes to access memory that appears contiguous and fast like local memory, while actually residing in network-attached memory resources, effectively adding another dimension to the memory architecture that combines the speed of local memory with the capacity of distributed memory
Data Source
AI summary
A first process executing on a first HPC compute node of an HPC cluster sends a write command, including a global identifier for a memory window that is globally accessible to processes executing in the HPC cluster, to an HPC memory node of the HPC cluster instructing the HPC memory node to write first data to the memory window allocated at the HPC memory node. The first data is sent by the first process to a memory window allocated on the HPC memory node. The first data is written by the HPC memory node to the memory window that includes randomly accessed, addressable memory locations.