SSD Burst Buffer Nodes for Supercomputing Checkpoint Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current supercomputing storage architectures face challenges in achieving the required bandwidth and capacity for Exascale computing due to the increasing number of cores with smaller memories per core, leading to non-sequential workloads and random access patterns, which result in inefficient disk utilization and increased costs.
Innovation Solution
The integration of Solid-State Drive (SSD) burst buffer nodes with asynchronous migration between SSDs and slower disk storage, along with the use of the Parallel Log-Structured File System (PLFS) to present a file system interface and optimize data layout for sequential access, allowing explicit user management of data placement and maximizing bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If magnetic disk storage is used directly for checkpoint data storage, then capacity is provided, but bandwidth and throughput are insufficient for Exascale computing workloads
Solution Approach 1:
The storage system is segmented into two distinct components: SSD burst buffer nodes for high-speed data transfer and magnetic disk storage for capacity. This segmentation allows each component to optimize for its specific function, with SSDs handling the bandwidth-intensive checkpoint operations and disks providing economical capacity storage.
Solution Approach 2:
SSD burst buffer nodes serve as intermediary components between compute nodes and magnetic disk storage. These intermediaries absorb the high-speed write operations from compute nodes and then asynchronously migrate data to disk storage, mediating between the conflicting requirements of high bandwidth and large capacity.
2Power
If the number of cores is increased for Exascale computing, then computational power is improved, but random access patterns increase leading to inefficient disk utilization
Solution Approach 1:
The SSD burst buffer acts as an intermediary that absorbs random access patterns from multiple compute nodes with increased cores. By providing a high-speed buffer layer, it prevents these random access patterns from directly impacting magnetic disk utilization efficiency, allowing disks to operate more efficiently.
Solution Approach 2:
The system segments the I/O workload by separating random access operations (handled by SSD burst buffers near compute nodes) from sequential access operations (handled by magnetic disk storage). This segmentation allows computational power to scale with core count while disk utilization remains efficient.
3Productivity
If SSD burst buffer nodes are integrated, then bandwidth and latency are improved, but system complexity increases
Solution Approach 1:
The SSD burst buffer nodes perform multiple functions: they serve as high-speed checkpoint write targets for compute nodes, act as buffer for asynchronous migration to disk, and provide a file system interface to applications. This multi-functionality reduces the need for additional specialized components, managing system complexity.
Solution Approach 2:
The SSD burst buffer nodes autonomously manage the asynchronous migration of checkpoint data to magnetic disk storage without requiring continuous intervention from compute nodes. This self-service capability simplifies the overall system control architecture while maintaining high bandwidth performance.
4Quantity of substance
If checkpoint data is written directly to magnetic disk storage, then capacity is utilized, but access latency and jitter increase
Solution Approach 1:
The storage path is segmented into two stages: first, checkpoint data is rapidly written to SSD burst buffers with low latency; second, data is asynchronously migrated to magnetic disk storage for capacity retention. This segmentation eliminates the latency penalty of direct disk writing while maintaining capacity efficiency.
Solution Approach 2:
SSD burst buffer nodes serve as intermediaries that absorb the latency-sensitive checkpoint write operations, allowing compute nodes to complete checkpoints without experiencing the high access latency of magnetic disk storage. The SSD layer shields the system from disk latency while disks provide capacity.
Data Source
AI summary
A parallel supercomputing cluster includes compute nodes interconnected in a mesh of data links for executing an MPI job, and solid-state storage nodes each linked to a respective group of the compute nodes for receiving checkpoint data from the respective compute nodes, and magnetic disk storage linked to each of the solid-state storage nodes for asynchronous migration of the checkpoint data from the solid-state storage nodes to the magnetic disk storage. Each solid-state storage node presents a file system interface to the MPI job, and multiple MPI processes of the MPI job write the checkpoint data to a shared file in the solid-state storage in a strided fashion, and the solid-state storage node asynchronously migrates the checkpoint data from the shared file in the solid-state storage to the magnetic disk storage and writes the checkpoint data to the magnetic disk storage in a sequential fashion.