Burst Buffer Network Aggregator for HPC Checkpoint Data Coalescing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High performance computing clusters face inefficiencies in storing checkpoint data due to stringent coherency protocols and static file layouts in parallel file systems, limiting their ability to scale and achieve peak performance.
Innovation Solution
A data migration system that employs a Burst Buffer tier between high performance compute clusters and a HDD-based parallel File System, where checkpoint data is stored in the Burst Buffer tier and then efficiently transferred to the File System, with data coalescing duties assigned to the File System's I/O nodes to optimize bandwidth and reduce unnecessary network traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If checkpoint data is written directly to the parallel File System, then data storage is achieved, but the I/O cycle time increases and compute nodes must wait for write completion
Solution Approach 1:
The patent extracts the bottleneck of direct-to-disk writing by introducing a Burst Buffer tier as an intermediate storage layer. Compute nodes write checkpoint data to the Burst Buffer tier (which has faster I/O characteristics) rather than directly to the parallel File System on HDDs, thereby reducing the I/O cycle time while ensuring checkpoint data is still reliably stored in the end through subsequent migration to the File System.
Solution Approach 2:
The Burst Buffer tier acts as an intermediary between compute nodes and the parallel File System. It temporarily holds checkpoint data and allows compute nodes to complete their I/O cycles faster, while background processes handle the migration of data from the Burst Buffer tier to the File System, thus mediating between the need for fast checkpointing and reliable long-term storage.
2Productivity
If multiple compute nodes write checkpoint data simultaneously to the parallel File System, then checkpointing parallelism is achieved, but coherency protocol overhead and network traffic increase
Solution Approach 1:
The patent segments the checkpoint data write path by having compute nodes write to individual Burst Buffer tier nodes rather than directly to shared File System resources. This segmentation allows parallel writes without immediate coherency conflicts, as each Burst Buffer node independently accepts data. Coherency and data consolidation are handled later by the BBNA process, reducing the overhead during the critical checkpoint writing phase.
Solution Approach 2:
The system performs preliminary actions by having compute nodes write checkpoint data to the Burst Buffer tier first, before the data needs to be consolidated or made coherent for File System storage. This preliminary write to a more permissive storage tier allows parallelism without immediate coherency constraints, and the coherency/consolidation work is deferred to background processes.
3Loss of energy
If data is coalesced at compute nodes before writing to the File System, then network traffic is reduced, but compute node processing overhead increases
Solution Approach 1:
The patent extracts the data coalescing function from compute nodes and relocates it to the File System side through the BBNA (Burst Buffer Network Aggregator) process. Compute nodes simply write checkpoint data to the Burst Buffer tier without performing coalescing operations, thereby reducing their processing overhead. The BBNA process running on File Servers or dedicated I/O nodes performs the coalescing of data from multiple Burst Buffer nodes, reducing network traffic to the File System while avoiding increased compute node complexity.
Data Source
AI summary
A data migrating system and method are provided in which a Burst Buffer Network Aggregator (BBNA) process is configured either on the File Servers or on the File System's dedicated I/O nodes to coalesce data fragments stored in participating Burst Buffer nodes under the direction of a primary BB node appointed by a data generating entity prior to transfer of the full data stripe into the File System. The “write” request in the form of a full data stripe is distributed into a plurality of data fragments among participating BB nodes along with corresponding metadata. The primary BB node gathers the metadata from the participating BB nodes, sends the metadata list to the BBNA unit, responsive to which the BBNA unit allocates a buffer sufficient to store the full data stripe, and transfers data fragments from participating BB nodes into the full data stripe buffer, thereby coalescing the data fragments into the full data stripe, which is subsequently transferred from the buffer in the BBNA unit into the File System.