MPI Alltoall Aggregation via Network Interface Controller
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-performance computing (HPC) systems using Message Passing Interface (MPI) protocols face network congestion and overhead due to the large number of messages transmitted during collective operations like alltoall and alltoallv, which leads to suboptimal bandwidth utilization, especially when message payloads are small.
Innovation Solution
Network interface controllers (NICs) perform message aggregation at the initiator node and disaggregation at responder nodes, reducing the number of messages in the fabric by combining payloads into a single aggregated message with a shared transport header, thereby minimizing I/O overhead and improving bandwidth utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If MPI alltoall and alltoallv collective operations transmit messages individually through the network fabric, then each process can communicate with every other process, but the number of messages becomes excessively large (N*(N-1)*P^2), causing network congestion and high I/O overhead
Solution Approach 1:
The patent applies message aggregation at the initiator node where multiple individual MPI messages are merged into a single aggregated message. This combining operation reduces the total number of messages in the network fabric from O(P^2) to O(P), directly resolving the contradiction between maintaining communication versatility and reducing message quantity.
Solution Approach 2:
The network interface controller acts as an intermediary between the MPI processes and the network fabric. It performs aggregation and disaggregation operations, mediating the communication to reduce message count while maintaining the ability of all processes to communicate with each other.
2Ease of operation
If each message is transmitted with its own transport header, then message routing and delivery can be controlled, but the I/O overhead increases significantly due to repeated header postings
Solution Approach 1:
Multiple individual messages with separate transport headers are merged into a single aggregated message with one shared transport header. This reduces the number of header postings from O(P^2) to O(P), significantly decreasing I/O overhead while the header still provides necessary routing control for the aggregated message.
Solution Approach 2:
The aggregated message is segmented into multiple individual messages at the responder node through disaggregation. This allows the single aggregated message to serve multiple destinations while maintaining the ability to route and deliver messages to specific processes after aggregation.
3Productivity
If the number of messages in the fabric is reduced through aggregation, then bandwidth utilization improves, but the complexity of message aggregation and disaggregation operations increases
Solution Approach 1:
The network interface controller performs aggregation and disaggregation operations autonomously as part of its standard functionality. The system leverages existing NIC capabilities and MPI implementation details to handle message aggregation without requiring complex external coordination or additional hardware, thus improving bandwidth utilization while keeping complexity manageable.
Data Source
AI summary
An MPI collective operation carried out in a fabric of network elements by transmitting MPI messages from all the initiator processes in an initiator node to designated ones of the responder processes in respective responder nodes. Respective payloads of the MPI messages are combined in a network interface device of the initiator node to form an aggregated MPI message. The aggregated MPI message is transmitted through the fabric to network interface devices of responder nodes, disaggregating the aggregated MPI message into individual messages, and distributing the individual messages to the designated responder node processes.