GPU Coalescing Agent for Network Communication Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data movement requests between distributed GPUs over a network result in low network efficiency due to small payloads, and existing coalescing mechanisms are not leveraged when data transfers are performed via networks like InfiniBand or Ethernet, leading to poor performance.
Innovation Solution
A coalescing agent in the communication engine analyzes communication requests across multiple threads and requests, including those with non-contiguous memory locations, to coalesce them into larger, more efficient packets before transmission over the network, while direct P2P transfers are handled separately for efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If GPU threads send communication requests with small payloads over network connections, then individual fine-grain requests can be transmitted, but network efficiency and performance deteriorate
Solution Approach 1:
The coalescing agent merges multiple fine-grain communication requests from multiple GPU threads into a single coalesced request with aggregated payload. This combining operation transforms many small individual requests into one larger request, thereby improving network efficiency while maintaining the capability to handle fine-grain data movements.
2Productivity
If existing coalescing mechanisms are used for P2P transfers, then efficient direct transfers are achieved, but network transfers between distributed GPUs do not benefit from coalescing
Solution Approach 1:
The coalescing agent is designed to provide universal coalescing functionality that works for both P2P transfers and network transfers between distributed GPUs. By making the coalescing mechanism multi-functional, the system achieves efficient direct transfers for P2P while also enabling network transfers to benefit from the same coalescing operations.
3Extent of automation
If warp-level coalescing is performed using software techniques, then requests from multiple threads can be detected and coalesced, but packet sizes remain small (256 B) and network efficiency is still low
Solution Approach 1:
The coalescing agent extends coalescing beyond the traditional warp-level (single dimension) to include multiple warps and cooperative thread arrays (CTAs) (multiple dimensions). This dimensional expansion allows aggregation of requests from a broader scope of threads, resulting in significantly larger packet sizes that are much more efficient for network transport.
4Device complexity
If coalescing is limited to contiguous memory locations, then simpler coalescing logic can be used, but the degree of coalescing is reduced and non-contiguous memory access patterns cannot be optimized
Solution Approach 1:
The coalescing agent acts as an intermediary that receives communication requests, analyzes their memory location patterns, and determines whether they can be coalesced regardless of contiguity. This intermediary layer enables the system to handle both contiguous and non-contiguous memory access patterns while maintaining coalescing efficiency through intelligent request grouping.
Data Source
AI summary
Apparatuses, systems, and techniques are directed to automatic coalescing of GPU-initiated network communications. In one method, a communication engine receives, from a shared memory application executing on a first graphics processing unit (GPU), a first communication request assigned to or having a second GPU as a destination to be processed. The communication engine determines that the first communication request satisfies a coalescing criterion and stores the first communication request in association with a group of requests that have a common property. The communication engine coalesces the group of requests into a coalesced request and transports the coalesced request to the second GPU over a network.