Parallel APD Data Transfer Using Helper GPUs for Higher Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer systems with multiple accelerated processing devices (APDs) often fail to utilize their full potential due to complexities in software design, leading to underutilization of interconnect bus bandwidth and inefficient data transfer between APDs.
Innovation Solution
Implementing a technique where data is transmitted directly to a target APD and indirectly through a helper APD using a high-speed inter-APD interconnect, allowing data to be transferred in parallel over independent interconnect buses, thereby increasing effective bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred only through direct interconnect bus connections to APDs, then the system architecture remains simple, but the effective bandwidth is limited by the capacity of individual interconnect buses
Solution Approach 1:
The patent introduces intermediary APDs that act as mediators to forward data between source and destination APDs. When data needs to be transferred to a target APD, the system can route data through one or more intermediate APDs connected via high-speed inter-APD interconnects, thereby increasing effective bandwidth without requiring direct high-bandwidth connections between all APD pairs.
Solution Approach 2:
The patent transforms the traditional single-dimension data transfer path (source APD → interconnect bus → destination APD) into a multi-dimensional network topology where data can travel through multiple APDs via high-speed inter-APD interconnects. This creates additional transfer paths and dimensions for data flow, effectively increasing bandwidth utilization across the system.
2Productivity
If multiple APDs are deployed to increase processing capacity, then computational power is improved, but software complexity increases making it difficult to utilize full potential
Solution Approach 1:
The patent creates a universal data transfer mechanism that works across different APD configurations and topologies. The same inter-APD interconnect infrastructure and forwarding protocol can handle various scenarios including direct transfers, multi-hop transfers, and transfers involving multiple helper APDs, simplifying software design while supporting multiple APDs.
Solution Approach 2:
The system enables APDs to serve multiple functions including their primary processing role and a secondary role as data forwarding intermediaries. Helper APDs automatically forward received data to target APDs without requiring complex software coordination, allowing APDs to self-organize into efficient data transfer paths.
3Speed
If high-speed inter-APD interconnects are implemented, then data transfer efficiency is improved, but the system architecture becomes more complex
Solution Approach 1:
The patent merges the traditional interconnect bus architecture with high-speed inter-APD interconnects into a unified system. Both connection types coexist and work together, with the inter-APD interconnects providing high-speed paths where beneficial while maintaining compatibility with existing interconnect bus structures, thereby increasing speed without completely redesigning the architecture.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques for improving data transfer in a system having multiple accelerated processing devices ("APDs") are described herein. In such a system, multiple APDs are coupled to a processor (e.g., a central processing unit ("CPU")) via a general interconnect fabric and to each other via a high speed interconnect. The techniques herein increase the effective bandwidth for transfer of data between the CPU and the APD by transmitting data to both APDs through the portion of the interconnect fabric coupled to each respective APD. Then, one of the APDs transfers data to the other APD or to the processor via the high speed inter-APD interconnect. Although data transferred "indirectly" through the helper APD takes slightly more time to be transferred than a direct transfer, the total effective bandwidth to the target is increased due to the high-speed inter- APD interconnect.