Dynamic Inter-GPU Link Interface Management for Neural Network Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network training on multi-processing systems faces inefficiencies due to bursty data transfer between processing units, leading to excessive power consumption and prolonged training times, as current methods fix clock frequencies and link interfaces at maximum values, causing compute units to starve for power.
Innovation Solution
A computing system dynamically adjusts the clock frequency and link width of inter-processor links based on data transfer size and layer computation time, optimizing power sharing between compute units and link interfaces to minimize execution time during neural network training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If clock frequencies and link interfaces are fixed at maximum values to achieve shortest data transfer time, then data transfer speed is improved, but power consumption increases significantly and compute units starve for power
Solution Approach 1:
The patent applies dynamics by making the clock frequency and link interface configuration adjustable rather than fixed. The system dynamically changes these parameters based on the actual data transfer requirements of different neural network layers, allowing the inter-processor link to operate at maximum speed only when necessary and at lower speeds when less data needs to be transferred, thus resolving the contradiction between speed and power consumption
Solution Approach 2:
The patent changes the parameters of the inter-processor link (clock frequency and link interface width) based on the data transfer size requirements. By adjusting these parameters to match the actual needs of each neural network layer's data transfer, the system achieves optimal performance without excessive power consumption, directly addressing the technical contradiction
2Productivity
If power is allocated to link interfaces for bursty data transfers, then data transfer efficiency is improved, but compute units experience power starvation
Solution Approach 1:
The system dynamically adjusts power allocation to the inter-processor link based on the bursty data transfer requirements of different neural network layers. During backward propagation where large data transfers occur, the link receives sufficient power; during compute-intensive phases with minimal data transfer, power is reallocated to compute units, thus balancing productivity and power availability
Solution Approach 2:
The patent implements periodic adjustment of power allocation corresponding to the periodic nature of neural network training phases (forward propagation, backward propagation, parameter updates). The system rhythmically shifts power between link interfaces and compute units based on which phase is active, ensuring data transfer efficiency when needed while preventing compute unit power starvation
3Loss of time
If maximum power is provided to inter-processor links during data transfer, then transfer time is reduced, but overall training time increases due to power management overhead
Solution Approach 1:
The system performs preliminary analysis of the neural network layer characteristics and data transfer requirements before execution. By pre-determining the optimal clock frequency and link interface configuration for each layer, the system avoids runtime power management overhead and ensures that maximum power is provided to the link only when actually needed for data transfer, thus reducing both transfer time and overall training time
Data Source
AI summary
Systems, apparatuses, and methods for proactively managing inter-processor network links are disclosed. A computing system includes at least a control unit and a plurality of processing units. Each processing unit of the plurality of processing units includes a compute module and a configurable link interface. The control unit dynamically adjusts a clock frequency and a link width of the configurable link interface of each processing unit based on a data transfer size and layer computation time of a plurality of layers of a neural network so as to reduce execution time of each layer. By adjusting the clock frequency and the link width of the link interface on a per-layer basis, the overlapping of communication and computation phases is closely matched, allowing layers to complete more quickly.