Heterogeneous CPU-GPU CNN Training Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for training Convolutional Neural Networks (CNNs) rely solely on GPUs, leading to underutilization of CPU resources and limitations in model size due to GPU memory constraints, with no effective integration of CPU and GPU for parallel processing.
Innovation Solution
The HetNet and HybNet approaches partition data and model layers between CPUs and GPUs for parallel training, allowing for static or dynamic partitioning to balance workload and merge gradients using a weighted sum, enabling efficient utilization of both processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If GPU is used for training CNN, then training speed is improved, but CPU resources are wasted
Solution Approach 1:
The patent segments the CNN training workload by partitioning data batches between CPU and GPU. The CPU processes some batches while the GPU processes others, allowing both processors to work in parallel. This segmentation resolves the contradiction by utilizing CPU resources that would otherwise be idle while maintaining GPU-accelerated training speed.
2Productivity
If GPU is used for training CNN, then training speed is improved, but model size is restricted due to limited GPU memory
Solution Approach 1:
The patent introduces a new dimension to the training architecture by adding CPU processing capability to the traditional GPU-only approach. This dimensional expansion allows the system to handle larger models by distributing computation across both CPU and GPU, effectively bypassing the GPU memory limitation while preserving training speed advantages.
3Productivity
If CPU and GPU are used together for training, then hardware utilization is improved, but system complexity increases
Solution Approach 1:
The patent implements feedback mechanisms to dynamically manage the CPU-GPU training system. By monitoring system state and adjusting workload distribution between CPU and GPU based on performance feedback, the system optimizes hardware utilization while managing complexity through adaptive control rather than static complex architecture.
Data Source
AI summary
Convolution neural networks are able to be trained using a GPU and a CPU. To efficiently utilize a device's resources, the HetNet and HybNet approaches have been developed. The HetNet approach separates batches into partitions such that the GPU and CPU process separate batches. The HybNet approach separates the layers of a convolution neural network for the GPU and CPU.