Multi-Processor Deep Learning Pipelines for Faster Network Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processors struggle to efficiently execute deep learning networks due to the high computational demands and repetitive tasks, which can be optimized through a multi-processor system with a pipeline architecture that leverages a general-purpose graphics processing unit (GPGPU) for parallel processing.
Innovation Solution
A multi-processor system with a pipeline architecture that utilizes a GPGPU for executing deep learning networks, where a network workload analyzer distributes computations across processing cores, optimizing tasks for efficient execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If a single graphics processor is used to execute deep learning networks, then the device complexity is low, but the processing speed and computational power are insufficient
Solution Approach 1:
The deep learning network execution is divided into multiple stages (data loading, preprocessing, forward propagation, backward propagation, parameter update) that are distributed across different processing cores in the multi-processor system, allowing parallel execution of independent stages
Solution Approach 2:
Multiple processing cores are combined into a unified multi-processor system with shared memory and coordination mechanisms, merging their computational power to handle deep learning workloads collectively rather than using a single processor
2Loss of time
If traditional sequential processing is used for deep learning networks, then the device complexity is low, but the processing time is excessive
Solution Approach 1:
Data preprocessing operations (normalization, augmentation, formatting) are performed in advance and stored in memory, so that when the deep learning network execution begins, the processing cores can immediately start computation without waiting for data preparation
Solution Approach 2:
The pipeline architecture ensures continuous data flow through the system where data loading, preprocessing, and network computation occur in overlapping time periods, eliminating idle time and maintaining continuous productive action across all cores
3Productivity
If a multi-processor pipeline system is implemented, then the processing speed increases, but the device complexity increases
Solution Approach 1:
The processing cores are designed to execute multiple types of operations (data loading, preprocessing, forward propagation, backward propagation, parameter update) rather than being dedicated to single functions, reducing the need for specialized hardware for each operation type
4Measurement precision
If high capacity networks are trained on larger datasets, then the measurement precision and model accuracy improve, but the computational processing demand increases significantly
Solution Approach 1:
Large datasets are divided into batches that are processed in parallel across multiple cores, and the network training is segmented into alternating phases of forward propagation and backward propagation that can be efficiently distributed across the processor pipeline
Data Source
AI summary
Methods and systems are disclosed using an execution pipeline on a multi-processor platform for deep learning network execution. In one example, a network workload analyzer receives a workload, analyzes a computation distribution of the workload, and groups the network nodes into groups. A network executor assigns each group to a processing core of the multi-core platform so that the respective processing core handle computation tasks of the received workload for the respective group.