Neural Network Parallelization on Manycore Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for processing neural network models in manycore systems face inefficiencies in parallelization, particularly in assigning cores to layers and optimizing communication costs, which hinders rapid convergence during training and inference operations.
Innovation Solution
A method that generates a profiling result for a target neural network based on model and architecture information, determines an assignment strategy to assign cores to layers, and creates a parallelization strategy for the manycore system, prioritizing computational operations over communication traffic to optimize processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If model parallelization and data parallelization are used to speed up neural network processing, then processing speed improves, but system complexity and communication overhead increase
Solution Approach 1:
The patent segments the neural network model into multiple layers and assigns them to different cores in a manycore system. Each core processes specific layers independently, enabling parallel computation while maintaining manageable system complexity through modular organization of processing tasks.
Solution Approach 2:
The patent introduces a new dimension of parallelization by utilizing manycore architecture with multiple cores within clusters, transitioning from traditional single-core or multi-core systems. This dimensional expansion enables simultaneous execution of multiple neural network layers across numerous cores, significantly accelerating processing speed.
2Productivity
If more cores are assigned to process neural network layers in parallel, then processing throughput increases, but communication cost between cores increases
Solution Approach 1:
The patent applies local quality by organizing cores into clusters and assigning specific neural network layers to specific clusters. This localized assignment reduces unnecessary communication between distant cores, as each cluster processes its assigned layers independently with minimal inter-cluster communication, thereby reducing communication costs while maintaining high throughput.
3Speed
If neural network layers are distributed across multiple cores, then convergence speed improves, but assignment optimization complexity increases
Solution Approach 1:
The patent employs preliminary action by pre-profiling neural network layers to determine their computational characteristics before assignment. This pre-analysis enables optimized assignment strategies that distribute layers across cores based on their computational demands, achieving rapid convergence while simplifying the optimization process through advance planning rather than complex real-time decision-making.
Data Source
AI summary
A parallelization method includes: generating a profiling result by performing profiling on a target neural network based on model information of the target neural network and architecture information of a manycore system; determining an assignment strategy to assign a plurality of cores of each of a plurality of clusters of the manycore system to a plurality of layers of the target neural network, based on the profiling result; and generating a parallelization strategy for parallel processing of the manycore system based on the assignment strategy.