Neural Network Layer Segmentation for Model Parallelization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks face limitations in computation and parameter management, especially when dealing with large models or real-time object detection tasks, as conventional methods like data parallelization do not effectively improve the network structure and can lead to temporal bottlenecks in communication between arithmetic units.
Innovation Solution
An electronic apparatus and method that automatically divides each layer of a trained model into semantically related groups, enabling model parallelization by initializing parameter matrices and split variables, calculating new parameter matrices with block-diagonal structures, and optimizing the model using stochastic gradient descent to minimize loss functions and regularization terms, thereby reducing computation and parameter requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data parallelization is used to accelerate model execution, then computation speed is improved, but communication bottlenecks occur between arithmetic units
Solution Approach 1:
The trained model is divided into multiple independent sub-models based on semantic relationships between layers. Each sub-model can be executed independently on separate arithmetic units without requiring frequent inter-unit communication, thereby eliminating the temporal bottlenecks associated with data parallelization while maintaining accelerated execution speed.
2Quantity of substance
If the number of model parameters is reduced, then computation requirements are decreased, but model performance may degrade
Solution Approach 1:
The model parameters are segmented into distinct sub-models based on semantic relationships. This segmentation allows for more efficient parameter utilization and reduced redundancy while maintaining the essential functional capabilities of the original model, thus decreasing computation requirements without significant performance degradation.
Solution Approach 2:
Different sub-models are optimized with parameter quantities tailored to their specific functional requirements rather than uniformly across the entire model. This local optimization reduces the total number of parameters needed while ensuring each sub-model has sufficient parameters for its specific task, maintaining overall model performance.
3Productivity
If sequential computations are divided into multiple arithmetic units, then computation is accelerated, but temporal bottlenecks occur in communication between units
Solution Approach 1:
The sequential computation chain is segmented into independent sub-models that can be processed in parallel. Each sub-model represents a semantically coherent portion of the original model, allowing arithmetic units to operate independently without requiring complex communication infrastructure, thus accelerating computation while simplifying the system architecture.
Data Source
AI summary
An electronic apparatus is provided. The electronic apparatus includes: a memory storing a trained model including a plurality of layers; and a processor initializing a parameter matrix and a plurality of split variables of a trained model, calculating a new parameter matrix having a block-diagonal matrix for the plurality of split variables and the trained model to minimize a loss function for the trained model, a weight decay regularization term, and an objective function including a split regularization term defined by the parameter matrix and the plurality of split variables, vertically splitting the plurality of layers according to the group based on the computed split parameters and reconstruct the trained model using the computed new parameter matrix as parameters of the vertically split layers.