Neural Network Segmentation for Execution Unit Load Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks face inefficiencies due to varying operation times among operator groups assigned to execution units, leading to low work efficiency as operators must wait for the slowest group to complete operations.
Innovation Solution
A method for segmenting neural networks by dividing operators into sub-operators and assigning them to execution units based on operation time standards, such as using an average operation time threshold, to minimize time discrepancies and optimize assignment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If operator groups are assigned to execution units based on simple division, then the neural network can be processed in parallel, but the operation time difference among execution units increases, causing waiting time and reducing work efficiency
Solution Approach 1:
The patent segments operators into sub-operators and further divides them into multiple operator groups, assigning different groups to different execution units. This multi-level segmentation allows for more granular control over task distribution, enabling better balancing of workloads across execution units while maintaining parallel processing capabilities
Solution Approach 2:
The patent changes the parameter of operator grouping by introducing multiple segmentation methods and selecting the optimal one based on operation time discrete degree. By dynamically adjusting how operators are grouped and assigned, the system optimizes the distribution of computation time across execution units, minimizing waiting time while preserving parallel processing efficiency
2Productivity
If operator groups are assigned to execution units, then load balancing can be achieved, but the discrete degree of operation time among execution units is difficult to control, reducing work efficiency
Solution Approach 1:
The patent calculates the operation time discrete degree as a feedback metric to evaluate the quality of operator group assignment. By measuring the variance in operation times across execution units and using this information to select the optimal segmentation method, the system continuously optimizes the assignment to achieve better load balancing with controlled operation time differences
Solution Approach 2:
The patent introduces dynamic selection of segmentation methods based on the calculated discrete degree of operation times. Rather than using a fixed assignment strategy, the system adapts its segmentation approach according to the actual operation time characteristics, enabling precise control over the distribution of computation times while maintaining effective load balancing
Data Source
AI summary
A method for segmenting a neural network, implemented in an electronic device, includes assigning a plurality of operator groups to a plurality of execution units based on a plurality of segmentation methods; determining operation times of the plurality of execution units and discrete degree of the operation times of the plurality of execution units in each of the plurality of segmentation methods; and segmenting the neural network according to the segmentation method with a lowest discrete degree.