Neural Network Processor Allocation by Layer Slicing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices processing neural network models face variability in processing speed and accuracy due to differences in processor performance and layer characteristics, necessitating a method to optimize layer allocation across multiple processors.
Innovation Solution
The method involves allocating slices of neural network layers to processors based on processing times, including switching times between processors, to optimize processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If layers are allocated to processors without considering processing times, then device complexity is reduced, but processing speed decreases
Solution Approach 1:
The patent implements dynamic layer allocation by calculating processing times for each processor-layer combination and using this information to determine optimal assignments. The system dynamically adjusts the allocation based on measured performance characteristics rather than using fixed or heuristic rules, thereby optimizing processing speed while managing complexity through data-driven decision making.
Solution Approach 2:
The patent changes the allocation parameter from simple one-to-one mapping to a many-to-many mapping that considers processing time parameters. By introducing processing time as a key parameter in the allocation decision and using slices to represent portions of layers, the system optimizes the balance between processing speed and allocation complexity through parameter-based control.
2Loss of time
If all layers are processed by a single processor, then allocation complexity is minimized, but processing time increases
Solution Approach 1:
The patent segments layers into slices, where each slice can be independently allocated to different processors. This segmentation enables parallel processing while maintaining manageable complexity through modular units. The slicing approach allows the system to distribute computational work across multiple processors without requiring complex coordination, thereby reducing processing time while keeping allocation complexity at acceptable levels.
3Reliability
If layers are allocated based on processor performance without considering switching times, then allocation simplicity is maintained, but processing accuracy decreases
Solution Approach 1:
The patent incorporates feedback mechanisms by measuring actual processing times for each processor-layer combination and using this feedback to refine allocation decisions. The system continuously monitors performance characteristics and adjusts allocations accordingly, thereby improving processing accuracy. This feedback-driven approach manages complexity by using empirical data to guide allocation rather than relying on theoretical or static performance models.
Data Source
AI summary
A method of processing a neural network model by using a plurality of processors includes allocating at least one slice to each layer from among a plurality of layers included in the neural network model, allocating each layer from among the plurality of layers to the plurality of processors based on respective processing times of the plurality of processors for processing each of the at least one slice, and processing the neural network model by using the plurality of processors based on a result of the allocation.