Neural Network Partitioning Across Accelerators for Speed and Power Balance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As artificial neural network models become larger, they require significant resources to operate and manage efficiently, especially when resources are limited, necessitating improved methods for execution.
Innovation Solution
The method involves dividing the neural network model into multiple node groups and allocating these groups to hardware accelerators with varying operating speeds, recording activation area information and call counts, and adjusting the allocation based on this data to optimize execution time and reduce power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the artificial neural network model is divided into node groups and allocated to multiple hardware accelerators, then the execution time is reduced and productivity is improved, but the device complexity increases due to multiple accelerators and allocation management
Solution Approach 1:
The artificial neural network model is divided into multiple node groups, where each node group contains a subset of nodes from the original model. This segmentation allows parallel execution on multiple hardware accelerators, reducing overall execution time while managing complexity through structured division of the computational workload
Solution Approach 2:
The allocation of node groups to hardware accelerators is dynamically adjusted based on recorded activation area information and call counts. The system learns from actual usage patterns and reconfigures the allocation to optimize performance, making the system adaptable rather than static
2Speed
If node groups are allocated to faster hardware accelerators, then the execution speed is improved, but the power consumption increases due to higher operating speeds
Solution Approach 1:
The system changes the allocation parameters by recording activation area information and call counts, then uses this data to dynamically adjust which node groups are assigned to faster (higher power consumption) versus slower (lower power consumption) hardware accelerators, optimizing the balance between speed and energy usage
Solution Approach 2:
The allocation configuration is made dynamic through continuous recording of activation patterns and call frequencies, allowing the system to adaptively reconfigure accelerator assignments based on actual workload characteristics rather than using a fixed allocation strategy
3Quantity of substance
If the neural network model is divided into multiple node groups and allocated across hardware accelerators, then memory and computation limitations are overcome, but the system complexity increases due to allocation management
Solution Approach 1:
By dividing the neural network model into node groups that can be distributed across multiple hardware accelerators, the system overcomes individual memory and computation limitations of single accelerators, enabling processing of larger models that would otherwise not fit in memory
Solution Approach 2:
The system implements feedback mechanisms by recording activation area information and call counts from actual model execution, then uses this feedback to optimize the allocation of node groups to hardware accelerators, automatically improving the system without manual intervention
Data Source
AI summary
A method of operating an artificial neural network model including a plurality of nodes includes: dividing the artificial neural network model into a divided artificial neural network including plurality node groups using a first grouping manner, allocating the plurality of node groups to a plurality of first hardware accelerators and a plurality of second hardware accelerators using a first corresponding manner to generate an allocation, executing the divided artificial neural network model on a plurality of input values to generate a plurality of inference results values, for each of the plurality of inference result values, recording activation area information of the plurality of node groups and a call count, and performing at least one of a first operation to change the allocation and a second operation to change the divided artificial neural network based on the activation area information and the call count.