Neural Network Subgraph Processing With Dynamic Hardware Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently processing increased operations using artificial neural networks due to the need for optimal allocation of hardware resources based on the current execution state and subgraph characteristics.
Innovation Solution
An artificial neural network module that dynamically allocates hardware and adjusts resource settings, such as voltage and frequency, based on the current operation state and subgraph characteristics using a neural network manager and dispatcher to optimize performance and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hardware resources are statically allocated for neural network operations, then device complexity is reduced and ease of operation is improved, but productivity and energy efficiency deteriorate due to inability to adapt to varying operation states
Solution Approach 1:
The system dynamically allocates hardware resources (PE units, memory blocks, DSP units) based on real-time operation states and subgraph characteristics. The neural network manager continuously monitors operation states and adjusts resource allocation accordingly, transforming static hardware allocation into a dynamic adaptive system that optimizes processing speed for different neural network workloads.
Solution Approach 2:
The system implements feedback mechanisms where the neural network manager receives information about current operation states and performance metrics, then adjusts hardware allocation and resource settings based on this feedback. This closed-loop control enables the system to adapt to varying computational demands and optimize productivity while managing complexity through automated decision-making.
2Productivity
If hardware resources are over-provisioned to handle peak loads, then productivity during peak operations is improved, but use of energy worsens due to resources operating at low utilization during non-peak periods
Solution Approach 1:
The system uses dynamic resource allocation to match hardware utilization with actual computational demands. Instead of maintaining fixed over-provisioned resources, the system scales resource allocation up or down based on real-time operation states, ensuring high productivity during peak loads while minimizing energy consumption during lower utilization periods.
Solution Approach 2:
The system changes operational parameters (frequency, voltage, resource allocation) based on workload characteristics. The neural network manager adjusts hardware resource settings and operating parameters dynamically, allowing the system to maintain peak processing capacity when needed while reducing power consumption during normal operation through parameter optimization.
3Use of energy by moving object
If fine-grained resource allocation is implemented for each subgraph, then energy efficiency and productivity are improved, but device complexity and difficulty of control increase
Solution Approach 1:
The neural network manager acts as an intermediary layer between the computational graph compiler and the hardware resources. It receives subgraph information from the compiler, analyzes operation states, and automatically makes fine-grained resource allocation decisions. This intermediary abstracts the complexity of fine-grained control, making the system manageable while achieving high energy efficiency through precise resource matching.
Solution Approach 2:
The system implements automated self-service mechanisms where the neural network manager independently performs resource allocation, performance monitoring, and optimization without requiring manual intervention. This automation handles the complexity of fine-grained resource control, enabling energy-efficient allocation across multiple subgraphs while reducing the burden on operators.
4Adaptability or versatility
If multiple hardware types are allocated for different operation types, then adaptability and productivity are improved, but device complexity increases due to heterogeneous resource management
Solution Approach 1:
The neural network manager implements a universal resource allocation framework that can manage multiple hardware types (PUs, memory blocks, DSP units) through a unified interface and control mechanism. This multi-functional manager handles diverse hardware resources consistently, enabling adaptable resource matching for different operation types while abstracting away the complexity of heterogeneous management.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
A method for an artificial neural network operation on a plurality of subgraphs may include generating a resource determination trigger corresponding to a target subgraph among the plurality of subgraphs included in a target neural network model; generating a control signal for hardware allocated to the target subgraph and driving resource settings in response to the resource determination trigger; changing, based on the control signal, at least one of hardware allocated to the target subgraph and driving resource settings; and performing an operation on the target subgraph based on the changed hardware and driving resource settings.