Reinforcement Learning DNN Allocation to Processing Units
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Efficient allocation of multiple deep neural networks (DNNs) to various processing units is challenging due to considerations such as utilization, temperature, process time, accuracy, energy consumption, and memory access competition, which existing methods fail to address effectively.
Innovation Solution
A system utilizing reinforcement learning, specifically Q-learning, to select the optimal state and action for allocating DNNs to processing units based on preset states and actions, updating the Q-table to improve allocation efficiency, thereby optimizing the distribution of DNNs across multiple processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple DNNs are allocated to multiple processing units, then processing capacity and functionality are improved, but allocation complexity and resource management difficulty increase
Solution Approach 1:
The system employs reinforcement learning agents that autonomously make allocation decisions without manual intervention. The agent continuously learns from environmental feedback (rewards based on utilization, temperature, process time, accuracy, energy consumption, and memory access) to optimize DNN-to-processing-unit mapping automatically, resolving the complexity of manual resource management while maintaining high processing capacity
Solution Approach 2:
The allocation system incorporates a reward function that provides continuous feedback on allocation quality based on multiple parameters including processing utilization, temperature control, process time, accuracy, energy consumption, and memory access patterns. This feedback mechanism enables the reinforcement learning agent to iteratively improve allocation decisions, balancing productivity with resource management complexity
2Speed
If DNN processing is performed on mobile devices to reduce network latency and communication burden, then response time is improved, but computational and memory requirements increase
Solution Approach 1:
The system allocates different DNNs to different processing units based on their specific characteristics and requirements. Each processing unit (CPU, GPU, NPU) is assigned DNNs that match its computational strengths, optimizing the balance between response time and energy consumption. This localized optimization ensures that computationally intensive tasks are handled by appropriate units while minimizing overall energy usage
Solution Approach 2:
The reinforcement learning agent dynamically adjusts DNN allocations based on real-time system state changes including temperature, energy consumption, and processing workload. This dynamic adaptation allows the system to respond to varying computational demands while optimizing energy usage, transitioning between high-performance and energy-saving modes as needed
3Reliability
If DNN allocation considers multiple parameters (utilization, temperature, process time, accuracy, energy consumption, memory access), then allocation optimization is improved, but decision-making complexity increases
Solution Approach 1:
The reinforcement learning agent autonomously processes and integrates multiple allocation parameters without external intervention. The agent internally manages the complexity of balancing utilization, temperature, process time, accuracy, energy consumption, and memory access by learning optimal trade-offs through environmental interaction, transforming multi-parameter decision complexity into automated intelligent control
Solution Approach 2:
The system transforms complex multi-parameter optimization into a structured reward function that quantifies allocation quality. By converting multiple parameters into a unified reward signal, the reinforcement learning agent can systematically optimize allocation decisions without being overwhelmed by the complexity of individual parameter management, achieving reliable optimization through parameter integration
Data Source
AI summary
Provided is a system configured to respectively allocate a plurality of deep neural networks to a plurality of processing units according to a particular action having maximum quality in a particular state. In addition, provided is a system configured to respectively allocate a plurality of deep neural networks to a plurality of processing units according to an action having maximum quality in a current state, and to update quality of an action selected in the current state by using a calculated reward based on a process of the plurality of deep neural networks by the allocated plurality of processing units.