Dynamic DNN Partitioning for Edge-Cloud Inference Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for partitioning deep neural networks in edge computing environments are cumbersome and inefficient, requiring complex modeling processes to determine optimal local exit and partition points, which are static and do not adapt to environmental variables like resource utilization and network conditions.
Innovation Solution
A method using reinforcement learning to dynamically determine multiple exit points and partition points for distributed processing between edge devices and clouds, where a processor sets environment and training variables, selects actions based on experience data, and outputs optimal combinations of exit and partition points for efficient inference operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If a distributed DNN structure with static exit points is used, then inference latency is reduced, but adaptability to environmental variables deteriorates
Solution Approach 1:
The patent transforms the static exit point structure into a dynamic one by using reinforcement learning to adaptively determine optimal exit points and partitioning strategies based on real-time environmental conditions such as network status, device resources, and inference requirements, thereby resolving the contradiction between reduced latency and adaptability
2Measurement precision
If complex modeling processes are used to determine optimal partition points, then inference accuracy is improved, but device complexity increases
Solution Approach 1:
The system employs reinforcement learning agents that autonomously learn optimal partitioning strategies through interaction with the environment, eliminating the need for manual complex modeling processes while maintaining high inference accuracy, thus reducing device complexity
3Measurement precision
If more layers are processed at the edge device, then inference accuracy is improved, but energy consumption increases
Solution Approach 1:
The patent dynamically adjusts the number of layers processed at the edge device versus the cloud based on real-time conditions including energy constraints, using reinforcement learning to optimize the partitioning point, thereby achieving high accuracy while minimizing energy consumption
Data Source
AI summary
A processor partitions a deep neural network having a plurality of exit points and at least one partition point in a branch corresponding to each of the exit points, for distributed processing in an edge device and a cloud. The processor sets environmental variables and training variables for training, selects an action to move at least one of an exit point and a partition point from a combination of the exit point and the partition point corresponding to a current state, performs the training by accumulating experience data using a reward according to the selected action and then moves to a next state, and outputs a combination of an optimal exit point and a partition point as a result of the training.