Dynamic DNN Split Point and Container Activation in Edge Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing split computing techniques fail to optimize deep neural network (DNN) inference latency and resource efficiency in serverless edge computing environments, as they do not consider dynamic container instance activation times, leading to suboptimal performance and resource waste.
Innovation Solution
A method using Constrained Markov Decision Process (CMDP) and linear programming to dynamically determine the optimal DNN splitting point and container instance activation status, minimizing inference latency and resource consumption by formulating a stochastic policy based on real-time resource information and data transmission rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If the container instance is maintained in an inactive state to reduce resource waste, then resource efficiency is improved, but a long initial response time is required for instance activation
Solution Approach 1:
The system pre-activates container instances before they are actually needed by predicting future request patterns. This preliminary action allows the container to be ready in advance, eliminating the long activation time penalty when requests arrive, while still maintaining low resource consumption by activating only when predicted to be needed.
Solution Approach 2:
The system dynamically adjusts the activation status of container instances based on real-time and historical request patterns. Rather than a static active/inactive state, the system continuously optimizes the state to balance resource efficiency and response time, using machine learning predictions to determine when activation is warranted.
2Loss of time
If the container instance is active at all times to ensure short initial response time, then response time is improved, but resource availability is low
Solution Approach 1:
Instead of being continuously active, the system performs preliminary activation only when predicted to be needed based on request patterns. This selective pre-activation ensures short response times when requests arrive during predicted active periods, while avoiding the resource waste of continuous activation during idle periods.
Solution Approach 2:
The system uses feedback from historical request patterns and real-time conditions to dynamically determine when to activate container instances. This feedback mechanism allows the system to learn optimal activation timing, reducing resource consumption while maintaining service level agreements for response time.
3Ease of manufacture
If existing split computing techniques are applied without considering container instance activation, then the method is simple to implement, but desired inference latency cannot be achieved
Solution Approach 1:
The system segments the DNN model into head and tail models at optimal splitting points, distributing computation between mobile device and edge cloud. This segmentation enables the system to achieve desired inference latency by performing computationally intensive operations on the edge cloud while keeping data processing on the mobile device, all while managing container activation status.
Solution Approach 2:
The system dynamically changes the splitting point parameter of the DNN model based on real-time conditions including container activation status, network conditions, and device capabilities. This parameter adjustment allows the system to optimize inference latency adaptively while managing the complexity of container instance management.
Data Source
AI summary
Disclosed is a split computing device operating in a serverless edge computing environment. The split computing device includes a transceiver configured to receive resource information of a terminal from the terminal and to measure a data transmission rate between the terminal and the split computing device in a process of receiving the resource information of the terminal; and a splitting point deriver configured to determine a splitting point of a deep neural network (DNN) model for split computing and an activation status of a container instance for each of tail models of a DNN corresponding to the respective splitting points using the resource information of the terminal, the data transmission rate, and resource information of the split computing device.