Deep Learning Model Slicing for Heterogeneous Embedded Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Heterogeneous embedded systems face challenges in real-time deep learning inference due to difficulties in processing large amounts of data, such as video data, without cloud support, as existing systems struggle to efficiently allocate tasks across different computing devices with varying performance, power consumption, and communication loads.
Innovation Solution
A method involving a model slicing apparatus that collects execution time and power consumption data for each layer of a deep learning model across various computing devices, predicts performance and power costs, and uses reinforcement learning to slice the model, allocating different layers to devices based on these costs within a given execution time limit, optimizing performance and power efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single heterogeneous embedded system processes deep learning inference without cloud support, then system independence and real-time processing capability are improved, but processing performance and power efficiency deteriorate due to limited computing resources
Solution Approach 1:
The deep learning model is divided into multiple layers that are distributed across different computing devices in the heterogeneous embedded system. Each computing device processes specific model layers, enabling parallel computation and improving overall processing performance while maintaining system independence.
Solution Approach 2:
The system transitions from single-device sequential processing to multi-device parallel processing by adding the dimension of spatial distribution across heterogeneous computing devices, thereby enhancing processing capacity without relying on cloud support.
2Productivity
If deep learning model layers are distributed across multiple computing devices, then processing capability is improved, but system complexity increases due to task allocation and communication coordination
Solution Approach 1:
The optimal allocation of model layers to computing devices is determined in advance using reinforcement learning. This pre-computed allocation strategy eliminates the need for complex real-time decision-making and coordination, simplifying the system operation while maintaining high processing capability.
Solution Approach 2:
The reinforcement learning model uses feedback from performance metrics and power consumption data to optimize the allocation of model layers. This feedback mechanism enables the system to adaptively improve its performance without requiring complex manual configuration or real-time complex coordination.
3Speed
If model layers are allocated to computing devices based on performance optimization, then processing speed is improved, but power consumption increases
Solution Approach 1:
The reinforcement learning model optimizes allocation by considering multiple parameters simultaneously, including processing speed and power consumption. By changing the optimization criteria to balance both parameters, the system achieves acceptable processing speed while controlling power consumption through intelligent device selection and layer assignment.
Data Source
AI summary
A method of slicing a deep learning model for a heterogeneous embedded system includes collecting, by a model slicing apparatus, an execution time and power consumption when each layer corresponding to one layer of a deep learning model including a plurality of layers is executed in each computing device of the heterogeneous embedded system, predicting, by the model slicing apparatus, a performance cost and a power cost when each of the layers is executed in each of the computing devices using the execution time and the power consumption, predicting, by the model slicing apparatus, a communication cost when transmitting information from each of the layers to a next layer in each of the computing devices, and slicing, by the model slicing apparatus, the plurality of layers so that different sliced layers are allocated to each of the computing devices based on the performance cost, the power cost, and the communication cost in a given execution time limit condition using a reinforcement learning model.