Edge DNN Partitioning for Latency and Security
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning applications on edge devices are hindered by the computational limitations of these devices, as most Deep Neural Networks (DNNs) are too heavy to run efficiently on standard edge devices, leading to the need for expensive additional hardware or cloud offloading, which can introduce latency and security issues.
Innovation Solution
An orchestration system that efficiently partitions and distributes a deep neural network across edge devices within a network, by analyzing the neural network model to determine optimal split points and deploying workload portions to computing devices with suitable resources, thereby optimizing throughput and reducing the need for additional hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep neural networks are run on edge devices, then data security and latency are improved, but computational limitations prevent heavy DNNs from running efficiently
Solution Approach 1:
The patent divides a deep neural network into multiple partitions, where early layers are deployed to edge devices for local processing and later layers are deployed to cloud or remote edge servers. This segmentation allows the system to maintain data security for local processing while overcoming computational limitations by offloading heavy computations to more powerful remote resources.
2Productivity
If additional dedicated hardware (GPUs, TPUs) is purchased to run DNNs on edge devices, then processing capability is improved, but cost and engineering effort increase significantly
Solution Approach 1:
The patent combines edge devices with standard processors with remote cloud or edge server resources that have dedicated hardware accelerators. This merging allows the system to achieve high processing capability for deep neural networks without requiring expensive dedicated hardware at the edge, as the remote resources provide the necessary computational power while the edge device handles data collection and preliminary processing.
3Productivity
If cloud offloading is used to run heavy DNNs, then computational power is improved, but latency and network dependency increase
Solution Approach 1:
The patent segments the neural network processing pipeline so that early layers that require minimal computation are executed locally at the edge device, providing low-latency responses for time-sensitive operations. Later layers that require heavy computation are executed remotely, utilizing cloud or edge server resources. This segmentation optimizes the balance between latency and computational power by handling different processing stages at different locations.
4Device complexity
If a monolithic pre-trained DNN is deployed on edge devices, then model simplicity is maintained, but the model is too heavy for standard edge device resources
Solution Approach 1:
The patent partitions the monolithic pre-trained DNN into multiple segments that can be distributed across edge devices and remote servers. Each segment is deployed to appropriate resources based on computational requirements and data locality considerations. This segmentation allows the heavy model to run on standard edge devices without requiring hardware upgrades, as the computational load is distributed across multiple locations and resources.
Data Source
AI summary
Techniques and mechanisms are described for enabling a user to run heavy deep learning workloads on standard edge networks without off-loading computation to a cloud, leveraging the available edge computing resources, and efficiently partitioning and distributing a Deep Neural Network (DNN) over a network. The techniques enable the user to split a workload into multiple parts and process the workload on a set of smaller, less capable compute nodes in a distributed manner, without compromising on performance, and while meeting a Service Level Objective (SLO).