DNN Partitioning via Peer-to-Peer Routing for Edge Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning algorithms face challenges in deployment on resource-constrained embedded devices due to high computation time and energy consumption, particularly in local area networks, where cloud-based solutions suffer from latency and energy inefficiencies.
Innovation Solution
Implementing a peer-to-peer algorithm within a local area network that allows for variable partitioning of deep neural networks and uses dedicated peer-to-peer protocols for feature map routing, enabling edge devices to process images without sending bulky frames over the network, thereby optimizing resource usage and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If cloud-based DNN hosting is used to leverage powerful GPU clusters, then computation power is improved, but end-to-end inference latency and energy consumption increase excessively
Solution Approach 1:
The DNN is segmented into multiple layers that are distributed across different computational points in the network. Each CP processes a specific subset of layers locally, eliminating the need to transmit entire images to remote cloud servers and reducing inference latency while maintaining computation power.
Solution Approach 2:
The architecture transitions from a centralized cloud-based model to a distributed peer-to-peer network model, adding the dimension of spatial distribution across multiple network nodes. This enables parallel processing of different DNN layers across the network, reducing overall inference time.
2Power
If cloud-based DNN hosting is used, then computation power is improved, but energy consumption increases due to intensive network connections
Solution Approach 1:
By segmenting the DNN across multiple local computational points, the system eliminates the need for intensive continuous network connections to remote cloud servers. Each CP processes data locally, reducing network transmission energy consumption while maintaining computation power.
3Adaptability or versatility
If DNN is partitioned between edge devices and cloud, then resource utilization is improved, but latency increases due to bandwidth limitations
Solution Approach 1:
The DNN is segmented into layers distributed across edge devices and computational points in the local network. This segmentation enables parallel processing of different layers, maintaining resource utilization benefits while reducing latency by keeping processing within the local network rather than requiring cloud communication.
Solution Approach 2:
Multiple computational points in the local network can serve multiple functions: processing data from multiple different edge devices, handling different layers of the same DNN, and providing redundant processing capacity. This multi-functionality maintains resource utilization while reducing latency.
4Device complexity
If trivial fixed delivery order topology is used, then system complexity is reduced, but flexibility and load adaptation capability are lost
Solution Approach 1:
The system implements dynamic routing where computational points can adaptively select which edge devices to serve and which DNN layers to process based on current network conditions and load. This dynamic behavior provides load adaptation capability without significantly increasing system complexity.
Solution Approach 2:
Each computational point autonomously determines its processing responsibilities and routing decisions based on local conditions, eliminating the need for complex centralized control while maintaining adaptability to changing network loads.
Data Source
AI summary
A system and method of executing a deep neural network (DNN) in a local area network (LAN) may include executing a partitioned deep neural network in multiple computational nodes (CPs) in devices operating on the LAN. An image frame may be captured by a device. The image frame may be processed by a first layer of the partitioned neural network by a CP operating on the device. In response to the device that captured the image frame determining to request processing assistance from another CP, a request using a peer-to-peer protocol to other CPs on the LAN may be performed. A feature map may be communicated to another CP selected using the peer-to-peer protocol to process the feature map by a next layer of the DNN.