Hidden-Layer Routing for Disaggregated Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning applications on edge devices are hindered by the need for expensive hardware acceleration or downsizing neural networks, which is inefficient and resource-intensive, especially in resource-constrained networks.
Innovation Solution
The system enables dynamic partitioning and distribution of hidden layers of a deep-learning model across multiple edge devices using IP routing, allowing for a disaggregated artificial neural network that can efficiently perform radio resource management (RRM) without requiring additional hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deep learning applications are run on edge devices, then processing capability is improved, but hardware cost and complexity increase
Solution Approach 1:
The neural network is divided into multiple layers that are distributed across different edge devices. Each device processes a specific layer or subset of layers, transforming the monolithic processing task into segmented parallel operations. This allows complex deep learning workloads to be executed on resource-constrained devices without requiring each device to have full hardware acceleration capabilities.
Solution Approach 2:
The system enables general-purpose edge devices to perform deep learning inference by implementing a unified layer processing interface. Any edge device with basic computational capabilities can participate in the distributed inference process, eliminating the need for specialized hardware acceleration units on each device while maintaining processing capability.
2Device complexity
If neural networks are downsized to fit resource-constrained devices, then device requirements are reduced, but model accuracy and performance deteriorate
Solution Approach 1:
Instead of downsizing the entire neural network model, the system segments the full-precision model into layers distributed across multiple devices. Each device handles a portion of the computation with full model precision maintained throughout the distributed processing chain, avoiding accuracy degradation while reducing individual device requirements.
Solution Approach 2:
Multiple edge devices are merged into a collaborative computing system where each device contributes its computational resources. The combined processing power of multiple devices replaces the need for a single powerful device or a downsized model, maintaining full model accuracy while adapting to resource constraints.
3Productivity
If deep learning workloads are distributed across multiple edge devices, then processing throughput is improved, but network communication overhead increases
Solution Approach 1:
The system optimizes data flow by routing intermediate tensors through local network connections between nearby edge devices rather than centralized cloud infrastructure. This local quality approach minimizes communication distance and energy consumption while maintaining distributed processing throughput benefits.
Data Source
AI summary
Novel techniques and mechanisms enable processing of heavy deep learning workloads on standard edge network devices to optimize the overall inference throughput of the network while meeting Service Level Agreement(s) (SLAs). The techniques can include receiving a deep learning model, determining a graph structure of the deep learning model including neurons organized in layers (the layers including an input layer, a plurality of hidden layers, and an output layer), assigning to a first IP subnet, at least a part of a first hidden layer of the plurality of hidden layers, assigning to a second IP subnet, at least a part of a second hidden layer of the plurality of hidden layers, and deploying the parts of the first and second hidden layers to edge devices as containerized applications with assigned IP addresses, which may be hidden from the user and/or third party application.