Distributed Neural Network Partitioning Balancing Edge Power and Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Edge devices with limited resources and power constraints struggle to execute complex neural networks within predefined power and latency thresholds, necessitating a method to partition neural networks across multiple Edge nodes for efficient execution.
Innovation Solution
A neural network partitioning system that determines a first portion of layers to be executed on a sustainably-powered Edge node and a second portion on a more powerful Edge node or cloud data center, based on ambient conditions, energy consumption, and service level agreement timeframe.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the entire neural network is executed on a single Edge node, then computation latency is reduced, but power consumption exceeds available power supply
Solution Approach 1:
The patent divides the neural network into multiple partitions (first portion and second portion) that can be distributed across different Edge nodes. This segmentation allows the computation to be spread across multiple devices, reducing the power consumption burden on any single node while maintaining low latency through localized processing.
2Use of energy by moving object
If the neural network is partitioned across multiple Edge nodes, then power consumption is reduced, but transmission time increases
Solution Approach 1:
The patent optimizes the partitioning by considering the specific characteristics of each Edge node, including its computational capabilities and power supply. The first Edge node is selected to execute the first portion of the neural network based on its sustainable power availability, while the second node executes the second portion. This local optimization minimizes transmission time while respecting power constraints.
3Power
If a more powerful Edge node or cloud data center executes the second portion of the neural network, then computation capability is improved, but transmission energy consumption increases
Solution Approach 1:
The patent introduces an intermediary selection process that identifies a second Edge node with appropriate computational capabilities to execute the second portion of the neural network. This intermediary node serves as a middle ground between the first Edge node and the cloud data center, providing sufficient computation power while minimizing transmission distance and energy consumption.
Data Source
AI summary
Methods, apparatus, systems, and articles of manufacture are disclosed to partition neural network models for executing at distributed Edge nodes. An example apparatus includes processor circuitry to perform at least one of first, second, or third operations to instantiate power consumption estimation circuitry to estimate a computation energy consumption for executing the neural network model on a first edge node, network bandwidth determination circuitry to determine a first transmission time for sending an intermediate result from the first edge node to a second or third edge node, power consumption estimation circuitry to estimate a transmission energy consumption for sending the intermediate result to the second or the third edge node, and neural network partitioning circuitry to partition the neural network model into a first portion to be executed at the first edge node and a second portion to be executed at the second or third edge node.