Distributed Neural Network Load Splitting for Energy-Aware Offloading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed neural networks face challenges with static load distribution, where lower layers are processed locally but lack abstraction for final outputs, require specific exit layers, and are limited to classification tasks, especially in resource-constrained devices.
Innovation Solution
A dynamic load distribution method that estimates energy usage for processing and transmission in resource-constrained devices, allowing flexible offloading to edge clouds based on energy efficiency, using encoding and compression techniques to optimize power consumption and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If lower layers are processed locally in distributed neural networks, then energy consumption is reduced, but processing accuracy and abstraction quality deteriorate
Solution Approach 1:
The patent implements dynamic load distribution where the division of processing layers between local device and cloud service is not fixed but adapts based on real-time conditions such as energy availability, task complexity, and performance requirements. This allows the system to optimize the balance between energy consumption and processing accuracy dynamically rather than using a static lower-layer-local approach
Solution Approach 2:
The system changes key parameters including the split point between local and cloud processing, compression levels for layer outputs, and selection of which layers to process locally based on energy constraints and accuracy requirements. This enables flexible adjustment of the trade-off between energy savings and processing quality
2Device complexity
If static load distribution is used, then device complexity is reduced, but adaptability and versatility worsen
Solution Approach 1:
The patent introduces dynamic decision-making mechanisms that adjust load distribution based on task type, energy availability, and performance requirements. The system can adapt to different tasks such as classification, object detection, and segmentation by selecting appropriate processing strategies without requiring complex hardware modifications
Solution Approach 2:
The system designs a universal load distribution framework that can handle multiple task types (classification, object detection, segmentation) and different energy scenarios through software-based control. This avoids the need for task-specific hardware configurations while maintaining high adaptability across diverse applications
3Measurement precision
If layer output is transmitted to cloud service, then processing accuracy is improved, but energy consumption and transmission time increase
Solution Approach 1:
The patent applies compression techniques to layer outputs before transmission, adjusting the compression level based on the trade-off between transmission energy savings and accuracy preservation. The system selectively compresses or transmits full-precision data depending on the importance of the layer and energy constraints
Solution Approach 2:
The system applies different processing and transmission strategies to different layers based on their specific characteristics and importance. Critical layers that require high precision are transmitted with minimal compression, while less critical layers undergo higher compression or are processed locally, optimizing the overall energy-accuracy trade-off
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
A method for dynamic load distribution for a distributed neural network is disclosed. The method comprises estimating (103), in a device of the neural network, an energy usage for processing at least one non-processed layer in the device, and estimating (106), in the device of 5 the neural network, an energy usage for transmitting layer output of at least one processed layer to a cloud service of the neural network for processing. The method further comprises comparing (107), in the device of the neural network, the estimated energy usage for processing the at least one non-processed layer in the device with the estimated energy usage for transmitting the layer output of the at least one processed layer to the cloud service. The 10 method furthermore comprises determining (108) to process the at least one non-processed layer in the device when the estimated energy usage for transmitting the layer output of the at least one processed layer to the cloud service is equal or greater than the estimated energy usage for processing the at least one non-processed layer, and determining (110) to transmit the layer output of the at least one processed layer to the cloud service for processing 15 subsequent layers when the estimated energy usage for transmitting the layer output of the at least one processed layer to the cloud service is less than the estimated energy usage for processing the at least one non-processed layer in the device. Corresponding computer program product, apparatus, cloud service assembly, and system are also disclosed.20