Energy-Aware DNN Deployment on Edge Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning solutions lack visibility into energy-saving capabilities and fail to efficiently manage energy consumption in edge inference computing environments, as they do not account for the unique energy characteristics of edge devices and do not deploy workloads based on energy efficiency.
Innovation Solution
The method involves energy rating deep neural network models using software optimization techniques like quantization and network pruning, and energy scoring edge devices based on their static and dynamic energy characteristics, enabling an energy-aware deployment policy to match energy-rated models with suitable edge devices for efficient inference computing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If deep neural network models are deployed on edge devices without energy-aware deployment, then model deployment simplicity is improved, but energy efficiency deteriorates
Solution Approach 1:
The system performs preliminary actions by pre-rating deep neural network models based on their energy efficiency characteristics and pre-scoring edge devices based on their energy properties before deployment. This allows the actual deployment process to simply involve matching pre-computed ratings and scores, maintaining ease of operation while achieving energy efficiency through the preliminary characterization work.
Solution Approach 2:
The patent introduces energy ratings for models and energy scores for devices as intermediary metrics that mediate between the deployment process and energy efficiency requirements. These intermediaries enable simple deployment procedures while ensuring energy-efficient matching, as the complex energy efficiency evaluation is encapsulated within the rating and scoring mechanisms rather than the deployment process itself.
2Use of energy by moving object
If energy-rated models are matched with energy-scored devices, then energy efficiency is improved, but deployment complexity increases
Solution Approach 1:
The patent segments the deployment system into distinct components: a model rating component that evaluates energy efficiency of deep neural network models, a device scoring component that assesses energy properties of edge devices, and a matching component that pairs them. This segmentation allows each component to focus on specific tasks, making the overall system manageable despite the increased functionality for energy-efficient matching.
Solution Approach 2:
The system changes parameters by introducing energy ratings and energy scores as new characterization parameters for models and devices respectively. By transforming the deployment problem into a parameter-based matching problem, the system achieves energy efficiency through parameter-driven selection while keeping the deployment process relatively simple through automated parameter comparison and matching algorithms.
3Use of energy by moving object
If software optimization and hardware accelerators are used during training, then model energy efficiency is improved, but training complexity increases
Solution Approach 1:
The patent implements feedback mechanisms during training by monitoring energy consumption and using this information to guide optimization decisions. The system provides feedback about energy efficiency to the training process, allowing it to adjust model parameters, select appropriate hardware accelerators, and optimize software implementations based on actual energy performance data, thereby achieving energy efficiency through informed iterative improvement.
Solution Approach 2:
The patent employs composite approaches by combining multiple optimization techniques including software-based optimizations (such as algorithm improvements, quantization) and hardware-based accelerators (such as specialized processing units) to achieve model energy efficiency. This composite strategy leverages the strengths of both software and hardware approaches while managing complexity through integrated optimization frameworks that coordinate multiple techniques.
Data Source
AI summary
Deploying energy-rated deep neural network models on energy-scored edge devices is provided. An overall energy efficiency rating is assigned to a deep neural network model based on utilizing software optimization and hardware accelerators during training of the deep neural network model. Energy scores are assigned to respective edge devices in an edge inference computing environment based on properties of each respective edge device. Particular edge devices are selected that have a corresponding energy score within a defined edge device energy score range for the overall energy efficiency rating that corresponds to the deep neural network model. The deep neural network model is deployed to the particular edge devices that have a corresponding energy score within the defined edge device energy score range for the overall energy efficiency rating that corresponds to the deep neural network model.