ML Inference Energy Prediction Using Model Properties and Counters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Predicting energy consumption of machine learning inference operations on resource-constrained edge devices is desirable due to their limited battery power, which is crucial for determining their capability to execute such operations and manage other software tasks effectively.
Innovation Solution
An apparatus and method for predicting energy consumption of machine learning models by analyzing model property values and performance counters using a prediction model, which includes at least one memory and a processor configured to determine energy consumption based on these factors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning inference operations are executed on edge devices, then the device can perform intelligent processing locally, but the energy consumption increases due to limited battery power
Solution Approach 1:
The system performs preliminary analysis of model property values and determination of performance counters before actual inference execution. This preliminary characterization allows the device to predict energy consumption in advance, enabling better power management decisions before the energy-intensive inference operation begins
Solution Approach 2:
The system uses a prediction model that incorporates performance counters and model property values to provide feedback about expected energy consumption. This feedback mechanism enables the edge device to make informed decisions about whether to execute the inference operation, thereby managing the trade-off between local processing capability and power consumption
2Productivity
If the device executes machine learning models, then it can perform inference operations, but it may overload the device due to limited resources
Solution Approach 1:
The system performs preliminary analysis of model property values and determination of performance counters before actual inference execution. This preliminary characterization allows the device to assess whether it has sufficient resources to handle the incoming inference operation, preventing overload conditions before they occur
Solution Approach 2:
The prediction model provides feedback about expected resource utilization and energy consumption based on model properties and performance counters. This feedback enables the system to make real-time decisions about operation execution, ensuring device stability while maintaining productivity
Data Source
AI summary
Processing circuitry of one or more computing devices obtains model property values associated with a machine learning model by analyzing the machine learning model by the processing circuitry. The processing circuitry determines, based on the model property values, performance counters associated with the machine learning model executing on a processor, by analyzing, using the processing circuitry, the machine learning model and stored data associated with the processor. The processing circuitry predicts, using a prediction model stored at the one or more computing devices, an energy consumption value of executing the machine learning model on the processor based on the performance counters.