NPU Frequency Prediction Using Static and Runtime Model Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing NPU DVFS technology inaccurately predicts operating frequency based on past information, leading to unnecessary power loss due to insufficient consideration of neural network model characteristics and hardware correlations.
Innovation Solution
A frequency control method and system for NPUs that extracts static and dynamic feature data from neural network models to predict operating frequency, using a prediction model constructed with scaling factors to optimize power consumption while meeting execution time targets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing NPU DVFS technology uses only past operating frequency information for prediction, then the prediction method is simple, but the prediction accuracy is low resulting in unnecessary power loss
Solution Approach 1:
The patent segments the feature data into two distinct categories: static feature data (determined within offline time) and dynamic feature data (determined within runtime). This segmentation allows the system to process different types of information separately and combine them for more accurate frequency prediction, directly addressing the limitation of using only past frequency information.
Solution Approach 2:
The patent performs preliminary extraction and processing of static feature data during offline time before the NPU executes the neural network model. This preliminary action enables the system to prepare prediction inputs in advance, improving prediction accuracy without adding runtime complexity.
2Productivity
If NPU operates at high frequency to meet execution time targets, then performance is improved, but power consumption increases
Solution Approach 1:
The patent implements dynamic frequency adjustment by constructing a prediction model that determines the optimal operating frequency based on extracted feature data. The system dynamically scales the NPU frequency to match the actual computational requirements of each neural network model, avoiding both over-provisioning (wasted power) and under-provisioning (missed deadlines).
Solution Approach 2:
The patent changes the operating frequency parameter of the NPU based on predicted workload characteristics. By adjusting this critical parameter according to model-specific features rather than using fixed or简单地 scaled frequencies, the system optimizes the balance between execution speed and power consumption.
3Loss of energy
If NPU frequency is predicted based only on past information without considering model characteristics, then the control method is simple, but power efficiency is poor
Solution Approach 1:
The patent creates a universal prediction framework that can handle multiple types of neural network models with different characteristics. The feature extraction system is designed to work across various model types, extracting relevant static and dynamic features that capture model-specific properties, enabling accurate frequency prediction for diverse workloads through a single unified system.
Solution Approach 2:
The patent incorporates feedback mechanisms where the prediction model uses both static features (from offline analysis) and dynamic features (from runtime monitoring) to continuously refine frequency predictions. This feedback loop allows the system to learn from actual execution patterns and improve power efficiency over time while adapting to different model characteristics.
Data Source
AI summary
A frequency control method for a neural processing unit according to at least one embodiment includes receiving information about a neural network model to be executed on a neural processing unit (NPU), extracting static feature data determined within offline time of the neural network model and dynamic feature data determined within runtime of the neural network model from information about the NPU and information about the neural network model, generating a prediction model for predicting operating frequency of the NPU for executing the neural network model based on the static feature data and the dynamic feature data, and controlling the operating frequency of the NPU based on the prediction model.