Hardware Metric Predictor for Neural Network Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for predicting hardware metrics, such as energy consumption and latency, for neural networks on target hardware are impractical due to the need for extensive training data and costly measurement processes, especially when simulators are unavailable or inaccurate.
Innovation Solution
A neural network is trained using a modest set of measurement data to predict hardware metrics for novel hardware targets, employing a transformer architecture and a ground truth set to infer relationships between neural network architectures and their corresponding metrics, allowing for few-shot predictions without requiring extensive data from the target hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a machine learnable model is trained to predict hardware metrics, then prediction accuracy improves, but the amount of training data required increases substantially
Solution Approach 1:
The system performs preliminary actions by collecting hardware metric measurements from multiple source hardware platforms before training begins. These pre-collected measurements from diverse hardware sources serve as foundational training data, enabling the model to learn hardware-specific patterns without requiring extensive data collection from the target hardware platform itself.
Solution Approach 2:
The trained hardware metric predictor is designed to be universally applicable across multiple target hardware platforms. By training on data from various source hardware types, the single model learns transferable patterns that enable accurate predictions on different hardware architectures, reducing the need for platform-specific data collection.
2Measurement precision
If hardware metrics are measured on target hardware, then prediction accuracy improves, but time consumption and cost increase
Solution Approach 1:
Instead of measuring hardware metrics directly on the target hardware, the system creates a predictive model by copying and analyzing measurements from source hardware platforms. The trained model then serves as a virtual copy that can predict target hardware metrics without requiring physical access to or repeated testing on the actual target device, eliminating iterative compile-upload-test cycles.
Solution Approach 2:
Hardware metric measurements from source platforms are collected and processed in advance before the actual prediction is needed. This preliminary data collection and model training enables rapid predictions on target hardware without requiring time-consuming measurement cycles on the target device itself.
3Loss of time
If simulators are used to estimate hardware costs, then measurement time reduces, but accuracy deteriorates
Solution Approach 1:
The system introduces a trained machine learnable model as an intermediary between hardware architecture descriptions and actual hardware metric values. This model, trained on real measurements from source hardware, acts as a mediator that translates architectural parameters into accurate metric predictions without requiring direct simulation or physical measurement, achieving both speed and accuracy.
Data Source
Figure 1a~1b
Figure 2a~2b
Figure 2c~3
AI summary
Some embodiments are directed to training a hardware metric predictor. The hardware metric predictor being configured to receive as input a query description of a neural network architecture and to produce as output a predicted hardware metric predicted to be incurred by a neural network corresponding to the query description when run on the target hardware. Method may include giving as training input a number of input/output pairs of a given training function.