Neural Network Hardware Co-Design via Event-Based Simulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing complexity of deep neural networks (DNNs) on embedded devices makes it difficult to manually optimize hardware performance, as existing methods require time-consuming training and extensive manual modeling for accurate predictions, especially during early phases of DNN system design, and struggle with balancing limited on-chip resources with network requirements.
Innovation Solution
An end-to-end simulation-based methodology for performance predictions of DNN workloads on ML accelerators, which automates the creation of simulation environments and allows for joint optimization of neural network parameters and accelerator architecture, using event-based simulation frameworks like SimPy and machine learning compilers like TVM to generate Petri net graphs for accurate hardware-software co-design.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If ML-based approaches are used to train statistical models on real hardware measurements, then prediction accuracy for complex architectures is improved, but training time and data requirements increase significantly
Solution Approach 1:
The patent creates simulation environments in advance that can generate training data without requiring actual hardware measurements. The simulation framework pre-computes performance metrics for various neural network configurations on virtual hardware models, enabling the statistical model to be trained beforehand on diverse architectures and workloads, thus avoiding time-consuming real hardware training during early design phases
Solution Approach 2:
The patent uses virtual hardware models that copy the essential behavior and characteristics of physical hardware accelerators. These simulation environments replicate memory hierarchies, data buses, processing elements, and timing constraints of actual hardware, allowing statistical models to be trained on realistic hardware behavior data generated by the virtual models rather than requiring direct access to physical devices
2Speed
If analytical models are used to provide rapid predictions, then prediction speed is improved, but accuracy for complex architectures deteriorates
Solution Approach 1:
The patent pre-trains statistical models using simulation environments that capture complex hardware behavior before actual predictions are needed. By performing the heavy computational work of generating training data and fitting models in advance on virtual hardware, the system enables fast predictions during actual design iterations without sacrificing accuracy for complex architectures
Solution Approach 2:
The patent introduces statistical models as an intermediary between simulation environments and final predictions. The statistical models learn from simulation data and serve as a mediator that can rapidly predict performance metrics for new neural network configurations without requiring real-time simulation execution, thus achieving both speed and accuracy
3Ease of operation
If manual optimization of hardware performance is performed, then customization and control are improved, but development time and complexity increase
Solution Approach 1:
The patent implements automated tools that enable the system to optimize itself by automatically evaluating neural network configurations on virtual hardware models. The framework autonomously performs design space exploration, generates performance predictions using statistical models, and identifies optimal hardware configurations without requiring manual intervention, thus reducing development time while maintaining customization capabilities
Solution Approach 2:
The patent creates closed-loop feedback mechanisms where performance predictions from simulation environments guide hardware optimization decisions. The system continuously monitors performance metrics from virtual hardware models, compares them against target specifications, and automatically adjusts neural network configurations or hardware parameters, providing ongoing feedback that accelerates the optimization process
4Measurement precision
If simulation environments are created manually, then modeling accuracy is improved, but setup effort and complexity increase
Solution Approach 1:
The patent implements automated framework that generates simulation environments by automatically translating hardware architecture descriptions into executable simulation models. The system self-configures simulation parameters, creates virtual hardware models, and sets up performance evaluation metrics without manual intervention, maintaining accuracy while significantly reducing setup complexity and effort
Data Source
AI summary
A method for ascertaining a performance of a machine learning system on a processing unit. The method includes: creating a hardware model of the processing unit from a provided technical specification of the processing unit and creating a simulation graph based on the machine learning system; simulating an implementation of the machine learning system on the processing unit using the hardware model and the graph, the simulation being an event-based simulation, and ascertaining the performance based on the result of the simulation.