Neural Network Power Profile Estimation for AI Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current power modeling tools for AI accelerators are inadequate for accurately estimating power consumption in neural networks due to rapid evolution of AI hardware architectures and lack of fine-grained power profiling capabilities, leading to inefficiencies in design and management.
Innovation Solution
A system that estimates power profiles for neural networks on a per-layer and per-workload basis, utilizing a neural network compiler, performance simulator, and power simulator to model hardware architecture abstractly, providing fine-grained power optimization and quick turn-around times, while considering hardware efficiency and leveraging tools like Synopsys PrimePower for calibration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current power modeling tools are used, then average power estimation is possible, but fine-grained power profiling capability is lacking
Solution Approach 1:
The patent segments the neural network model into individual layers and operations, enabling power consumption to be measured and estimated at each layer level. This segmentation transforms the undifferentiated average power measurement into granular, layer-specific power profiles, directly resolving the contradiction between measurement precision and device complexity by breaking down the complex measurement task into manageable hierarchical units.
2Measurement precision
If detailed power profiling is implemented, then accurate power estimation is achieved, but turn-around time increases
Solution Approach 1:
The system performs preliminary actions by pre-compiling the neural network model into an intermediate representation that includes power-related metadata before actual power measurement begins. This preliminary compilation phase prepares the model structure, layer information, and operation details in advance, enabling rapid iterative power profiling without repeatedly parsing the entire model, thus achieving accurate power estimation with reduced turn-around time.
3Measurement precision
If power models are built for next generation AI accelerators, then accurate power modeling is achieved, but the process becomes challenging and time consuming
Solution Approach 1:
The patent creates an abstract copy of the AI accelerator architecture in the form of a performance simulator model that replicates the hardware structure, memory hierarchy, and computational units. This virtual copy allows power profiling to be performed on the simulated architecture before physical hardware is available, enabling accurate power modeling for next-generation devices without requiring actual hardware prototypes, thus reducing the challenge and time associated with building power models for evolving architectures.
Data Source
AI summary
Technology for estimating neural network (NN) power profiles includes obtaining a plurality of workloads for a compiled NN model, the plurality of workloads determined for a hardware execution device, determining a hardware efficiency factor for the compiled NN model, and generating, based on the hardware efficiency factor, a power profile for the compiled NN model on one or more of a per-layer basis or a per-workload basis. The hardware efficiency factor can be determined on based on a hardware efficiency measurement and a hardware utilization measurement, and can be determined on a per-workload basis. A configuration file can be provided for generating the power profile, and an output visualization of the power profile can be generated. Further, feedback information can be generated to perform one or more of selecting a hardware device, optimizing a breakdown of workloads, optimizing a scheduling of tasks, or confirming a hardware device design.