Accelerator-Aware Weight Training for Energy-Efficient Multiply-Accumulate
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning models consume excessive energy during multiplication and accumulation operations due to inefficient utilization of accelerators, leading to high computational costs.
Innovation Solution
Customize the training of weight matrices in deep learning models based on the energy consumption characteristics of specific accelerators, such as microring resonators, synapse memory cells, and memristors, by using loss functions or pruning techniques to optimize energy efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If deep learning models use traditional training methods without accelerator-specific optimization, then model training is simple and fast, but energy consumption during multiplication and accumulation operations is excessive
Solution Approach 1:
The patent applies parameter changes by modifying the training process to incorporate accelerator-specific characteristics. Different loss functions are used during training based on the accelerator type (e.g., quantization-aware training for neuromorphic accelerators, precision-aware training for FPGA). This transforms the training parameters and objectives to align with the energy consumption characteristics of specific accelerators, thereby reducing energy consumption during inference while maintaining model accuracy.
Solution Approach 2:
The patent implements preliminary action by optimizing the weight matrices during the training phase before deployment. By pre-training models with accelerator-specific loss functions and constraints, the weight matrices are shaped to be more compatible with the target accelerator's architecture and energy characteristics. This preliminary optimization ensures that when the model is deployed on the accelerator, energy consumption is minimized without requiring further modification during inference.
2Use of energy by moving object
If weight matrices are optimized for specific accelerator types, then energy efficiency improves, but training time and computational cost increase
Solution Approach 1:
The patent applies local quality by tailoring the training process to specific accelerator types rather than using a one-size-fits-all approach. Different loss functions and training constraints are applied locally to different accelerator architectures (e.g., quantization-aware training for neuromorphic, precision-aware training for FPGA). This localized optimization ensures that each accelerator type receives training tailored to its specific energy characteristics, improving energy efficiency without unnecessarily extending training time for all models.
Solution Approach 2:
The patent implements partial action by applying accelerator-specific optimization only when necessary. The system can detect the target accelerator type and apply specialized training only for those cases, while using standard training for generic or unknown accelerators. This partial application of optimization strategies reduces the overall training time penalty while still achieving significant energy efficiency improvements for the targeted accelerator types.
3Measurement precision
If conventional training methods are used, then training process is straightforward, but computational accuracy may degrade when deployed on specialized accelerators
Solution Approach 1:
The patent applies feedback by incorporating loss functions that provide guidance during training based on the target accelerator's characteristics. Quantization-aware training uses feedback from quantization errors, while precision-aware training uses feedback from computational precision requirements. This feedback mechanism ensures that the weight matrices are trained to produce accurate results when deployed on the specific accelerator type, maintaining computational accuracy while simplifying the training process through automated accelerator-type detection and configuration.
Data Source
AI summary
Customization of deep learning models for accelerators of multiplication and accumulation operations. Based on a type of an accelerator to be used to implement the computation of an artificial neural network, a weight matrix of an artificial neural network can be adjusted, during training or via re-training, based on energy consumption characteristics of the type of accelerators. Patterns of weights that can consume more energy in computations implemented via the accelerator can be suppressed via penalizing by a loss function during training, or via pruning and re-training. The adjusted weight matrix can be configured in a computing device having an accelerator of the type. When the computing device performs computations of the artificial neural network using the weight matrix, the accelerator can be used to accelerate multiplication and accumulation operations involving the weight matrix.