Quantization-Aware Neural Architecture Search for Accuracy-Energy Trade-offs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The use of quantized-precision floating-point formats in artificial neural networks (ANNs) can lead to a loss in accuracy and requires optimization to balance performance, energy usage, and storage requirements.
Innovation Solution
Quantization-aware neural architecture search (QNAS) is employed to learn an optimized architecture for ANNs using neural architecture search techniques, predicting hyperparameters and quantization parameters to configure the ANN structure and operation, including bit width for mantissa representation, to improve accuracy and inference time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If quantized-precision floating-point formats are used in ANNs, then performance and energy usage are improved, but accuracy is lost
Solution Approach 1:
The patent applies parameter changes by systematically varying quantization parameters (bit width for mantissa and exponent, block size) to find the optimal configuration that balances energy efficiency and accuracy. The neural architecture search process explores different parameter combinations to identify the best trade-off point.
Solution Approach 2:
The patent implements dynamics by making the quantization configuration adaptive rather than static. The system dynamically selects different quantization parameters for different layers and operations based on their specific requirements, allowing each part of the network to operate at its optimal precision level.
2Quantity of substance
If quantized-precision floating-point formats are used in ANNs, then storage requirements are reduced, but accuracy is lost
Solution Approach 1:
The patent uses parameter changes to optimize the balance between storage efficiency and accuracy by adjusting quantization parameters. The system explores different bit width configurations to find the minimum storage requirement that maintains acceptable accuracy levels.
Solution Approach 2:
The patent applies local quality by allowing different parts of the neural network to use different quantization precisions. Critical layers maintain higher precision while less sensitive layers use lower precision, optimizing the overall accuracy-storage trade-off.
3Productivity
If quantized-precision floating-point formats are used in ANNs, then inference time is reduced, but accuracy is lost
Solution Approach 1:
The patent applies parameter changes by optimizing quantization parameters specifically for inference performance. The neural architecture search process identifies parameter configurations that maximize inference speed while maintaining acceptable accuracy thresholds.
4Measurement precision
If neural architecture search is used to optimize quantized ANN, then accuracy and inference time are improved, but device complexity increases
Solution Approach 1:
The patent applies self-service by using the neural architecture search process to automatically discover and configure optimal quantization parameters without manual intervention. The system self-optimizes its own architecture based on performance metrics, reducing the need for expert tuning while achieving high accuracy.
Data Source
AI summary
Quantization-aware neural architecture search (“QNAS”) can be utilized to learn optimal hyperparameters for configuring an artificial neural network (“ANN”) that quantizes activation values and/or weights. The hyperparameters can include model topology parameters, quantization parameters, and hardware architecture parameters. Model topology parameters specify the structure and connectivity of an ANN. Quantization parameters can define a quantization configuration for an ANN such as, for example, a bit width for a mantissa for storing activation values or weights generated by the layers of an ANN. The activation values and weights can be represented using a quantized-precision floating-point format, such as a block floating-point format (“BFP”) having a mantissa that has fewer bits than a mantissa in a normal-precision floating-point representation and a shared exponent.