Neural Network Compute Engines With Dynamic Precision Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning processing technologies face challenges in efficiently managing precision for neural network compute operations, particularly in balancing accuracy and computational resources.
Innovation Solution
The implementation of dynamic precision techniques within neural network compute operations, utilizing a combination of high-precision and low-precision hardware, such as fused compute engines and embedded FPGAs, to adapt precision based on workload requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-precision hardware is used for neural network compute operations, then measurement precision is improved, but use of energy increases
Solution Approach 1:
The system dynamically adjusts precision levels based on workload requirements and performance targets. The compute engine switches between high-precision and low-precision modes, and between different data format representations (e.g., floating-point, fixed-point, integer), to optimize the trade-off between computational accuracy and energy consumption for different neural network operations.
Solution Approach 2:
Different precision levels are applied to different portions of the neural network computation based on their specific requirements. Critical layers or operations that require high accuracy use high-precision hardware, while less sensitive operations use low-precision hardware, thereby reducing overall energy consumption while maintaining necessary measurement precision where required.
2Productivity
If dynamic precision techniques are implemented, then productivity is improved, but device complexity increases
Solution Approach 1:
The compute engine is designed with multi-functional capabilities to handle various data formats (floating-point, fixed-point, integer) and precision levels within a single hardware unit. This universal design allows the system to achieve dynamic precision adjustment without requiring separate specialized hardware for each precision level, thereby improving productivity while limiting the increase in device complexity.
Solution Approach 2:
The system changes operational parameters such as data format representation and precision level based on the specific computational requirements. By dynamically adjusting these parameters rather than hardware configuration, the system achieves high productivity while avoiding the complexity of multiple fixed-precision hardware units.
Data Source
AI summary
In an example, an apparatus comprises a compute engine comprising a high precision component and a low precision component; and logic, at least partially including hardware logic, to receive instructions in the compute engine; select at least one of the high precision component or the low precision component to execute the instructions; and apply a gate to at least one of the high precision component or the low precision component to execute the instructions. Other embodiments are also disclosed and claimed.