Dynamic Precision Compute Engines for Neural Network Speed
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning algorithms, particularly neural networks, face challenges in efficiently utilizing parallel processing architectures like GPGPUs due to inefficiencies in thread execution and precision management, leading to suboptimal performance in training and deployment tasks.
Innovation Solution
Implement a hybrid precision computing system within GPGPUs, combining high and low precision components with dynamic precision management, and utilize FPGA acceleration for specific operations, along with a unified memory architecture for efficient data access and coherence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high precision components are used for all neural network operations, then computation accuracy is improved, but energy consumption and processing speed decrease
Solution Approach 1:
The system dynamically switches between high precision and low precision components based on the specific neural network operation being performed. Different precision levels are assigned to different computational tasks (e.g., high precision for weight updates, low precision for activation functions) to optimize both accuracy and speed, rather than using a fixed precision level for all operations
Solution Approach 2:
Different precision levels are applied to different parts of the neural network computation pipeline. High precision is used where it matters most (e.g., gradient calculations, weight updates) while low precision is used for operations where it has minimal impact (e.g., activation functions, normalization), achieving local optimization of the computational process
2Measurement precision
If high precision components are used for all neural network operations, then computation accuracy is improved, but energy consumption increases
Solution Approach 1:
The system changes the precision parameter dynamically based on the computational task. By adjusting the precision level (e.g., switching between 32-bit and 16-bit floating point) for different operations, the system reduces energy consumption for operations that don't require high precision while maintaining accuracy where needed
Solution Approach 2:
The precision level is made dynamic rather than static, allowing the system to adapt to different computational requirements in real-time. This dynamic adjustment enables energy-efficient processing by using lower precision when high accuracy is not critical
3Productivity
If dynamic precision switching is implemented, then energy efficiency and speed are improved, but device complexity increases
Solution Approach 1:
The system merges high precision and low precision compute engines into a single unified architecture that can dynamically switch between them. This integration reduces the overall complexity compared to having separate systems, while still providing the benefits of dynamic precision switching for optimized performance
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
In an example, an apparatus comprises a compute engine comprising a high precision component and a low precision component; and logic, at least partially including hardware logic, to receive instructions in the compute engine; select at least one of the high precision component or the low precision component to execute the instructions; and apply a gate to at least one of the high precision component or the low precision component to execute the instructions. Other embodiments are also disclosed and claimed.