Chiplet Matrix Engines for Adaptive Precision in Neural Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network training methods face challenges in balancing accuracy and storage/compute costs, particularly when using lower precision weights and activations, which can impact memory bandwidth and compute efficiency.
Innovation Solution
Implement dynamic datatype adjustment during neural network training to optimize precision based on computational requirements, leveraging techniques such as pipelining and SIMT architectures to enhance parallel processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If lower precision weights and activations are used in neural network training, then memory bandwidth requirements and compute efficiency are improved, but accuracy deteriorates
Solution Approach 1:
The patent applies dynamic datatype adjustment during neural network training, where the precision of weights and activations is adaptively changed based on computational requirements and convergence criteria. The system transitions from fixed-precision approaches to dynamic precision adjustment, allowing the network to use lower precision when appropriate while maintaining accuracy when needed, thus resolving the contradiction between compute efficiency and accuracy.
Solution Approach 2:
The patent changes the precision parameter of data types dynamically during training. By adjusting the precision parameter based on monitoring convergence criteria and computational needs, the system can switch between different precision levels (e.g., FP32, FP16, INT8) to optimize the trade-off between compute efficiency and accuracy for different layers and training stages.
2Quantity of substance
If lower precision weights and activations are used in neural network training, then memory bandwidth requirements are reduced, but accuracy deteriorates
Solution Approach 1:
The system dynamically adjusts data type precision during training based on actual memory bandwidth requirements and accuracy monitoring. By implementing conditional precision adjustment, the network uses lower precision (reducing memory bandwidth) only when convergence criteria are met, while maintaining higher precision when accuracy requirements demand it, thus resolving the contradiction between memory bandwidth and accuracy.
3Productivity
If dynamic datatype adjustment is implemented, then compute efficiency and memory bandwidth are improved, but device complexity increases
Solution Approach 1:
The patent implements self-service mechanisms where the neural network training system automatically monitors its own convergence criteria and computational needs, then autonomously adjusts data type precision without external intervention. The system uses built-in monitoring of training metrics and automatic precision selection, eliminating the need for complex external control systems while achieving dynamic optimization.
4Productivity
If dynamic datatype adjustment is implemented, then performance is optimized, but complexity of operation increases
Solution Approach 1:
The system performs self-service by automatically monitoring training convergence and computational requirements, then autonomously adjusting precision settings without requiring manual configuration or complex operational intervention. The automatic precision adjustment based on monitored convergence criteria simplifies operation while achieving performance optimization.
Data Source
Figure 1
Figure 2A
Figure 2B~2C
AI summary
One embodiment provides a graphics processor comprising a base die including a plurality of chiplet sockets and a plurality of chiplets coupled with the plurality of chiplet sockets. At least one of the plurality of chiplets including first circuitry configured to perform a multi-dimensional matrix multiply accumulate operation to facilitate training for a neural network and second circuitry configured to perform dynamic datatype adjustment during the training for the neural network. The dynamic datatype adjustment is performed based on statistics generated based on output of the multi-dimensional matrix multiply accumulate operation, neural network model metadata, and training metadata associated with training operations to be performed for the neural network.