Mixed Precision Neural Network Accelerator Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models, particularly neural networks, are computationally expensive and energy-intensive, making real-time feature extraction and training inefficient, especially when executed on general-purpose processors.
Innovation Solution
A neural network accelerator is configured to use mixed precision data formats, converting input tensors from normal-precision floating-point to quantized-precision floating-point formats for operations and back to normal-precision for output, optimizing computational complexity and energy efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If normal-precision floating-point format is used for neural network operations, then computational accuracy is maintained, but computational complexity and energy consumption increase
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the precision format of floating-point numbers during neural network operations. The system converts between normal-precision and quantized-precision formats based on the specific computational context, allowing energy consumption to be reduced while maintaining sufficient accuracy for each operation type.
Solution Approach 2:
The system implements dynamic precision switching where the precision level is not fixed but adapts during computation. Different layers and operations within the neural network use different precision levels, with the system dynamically selecting appropriate precision formats to balance accuracy requirements against energy consumption constraints.
2Measurement precision
If normal-precision floating-point format is used for neural network operations, then computational accuracy is maintained, but processing speed decreases
Solution Approach 1:
The patent changes the precision parameter of floating-point representations based on computational needs. By using quantized-precision formats for operations where full precision is not critical, the system achieves faster processing speeds while maintaining adequate accuracy through selective use of normal-precision formats when needed.
Solution Approach 2:
The system dynamically adjusts precision levels during different phases of neural network computation. Training operations that require high accuracy use normal-precision formats, while inference operations can utilize quantized-precision formats for faster execution, creating a dynamic adaptation between speed and accuracy requirements.
3Use of energy by moving object
If quantized-precision floating-point format is used for all operations, then computational complexity and energy consumption are reduced, but computational accuracy deteriorates
Solution Approach 1:
The patent applies local quality by assigning different precision levels to different parts of the neural network computation. Critical operations such as gradient calculations and weight updates maintain normal-precision formatting, while less sensitive operations like activation functions and inference computations use quantized-precision formats, ensuring accuracy is preserved where needed while reducing energy consumption elsewhere.
Solution Approach 2:
The computational process is segmented into different precision zones. The system divides neural network operations into segments that require high precision and segments that can tolerate quantization, processing each segment at the appropriate precision level. This segmentation allows the system to reduce overall energy consumption without compromising the accuracy of critical computations.
Data Source
AI summary
Technology related to training a neural network accelerator using mixed precision data formats is disclosed. In one example of the disclosed technology, a neural network accelerator is configured to accelerate a given layer of a multi-layer neural network. An input tensor for the given layer can be converted from a normal-precision floating-point format to a quantized-precision floating-point format. A tensor operation can be performed using the converted input tensor. A result of the tensor operation can be converted from the block floating-point format to the normal-precision floating-point format. The converted result can be used to generate an output tensor of the layer of the neural network, where the output tensor is in normal-precision floating-point format.