Floating-Point Unit Stochastic Rounding for Deep Learning Bias
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning technologies face challenges in achieving improvements in accuracy, performance, and energy efficiency, particularly in the training and inference processes of neural networks.
Innovation Solution
The implementation of a deep learning accelerator that utilizes an array of processing elements with compute and routing elements, each equipped with a floating-point unit capable of stochastic rounding, performing flow-based computations on wavelets to reduce systematic bias in long dependency chains and enhance training and inference efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deterministic rounding is used in floating-point units, then computational speed is maintained, but systematic bias accumulates in long dependency chains reducing accuracy
Solution Approach 1:
The patent changes the rounding parameter from deterministic to stochastic, introducing randomness into the rounding process. This transforms the rounding behavior from a fixed rule-based system to a probabilistic system that reduces systematic bias accumulation while maintaining computational efficiency through hardware-level implementation.
2Measurement precision
If stochastic rounding is implemented in floating-point units, then systematic bias is reduced improving accuracy, but additional computational overhead is introduced
Solution Approach 1:
The patent extracts the random number generation function from the main computational path and implements it as a separate entropy source module. This allows stochastic rounding to be implemented by combining pre-generated random numbers with rounding decisions, reducing the computational overhead and hardware complexity compared to generating random numbers for every rounding operation.
Solution Approach 2:
The patent generates random numbers in advance and stores them in buffers before they are needed for rounding operations. This preliminary action eliminates the need for real-time random number generation during critical computational paths, reducing hardware complexity and maintaining computational speed.
3Measurement precision
If higher precision floating-point operations are used, then computational accuracy is improved, but energy consumption increases
Solution Approach 1:
The patent changes the precision parameter dynamically based on the computational context. During training phases where accuracy is critical, higher precision is used. During inference phases or in less critical computations, lower precision is employed. This dynamic parameter adjustment optimizes the trade-off between accuracy and energy consumption.
Data Source
AI summary
Techniques in advanced deep learning provide improvements in one or more of accuracy, performance, and energy efficiency. An array of processing elements comprising a portion of a neural network accelerator performs flow-based computations on wavelets of data. Each processing element has a respective compute element and a respective routing element. Each compute element has a respective floating-point unit enabled to perform stochastic rounding, thus in some circumstances enabling reducing systematic bias in long dependency chains of floating-point computations. The long dependency chains of floating-point computations are performed, e.g., to train a neural network or to perform inference with respect to a trained neural network.