Neural Network Accelerator with Programmable Exponent Bias
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning technologies face challenges in achieving improvements in accuracy, performance, and energy efficiency, particularly in neural network training and inference processes.
Innovation Solution
The implementation of a deep learning accelerator system that utilizes an array of processing elements with compute and routing elements, enabling flow-based computations on wavelets, and incorporating features like programmable exponent bias for floating-point operations, stochastic rounding, data structure descriptors, fabric vectors, neuron smearing, and microthreading to enhance processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional deep learning processing is used, then implementation is simpler, but accuracy and energy efficiency are insufficient
Solution Approach 1:
The patent applies parameter changes by introducing a programmable exponent bias mechanism that dynamically adjusts the exponent bias value for floating-point operations. This allows the system to optimize numerical precision and accuracy for different neural network workloads by changing the exponent bias parameter, thereby improving measurement precision without requiring fundamental changes to the processing architecture.
Solution Approach 2:
The patent segments the deep learning processing into distinct functional units including compute elements, routing elements, and specialized floating-point units with stochastic rounding capabilities. This segmentation allows each component to be optimized independently for its specific function, improving overall accuracy while managing complexity through modular design.
2Productivity
If conventional training methods are used, then process is simpler, but training time is longer
Solution Approach 1:
The patent implements continuous propagation through the neural network by maintaining persistent data structures and avoiding repeated memory accesses. The system keeps activation values and gradients in accessible memory states, enabling continuous computational flow without interruption, which significantly reduces training time while maintaining simple programming interfaces.
Solution Approach 2:
The patent performs preliminary actions by pre-computing and storing intermediate results in data structure descriptors and fabric vectors during the forward pass. These pre-computed values are readily available for the backward pass, eliminating the need to re-compute them and thereby reducing overall training time.
3Speed
If conventional inference processing is used, then system is simpler, but inference latency is higher
Solution Approach 1:
The patent replaces conventional mechanical memory access patterns with a dataflow-based computational model where data moves continuously through processing elements. This substitution eliminates memory bottlenecks and enables parallel processing of inference operations, significantly reducing inference latency while maintaining system simplicity through hardware-level optimizations.
4Use of energy by moving object
If standard floating-point operations are used, then compatibility is better, but energy efficiency is lower
Solution Approach 1:
The patent applies local quality by implementing stochastic rounding in specific floating-point units where it is most beneficial for energy efficiency, while maintaining standard floating-point operations in other parts of the system. This localized application of custom rounding behavior optimizes energy consumption in critical paths while preserving compatibility and versatility in other operations.
Solution Approach 2:
The patent changes the rounding parameter from deterministic to stochastic, which fundamentally alters the energy characteristics of floating-point operations. This parameter change enables more efficient hardware implementation by reducing the need for complex correction logic, thereby improving energy efficiency while maintaining IEEE 754 compatibility through controlled stochastic behavior.
5Measurement precision
If detailed data tracking is implemented, then accuracy is improved, but computational overhead increases
Solution Approach 1:
The patent merges the data structure descriptors with the computational data flow, combining metadata about data characteristics directly with the data being processed. This merging eliminates the need for separate tracking structures, reducing computational overhead while maintaining detailed information about data precision, dimensions, and transformation requirements throughout the neural network computation.
Data Source
AI summary
Techniques in advanced deep learning provide improvements in one or more of accuracy, performance, and energy efficiency. An array of processing elements comprising a portion of a neural network accelerator performs flow-based computations on wavelets of data. Each processing element has a respective compute element and a respective routing element. Each compute element has a respective floating-point unit enabled to optionally and/or selectively perform floating-point operations in accordance with a programmable exponent bias and/or various floating-point computation variations. In some circumstances, the programmable exponent bias and/or the floating-point computation variations enable neural network processing with improved accuracy, decreased training time, decreased inference latency, and/or increased energy efficiency.