Stochastic Rounding Neural Processor Circuit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks face performance issues due to high bandwidth consumption and power usage when relying solely on CPUs for operations, leading to accuracy and speed problems during training and inference, and potential biases like overfitting.
Innovation Solution
A neural processor circuit with a neural engine and post-processing circuit that performs stochastic rounding, using a random bit generator, adder circuit, and rounding circuit to intentionally introduce noise, enhancing model performance during training and inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If CPU is used to perform neural network operations, then ease of operation is improved, but bandwidth consumption and power usage increase
Solution Approach 1:
The patent replaces the general-purpose CPU (mechanical/electronic system) with a specialized neural processing unit that uses stochastic rounding operations. This substitution reduces power consumption by dedicating hardware specifically for neural network computations rather than using a general-purpose processor
Solution Approach 2:
The patent changes the computational approach by introducing stochastic rounding as a fundamental operation in the neural processing pipeline. This parameter change in the computational method enables more efficient hardware implementation that consumes less power while maintaining computational effectiveness
2Ease of operation
If CPU is used to perform neural network operations, then ease of operation is improved, but processing speed decreases
Solution Approach 1:
The patent replaces the general-purpose CPU with a specialized neural processing unit optimized for specific neural network operations. This substitution enables faster processing by using hardware architectures designed specifically for convolutional and stochastic rounding operations rather than general-purpose instruction execution
3Manufacturing precision
If standard rounding is used in neural processing, then manufacturing precision is maintained, but performance accuracy deteriorates due to bias
Solution Approach 1:
The patent converts the typically harmful effect of rounding errors and noise into a beneficial feature by deliberately introducing stochastic rounding. This intentional noise injection prevents overfitting and improves generalization performance, transforming what is usually considered an error source into a performance-enhancing mechanism
Solution Approach 2:
The patent changes the rounding parameter from deterministic (standard rounding) to probabilistic (stochastic rounding). This parameter change in the rounding operation introduces controlled randomness that improves model performance during training and inference while maintaining numerical precision
Data Source
AI summary
Embodiments relate to a neural processor circuit that includes a neural engine and a post-processing circuit. The neural engine performs a computational task related to a neural network to generate a processed value. The post-processing circuit includes a random bit generator, an adder circuit and a rounding circuit. The random bit generator generates a random string of bits. The adder circuit adds the random string of bits to a version of the processed value to generate an added value. The rounding circuit truncates the added value to generate an output value of the computational task. The random bit generator may include a linear-feedback shift register (LFSR) that generates random numbers based on a seed. The seed may be derived from a master seed that is specific to a task of the neural network.