DNN Accelerator Architecture for Skipping Zero-Valued Neurons
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face significant performance and energy efficiency challenges due to a large fraction of ineffectual operations where one input of a multiplication is zero, which wastes time and resources without contributing to the final result.
Innovation Solution
An accelerator system that eliminates or skips over most ineffectual operations by using a Zero-Free Neuron Array Format (ZFNAf) to store and process only non-zero neurons, allowing independent processing of neuron lanes and maintaining wide, aligned memory accesses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional DNN computation is performed with all neurons including zero-valued neurons, then complete computational accuracy is maintained, but processing speed and energy efficiency deteriorate due to ineffectual operations
Solution Approach 1:
The patent extracts and removes zero-valued neurons from the computation stream, processing only non-zero neurons. This is achieved through a selective neuron processing mechanism that identifies and eliminates ineffectual operations where one input to multiplication is zero, thereby improving processing speed and energy efficiency without affecting the final computational result.
Solution Approach 2:
The patent introduces dynamic processing by allowing different processing paths based on neuron values. The system dynamically adjusts computation by skipping zero-valued neurons while maintaining processing for non-zero neurons, creating a flexible pipeline that adapts to the actual data distribution and reduces wasted computational cycles.
2Productivity
If zero-valued neurons are eliminated from processing, then energy efficiency and processing speed improve, but computational accuracy may deteriorate
Solution Approach 1:
The patent converts the harmful effect of zero-valued neurons (wasting computational resources) into a benefit by systematically identifying and eliminating them. The mechanism uses the zero value property itself as a signal to skip computation, transforming what was previously a source of inefficiency into a optimization opportunity that improves both speed and energy efficiency while preserving accuracy.
3Loss of energy
If selective processing of non-zero neurons is implemented, then energy efficiency improves by reducing ineffectual operations, but device complexity increases due to additional processing mechanisms
Solution Approach 1:
The patent segments the neuron processing pipeline into distinct functional units: a dispatcher that routes neurons, processing elements that execute computations, and memory structures that store neuron data. This segmentation allows each component to be optimized independently and simplifies the control logic for selective processing, managing complexity through modular organization rather than monolithic design.
Solution Approach 2:
The patent introduces a dispatcher as an intermediary component between memory and processing elements. This mediator manages the complex task of selective neuron distribution by reading neurons from memory, identifying non-zero values, and routing them to appropriate processing elements, thereby isolating the complexity of selective processing from the core computation units.
4Speed
If all neurons are processed in parallel, then processing speed is maximized, but energy consumption increases due to processing ineffectual zero-valued neurons
Solution Approach 1:
The patent applies partial action by processing only the necessary subset of neurons (non-zero values) rather than all neurons in parallel. This selective approach performs slightly less total computation (skipping zero-valued neurons) but achieves the same effective result with reduced energy consumption, as the skipped operations would have contributed nothing to the final output.
Data Source
AI summary
Described is a system, integrated circuit and method for reducing ineffectual computations in the processing of layers in a neural network. One or more tiles perform computations where each tile receives input neurons, offsets and synapses, and where each input neuron has an associated offset. Each tile generates output neurons, and there is also an activation memory for storing neurons in communication with the tiles via a dispatcher and an encoder. The dispatcher reads neurons from the activation memory and communicates the neurons to the tiles and reads synapses from a memory and communicates the synapses to the tiles. The encoder receives the output neurons from the tiles, encodes them and communicates the output neurons to the activation memory. The offsets are processed by the tiles in order to perform computations only on non-zero neurons. Optionally, synapses may be similarly processed to skip ineffectual operations.