Neural Network Accelerator Tile Exploiting Input Sparsity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional Neural Networks (CNNs) face inefficiencies due to the dominance of ineffectual weights and activations in convolutional layers, leading to increased execution time and resource consumption, as existing hardware accelerators do not effectively exploit sparsity in neural networks.
Innovation Solution
A neural network accelerator tile is designed to exploit input sparsity by using weight lanes and activation lanes, with a weight memory, activation selection unit, multiplexers, and combination units to statically promote effectual weights and selectively process only effectual activation bits, reducing unnecessary computations and resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional hardware accelerators process all weights and activations in convolutional layers, then computation completeness is maintained, but execution time and energy consumption increase due to processing ineffectual elements
Solution Approach 1:
The patent extracts and removes ineffectual weights and activations from the computation pipeline. By identifying zero or near-zero elements and excluding them from processing, the system reduces the computational workload while maintaining accurate inference results, thereby decreasing energy consumption and execution time without sacrificing productivity
Solution Approach 2:
The system changes the parameter of data representation by using sparse data structures and dynamic computation graphs. Instead of processing dense matrices, the system reconfigures the computation to only process non-zero elements, transforming the computational paradigm to achieve both speed and energy efficiency
2Productivity
If all activation values are processed through the computation pipeline, then computational completeness is maintained, but resource consumption increases due to processing unnecessary activation bits
Solution Approach 1:
The patent extracts and removes ineffectual activation values (zeros or near-zero elements) from the computation pipeline. By identifying and excluding these unnecessary elements, the system reduces the quantity of data processed while maintaining complete and accurate inference results, thereby decreasing resource consumption without sacrificing throughput
Solution Approach 2:
Instead of processing all activation values completely, the system applies partial action by only processing the necessary non-zero elements. This selective processing approach reduces resource consumption while maintaining sufficient computational accuracy for reliable inference
3Productivity
If weight lanes and activation lanes are implemented with multiplexers and combination units, then sparsity exploitation is achieved, but device complexity increases
Solution Approach 1:
The patent segments the computation pipeline into distinct functional units: weight lanes, activation lanes, multiplexers for selection, and combination units for computation. This segmentation allows each component to handle specific tasks efficiently, enabling sparsity exploitation while keeping the overall architecture modular and manageable despite the increased complexity
Solution Approach 2:
The system introduces dynamic elements including programmable multiplexers that can be configured based on sparsity patterns and adaptive computation graphs. These dynamic components allow the hardware to reconfigure itself for different computational tasks, achieving high computational efficiency while managing complexity through flexibility rather than fixed complex paths
Data Source
AI summary
Described is a neural network accelerator tile for exploiting input sparsity. The tile includes a weight memory to supply each weight lane with a weight and a weight selection metadata, an activation selection unit to receive a set of input activation values and rearrange the set of input activation values to supply each activation lane with a set of rearranged activation values, a set of multiplexers including at least one multiplexer per pair of activation and weight lanes, where each multiplexer is configured to select a combination activation value for the activation lane from the activation lane set of rearranged activation values based on the weight lane weight selection metadata, and a set of combination units including at least one combination unit per multiplexer, where each combination unit is configured to combine the activation lane combination value with the weight lane weight to output a weight lane product.