Neural Network Accelerator Dual-Mode Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network accelerators are inefficient as they treat all layers uniformly, leading to slower processing times and unnecessary power usage due to their inability to effectively handle both dense and sparse layers within neural networks.
Innovation Solution
The development of a processor accelerator that operates in two modes: one for dense layers and another for sparse layers, where the mode is determined by a software element, allowing for immediate multiplication of row and column data sets in dense layers and compression of data sets to eliminate zeros in sparse layers, thereby optimizing processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a uniform processing mode is used for all neural network layers, then the accelerator can maintain consistent hardware architecture, but processing efficiency decreases for both dense and sparse layers
Solution Approach 1:
The accelerator dynamically switches between two processing modes (dense mode and sparse mode) based on the input data characteristics. The system includes a mode selection mechanism that determines whether to apply dense layer processing or sparse layer processing, allowing the hardware to adapt its behavior to match the computational requirements of different layer types, thereby resolving the contradiction between architectural consistency and processing efficiency
Solution Approach 2:
The patent implements different processing strategies for different parts of the neural network based on layer characteristics. Dense layers receive standard multiplication operations while sparse layers receive optimized operations that skip zero elements. This localized adaptation of processing quality to match local data characteristics improves overall efficiency without requiring complete architectural redesign
2Speed
If the accelerator optimizes for dense layers, then processing speed for dense workloads is improved, but performance degrades for sparse layers
Solution Approach 1:
The accelerator is designed with multi-functionality to handle both dense and sparse layer processing. The system includes a mode selection mechanism that routes dense layers to standard processing units while directing sparse layers to optimized processing paths. This universal design allows a single accelerator to efficiently process various layer types without sacrificing performance in either category
3Speed
If the accelerator optimizes for sparse layers, then processing speed for sparse workloads is improved, but performance degrades for dense layers
Solution Approach 1:
The accelerator incorporates dedicated processing paths for both sparse and dense operations. When sparse layer processing is detected, the system activates optimized routines that exploit sparsity patterns. When dense layers are encountered, standard efficient routines are used. This multi-functional capability ensures high performance across different workload types
4Device complexity
If uniform compression and arrangement techniques are applied to all layers, then hardware resource utilization is improved, but processing time increases due to inefficiencies in manipulating different layer types
Solution Approach 1:
The system dynamically adjusts its processing strategy based on the detected layer type. For dense layers, standard multiplication and accumulation operations are performed without compression. For sparse layers, the system activates compression techniques that eliminate zero elements before processing. This dynamic adaptation prevents time loss from applying inappropriate manipulation techniques while maintaining efficient hardware utilization
Data Source
AI summary
An accelerator for increasing the processing speed of a processor. The accelerator operates in two distinct modes. In a first mode for dense layer processing, row data sets and column data sets are sent to a multiplier for multiplication. In a second mode for sparse layer processing compressed row data sets are received by a row multiplexer and compressed column data sets are received by a column multiplexer. Each multiplexer is configured to compare the indexes of data sets with one another to determine matching indexes. When indexes match, the matching data sets are selected and sent to the multiplier for multiplication. When indexes do not match, data sets are stored in memory devices for subsequent cycles.