Configurable Pooling Unit for Neural Network Accelerator Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network accelerators face inefficiencies in processing diverse neural network structures due to fixed pipeline configurations, which limit their ability to handle non-classical neural networks effectively, especially when dealing with operations like depth-wise convolutions and pooling operations that require flexible pipeline formations.
Innovation Solution
A neural network accelerator with a crossbar that dynamically forms pipelines from hardware processing units, including a convolution processing unit and a configurable pooling processing unit capable of performing depth-wise convolution and pooling operations, allowing for flexible ordering and combination of operations in each hardware pass.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a fixed pipeline configuration is used in neural network accelerators, then the device complexity is reduced and manufacturing is easier, but the adaptability to handle diverse neural network structures (especially non-classical networks with depth-wise convolutions and pooling operations) deteriorates
Solution Approach 1:
The patent implements dynamic pipeline configuration where the crossbar can be reconfigured at runtime to form different pipeline topologies. The pooling processing unit can be dynamically connected to either the convolution processing unit or the activation processing unit based on the operation type, enabling the system to adapt to diverse neural network structures including depth-wise convolutions and pooling operations without requiring multiple fixed dedicated pipelines
Solution Approach 2:
The pooling processing unit is designed with multi-functionality to handle both pooling operations and depth-wise convolution operations. By configuring the crossbar to connect the pooling processing unit with appropriate neighboring units based on operation type, a single processing unit serves multiple functions, reducing the need for separate dedicated hardware for each operation type while maintaining high adaptability
2Productivity
If multiple dedicated processing units are provided for different operations (convolution, pooling, activation), then the productivity for specific operations is improved, but the device complexity increases
Solution Approach 1:
The pooling processing unit is designed to perform multiple functions including both pooling operations and depth-wise convolution operations. This multi-functional design allows the system to maintain high processing throughput for diverse operations while reducing the total number of dedicated processing units required, thereby balancing productivity with device complexity
3Adaptability or versatility
If a flexible dynamic pipeline configuration is implemented, then the adaptability to diverse operations is improved, but the use of energy for crossbar switching and configuration increases
Solution Approach 1:
The system performs preliminary configuration of the crossbar based on operation type before executing the actual neural network operations. By pre-configuring the pipeline topology to match the incoming operation (e.g., connecting pooling unit to convolution unit for depth-wise convolution, or to activation unit for pooling), the system minimizes runtime reconfiguration overhead and energy consumption while maintaining high adaptability
Data Source
AI summary
A neural network accelerator includes a plurality of hardware processing units, each hardware processing unit comprising hardware to accelerate performing one or more neural network operations on data; and a crossbar coupled to each hardware processing unit of the plurality of hardware processing units and configured to selectively form, from a plurality of selectable pipelines, a pipeline from one or more of the hardware processing units of the plurality of hardware processing units to process input data to the neural network accelerator. The plurality of hardware processing units comprising (i) a convolution processing unit configured to accelerate performing convolution operations on data, and (ii) a configurable pooling processing unit configured to selectively perform an operation of a plurality of selectable operations on data, the plurality of selectable operations comprising a depth-wise convolution operation and one or more pooling operations.