Group Convolution Hardware Accelerator Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware architectures for neural networks face inefficiencies in executing group convolutions, particularly in terms of compute efficiency and hardware utilization, compared to depthwise and full convolutions.
Innovation Solution
The implementation of group convolutions on a hardware neural network accelerator involves partitioning the input feature map along the channel dimension, mapping these partitions to multiply accumulate cells, and using a broadcast input bus to optimize computations, thereby improving compute efficiency and hardware utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If group convolutions are implemented on existing hardware architectures, then the neural network can process a wider range of computer vision tasks, but compute efficiency and hardware utilization are insufficient compared to depthwise and full convolutions
Solution Approach 1:
The input feature map is partitioned into multiple partitions along the channel dimension, and each partition is mapped to a separate subset of MACs. This segmentation enables parallel processing of different partitions while maintaining the group convolution structure, thereby improving compute efficiency without sacrificing the versatility of handling various computer vision tasks
Solution Approach 2:
The patent introduces a new dimension of parallelism by partitioning along the channel dimension and mapping to multiple MAC subsets. This dimensional transformation allows group convolutions to utilize hardware resources more effectively, achieving compute efficiency comparable to depthwise and full convolutions while maintaining adaptability
2Adaptability or versatility
If group convolutions are implemented on existing hardware architectures, then the neural network can process a wider range of computer vision tasks, but hardware utilization is insufficient compared to depthwise and full convolutions
Solution Approach 1:
The hardware architecture is designed to handle multiple types of convolutions (full, depthwise, and group) using the same MAC array and partitioning mechanism. By making the hardware universal, it can efficiently process group convolutions without requiring dedicated specialized units, thereby improving hardware utilization while maintaining the ability to handle various computer vision tasks
Solution Approach 2:
The patent employs dynamic partitioning and mapping strategies that can adapt to different convolution types and hardware configurations. This dynamic approach allows the same hardware to be optimally utilized for different tasks, improving overall hardware utilization while maintaining versatility
3Productivity
If partitions are mapped to MACs in parallel, then compute efficiency is improved, but latency and off-chip communication may increase
Solution Approach 1:
The patent performs preliminary actions by pre-fetching and caching weights and input data into on-chip memory before the actual computation. This preparation reduces the need for off-chip communication during parallel processing, thereby improving compute efficiency without significantly increasing latency
Solution Approach 2:
The parallel processing of partitions is designed to maintain continuous useful action by overlapping data transfer and computation operations. While some partitions are being processed, other partitions are having their data transferred and prepared, ensuring that the hardware remains continuously utilized without idle time that would increase latency
Data Source
AI summary
Methods, systems, and apparatus, including computer-readable media, are described for processing an input image using integrated circuit that implements a convolutional neural network with a group convolution layer. The processing includes determining a mapping of partitions along a channel dimension of an input feature map to multiply accumulate cells (MACs) in a computational unit of the circuit and applying a group convolution to the input feature map. Applying the group convolution includes, for each partition: providing weights for the group convolution layer to a subset of MACs based on the mapping; providing, via an input bus of the circuit, an input of the feature map to each MAC in the subset; and computing, at each MAC in the subset, a product using the input and a weight for the group convolution layer. An output feature map is generated for the group convolution layer based on an accumulation of products.