CNN Accelerator Hardware with Image-to-Column and GEMM Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Designing hardware accelerators for convolutional neural networks (CNNs) that can efficiently operate on edge devices is challenging due to limited memory and compute resources, and the complexity of supporting various CNN layers and sparse inputs.
Innovation Solution
A hardware accelerator is developed, incorporating an image-to-column block and a General Matrix-Matrix Multiplication (GEMM) block with a systolic array, capable of handling sparsity through metadata and selective application of weights and results, and featuring dynamic reconfigurability to adapt to different CNN layer attributes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hardware acceleration is implemented for CNN operations on edge devices, then processing speed and energy efficiency are improved, but device complexity and memory requirements increase
Solution Approach 1:
The hardware accelerator is divided into distinct functional blocks: image-to-column conversion unit, GEMM block with systolic array, and sparsity handling units. Each block performs a specific function, allowing the system to achieve high processing speed while managing complexity through modular design. The segmentation enables parallel processing of different CNN operations simultaneously.
Solution Approach 2:
The hardware accelerator incorporates dynamic reconfiguration capabilities through mode selectors that can switch between different operational modes (e.g., dense mode, sparsity mode, different GEMM configurations). This dynamic adaptability allows the system to optimize performance for different CNN layer types and sparsity patterns while efficiently utilizing available hardware resources on edge devices.
2Adaptability or versatility
If the hardware accelerator supports various CNN layer types with unique features, then versatility is improved, but device complexity increases
Solution Approach 1:
The hardware accelerator implements a universal architecture that can handle multiple CNN layer types (convolution, fully connected, pooling) through a common GEMM block and image-to-column conversion unit. By transforming different layer operations into unified matrix multiplication forms, the system achieves versatility without proportionally increasing complexity. The same core hardware blocks serve multiple functions depending on configuration.
3Productivity
If sparsity in feature map inputs and filter weights is handled through metadata and selective application, then processing efficiency is improved, but device complexity and memory usage increase
Solution Approach 1:
The hardware extractor identifies and extracts only the non-zero elements from sparse feature maps and filter weights, storing them in compact representations. Metadata structures track the positions and values of these non-zero elements. By extracting only the essential information needed for computation, the system achieves high processing efficiency while minimizing memory usage compared to storing complete dense matrices.
4Use of energy by moving object
If data transfer is reduced through local buffer exploitation, then energy efficiency is improved, but device complexity increases
Solution Approach 1:
The hardware accelerator pre-loads and stores frequently accessed data elements (such as filter weights and feature map regions) into local buffers before they are needed for computation. The image-to-column conversion unit prepares data in advance in the correct format for the GEMM block. This preliminary action reduces the need for repeated data transfers from main memory during computation, significantly improving energy efficiency while the added complexity is managed through systematic buffer management.
Data Source
AI summary
A hardware accelerator for neural network applications can include an image-to-column block and a general matrix-matrix multiplication (GEMM) block. The image-to-column block includes an input controller coupled to receive an input feature map from a memory block; a series of patch units configured in a ring network and coupled to the input controller to receive new elements of the input feature map; and an output controller coupled to receive each output patch from the series of patch units. The GEMM block can be a dynamically reconfigurable unit that can be configured as a tall array or individual square arrays. The described hardware accelerator can handle sparsity in both the feature map inputs (output from the image-to-column block) and the filter/weight inputs to the GEMM block.