Configurable Neural Network Accelerator for Flexible Layer Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
General-purpose compute engines, especially GPUs, are limited in processing power and cannot keep up with the increasing computational demands of neural network (NN) processing, and prior NN accelerators are inflexible when implementing different types and shapes of NN layers, leading to inefficiencies in memory access and computation.
Innovation Solution
A configurable NN accelerator architecture that employs a two-dimensional array of registers for data reuse and flexible implementation of various NN layer types and shapes, allowing for efficient computation and reduced storage requirements through dynamic reconfiguration of compute pipes and data processing circuits.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general-purpose compute engines (GPUs) are used for NN processing, then flexibility in implementation is maintained, but processing power and computational speed are insufficient
Solution Approach 1:
The system segments NN processing into distinct layers (convolutional layers, fully-connected layers, etc.) and implements dedicated hardware circuits for each layer type. This segmentation allows each circuit to be optimized for its specific function while maintaining overall system flexibility through configurable parameters that adapt to different NN architectures.
Solution Approach 2:
The accelerator employs dynamic reconfiguration capabilities where compute blocks can be programmatically configured to handle different layer types and shapes. The system dynamically adjusts operational parameters such as input/output feature map dimensions, number of filters, and stride values to match the specific NN layer being processed, enabling both high speed and adaptability.
2Productivity
If dedicated NN accelerators are used, then processing speed is improved, but flexibility in implementing different layer types and shapes is reduced
Solution Approach 1:
The accelerator design implements universal compute blocks that can perform multiple functions depending on configuration. The same hardware circuit can process convolutional layers, fully-connected layers, or other NN layer types by programmatically adjusting operational parameters. This multi-functionality maintains high processing speed while enabling flexibility across different NN architectures.
Solution Approach 2:
The system achieves flexibility through dynamic parameter changes rather than structural reconfiguration. Key parameters such as input feature map size, output feature map size, number of filters, kernel size, and stride values can be programmatically adjusted to match different layer requirements. This allows the same hardware to adapt to various layer types and shapes while maintaining optimized processing performance.
3Device complexity
If rigid accelerator architecture is used, then hardware complexity is reduced, but memory access efficiency deteriorates due to inability to optimize for different layer shapes
Solution Approach 1:
The system performs preliminary configuration of operational parameters before processing each layer. By pre-configuring the compute blocks with the specific dimensions, shapes, and parameters of the upcoming layer, the system optimizes memory access patterns in advance. This preliminary action enables efficient memory utilization without requiring complex dynamic reconfiguration hardware.
Data Source
AI summary
A scalable neural network accelerator may include a first circuit for selecting a sub array of an array of registers, wherein the sub array comprises LH rows of registers and LW columns of registers, and wherein LH and RH are integers. The accelerator may also include a register for storing a value that determines LH. In addition, the accelerator may include a first load circuit for loading data received from the memory bus into registers of the sub array.