Dynamic Tile Neural Network Accelerator for MAC Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network accelerators have fixed computational engine sizes, leading to inefficient utilization and high data transfer latency, especially in workloads with many convolution layers, as they do not match the dynamic nature of computational problems.
Innovation Solution
A neural network accelerator with dynamic tiling capabilities, where processing engines can be re-configured into tiles of various sizes, and multi-functional computational cells with edge cells performing vector operations, allowing for direct data paths and reduced latency through a configuration controller that adjusts data flow and tile sizes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If fixed-size computational engines are used, then hardware structure is simple, but MAC utilization efficiency deteriorates
Solution Approach 1:
The patent implements dynamic tile sizing where computational engines can be reconfigured into tiles of various sizes based on workload requirements. The configuration controller dynamically adjusts the number and size of active tiles during runtime, transforming the static hardware architecture into a dynamic system that adapts to different computational demands, thereby resolving the contradiction between structural simplicity and utilization efficiency.
Solution Approach 2:
The patent designs multi-functional computational cells that can perform both 2D multiplication operations and vector operations. Edge cells specifically are configured to handle vector operations while regular computational cells handle 2D multiplication. This multi-functionality allows the same hardware resources to serve multiple purposes, improving MAC utilization without significantly increasing hardware complexity.
2Device complexity
If fixed-size computational engines are used, then device complexity is low, but data transfer latency increases
Solution Approach 1:
The dynamic tile configuration allows the system to optimize data transfer paths based on the current computational workload. By adjusting tile sizes dynamically, the configuration controller can minimize data movement between memory and computational units, reducing data transfer latency while maintaining manageable device complexity through a unified configuration mechanism.
3Power
If parallel fixed hardware is used, then computational power is high, but efficiency deteriorates due to unused engines
Solution Approach 1:
The patent employs dynamic tile sizing where the configuration controller activates only the necessary number and size of computational tiles based on the current workload requirements. This dynamic adjustment ensures that computational power is fully utilized without leaving engines idle, thereby resolving the contradiction between having high computational power through parallelism and maintaining processing efficiency by avoiding unused engines.
4Adaptability or versatility
If multi-functional cells are implemented, then versatility improves, but cell design complexity increases
Solution Approach 1:
The patent implements multi-functional computational cells where edge cells can perform both 2D multiplication and vector operations, while regular computational cells perform 2D multiplication. The configuration controller manages the different operational modes through unified control signals. This approach achieves versatility without excessive design complexity by using a regular cellular structure with controlled functional variations at the edges.
Data Source
AI summary
A dynamic-tile neural network accelerator allows for the number and size of computational tiles to be re-configured. Each sub-array of computational cells has edge cells on the left-most column that have an added vector mux that feeds the cell output back to an adder-comparator to allow Rectified Linear Unit (ReLU) and pooling operations that combine outputs shifted in from other cells. The edge cells drive external output registers and receive external weights. The weights and outputs are shifted in opposite directions horizontally between cells while control and input data are shifted in a same direction vertically between cells. A column of row data selectors is inserted between sub-arrays to bypass weights and output data around sub-arrays, while a row of column data selectors are inserted between sub-arrays to bypass control and input data. Larger tiles are configured by passing data directly through these selectors without bypassing.