AI Accelerator Layer-Specific Dataflow Pipelining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI accelerators are limited by a single fixed dataflow that does not optimize performance for diverse neural network layer types, such as convolutional and fully connected layers, leading to suboptimal efficiency and throughput.
Innovation Solution
Implementing a pipelined architecture with separate convolutional and fully connected cores, each optimized for weight stationary and input stationary dataflows, respectively, to improve latency and throughput by reducing interconnect overhead and enhancing PE utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single fixed dataflow architecture is used, then device complexity is reduced, but productivity and adaptability to different neural network layer types deteriorate
Solution Approach 1:
The processing system is segmented into separate convolutional cores and fully connected cores, each optimized for specific neural network layer types. This segmentation allows each core to use the optimal dataflow architecture for its specific function, improving overall productivity while maintaining manageable complexity through modular design
Solution Approach 2:
The system dynamically switches between different dataflow architectures (weight stationary for convolutional layers, input stationary for fully connected layers) based on the layer type being processed. This dynamic adaptation enables the system to optimize productivity for different workloads without requiring a completely different hardware architecture for each case
2Ease of manufacture
If a single fixed dataflow architecture is used, then ease of manufacture is improved, but adaptability to diverse neural network layer types deteriorates
Solution Approach 1:
The accelerator design incorporates multiple specialized cores (convolutional and fully connected) within a single device, enabling it to handle diverse neural network layer types. This multi-functionality approach maintains ease of manufacture by integrating multiple functions into one device while achieving high adaptability to different computational requirements
3Productivity
If separate pipelined cores are implemented, then productivity and latency performance are improved, but device complexity increases
Solution Approach 1:
The system divides the processing workload into separate pipelined convolutional and fully connected cores, allowing independent optimization of each core's dataflow architecture. This segmentation improves productivity by eliminating bottlenecks in specific layer types while managing complexity through clear functional separation
Solution Approach 2:
The pipelined architecture enables continuous processing by overlapping computation across different layers and time steps. Multiple operations can proceed simultaneously in different pipelines, maintaining continuous useful action and improving throughput without requiring complete sequential execution
Data Source
AI summary
Systems and methods for a pipelined heterogeneous dataflow for an artificial intelligence accelerator are disclosed. A pipelined processing core includes a first processing core configured to have a first type of dataflow and a second processing core configured to have a second type of dataflow. The first processing core includes a matrix array of PEs arranged in columns and rows, each of the PEs configured to perform a MAC operation based on an input and a weight. The second processing core is configured to receive an output from the first processing core. The second processing core includes a column of PEs configured to perform MAC operations.