Adaptive MAC Array Scheduling for CNN Tile Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face inefficiencies in processing capacity and memory bandwidth, particularly in operations requiring large numbers of multiply-accumulate (MAC) units, which are power-intensive and have high IC gate counts, limiting their performance in applications like autonomous driving.
Innovation Solution
The solution involves cutting input and output activation data into tiles in three dimensions and using a 3D convolution computation core with configurable MAC arrays and adaptive scheduling to perform various convolution operations efficiently, including multi-precision and dilated convolutions, thereby optimizing MAC array utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If large numbers of MAC units are used to increase processing capacity, then convolution operation performance is improved, but power consumption and IC gate count increase significantly
Solution Approach 1:
The patent divides the activation data into multiple tiles and processes them in parallel across multiple MAC arrays. This segmentation allows the system to achieve high processing capacity by distributing work across many smaller, more power-efficient units rather than relying on a few large MAC units, thereby improving productivity while controlling power consumption.
Solution Approach 2:
The patent introduces the dimension of data tiling and parallel processing streams, transforming the computational approach from sequential MAC operations to concurrent processing across multiple data tiles. This dimensional expansion in the data flow organization enables scalable productivity improvement without linearly increasing power consumption.
2Speed
If more MAC units are deployed to handle large convolution operations, then computation speed is improved, but device complexity and IC gate count increase
Solution Approach 1:
The patent designs a universal MAC array architecture that can be configured to handle different convolution scenarios through adaptive scheduling. The same MAC array infrastructure serves multiple functions by dynamically allocating and configuring arrays for different tile processing needs, achieving high computation speed without proportionally increasing device complexity.
Solution Approach 2:
The patent implements dynamic configuration of MAC arrays through adaptive scheduling that adjusts the number and arrangement of active MAC arrays based on the specific convolution operation requirements. This dynamic approach allows the system to optimize computation speed for different workloads without permanently increasing device complexity.
3Ease of operation
If activation data is processed as whole large arrays, then memory access is simplified, but memory bandwidth requirements and computation overhead increase
Solution Approach 1:
The patent segments large activation data arrays into smaller tiles that can be processed independently by different MAC arrays. This segmentation reduces the memory bandwidth required for each processing unit while maintaining overall system efficiency, as each tile can be loaded and processed with smaller, more manageable memory transactions.
Solution Approach 2:
The patent performs preliminary tiling and organization of activation data before convolution processing, preparing the data in a format optimized for parallel MAC array processing. This preliminary action enables more efficient memory bandwidth utilization during the actual convolution operation, as data is already arranged in tile-ready format.
Data Source
AI summary
The present invention relates to convolution neural networks (CNN) and methods for improving computational efficiency of multiply accumulate (MAC) array structure Specifically, the invention relates to cutting of activation data into a number of tiles for increasing overall computation efficiency. The invention discloses techniques to cut an activation data into a plurality of tiles by using a 3-D convolution computation core and support bigger tensor sizes. Lastly, the invention provides adaptive scheduling of MAC array to achieve high utilization in multi-precision neural network acceleration.