Data Format Converter for Systolic Array MAC Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systolic arrays in AI engines experience low MAC utilization in the first couple of convolution layers due to the mismatch between the number of image data channels and processing elements, leading to inefficient computation in image processing tasks.
Innovation Solution
A data format converter rearranges the input image data from three channels (RGB) to a greater number of channels, closer to the number of MAC processing elements in the systolic array, enhancing MAC PE utilization in the first and second convolution layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If input image data is provided in three channels (RGB), then the data format is simple and easy to process, but the MAC PE utilization in the first couple of convolution layers is low
Solution Approach 1:
The patent segments the three-channel RGB data into multiple separate channels by dividing the pixel data into groups and assigning each group to a separate channel. This segmentation increases the number of channels from 3 to a greater number, allowing better utilization of MAC PEs in the first couple of convolution layers while maintaining the ability to process image data efficiently
Solution Approach 2:
The patent transforms the data from a three-dimensional RGB format into a multi-channel format by adding a channel dimension. This dimensional change allows the data to be represented as multiple separate channels rather than three combined channels, thereby increasing MAC PE utilization without losing the essential image information
2Productivity
If the number of channels is increased to match the number of MAC PEs, then MAC PE utilization improves significantly, but the data processing complexity increases
Solution Approach 1:
The patent performs the channel rearrangement and data formatting operations before the data enters the systolic array for convolution processing. By preparing the data in advance and converting it to the appropriate multi-channel format upstream, the system avoids adding complexity within the critical convolution path while still achieving high MAC PE utilization
Solution Approach 2:
The patent introduces an intermediary data format conversion layer between the input data and the convolution operation. This intermediary layer handles the complexity of channel rearrangement and data formatting, separating the complex data preparation tasks from the simpler convolution computation, thereby enabling high MAC PE utilization without burdening the main processing path
Data Source
AI summary
A data format converter rearranges data of an input image for input to a systolic array of multiply and accumulate processing elements. The image has a pixel height and a pixel width in a number of channels equal to a number of colors per pixel. The data format converter rearranges the data to a second, greater number of channels and inputs the second number of channels to one side of the systolic array. The second number of channels is less than or equal to the number of MAC PEs on the one side of the systolic array, and results in greater MAC PE utilization in the systolic array.