Configurable Systolic Array for Neural Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systolic arrays used in neural network processing face inefficiencies due to mismatches in the number of input and output data sets, leading to reduced utilization rates and increased processing times, resulting in wasted computing resources and degraded performance.
Innovation Solution
A dynamically configurable array of processing elements with multiple adders and multipliers, capable of expanding or shrinking the number of input and output data sets, allowing for flexible processing configurations to match the requirements of different neural network layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the systolic array is configured with fixed number of rows and columns, then the hardware structure is simple, but the utilization rate decreases when there is a mismatch between input and output data sets
Solution Approach 1:
The patent implements dynamic reconfiguration of the systolic array by allowing processing elements to be dynamically assigned to different input data sets. The controller can change the mapping between input data sets and array rows during operation, enabling the fixed hardware structure to adapt to varying computational requirements and maintain high utilization rates
Solution Approach 2:
Each processing element in the systolic array is designed to handle multiple input data sets through dynamic configuration. The same physical array can process different numbers and sizes of input data sets by reassigning the mapping between inputs and processing elements, making the hardware universally applicable to various neural network layer configurations
2Productivity
If the systolic array processes multiple input data sets with fixed configuration, then the processing throughput is limited, but the processing time increases due to sequential processing
Solution Approach 1:
The patent segments the processing of multiple input data sets by dividing them into batches that can be processed in parallel across different rows of the systolic array. The controller manages the segmentation and assignment of input data sets to array rows, enabling simultaneous processing of multiple data sets and reducing overall processing time
Solution Approach 2:
The systolic array maintains continuous operation by dynamically reconfiguring the mapping between input data sets and processing elements between batches. This ensures that all processing elements remain actively engaged in computation throughout the processing of multiple data sets, eliminating idle time and maximizing throughput
Data Source
AI summary
Disclosed herein are techniques for performing neural network computations. In one embodiment, an apparatus includes an array of processing elements, the array having configurable dimensions. The apparatus further includes a controller configured to set the dimensions of the array of processing elements based on at least one of: a first number of input data sets to be received by the array, or a second number of output data sets to be output by the array.