Systolic Array Processing Element Parallel Data Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systolic arrays in neural networks often underutilize their processing capacity due to the mismatch between the precision of supported data types and the specific requirements of applications, leading to inefficient utilization and performance issues.
Innovation Solution
Implementing a method where each processing element in the systolic array performs multiple computations simultaneously, utilizing the full computing capacity by processing multiple input data elements in parallel, and generating multiple output data elements concurrently, thereby optimizing the systolic array's performance across various data types, including 4-bit, 8-bit, and 16-bit data types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a processing element performs a single computation on single input data elements, then the computation precision is maintained, but the utilization rate of the systolic array is low
Solution Approach 1:
The patent segments the computation process by dividing input data elements into groups that can be processed simultaneously. Each processing element handles multiple segmented data elements in parallel, increasing utilization without requiring a larger array. The segmentation allows the system to process 4-bit, 8-bit, and 16-bit data types efficiently by grouping compatible elements together.
Solution Approach 2:
The processing element is designed with multi-functionality to handle various data types (4-bit, 8-bit, 16-bit) and perform multiple computations simultaneously. This universal design allows a single processing element to serve multiple purposes, increasing the utilization rate without proportionally increasing device complexity.
2Speed
If the systolic array processes multiple input data elements in parallel, then the processing speed is improved, but the mismatch between data type precision and application requirements increases
Solution Approach 1:
The systolic array implements dynamic configuration where processing elements can be reconfigured on-the-fly to handle different data type precisions (4-bit, 8-bit, 16-bit). This dynamic adaptability allows the system to optimize processing speed for specific applications while maintaining versatility across different precision requirements through runtime reconfiguration.
Solution Approach 2:
The patent changes the precision parameter of data types being processed by implementing parallel processing paths that can handle different bit depths simultaneously. The system can process multiple input data elements with different precision levels in parallel, adapting to application requirements while maintaining high processing speed through parameter flexibility.
3Productivity
If a processing element generates a single output data element per computation cycle, then the computation accuracy is maintained, but the productivity of the systolic array is reduced
Solution Approach 1:
The patent adds a dimensional aspect to output generation by producing multiple output data elements across different dimensions simultaneously. Instead of generating one output per cycle, the processing element generates multiple outputs in parallel across different computational dimensions, increasing throughput while maintaining accuracy through consistent application of computational rules across all output channels.
Data Source
AI summary
A processing element (PE) of a systolic array can perform neural networks computations on two or more data elements of an input data set using the same weight. Thus, two or more output data elements corresponding to an output data set may be generated. Based on the size of the input data set and an input data type, the systolic array can process a single data element or multiple data elements in parallel.