Systolic Array Transposed Convolution Memory Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Transposed convolution operations in neural network processors require a large number of memory operations for generating dilated input and rotated weight data matrices, leading to efficiency degradation due to memory access latency.
Innovation Solution
A neural network processor configuration that includes a memory, systolic array, and controller, where the controller loads weight and input data elements into the systolic array to perform partial sums, avoiding the need for extensive memory copying and dilation operations by using a summation buffer to accumulate results and handle zero output elements efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional memory-based transposed convolution operations are used, then the operation can be performed with standard memory access patterns, but the number of memory operations increases significantly leading to efficiency degradation
Solution Approach 1:
The transposed convolution operation is segmented into distinct functional units within the systolic array architecture. The array is divided into multiple processing elements that each handle specific computational tasks, allowing parallel processing of different portions of the input data matrix. This segmentation enables simultaneous computation of multiple output elements, reducing overall execution time and memory access requirements.
Solution Approach 2:
The systolic array acts as an intermediary computational structure between memory and the final output. Instead of directly accessing memory for each computation, data flows through the systolic array where intermediate calculations are performed in a pipelined manner. This intermediary structure buffers data flows and performs computations in between memory accesses, reducing the frequency and latency of memory operations.
2Reliability
If dilated input data matrix and rotated weight data matrix are generated through copying operations, then the transposed convolution can be performed correctly, but the device complexity and memory operations increase
Solution Approach 1:
Instead of creating dilated input matrices and rotated weight matrices through copying and transformation operations, the systolic array inverts the approach by directly computing the transposed convolution using the original input and weight matrices. The computational pattern within the systolic array inherently produces the correct transposed convolution results without requiring explicit dilation or rotation steps, thereby reducing device complexity.
Solution Approach 2:
The patent extracts the essential computational pattern of transposed convolution and implements it directly within the systolic array logic. Rather than extracting data into separate dilated and rotated matrices, the computation is performed in-place within the array structure, taking out only the necessary computational operations and eliminating the need for intermediate data transformation steps.
Data Source
AI summary
In one example, a neural network accelerator can execute a set of instructions to: load a first weight data element from a memory into a systolic array, the first weight data element having first coordinates; extract, from the instructions, information indicating a first subset of input data elements to be obtained from the memory, the first subset being based on a stride of a transposed convolution operation and second coordinates of first weight data element in a rotated array of weight data elements; based on the information, obtain the first subset of input data elements from the memory; load the first subset of input data elements into the systolic array; and control the systolic array to perform first computations based on the first weight data element and the first subset of input data elements to generate output data elements of an array of output data elements.