Systolic Array Dataflow for High-Utilization Depthwise Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The utilization ratio of systolic arrays is low for depthwise convolutions due to low reusability calculations, leading to reduced computational efficiency, and adding data paths for connectivity results in significant overhead.
Innovation Solution
A method and device that preload weight values into PE chains through column input links, supply input values in column-major order, and utilize empty slots for overlapping calculations, allowing simultaneous execution of multiple depthwise convolutions without altering the conventional lattice structure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a conventional lattice-type systolic array structure is used for depthwise convolution, then the structure remains simple and unchanged, but the utilization ratio of PEs becomes very low (1/N) and computational efficiency is reduced
Solution Approach 1:
The patent dynamically configures the systolic array by selectively activating and deactivating PEs based on the convolution type. For depthwise convolution, it dynamically reconfigures data flow patterns and PE utilization without physically changing the lattice structure, thereby adapting the fixed hardware to variable computational requirements and improving utilization ratio.
Solution Approach 2:
The patent changes operational parameters such as data flow direction, PE activation state, and calculation mode to optimize performance for different convolution types. By modifying these parameters rather than the physical structure, the system achieves high computational efficiency for depthwise convolution while maintaining the simple lattice architecture.
2Productivity
If additional data paths are added to maximize PE usage for depthwise convolution, then computational efficiency improves, but significant overhead occurs in connecting PEs
Solution Approach 1:
The patent makes the existing data paths in the systolic array multi-functional by designing them to support both standard convolution and depthwise convolution operations. The same data paths are reused for different purposes depending on the operation type, eliminating the need for additional dedicated data paths for depthwise convolution and reducing overhead.
Solution Approach 2:
The systolic array is designed to self-adapt to different convolution types by internally reconfiguring data flow patterns and PE utilization without requiring external additional connections. The existing infrastructure serves multiple functions through intelligent resource allocation and operational mode switching, reducing the need for extra hardware overhead.
Data Source
AI summary
The present disclosure includes a first step of preloading weight values of each filter into respective processing element (PE) chains through a column input link, a second step of supplying an input value to the column input link in the same order as the weight values of the preloaded filters, a third step of starting accumulation in a column output link matching the column input link when a first input value among the input values reaches a top end of the PE chains, a fourth step of moving while further accumulating a product of a facing input value and the weight values of the filter loaded at a corresponding position when the accumulated value moves and faces the input value supplied, and a fifth step of outputting an output value accumulated by multiplication when the accumulated value to be moved reaches a lowest end of the PE chains.