Systolic Array Calculation Circuit for Deep Learning MAC Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning systems face challenges with high power consumption and long calculation times due to the need for a large number of MAC (Multiply-Accumulate) calculations, which affects the operation performance of MAC operation circuits.
Innovation Solution
A calculation circuit with a systolic array architecture that includes multiple MAC circuits, allowing for simultaneous data and weight value propagation through multiple paths, enabling efficient generation and propagation of drain values, thereby reducing calculation time and improving throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a large number of MAC calculations are performed to process big data in deep learning systems, then the processing capability and accuracy are improved, but the power consumption and calculation time increase significantly
Solution Approach 1:
The calculation circuit is divided into multiple calculator groups arranged in a systolic array configuration with rows and columns. Each calculator group processes a portion of the data independently, allowing parallel execution of MAC operations. This segmentation enables the system to handle large-scale deep learning computations by distributing the workload across multiple units, improving processing capability while managing power consumption through selective activation of calculator groups.
Solution Approach 2:
The patent introduces a two-dimensional systolic array structure with multiple rows and columns of calculator groups, transforming the traditional one-dimensional processing architecture. Data values propagate through rows while weight values propagate through columns, enabling simultaneous multi-dimensional processing. This dimensional expansion allows the system to perform numerous MAC calculations in parallel, significantly enhancing processing capability without proportionally increasing power consumption.
2Productivity
If more MAC calculations are performed to process more data, then the deep learning processing accuracy is improved, but the calculation time becomes longer
Solution Approach 1:
The systolic array architecture enables continuous data flow through the calculator groups, with data values propagating sequentially from one calculator group to the next in each row. This continuous action eliminates idle time between calculations, as each calculator group continuously performs MAC operations on incoming data and weight values. The pipeline-like operation ensures that processing capability is maximized without unnecessary delays, reducing overall calculation time for deep learning tasks.
Solution Approach 2:
By segmenting the calculation circuit into multiple calculator groups that operate in parallel, the patent divides the overall calculation task into smaller sub-tasks that can be executed simultaneously. Each calculator group processes a specific portion of the data stream, allowing the system to complete numerous MAC calculations in parallel rather than sequentially. This segmentation directly reduces calculation time while maintaining high processing capability.
3Device complexity
If data values are propagated through multiple calculator groups in the same row through a single data path, then the device complexity is reduced, but the data propagation time increases
Solution Approach 1:
The patent resolves the trade-off by introducing a two-dimensional data propagation structure where data values move horizontally through rows of calculator groups while weight values move vertically through columns. This dimensional approach allows data to reach multiple calculator groups in parallel by utilizing both row and column pathways, maintaining simple data path structures while achieving fast data propagation speeds through the systolic array architecture.
4Device complexity
If multiple drain value sets are propagated through the same drain path at different times, then the device complexity is reduced, but drain conflicts occur and throughput decreases
Solution Approach 1:
The patent eliminates drain conflicts by implementing a two-dimensional drain path structure where each calculator group has dedicated drain paths in both row and column directions. This dimensional expansion allows multiple drain value sets to be propagated simultaneously through different paths without conflicts, significantly improving hourly throughput while maintaining manageable device complexity through the systematic organization of the systolic array.
Data Source
AI summary
A calculation circuit may include a plurality of calculator groups constituting a systolic array composed of a plurality of rows and columns, wherein calculator groups included in each of the rows propagate a data value set through a single data path corresponding to the row in a data propagation direction, and propagate a plurality of drain value sets through a plurality of drain paths corresponding to the row in a drain propagation direction, and wherein a calculator group of the calculator groups included in each of the rows comprises a plurality of MAC (Multiplier-Accumulator) circuits, and the MAC circuits generate drain values respectively included in the drain value sets at the same time. The calculator groups included in each column may further propagate a weight value set corresponding to the column through a plurality of weight data paths corresponding to the column.