Activation-Cache MAC Array Layout for Sparse Compute Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital systems face inefficiencies in performing multiply-accumulate operations due to large MAC arrays that process data at a lower rate and encounter varying sparsity speed-ups, leading to uneven processing times across different sections of the array.
Innovation Solution
A multiply-accumulate array circuit with an activation cache that divides input activation arrays into sub-slices, storing spatially local activations in cache lines and coupling them to multiple smaller MA groups, allowing independent processing and efficient reuse across groups.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If a single large MAC array is used to process large input activation arrays, then the computational capacity is increased, but the processing rate is reduced due to lower fetch bandwidth and delayed processing start
Solution Approach 1:
The patent divides a single large MAC array into multiple smaller MAC arrays (e.g., four 8x8 MAC arrays instead of one 32x32 array). This segmentation allows each smaller array to process data independently with higher fetch bandwidth, reducing the delay in processing start while maintaining the overall computational capacity through parallel operation of multiple arrays.
2Productivity
If sparsity speed-up techniques are applied to skip zero value multiplications, then computational efficiency is improved, but different sections of the MAC array complete processing at different times resulting in uneven processing
Solution Approach 1:
By segmenting the large MAC array into multiple smaller arrays, the patent ensures that sparsity speed-up techniques can be applied more effectively within each smaller section. Each smaller MAC array processes a subset of activations, and when zeros are encountered, the skip operations occur within manageable sections, reducing the variation in processing completion times across the overall system.
Solution Approach 2:
The patent applies local quality by allowing different sections of the MAC array to have different processing characteristics. Each smaller MAC array can independently optimize its operation based on the sparsity patterns of its specific input data, enabling tailored sparsity speed-up strategies for different regions while maintaining overall processing coherence.
3Device complexity
If a single large MAC array is used, then the device complexity is reduced, but the layout flexibility and processing efficiency are compromised
Solution Approach 1:
The patent segments the MAC array into multiple smaller arrays that can be arranged in a grid configuration. This segmentation improves layout flexibility by allowing the smaller arrays to be positioned and connected in various configurations that optimize data flow and memory access patterns, while the overall structure remains relatively simple through modular design.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
Embodiments of the present disclosure include a multiply-accumulator (MAC) array circuit comprising an activation cache and a plurality of multiply-accumulator (MA) groups. The activation cache comprises cache lines configured to store sub-slices of an input activation array. The cache lines are coupled to particular MA groups. Activations stored in the cache lines may be used and reused across multiple MA groups.