Sparse Array Multiplication Accelerator With Dynamic MAC Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processors struggle to efficiently perform array multiplication with sparse arrays due to fixed compression rates and inefficient utilization of multiply-accumulate circuits, especially when the sparsity ratio exceeds 50%, leading to suboptimal performance and power consumption.
Innovation Solution
A hardware accelerator that stores sparse arrays in a compressed format using a mask array and non-zero element array, allowing dynamic loading of non-zero elements and utilizing a variable number of multiply-accumulate circuits based on actual sparsity, reducing memory footprint and improving performance by a factor of N/M.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If fixed compression rates are used for sparse arrays, then storage efficiency is improved, but computational performance deteriorates when sparsity ratio exceeds 50%
Solution Approach 1:
The patent implements dynamic configuration of multiply-accumulate circuits based on actual sparsity ratios. The system adjusts the number of active MAC circuits and compression rate according to the measured sparsity of input data, transitioning from fixed to dynamic resource allocation. This allows the accelerator to optimize between storage efficiency and computational performance adaptively.
Solution Approach 2:
The system changes operational parameters (compression rate, number of MAC circuits) based on the sparsity ratio of the input sparse array. When sparsity exceeds 50%, the system adjusts parameters to favor computational performance; when sparsity is lower, it optimizes for storage efficiency. This parameter adaptation resolves the contradiction between fixed compression benefits and variable performance requirements.
2Productivity
If all multiply-accumulate circuits are activated, then computational throughput is improved, but power consumption increases for high sparsity ratios
Solution Approach 1:
The patent applies partial action by activating only the necessary number of multiply-accumulate circuits based on the actual sparsity ratio. Instead of always activating all MAC circuits for maximum throughput, the system activates only M/N × 100% of circuits where M is the sparsity ratio and N is the total number of circuits. This partial activation maintains adequate throughput while significantly reducing power consumption for high sparsity inputs.
Solution Approach 2:
The system dynamically adjusts the number of active MAC circuits according to the sparsity characteristics of the input data. The controller measures the sparsity ratio and configures the appropriate number of MAC circuits to activate, creating a dynamic power-throughput tradeoff that adapts to different workloads.
3Quantity of substance
If compression rate is increased for high sparsity, then memory footprint is reduced, but access time to non-zero elements increases
Solution Approach 1:
The patent segments the sparse array processing into two distinct phases: compression phase and computation phase. During compression, the system achieves high compression ratios by storing only non-zero elements. During computation, the system uses dedicated hardware (MAC circuits) that processes the compressed format efficiently without requiring decompression. This segmentation allows the system to achieve both low memory footprint and fast access time by optimizing each phase separately.
Solution Approach 2:
The system introduces a specialized compression format as an intermediary representation between storage and computation. This intermediate format stores only non-zero elements with their positions, enabling efficient storage while allowing direct access to non-zero elements for computation without full decompression. The intermediary format bridges the gap between compressed storage and fast computation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method and apparatus for multiplying a first array including a plurality of equal-sized subgroups of elements, each including at least a minimum number of zero-elements, by a second array, by, for each subgroup of elements of the first array: loading a subgroup mask indicating locations of non-zero elements within the subgroup of elements of the first array, from memory into a first register; loading, from memory into a second register, the non-zero elements in the subgroup of elements of the first array; loading, from memory into a third register, a subgroup of elements of the second array corresponding to the subgroup of elements of the first array; and multiplying each of the non-zero elements of the first array by the corresponding elements of the second array, wherein the corresponding elements of the second array are selected according to the subgroup mask.