Sparse Tensor Circuit for FPGA AI Matrix Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing configurable integrated circuits face challenges in efficiently implementing structured sparsity in AI models due to high resource demands for routing and logic, particularly in field programmable gate arrays (FPGAs), which can offset the reduction in digital signal processing blocks.
Innovation Solution
A tensor circuit block is introduced that utilizes structured sparsity by streaming weights from a sparse weight matrix and storing activations in registers, employing multiplexer circuits to align activations with weights, and multipliers to generate products summed by a summation block, with interchangeable dense and sparse modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If structured sparsity is implemented using traditional FPGA approaches, then the number of non-zero weights is reduced, but routing and logic resources increase significantly
Solution Approach 1:
The patent segments the weight matrix into structured sparse blocks where zero-weight positions are systematically identified and excluded from processing. By dividing the matrix into manageable segments with known sparsity patterns, the system can efficiently route only non-zero weights to dedicated multiplier circuits, reducing overall routing complexity while maintaining computational functionality.
Solution Approach 2:
The invention extracts and removes zero-weight elements from the processing pipeline by using sparsity information to selectively disable routing paths. This extraction approach allows the system to focus computational resources only on non-zero weights, significantly reducing the logic and routing requirements while preserving the essential computational function.
2Adaptability or versatility
If more logic resources are allocated to handle structured sparsity, then processing capability is maintained, but the reduction in DSP blocks is offset
Solution Approach 1:
The patent implements multi-functional logic resources that can dynamically adapt between dense and sparse matrix processing modes. The same logic blocks serve multiple purposes: handling both fully dense matrices and structured sparse matrices, thereby maximizing the utilization efficiency of available DSP blocks while maintaining versatile processing capability across different data types.
Solution Approach 2:
The system dynamically reconfigures its logic resources based on the sparsity characteristics of the input data. By monitoring and adapting to the actual sparsity pattern, the logic can flexibly allocate routing paths and computational resources in real-time, ensuring optimal productivity whether processing dense or sparse matrices without requiring separate dedicated hardware for each case.
Data Source
AI summary
A tensor circuit includes first storage circuits coupled to store first activation values from an activation matrix, second storage circuits coupled to store second activation values from the activation matrix, multiplexer circuits configurable to output a subset of the first and the second activation values stored in the first and the second storage circuits, multiplier circuits coupled to multiply weight values from a sparse weight matrix by the subset of the first and the second activation values output by the multiplexer circuits to generate products, and a summation circuit coupled to sum the products.