Pruned Sub-Matrix Multiplication Device for AI Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing semiconductor devices face challenges in performing matrix multiplication efficiently, leading to increased operation time and resource consumption in artificial intelligence models.
Innovation Solution
A matrix multiplication device and system that utilize a weight memory circuit, input matrix buffer, and processing element arrays to perform matrix multiplication with pruned sub-matrices, reducing computation and resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional matrix multiplication is performed using full weight matrices, then computation accuracy is maintained, but operation time and resource consumption increase significantly
Solution Approach 1:
The weight matrix is divided into multiple sub-matrices, and each sub-matrix is further pruned to retain only significant weights. This segmentation allows parallel processing of multiple sub-matrices, significantly reducing the overall computation time while maintaining computational accuracy through selective weight retention.
Solution Approach 2:
The weight matrix undergoes pruning operations before the actual matrix multiplication to eliminate insignificant weights. This preliminary action reduces the number of computations required during the multiplication process, thereby decreasing operation time without substantially compromising the result accuracy.
2Productivity
If pruning is applied to reduce computation amount, then operation speed improves, but computation precision may deteriorate
Solution Approach 1:
Different pruning strategies and threshold values are applied to different sub-matrices based on their specific characteristics. This local quality approach ensures that each sub-matrix is pruned appropriately, retaining sufficient precision for its specific computational requirements while optimizing overall computation efficiency.
Solution Approach 2:
The pruning process incorporates feedback mechanisms where the impact of pruning on computation results is evaluated, and pruning thresholds are adjusted accordingly. This feedback loop ensures that computation precision is maintained at acceptable levels while still achieving significant reductions in computation amount.
3Productivity
If multiple processing element arrays are used to parallelize computation, then productivity increases, but device complexity increases
Solution Approach 1:
The computation task is segmented into multiple sub-matrices that can be processed in parallel by different processing element arrays. This segmentation enables efficient utilization of multiple processing units while keeping the control logic relatively simple, as each processing element handles a well-defined sub-matrix independently.
Solution Approach 2:
The multiple processing element arrays are designed with universal functionality to handle different types of matrix operations. This multi-functionality reduces the need for specialized hardware for each operation type, thereby managing device complexity while maintaining high parallel computation capability.
Data Source
AI summary
A matrix multiplication device including a weight memory circuit, an input matrix buffer, a first processing element array and a second processing element array is provided. The weight memory circuit stores a first pruned sub-matrix and a second pruned sub-matrix. The input matrix buffer receives an input matrix including a plurality of input elements, outputs a first plurality of input elements corresponding to a first plurality of residual weights of the first pruned sub-matrix, and outputs a second plurality of input elements corresponding to a second plurality of residual weights of the second pruned sub-matrix. The first processing element array receives the first plurality of residual weights and the first plurality of input elements, and outputs a first sub-output matrix. The second processing element array receives the second plurality of residual weights and the second plurality of input elements, and output a second sub-output matrix.