Bitmap-Scheduled Matrix Computing Chip for Dynamic Sparsity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing matrix computing methods in AI and HPC fail to efficiently handle scenarios where the sparsity degree of matrices dynamically changes, leading to increased computational costs and reduced efficiency.
Innovation Solution
A chip design incorporating buffers, multiplexers, and scheduling modules that utilize bitmaps to selectively process non-zero elements, allowing dynamic sparsity management and efficient matrix operations without precision loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If pruning technology is used to sparsify matrices before computing, then computing amount is reduced and computing efficiency is improved, but the system cannot handle scenarios where sparsity degree dynamically changes
Solution Approach 1:
The system dynamically adjusts the computing process based on the actual sparsity of input matrices. The scheduling module generates selection signals in real-time based on bitmap information, allowing the system to adapt to varying sparsity degrees without requiring pre-pruning of matrices. This dynamic adaptation resolves the contradiction by making the computing efficiency improvement contingent on actual sparsity patterns rather than fixed pre-processing.
Solution Approach 2:
The system changes the parameter of sparsity representation from fixed pre-determined values to dynamic bitmap-based indicators. By using bitmaps to represent non-zero element positions and dynamically generating selection signals based on these bitmaps, the system can handle varying sparsity degrees adaptively while maintaining reduced computing amounts through selective processing of non-zero elements.
2Measurement precision
If all elements in matrices are processed during computing, then computing precision is maintained, but computing amount increases and efficiency decreases
Solution Approach 1:
The system extracts and processes only the non-zero elements of matrices by using bitmaps to identify positions of non-zero elements and generating selection signals to retrieve only these elements from buffers. This extraction approach maintains computing precision by ensuring all non-zero elements are processed while eliminating unnecessary computations on zero elements, thus resolving the contradiction between precision and efficiency.
Solution Approach 2:
The system performs partial action by processing only the necessary non-zero elements rather than all elements in the matrix. The bitmap-based selection mechanism ensures that exactly the right elements (non-zero ones) are processed, avoiding both insufficient processing (which would reduce precision) and excessive processing (which would reduce efficiency).
Data Source
AI summary
This application provides a matrix computing method, a chip, and a related device. The chip includes a first buffer, is configured to buffer a first vector, and a second buffer is configured to buffer a second vector. A scheduling module generates a selection signal based on a bitmap of the first vector. The selection signal may cause the processing element to obtain, from the first buffer, a group of non-zero elements in the first vector, and cause the processing element to obtain, from the second buffer, a group of elements in the second vector. An operation is performed between the first vector and the second vector based on the group of non-zero elements in the first vector and the group of elements in the second vector. In this application, an element whose value is 0 in one vector may be excluded from computing, to reduce a computing amount.