Matrix Calculation Device with Pre-fetch Circuit for Sparse Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing matrix multiplication calculation methods in electronic circuits are inefficient due to the need for extensive hardware configuration and unnecessary calculations, particularly in neural network operations where sparse matrices are common, leading to redundant calculations.
Innovation Solution
A matrix calculation device comprising a storing unit, a multiply accumulate (MAC) circuit, a pre-fetch circuit, and a control circuit that pre-fetches data from matrices to determine whether to perform MAC calculations, thereby reducing unnecessary calculations by identifying zero rows and columns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional matrix multiplication is performed on all elements, then complete calculation accuracy is achieved, but calculation time increases and efficiency decreases
Solution Approach 1:
The pre-fetch circuit retrieves data from storage before the MAC circuit performs calculations. By anticipating which data will be needed and loading it in advance, the system avoids idle waiting time during computation, thereby reducing overall calculation time while maintaining accuracy for non-sparse matrices
Solution Approach 2:
The control circuit performs partial MAC calculations by skipping operations on zero elements identified through pre-fetching. Instead of computing all possible products, the system selectively executes only necessary calculations, significantly reducing computation time while preserving calculation accuracy for non-zero elements
2Productivity
If pre-fetch circuit is added to reduce calculations, then calculation efficiency improves, but device complexity increases
Solution Approach 1:
The matrix calculation system is divided into distinct functional modules: a pre-fetch circuit for data retrieval, a control circuit for decision-making, and a MAC circuit for computation. This segmentation allows each component to be optimized independently and simplifies the overall architecture by assigning specific responsibilities to each unit, reducing implementation complexity
Solution Approach 2:
The pre-fetch circuit acts as an intermediary between storage and the MAC circuit, filtering and preparing data before it reaches the computation unit. This intermediary layer reduces the burden on the MAC circuit by pre-processing data and enabling the control circuit to make informed decisions about which calculations to perform, improving efficiency without significantly increasing complexity
3Measurement precision
If all MAC calculations are performed, then complete result accuracy is achieved, but number of storage accesses increases
Solution Approach 1:
The pre-fetch circuit performs preliminary data retrieval from storage before the MAC circuit needs the data. By loading required data in advance and organizing it in the pre-fetch buffer, the system reduces the number of times the MAC circuit and control circuit need to access storage during the calculation phase, thereby reducing total storage access count while maintaining result accuracy
Solution Approach 2:
The control circuit performs partial storage accesses by selectively retrieving only non-zero elements and their corresponding indices. Instead of accessing storage for every matrix element, the system intelligently accesses only the necessary data required for accurate calculation, significantly reducing the quantity of storage accesses while preserving result accuracy
Data Source
AI summary
A matrix calculation device including a storing unit, a multiply accumulate (MAC) circuit, a pre-fetch circuit, and a control circuit, and an operation method thereof are provided. The storing unit stores a first and second matrixes. The MAC circuit is configured to execute MAC calculation. The pre-fetch circuit pre-fetches at least one column of the first matrix from the storing unit to act as pre-fetch data, pre-fetches at least one row of the second matrix from the storing unit to act as the pre-fetch data, or pre-fetches at least one column of the first matrix and at least one row of the second matrix from the storing unit to act as the pre-fetch data. The control circuit decides whether to perform the MAC calculation on a current column of the first matrix and a current row of the second matrix through the MAC circuit according to the pre-fetch data.