Sparse Matrix Vector Product via Shifting and Addition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial neural networks (ANNs) face challenges in efficiently computing matrix vector products due to high computational effort and energy consumption, particularly during inference, which is exacerbated by the need for accurate calculations and the inefficiencies of standard hardware implementations.
Innovation Solution
The proposed solution involves representing the matrix as a product of sparse submatrices, with each submatrix approximated by a series of sparse matrices containing only a limited number of non-zero elements, allowing for efficient computation through shifting and addition operations without the need for multiplication units, and implementing this on reconfigurable hardware like FPGAs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If standard hardware implementations are used for matrix vector multiplication, then computational accuracy is maintained, but computational effort and energy consumption are high
Solution Approach 1:
The patent changes the representation parameters of the matrix by decomposing it into sparse submatrices with specific structures (limited non-zero elements per row). This parameter transformation allows the use of simplified computation operations (shifting and addition instead of full multiplication) while maintaining acceptable accuracy, thereby reducing energy consumption significantly
Solution Approach 2:
The patent substitutes the standard multiplication operation with a different mechanical/computational mechanism: shifting and addition operations. By representing matrix elements in a format that enables these alternative operations, the system achieves the same computational goal with lower energy requirements and simpler hardware
2Measurement precision
If standard hardware implementations are used for matrix vector multiplication, then computational accuracy is maintained, but hardware resources and computational effort are high
Solution Approach 1:
The matrix is transformed into a product of sparse submatrices with specific structural parameters (limited non-zero elements per row). This parameter change enables the use of simpler hardware components that perform shifting and addition rather than full multiplication units, reducing hardware complexity while preserving accuracy
Solution Approach 2:
The patent segments the original matrix into multiple sparse submatrices. Each submatrix can be processed independently with simplified operations, and the results are combined to produce the final output. This segmentation reduces the complexity of individual processing units while maintaining overall computational accuracy
3Device complexity
If matrix is represented as product of sparse submatrices, then hardware resources are reduced, but computational complexity of decomposition increases
Solution Approach 1:
The computationally intensive matrix decomposition into sparse submatrices is performed once during a preprocessing stage, before the actual matrix-vector multiplication. This preliminary action separates the one-time decomposition cost from the repeated multiplication operations, making the overall system more efficient for multiple inference operations
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
An apparatus for computing a matrix vector product of a given matrix and an arbitrary vector is described. The given matrix is represented by S submatrices, with S≥1, with each submatrix representing a vertical slice of the given matrix, and with each submatrix approximated by the product of P further matrices, with P≥1. Each further matrix is a sparse matrix and includes in each row a certain number of elements unequal to zero. The apparatus comprises S processing chains, wherein each processing chain is to receive the arbitrary vector and comprises P processing blocks. Each processing block is to multiply a block input vector and an associated further matrix by shifting the elements of the block input vector according to the values of the elements in the associated further matrix which are unequal to zero, and by combining the shifted elements of the block input vector to obtain respective elements of a block output vector.