Encoded Weight Blocks for Sparse ANN Matrix Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial neural network (ANN) models, particularly deep neural networks (DNNs) and convolutional neural networks (CNNs), face inefficiencies in matrix multiplication due to the presence of numerous zero values in sparse matrices, leading to excessive memory allocation and inefficient computation.
Innovation Solution
A matrix encoding process that converts ANN weight tensors into a block-based format, allowing for fixed storage size while balancing zero, small, and large non-zero values, with a matrix multiply accelerator (MMA) to perform convolutions on reconstructed tensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If standard matrix multiplication techniques are applied to sparse and ANN matrices, then computation can be performed, but memory allocation is excessive and power consumption is high due to the large number of zero values
Solution Approach 1:
The matrix is divided into blocks of fixed size, and each block is independently encoded to store only non-zero elements with their positions. This segmentation allows the system to process sparse and ANN matrices by handling small portions at a time, reducing overall memory requirements while maintaining computation efficiency through optimized block processing.
Solution Approach 2:
The invention extracts and removes zero values from the matrix representation by using a compressed format that stores only non-zero elements along with their position information. This extraction of unnecessary zero data significantly reduces memory allocation and power consumption while preserving the essential computational information needed for matrix operations.
2Productivity
If standard matrix multiplication techniques are applied to sparse and ANN matrices, then computation can be performed, but power consumption is high in devices with power constraints
Solution Approach 1:
By segmenting the matrix into blocks and processing them independently with optimized encoding, the system reduces the total number of operations required, thereby lowering power consumption in mobile and resource-constrained devices while maintaining acceptable computation efficiency.
Solution Approach 2:
The extraction of zero values from the matrix representation reduces the amount of data that needs to be processed, directly decreasing power consumption. The compressed format stores only essential non-zero information, enabling efficient computation with reduced energy requirements for devices with power constraints.
3Ease of operation
If standard matrix multiplication techniques are applied to sparse and ANN matrices, then computation can be performed, but the large number of zero operands makes the process inefficient
Solution Approach 1:
The matrix is segmented into blocks that are processed independently, with each block encoded to exclude zero values. This segmentation simplifies the computation by focusing only on non-zero elements while maintaining the structural integrity of the matrix, making the operation both simpler and more efficient.
Solution Approach 2:
By extracting and removing zero values from the matrix representation, the system eliminates unnecessary operations. The compressed format stores only non-zero elements with their positions, making computation simpler by reducing the number of operands while improving efficiency by eliminating redundant calculations involving zero values.
Data Source
AI summary
A matrix multiplication system and method are provided. The system includes a memory that stores one or more weight tensors, a processor and a matrix multiply accelerator (MMA). The processor converts each weight tensor into an encoded block set that is stored in the memory. Each encoded block set includes a number of encoded blocks, and each encoded block includes a data field and an index field. The MMA converts each encoded block set into a reconstructed weight tensor, and convolves each reconstructed weight tensor and an input data tensor to generate an output data matrix.