Matrix Multiplier Using Binary Quantized Weights for Faster AI Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing matrix multiplication technologies in artificial intelligence models are inefficient, as they spend most of their operation time on calculating output matrices through multiplication of input matrices and weight matrices, leading to high computational requirements.
Innovation Solution
A matrix multiplier and matrix multiplication device are designed to perform matrix multiplication with improved speed and reduced computation by using an input vector scaler, data type converters, and a processing element array that applies binary coding quantization (BCQ) to generate quantization sign values and scale coefficients, thereby reducing the number of calculations required.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional matrix multiplication is used to calculate output matrices, then computation accuracy is maintained, but computational load increases and operation time increases
Solution Approach 1:
The patent applies binary coding quantization to transform weight matrix parameters from high-precision floating-point format to low-precision binary format. This parameter transformation reduces the computational complexity of matrix multiplication while maintaining acceptable accuracy for AI inference tasks, directly addressing the contradiction between operation speed and computational load
Solution Approach 2:
The patent differentiates between different data types in the matrix multiplication process. Input vectors are processed in floating-point format to maintain precision where needed, while weight matrices are converted to binary format for efficient computation. This local differentiation of data quality allows the system to optimize computation speed without sacrificing critical accuracy requirements
2Quantity of substance
If binary coding quantization is applied to reduce computation amount, then computational efficiency improves, but device complexity increases
Solution Approach 1:
The patent performs binary coding quantization of the weight matrix in advance, before the actual matrix multiplication operation. This preliminary transformation converts complex floating-point weights into simple binary codes, reducing the computational burden during inference. The quantization process is done once during model preparation, not during each computation, which minimizes the impact on operational complexity
Solution Approach 2:
The patent creates a binary-coded version of the weight matrix as a simplified copy for computation purposes. Instead of directly using the original complex weight matrix, the system works with this binary representation that preserves essential information while enabling faster computation. This copying approach separates the complexity of the original model from the efficiency of the computation
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A matrix multiplier includes an input vector scaler configured to generate a first scaled input vector based on a first input vector and a plurality of quantization scale coefficients, a first data type converter configured to generate a first fixed-point scaled input vector based on the first scaled input vector, a processing element array including a first processing element configured to generate a first fixed-point output element based on the first fixed-point scaled input vector and first plurality of quantization sign values and a second processing element configured to generate a second fixed-point output element based on the first fixed-point scaled input vector and second plurality of quantization sign values, and a second data type converter configured to generate first and second output elements by converting data type of the first and second fixed-point output elements, and to output a first output vector including the first and second output elements.