Matrix Multiplication Quantization with Dynamic Constraint Restoration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current matrix multiplication methods, such as General Matrix Multiplication (GEMM) and Quantization Matrix Multiplication, face challenges with float-type data processing, including high computational time and memory usage, and large computational errors due to fixed empirical values for inverse quantization.
Innovation Solution
A processor-implemented method that determines a first and second multiplication matrix based on input multiplicand and multiplier matrices, using quantization to handle float-type data, and calculates a matrix restoration constraint value to restore the multiplication result, allowing for efficient computation and reduced errors by dynamically determining constraint values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If General Matrix Multiplication (GEMM) is used for float-type data processing, then computation accuracy is maintained, but computational time and memory usage increase significantly
Solution Approach 1:
The patent changes the data type parameter from float to quantized integer types, and dynamically adjusts the quantization parameters (maximum and minimum values) based on the specific matrix multiplication task. This allows the system to use faster integer arithmetic while maintaining acceptable accuracy through adaptive parameter selection.
Solution Approach 2:
The patent introduces dynamic selection of quantization parameters (maximum and minimum values) based on the input matrices. Instead of using fixed quantization parameters, the system adapts the parameters to the specific data range and distribution, enabling optimal balance between speed and accuracy for each computation task.
2Productivity
If quantization matrix multiplication with fixed empirical values is used, then computational speed increases, but computation errors increase due to large computational errors
Solution Approach 1:
The patent dynamically changes the quantization parameters (maximum and minimum values) based on the actual input data characteristics. By selecting parameters that match the specific computation task rather than using fixed empirical values, the system reduces quantization errors while maintaining the speed benefits of integer arithmetic.
Solution Approach 2:
The system incorporates a feedback mechanism where the quantization parameters are selected based on the actual distribution and range of the input matrices. This adaptive selection process provides feedback from the data characteristics to the parameter selection, ensuring optimal quantization for each specific computation task.
3Productivity
If quantization matrix multiplication with fixed empirical values is used, then computational speed increases, but device complexity increases due to complicated computation with respect to maximum and minimum
Solution Approach 1:
The patent performs preliminary analysis of the input matrices to determine appropriate quantization parameters before executing the main computation. By pre-determining the maximum and minimum values based on the input data characteristics, the system avoids complex runtime calculations while ensuring accurate quantization for the specific task.
Data Source
AI summary
A processor-implemented method includes: determining a first multiplication matrix and a second multiplication matrix, based on an input multiplicand matrix and an input multiplier matrix that are generated from an input signal; determining a matrix to be restored, based on the first multiplication matrix and the second multiplication matrix; determining a matrix restoration constraint value, based on the matrix to be restored; determining a multiplication result of the input multiplicand matrix and the input multiplier matrix, based on the matrix restoration constraint value and the matrix to be restored; and analyzing the input signal based on the multiplication result.