Quantized Vector Computing With Precomputed Error Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Quantization error in vector computing leads to inaccuracies in AI and ML applications due to precision loss when floating point data is converted to integer data, which can be mitigated by increasing circuit complexity and cost.
Innovation Solution
Compensating input data vectors with a compensation vector after quantization to reduce quantization error without requiring higher precision, using techniques applicable in semiconductor devices like NAND or NOR flash memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If quantization is used to convert floating point data to integer data for vector computing, then circuit complexity and cost are reduced, but computing accuracy deteriorates due to precision loss
Solution Approach 1:
The compensation vector is pre-calculated based on the statistical distribution characteristics of the quantized data. This preliminary computation captures the systematic quantization error patterns, allowing the compensation to be applied efficiently during vector computing operations without increasing real-time computational complexity
Solution Approach 2:
The invention introduces a compensation vector parameter that adjusts the quantized data values to counteract systematic errors. By modifying the data representation to include this compensation parameter, the accuracy is improved while maintaining the integer-based quantized format and avoiding increased circuit complexity
2Productivity
If quantization is used to convert floating point data to integer data, then computing speed is improved, but computing accuracy deteriorates due to quantization error
Solution Approach 1:
The compensation vector is pre-computed based on the statistical properties of the quantized data distribution. This allows the compensation to be applied as a simple vector addition during computing operations, maintaining high computing speed while correcting accuracy issues
Solution Approach 2:
The compensation vector acts as an intermediary that bridges the gap between quantized integer data and the original floating point accuracy requirements. It mediates the trade-off by providing error correction without requiring the system to use higher precision data formats
Data Source
AI summary
A method for performing a computing task includes: extracting one or more features from a user content; converting the features to a floating point query vector; quantizing the floating point query vector; obtaining a database vector including one or more floating point feature vectors; determining a compensation vector based on a data distribution of the floating point query vector; quantizing the floating point feature vectors; determining an error function based on a difference between data distributions of i) the quantized query vector compensated with the compensation vector, and ii) the floating point query vector; determining, based on the error function, values of the compensation vector corresponding to the quantized feature vectors; combining the quantized query vectors and the values of the compensation vector to obtain one or more compensated query vectors; and performing the computing task using the compensated query vectors and the quantized feature vectors to obtain an output.