ML Model Execution Optimization via Logarithmic Weight Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models, particularly deep neural networks, face inefficiencies in execution due to resource-intensive computations and storage requirements, especially on embedded devices with limited resources, where weight representation often uses high-bit precision leading to increased power consumption and reduced throughput.
Innovation Solution
The proposed solution involves a system for optimizing machine learning model execution by quantizing weights and activation matrices, transforming them into alternate representations like logarithmic or frequency domains, and rearranging them into sparse matrix structures, which allows for more efficient processing on hardware like FPGAs and CGRAs, reducing storage and energy costs through techniques such as packing, blocking, and fusing layers into pipelines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-bit precision weight representation is used in machine learning models, then model accuracy is maintained, but power consumption increases and throughput decreases
Solution Approach 1:
The patent transforms weight parameters from linear domain to logarithmic domain using the relationship w = exp(w_log), where w_log is stored with reduced precision (e.g., 8-bit integers instead of 32-bit floats). This parameter transformation allows the same computational accuracy to be achieved with lower precision storage, directly reducing power consumption and memory bandwidth requirements while maintaining model accuracy.
2Measurement precision
If high-bit precision weight representation is used in machine learning models, then model accuracy is maintained, but execution throughput decreases
Solution Approach 1:
By changing the parameter representation from linear to logarithmic domain, the patent enables reduced precision arithmetic operations. The transformation w = exp(w_log) allows weights to be computed and stored with fewer bits, which increases execution throughput by reducing computational complexity and memory access requirements while preserving the necessary precision for accurate model execution.
3Use of energy by moving object
If quantization is applied to reduce precision, then power consumption and storage requirements are reduced, but model accuracy may deteriorate
Solution Approach 1:
The patent applies parameter transformation to the logarithmic domain where quantization has less impact on model accuracy. By storing quantized values in the logarithmic domain (w_log instead of w), the reconstruction error after quantization is minimized because the logarithmic transformation compresses the dynamic range of weight values, allowing efficient quantization while maintaining high model accuracy.
4Quantity of substance
If quantization is applied to reduce precision, then storage requirements are reduced, but model accuracy may deteriorate
Solution Approach 1:
The patent transforms weight parameters to logarithmic domain (w_log) before quantization, which compresses the value distribution and allows for more efficient quantization. This transformation enables storing weights with fewer bits (reducing storage requirements) while the logarithmic relationship ensures that the reconstructed weights maintain sufficient precision for accurate model execution, thus resolving the trade-off between storage efficiency and model accuracy.
Data Source
AI summary
Methods, apparatus, systems and articles of manufacture to optimize execution of a machine learning model are disclosed. An example apparatus includes a quantizer to quantize a layer of a model based on an execution constraint, the layer of the model represented by a matrix. A packer is to pack the quantized layer of the matrix to create a packed layer represented by a packed matrix, the packed matrix having non-zero values of the matrix grouped together along at least one of a row or a column of the matrix. A blocker is to block the packed layer into a blocked layer by dividing the non-zero values in the packed matrix into blocks. A fuser is to fuse the blocked layer into a pipeline. A packager is to package the pipeline into a binary.