Partial Quantization for Edge Machine Learning Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models on edge devices require significant computing resources and internet connectivity for training and quantization, leading to security concerns and limited operational flexibility due to the need for cloud-based processing.
Innovation Solution
Implementing partial quantization directly on edge devices to reduce model complexity, allowing for training and quantization without a host machine or cloud connection, using affine mapping to convert floating-point numbers to integers, and facilitating transfer learning by modifying only a subset of layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning models are trained and quantized on portable devices, then data security and operational flexibility are improved, but computing power and storage requirements exceed device capabilities
Solution Approach 1:
The patent segments the machine learning model into multiple layers and applies quantization selectively to different layers. Some layers are quantized while others remain in floating-point format, allowing the model to be processed on edge devices with limited resources while maintaining accuracy. This segmentation enables training and quantization to occur locally without requiring cloud computing resources.
Solution Approach 2:
The patent changes the numerical precision parameter of model weights and activations from full floating-point to quantized formats (e.g., INT8, INT4). This parameter change reduces the computational complexity and memory requirements, enabling the model to run on portable devices while maintaining acceptable accuracy. The quantization process transforms high-precision parameters into lower-precision representations that fit within device constraints.
2Device complexity
If full quantization is applied to reduce model complexity, then memory storage and computing requirements are reduced, but model accuracy and performance deteriorate
Solution Approach 1:
The patent applies different quantization qualities to different parts of the model. Critical layers that require high precision are kept in floating-point or use higher precision quantization, while less sensitive layers are quantized to lower precision. This local quality approach ensures that model accuracy is maintained in important regions while reducing overall complexity in less critical areas.
Solution Approach 2:
The patent applies partial quantization rather than full quantization across all layers. By selectively quantizing only certain layers and keeping others in higher precision, the patent achieves a balance between model complexity reduction and accuracy preservation. This partial action approach avoids the excessive quantization that would degrade performance while still achieving the benefits of reduced complexity.
3Power
If cloud-based processing is used for model training and quantization, then computational capabilities are sufficient, but internet connectivity is required and security risks increase
Solution Approach 1:
The patent enables the portable device to perform model training and quantization operations independently without requiring external cloud services. The device uses its own computational resources, augmented by efficient algorithms, to execute the full machine learning pipeline locally. This self-service capability eliminates the need for internet connectivity and cloud dependencies, providing operational flexibility and security.
Data Source
AI summary
A method for partial quantization to achieve full quantized model includes quantizing a plurality of weights and a respective activation function from each of a plurality of respective layers of an original Machine Learning Model (MLM) to generate a quantized MLM comprising a plurality of frozen quantized weights. The plurality of frozen quantized weights are extracted from at least one frozen layer of the layers of the quantized MLM. The plurality of weights are quantized from at least one updated layer of an updated MLM to generate a plurality of updated quantized weights. The respective activation function of the at least one updated layer of the updated MLM is quantized from a difference between the original MLM and the updated MLM, to form a respective quantized activation function. A new quantized MLM is generated from the frozen quantized weights, the updated quantized weights and the respective quantized activation function.