Cross-Platform Model Quantization for Faster AI Deployment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current model deployment technologies are time-consuming, error-prone, and face challenges in cross-platform deployment due to significant performance and resource disparities among devices, requiring manual adjustments that are inefficient and prone to errors.
Innovation Solution
A model optimization method involving automated strategies such as model weight adjustment, dynamic quantization, operator lists, and resource assessment-based optimization to optimize model performance across different devices, including transformation functions, dynamic quantization, and lightweight deployment through operator lists and parameter fusion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If manual adjustment and optimization of models is performed during deployment, then model performance can be optimized for specific devices, but the process becomes time-consuming and error-prone
Solution Approach 1:
The patent applies preliminary action by pre-processing the training dataset with transformation functions that simulate target device characteristics before model training. This allows the model to be pre-optimized for specific device types, eliminating the need for manual post-deployment adjustments and reducing deployment time while maintaining performance optimization.
Solution Approach 2:
The system implements self-service through automated model optimization that performs dataset transformation and model retraining based on target device specifications without human intervention. The automated pipeline selects appropriate transformation functions, generates transformed datasets, and retrains models automatically, making the process error-free and time-efficient.
2Ease of operation
If traditional model deployment methods are used across different devices, then deployment simplicity is maintained, but performance and resource efficiency degrade due to device heterogeneity
Solution Approach 1:
The patent applies local quality by applying device-specific transformation functions to training data based on the characteristics of target devices. Different transformation functions are applied locally according to device type (e.g., mobile, embedded, server), allowing each model instance to be optimized for its specific deployment environment while maintaining a unified deployment process.
3Use of energy by moving object
If model quantization is applied to reduce resource consumption, then resource efficiency improves, but model precision may deteriorate
Solution Approach 1:
The patent applies preliminary action by performing quantization-aware training during the model optimization phase. The transformation functions and quantization parameters are predetermined based on target device characteristics, allowing the model to be trained with quantization constraints already in place. This preliminary quantization preparation ensures that when the model is deployed with reduced precision for lower power consumption, the accuracy loss is minimized because the model has already adapted to the quantized representation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure provides a model optimization method capable of implementing cross-platform intelligent model deployment. A model optimization method executed on a first device may include: performing training using a first training dataset to obtain a first model; transforming the first training dataset with a 5 transformation function to obtain a transformed dataset, and generating a second training dataset based on the first training dataset and the transformed dataset; training the first model using the second training dataset to obtain an adjusted first model; performing performance tests on the first model and the adjusted first model using a test dataset to respectively obtain a first performance metric and a second performance metric, and calculating a performance metric difference between the first and second performance metrics; generating an adjusted second 10 training dataset based on the performance metric difference, which is configured to train the adjusted first model to obtain a second model; and performing dynamic quantization on the second model to obtain an optimized model deployable to a second device.