A model compression method, device and readable storage medium
By employing a joint optimization method combining model quantization and knowledge distillation, the challenge of deploying deep neural network models on resource-constrained devices is addressed, achieving high-precision model compression applicable to tasks such as image classification, object detection, and speech recognition.
CN118586449BActive Publication Date: 2026-07-10SHENZHEN MICROBT ELECTRONICS TECH CO LTD
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN MICROBT ELECTRONICS TECH CO LTD
- Filing Date
- 2023-03-03
- Publication Date
- 2026-07-10
AI Technical Summary
Technical Problem
The large number of parameters and computational demands of deep neural network models make them difficult to deploy on hardware devices with limited resources, and the accuracy of compressed models is hard to guarantee.
Method used
By employing a joint optimization method of model quantization and knowledge distillation, the student network model is first quantized, and then the knowledge transfer from the teacher network model is used to optimize the parameters of the quantized student network model to improve accuracy.
Benefits of technology
While compressing the model, maintain or improve its accuracy so that it can be deployed on resource-constrained hardware devices, reducing the consumption of computing and storage resources.
✦ Generated by Eureka AI based on patent content.
Smart Images
Figure CN118586449B_ABST
Abstract
Embodiments of the present application provide a model compression method, device and readable storage medium. The method comprises: obtaining a pre-trained teacher network model and a student network model, the teacher network model and the student network model being floating point models and having the same model type; performing model quantization on the student network model to obtain a quantized student network model; inputting training data in a training set into the teacher network model and the quantized student network model in sequence respectively, adjusting parameters of the quantized student network model according to a preset loss function, and obtaining a target network model when an iteration stop condition is met; the preset loss function comprises a first distillation loss function, and the first distillation loss function is determined according to features output by each corresponding segment in the quantized student network model and the teacher network model. Embodiments of the present application can improve the accuracy of the model while compressing the model.
Need to check novelty before this filing date? Find Prior Art