Model distillation method, device and electronic equipment

By generalizing the unlabeled data and integrating multiple teachers' network models, determining pseudo-labels and optimizing the student network model with the recognition results, the problem of high labor costs of the existing model distillation method is solved, and efficient model training and cost-reducing effect is achieved.

CN114519416BActive Publication Date: 2025-03-25SHENZHEN INTELLIFUSION TECHNOLOGIES CO LTD
2 Cites 0 Cited by

Patent Information

Application Number
CN202111680113.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-30
Publication Date
2025-03-25
Estimated Expiration
2041-12-30

AI Technical Summary

Technical Problem

Existing model distillation methods rely on a large amount of labeled data, resulting in excessive labor costs.

Method used

By performing a first generalization process on the unlabeled data and inputting it into at least two teacher network models, multiple recognition results are obtained and pseudo-labels are determined; then the second generalization process on the unlabeled data and input it into the student network model, and the student network model is optimized based on the pseudo-labels and identification results.

Benefits of technology

Reduces dependence on labeled data, reduces labor tagging costs, and achieves the same or similar effects as training student network models using labeled data.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The embodiment of the present invention relates to a model distillation method, device and electronic device, which includes: after performing a first generalization process on the i-th unlabeled data, inputting them into at least two teacher network models respectively to obtain a set of first recognition results; obtaining a second recognition result based on the set of first recognition results; when the predicted probability of the first candidate label in at least one candidate label is greater than or equal to a preset probability threshold, setting the first candidate label as a pseudo label of the unlabeled data; after performing a second generalization process on the i-th unlabeled data, inputting them into a student network model to obtain a third recognition result; optimizing the student network model based on the second recognition result, the third recognition result, and the pseudo label until the student network model meets the preset standard. In this way, the cost of human labeling is greatly reduced.
Need to check novelty before this filing date? Find Prior Art