A boundary sample data enhancement method and device for knowledge distillation
By iteratively modifying samples according to the decision boundary of the teacher model during the knowledge distillation process, boundary samples suitable for knowledge distillation are generated, which solves the problem of insufficient sample expansion in the knowledge distillation of existing data augmentation methods and improves the classification accuracy and knowledge transfer efficiency of the student model.
CN114219042BActive Publication Date: 2026-07-17HARBIN INST OF TECH SHENZHEN GRADUATE SCHOOL
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN INST OF TECH SHENZHEN GRADUATE SCHOOL
- Filing Date
- 2021-12-21
- Publication Date
- 2026-07-17
AI Technical Summary
Technical Problem
Existing data augmentation methods cannot effectively expand suitable samples during knowledge distillation, resulting in poor training performance of student models.
Method used
By iteratively modifying samples based on the decision boundary of the teacher model, boundary samples suitable for knowledge distillation are generated. The DeepFool algorithm is used for adversarial attacks, and the sample that is furthest away from other samples is selected as the base sample for the next round of iteration.
Benefits of technology
It improves the classification accuracy of student models, fully utilizes the knowledge transfer efficiency of teacher models, and is suitable for model deployment on devices with weak computing power.
✦ Generated by Eureka AI based on patent content.
Smart Images
Figure CN114219042B_ABST
Abstract
The application discloses a boundary sample data enhancement method and device for knowledge distillation and a computer storage medium. The method comprises the following steps: before knowledge distillation is performed, the output of a teacher model is used to modify samples in each original data set along the decision boundary of the teacher model step by step, and a plurality of boundary samples suitable for knowledge distillation are expanded. In each iteration, the original sample or each sample modified in the last iteration is used as a basic sample, the approximate tangent plane of the decision boundary near the sample is calculated by using the output of the teacher model, and the sample is modified along multiple directions on the tangent plane; then, the modified sample is modified to be located near the boundary; finally, a plurality of samples farthest from other basic samples are selected as the result of the modification in the round and the basic samples for the next iteration. The application can meet the demand for data enhancement in current image classifier knowledge distillation.
Need to check novelty before this filing date? Find Prior Art