The application discloses an adversarial knowledge
distillation method and
system based on feature semantic guidance and feature
orthogonalization, which firstly acquires an image classification dataset, extracts text features, and constructs a
semantic feature space.Secondly, a knowledge
distillation framework including a teacher model, a student model and a projection head is constructed, the input of the teacher model and the student model is an image in the dataset, and the output is an image category prediction result. Then, the student
model parameters are frozen, the projection head is updated through
momentum tracking, in the feature level semantic adversarial sample generation stage, all network parameters are frozen, and normalized disturbance is constructed to obtain adversarial features. Finally, in the student model multi-objective joint optimization stage, the teacher model and the two projection heads are fixed, the student
model parameters are updated, the classification loss, the
distillation loss, the consistency loss and the
orthogonalization loss are introduced, and optimization is completed. The application improves the fine-grained discrimination and generalization ability of the student
model image classification, and does not introduce additional reasoning overhead.