Image classification method based on vector standardization and knowledge distillation

Abstract

The invention discloses an image classification method based on vector standardization and knowledge distillation, and the method comprises the following steps: (1), constructing and training a teacher model which employs a deep convolutional neural network; (2) constructing a student model smaller than the teacher model, wherein the student model also adopts a deep convolutional neural network; (3) training the student model by using a distillation loss function, and standardizing probability coding vectors output by the student model and the teacher model in the training process; and (4) inputting a to-be-classified image into the trained student model, and carrying out classification prediction. By utilizing the method, the performance of the student network can be improved and the image classification precision can be improved under the condition of not introducing extra parameters and calculation overhead.

Description

technical field [0001] The invention belongs to the technical field of image classification, and in particular relates to an image classification method based on vector standardization and knowledge distillation. Background technique [0002] With the advent of the era of artificial intelligence, deep learning technology has been widely used in a variety of image classification fields: such as face recognition, automatic driving, fault detection, etc. [0003] Model compression is one of the hottest issues in deep learning model deployment. It requires the model to maintain a certain accuracy when the parameters are reduced. Currently, the most popular methods include parameter quantization, model pruning, knowledge distillation, etc. Among them, the method of knowledge distillation has a better effect and has received extensive attention. [0004] In knowledge distillation, a larger model (teacher) transfers knowledge to a smaller model (student). On some datasets, know...

AI Technical Summary

Problems solved by technology

[0006] However, knowledge distillation faces a problem known as the "capacity gap"

This describes a problem in the distillation process, where the performance of the student network degrades if the teacher network becomes too large

This leads to the need to manually select the appropriate size of the teacher network problem when performing distillation

In addition, this also makes it impossible to simply use a larger teacher network to enhance the performance of the student network.

Images