Method and device for detecting knowledge migration of network

Abstract

The invention discloses a method and device for detecting knowledge migration of the network. A first neural network model is monitored through a training sample for training, firstly, according to the overlapping degree of a target window of the first neural network and a marking window, the effective classification and detection knowledge is respectively extracted, a small model is made to learn, and monitoring of the small model on a training set is jointed to carry out joint training. A technical problem of large precision loss of the small model because of utilizing the small model to carry out knowledge migration for characteristic graphs of a large model or a target integrity in the prior art is solved.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a knowledge transfer method and device for a detection network. Background technique [0002] The knowledge transfer method of the detection network is to transfer the characteristics of the large model of the neural network to the small model, so that the small model can learn the characteristics of the large model. At the same time, it has the characteristics of small size and fast speed, and can be used in embedded low-end devices. This technology can be widely used in target detection tasks such as faces, irises, and palms, and has a wide range of applications, such as attendance machines, face recognition SDKs, and face capture cameras and other products and industries. At present, this technology is mainly proposed by Hinton KnowledgeDistill, FitNets proposed by Romero, Attention Transfer proposed by Sergey, among them, KnowledgeDistill is to let the small model want to learn...

AI Technical Summary

Problems solved by technology

This technology can be widely used in target detection tasks such as faces, irises, and palms, and has a wide range of applications, such as attendance machines, face recognition SDKs, and face capture cameras and other products and industries. At present, this technology is mainly proposed by Hinton KnowledgeDistill, FitNets proposed by Romero, Attention Transfer proposed by Sergey, among them, KnowledgeDistill is to let the small model want to learn the probability distribution of the output of the large model, instead of learning the one-hot label in the training set; FitNets not only learns the final result of the large model, but also allows The small model learns the middle layer of the large model, which is the so-called hint training; Attention Transfer transfers the attention map of the large model to the small network. KnowledgeDistill, FitNets and Attention Transfer use the small model to the feature map or target of the large model. In the overall knowledge transfer, there is a technical problem of large loss of precision for small models

Images