Unlock instant, AI-driven research and patent intelligence for your innovation.

Model distillation method and device

A model and student model technology, applied in deep learning, computer vision, and artificial intelligence fields, can solve problems such as slow prediction speed and inability to meet the needs of image processing

Pending Publication Date: 2021-03-19
BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Some models cannot meet the needs of image processing because of their slow prediction speed

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Model distillation method and device
  • Model distillation method and device
  • Model distillation method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0018] Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

[0019] It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0020] figure 1 An exemplary system architecture 100 of an embodiment of t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a model distillation method and device, and relates to the technical field of artificial intelligence, in particular to the technical field of deep learning and computer vision. The method comprises the steps of obtaining batch teacher features corresponding to a teacher model and batch student features corresponding to a student model; determining a teacher similarity setcorresponding to the batch teacher features and a student similarity set corresponding to the batch student features; determining the weight of the loss value of the feature of the image based on thedifference value corresponding to the image; and weighting the loss value of the feature of each image in the batch images, and training a student model by using a weighting result. According to the method, the weight of the loss value can be determined by utilizing the difference value of the feature similarity between the student model and the teacher model, so that the model is accurately distilled. Through the distillation process, the detection capability of the model can be improved, the delay of execution equipment can be reduced, and the occupation and consumption of computing resources such as a memory can be reduced.

Description

technical field [0001] The present application relates to the technical field of artificial intelligence, specifically to the technical field of deep learning and computer vision, especially to the method and device of model distillation. Background technique [0002] With the development of Internet technology, more and more platforms need to use models to predict images. Some models cannot meet the needs of image processing because of their slow prediction speed. [0003] In related technologies, model distillation can be used to supervise the training process of the student model through a trained teacher model. The teacher model usually has some kind of predictive ability, such as a strong predictive ability for a certain target. For example, it can be the ability to detect human faces, or the ability to detect special shapes. Contents of the invention [0004] Provided are a model distillation method, device, electronic equipment and storage medium. [0005] Accor...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/08G06V10/764
CPCG06N3/08G06V10/454G06V10/82G06V10/761G06V10/764G06N3/045G06F18/22G06F17/16G06F18/217
Inventor 杨馥魁温圣召韩钧宇
Owner BEIJING BAIDU NETCOM SCI & TECH CO LTD