Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Neural network training method and device and computer readable storage medium

A neural network training and neural network technology, applied in the fields of electronic equipment and computer-readable storage media, neural network training methods and devices, can solve the problems of many teacher model resources, long required time, long training time, etc. Training accuracy, high accuracy, and the effect of improving training speed

Pending Publication Date: 2020-01-17
MEGVII BEIJINGTECH CO LTD
View PDF0 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] At present, in the way of training the neural network through the distillation method, the teacher model requires many resources and takes a long time to train.
Moreover, it cannot be optimized uniformly. First, there must be a teacher model, and then distillation, time series, takes a long time

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Neural network training method and device and computer readable storage medium
  • Neural network training method and device and computer readable storage medium
  • Neural network training method and device and computer readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] The principle and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are given only to enable those skilled in the art to better understand and implement the present disclosure, rather than to limit the scope of the present disclosure in any way.

[0029] It should be noted that although expressions such as "first" and "second" are used herein to describe different modules, steps, data, etc. of the embodiments of the present disclosure, expressions such as "first" and "second" are only for A distinction is made between different modules, steps, data, etc., without implying a particular order or degree of importance. In fact, expressions such as "first" and "second" can be used interchangeably.

[0030] Currently, the teacher model is used to train the student model. Because the teacher model requires a lot of resources and takes a long time to train, the training cost ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a neural network training method and device, and the method comprises the steps: obtaining a plurality of feature maps outputted by a plurality of intermediate layers of a to-be-trained neural network; performing feature extraction on the plurality of feature maps through a feature extraction network to obtain first feature output of each intermediate layer; according to theplurality of first feature outputs and second feature outputs output by the to-be-trained neural network, calculating to obtain a first loss; based on the first loss, parameters of the plurality of intermediate layers are adjusted. Through a self-distillation mode, each intermediate layer of the neural network model and result features extracted in each iteration are output to serve as self supervision signals to be fully utilized, a result can be converged more quickly, training is completed, and time and resources are saved.

Description

technical field [0001] The present disclosure generally relates to the field of artificial intelligence, and specifically relates to a neural network training method and device, electronic equipment, and a computer-readable storage medium. Background technique [0002] With the rise of deep learning in recent years, people have achieved excellent results in many fields such as image classification, speech recognition, natural language processing, policy AI, and automatic driving. However, relying on complex neural networks and large data sets to achieve good results is based on powerful computing power. With the deepening of the neural network layers and the continuous expansion of the data set, the computational requirements for training the neural network and the trial and error cost of adjusting parameters are also getting higher and higher, which is a huge time cost for neural network training. [0003] At present, in the way of training the neural network through the d...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06N3/04G06K9/62
CPCG06N3/045G06F18/214
Inventor 李亮亮
Owner MEGVII BEIJINGTECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com