Processing method and device based on optimized neural network, and electronic system

A technology of neural network and processing method, applied in the field of neural network processing method, device and electronic system, capable of solving the problem of high delay of convolutional neural network

Inactive Publication Date: 2020-02-28
MEGVII BEIJINGTECH CO LTD
View PDF0 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, most network pruning only focuses on the amount of parameters and calculations, which leads to the problem of high delay in the convolutional neural network after network pruning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Processing method and device based on optimized neural network, and electronic system
  • Processing method and device based on optimized neural network, and electronic system
  • Processing method and device based on optimized neural network, and electronic system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0030] First, refer to figure 1 An example electronic system 100 for implementing the optimized neural network-based processing method, apparatus and electronic system of the embodiments of the present invention will be described.

[0031] like figure 1A schematic structural diagram of an electronic system is shown, the electronic system 100 includes one or more processing devices 102, one or more storage devices 104, input devices 106, output devices 108 and one or more data acquisition devices 110, these components The interconnections are via bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that figure 1 The components and structures of the electronic system 100 shown are exemplary rather than limiting, and the electronic system may also have other components and structures as required.

[0032] The processing device 102 may be a smart terminal, or a device including a central processing unit (CPU) or other forms of processing un...

Embodiment 2

[0039] This embodiment provides a processing method based on an optimized neural network, which is executed by the processing device in the electronic system; the processing device may be any device or chip capable of data processing. like figure 2 A flow chart of a processing method based on an optimized neural network is shown, and the processing method based on an optimized neural network includes the following steps:

[0040] Step S202, obtain the convolution kernel used by each convolution layer in each training of the convolutional neural network, and obtain the cross entropy loss and delay loss corresponding to each training according to the convolution kernel used by each convolution layer, cross entropy The loss and delay loss correspond to the convolution layer identification vector for each training; where each element in the above convolution layer identification vector is the number of convolution kernels used by the corresponding convolution layer.

[0041] Con...

Embodiment 3

[0052] This embodiment provides another processing method based on an optimized neural network, which is implemented on the basis of the above-mentioned embodiments; this embodiment focuses on obtaining the volume used by each convolutional layer in each training of the convolutional neural network The product kernel, according to the convolution kernel used by each convolution layer, obtains the cross-entropy loss and delay loss corresponding to each training, and the specific implementation method of the cross-entropy loss and delay loss corresponding to the convolution layer identification vector of each training . like image 3 The flow chart of another processing method based on the optimized neural network is shown, and the processing method based on the optimized neural network in this embodiment includes the following steps:

[0053] Step S302, obtaining the identification of the convolution kernel used by each convolution layer in each training of the convolutional n...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a processing method and device based on an optimized neural network and an electronic system. The method comprises the steps of acquiring a convolution kernel used by each convolution layer in each time of training of the convolutional neural network, acquiring cross entropy loss and delay loss corresponding to each time of training according to the convolution kernel used by each convolution layer, wherein the cross entropy loss and the delay loss correspond to a convolution layer identification vector of each time of training; calculating an overall loss value corresponding to each training based on a preset loss function and the cross entropy loss and delay loss corresponding to the convolutional layer identification vector; and screening out a convolution kernelfinally used by each convolution layer according to the overall loss value corresponding to each training. According to the method, the overall loss value of the convolutional neural network is comprehensively considered based on the cross entropy loss and the delay loss corresponding to each time of training so as to perform dual-objective optimization on the accuracy and the delay of the convolutional neural network, and the convolutional neural network which meets the delay requirement and is relatively high in accuracy can be screened out.

Description

technical field [0001] The present invention relates to the technical field of convolutional neural networks, in particular to a processing method, device and electronic system based on an optimized neural network. Background technique [0002] With the rapid development of convolutional neural networks, the parameters of convolutional neural network models usually increase to the order of millions, tens of millions or hundreds of millions, and the occupied space exceeds the storage devices of various current mobile terminals. Therefore, the convolutional neural network model has extremely high requirements on computers and storage devices, which exceeds the computing limits of various current mobile terminal devices, and limits the application of convolutional neural network models on mobile terminal devices. [0003] In related technologies, the network pruning technology is generally used to screen the redundancy in the model without affecting the accuracy of the convolut...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/04
CPCG06N3/045
Inventor 李运
Owner MEGVII BEIJINGTECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap