Convolutional neural network channel self-selection compression and acceleration method based on knowledge migration

A convolutional neural network and neural network technology, applied in the field of convolutional neural network channel self-selection compression and acceleration, can solve the problem of capacity reduction and achieve high compression ratio and acceleration ratio

Inactive Publication Date: 2019-07-09
SOUTH CHINA UNIV OF TECH
View PDF1 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the pruning of the network is carried out on the already trained network, and it will not be restored after pruning, so the problem of capacity reduction still exists

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Convolutional neural network channel self-selection compression and acceleration method based on knowledge migration
  • Convolutional neural network channel self-selection compression and acceleration method based on knowledge migration
  • Convolutional neural network channel self-selection compression and acceleration method based on knowledge migration

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0028] The present invention will be further described in detail below with reference to the drawings and embodiments, but the implementation of the present invention is not limited to this.

[0029] Such as figure 1 As shown, the method for self-selected compression and acceleration of convolutional neural network channels based on knowledge transfer provided by this embodiment includes the following steps:

[0030] S1. Extract the migration guidance knowledge from the trained complex convolutional neural network CN1 with good enough performance.

[0031] The migration guidance knowledge can be extracted from different locations in the network. In this embodiment, the ResNet56 network is used as the complex convolutional neural network CN1, such as figure 2 As shown, the residual module in the figure includes two convolutional layers of 3*3 kernel size, and in the first residual module of each stage, the step size of the first convolutional layer is 2, use For dimensionality reduct...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a convolutional neural network channel self-selection compression and acceleration method based on knowledge migration. The method comprises the following steps: S1, extractingmigration guidance knowledge from a trained complex convolutional neural network CN1; S2, setting randomly initialized coefficient masks for all convolutional layer channels of the target convolutional neural network ON1; S3, setting the output of each channel of the target network as a product of the original output of the channel and the corresponding coefficient mask; and S4, optimizing the target network under the guidance of the migration guidance knowledge, carrying out channel soft cutting according to the coefficient mask, and carrying out channel hard cutting until the target networkconverges. According to the method, the network can automatically select the channel to be cut off, manual selection operation is omitted, the expression capacity of the network is reserved and the generalization performance is improved through soft cut-off operation and a knowledge migration method, so that a higher compression ratio and a higher acceleration ratio are realized.

Description

technical field [0001] The invention relates to the technical field of deep learning and computer vision algorithms, in particular to a method for self-selecting compression and acceleration of convolutional neural network channels based on knowledge transfer. Background technique [0002] In recent years, the development of artificial intelligence has attracted more and more attention from all walks of life, especially in the image field such as computer vision, which has attracted the favor of scholars and industrial producers. Driven by the wave of artificial intelligence, visual recognition tasks such as face recognition and target positioning have achieved remarkable results, and the achievement of these results is inseparable from the innovation and development of deep convolutional neural network algorithms. However, the existing deep convolutional neural network models are very large in terms of storage and calculation, which hinders the application of this model on ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/08G06N3/04
CPCG06N3/082G06N3/045
Inventor 郭礼华陈达武
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap