Convolution neural network compression method and decompression method based on compressed sensing principle

A convolutional neural network and compression method technology, applied in neural learning methods, biological neural network models, neural architectures, etc., can solve the problem of not considering weight conversion, loss of information, etc., to reduce the impact of accuracy, high compression rate, the effect of preventing information loss

Active Publication Date: 2018-03-23
NANJING UNIV
View PDF3 Cites 29 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, since the calculation of the deep neural network requires the use of high-performance GPU and a large amount of memory, and the current embedded devices cannot provide such high-performance computing power and memory capacity, the demand for neural network compression technology is urgen

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Convolution neural network compression method and decompression method based on compressed sensing principle
  • Convolution neural network compression method and decompression method based on compressed sensing principle
  • Convolution neural network compression method and decompression method based on compressed sensing principle

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0076] The convolutional neural network used in this embodiment is YOLOv2, and the training data is VOC2012.

[0077] The specific compression process is:

[0078] i. The YOLOv2 convolutional neural network has a total of 22 convolutional layers, and the weights of each convolutional layer are divided into 15×15 matrix blocks through the preprocessing process of this method. For example, the weights of the first convolutional layer are 32×3×3=288, which can be divided into two matrix blocks of 15×15, but the data of the second matrix block is less than 225, and the first 63 are used for the vacant positions. The average of the weights is used to complete.

[0079] ii. The preprocessing results are subjected to the compression process of this method, that is, through the steps of DCT transformation, pruning, and dimensionality reduction sampling in sequence. When pruning, the pruning threshold ρ can be manually adjusted, and different ρ values ​​can be set in turn to observe ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a convolution neural network compression method and a decompression method based on a compressed sensing principle. The compression method comprises a preprocessing step in which weights of layers in the convolution neural network are preprocessed to a series of matrix, a compression step in which the preprocessing result obtained in the preprocessing step is compressed toobtain a compressed weight, a training step in which the compressed weight is trained, a coding step in which the compressed weight trained in the training step is coded, and a model generation step in which a compressed convolution neural network model file is generated according to the coding result obtained in the coding step. Compared with other method, the convolution neural network compression method based on the compressed sensing principle has a higher compression ratio than the current popular direct pruning quantization method, and excessive information loss can be prevented throughkeeping low-frequency information in a frequency domain.

Description

technical field [0001] The invention relates to a method for compressing a convolutional neural network model, which belongs to the technical field of deep learning. Background technique [0002] Since the Alexnet convolutional neural network won the first place in the ImageNet image classification competition in 2012, and its accuracy far exceeded the second place, deep learning technology ushered in a boom. In the past 5 years, deep learning technology can be said to have entered all walks of life, including automatic driving, recommendation systems, medical imaging, game AI and other technical fields. Deep learning technology has been used and achieved better performance than before. However, since the calculation of the deep neural network requires high-performance GPU and a large amount of memory, and the current embedded devices cannot provide such high-performance computing power and memory capacity, there is an urgent need for neural network compression technology. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/04G06N3/08
CPCG06N3/08G06N3/045
Inventor 路通孟周宇巫义锐
Owner NANJING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap