Deep convolutional neural network compression method based on Tucker algorithm

A deep convolution and neural network technology, applied in the field of deep neural network, can solve the problems of difficult to compress a large number of parameters and huge amount of calculation, and achieve the effect of reducing compression time and system overhead, avoiding convolution kernels, and high compression multiples.

Inactive Publication Date: 2019-11-08
UNIV OF ELECTRONIC SCI & TECH OF CHINA
View PDF0 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The typical size of the convolution kernel of the hidden layer is 3×3, so the decomposition of this dim

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Deep convolutional neural network compression method based on Tucker algorithm
  • Deep convolutional neural network compression method based on Tucker algorithm
  • Deep convolutional neural network compression method based on Tucker algorithm

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0060] Such as figure 1 Shown the present invention is based on the depth convolutional neural network compression method of Tucker algorithm, comprises:

[0061] A. Obtain a deep convolutional neural network model, the deep convolutional neural network model can be deep convolutional neural network models such as AlexNet, VGG or ResNet, adopt the ResNet deep convolutional neural network model in the present embodiment, as figure 2 As shown, the ResNet deep convolutional neural network model includes a convolutional layer and a fully connected layer.

[0062] B. use the EVBMF algorithm (empirical variational Bayesian matrix decomposition, Empirical Variational BayesMatrix Factorization) to estimate the decomposition rank of each hidden layer parameter in the described depth convolutional neural network model successively, specifically:

[0063] B1. Traverse each convolutional layer of the deep convolutional neural network model in turn, and extract the kernel parameter K∈R o...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a deep convolutional neural network compression method based on a Tucker algorithm. The method comprises the steps of A, obtaining a deep convolutional neural network model; B, sequentially estimating a decomposition rank of each hidden layer parameter in the deep convolutional neural network model by using an EVBMF algorithm; C, according to the parameter tensor of the hidden layer in the deep convolutional neural network model and the corresponding decomposition rank, sequentially decomposing each parameter tensor through a Tucker algorithm to generate a plurality oflow-rank sub-tensors; and D, generating a new hidden layer through the low-rank sub-tensor, and replacing the original hidden layer in the deep convolutional neural network model with the new hiddenlayer to generate a new deep convolutional neural network model. According to the deep convolutional neural network compression method based on the Tucker algorithm, the compression multiple can be greatly improved, and the compression time and the system overhead are effectively reduced.

Description

technical field [0001] The invention relates to the technical field of deep neural networks, in particular to a deep convolutional neural network compression method based on the Tucker algorithm. Background technique [0002] In recent years, deep learning technology represented by convolutional neural network has achieved remarkable results in natural language processing, autonomous driving, object tracking and other fields. Thanks to the readily available datasets and image processors (GPUs) with ever-increasing computing performance in the information age, deep learning techniques have achieved performance beyond traditional computer vision techniques. In order to learn more generalized features from existing data, convolutional neural networks are designed to become more and more complex, and the number of model layers, number of parameters, memory usage, and hard disk storage of the network also increase. Therefore, it is difficult to implement deep learning without hi...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/04
CPCG06N3/045
Inventor 袁国慧贺晨王卓然彭真明曲超范文澜赵浩浩张鹏年赵学功王慧何艳敏蒲恬周宇
Owner UNIV OF ELECTRONIC SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap