Method and system for compressing sparse weight matrix of convolutional neural network full connection layer

A convolutional neural network and weight matrix technology, applied in neural learning methods, biological neural network models, neural architectures, etc., can solve the problem that the compression rate of sparse weight matrix is ​​not very ideal, and achieve the effect of high compression rate

Active Publication Date: 2019-07-26
XI AN JIAOTONG UNIV
View PDF6 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The compression ratio of these methods fo

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for compressing sparse weight matrix of convolutional neural network full connection layer
  • Method and system for compressing sparse weight matrix of convolutional neural network full connection layer
  • Method and system for compressing sparse weight matrix of convolutional neural network full connection layer

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0053] see figure 1 , figure 1 Shown is a schematic diagram of the entire fully connected layer sparse weight matrix compression optimization principle. A method for compressing and optimizing a sparse weight matrix of a fully connected layer according to an embodiment of the present invention includes two steps: lossless compression and lossy compression.

[0054] (1) The lossless compression steps are as follows: After unstructured pruning and retraining, the weight matrix of the fully connected layer is a large sparse matrix A. Decompose a large sparse matrix A into a position matrix B and a non-zero value array C; where the size of the position matrix B is the same as the sparse weight matrix A; numerically, the corresponding position of the non-zero value in matrix B in matrix A The value is 1, and the value of other positions is 0.

[0055] In this embodiment, the convolutional neural network model selected for the specific experiment is CaffeNet, and the experimental...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method and a system for compressing a sparse weight matrix of a convolutional neural network full connection layer, and the method comprises the following steps: 1, obtaininga full connection layer sparse weight matrix A of a convolutional neural network to be optimized, and decomposing the weight matrix A into a position matrix B and a non-zero value array C; wherein the size of the position matrix B is consistent with that of the sparse weight matrix A; in the position matrix B, the numerical value of the corresponding position with the non-zero value in the sparseweight matrix A is 1, and the numerical values of other positions are 0; and step 2, taking the position matrix B obtained in the step 1 as a binary matrix, and carrying out lossless compression meeting a preset image compression standard. According to the method, the compression rate of the sparse weight matrix of the full connection layer can be improved, and more storage space is saved.

Description

technical field [0001] The invention belongs to the field of computer artificial intelligence, the field of deep neural network optimization technology and the field of picture recognition technology, and in particular relates to a method and system for compressing a sparse weight matrix of a fully connected layer of a convolutional neural network. Background technique [0002] In the field of artificial intelligence, deep neural network is one of the cornerstones, and its complexity and portability directly affect the application of artificial intelligence in life. Research on the acceleration and compression optimization of deep networks can make artificial intelligence more convenient to realize and serve life more conveniently. [0003] At present, a sparse matrix compression method CSR (Compressed sparse row) or CSC (Compressed sparse column) is often used to obtain a sparse weight matrix after unstructured pruning of the fully connected layer of the convolutional neura...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04N19/129H04N19/42G06N3/08G06N3/04
CPCH04N19/42H04N19/129G06N3/082G06N3/045G06N3/044
Inventor 梅魁志张良王晓张增赵英海常蕃张向楠鄢健宇
Owner XI AN JIAOTONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap