Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Neural network compression method using block cyclic matrix

A technology of neural network and circulant matrix, which is applied in the field of neural network compression using block circulant matrix, can solve problems such as hardware matching and projection correlation performance degradation, and achieve the effects of ensuring accuracy, increasing convergence speed, and improving compression performance

Inactive Publication Date: 2020-04-21
NANCHANG UNIV
View PDF3 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] However, the schemes of the above three disclosed invention patents cannot fully match the hardware, and at the same time, for the block circulant matrix, the weight tensor is composed of the block circulant matrix; the projection correlation leads to serious performance degradation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Neural network compression method using block cyclic matrix
  • Neural network compression method using block cyclic matrix
  • Neural network compression method using block cyclic matrix

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0074] Embodiment 1: VGG16 experiment

[0075] When the sign vector is generated, n_sign=block size, n_sign=32, block_sign=25. Among them, the expansion rule is: cyclically shift block_sign-1 times to the right from the basic symbol vector, and shift right by 1 bit each time, and splice the obtained structure into an 800-dimensional symbol vector.

[0076] The penultimate fully connected layer of the VGG16 network is replaced by the recurrent neural network layer proposed in this scheme, and the convolutional layer of the network is pruned by a fine-grained compression method.

[0077] The above experiments were performed on the large data set ImageNet2012, and the compression rate and Top-1 Acc were calculated. The experimental results are shown in Table 1, Table 2 and Table 3 below.

[0078] The results show that compared with the uncompressed VGG16 network, the number of network parameters and complexity is greatly reduced at the expense of little accuracy performance.

Embodiment 2

[0079] Example 2: ResNet50 experiment

[0080] When the sign vector is generated, n_sign=40, block_sign=20. Among them, the expansion rule is: repeat the basic symbol vector 20 times, and concatenate it into an 800-dimensional symbol vector.

[0081] Replace all the fully connected layers of ResNet with the recurrent neural network layer proposed in this scheme, and use the fine-grained compression method to prune the convolutional layer of the network.

[0082] The above experiments were performed on the large data set ImageNet2012, and the compression rate and Top-1 Acc were calculated. The experimental results are shown in Table 1, Table 2 and Table 3 below.

[0083] The results show that compared with the uncompressed ResNet50 network, the number of network parameters and complexity is greatly reduced at the expense of little accuracy performance.

[0084] Table 1 Top-1Acc (%) of the network

[0085]

[0086] Table 2 Parameter compression ratio of the network

[00...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a neural network compression method using a block cyclic matrix. The invention relates to the field of neural network compression. The method comprises the following steps: reading a longest basic random symbol vector in a neural network, generating a symbol vector equal to an input dimension of the layer on each layer of the neural network, multiplying the symbol vector byan input vector element to obtain a new input vector, training to form a new block cycle network, and storing the longest basic random symbol vector and neural network model parameters; and pruning the model by adopting a fine-grained neural network compression method, and further reducing the complexity of the model. By introducing the random symbol vectors, the correlation between the projection vectors is reduced, so that the convergence of the model is ensured, and the purpose of effectively reducing storage and bandwidth is achieved. And meanwhile, the problem of performance reduction caused by increase of the block size when the block cyclic matrix compression neural network is processed is avoided.

Description

technical field [0001] The invention relates to the field of neural network compression methods, in particular to a neural network compression method using a block cyclic matrix. Background technique [0002] In recent years, deep neural networks have made great progress, and many algorithms can be calculated in real time on a graphics processing unit (GPU), and have achieved great success in computer vision, natural language processing and other fields. How to compress the recurrent neural network and reduce its parameter quantity and complexity without significantly affecting the performance of the recurrent neural network has become a research hotspot in academia and industry. [0003] Chinese invention patent (publication number: CN109389221A) discloses a neural network compression method based on model calculation. the number of parameters. Quantizing the neural network model can reduce the storage space occupied by the internal weights of the model and improve the ca...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06N3/08G06N3/04
CPCG06N3/082G06N3/045G06N3/044
Inventor 杨宇钢胡凌燕
Owner NANCHANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com