Coding and decoding method based on block cyclic sparse matrix neural network

A neural network and sparse matrix technology, applied in the field of sparse deep neural network compression, can solve problems such as complex encoding and decoding methods, irregular operations, and unbalanced loads, and achieve the effects of reducing storage requirements, improving throughput, and facilitating hardware implementation

Active Publication Date: 2019-01-18
NANJING UNIV
View PDF9 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The current accelerators for sparse neural networks have problems such as unb

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Coding and decoding method based on block cyclic sparse matrix neural network
  • Coding and decoding method based on block cyclic sparse matrix neural network
  • Coding and decoding method based on block cyclic sparse matrix neural network

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0037] The solution of the present invention will be described in detail below in conjunction with the accompanying drawings.

[0038] The encoding and decoding method described in this solution is mainly designed for the fully connected deep neural network, and combines the characteristics of block cyclic matrix and sparse matrix for network compression.

[0039] The calculation formula of the fully connected layer algorithm is as follows:

[0040] y=f(Wa+b) (1)

[0041] Among them, a is the excitation vector of the calculation input, y is the output vector, b is the bias, f is the nonlinear function, and W is the weight matrix.

[0042] The operation of each element value of the output vector y in formula (1) can be expressed as:

[0043]

[0044] In formula (2), i represents the row number of the element, j represents the column number of the element, and n represents the number of input stimuli (the total number of columns of the weight matrix).

[0045] Therefore, t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a coding and decoding method based on a block cyclic sparse matrix neural network, and the method comprises the steps: compressing and coding a fully connected neural networkwith a block cyclic sparse weight matrix; using the cyclic property and sparsity of the weight matrix to design a coding method using a mask matrix and a non-zero value list; using the sparsity of theinput excitation to design a coding method using a mask vector and a non-zero value list. The method makes full use of the characteristics of the mask matrix and the cyclic matrix, and employs a hardware-friendly decoding method. The beneficial effects are that the method performs the simultaneous compressing and coding of a sparse input excitation vector and the weight matrix, effectively reduces the storage space required for the data and the number of storage accesses required to carry the data in the operation process. In the process of neural network operation, the method can effectivelyreduce the energy consumed by the memory access, brings convenience to a processor for skipping unnecessary operations, and improves the throughput of a hardware system.

Description

technical field [0001] The invention relates to a sparse deep neural network compression method, in particular to an encoding and decoding method based on a block cycle sparse matrix neural network. Background technique [0002] Deep neural networks are widely used in the field of artificial intelligence, especially in the field of image recognition, which has achieved the best accuracy so far. A large-scale deep neural network has high computational complexity and contains a large number of operational parameters, so it has high requirements for the operational performance of the processor. Processing deep neural networks in resource-constrained systems such as embedded systems requires high processor power efficiency. The study found that the deep neural network contains a large number of zero elements, and the sparsity of the neural network will be higher after pruning technology. Therefore, using the sparsity of deep neural networks to design special hardware accelerat...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H03M7/30G06N3/02G06F17/16
CPCG06F17/16G06N3/02H03M7/30
Inventor 潘红兵秦子迪朱杏伟孙华庆苏岩朱棣吴加维沈庆宏
Owner NANJING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap