Pattern-based convolutional neural network pruning method and pattern sensing accelerator

A convolutional neural network and pattern technology, applied in the field of convolutional neural networks, can solve problems such as inability to accelerate computation, insufficient to maintain model accuracy, and architecture that cannot skip redundant computations.

Active Publication Date: 2020-07-03
INST FOR INTERDISCIPLINARY INFORMATION CORE TECH XIAN CO LTD
View PDF5 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This approach is not sufficient to maintain the accuracy of the entire model, since error propagation through the layers is ignored
Furthermore, experiments are limited to small neural networks and datasets, such that the resulting architectures cannot skip redundant computations and thus cannot accelerate computations

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Pattern-based convolutional neural network pruning method and pattern sensing accelerator
  • Pattern-based convolutional neural network pruning method and pattern sensing accelerator
  • Pattern-based convolutional neural network pruning method and pattern sensing accelerator

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0082] The present invention will be further described in detail below in conjunction with specific embodiments, which are explanations of the present invention rather than limitations.

[0083] In the traditional way, the filter is filled with K values, so that the data sequence of K is used to describe the filter. However, when simply employing pruning methods, filters often exhibit sparsity, so we can exploit sparsity masks to represent non-zero locations.

[0084] With the help of the sparsity mask, only the non-zero parts are required, such as Figure 3a The raw representation of the filter shown at the top, and the pattern-based one shown at the bottom using the pattern mask and sequence of non-zeros. Encoding the sparsity mask has a small overhead. Taking the most commonly used 3×3 filter as an example, since at most A pattern type, 9-bit word length code can represent all possibilities. To obtain regular pruning, using the same length of non-zero sequences is bene...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a pattern-based convolutional neural network pruning method and a pattern sensing accelerator. Firstly, a pattern-based convolutional neural network (PCNN) filter is provided,regular pruning in the filter is realized based on a multi-knapsack framework, and the method can be orthogonal to an existing pruning method, is similar to fine-grained sparsity, but still keeps certain regularity to a certain extent. In each layer, the number of non-zeros in the filter is limited to be the same, so that the calculation workloads of different convolution windows can be the same.Based on the uniform sparsity ratio in the filter, the pattern type in one layer is very limited. The filters are described with pattern masks with very few bits and corresponding non-zero sequences rather than all values, thereby significantly reducing memory size and computational complexity, and being very friendly to both accuracy and hardware. The PCNN pruning method can be orthogonal to other pruning methods such as filter pruning, the PCNN pruning method can realize nine-time weight pruning, and the accuracy loss can be ignored.

Description

technical field [0001] The invention relates to a convolutional neural network method and hardware optimization, in particular to a pattern-based convolutional neural network pruning method and a pattern-aware accelerator. Background technique [0002] Neural network is one of the most important algorithms in the field of artificial intelligence at present, and it can be widely used in fields such as face recognition, self-driving and health care. Traditionally, a neural network consists of three distinct layers, including a convolutional layer, a pooling layer, and a fully connected layer, which implement feature extraction, downsampling, and classification functions, respectively. With the development of network architectures, convolutional layers have become the main bottleneck of computation and storage. Therefore, in order to bring artificial intelligence to various fields with lower overhead, it is urgent to solve a large number of calculations and parameters. [000...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06F9/50G06N3/04G06N3/063G06N3/08
CPCG06F9/5016G06N3/063G06N3/082G06V40/172G06N3/045Y02D10/00
Inventor 马恺声谭展宏
Owner INST FOR INTERDISCIPLINARY INFORMATION CORE TECH XIAN CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products