A depth neural network model clipping method based on statistical feature of feature graph

Abstract

The invention discloses a depth neural network model cutting method based on the statistical characteristics of a characteristic map. The method comprises the following steps: step 1, aiming at the characteristic layer in the depth neural network model, calculating the statistical characteristics of the characteristic map corresponding to each output channel; The characteristic layer consists of convolution layer and activation layer, or of convolution layer, normalization layer and activation layer. 2, accord to that statistical characteristics of the characteristic map correspond to each output channel in the characteristic layer, calculating evaluation indexes of each output channel in the characteristic lay; 3, judging that importance of each output channel in the characteristic layeraccord to the evaluation index, and removing the unimportant output channels and the corresponding parameter. The invention can effectively reduce the dimension of the neural network characteristic layer, improve the operation efficiency of the network model, reduce the network scale, and has little influence on the accuracy.

Description

technical field [0001] The invention belongs to the fields of artificial intelligence and pattern recognition, and in particular relates to deep neural network model compression. Background technique [0002] Deep learning (deep learning) has achieved remarkable results in solving advanced abstract cognitive problems, bringing artificial intelligence to a new level, and providing a technical basis for high-precision, multi-type target detection, recognition and tracking. However, complex calculations and huge resource requirements mean that neural networks can only be deployed on high-performance computing platforms, which limits their application on mobile devices. In 2015, Deep Compression, published by Han, applied network model clipping, weight sharing, quantization, and encoding to model compression, making model storage achieve good results, and also aroused researchers' research on network compression methods. The current research on deep learning model compression m...

AI Technical Summary

Problems solved by technology

[0005] (3) The sparseness of the kernel. During the training process, the update of the weight is induced to make it more sparse. For the sparse matrix, a more compact storage method can be used, but the operation efficiency of the sparse matrix operation on the hardware platform is not high. High, easily affected by bandwidth, so the speedup is not obvious

[0008] a. The sparse method of the kernel only considers the compressed storage of the network, the compression effect is not obvious during operation, and the speed is not significantly improved;

[0009] b. Using the size of the weight as the evaluation index only considers the numerical characteristics of the weight itself, and does not consider the data characteristics of the network layer, so the compression effect is not high;

[0010] c. The calculation of the evaluation index is more complex and consumes more computing power;

[0011] d. The method of random cropping is too random, and it is easy to destroy the parameter characteristics of the network itself;

Images