Method for batch normalization layer pruning in deep neural networks

Abstract

A method of pruning a batch normalization layer from a pre-trained deep neural network model is proposed. The pre-trained deep neural network model is inputted as a candidate model. The candidate model is pruned by removing the at least one batch normalization layer from the candidate model to form a pruned candidate model only when the at least one batch normalization layer is connected to and adjacent to a corresponding linear operation layer. Weights of the corresponding linear operation layer are adjusted to compensate for the removal of the at least one batch normalization. The pruned candidate model is then output and utilized for inference. In the method, by pruning batch normalization layers connecting with a linear operation layer in a pre-trained DNN, the size of the DNN is reduced and implementation requirements are also reduced. Model inference can be achieved with much increased speed and much decreased computational requirements while guaranteeing accuracy during inference.

Description

technical field [0001] The present invention relates to a deep neural network with at least one batch normalization layer, and more particularly to a method for pruning a batch normalization layer from a pre-trained deep neural network model. Background technique [0002] Large-scale deep neural network (deep neural network, DNN) has significant functions in the fields of machine vision, image recognition and speech processing. However, these modern deep neural networks usually contain many layers, have very large models, and require high computational intensity. These characteristics make it very difficult to use deep neural networks on end-user devices (e.g., mobile phones) with low memory and low computing power. Modern deep neural networks such as ResNet, MobileNet, GoogLeNet, and Xception contain multiple batch normalization layers. These batch normalization layers are usually placed before or after convolutional, dense or deep convolutional layers to help train the n...

AI Technical Summary

Problems solved by technology

However, in the model inference stage, these batch normalization layers will consume a lot of computing resources and cause the device to require huge computing power to handle the training of the neural network.

Therefore, it is still difficult to import deep neural networks with batch normalization layers to all types of hardware, such as central processing units, graphics processing units, digital signal processors, etc.

Images