Neural network compression method based on Taylor expansion and data driving

Abstract

The invention discloses a neural network compression method based on Taylor expansion and data driving, and the method comprises the steps: training an original neural network model through employing a training data set until convergence, and obtaining a trained neural network model; evaluating the data set for evaluation and the training neural network model to obtain an importance evaluation value of each convolution kernel; and removing unimportant convolution kernel channels to compress the neural network model, so that the effect of compressing the parameter quantity and the model size of the neural network model can be achieved on the premise of ensuring the performance of the neural network model as much as possible, thereby realizing optimization of the neural network model in the prior art, improving the precision of the compressed network, and reducing calculation memory space.

Description

technical field [0001] The invention relates to the technical field of neural network optimization, in particular to a neural network compression method based on Taylor expansion and data drive. Background technique [0002] Convolutional neural networks are widely used in various fields of computer vision, including object classification and localization, pedestrian and vehicle detection, and video classification. Expanding the scale of the data set and the parameters of the neural network model are the reasons for the success of deep learning. In recent years, the number of layers of the neural network model has ranged from 8 layers of AlexNet to more than 100 layers of ResNet-152. By increasing the number of layers dramatically, deep neural networks achieve better performance than humans on the ImageNet classification task. However, while the neural network model is getting bigger and bigger, devices with limited computing resources, such as smartphones and embedded devi...

AI Technical Summary

Problems solved by technology

However, while the neural network model is getting bigger and bigger, devices with limited computing resources, such as smart phones and embedded devices, are gradually unable to carry and run those neural network models that have outstanding performance but consume a lot of resources (computing power and graphics card memory). Even with cloud computing services, predictive latency and energy consumption are important considerations

Images