Neural network model compression and acceleration method based on entropy attention

Abstract

The invention belongs to the technical field of neural networks, and relates to a neural network model compression and acceleration method based on entropy attention. According to the invention, a large parameter is constructed; a teacher network model with a large calculation amount and excellent performance is used for carrying out strong supervised learning on a student network with a small parameter amount, a small calculation amount and poor performance, a small model with a small parameter amount, a small calculation amount and excellent performance is finally obtained through the learning process, and the small model can meet the requirements for real-time performance and precision of a real scene.

Description

technical field [0001] The invention belongs to the technical field of neural networks and relates to a method for compressing and accelerating neural network models based on entropy attention. Background technique [0002] In recent years, the convolutional neural network has developed very rapidly. With the continuous improvement of the theory and the support of modern large-scale computing platforms, the convolutional neural network has made great progress. It has applications in different fields, and has shown very good performance in different applications. [0003] The convolutional neural network is a computationally intensive network model, and its superior performance depends on the inclusion of millions or even tens of millions of convolutional neural network models. The training of the model involves a large number of matrix operations, so the requirements for the computing platform are relatively high. , due to the advantages of large-scale parallel computing of...

AI Technical Summary

Problems solved by technology

Another example is that in the ImageNet Challenge in 2014, the VGGNet series models achieved very good results that year, such as the VGG16 model, which contains 13 layers of convolution, 3 fully connected layers, contains hundreds of millions of parameters, and the amount of parameters is huge. , although the performance is improved, it takes a lot of training time, and the inference time will also take a long time

Although the increase in the number of parameters of the model can increase performance, it is not suitable for low-power, low-storage, and low-bandwidth embedded device applications. If a model has too many parameters, it will undoubtedly limit its application in engineering

Images