Compression method and device for deep learning model

Abstract

The invention discloses a compression method and device for a deep learning model, and the method comprises the following steps: carrying out the pruning of the deep learning model according to a pruning threshold value; carrying out the fine tuning of the pruned deep learning model; calculating the accuracy of the deep learning model after fine tuning; and retraining the deep learning model after fine tuning if the accuracy is less than a preset threshold value. Through the addition of an accuracy test, it indicates that too many parameters are deleted in compression at a former step if the accuracy is less than the threshold value, and the set accuracy cannot be achieved by fine tuning through a little data, so a large amount of data is needed for the retraining of the model. The method is simple in compression steps, and the principles are easy to understand. The method is high in compression efficiency. The method can reduce the storage and computing consumption of a large-scale deep learning model, keeps the performance and accuracy, and improves the practicality of the deep learning model.

Description

technical field [0001] The invention relates to the field of deep learning, in particular to a compression method and device for a deep learning model. Background technique [0002] Deep learning is a new field in machine learning research by establishing a neural network that imitates the human brain for analysis and learning, and uses the mechanism of the human brain to explain data, such as images, sounds, and data in this paper. [0003] Deep learning allows an artificial neural network to learn its statistical laws from a large number of training data samples to make predictions about unknown events. Its essence is to build a network structure with many hidden layers and automatically learn more useful features based on massive data, thereby improving the accuracy of classification or prediction. Therefore, deep learning has been widely used in image, sound and other fields recently. [0004] During the training and learning process, the neural network continuously im...

AI Technical Summary

Problems solved by technology

[0015] In order to overcome the deficiencies of the prior art, one of the purposes of the present invention is to provide a compression method for deep learning models, which can solve the problem that the existing direct construction of small models requires a lot of experiments, and the efficiency is low; removal is not important The scheme of pruning the network and then training based on the pruned network has a large upper limit of compression, which is easy to lose more information and reduce the performance of the model; the code steps of the deep compression scheme are more complicated, and the network structure needs to be understood Deeper understanding, more difficult questions

[0016] The second object of the present invention is to provide a compression device for deep learning models, which can solve the problem that the existing direct construction of small models requires a lot of experiments, and the efficiency is low; remove unimportant connections to pruning the network, In the training scheme based on the pruned network, the compression has a large upper limit, and it is easy to lose more information and reduce the performance of the model; the code steps of the deep compression scheme are more complicated, requiring a deeper understanding of the network structure, which is more difficult The problem

[0017] The third object of the present invention is to provide a compression device for deep learning models, which can solve the problem that the existing direct construction of small models requires a lot of experiments, and the efficiency is low; remove unimportant connections and pruning the network, In the training scheme based on the pruned network, the compression has a large upper limit, and it is easy to lose more information and reduce the performance of the model; the code steps of the deep compression scheme are more complicated, requiring a deeper understanding of the network structure, which is more difficult The problem

Images