Sensitivity analysis and reinforcement learning neural network pruning method, system and device

Abstract

The invention discloses a sensitivity analysis and reinforcement learning neural network pruning method, system and device, and the method comprises the steps: setting a sparsity threshold, and selecting a low-sensitivity weight for pruning; obtaining a cutting method and precision, and determining a weight needing random pruning according to the sensitivity weight; carrying out random clipping oneach selected weight, and putting a pruning method and precision of multiple times of random clipping into a buffer area; a training reinforcement learning step: training a reinforcement learning agent by using the data in the buffer area, and putting a cutting method and precision generated after training into the buffer area; and repeating until the network precision reaches a preset value. According to the method, the low-sensitivity weight is selected for pruning, and the sparsity threshold value of each weight is set to ensure that the network descending precision is kept within the preset range after the cut weight is cut by adopting the current sparsity process. Under the condition that the network precision is ensured, the compression ratio of the neural network is improved to themaximum extent.

Description

technical field [0001] This application relates to the field of deep learning compression technology, in particular, to a neural network pruning method based on sensitivity analysis and reinforcement learning. Background technique [0002] Pruning (prune) is a compression technique of convolutional neural network (CNN), which is mainly used to reduce the calculation amount of convolutional neural network (CNN). The pruning algorithm usually achieves the purpose of reducing the calculation amount of the entire neural network by cutting out unimportant tensors in the weight of the neural network. [0003] Which tensors in the neural network weights are not important are determined by their sparsity. Sparsity is used to measure the number of 0s in the tensor and the size of the tensor. Therefore, cutting out the tensor with higher sparsity in the weight can achieve the purpose of compressing the convolutional neural network (CNN). [0004] The principle of convolutional neur...

AI Technical Summary

Problems solved by technology

[0008] The present invention proposes a neural network pruning method based on sensitivity analysis and reinforcement learning in order to solve the problem that the training data in the reinforcement learning method cannot contain all information and the network accuracy is not high

Images