Neural network interlayer activation value quantification method and apparatus

Abstract

A neural network interlayer activation value quantification method and apparatus, wherein the method comprises: acquiring an activation value of the previous activation layer of a neural network; obtaining an activation value limit of a current activation layer according to the activation value of the previous activation layer; determining respective activation values of the current activation layer according to the activation value limit of the current activation layer and the activation value of the previous activation layer. Therefore, the activation value information of the previous activation layer can be transmitted to the current activation layer. Compared with the fixedly set limit in the prior art, the technical solution provided by the present invention can dynamically adapt to the change of the activation value according to the activation value of the previous activation layer, thereby reducing the estimated deviation of quantization.

Description

technical field [0001] The invention relates to the technical field of neural networks, in particular to a method and device for quantifying activation values ​​between layers of a neural network. Background technique [0002] With the development of artificial intelligence, especially the development of convolutional neural network, it has been widely used in the field of intelligent monitoring and has become an indispensable tool, such as face recognition, vehicle detection, object recognition, etc. However, as the number of layers of modern convolutional neural networks deepens, the complexity of the network becomes larger and larger. For example, for a convolutional neural network, the number of convolutional layers can exceed 10 layers. In addition, all convolutional layers The amount of calculation accounts for almost 80% of the entire network's calculation amount. As a result, similar convolutional neural networks cannot be run on embedded devices such as surveillanc...

AI Technical Summary

Problems solved by technology

If the estimation is inaccurate, it will cause the quantized convolutional neural network model to deviate from the optimal model in the optimization space

The applicant found that: the existing quantization algorithms predetermine the maximum and minimum values ​​of quantization according to a certain fixed model, without considering that the activation of the convolutional neural network changes dynamically during the training process, which leads to Quantify the estimated deviation of the maximum and minimum values

Images