Method for quantizing PRELU activation function

Abstract

The invention provides a method for quantizing an activation function as PRELU, which comprises the following steps: S1, data quantization: to-be-quantized data is quantized according to the following formula (1) to obtain low-bit data, and the variable description of the formula (1) is as follows: Wf is an array of full-precision data, Wq is quantized data, maxw is the maximum value in the full-precision data Wf, minw is the minimum value in the full-precision data Wf, b is the quantized bit width; S2, the PRELU activation function is quantized, a quantization formula is shown as a formula (2), and variables of the formula (2) indicate that when the value of xi is larger than 0, the value of xi needs to be multiplied by a parameter q1, if the value of xi is smaller than 0, the value of xi needs to be multiplied by a parameter ac, and c is a channel where xi is located; specific parameters illustrate that x is a three-dimensional array, namely {h, w, c}, and h, w and c are respectively the length, width and channel number of the array; the parameter a is a one-dimensional array {c}, and the values of c and c in x are equal; q1 is the quantization of 1.0; ac is the value of the cth channel in the parameter a.

Description

technical field [0001] The present invention relates to the technical field of neural network acceleration, in particular to a method for quantifying PRELU activation functions. Background technique [0002] In recent years, with the rapid development of science and technology, the era of big data has arrived. Deep learning uses deep neural network (DNN) as a model, and has achieved remarkable results in many key areas of artificial intelligence, such as image recognition, reinforcement learning, and semantic analysis. As a typical DNN structure, convolutional neural network (CNN) can effectively extract hidden layer features of images and accurately classify images. It has been widely used in the field of image recognition and detection in recent years. [0003] In particular, real-time quantization of the feature map: dequantize the result of the convolution operation into a full-precision number, and then complete the quantization of the feature map according to the maxi...

AI Technical Summary

Problems solved by technology

[0011] In order to solve the above technical problems, this application proposes a method of quantizing the activation function as PRELU, which aims to overcome the defects in the above-mentioned prior art, and proposes a method of quantizing PRELU to solve the problem of activation of the existing low-bit model reasoning process. The problem of using full precision calculation when the function is PRELU

Images