Model quantitative training method and device, machine readable storage medium and electronic equipment

Abstract

The invention discloses a model quantitative training method and device, a machine readable storage medium and electronic equipment, and the method comprises the steps: obtaining a network parameter of a to-be-quantized layer for each to-be-quantized layer in a to-be-quantized model, the network parameter being a weight value or an activation value in a network; judging whether the maximum value and the minimum value in the network parameters of the to-be-quantized layer are equal or not, or whether the maximum value is smaller than a preset parameter threshold value or not; if the maximum value and the minimum value in the network parameters of the to-be-quantized layer are equal or the maximum value is smaller than a parameter threshold value, taking the sum of the maximum value and a preset numerical value as a new maximum value; and performing network quantization according to the maximum value and the minimum value of the parameters in the to-be-quantized layer to obtain a targetmodel. According to the scheme, the problem that training cannot be carried out when a certain layer is 0, the value of the certain layer is completely equal or the value of the certain layer is veryclose to 0 in the quantitative training process can be solved, so that the to-be-quantized model can be subjected to normal quantitative training.

Description

technical field [0001] The present application relates to the technical field of neural networks, and in particular, to a model quantization training method, device, machine-readable storage medium, and electronic equipment. Background technique [0002] With the rapid development of deep learning, the accuracy of deep learning models has been continuously improved. However, deep learning models with higher accuracy often need to rely on high-performance GPUs. When these deep learning models are applied, they also need to consume huge hardware resources and are not suitable for mobile terminals. At present, in order to solve the application of high-precision deep learning models on mobile terminals, the method of quantifying the model is usually used to obtain a model that can be used on mobile terminals. However, the current model quantization method will appear in a certain layer of the model. When the network parameters are equal, for example, they are all equal to 0, o...

AI Technical Summary

Problems solved by technology

When these deep learning models are applied, they also need to consume huge hardware resources, and are not suitable for mobile terminals, etc.

At present, in order to solve the application of high-precision deep learning models on mobile terminals, the method of quantifying the model is usually used to obtain a model that can be used on mobile terminals. However, the current model quantization method will appear in a certain layer of the model. The network parameters are equal, for example, they are all equal to 0, or when the parameters of a certain layer are very close to 0, the model cannot be quantified.

Images