Neural network quantification method and device and computer readable storage medium

Abstract

The invention provides a neural network quantification method and device, and a computer readable storage medium. The method comprises the steps of determining distribution data of activation output of a target network layer of a neural network according to a correction data set; determining the target quantization range of the target network layer according to the distribution data; and performing fixed-point quantization on the target network layer according to the target quantization range and the target quantization bit width. By utilizing the method, the precision loss in neural network quantization can be reduced, and a better quantization effect is achieved.

Description

technical field

[0001] The invention belongs to the field of neural network calculation, and in particular relates to a neural network quantization method, device and computer-readable storage medium. Background technique

[0002] This section is intended to provide a background or context for implementations of the invention that are recited in the claims. The descriptions herein are not admitted to be prior art by inclusion in this section.

[0003] In recent years, with the rapid development of deep learning, deep learning has been proved to have good results in tasks including image classification (Image Classification), object detection (Object Detection), natural language processing (Natural Language Processing) and so on. Deep learning uses a large amount of data to train a neural network model with functions such as analysis and prediction. However, as the scale of the neural network model increases, more storage resources, bandwidth resources, and computing resourc...

Images