Data fixed-point processing method, device, electronic equipment and computer storage medium

Abstract

The invention provides a data fixed-point processing method. The method includes the following steps: for a trained first convolutional neural network model, extracting an original weight parameter and original feature map of the ith data layer of the first convolutional neural network model, wherein i is the number of layers of the network and is a positive integer; performing fixed-point processing on the original weight parameter and original feature map to obtain a first weight parameter and a first feature map; converting the first weight parameter and first feature map into floating point numbers respectively to obtain a second weight parameter and a second feature map; and replacing the original weight parameter with the second weight parameter, and replacing the original feature map with the second feature map. The data fixed-point processing method provided by the invention can ensure that relatively small precision loss can be achieved at relatively small quantification bit width.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a fixed-point data processing method, device, electronic equipment and computer storage medium. Background technique [0002] Convolutional neural network is an important algorithm in machine learning algorithm. With the continuous research of people, the recognition accuracy of convolutional neural network has been continuously improved, and at the same time, the computational and storage complexity of its algorithm has also increased. [0003] With the large-scale popularity of deep learning and Graphic Processing Unit (GPU) technology, tasks such as video-based target detection and recognition have gradually emerged in people's field of vision and are changing our lives all the time. Although GPU technology has powerful computing capabilities, its power consumption and area are very high, which also limits its application in many fields, especially the expansion of ...

AI Technical Summary

Problems solved by technology

However, due to the high complexity of deep learning algorithms, usually only a relatively low quantization bit width can

Images