Deep convolutional neural network-based inter-layer non-uniform K-means clustering fixed-point quantification method

Abstract

The invention discloses a deep convolutional neural network-based inter-layer non-uniform K-means clustering fixed-point quantification method. The method includes the following steps that: step 1, a part of images of a deep convolutional neural network which can be correctly identified are selected, and feature mappings (Feature Map) generated in an identification process are extracted; step 2, inter-layer non-regular quantification is performed on the feature mappings in the deep convolutional neural network, and the maximum number of quantification bits of each layer of the convolutional network is determined with the precision of the model maintained; step 3, for each convolutional layer in the model, a K-means clustering algorithm is used to determine fixed-point values satisfying feature mapping distribution, the range of the fixed-point values is made to be located in a range which can be expressed by the maximum number of quantification bits, and the fixed-point values are adopted to represent values in the feature mappings and are stored in the form of indexes; and step 4, a neural network model fine tuning method is adopted perform fine tuning on the model, so that error caused by the quantification can be eliminated. With the inter-layer non-uniform K-means clustering fixed-point quantification method adopted, the storage cost of the feature mappings of the deep convolutional neural network can be greatly reduced with the precision of the model maintained. The method is innovative.

Description

technical field [0001] The invention relates to the field of deep learning model compression, in particular to the fixed-point field of deep convolutional neural networks oriented to embedded systems. Background technique [0002] With the rapid development of artificial intelligence, more and more applications designed with deep learning-led algorithms have appeared in people's lives, work and entertainment. However, a deep neural network is often composed of dozens or even hundreds of convolutional layers, and the feature maps generated during the calculation process need to occupy a large amount of storage space. This means that the product area is greatly increased for embedded applications. Therefore, it is of great significance to study the fixed-point compression problem of deep convolutional neural networks to reduce the storage overhead of feature maps and improve the practical value of deep learning. [0003] The fixed-point quantization of feature maps in the cu...

AI Technical Summary

Problems solved by technology

[0004] Purpose of the invention: the technical problem to be solved by the present invention is to provide a non-uniform K-average clustering fixed-point quantization method based on deep convolutional neural network layers for the problem of excessive feature map storage overhead in deep convolutional neural networks, Thus, the storage overhead can be greatly reduced while maintaining the accuracy of the model

Images