Method and system for quantitative training of convolutional neural network weight parameters

Abstract

The invention belongs to the technical field of artificial intelligence and in particular to a method and system for the quantitative training of convolutional neural network (CNN) weight parameters.The CNN includes a convolution layer, a normalization layer, a scaling layer, a full connection layer, and a pooling layer. The method includes: updating the scaling layer according to a weight parameter of the normalization layer; removing the normalization layer; quantizing a weight parameter of the scaling layer by using an exponential quantization method; adjusting a weight parameter of the convolution layer according to a quantization process of the weight parameter of the scaling layer; and quantizing the weight parameter of the convolution layer by using a group recursive method; updating the weight parameter of the convolution layer according to the weight parameter of the scaling layer; and removing the scaling layer. Without a decrease in network precision, method and system greatly reduce network computing complexity, the storage capacity and the transmission bandwidth of the weight parameters, and achieve hardware implementation without multipliers so as to achieve fast calculation acceleration..

Description

technical field [0001] The invention belongs to the technical field of artificial intelligence, and in particular relates to a convolutional neural network weight parameter quantization training method and system. Background technique [0002] With the development of artificial intelligence, deep learning has become a very popular research direction, attracting great attention and being widely used in popular fields such as computer vision, speech recognition, and automatic driving. Among them, image recognition application, as one of the core application fields, has received extensive attention and achieved many remarkable results. In image classification and recognition, Convolutional Neural Network (CNN for short) has the best performance. CNN network is a feed-forward neural network, mainly composed of one or more convolutional layers, fully connected layers and their associated weight parameters, supplemented by normalization layer and pooling layer, usually connected ...

AI Technical Summary

Problems solved by technology

However, most of the previous work only focused on shallow and simple convolutional neural network models such as AlexNet [5] and VGG [6] network structures

However, for processing the most advanced convolutional neural network models, it is necessary to deal with more complex networks and greater computing and storage requirements.

In this case, bandwidth and storage space become a major challenge

Especially for embedded applications, although hardware accelerators are used, due to power consumption constraints, it is difficult for complex CNN networks to achieve real-time processing calculation speeds

Images