Quantification method for deep learning network parameters

Abstract

The invention provides a quantification method for deep learning network parameters. The method comprises the following steps: (1) constructing a deep learning network to generate training data; (2) training the constructed deep learning network by using a large amount of training data, and determining network parameters; (3) learning parameters are extracted, and hyper-parameters are determined;(4) designing a quantizer, determining a specific expression of the designed soft step function according to hyper-parameters, and introducing learnable parameters to enable the shape of the soft stepfunction to be adjustable; (5) introducing the soft step function with the learning parameters into a deep learning network, quantifying the learning parameters, learning quantizer parameters throughthe same training data, and adopting an annealing strategy in the training process; and (6) fixing the trained quantizer parameters, and using the quantizer to quantize the deep learning network parameters. According to the method, the performance loss caused by quantification can be effectively reduced, and the storage overhead required by the deep network is greatly reduced.

Description

technical field [0001] The invention belongs to the field of deep learning, and is a quantification method for deep learning network parameters. Background technique [0002] The subject of deep learning network has been gradually created since 2006 with the introduction of the learning algorithm of the deep belief network based on the layered restricted Boltzmann machine. It is an emerging field in the field of artificial intelligence. Discipline, the main content of its research is the modeling and algorithm learning of multi-layer neural networks. Deep learning network methods have been successfully applied in many other fields, such as image processing, natural language processing, etc. [0003] Deep learning is an emerging multi-layer neural network learning algorithm, which has attracted widespread attention in the field of machine learning because it alleviates the local minimum in traditional network training. With the development in recent years, the deep learning...

AI Technical Summary

Problems solved by technology

With its multi-layer network structure, the deep network has been well applied in many fields, but at the same time, with the increase of the network level, a large number of network parameters follow, which is not only difficult to learn, but also difficult to learn in the storage network. A large amount of hardware overhead is required when constructing

Images