Method and system capable of switching bit-wide quantized neural network on line

Abstract

The invention provides a method and a system for switching bit-wide quantized neural network on line, and the method comprises the steps: enabling deep neural networks with different bit widths to beintegrated into a super network, wherein all the networks with different bit widths share the same network architecture; the super network operating at different bit widths, for any bit width, obtaining a corresponding network intermediate layer feature, and processing each network intermediate layer feature by adopting a corresponding batch normalization layer; training the super network throughsupervised learning, and simulating quantization noise in a super-network training stage until a consistency loss function between a low-bit mode and a high-bit mode converges, so as to obtain a trained super network; and extracting the quantized neural network of the target bit from the trained super network by using a preset quantizer to perform low-bit reasoning. While the neural network is nottrained again, the bit width can be switched at will to adapt to different hardware deployment environments.

Description

technical field [0001] The invention relates to the fields of computer vision and image processing, in particular to a method and system for a quantized neural network capable of switching bit widths online. Background technique [0002] With the increasing complexity of deep neural networks (DNNs), there are often great challenges in deploying deep neural networks. Therefore, model compression and model acceleration have received increasing attention in machine learning. An important research direction is quantization of deep neural networks, which quantizes model weights and intermediate layer activations to smaller bit widths. Due to the reduced bit width, quantized deep neural networks have smaller model sizes and can leverage efficient fixed-point computations for fast inference. However, when the bit width is reduced to 4 bits or even smaller, there will be a significant loss of precision. To alleviate this problem, quantization-aware training is commonly employed t...

AI Technical Summary

Problems solved by technology

But it can only quantize the neural network to 8 bits, and cannot flexibly change the bit width during network operation

Images