Dynamic mixing precision model construction method and system

Abstract

The invention discloses a dynamic mixing precision model construction method and system, and relates to a deep neural network technology. Aiming at the convertible state number and precision problems in the prior art, the scheme is provided, and the method is characterized by constructing a mixed precision state conversion table according to an average trace of a Hessian matrix of parameters of different blocks in a full-precision model and an optional parameter precision table S; randomly sampling a plurality of mixed precision sub-models by adopting each training iteration in the training process, and performing quantization operation by using an improved quantization function to obtain a mixed precision model; and forming a mixed bit deployment state table according to the actual deployment requirements to carry out actual deployment. The method has the advantages that the search space and the calculation amount required by training are reduced by utilizing the Hessian matrix average trace and the random sampling, and meanwhile, the improved quantization function can be directly migrated among different quantization bits, and the quantization error is small, so that the mixed precision model can be adaptively deployed in a lower bit scene; and meanwhile, the precision in a higher bit scene is also improved.

Description

technical field [0001] The present invention relates to deep neural network technology, in particular to a dynamic mixed precision model construction method and system. Background technique [0002] In recent years, more and more smart devices have deployed deep neural models, such as mobile phones and VR glasses. Deep neural models usually require large memory footprint and expensive computing operations, and smart devices have limited memory and computing resources. Considering more complex practical application scenarios, even for the same device, the requirements for deep neural models will vary due to changes in battery conditions, hardware aging and other factors. This brings greater difficulty to the deployment of smart devices, which is not conducive to the lightweight application of neural network algorithms. [0003] In order to solve the high memory usage and computing requirements of deep neural networks, researchers proposed a method for quantizing and compres...

AI Technical Summary

Problems solved by technology

[0008] However, the current adaptive quantization model has problems that are difficult to apply in low-bit scenarios, and the convertible states are limited. These two drawbacks greatly limit the promotion and use of adaptive quantization models.

Images