After-training quantization compression method and system in speech recognition task

Abstract

The invention discloses an after-training quantization compression method and system in a speech recognition task. The related method comprises the steps: carrying out the scaling of an input vector and each row of a weight matrix through a diagonal matrix, carrying out quantization, and carrying out estimation to obtain a theoretical expectation error of a calculation result after quantization; setting that truncation does not occur in calculation after quantization of a scaled weight matrix, establishing corresponding constraint conditions, and solving a diagonal matrix which enables a theoretical expected error to be minimum by using an alternative iteration optimization algorithm; or, introducing a pre-truncation boundary to adjust the constraint condition, and then using an alternative iteration optimization algorithm to solve the diagonal matrix which enables the theoretical expectation error to be minimum. According to the scheme and the quantification method, through finer scaling and an optimization algorithm based on theoretical derivation, while the model precision loss is well controlled, the model reasoning efficiency is obviously improved, that is, the storage space and the operation time are obviously reduced, and the application scene of a speech recognition model is expanded.

Description

technical field [0001] The invention relates to the technical field of deep learning and artificial intelligence, in particular to a post-training quantization compression method and system in speech recognition tasks. Background technique [0002] The speech recognition task is the process of converting speech into text, and it is also one of the most important and common tasks in the field of deep learning and artificial intelligence. At present, deep neural network models, such as end-to-end VGG-Transformer model, Conformer model, etc., often achieve the best results in speech recognition tasks, and the further improvement of the effect is usually accompanied by the increase of model complexity, which brings problems. The ever-increasing storage and computing requirements limit its application and deployment in scenarios such as mobile terminals and embedded devices. [0003] In order to reduce the storage and computing costs of the deep speech recognition model, the fol...

AI Technical Summary

Problems solved by technology

Quantization is divided into quantization training and post-training quantization. The former requires additional training, which will increase a large amount of calculations, and is not suitable for situations where training data is difficult to obtain due to privacy or permission issues.

[0004] For post-training quantization, the common method is to use a uniform scaling ratio for the weight matrix. However, the distribution of values ​​in each row and column of the weight matrix is ​​often inconsistent, which will increase the calculation error caused by quantization; in addition, when selecting the optimal scaling ratio, it is often used The minimum information loss (KL divergence) criterion does not necessarily mean the minimum loss of model accuracy

Images