ASR Model Quantization and Sparsity Aware Fine-Tuning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The deployment of large-scale universal speech models for automatic speech recognition is hindered by their massive size, requiring significant memory and computational resources, necessitating efficient fine-tuning and model compression algorithms.
Innovation Solution
A computer-implemented method for fine-tuning a pre-trained automatic speech recognition (ASR) model using quantization and sparsity aware training with native integer operations, involving pruning weights using a sparsity mask and quantizing weights based on integers with a fixed-bit width, thereby reducing model size while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large-scale universal speech models are deployed for automatic speech recognition, then recognition quality is improved, but memory and computational resource requirements increase significantly
Solution Approach 1:
The patent extracts and removes redundant or less important weights from the model using pruning techniques. By identifying and eliminating unnecessary connections in the neural network, the model size is reduced while attempting to preserve the essential functionality for speech recognition quality.
Solution Approach 2:
The patent changes the precision parameters of model weights by quantizing them from high precision (e.g., 32-bit floating point) to low precision (e.g., 8-bit integers or lower). This parameter transformation reduces memory requirements and computational complexity while maintaining acceptable recognition performance.
2Quantity of substance
If model compression is applied to reduce ASR model size, then deployment cost is reduced, but recognition accuracy may deteriorate
Solution Approach 1:
The patent applies preliminary quantization-aware training before final model compression. During this preliminary phase, the model is trained with quantized weights to adapt its parameters to the lower precision representation, thereby compensating for potential accuracy losses that would occur with direct compression of a fully trained high-precision model.
Solution Approach 2:
The patent incorporates feedback mechanisms during the fine-tuning process where the model performance is continuously monitored and used to guide further pruning and quantization decisions. This iterative feedback loop allows the system to identify which weight reductions have minimal impact on accuracy and which require more careful handling.
3Use of energy by moving object
If quantization and pruning are applied during fine-tuning, then computational resources are reduced, but training complexity increases
Solution Approach 1:
The patent segments the fine-tuning process into distinct phases: initial quantization-aware training, followed by pruning, then further fine-tuning, and finally evaluation. This segmentation allows each sub-task to be optimized independently and makes the overall complex process more manageable and systematic.
Data Source
AI summary
A method includes obtaining a plurality of training samples that each include a respective speech utterance and a respective textual utterance representing a transcription of the respective speech utterance. The method also includes fine-tuning, using quantization and sparsity aware training with native integer operations, a pre-trained automatic speech recognition (ASR) model on the plurality of training samples. Here, the pre-trained ASR model includes a plurality of weights and the fine-tuning includes pruning one or more weights of the plurality of weights using a sparsity mask and quantizing each weight of the plurality of weights based on an integer with a fixed-bit width. The method also includes providing the fine-tuned ASR model to a user device.