Personalized ASR Quantization Under Mobile Memory Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic speech recognition (ASR) models, particularly large transformer-based architectures, are impractical for deployment on mobile devices due to their size and lack of consideration for user diversity and privacy, with current quantization methods failing to account for out-of-domain data during deployment.
Innovation Solution
A method for personalizing ASR models by detecting layer sensitivity to quantization using unlabelled local data, determining optimal bit depths, and applying scaling factors to generate a compressed model that meets memory constraints while preserving privacy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large transformer-based ASR models are deployed, then recognition accuracy is improved, but device memory requirements and computational resources become impractical for mobile devices
Solution Approach 1:
The patent segments the model quantization process into two distinct phases: a training phase where the full-precision model is trained on labelled data, and a deployment phase where the pre-trained model is quantized using unlabelled local data. This segmentation allows the model to achieve high accuracy through full-precision training while enabling efficient deployment through post-training quantization, resolving the contradiction between accuracy and model size for mobile devices
Solution Approach 2:
The patent applies preliminary action by pre-training the ASR model to full precision before deployment, then performing quantization as a preliminary step before actual use. The method pre-computes sensitivity metrics and determines optimal bit depths for each layer in advance, allowing the model to be compressed to fit mobile devices while maintaining accuracy. This preliminary quantization preparation enables the model to be deployed on resource-constrained devices without sacrificing recognition performance
2Quantity of substance
If model quantization is applied to compress the model, then memory usage is reduced, but performance degrades on out-of-domain data
Solution Approach 1:
The patent applies local quality by determining different optimal bit depths for different layers of the neural network based on their individual sensitivity to quantization. Each layer is analyzed separately using sensitivity metrics computed from local data, allowing the quantization strategy to be tailored to the specific characteristics of each layer. This layer-wise differentiated quantization preserves performance on out-of-domain data while achieving compression, as critical layers maintain higher precision while less sensitive layers are compressed more aggressively
Solution Approach 2:
The patent changes parameters by dynamically determining the optimal bit depth for each layer based on sensitivity analysis rather than using a fixed uniform quantization scheme. The method computes sensitivity metrics from local data and adjusts the bit depth parameter for each layer accordingly, allowing the model to adapt to local data distributions and maintain performance on out-of-domain data while achieving effective compression
3Measurement precision
If layer-wise sensitivity detection is performed using local data, then quantization accuracy is improved, but computational time and resources increase
Solution Approach 1:
The patent extracts only the essential information needed for quantization by using unlabelled local data to compute sensitivity metrics, rather than requiring full labelled datasets. The method extracts sensitivity characteristics from local data samples and uses these extracted metrics to determine optimal bit depths, avoiding the need for time-consuming retraining or extensive computational analysis. This extraction approach achieves high quantization accuracy while minimizing computational time and resource requirements
Data Source
AI summary
The present techniques generally relate to a computer-implemented method for personalising automatic speech recognition (ASR) or other general-purpose models using mixed precision (MP) quantization. Each model is personalised on a user device or server to a target memory budget B using user data.