Compressed Finite State Transducers for ASR Memory Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The size of transducers used in automatic speech recognition (ASR) systems, particularly those incorporating language models, becomes a barrier due to runtime resource restrictions, limiting the deployment of more complex models despite potential accuracy gains.
Innovation Solution
The implementation of compressed finite state transducers (FSTs) that are created during a training phase and decompressed at runtime, reducing the computational resources needed for high-quality ASR processing by storing and using compressed FSTs, which are specifically designed to minimize memory and processing requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If compressed FSTs are used, then memory usage is reduced, but decompression processing is required at runtime
Solution Approach 1:
The FST is compressed during the training phase (preliminary action) before deployment. This pre-compression reduces the memory footprint of the model that needs to be stored and loaded at runtime, while the decompression operation is performed once during initialization rather than continuously during inference, thus resolving the contradiction between reduced memory usage and added processing complexity.
2Measurement precision
If larger FSTs are used for better ASR performance, then recognition accuracy improves, but runtime resource restrictions are exceeded
Solution Approach 1:
The FST is transformed from an uncompressed format to a compressed format, changing its physical representation while maintaining its functional properties. This parameter change (compression ratio) allows the same linguistic model to occupy less memory space, enabling larger, more accurate models to be deployed within fixed runtime resource constraints without sacrificing recognition accuracy.
3Volume of stationary object
If FST compression is implemented, then model size is reduced, but additional compression/decompression steps are added
Solution Approach 1:
Compression is performed as a preliminary action during model training and preparation, before the model is deployed to the runtime environment. The compressed model is stored in this compact form, and decompression is performed once during system initialization. This timing strategy reduces the visible overhead during actual speech recognition operations, as the compression/decompression steps occur outside the main inference loop.
Data Source
AI summary
Compact finite state transducers (FSTs) for automatic speech recognition (ASR). An HCLG FST and/or G FST may be compacted at training time to reduce the size of the FST to be used at runtime. The compact FSTs may be significantly smaller (e.g., 50% smaller) in terms of memory size, thus reducing the use of computing resources at runtime to operate the FSTs. The individual arcs and states of each FST may be compacted by binning individual weights, thus reducing the number of bits needed for each weight. Further, certain fields such as a next state ID may be left out of a compact FST if an estimation technique can be used to reproduce the next state at runtime. During runtime portions of the FSTs may be decompressed for processing by an ASR engine.