Multi-Frame Blank Decoding for ASR Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conversational AI systems using automatic speech recognition (ASR) techniques face inefficiencies due to processing empty or unhelpful audio frames, leading to increased resource usage and deployment costs.
Innovation Solution
Implementing a neural network with multi-frame blanks that allow for skipping of certain frames during processing, trained using a probability lattice to minimize loss and adjust weights for token emissions, reducing computational load and improving speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all audio frames are processed individually, then complete speech recognition is achieved, but resource usage and processing time increase significantly
Solution Approach 1:
The audio input is divided into discrete frames, and the system selectively processes only certain frames rather than all frames. This segmentation allows the system to identify and process only the frames containing useful speech information, reducing overall processing load while maintaining recognition accuracy.
Solution Approach 2:
The system extracts and processes only the essential frames that contain meaningful speech content, discarding or skipping empty or unhelpful frames. This extraction approach removes the harmful element of processing redundant data while preserving the useful speech information.
2Reliability
If all audio frames are processed individually, then no speech information is missed, but computational resources and deployment costs increase
Solution Approach 1:
The system extracts only the necessary frames containing speech information for processing, eliminating the waste of computational resources on empty frames. This selective extraction reduces energy consumption and deployment costs while maintaining recognition accuracy.
Solution Approach 2:
Instead of processing all frames (excessive action), the system processes only the subset of frames that contain useful speech content (partial action). This approach avoids the waste of computational resources on redundant processing while ensuring all necessary speech information is captured.
3Device complexity
If single blank decoding is used, then processing is simpler, but frames with no sound or unhelpful content cannot be skipped efficiently
Solution Approach 1:
The system implements multi-frame blank decoding that allows the decoder to skip multiple consecutive frames that contain no useful information. This skipping mechanism enables efficient bypass of empty or unhelpful frames, improving processing efficiency without significantly increasing decoding complexity.
Data Source
AI summary
Systems and methods provide for a machine learning system to train a machine learning model to output a multi-frame blank symbol when processing an auditory input. For example, as the system generates paths through a probability lattice, one or more paths include a multi-frame blank that skips at least one frame associated with the probability lattice. The inclusion of the multi-frame blank symbol may increase a total number of potential paths through the probability lattice, and may allow the machine learning model to more quickly and accurately process audio frames, while disregarding audio frames of less value. In deployment, when an output of the machine learning model indicates a multi-frame blank symbol or token, one or more frames of the auditory input may be omitted from processing.