Label-Looping ASR Decoding to Reduce Redundant Predictor Calls
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional decoding algorithms for transducer models face inefficiencies in parallel and batch processing of speech utterances due to asynchronous processing of content and blank units, leading to redundant predictor network calls and idle processing of some utterances.
Innovation Solution
Implement label-looping processing that aggregates non-blank content units, updating the speech state only when a new content unit is identified, and optimizing predictor network calls by synchronizing processing across utterances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional decoding algorithms process speech utterances in parallel batches, then throughput is improved, but redundant predictor network calls and idle processing occur due to asynchronous content and blank unit processing
Solution Approach 1:
The patent extracts and separates the processing of content units from blank units. The label-looping mechanism extracts only the necessary predictor network calls by identifying when content units actually change the speech state, eliminating redundant calls that occur in conventional asynchronous processing where the predictor is called for every time step regardless of whether content is present.
Solution Approach 2:
The patent performs preliminary aggregation of content units within each time step before triggering predictor network calls. By pre-processing and consolidating content units at each time step, the system determines in advance whether a predictor call is necessary, preventing idle processing and redundant calls that would otherwise occur during asynchronous execution.
2Adaptability or versatility
If asynchronous processing is used for content and blank units, then processing flexibility is improved, but idle processing occurs when some utterances finish earlier than others
Solution Approach 1:
The patent merges the processing timelines of multiple utterances by synchronizing predictor network calls across the batch. The label-looping mechanism ensures that all utterances progress through time steps in unison, with predictor calls triggered only when content units are identified. This combining of processing rhythms eliminates idle time while preserving the flexibility to handle variable-length utterances.
3Ease of operation
If frame-looping is used to process each frame sequentially, then processing simplicity is improved, but processing efficiency deteriorates due to redundant operations
Solution Approach 1:
The patent segments the processing into two distinct loops: an outer label-looping structure that manages time steps and predictor calls, and an inner frame-looping structure that processes individual frames. This segmentation allows the simple frame-by-frame processing to occur within a more efficient outer loop that triggers predictor calls only when necessary, combining simplicity with improved productivity.
Data Source
AI summary
Disclosed are apparatuses, systems, and techniques that use label-looping processing for efficient automatic speech recognition (ASR). The techniques include performing a plurality of iterations of an outer processing loop to identify content units (CUs) of a media item having multiple frames. An individual iteration of the outer processing loop includes updating, using a first neural network (NN) and identified non-blank CU, a state of the media item and performing one or more iterations of an inner processing loop. An individual iteration of the inner processing loop includes processing, using a second NN, the state of the media item and an individual frame to predict a CU associated with the individual frame. The iterations of the inner processing loop are performed until the predicted CU corresponds to a non-blank CU. The identified plurality of CUs is used to generate a representation of the media item.