Speech Recognition Decoding with Reuse Tokens for Faster Interim Output
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition technologies struggle with responsiveness, as they often require sequential decoding of entire input speech before providing intermediate results, leading to delayed output and increased processing time, especially when partial decoding is used with short intervals.
Innovation Solution
A neural network-based decoding method that reuses tokens from previous decoding steps by identifying a stable time where the decoding result remains unchanged, allowing for partial decoding with reduced processing load and immediate output of recognition results during speech input.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sequential decoding of entire input speech is performed, then decoding accuracy is improved, but responsiveness deteriorates due to delayed output
Solution Approach 1:
The patent segments the decoding process into multiple independent decoding steps that can be performed on different portions of the input speech sequence. Each decoding step processes a subset of tokens and generates intermediate results independently, allowing parallel execution. This segmentation enables the system to provide partial recognition results before completing the entire decoding process, thereby improving responsiveness while maintaining decoding accuracy through multiple decoding passes.
2Loss of time
If partial decoding with short intervals is used, then responsiveness is improved, but processing load increases
Solution Approach 1:
The patent performs preliminary actions by pre-processing the input speech sequence into fixed-length chunks or segments before the actual decoding process. These pre-segmented sequences are prepared in advance and can be quickly processed during decoding. By organizing the input data structure beforehand into manageable segments with overlapping boundaries, the system reduces the computational complexity during each decoding step, thereby lowering the overall processing load while maintaining short decoding intervals for improved responsiveness.
3Reliability
If entire speech sequence is decoded, then complete recognition result is obtained, but intermediate results are delayed
Solution Approach 1:
The patent implements continuous useful action by performing multiple decoding steps that progressively process the input speech sequence from different starting points. Each decoding step generates valid intermediate recognition results that can be output immediately, while subsequent decoding steps continue processing to refine and complete the recognition. This continuous approach ensures that complete recognition results are eventually obtained while providing useful intermediate results at each step, eliminating the delay associated with waiting for the entire sequence to be decoded before providing any output.
Data Source
AI summary
A decoding method includes receiving an input sequence corresponding to an input speech at a current time; and in a neural network (NN) for speech recognition, generating an encoded vector sequence by encoding the input sequence, determining reuse tokens from candidate beams of two or more previous times by comparing the candidate beams of the previous times, and decoding one or more tokens subsequent to the reuse tokens based on the reuse tokens and the encoded vector sequence.