Chunking and Overlap Decoding for Streaming RNN Transducers

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

RNN-T based speech recognition systems face challenges with less reliable time alignments for words compared to hybrid speech recognition models, affecting accuracy.

Innovation Solution

A chunking and overlap decoding strategy is implemented for streaming RNN transducers, where digital speech is split into overlapping chunks, and bidirectional encoder embeddings are computed and combined to improve interpretation by the speech recognition system.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If RNN-T based speech recognition is used, then inference speed and memory efficiency are improved, but time alignment reliability deteriorates

Engineering Contradiction:

Improveinference speedVSAvoidtime alignment reliability

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The speech signal is divided into multiple overlapping chunks, where each chunk is processed independently by the RNN-T model. This segmentation allows parallel processing of different time segments while maintaining temporal coherence through the overlap region, thereby improving inference speed without sacrificing time alignment reliability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Embeddings are pre-computed for each chunk before the main decoding process. By preparing these representations in advance, the model reduces computational overhead during real-time inference, enabling faster processing while preserving the temporal structure needed for accurate time alignments.

Inventive Principle:

Principle #10Preliminary action

2Loss of information

If digital speech is split into overlapping chunks, then information preservation across chunks is improved, but processing complexity increases

Engineering Contradiction:

Improveinformation preservationVSAvoidprocessing complexity

Core Design Contradiction:

Loss of informationVSDevice complexity

Solution Approach 1:

The speech stream is segmented into fixed-size overlapping chunks with a defined overlap duration. This segmentation strategy preserves contextual information across chunk boundaries while maintaining a systematic processing framework that manages complexity through regular patterns.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Results from multiple overlapping chunks are merged through a combination strategy that resolves redundancies and integrates information from different segments. This merging process preserves complete information while eliminating duplicate computations, balancing information preservation with processing efficiency.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11942078B2Chunking and overlap decoding strategy for streaming RNN transducers for speech recognition

Publication Date: 2024.03.26 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A computer-implemented method is provided for improving accuracy recognition of digital speech. The method includes receiving the digital speech. The method further includes splitting the digital speech into overlapping chunks. The method also includes computing a bidirectional encoder embedding of each of the overlapping chunks to obtain bidirectional encoder embeddings. The method additionally includes combining the bidirectional encoder embeddings. The method further includes interpreting, by a speech recognition system, the digital speech using the combined bidirectional encoder embeddings.