Multi-Stream RNN-T Architecture for Joint Output Sequence Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional recurrent neural network transducers (RNN-Ts) require separate models for generating multiple output sequences, leading to inefficient use of computing resources and longer training times due to the lack of joint learning of dependencies between sequences.
Innovation Solution
A multi-stream recurrent neural network transducer (MS RNN-T) model processes multiple input sequences jointly, using a single unified model to learn dependencies between output sequences, thereby conserving computing resources and enabling faster training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate RNN-T models are used for each output sequence, then each output can be generated independently, but computing resources are wasted and training time increases
Solution Approach 1:
The patent combines multiple separate RNN-T models into a single unified model that processes multiple input sequences and generates multiple output sequences simultaneously. The model architecture integrates multiple encoders (e.g., acoustic encoder, language encoder) and a joint network that handles multiple output tasks, allowing joint training and shared parameter learning, thereby reducing training time and computing resource consumption while maintaining the ability to generate multiple output sequences independently
2Use of energy by moving object
If a single unified model is used to process multiple input sequences, then computing resources are conserved and training is faster, but the model must handle increased complexity in managing multiple streams
Solution Approach 1:
The unified model employs universal encoder structures and shared network components that can process different types of input sequences (e.g., audio, text, gestures) and generate different types of output sequences. The encoders and joint network are designed to handle multiple streams simultaneously through shared parameter spaces and unified processing architectures, reducing computing resource consumption while managing complexity through standardized multi-functional components
Data Source
AI summary
Techniques are disclosed that enable generating jointly probable output by processing input using a multi-stream recurrent neural network transducer (MS RNN-T) model. Various implementations include generating a first output sequence and a second output sequence by processing a single input sequence using the MS RNN-T, where the first output sequence is jointly probable with the second output sequence. Additional or alternative techniques are disclosed that enable generating output by processing multiple input sequences using the MS RNN-T. Various implementations include processing a first input sequence and a second input sequence using the MS RNN-T to generate output. In some implementations, the MS RNN-T can be used to process two or more input sequences to generate two or more jointly probable output sequences.