Non-Autoregressive Decoder for Parallel Sequence Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autoregressive (AR) models for generating sequences, such as text, are limited by slow sampling, difficulty in handling incomplete sequences, and restrictive neural network architectures, which hinder their efficiency and flexibility in practical applications.
Innovation Solution
A non-autoregressive neural network system is developed, utilizing a decoder neural network that processes a current output sequence conditioned on a context input to generate scores for output tokens, allowing for parallel generation of output sequences and overcoming the limitations of AR models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If autoregressive models are used for sequence generation, then training scalability is improved, but sampling speed becomes prohibitively slow
Solution Approach 1:
The patent segments the sequence generation process by predicting multiple tokens simultaneously at different positions rather than sequentially. The decoder neural network processes the entire output sequence in parallel, computing scores for all output tokens at once, which divides the generation task into independent positional segments that can be processed concurrently.
Solution Approach 2:
The patent transitions from a temporal dimension of processing (autoregressive step-by-step generation) to a spatial dimension of processing (parallel prediction across all positions). By reordering the generation process from sequential time-steps to parallel position-based predictions, the system achieves simultaneous computation of multiple tokens.
2Adaptability or versatility
If autoregressive models are used for sequence generation, then training performance is improved, but architectural flexibility is severely limited
Solution Approach 1:
The patent inverts the traditional autoregressive approach by removing the causal constraint that forces left-to-right processing. Instead of predicting tokens in sequential order where each prediction depends on previous tokens, the system predicts all tokens simultaneously based on the entire context, fundamentally reversing the processing paradigm.
Solution Approach 2:
The decoder neural network is designed to handle multiple generation tasks and conditioning scenarios universally. It can process various types of context inputs and generate different kinds of output sequences without requiring architecture changes, making the model adaptable to machine translation, text completion, and other sequence generation tasks.
3Adaptability or versatility
If autoregressive models are used for sequence generation, then sequence generation capability is improved, but ability to handle incomplete sequences is difficult
Solution Approach 1:
The system performs preliminary action by predicting tokens for all positions simultaneously before any gaps need to be filled. The decoder neural network processes the incomplete sequence as a whole, predicting both complete and missing portions in parallel, which allows gaps to be filled naturally during the generation process without requiring special handling.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output sequences using a non-auto-regressive neural network.