Sequence-to-Sequence Neural Networks with Look-Ahead Tree Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sequence transduction neural networks often produce output sequences with high likelihood but may not be the most useful in practice, leading to sub-optimal results, and require significant computing and memory resources.
Innovation Solution
A method combining a look-ahead tree search, such as Monte Carlo tree search, with a sequence-to-sequence neural network system to generate output sequences, using a value neural network to evaluate sequences and guide the tree search for improved metric-based output generation, allowing for more diverse and desirable sequence outputs with reduced computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If maximum likelihood training is used to generate output sequences, then the sequence model likelihood is improved, but the usefulness and performance according to practical metrics deteriorates
Solution Approach 1:
The patent changes the objective function parameter from maximum likelihood to metric-based optimization. The value neural network is trained to predict sequence-level metrics (such as BLEU score, ROUGE score, or other task-specific metrics) rather than token-level likelihoods. This parameter change allows the model to directly optimize for practical usefulness rather than statistical probability.
Solution Approach 2:
The patent introduces feedback through the value neural network that evaluates partial sequences and provides guidance back to the sequence generation process. The value network predicts the expected metric score for a given sequence, and this feedback is used to guide beam search or other decoding strategies to prefer sequences with higher predicted metric values, creating a closed-loop optimization system.
2Device complexity
If traditional sequence transduction methods are used, then implementation simplicity is maintained, but computing and memory requirements increase
Solution Approach 1:
The patent applies preliminary action by training the value neural network in advance to predict metric scores for sequences. During generation, instead of computing expensive metric evaluations on finished sequences or performing exhaustive search, the pre-trained value network provides quick predictions that guide the generation process, significantly reducing computational resources required during inference while maintaining implementation simplicity.
Data Source
AI summary
A computer-implemented method for generating an output token sequence from an input token sequence. The method combines a look ahead tree search, such as a Monte Carlo tree search, with a sequence-to-sequence neural network system. The sequence-to-sequence neural network system has a policy output defining a next token probability distribution, and may include a value neural network providing a value output to evaluate a sequence. An initial partial output sequence is extended using the look ahead tree search guided by the policy output and, in implementations, the value output, of the sequence-to-sequence neural network system until a complete output sequence is obtained.