Best-K Graph Search for Diverse Neural Text Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing natural language processing (NLP) models generate repetitive and similar output text sequences due to beam search algorithms, lacking diversity and requiring high computational effort for alternative methods.
Innovation Solution
A graph-based NLP text decoding framework called best-k search, which uses a directed search graph with a greedy search algorithm and temporal decay to generate diverse and high-quality outputs efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If beam search algorithms are used for text generation, then decoding efficiency is improved, but output diversity deteriorates
Solution Approach 1:
The patent segments the search process by maintaining k separate candidate sequences instead of a single beam, allowing multiple diverse paths to be explored simultaneously. Each candidate sequence is independently scored and maintained, preventing the convergence to repetitive outputs while preserving decoding efficiency through structured candidate management.
Solution Approach 2:
The patent changes the selection parameter from purely probability-based (beam search) to a composite scoring mechanism that incorporates diversity metrics. By modifying the selection criteria to consider both likelihood and diversity, the system achieves better output variety without sacrificing decoding speed, as the scoring can be computed efficiently during the existing search process.
2Object-generated harmful factors
If alternative decoding methods are used to improve diversity, then output diversity is improved, but computational effort increases
Solution Approach 1:
The patent applies partial action by maintaining only k candidate sequences rather than exploring all possible sequences. This partial exploration achieves sufficient diversity for practical applications while avoiding the exponential computational cost of exhaustive search. The parameter k acts as a control that balances diversity against computational resources.
Solution Approach 2:
The patent performs preliminary scoring and filtering of candidate sequences at each decoding step, eliminating low-potential candidates before they can consume excessive computational resources. By pre-evaluating candidates using the composite score (combining likelihood and diversity metrics), the system avoids wasting computation on paths unlikely to produce diverse or high-quality outputs.
3Speed
If beam search is used for text generation, then decoding speed is maintained, but output quality deteriorates
Solution Approach 1:
The patent introduces feedback through the composite scoring mechanism that continuously evaluates candidate sequences based on both probability and diversity metrics. This feedback guides the selection of next tokens, ensuring that high-quality diverse outputs are generated. The feedback is computed efficiently during the decoding process, maintaining speed while improving quality through informed candidate selection.
Data Source
AI summary
Embodiments described herein provide systems and methods for text generation using a natural language processing model. Natural language input is received and encoded by an encoder. A decoder generates a plurality of next-node probabilities for a set of candidate nodes that are next to at least one of K previously decoded paths of nodes on a directed search graph. Scores are computed for each of the candidate nodes and done in parallel among the K previously decoded paths. The highest scoring K nodes are selected for addition to the directed search graph. The search continues until a search budget is exhausted. Generated sequences are output. These sequences may represent a plurality of sentences or questions, depending on the task.