Text Generation Scoring With Repetition Penalties for Diverse Outputs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text generation methods using beam search result in high similarity among generated texts, leading to low diversity.
Innovation Solution
A method that involves generating a plurality of candidate texts by using candidate words as suffixes of previous texts, identifying repeated words, applying penalty processing on their scores, and selecting a target text set based on set scores to enhance diversity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If beam search is used to generate texts, then the conditional probability value of the generated text sequence is maximized, but the diversity of the generated texts becomes low
Solution Approach 1:
The patent changes the scoring parameter by introducing a penalty mechanism. Instead of using only the original conditional probability scores, the system subtracts penalty values from scores of repeated words, creating a modified scoring system that balances probability maximization with diversity enhancement through parameter transformation
Solution Approach 2:
The patent implements feedback by analyzing the generated text set to identify repeated words, calculating penalty values based on repetition frequency, and using this feedback to adjust the scoring of candidate texts. This closed-loop feedback mechanism guides the selection process toward more diverse outcomes while maintaining quality
Data Source
Figure 1~2
Figure 3
Figure 4~6
AI summary
Disclosed in embodiments of the present application are a text generation method and apparatus, a device, and a storage medium. The method comprises: acquiring first texts generated at a previous moment; determining a plurality of first candidate words corresponding to each first text, and using each first candidate word as a suffix of the first text to generate a plurality of first candidate texts; traversing and combining the plurality of first candidate texts corresponding to the preset number of first texts to determine a plurality of candidate text sets; determining, based on each word in each first candidate text, repeated words in each candidate text set, and performing penalty processing on word scores corresponding to the repeated words to determine target scores corresponding to the repeated words; determining, based on the target scores corresponding to the repeated words and word scores corresponding to non-repeated words in each candidate text set, a set score corresponding to each candidate text set; and determining a target text set based on the set score, and determining second texts generated at a current moment based on the target text set.