Neural Unlikelihood Training for Diverse Keyphrase Generation

The patent applies preliminary action by performing unlikelihood training during the model training phase to preemptively reduce repetition. The model is trained with a loss function that penalizes repetitive token generation, so that when inference occurs, the model inherently produces diverse keyphrases without requiring exhaustive beam search or post-processing deduplication. This shifts the diversity-enhancing action to the training stage rather than the inference stage.

The patent converts the harmful effect of repetition into a benefit by using unlikelihood training. The loss function specifically targets repetitive tokens and penalizes them, turning the problem of repetition into a learning signal that guides the model to generate diverse keyphrases. The harmful redundancy is transformed into a constructive training objective that improves diversity while reducing computational waste.

The patent performs the deduplication action preliminarily during training through unlikelihood training. The model learns to avoid generating repetitive keyphrases by being penalized for doing so during training. This preliminary learning eliminates the need for post-processing deduplication, as the model produces diverse keyphrases directly during inference, saving significant processing time.

The model performs self-service by inherently generating diverse keyphrases without requiring external post-processing intervention. Through unlikelihood training, the model develops an internal mechanism to avoid repetition, making it self-sufficient in producing high-quality diverse keyphrases without needing additional processing steps to remove duplicates.

The patent replaces mechanical rule-based systems with a neural network-based sequence-to-sequence model trained with unlikelihood training. This substitution transitions from rigid, pre-programmed rules to a learned, adaptive system that can generate diverse and context-appropriate keyphrases. The neural model captures complex patterns in the data that rule-based systems cannot, improving both diversity and quality while maintaining computational efficiency.

The patent changes the fundamental parameter of how keyphrases are generated - from fixed rule-based selection to probabilistic neural generation with unlikelihood training. By modifying the generation mechanism to use a trained model with a specialized loss function that penalizes repetition, the system achieves high diversity without sacrificing extraction speed, as the model generates keyphrases directly during inference.

The patent applies anti-weight by introducing a counterbalancing force against repetition through the unlikelihood training loss function. While standard training objectives push the model toward accurate predictions (which can lead to repetition), the unlikelihood loss acts as a counterweight that penalizes repetitive tokens. This creates a balanced optimization process where the model learns to generate both accurate and diverse keyphrases simultaneously.

The patent introduces dynamics by making the training objective adaptive and multi-faceted. Instead of a static single objective, the model is trained with a dynamic loss function that combines accuracy-oriented terms with diversity-oriented unlikelihood penalties. This dynamic training approach allows the model to adaptively balance between generating accurate keyphrases and maintaining diversity, adjusting its predictions based on both quality and repetition considerations.