Translation Model Loss Adjustment for Repetition Hallucination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Translation models struggle to distinguish between repetitive content caused by translation hallucination and inherent repetition in the input information, leading to reduced translation quality.
Innovation Solution
Adjust the translation loss of the model based on the similarity between the contribution degrees of input tokens to different output tokens, reflecting the probability of translation hallucination, to optimize the suppression intensity of repetitive content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the translation model is trained to minimize the probability of generating repetitive content, then translation hallucination is reduced, but inherent repetitions in the input information are mistakenly suppressed, leading to a decline in translation quality
Solution Approach 1:
The patent applies local quality by differentiating the treatment of repetitive content based on its source. The loss function adjusts penalties locally: high penalty for repetitions caused by translation hallucination (low contribution degree from input tokens), and low or no penalty for repetitions inherent in the input information (high contribution degree). This localized differentiation resolves the contradiction by enabling precise, context-aware suppression of translation hallucination while preserving legitimate repetitions.
Solution Approach 2:
The patent changes the parameter of loss calculation by introducing a contribution degree metric that measures how much each input token contributes to output tokens. The loss function dynamically adjusts based on this parameter, comparing contribution degrees between current and previous output tokens. This parameter change enables the model to distinguish between harmful repetitions (translation hallucination) and legitimate repetitions (input-inherent), resolving the detection accuracy issue while maintaining translation quality.
2Object-generated harmful factors
If the translation model suppresses repetitive content to avoid translation hallucination, then the probability of translation hallucination decreases, but translation quality deteriorates due to misjudgment of inherent repetitions
Solution Approach 1:
The patent applies dynamics by making the suppression mechanism adaptive rather than static. The loss function dynamically adjusts the suppression intensity based on the contribution degree comparison. When the model detects that output tokens have low contribution from input tokens (indicating translation hallucination), it applies strong suppression. When output tokens have high contribution from input tokens (indicating legitimate translation), it applies minimal or no suppression. This dynamic adjustment resolves the contradiction by enabling context-sensitive suppression that eliminates translation hallucination while preserving translation quality.
Data Source
AI summary
A translation model training method comprises: acquiring a first translation loss, which is positively correlated with a probability that a target output token and a preceding output token are the same token, the target output token being the token expected to be output when translating a plurality of input tokens included in input information, and the preceding output token being the token obtained by the translation model when translating the plurality of input tokens before obtaining the target output token; acquiring a first contribution degree of the plurality of input tokens to the target output token and a second contribution degree of the plurality of input tokens to the preceding output token; adjusting the first translation loss based on a similarity between the first contribution degree and the second contribution degree to obtain a second translation loss; and training the translation model based on the second translation loss.