A machine translation hallucination detection method based on a large language model
By combining Hidden Markov Models and Large Language Models, word alignment relationships are established and multi-dimensional risk scoring is optimized. This solves the accuracy and applicability issues of machine translation hallucination detection in large language model systems, and achieves more efficient machine translation hallucination recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- EAST CHINA JIAOTONG UNIVERSITY
- Filing Date
- 2026-05-06
- Publication Date
- 2026-06-02
AI Technical Summary
Existing machine translation illusion detection methods lack accuracy and applicability in systems based on large language models, and are unable to effectively identify semantically added, replaced, or fictitious information in machine translations.
We employ a large language model-based approach, establishing word alignment relationships through a Hidden Markov Model, searching for semantically matching, conflicting, and missing word pairs using the large language model, optimizing a multi-dimensional risk scoring template using an error response sample set, decomposing machine translation hallucinations into seven risk probabilities, and finally predicting machine translation hallucinations using empirical thresholds.
It significantly improves the accuracy and applicability of machine translation hallucination detection. Experimental results show that its performance is superior to several well-known methods, thus enhancing the effectiveness of machine translation hallucination detection.
Smart Images
Figure CN122133039A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of natural language processing technology, specifically to a machine translation illusion detection method based on a large language model. Background Technology
[0002] Machine translation hallucination refers to the presence of translation information in the output of a machine translation system that is completely unrelated to the source language sentence. It is considered an outward manifestation of low-quality machine translation and a specific example of "pathological" translations produced by neural machine translation systems and large language model-based machine translation systems. It not only affects translation accuracy but also significantly undermines user confidence in machine translations. Compared to traditional translation errors, machine translation hallucinations typically do not manifest as grammatical or lexical deviations, but rather as the addition, substitution, or fabrication of information at the semantic level of the source language sentence. Because its semantic shifts often lack stable, observable surface features, it is particularly difficult to accurately identify by human or automated methods, especially in machine translation scenarios with significant language differences.
[0003] Existing research on automatic detection of machine translation hallucinations can be broadly categorized into white-box and black-box methods. White-box methods typically rely on signals within the translation model, such as attention distribution, word-level probabilities, or decoding uncertainties, to analyze the machine translation generation process. These internal signals are used to identify and detect offsets related to machine translation hallucinations. In structure-accessible machine translation systems, these methods can achieve detection without additional external models. However, white-box methods are highly dependent on the model's internal structure, making them difficult to directly adapt to machine translation systems based on large language models where internal signals are not readily accessible, thus limiting their application. Black-box methods, on the other hand, rely solely on semantic consistency between the machine translation and the source language sentence. They utilize unsupervised models, quality filters, pre-trained language models, or large language models as external detection tools to predict the presence of hallucinations in the machine translation. Although they do not depend on the translation model's internal parameters and have high versatility, the optimization objectives of their underlying models deviate from the hallucination detection task. Summary of the Invention
[0004] The purpose of this application is to address the above-mentioned shortcomings by providing a machine translation hallucination detection method based on a large language model, so as to improve the accuracy and applicability of existing machine translation hallucination detection methods.
[0005] The technical solution adopted in this invention is as follows: A machine translation hallucination detection method based on a large language model, the steps of which are as follows: Step S1: Select a pre-defined large language model prompt template based on the translation direction; Step S2: Establish word alignment relationships between the machine translation and the source language sentence; Hidden Markov Models are used to establish word alignment relationships between machine translations and source language sentences; Step S3: Search for word pairs that match semantically, conflict semantically, or omit semantically between the machine translation and the source language sentence; The large language model uses its own knowledge and combines the word alignment relationship between the machine translation and the source language sentence obtained in step S2 to search for semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence; Step S4: Utilize feedback information from the error response sample set to perform machine translation hallucination detection, multi-dimensional risk scoring, large language model prompts, and automatic template optimization. Step S5: Machine translation of hallucination risk probability score across seven dimensions; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the large language model prompt template optimized in step S4 is used to analyze the probability of semantic addition risk, syntactic logic addition risk, format noise risk, over-translation risk, factual error risk, implicit fiction risk and fabrication risk in the machine translation. Step S6: Predict whether machine translation illusion exists in the machine translation; Based on the seven dimensions of machine translation hallucination risk probability in step S5, calculate the overall probability of machine translation hallucination in the machine translation, and use empirical thresholds to predict whether machine translation hallucination exists in the machine translation.
[0006] Furthermore, in step S1, a pre-defined large language model prompt template is selected based on the translation direction, specifically as follows: Based on the language types of the source sentences in the translation and the language types of the machine translation, a pre-defined large language model prompt template is selected using the differences in language types: ; Where Route() represents the mapping function, l s and l t These represent the language types of the source language sentence and the machine translation, respectively. P near , P far , P spec This represents the sets of general language pairs, long-distance language pairs, and special language pairs, categorized according to their linguistic differences. T gen , T dist, T spec This represents the prompt templates for general large language models, long-distance large language models, and large language models for special language phenomena.
[0007] Furthermore, in step S2, word alignment relationships are established between the machine translation and the source language sentence, specifically as follows: Step S21: Input the machine translation and the source language sentence into the cross-lingual pre-trained language model XLM-R, respectively, and extract the vectorized representations of the words: ; ; in, v s , v h Let XLM-R() represent the vectorized representations of words in the source language sentence and the machine translation, respectively (for words containing multiple sub-words, the vectorized representation is the average of all sub-word representations), and let XLM-R() represent the output function of the cross-lingual pre-trained language model XLM-R. s 1 , s 2 , s m These represent the 1st word, 2nd word, and mth word in the source language sentence, respectively. vs 1 , vs 2 , vs m These represent the vectorized representations of the first word, the second word, and the m-th word in the source language sentence, respectively. h 1 , h 2 , h n These represent the 1st word, 2nd word, and nth word in the machine translation, respectively. vh 1 , vh 2 , vh n These represent the vectorized representations of the first word, the second word, and the nth word in the machine translation, respectively. m , n These represent the number of words in the source language sentence and the machine translation, respectively. Step S22: Use a hidden Markov model to establish an optimal word alignment model between words in the machine translation and words in the source language sentence; Assuming the words in the source language sentence represent the states of the Hidden Markov Model (HMM), the words in the machine translation represent the observation sequence of the HMM, and the mapping relationship between the words in the source language sentence and the machine translation represents the hidden states, the state transition probabilities of the HMM are: ; in, p ( j|i ) represents the state transition probability. i Indicates the source language sentence number i Each position value j This indicates the sentence in the machine translation that is different from the sentence in the source language. i The alignment value of each word; The emission probability of the Hidden Markov Model is: ; in, p ( h j | s i ) represents the emission probability, and CosineSimilarity() is the cosine similarity calculation function. s i , h j These represent the first and second sentences of the source language, respectively. i The word and the machine translation in the first word j One word, vs i , vh j These represent the first and second sentences of the source language, respectively. i Vectorized representation of the first word and the first word in machine translation j Vectorized representation of each word, where the symbol || represents the vector modulus operation; Therefore, the optimal word alignment relationship between words in the machine translation and words in the source language sentence is: ; Where A represents the optimal word alignment between the words in the machine translation and the words in the source language sentence. argmax This indicates a search for the maximum value. Step S23: Use the expectation-maximization algorithm to search for all possible alignment relationships between words in the machine translation and words in the source language sentence, and obtain the optimal word alignment between words in the machine translation and words in the source language sentence as the word alignment relationship between the machine translation and the source language sentence; Step S24: Convert the word alignment relationship between the machine translation and the source language sentence into a word alignment relationship string: ; in,String A The string representing word alignment relationship; a1 , a2 , am These represent the positions of the words in the machine translation that are aligned with the 1st, 2nd, and mth words in the source language sentence, respectively. h a1 , h a2 , h am These represent the first and second parts of the machine translation. a1 The word, the first a2 The word, the first am One word.
[0008] Furthermore, step S3 searches for semantically matched, semantically conflicting, and semantically missing word pairs between the machine translation and the source language sentence, specifically: Step S31: Input the word alignment relationship string from step S24 into the large language model. The large language model searches for semantically matching word pairs between the machine translation and the source language sentence based on the prompts. ; in, Set match Word pairs representing semantic matching between machine translation and source language sentences. prompt match LLM() is the output function of the large language model, which searches for semantically matching word pairs between the machine translation and the source language sentence. s is the source language sentence and h is the machine translation to be detected for hallucination. Step S32: Input the word alignment relationship string from step S24 into the large language model. The large language model searches for word pairs with semantic conflicts between the machine translation and the source language sentence based on the prompts. ; in, Set conflict Word pairs that indicate semantic conflicts between the machine translation and the source language sentence. prompt conflict To provide hints for semantically conflicting word pairs between machine translations and source language sentences in large language models; Step S33: Input the word alignment relationship string from step S24 into the large language model. The large language model searches for words omitted in the translation of the source language sentence according to the prompts, and combines the words omitted in the translation with "NULL" to form semantically omitted word pairs. ; in, Set omitWord pairs indicating semantic omissions between the machine translation and the source language sentence. prompt omit This provides hints for semantically missing word pairs between the machine translation and the source language sentence in a large language model. NULL indicates the missing word in the source language.
[0009] Furthermore, step S4 utilizes the feedback information from the error response sample set to perform machine translation hallucination detection, multi-dimensional risk scoring, large language model prompts, and automatic template optimization, specifically as follows: Step S41: Collect the sample set of incorrect responses predicted by the initial prompts of the large language model; Input some samples with multi-dimensional risk labels for machine translation illusion detection into a large language model to predict risk scores; ; ; in, prompt halu This represents the initial machine translation hallucination detection multi-dimensional risk scoring big language model prompt template. X This represents the entire sample consisting of all source language sentences and machine translations. X i , X j They represent the first and second samples in the sample set, respectively. i The first sample and the first j One sample, Y This represents the true labels of all samples consisting of all source language sentences and machine translations. Y i , Y j They represent the first and second samples in the sample set, respectively. i The true label of the sample and the first j The true labels of the samples, X={( s i , h i , Set match i , Set omit i , Set conflict i )} 1 N ,symbol s i , h i , Set match i , Setomit i , Set conflict i They represent the first and second samples in the sample set, respectively. i Source language sentence, first i The machine translation, the first i The machine translation and the first i Word pairs that are semantically matched between source language sentences, the first i The machine translation and the first i Word pairs with semantic conflicts between sentences in the source language, the first i The machine translation and the first i Semantic missing word pairs between sentences in the source language N This represents the number of samples, with the subscript 1 indicating that the count starts from 1. Y ={ y i}1 N ,sample x i The real label is y i If the initial machine translation hallucination detection, multi-dimensional risk scoring, and large language model prompt template prediction results match the true label, then the predicted label is... Otherwise, the predicted label is ; Represents the set of all correctly predicted samples. This represents the set of all incorrectly predicted samples. Collecting all incorrectly predicted samples is used as the error response sample set of the large language model. Step S42: Perform error analysis using a large language model; The initial machine translation hallucination detection multidimensional risk scoring big language model prompt template is combined with the predicted error sample, the real label, and the predicted label to form the tuple required for the error analysis stage. By using the error analysis prompt, the tuple is input into the big language model to obtain the analysis result of the original prompt judging the error. ; ; in, prompt analyze For error analysis prompts, For the first error response sample set of the large language model m One sample, reasons This is the analysis result of the original prompt indicating an error. Step S43: Generate a new machine translation hallucination detection multi-dimensional risk scoring big language model prompt template through two iterations; Input the initial machine translation hallucination detection multidimensional risk scoring big language model prompt template and error analysis results into the big language model to generate a new machine translation hallucination detection multidimensional risk scoring big language model prompt template; ; in, prompt gen To provide suggestions for generating new machine translation hallucination detection multidimensional risk scoring big language model prompt templates, prompt' A prompt template for the generated machine-translated hallucination detection multidimensional risk scoring large language model; The generated machine-translated hallucination detection multi-dimensional risk score big language model prompt template and big language model error response sample set are compared with the first... m Each sample underwent error analysis again; ; in, reasons’ Re-analyze the error results for the m-th sample; The generated machine translation hallucination detection multidimensional risk scoring big language model prompt template and error analysis results are input into the big language model to iteratively generate a new machine translation hallucination detection multidimensional risk scoring big language model prompt template; ; in, prompt This serves as a prompt template for a new machine-translated hallucination detection multidimensional risk scoring large language model, generated after two iterations. Step S44: Decompose the machine translation hallucination detection multi-dimensional risk scoring big language model prompt template into seven dimensions of machine translation hallucination detection big language model prompt; Generally, machine translation illusions include seven aspects: semantic addition, syntactic logic addition, format noise, overtranslation, factual errors, implicit fiction, and fabricated information. Therefore, a large language model is used to score the risk of machine translation illusions in seven dimensions. The machine translation illusion detection multi-dimensional risk scoring large language model prompt template is further decomposed into seven dimensions of machine translation illusion detection large language model prompts.
[0010] in, prompt semadd Add risk warnings from a language model to the semantics of hallucination detection in machine translation. prompt synlogadd Add risky language model hints to the syntactic logic of illusion detection in machine translation. prompt formnoise The language model provides a high risk of format noise in machine translation hallucination detection. prompt overtranA language model provides a high risk of overtranslation in machine translation hallucination detection. prompt facterr A language model suggests a high risk of factual errors in machine translation hallucination detection. prompt imfab Large language model suggestions for detecting hidden fictional risks in machine translation hallucinations. prompt fabinf The language model provides a high risk of fabricated information in machine translation hallucination detection.
[0011] Furthermore, step S5 involves a seven-dimensional machine translation hallucination risk probability score, specifically: Step S51: Analyze the probability of semantic addition risk in machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of semantic addition risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score semadd Add risk probabilities to the semantics present in machine translations. prompt semadd Add risk warnings from a language model to the semantics of hallucination detection in machine translation. s , h , l s , l t , Set match , Set omit , Set conflict These are: the source language sentence, the machine translation to be detected for hallucination, the language type of the source language sentence, the language type of the machine translation, the semantically matched word pairs between the machine translation and the source language sentence obtained in step S3, the semantically missing word pairs, and the semantically conflicting word pairs. Step S52: Analyze the probability of syntactic and logical risk additions in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of syntactic logic addition risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score senlogaddAdd risk probabilities to the syntactic logic present in machine translations; Step S53: Analyze the probability of format noise risk in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of format noise risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score formnoise This represents the probability of format noise risk in machine translation. Step S54: Analyze the probability of overtranslation risk in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of overtranslation risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score overtran This represents the probability of overtranslation risk in machine-translated text. Step S55: Analyze the probability of factual errors in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of factual error risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score overtran This represents the probability of factual errors in machine translation. Step S56: Analyze the probability of implicit fiction risk in machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of implicit fiction risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score imfab The probability of implicit fictional risks in machine translation; Step S57: Analyze the probability of fabricated information risk in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of fabricated information risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score fabinf This represents the probability of fabricated information in machine-translated text.
[0012] Furthermore, step S6 predicts whether machine translation illusion exists in the machine translation; Based on the seven dimensions of machine translation hallucination risk probability in step S5, calculate the overall probability of machine translation hallucination in the machine translation; Wherein, score is the overall probability of machine translation illusion in machine translation; Predicting the existence of machine translation illusions in machine translations using empirical thresholds; ; Among them, symbols Indicates the result of hallucination prediction. λ The threshold for classifying hallucinations in machine translation is empirically set at 0.53.
[0013] The beneficial effects of this invention are as follows: This invention decomposes the machine translation hallucination detection method based on a large language model into multiple progressive sub-tasks. From a bottom-up perspective, this invention first uses a Hidden Markov Model to establish word alignment relationships between the machine translation to be hallucinated and the source language sentence. Based on these word alignment relationships, a large language model is used to search for semantically matched, semantically conflicting, and semantically missing word pairs between the machine translation and the source language sentence. Second, feedback information from the error response sample set is used to automatically optimize the large language model prompt template for multi-dimensional risk scoring of machine translation hallucination detection. The optimized large language model prompt template is then used to analyze the risk probability of seven specific forms of machine translation hallucination in the machine translation, namely, semantic addition risk probability, syntactic logic addition risk probability, format noise risk probability, overtranslation risk probability, factual error risk probability, implicit fiction risk probability, and fabricated information risk probability. Finally, the risk probability scores of the seven dimensions of machine translation hallucination are summarized, and the existence of machine translation hallucination in the machine translation is predicted. Experimental results show that the performance of the method of this invention surpasses that of several well-known machine translation hallucination detection methods, significantly improving the machine translation hallucination detection effect. Attached Figure Description
[0014] Figure 1This is a schematic diagram of the machine translation hallucination detection method based on a large language model according to the present invention. Detailed Implementation
[0015] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0016] like Figure 1 As shown, this embodiment works as follows: A machine translation hallucination detection method based on a large language model, with the following steps: Step S1: Select a pre-defined large language model prompt template based on the translation direction; Step S2: Establish word alignment relationships between the machine translation and the source language sentence; Hidden Markov Models are used to establish word alignment relationships between the machine translation of the hallucination detection target and the source language sentence; Step S3: Search for word pairs that match semantically, conflict semantically, or omit semantically between the machine translation and the source language sentence; The large language model uses its own knowledge and combines the word alignment relationship between the machine translation and the source language sentence obtained in step S2 to search for semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence; Step S4: Utilize feedback information from the error response sample set to perform machine translation hallucination detection, multi-dimensional risk scoring, large language model prompts, and automatic template optimization. Step S5: Machine translation of hallucination risk probability score across seven dimensions; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the large language model prompt template optimized in step S4 is used to analyze the probability of semantic addition risk, syntactic logic addition risk, format noise risk, over-translation risk, factual error risk, implicit fiction risk and fabrication risk in the machine translation. Step S6: Predict whether machine translation illusion exists in the machine translation; Based on the seven dimensions of machine translation hallucination risk probability in step S5, calculate the overall probability of machine translation hallucination in the machine translation, and use empirical thresholds to predict whether machine translation hallucination exists in the machine translation. Furthermore, in step S1, a pre-defined large language model prompt template is selected based on the translation direction, specifically as follows: Based on the language types of the source sentences in the translation and the language types of the machine translation, a pre-defined large language model prompt template is selected using the differences in language types: ; Where Route() represents the mapping function, ls and l t These represent the language types of the source language sentence and the machine translation, respectively. P near , P far , P spec This represents a set of language pairs categorized by linguistic differences: general large languages, long-distance large languages, and special languages. T gen , T dist , T spec These represent general large language model prompt templates, long-distance large language model prompt templates, and large language model prompt templates for special language phenomena. Furthermore, in step S2, word alignment relationships are established between the machine translation and the source language sentence, specifically as follows: Step S21: Input the machine translation and the source language sentence into the cross-lingual pre-trained language model XLM-R, respectively, and extract the vectorized representations of the words: ; ; in, v s , v h Let XLM-R() represent the vectorized representations of words in the source language sentence and the machine translation, respectively (for words containing multiple sub-words, the vectorized representation is the average of all sub-word representations), and let XLM-R() represent the output function of the cross-lingual pre-trained language model XLM-R. s 1 , s 2 , s m These represent the first and second parts of the sentence in the source language, respectively. 1 The word, the first 2 The word, the first m One word; vs 1 , vs 2 , vs m These represent the first and second parts of the sentence in the source language, respectively. 1 The vectorized representation of the nth word, the nth 2 The vectorized representation of the nth word, the nth m Vectorized representation of each word; h 1 , h 2 , h nThese represent the first and second parts of the machine translation. 1 The word, the first 2 The word, the first n One word; vh 1 , vh 2 , vh n These represent the first and second parts of the machine translation. 1 The vectorized representation of the nth word, the nth 2 The vectorized representation of the nth word, the nth n Vectorized representation of each word; m , n These represent the number of words in the source language sentence and the machine translation, respectively. Step S22: Use a hidden Markov model to establish an optimal word alignment model between words in the machine translation and words in the source language sentence; Assuming the words in the source language sentence represent the states of the Hidden Markov Model (HMM), the words in the machine translation represent the observation sequence of the HMM, and the mapping relationship between the words in the source language sentence and the machine translation represents the hidden states, the state transition probabilities of the HMM are: ; in, p ( j|i ) represents the state transition probability. i Indicates the source language sentence number i Each position value j This indicates the sentence in the machine translation that is different from the sentence in the source language. i The alignment value of each word; The emission probability of the Hidden Markov Model is: ; in, p ( h j | s i ) represents the emission probability, and CosineSimilarity() is the cosine similarity calculation function. s i , h j These represent the first and second sentences of the source language, respectively. i The word and the machine translation in the first word j One word, vs i , vh j These represent the first and second sentences of the source language, respectively. i Vectorized representation of the first word and the first word in machine translation jVectorized representation of each word, where the symbol || represents the vector modulus operation; Therefore, the optimal word alignment relationship between words in the machine translation and words in the source language sentence is: ; Where A represents the optimal word alignment between the words in the machine translation and the words in the source language sentence. argmax This indicates a search for the maximum value. Step S23: Use the Expectation-Maximization Algorithm to search for all possible alignment relationships between words in the machine translation and words in the source language sentence to obtain the optimal word alignment between words in the machine translation and words in the source language sentence as the word alignment relationship between the machine translation and the source language sentence. Step S24: Convert the word alignment relationship between the machine translation and the source language sentence into a word alignment relationship string: ; in, String A The string representing word alignment relationship. s 1 , s 2 , s m These represent the first and second parts of the sentence in the source language, respectively. 1 The word, the first 2 The word, the first m One word; a1 , a2 , am These respectively represent the first and second parts of the sentence in the source language. 1 The word, the first 2 The word, the first m The position of words in the machine translation with word alignment; h a1 , h a2 , h am These represent the first and second parts of the machine translation. a1 The word, the first a2 The word, the first am One word.
[0017] Furthermore, step S3 searches for semantically matched, semantically conflicting, and semantically missing word pairs between the machine translation and the source language sentence, specifically: Step S31: Input the word alignment relationship string from step S24 into the large language model. The large language model searches for semantically matching word pairs between the machine translation and the source language sentence based on the prompts. ; in, Set match Word pairs representing semantic matching between machine translation and source language sentences. prompt match LLM() is the output function of the large language model, which provides hints for semantically matching word pairs between the machine translation and the source language sentence. Step S32: Input the word alignment relationship string from step S24 into the large language model. The large language model searches for word pairs with semantic conflicts between the machine translation and the source language sentence based on the prompts. ; in, Set conflict Word pairs that indicate semantic conflicts between the machine translation and the source language sentence. prompt conflict To provide hints for semantically conflicting word pairs between machine translations and source language sentences in large language models; Step S33: Input the word alignment relationship string from step S24 into the large language model. The large language model searches for words omitted in the translation of the source language sentence according to the prompts, and forms a semantically omitted word pair with "NULL". ; in, Set omit Word pairs indicating semantic omissions between the machine translation and the source language sentence. prompt omit This provides hints for semantically missing word pairs between the machine translation and the source language sentence in a large language model. NULL indicates the missing word in the source language.
[0018] Furthermore, step S4 utilizes the feedback information from the error response sample set to perform machine translation hallucination detection, multi-dimensional risk scoring, large language model prompts, and automatic template optimization, specifically as follows: Step S41: Collect the sample set of incorrect responses predicted by the initial prompts of the large language model; Input some samples with multi-dimensional risk labels for machine translation illusion detection into a large language model to predict risk scores; ; ; in, prompt halu This represents the initial machine translation hallucination detection multi-dimensional risk scoring big language model prompt template. X This represents the entire sample consisting of all source language sentences and machine translations. X i ,X j They represent the first and second samples in the sample set, respectively. i Sample and the first j Sample, Y This represents the true labels of all samples consisting of all source language sentences and machine translations. Y i , Y j They represent the first and second samples in the sample set, respectively. i The true label and the first sample j The true labels of the samples, X={( s i , h i , Set match i , Set omit i , Set conflict i )} 1 N ,symbol s i , h i , Set match i , Set omit i , Set conflict i They represent the first and second samples in the sample set, respectively. i Source language sentence, first i The machine translation, the first i The machine translation and the first i Word pairs that are semantically matched between source language sentences, the first i The machine translation and the first i Word pairs with semantic conflicts between sentences in the source language, the first i The machine translation and the first i Semantic missing word pairs between sentences in the source language N This represents the number of samples, with the subscript 1 indicating that the count starts from 1. Y ={ y i}1 N ,sample x i The real label is y i If the initial machine translation hallucination detection, multi-dimensional risk scoring, and large language model prompt template prediction results match the true label, then the predicted label is... Otherwise, the predicted label is ; Represents the set of all correctly predicted samples. This represents the set of all incorrectly predicted samples. Collecting all incorrectly predicted samples is used as the error response sample set of the large language model. Step S42: Perform error analysis using a large language model; The initial machine translation hallucination detection multi-dimensional risk scoring big language model prompt template is combined with the predicted error sample, its true label, and the predicted label to form the binary pair required for the error analysis stage. By using the error analysis prompt, this binary pair is input into the big language model to obtain the analysis results of the original prompt judging the error. ; ; in, prompt analyze For error analysis prompts, For the first error response sample set of the large language model m One sample, reasons For the analysis results; Step S43: Generate a new machine translation hallucination detection multi-dimensional risk scoring big language model prompt template through two iterations; Input the initial machine translation hallucination detection multidimensional risk scoring big language model prompt template and error analysis results into the big language model to generate a new machine translation hallucination detection multidimensional risk scoring big language model prompt template; ; in, prompt gen To provide suggestions for generating new machine translation hallucination detection multidimensional risk scoring big language model prompt templates, prompt' A prompt template for the generated machine-translated hallucination detection multidimensional risk scoring large language model; The generated machine-translated hallucination detection multi-dimensional risk score big language model prompt template and big language model error response sample set are compared with the first... m Each sample underwent error analysis again; ; in, reasons’ For the analysis results; The generated machine translation hallucination detection multidimensional risk scoring big language model prompt template and error analysis results are input into the big language model to iteratively generate a new machine translation hallucination detection multidimensional risk scoring big language model prompt template; ; in, promptA new machine translation hallucination detection multidimensional risk scoring big language model prompt template (2-shot prompt) generated after two iterations. Step S44: Decompose the machine translation hallucination detection multi-dimensional risk scoring big language model prompt template into seven dimensions of machine translation hallucination detection big language model prompt; Generally, machine translation illusions include seven aspects: semantic addition, syntactic logic addition, format noise, overtranslation, factual errors, implicit fiction, and fabricated information. Therefore, this patent uses a large language model to score the risk of machine translation illusions in seven dimensions. The machine translation illusion detection multi-dimensional risk scoring large language model prompt template is further decomposed into seven dimensions of machine translation illusion detection large language model prompts. ; in, prompt semadd Add risk warnings from a language model to the semantics of hallucination detection in machine translation. prompt synlogadd Add risky language model hints to the syntactic logic of illusion detection in machine translation. prompt formnoise The language model provides a high risk of format noise in machine translation hallucination detection. prompt overtran A language model provides a high risk of overtranslation in machine translation hallucination detection. prompt facterr A language model suggests a high risk of factual errors in machine translation hallucination detection. prompt imfab Large language model suggestions for detecting hidden fictional risks in machine translation hallucinations. prompt fabinf The language model provides a high risk of fabricated information in machine translation hallucination detection.
[0019] Furthermore, step S5 involves a seven-dimensional machine translation hallucination risk probability score, specifically: Step S51: Analyze the probability of semantic addition risk in machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of semantic addition risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score semadd Add risk probabilities to the semantics present in machine translations. prompt semaddAdd risk warnings from a language model to the semantics of hallucination detection in machine translation. s , h , l s , l t , Set match , Set omit , Set conflict These are: the source language sentence, the machine translation to be detected for hallucination, the source language type in the translation direction, the target language type in the translation direction, the semantically matched word pairs between the machine translation and the source language sentence obtained in step S3, the semantically omitted word pairs, and the semantically conflicting word pairs. Step S52: Analyze the probability of syntactic and logical risk additions in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of syntactic logic addition risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score senlogadd Add risk probabilities to the syntactic logic present in machine translations; Step S53: Analyze the probability of format noise risk in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of format noise risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score formnoise This represents the probability of format noise risk in machine translation. Step S54: Analyze the probability of overtranslation risk in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of overtranslation risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score overtran This represents the probability of overtranslation risk in machine-translated text. Step S55: Analyze the probability of factual errors in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of factual error risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score overtran This represents the probability of factual errors in machine translation.
[0020] Step S56: Analyze the probability of implicit fiction risk in machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of implicit fiction risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score imfab The probability of implicit fictional risks in machine translation; Step S57: Analyze the probability of fabricated information risk in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of fabricated information risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ; in, score fabinf This represents the probability of fabricated information in machine-translated text.
[0021] Furthermore, step S6 predicts whether machine translation illusion exists in the machine translation; Based on the seven dimensions of machine translation hallucination risk probability in step S5, calculate the overall probability of machine translation hallucination in the machine translation;
[0022] Wherein, score is the overall probability of machine translation illusion in machine translation; Predicting the existence of machine translation illusions in machine translations using empirical thresholds; ; Among them, symbols Indicates the result of hallucination prediction. λ The threshold for classifying hallucinations in machine translation is empirically set at 0.53.
[0023] The machine translation illusion detection method DHAR-Agent based on a large language model of this invention was tested on the publicly available machine translation illusion detection dataset HalOmi. Well-known machine translation illusion detection methods such as BLASER2-QE, COMET-QE, SONAR, XNLI, LOG_LOSS, and LaBSE were used as comparison methods.
[0024] Appendix Table 1: Overall Performance Comparison of Different Machine Translation Hallucination Detection Methods on the HalOmi Dataset
[0025] See Appendix Table 1: To verify the detection performance of the method of this invention, the machine translation hallucination detection method based on a large language model (denoted as DHAR-Agent) described in this invention was tested on the publicly available machine translation hallucination detection dataset HalOmi, and typical existing machine translation hallucination detection methods BLASER2-QE, COMET-QE, SONAR, XNLI, LOG_LOSS, and LaBSE were selected as comparison methods. The evaluation metrics used in the test included ROC-AUC, PR-AUC, F1 score, precision, and accuracy. The higher the value of each metric, the better the detection performance. The test results show that the method of this invention, DHAR-Agent, is superior to all the comparison methods in all four metrics: ROC-AUC, F1 score, precision, and accuracy. It is only slightly lower than the best comparison method, SONAR, in the PR-AUC metric, with a difference of only 0.04. Experimental results fully demonstrate that the present invention, through its overall strategy of explicitly constructing word alignment relationships, automatically optimizing prompt templates based on error sample feedback, and scoring hallucination risk from seven dimensions, can significantly improve the accuracy and reliability of machine translation hallucination detection.
[0026] In performance measurement, the main evaluation metrics used are the area under the overall receiver operating characteristic curve (ROC-AUC), the area under the precision-recall curve (PR-AUC), the harmonic mean of precision and recall (F1), precision, and accuracy. These metrics are used to evaluate the performance of machine translation hallucination detection methods from different perspectives. Larger values for ROC-AUC, PR-AUC, F1, precision, and accuracy indicate better system performance.
[0027] Appendix Table 2 presents a comparison of the overall performance of different machine translation hallucination detection methods on the HalOmi dataset. The data in Table 2 shows that the DHAR-Agent machine translation hallucination detection method, based on a large language model, outperforms all the compared machine translation hallucination detection methods in terms of the area under the overall receiver operating characteristic curve (ROC-AUC), the harmonic mean of precision and recall (F1), and precision and accuracy. It only falls short of the best compared machine translation hallucination detection method, SONAR, by 0.04 (0.873 vs. 0.877), in the area under the precision-recall curve (PR-AUC) metric. This demonstrates that the machine translation illusion detection method based on the large language model disclosed in this paper is effective in its strategies of explicitly establishing word alignment relationships between the machine translation and the source language sentences, utilizing feedback information from the error response sample set for multi-dimensional risk scoring of machine translation illusion detection, automatically optimizing the large language model prompt template, and using the optimized large language model prompt template to analyze the risk probability of seven specific forms of machine translation illusion in the machine translation, namely semantic addition risk probability, syntactic logic addition risk probability, format noise risk probability, overtranslation risk probability, factual error risk probability, implicit fiction risk probability, and fabrication information risk probability.
[0028] Since the method disclosed herein is not proposed for a specific large language model, the method disclosed herein can be applied to all large language models, such as large language models Llama, ChatGPT, ChatGLM, Gemini, Zephyr, etc.
[0029] Since the method disclosed herein is not specific to two particular languages, it has universal applicability.
[0030] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of the inventive concept are included in this invention and are protected by the appended claims.
Claims
1. A machine translation hallucination detection method based on a large language model, characterized in that: The steps are as follows: Step S1: Select a pre-defined large language model prompt template based on the translation direction; Step S2: Establish word alignment relationships between the machine translation and the source language sentence; Hidden Markov Models are used to establish word alignment relationships between machine translations and source language sentences; Step S3: Search for word pairs that match semantically, conflict semantically, or omit semantically between the machine translation and the source language sentence; The large language model uses its own knowledge and combines the word alignment relationship between the machine translation and the source language sentence obtained in step S2 to search for semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence; Step S4: Utilize feedback information from the error response sample set to perform machine translation hallucination detection, multi-dimensional risk scoring, large language model prompts, and automatic template optimization. Step S5: Machine translation of hallucination risk probability score across seven dimensions; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the large language model prompt template optimized in step S4 is used to analyze the probability of semantic addition risk, syntactic logic addition risk, format noise risk, over-translation risk, factual error risk, implicit fiction risk and fabrication risk in the machine translation. Step S6: Predict whether machine translation illusion exists in the machine translation; Based on the seven dimensions of machine translation hallucination risk probability in step S5, calculate the overall probability of machine translation hallucination in the machine translation, and use empirical thresholds to predict whether machine translation hallucination exists in the machine translation.
2. The machine translation hallucination detection method based on a large language model according to claim 1, characterized in that: In step S1, a pre-defined large language model prompt template is selected based on the translation direction, specifically as follows: Based on the language types of the source sentences in the translation and the language types of the machine translation, a pre-defined large language model prompt template is selected using the differences in language types: ； Where Route() represents the mapping function, l s and l t These represent the language types of the source language sentence and the machine translation, respectively. P near , P far , P spec This represents the sets of general language pairs, long-distance language pairs, and special language pairs, categorized according to their linguistic differences. T gen , T dist , T spec This represents the prompt templates for general large language models, long-distance large language models, and large language models for special language phenomena.
3. The machine translation hallucination detection method based on a large language model according to claim 2, characterized in that: Step S2 establishes word alignment relationships between the machine translation and the source language sentence, specifically as follows: Step S21: Input the machine translation and the source language sentence into the cross-lingual pre-trained language model XLM-R, respectively, and extract the vectorized representations of the words: ； ； in, v s , v h Let XLM-R() represent the vectorized representations of words in the source language sentence and the machine translation, respectively (for words containing multiple sub-words, the vectorized representation is the average of all sub-word representations), and let XLM-R() represent the output function of the cross-lingual pre-trained language model XLM-R. s 1 , s 2 , s m These represent the 1st word, 2nd word, and mth word in the source language sentence, respectively. vs 1 , vs 2 , vs m These represent the vectorized representations of the first word, the second word, and the m-th word in the source language sentence, respectively. h 1 , h 2 , h n These represent the 1st word, 2nd word, and nth word in the machine translation, respectively. vh 1 , vh 2 , vh n These represent the vectorized representations of the first word, the second word, and the nth word in the machine translation, respectively. m , n These represent the number of words in the source language sentence and the machine translation, respectively. Step S22: Use a hidden Markov model to establish an optimal word alignment model between words in the machine translation and words in the source language sentence; Assuming the words in the source language sentence represent the states of the Hidden Markov Model (HMM), the words in the machine translation represent the observation sequence of the HMM, and the mapping relationship between the words in the source language sentence and the machine translation represents the hidden states, the state transition probabilities of the HMM are: ； in, p ( j|i ) represents the state transition probability. i Indicates the source language sentence number i Each position value j This indicates the sentence in the machine translation that is different from the sentence in the source language. i The alignment value of each word; The emission probability of the Hidden Markov Model is: ； in, p ( h j | s i ) represents the emission probability, and CosineSimilarity() is the cosine similarity calculation function. s i , h j These represent the first and second sentences of the source language, respectively. i The word and the machine translation in the first word j One word, vs i , vh j These represent the first and second sentences of the source language, respectively. i Vectorized representation of the first word and the first word in machine translation j Vectorized representation of each word, where the symbol || represents the vector modulus operation; Therefore, the optimal word alignment relationship between words in the machine translation and words in the source language sentence is: ； Where A represents the optimal word alignment between the words in the machine translation and the words in the source language sentence. argmax This indicates a search for the maximum value. Step S23: Use the expectation-maximization algorithm to search for all possible alignment relationships between words in the machine translation and words in the source language sentence, and obtain the optimal word alignment between words in the machine translation and words in the source language sentence as the word alignment relationship between the machine translation and the source language sentence; Step S24: Convert the word alignment relationship between the machine translation and the source language sentence into a word alignment relationship string: ； in, String A The string representing word alignment relationship; a1 , a2 , am These represent the positions of the words in the machine translation that are aligned with the 1st, 2nd, and mth words in the source language sentence, respectively. h a1 , h a2 , h am These represent the first and second parts of the machine translation. a1 The word, the first a2 The word, the first am One word.
4. The machine translation hallucination detection method based on a large language model according to claim 3, characterized in that: Step S3 searches for semantically matched, semantically conflicting, and semantically missing word pairs between the machine translation and the source language sentence, specifically: Step S31: Input the word alignment relationship string from step S24 into the large language model. The large language model searches for semantically matching word pairs between the machine translation and the source language sentence based on the prompts. ； in, Set match Word pairs representing semantic matching between machine translation and source language sentences. prompt match LLM() is the output function of the large language model, which searches for semantically matching word pairs between the machine translation and the source language sentence. s is the source language sentence and h is the machine translation to be detected for hallucination. Step S32: Input the word alignment relationship string from step S24 into the large language model. The large language model searches for word pairs with semantic conflicts between the machine translation and the source language sentence based on the prompts. ； in, Set conflict Word pairs that indicate semantic conflicts between the machine translation and the source language sentence. prompt conflict To provide hints for semantically conflicting word pairs between machine translations and source language sentences in large language models; Step S33: Input the word alignment relationship string from step S24 into the large language model. The large language model searches for words omitted in the translation of the source language sentence according to the prompts, and combines the words omitted in the translation with "NULL" to form semantically omitted word pairs. ； in, Set omit Word pairs indicating semantic omissions between the machine translation and the source language sentence. prompt omit This provides hints for semantically missing word pairs between the machine translation and the source language sentence in a large language model. NULL indicates the missing word in the source language.
5. The machine translation hallucination detection method based on a large language model according to claim 4, characterized in that: Step S4 utilizes feedback information from the error response sample set to perform machine translation, hallucination detection, multi-dimensional risk scoring, large language model prompts, and automatic template optimization, specifically as follows: Step S41: Collect the sample set of incorrect responses predicted by the initial prompts of the large language model; Input some samples with multi-dimensional risk labels for machine translation illusion detection into a large language model to predict risk scores; ； ； in, prompt halu This represents the initial machine translation hallucination detection multi-dimensional risk scoring big language model prompt template. X This represents the entire sample consisting of all source language sentences and machine translations. X i , X j They represent the first and second samples in the sample set, respectively. i The first sample and the first j One sample, Y This represents the true labels of all samples consisting of all source language sentences and machine translations. Y i , Y j They represent the first and second samples in the sample set, respectively. i The true label of the sample and the first j The true labels of the samples, X={( s i , h i , Set match i , Set omit i , Set conflict i )} 1 N ,symbol s i , h i , Set match i , Set omit i , Set conflict i They represent the first and second samples in the sample set, respectively. i Source language sentence, first i The machine translation, the first i The machine translation and the first i Word pairs that are semantically matched between source language sentences, the first i The machine translation and the first i Word pairs with semantic conflicts between sentences in the source language, the first i The machine translation and the first i Semantic missing word pairs between sentences in the source language N This represents the number of samples, with the subscript 1 indicating that the count starts from 1. Y ={ y i }1 N ,sample x i The real label is y i If the initial machine translation hallucination detection, multi-dimensional risk scoring, and large language model prompt template prediction results match the true label, then the predicted label is... Otherwise, the predicted label is ; Represents the set of all correctly predicted samples. This represents the set of all incorrectly predicted samples. Collecting all incorrectly predicted samples is used as the error response sample set of the large language model. Step S42: Perform error analysis using a large language model; The initial machine translation hallucination detection multidimensional risk scoring big language model prompt template is combined with the predicted error sample, the real label, and the predicted label to form the tuple required for the error analysis stage. By using the error analysis prompt, the tuple is input into the big language model to obtain the analysis result of the original prompt judging the error. ； ； in, prompt analyze For error analysis prompts, For the first error response sample set of the large language model m One sample, reasons This is the analysis result of the original prompt indicating an error. Step S43: Generate a new machine translation hallucination detection multi-dimensional risk scoring big language model prompt template through two iterations; Input the initial machine translation hallucination detection multidimensional risk scoring big language model prompt template and error analysis results into the big language model to generate a new machine translation hallucination detection multidimensional risk scoring big language model prompt template; ； in, prompt gen To provide suggestions for generating new machine translation hallucination detection multidimensional risk scoring big language model prompt templates, prompt A prompt template for the generated machine-translated hallucination detection multidimensional risk scoring large language model; The generated machine-translated hallucination detection multi-dimensional risk score big language model prompt template and big language model error response sample set are compared with the first... m Each sample underwent error analysis again; ； in, reasons Re-analyze the error results for the m-th sample; The generated machine translation hallucination detection multidimensional risk scoring big language model prompt template and error analysis results are input into the big language model to iteratively generate a new machine translation hallucination detection multidimensional risk scoring big language model prompt template; ； in, prompt This serves as a prompt template for a new machine-translated hallucination detection multidimensional risk scoring large language model, generated after two iterations. Step S44: Decompose the machine translation hallucination detection multi-dimensional risk scoring big language model prompt template into seven dimensions of machine translation hallucination detection big language model prompts.
6. The machine translation hallucination detection method based on a large language model according to claim 5, characterized in that: Step S5 involves a seven-dimensional machine translation hallucination risk probability score, specifically: Step S51: Analyze the probability of semantic addition risk in machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of semantic addition risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ； in, score semadd Add risk probabilities to the semantics present in machine translations. prompt semadd Add risk warnings from a language model to the semantics of hallucination detection in machine translation. s , h , l s , l t , Set match , Set omit , Set conflict These are: the source language sentence, the machine translation to be detected for hallucination, the language type of the source language sentence, the language type of the machine translation, the semantically matched word pairs between the machine translation and the source language sentence obtained in step S3, the semantically missing word pairs, and the semantically conflicting word pairs. Step S52: Analyze the probability of syntactic and logical risk additions in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of syntactic logic addition risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ； in, score senlogadd Add risk probabilities to the syntactic logic present in machine translations; Step S53: Analyze the probability of format noise risk in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of format noise risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ； in, score formnoise This represents the probability of format noise risk in machine translation. Step S54: Analyze the probability of overtranslation risk in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of overtranslation risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ； in, score overtran This represents the probability of overtranslation risk in machine-translated text. Step S55: Analyze the probability of factual errors in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of factual error risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ； in, score overtran This represents the probability of factual errors in machine translation. Step S56: Analyze the probability of implicit fiction risk in machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of implicit fiction risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ； in, score imfab The probability of implicit fictional risks in machine translation; Step S57: Analyze the probability of fabricated information risk in the machine translation using a large language model; Based on the semantic matching, semantic conflict and semantic omission word pairs between the machine translation and the source language sentence obtained in step S3, the probability of fabricated information risk in the machine translation is analyzed using the machine translation illusion detection multi-dimensional risk scoring big language model prompt template optimized in step S4. ； in, score fabinf This represents the probability of fabricated information in machine-translated text.
7. The machine translation hallucination detection method based on a large language model according to claim 6, characterized in that: Step S6 predicts whether machine translation illusion exists in the machine translation; Based on the seven dimensions of machine translation hallucination risk probability in step S5, calculate the overall probability of machine translation hallucination in the machine translation; Wherein, score is the overall probability of machine translation illusion in machine translation; Predicting the existence of machine translation illusions in machine translations using empirical thresholds; ； Among them, symbols Indicates the result of hallucination prediction. λ The threshold for classifying hallucinations in machine translation is empirically set at 0.53.