Machine translation quality estimation method based on multi-semantic space

Through feature extraction and fusion of multisemantic space, combined with cross-language pre-training model X-MOD, the problems of insufficient feature accuracy of single semantic space and weak language representation ability are solved, and the effect of machine translation quality estimation and cross-language understand.

CN119005214AActive Publication Date: 2024-11-22EAST CHINA JIAOTONG UNIVERSITY

Patent Information

Application Number
CN202411498075.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-25
Publication Date
2024-11-22
Estimated Expiration
2044-10-25

AI Technical Summary

Technical Problem

The machine translation quality estimation method based on the pre-trained language model extracts features only in a single semantic space, making it difficult to accurately reflect the machine translation quality, and the cross-language pre-trained model's representation ability to low-resource languages ​​is weakened.

Method used

Using a multi-semantic space method, pseudo-reference translations and back-translations are generated through a dialogue large language model, combined with the cross-language pre-trained model X-MOD, extracting machine-translation quality characteristics of source language, target language and cross-language semantic space, and using multiple heads Attention networks and feedforward neural networks perform feature fusion and prediction.

Benefits of technology

It improves the accuracy and effectiveness of machine translation quality estimation, especially in low-resource language performance, and enhances the understanding and transfer ability of different languages.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119005214A_ABST
    Figure CN119005214A_ABST
Patent Text Reader

Abstract

The invention discloses a machine translation quality estimation method based on multiple semantic spaces. The method comprises machine translation quality estimation model training and machine translation quality estimation. The model training comprises the following steps: generating a pseudo reference translation and back translation by using a dialogue big language model; extracting machine translation quality features in the source language semantic space; extracting machine translation quality features in the target language semantic space; extracting machine translation quality features in the cross-language semantic space; extracting a multi-semantic space machine translation quality feature vector; predicting a machine translation quality score based on the multi-semantic space; and training the machine translation quality estimation model based on the multi-semantic space. The method for estimating the quality of the machine translation comprises the following steps of: normalizing a source language sentence and the machine translation to be subjected to quality estimation; and inputting the normalized source language sentence and the machine translation into the trained machine translation quality estimation model based on the multi-semantic space, and predicting a machine translation quality score.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a machine translation quality estimation method based on multi-semantic space. Background Art

[0002] The goal of machine translation quality estimation (QE) is to evaluate the quality of machine translation using only source language sentences without the need for manual reference translation. Studying machine translation quality estimation methods is of great significance to the research and application of machine translation.

[0003] According to the different feature extraction and model building methods, machine translation quality estimation can be divided into methods based on traditional machine learning, methods based on neural translation models, and methods based on pre-trained language models. Most of the early research on machine translation quality estimation was based on traditional machine learning methods, mainly using the "feature engineering + task modeling" method to estimate machine translation quality. This method manually specifies the features related to translation quality in the source language sentences and machine translations, uses computers to automatically perform linguistic analysis to extract relevant features, and uses traditional machine learning algorithms to predict the quality of machine translation. Methods based on neural translation models, such as the Predictor-Estimator model and the Bilingual Expert model, use transfer learning to extract machine translation quality features from some network layers of the neural network translation model. Methods based on pre-trained language models use language models pre-trained on massive text data to extract features that characterize the quality of machine translations, and build neural network models to predict the quality of machine translations.

[0004] At present, the machine translation quality estimation method based on the pre-trained language model only extracts the quality features of the source language sentence and the machine translation in a single semantic space, which makes it difficult for the extracted quality features to accurately reflect the quality of the machine translation. In order to effectively alleviate the above problems, this technology proposes a machine translation quality estimation method based on multi-semantic space, which generates a pseudo-reference translation of the source language sentence and the back translation of the machine translation by prompting a large language model, compares the three sentence pairs of source language sentence and back translation, source language sentence and machine translation, and pseudo-reference translation and machine translation, extracts the machine translation quality feature vectors of different semantic spaces, deeply fuses the machine translation quality feature vectors of different semantic spaces into multi-semantic space machine translation quality feature vectors, and uses the multi-semantic space machine translation quality feature vectors to predict the machine translation quality. The starting point of this method is that the more similar the machine translation is to the pseudo-reference translation, the more similar the source language sentence is to the back translation, and the higher the quality of the machine translation.

[0005] Machine translation quality estimation methods usually use the cross-language pre-training model XLM-R to extract features in the cross-language space composed of the source language and the target language. The cross-language pre-training model XLM-R adopts the strategy of expanding the vocabulary and increasing the model parameters. By performing unsupervised pre-training on large-scale monolingual datasets in 100 languages, it significantly improves its performance in tasks such as cross-language understanding and low-resource languages. However, as the number of languages ​​supported by the cross-language pre-training model XLM-R increases, the resources that can be allocated to a single language will decrease, which leads to the weakening of the model's representation ability for certain language types, especially low-resource languages. The cross-language pre-training model X-MOD is a modular cross-language pre-training model that combines shared and language-specific parameters. Specifically, the cross-language pre-training model X-MOD adds modules for each language in each transformer layer. These modules are bottleneck feed-forward neural network layers and are specific to the language type. Although the total number of parameters of the model increases linearly with the number of languages, the cost of training and inference does not increase because only the modules of the language are used on the input of the relevant language. The cross-language pre-training model X-MOD introduces modular components during pre-training, which can not only effectively alleviate the noise interference between different languages, but also achieve active migration of low-resource language performance. Therefore, this patent uses the cross-language pre-training model X-MOD to extract features that characterize the quality of machine translation. Summary of the invention

[0006] The present invention provides a machine translation quality estimation method based on multi-semantic space to improve the effect of machine translation quality estimation.

[0007] The technical solution adopted by the present invention is as follows: a method for machine translation quality estimation based on multi-semantic space, which is divided into machine translation quality estimation model training and machine translation quality estimation; wherein the machine translation quality estimation model training method steps are as follows: Step S1, obtaining a training set, and normalizing the training set to obtain a normalized training set; The training set consists of multiple different samples, each of which includes the source language sentence, the machine translation, and the human evaluation score of the machine translation; Step S2, using the large conversation language model to generate a pseudo reference translation and a back translation; Step S3, extracting machine translation quality features in the source language semantic space; Step S4, extracting machine translation quality features in the target language semantic space; Step S5, extracting machine translation quality features in a cross-language semantic space; Step S6, extracting multi-semantic space machine translation quality feature vector; The machine translation quality features in the source language semantic space in step S3, the machine translation quality features in the target language semantic space in step S4, and the machine translation quality features in the cross-language semantic space in step S5 are concatenated and input into the multi-head self-attention network layer to obtain a multi-semantic space machine translation quality feature vector; Step S7, predicting a machine translation quality score based on a multi-semantic space; Inputting the multi-semantic space machine translation quality feature vector in step S6 into the feed-forward neural network layer to predict the machine translation quality score based on the multi-semantic space; Step S8, training a machine translation quality estimation model based on a multi-semantic space; According to the machine translation quality score based on the multi-semantic space predicted in step S7 and the human evaluation score of the machine translation in the training set after normalization in step S1, the parameters of the machine translation quality estimation model based on the multi-semantic space are optimized by minimizing the mean square error loss on the training set to obtain a trained machine translation quality estimation model based on the multi-semantic space.

[0008] Furthermore, the training set in step S1 is composed of multiple different samples, each of which is specifically: Given a sample d={src, mt, y} in the training set, d represents a training sample, src represents the source language sentence, mt represents the machine translation, and y represents the human evaluation score of the machine translation mt.

[0009] Furthermore, in step S2, the dialogue language model is used to generate a pseudo reference translation and a back translation, specifically: Step S21, using translation hints to input the source language sentence in each sample in the normalized training set into the dialogue large language model, so that the source language sentence generates a translation in the corresponding target language; (1); Where pref represents the target language translation of the source language sentence generated by the conversational language model, LLM() represents the output function of the conversational language model, prompt pref represents a forward translation prompt, src represents the source language sentence; Step S22, using the target language translation pref of the source language sentence as a pseudo reference translation of the machine translation; Step S23, using translation hints to input the machine translation of each sample in the training set after normalization into the dialogue large language model, so that the machine translation generates a translation of the corresponding source language; (2); Among them, bt represents the translation of the source language of the machine translation generated by the dialogue large language model, and mt represents the machine translation; Step S24, using the source language translation bt of the machine translation as the back translation of the source language sentence.

[0010] Furthermore, in step S3, the quality features of the machine translation in the source language semantic space are extracted; specifically: Step S31, concatenating the source language sentence and the back translation in each sample in the training set after the normalization process into a source language sentence pair sequence; (3); Among them, X src_bt Represents a source language sentence pair sequence, the symbol "[s]" represents the start symbol of the source language sentence pair sequence, the symbol "[sep]" represents the separator between sentences, and the symbol "[ / s]" represents the end symbol of the source language sentence pair sequence; Step S32, inputting the source language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the source language semantic space; (4); (5); Among them, H src_bt represents the subword-level representation of the source language sentence pair sequence generated by the cross-language pre-training model X-MOD, X-MOD( ) represents the output function of the cross-language pre-training model X-MOD, and h src_bt It represents the unified representation in the source language semantic space, that is, the representation of the starting symbol "[s]" of the subword-level representation of the source language sentence sequence. The symbol "[0]" represents the operation of taking the first element of the representation array; Step S33, the source language sentence and the back translation in each sample in the normalized training set are independently input into the cross-language pre-training model X-MOD, and the subword-level representation of the source language sentence and the subword-level representation of the back translation are extracted; (6); (7); Among them, H src represents the subword-level representation of the source language sentence, H bt represents the back-translated subword-level representation; Step S34, obtaining an independent enhanced representation in the source language semantic space through average pooling and interactive reasoning enhancement; (8); (9); (10); Among them, hsrc represents the overall representation of the source language sentence, h bt represents the back-translated overall representation, v src_bt Indicates independent enhancement of representation in the source language semantic space; AvgPool() represents the average pooling function, “⊕” is a vector concatenation operation, which is used to fuse multiple representation information, the symbol “-” is a subtraction operation between vectors, which is used to capture the difference characteristics between sentence pairs, and the symbol “||” represents the absolute value operation of each vector element, " is the vector element-by-element multiplication operation, which is used to obtain the interactive features of the vectors; Step S35, concatenating the unified representation in the source language semantic space and the independently enhanced representation in the source language semantic space to obtain machine translation quality features in the source language semantic space; (11); Among them, e src_bt Represents the quality features of machine translation in the semantic space of the source language.

[0011] Furthermore, in step S4, the quality features of the machine translation in the target language semantic space are extracted, specifically: Step S41, concatenate the machine translation and pseudo reference translation in each sample in the training set after normalization into a target language sentence pair sequence: (12); Among them, X mt_pref Represents a sequence of target language sentence pairs; Step S42, input the target language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the target language semantic space: (13); (14); Among them, H mt_pref represents the subword-level representation of the target language sentence pair sequence generated by the cross-language pre-training model X-MOD, h mt_pref It represents the unified representation in the target language semantic space, that is, the representation of the start symbol “[s]” of the subword-level representation of the source language sentence sequence; Step S43, the machine translation and pseudo reference translation in each sample in the normalized training set are independently input into the cross-language pre-training model X-MOD, and the sub-word level representation of the machine translation and the sub-word level representation of the pseudo reference translation are extracted: (15); (16); Among them, H mt represents the subword-level representation of machine translation, Hpref represents the subword-level representation of the pseudo-reference translation; Step S44, obtain an independent enhanced representation in the target language semantic space through average pooling and interactive reasoning enhancement: (17); (18); (19); Among them, h mt represents the overall representation of the machine translation, h pref represents the overall representation of the pseudo-reference translation, v mt_pref It represents independent enhancement of representation in the target language semantic space; Step S45, the unified representation in the target language semantic space and the independently enhanced representation in the target language semantic space are concatenated to obtain the machine translation quality features in the target language semantic space: (20); Among them, e mt_pref Represents the quality features of machine translation in the target language semantic space.

[0012] Furthermore, in step S5, the quality features of machine translation in the cross-language semantic space are extracted, specifically: Step S51, concatenate the source language sentence and the machine translation in each sample in the normalized training set into a cross-language sentence pair sequence: (twenty one); Among them, X src_mt Represents a sequence of cross-language sentence pairs; Step S52, input the cross-language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the cross-language semantic space: (twenty two); (twenty three); Among them, H src_mt represents the subword-level representation of the cross-language sentence pair sequence generated by the cross-language pre-training model X-MOD, h src_mt It represents the unified representation in the cross-language semantic space, that is, the representation of the start symbol “[s]” of the subword-level representation of the source language sentence sequence; Step S53, the overall representation of the source language sentence and the overall representation of the machine translation are enhanced by interactive reasoning to obtain independent enhanced representations in the cross-language semantic space: (twenty four); Among them, v src_mt Indicates independent enhancement of representations in the cross-linguistic semantic space; Step S54, the unified representation in the cross-language semantic space and the independently enhanced representation in the cross-language semantic space are concatenated to obtain the machine translation quality features in the cross-language semantic space: (25); Among them, e src_mt Representing machine translation quality features in a cross-lingual semantic space.

[0013] Furthermore, in step S6, the multi-semantic space machine translation quality feature vector is extracted, specifically: The machine translation quality features in the source language semantic space in step S3, the machine translation quality features in the target language semantic space in step S4, and the machine translation quality features in the cross-language semantic space in step S5 are concatenated and input into the multi-head self-attention network layer to obtain the multi-semantic space machine translation quality feature vector: (26); (27); Among them, e multi is the multi-semantic space concatenation vector, Multi-head() is the multi-head self-attention network layer function, e qe is the quality feature vector of machine translation in multi-semantic space.

[0014] Furthermore, in step S7, the machine translation quality score based on the multi-semantic space is predicted, specifically: The multi-semantic space machine translation quality feature vector in step S6 is input into the feed-forward neural network layer to predict the machine translation quality score based on the multi-semantic space: (28); Among them, parameters W1, W2, b1, b2 are the parameters of the feedforward neural network layer, Tanh () is the activation function of the feedforward neural network layer, and QEscore is the machine translation quality score based on multi-semantic space.

[0015] Furthermore, the mean square error loss in step S8 is shown in formula (29); (29); Among them, Loss represents the mean square error loss on the training set, N represents the number of samples in the training set, i represents the i-th sample in the training set, and y (i) represents the human evaluation score of the machine translation of the i-th sample in the training set, QEscore (i) Represents the machine translation quality score based on multi-semantic space predicted by the i-th sample.

[0016] Furthermore, another technical solution adopted by the present invention is a method for estimating the quality of machine translation based on a multi-semantic space, wherein the machine translation quality estimation further includes the following steps: Step S9, normalizing the source language sentences and the machine translations to be estimated in terms of quality; Step S10, inputting the source language sentence and the machine translation after the normalization processing in step S9 into the machine translation quality estimation model based on multi-semantic space trained in step S8 to predict the machine translation quality score.

[0017] The beneficial effects of the present invention are as follows: the present invention decomposes the machine translation quality estimation method into: using a large conversation language model to generate a translation of a source language sentence and use it as a pseudo reference translation, using a large conversation language model to generate a translation of a machine translation and use it as a back translation; extracting machine translation quality features in the source language semantic space based on source language sentences and back translation, extracting machine translation quality features in the target language semantic space based on machine translation and back translation, and extracting machine translation quality features in the cross-language semantic space based on source language sentences and machine translation; splicing three aspects of machine translation quality features and using a multi-head self-attention network layer to extract a multi-semantic space machine translation quality feature vector; using a feedforward neural network layer to predict a machine translation quality score based on multi-semantic space; and effectively improving the effect of machine translation quality estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] FIG1 is a flow chart of a method for training a machine translation quality estimation model based on a multi-semantic space according to the present invention; Figure 2 It is a flow chart of a method for estimating the quality of machine translation based on multi-semantic space of the present invention; Figure 3 It is a schematic diagram of the structure of a machine translation quality estimation model based on multi-semantic space of the present invention. DETAILED DESCRIPTION

[0019] The present invention is described in detail below with reference to the accompanying drawings and implementation modes.

[0020] As shown in FIG1 , this embodiment works as follows: a method for machine translation quality estimation based on a multi-semantic space, which is divided into machine translation quality estimation model training and machine translation quality estimation; wherein the machine translation quality estimation model training method steps are as follows: Step S1, obtaining a training set, normalizing the training set, and obtaining a normalized training set; the training set is composed of a plurality of different samples, each sample including a source language sentence, a machine translation, and a human evaluation score of the machine translation; Step S2, using the dialogue large language model to generate a pseudo reference translation and a back translation; using the translation prompt to input the source language sentence in each sample in the training set after the normalization processing into the dialogue large language model, so that the source language sentence generates a translation in the corresponding target language, and the target language translation of the source language sentence is used as the pseudo reference translation of the machine translation, and using the translation prompt to input the machine translation of each sample in the training set after the normalization processing into the dialogue large language model, so that the machine translation generates a translation in the corresponding source language, and the source language translation of the machine translation is used as the back translation of the source language sentence; Step S3, extracting the quality features of machine translation in the source language semantic space; splicing the source language sentences and back translations in each sample in the normalized training set into a source language sentence pair sequence, inputting the source language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the source language semantic space, and at the same time, independently inputting the source language sentences and back translations in each sample in the normalized training set into the cross-language pre-training model X-MOD to extract the source language sentence subword-level representation and the back translation subword-level representation, obtaining an independent enhanced representation in the source language semantic space through average pooling and interactive reasoning enhancement, and splicing the unified representation in the source language semantic space and the independent enhanced representation in the source language semantic space to obtain the quality features of machine translation in the source language semantic space; Step S4, extracting the quality features of the machine translation in the target language semantic space; splicing the machine translation and pseudo-reference translation in each sample in the training set after normalization into a target language sentence pair sequence, inputting the target language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the target language semantic space, and at the same time independently inputting the machine translation and pseudo-reference translation in each sample in the training set after normalization into the cross-language pre-training model X-MOD to extract the sub-word level representation of the machine translation and the sub-word level representation of the pseudo-reference translation, obtaining an independent enhanced representation in the target language semantic space through average pooling and interactive reasoning enhancement, and splicing the unified representation in the target language semantic space and the independent enhanced representation in the target language semantic space to obtain the quality features of the machine translation in the target language semantic space; Step S5, extracting the quality features of the machine translation in the cross-language semantic space; splicing the source language sentence and the machine translation in each sample in the training set after the normalization process into a cross-language sentence pair sequence, inputting the cross-language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the cross-language semantic space, and at the same time enhancing the overall representation of the source language sentence and the overall representation of the machine translation through interactive reasoning to obtain an independent enhanced representation in the cross-language semantic space, and splicing the unified representation in the cross-language semantic space and the independent enhanced representation in the cross-language semantic space to obtain the quality features of the machine translation in the cross-language semantic space; Step S6, extracting a multi-semantic space machine translation quality feature vector; concatenating the machine translation quality features in the source language semantic space in step S3, the machine translation quality features in the target language semantic space in step S4, and the machine translation quality features in the cross-language semantic space in step S5, and inputting them into a multi-head self-attention network layer to obtain a multi-semantic space machine translation quality feature vector; Step S7, predicting the machine translation quality score based on the multi-semantic space; inputting the multi-semantic space machine translation quality feature vector in step S6 into the feed-forward neural network layer to predict the machine translation quality score based on the multi-semantic space; Step S8, training a machine translation quality estimation model based on a multi-semantic space; according to the machine translation quality score based on a multi-semantic space predicted in step S7 and the human evaluation score of the machine translation in the training set after normalization in step S1, optimizing the parameters of the machine translation quality estimation model based on a multi-semantic space by minimizing the mean square error loss on the training set, so as to obtain a trained machine translation quality estimation model based on a multi-semantic space.

[0021] like Figure 2 As shown, a machine translation quality estimation method based on multi-semantic space also includes: Step S9, normalizing the source language sentences and the machine translations to be estimated in terms of quality; Step S10, inputting the source language sentence and the machine translation after the normalization processing in step S9 into the machine translation quality estimation model based on multi-semantic space trained in step S8 to predict the machine translation quality score.

[0022] Figure 3 It is a schematic diagram of the structure of a machine translation quality estimation model based on multi-semantic space of the present invention; Furthermore, a sample in the training set in step S1 is specifically: Given a sample d={src, mt, y} in the training set, d represents a training sample, src represents the source language sentence, mt represents the machine translation, and y represents the human evaluation score of the machine translation mt; Furthermore, in step S2, the dialogue language model is used to generate a pseudo reference translation and a back translation, specifically: Step S21, using translation hints to input the source language sentence in each sample in the normalized training set into the dialogue large language model, so that the source language sentence generates a translation in the corresponding target language; (1); Where pref represents the target language translation of the source language sentence generated by the conversational language model, LLM() represents the output function of the conversational language model, prompt prefrepresents a forward translation prompt, src represents the source language sentence; Step S22, using the target language translation pref of the source language sentence as a pseudo reference translation of the machine translation; Optionally, the conversational large language model can use a large language model such as ChatGPT, Gemini, or Meta-Llama-3.1-405B-Instruct; Step S23, using translation hints to input the machine translation of each sample in the training set after normalization into the dialogue large language model, so that the machine translation generates a translation of the corresponding source language; (2); Among them, bt represents the translation of the source language of the machine translation generated by the dialogue large language model, and mt represents the machine translation; Step S24, using the source language translation bt of the machine translation as the back translation of the source language sentence.

[0023] Furthermore, in step S3, the quality features of the machine translation in the source language semantic space are extracted; specifically: Step S31, concatenating the source language sentence and the back translation in each sample in the training set after the normalization process into a source language sentence pair sequence; (3); Among them, X src_bt Represents a source language sentence pair sequence, the symbol "[s]" represents the start symbol of the source language sentence pair sequence, the symbol "[sep]" represents the separator between sentences, and the symbol "[ / s]" represents the end symbol of the source language sentence pair sequence; Step S32, inputting the source language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the source language semantic space; (4); (5); Among them, H src_bt represents the subword-level representation of the source language sentence pair sequence generated by the cross-language pre-training model X-MOD, X-MOD( ) represents the output function of the cross-language pre-training model X-MOD, and h src_bt It represents the unified representation in the source language semantic space, that is, the representation of the starting symbol "[s]" of the subword-level representation of the source language sentence sequence. The symbol "[0]" represents the operation of taking the first element of the representation array; Step S33, the source language sentence and the back translation in each sample in the normalized training set are independently input into the cross-language pre-training model X-MOD, and the subword-level representation of the source language sentence and the subword-level representation of the back translation are extracted; (6); (7); Among them, H src represents the subword-level representation of the source language sentence, H bt represents the back-translated subword-level representation; Step S34, obtaining an independent enhanced representation in the source language semantic space through average pooling and interactive reasoning enhancement; (8); (9); (10); Among them, h src represents the overall representation of the source language sentence, h bt represents the back-translated overall representation, v src_bt Indicates independent enhancement of representation in the source language semantic space; AvgPool() represents the average pooling function, “⊕” is a vector concatenation operation, which is used to fuse multiple representation information, the symbol “-” is a subtraction operation between vectors, which is used to capture the difference characteristics between sentence pairs, and the symbol “||” represents the absolute value operation of each vector element, " is the vector element-by-element multiplication operation, which is used to obtain the interactive features of the vectors; Step S35, concatenating the unified representation in the source language semantic space and the independently enhanced representation in the source language semantic space to obtain machine translation quality features in the source language semantic space; (11); Among them, e src_bt Represents the quality features of machine translation in the semantic space of the source language.

[0024] Furthermore, in step S4, the quality features of the machine translation in the target language semantic space are extracted, specifically: Step S41, concatenate the machine translation and pseudo reference translation in each sample in the training set after normalization into a target language sentence pair sequence: (12); Among them, X mt_pref Represents a sequence of target language sentence pairs; Step S42, input the target language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the target language semantic space: (13); (14); Among them, H mt_prefrepresents the subword-level representation of the target language sentence pair sequence generated by the cross-language pre-training model X-MOD, h mt_pref It represents the unified representation in the target language semantic space, that is, the representation of the start symbol “[s]” of the subword-level representation of the source language sentence sequence; Step S43, the machine translation and pseudo reference translation in each sample in the normalized training set are independently input into the cross-language pre-training model X-MOD, and the sub-word level representation of the machine translation and the sub-word level representation of the pseudo reference translation are extracted: (15); (16); Among them, H mt represents the subword-level representation of machine translation, H pref represents the subword-level representation of the pseudo-reference translation; Step S44, obtain an independent enhanced representation in the target language semantic space through average pooling and interactive reasoning enhancement: (17); (18); (19); Among them, h mt represents the overall representation of the machine translation, h pref represents the overall representation of the pseudo-reference translation, v mt_pref It represents independent enhancement of representation in the target language semantic space; Step S45, the unified representation in the target language semantic space and the independently enhanced representation in the target language semantic space are concatenated to obtain the machine translation quality features in the target language semantic space: (20); Among them, e mt_pref Represents the quality features of machine translation in the target language semantic space.

[0025] Furthermore, in step S5, the quality features of machine translation in the cross-language semantic space are extracted, specifically: Step S51, concatenate the source language sentence and the machine translation in each sample in the normalized training set into a cross-language sentence pair sequence: (twenty one); Among them, X src_mt Represents a sequence of cross-language sentence pairs; Step S52, input the cross-language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the cross-language semantic space: (twenty two); (twenty three); Among them, H src_mt represents the subword-level representation of the cross-language sentence pair sequence generated by the cross-language pre-training model X-MOD, h src_mt It represents the unified representation in the cross-language semantic space, that is, the representation of the start symbol “[s]” of the subword-level representation of the source language sentence sequence; Step S53, the overall representation of the source language sentence and the overall representation of the machine translation are enhanced by interactive reasoning to obtain independent enhanced representations in the cross-language semantic space: (twenty four); Among them, v src_mt Indicates independent enhancement of representations in the cross-linguistic semantic space; Step S54, the unified representation in the cross-language semantic space and the independently enhanced representation in the cross-language semantic space are concatenated to obtain the machine translation quality features in the cross-language semantic space: (25); Among them, e src_mt Representing machine translation quality features in a cross-lingual semantic space.

[0026] Furthermore, in step S6, the multi-semantic space machine translation quality feature vector is extracted, specifically: The machine translation quality features in the source language semantic space in step S3, the machine translation quality features in the target language semantic space in step S4, and the machine translation quality features in the cross-language semantic space in step S5 are concatenated and input into the multi-head self-attention network layer to obtain the multi-semantic space machine translation quality feature vector: (26); (27); Among them, e multi is the multi-semantic space concatenation vector, Multi-head() is the multi-head self-attention network layer function, e qe is the quality feature vector of machine translation in multi-semantic space.

[0027] Furthermore, in step S7, the machine translation quality score based on the multi-semantic space is predicted, specifically: The multi-semantic space machine translation quality feature vector in step S6 is input into the feed-forward neural network layer to predict the machine translation quality score based on the multi-semantic space: (28); Among them, parameters W1, W2, b1, b2 are the parameters of the feedforward neural network layer, Tanh () is the activation function of the feedforward neural network layer, and QEscore is the machine translation quality score based on multi-semantic space.

[0028] Furthermore, the mean square error loss in step S8 is shown in formula (29); (29); Among them, Loss represents the mean square error loss on the training set, N represents the number of samples in the training set, i represents the i-th sample in the training set, and y (i) represents the human evaluation score of the machine translation of the i-th sample in the training set, QEscore (i) Represents the machine translation quality score based on multi-semantic space predicted by the i-th sample.

[0029] The machine translation quality estimation method MSS based on multi-semantic space of the present invention is tested on the direct assessment (DA) annotation subtask in the sentence-level machine translation quality estimation benchmark dataset of the 8th International Conference on Machine Translation, which includes the translation quality evaluation of English-Marathi (En-Mr), English-Hindi (En-Hi), English-Tamil (En-Ta), English-Telugu (En-Te) and English-Gujarati (En-Gu) language pairs. The machine translation quality estimation method TransQuest, the machine translation quality estimation method UniTE; the machine translation quality estimation method CometKiwi, the machine translation quality estimation method CrossQE, the machine translation quality estimation method IOL Research and the machine translation quality estimation method EnsembleTQ are used as comparison methods. When evaluating the system performance, the official practice of the 8th International Conference on Machine Translation is followed, and the Spearman correlation coefficient is used as the main evaluation indicator for calculating the correlation between the machine translation quality estimation and the human evaluation results. The larger the Spearman correlation coefficient, the higher the correlation between the two, and the better the model performance.

[0030] Appendix 1: Performance of different machine translation quality estimation methods on the 8th International Conference on Machine Translation Sentence Level Machine Translation Quality Estimation DA dataset

[0031] Appendix 1 shows the performance of different machine translation quality estimation methods on the 8th International Conference on Machine Translation sentence-level machine translation quality estimation DA dataset. MSS is the machine translation quality estimation method based on multi-semantic space of the present invention. The data in Appendix 1 show that the machine translation quality estimation method based on multi-semantic space is superior to the compared machine translation quality estimation methods TransQuest, UniTE, CometKiwi, CrossQE, IOL Research and EnsembleTQ on average.

[0032] This also shows that the machine translation quality estimation method based on multi-semantic space disclosed in the present invention can fully compare the three sentence pairs of source language sentence and back translation, source language sentence and machine translation, and pseudo-reference translation and machine translation, and can effectively extract machine translation quality feature vectors in different semantic spaces, and can improve the effect of machine translation quality estimation on average.

[0033] Appendix 2 Correlation between machine translation quality estimation methods based on multi-semantic space and human evaluation results when different cross-language pre-training models extract translation quality features

[0034] Appendix 2 shows the correlation between the machine translation quality estimation method based on multi-semantic space and the human evaluation results when different cross-language pre-training models are used to extract translation quality features, where the symbol X-MOD row indicates that the machine translation quality estimation method based on multi-semantic space of the present invention uses the cross-language pre-training model X-MOD to extract features of multi-language space; the symbol XLM-R row indicates that the machine translation quality estimation method based on multi-semantic space of the present invention uses the cross-language pre-training model XLM-R to extract features of multi-language space. The results show that the correlation of the machine translation quality estimation method based on multi-semantic space that uses the cross-language pre-training model X-MOD to extract machine translation quality features is consistently higher than that of the machine translation quality estimation method based on multi-semantic space that uses the cross-language pre-training model XLM-R to extract machine translation quality features. The analysis of the machine translation quality features extracted by different cross-language pre-training models shows that the machine translation quality features extracted by the cross-language pre-training model X-MOD can more accurately reflect the translation quality than the machine translation quality features extracted by the cross-language pre-training model XLM-R, thereby improving the performance of the machine translation quality estimation method.

[0035] Since the method of the present disclosure is not proposed for two specific languages, the method of the present disclosure has universal applicability. Although the present disclosure is only experimentally verified in the translation directions of English-Marathi, English-Hindi, English-Tamil, English-Telugu and English-Gujarati, the present disclosure is also applicable to other language pairs, such as Chinese-Japanese and Chinese-Vietnamese.

[0036] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the inventive concept, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A machine translation quality estimation method based on multi-semantic space, which is divided into machine translation quality estimation model training and machine translation quality estimation; The machine translation quality estimation model training is characterized by: The steps are as follows: Step S1, obtaining a training set, and normalizing the training set to obtain a normalized training set; The training set consists of multiple different samples, each of which includes the source language sentence, the machine translation, and the human evaluation score of the machine translation; Step S2, using the large conversation language model to generate a pseudo reference translation and a back translation; Step S3, extracting machine translation quality features in the source language semantic space; Step S4, extracting machine translation quality features in the target language semantic space; Step S5, extracting machine translation quality features in a cross-language semantic space; Step S6, extracting multi-semantic space machine translation quality feature vectors; The machine translation quality features in the source language semantic space in step S3, the machine translation quality features in the target language semantic space in step S4, and the machine translation quality features in the cross-language semantic space in step S5 are concatenated and input into the multi-head self-attention network layer to obtain a multi-semantic space machine translation quality feature vector; Step S7, predicting a machine translation quality score based on a multi-semantic space; Inputting the multi-semantic space machine translation quality feature vector in step S6 into the feed-forward neural network layer to predict the machine translation quality score based on the multi-semantic space; Step S8, training a machine translation quality estimation model based on a multi-semantic space; According to the machine translation quality score based on the multi-semantic space predicted in step S7 and the human evaluation score of the machine translation in the training set after normalization in step S1, the parameters of the machine translation quality estimation model based on the multi-semantic space are optimized by minimizing the mean square error loss on the training set to obtain a trained machine translation quality estimation model based on the multi-semantic space.

2. The method for machine translation quality estimation based on multi-semantic space according to claim 1, characterized in that: The training set in step S1 is composed of multiple different samples, each of which is specifically: Given a sample d={src, mt, y} in the training set, d represents a training sample, src represents the source language sentence, mt represents the machine translation, and y represents the human evaluation score of the machine translation mt.

3. The method for machine translation quality estimation based on multi-semantic space according to claim 2, characterized in that: In step S2, the dialogue language model is used to generate a pseudo reference translation and a back translation, specifically: Step S21, using translation hints to input the source language sentence in each sample in the normalized training set into the dialogue large language model, so that the source language sentence generates a translation in the corresponding target language; (1); Where pref represents the target language translation of the source language sentence generated by the conversational language model, LLM() represents the output function of the conversational language model, prompt pref represents a forward translation prompt, src represents the source language sentence; Step S22, using the target language translation pref of the source language sentence as a pseudo reference translation of the machine translation; Step S23, using translation hints to input the machine translation of each sample in the training set after normalization into the dialogue large language model, so that the machine translation generates a translation of the corresponding source language; (2); Among them, bt represents the translation of the source language of the machine translation generated by the dialogue large language model, and mt represents the machine translation; Step S24, using the source language translation bt of the machine translation as the back translation of the source language sentence.

4. The method for machine translation quality estimation based on multi-semantic space according to claim 3, characterized in that: Step S3 extracts the quality features of the machine translation in the source language semantic space; Specifically: Step S31, concatenating the source language sentence and the back translation in each sample in the training set after the normalization process into a source language sentence pair sequence; (3); Among them, X src_bt Represents a source language sentence pair sequence, the symbol "[s]" represents the start symbol of the source language sentence pair sequence, the symbol "[sep]" represents the separator between sentences, and the symbol "[ / s]" represents the end symbol of the source language sentence pair sequence; Step S32, inputting the source language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the source language semantic space; (4); (5); Among them, H src_bt represents the subword-level representation of the source language sentence pair sequence generated by the cross-language pre-training model X-MOD, X-MOD( ) represents the output function of the cross-language pre-training model X-MOD, and h src_bt It represents the unified representation in the source language semantic space, that is, the representation of the starting symbol "[s]" of the subword-level representation of the source language sentence sequence. The symbol "[0]" represents the operation of taking the first element of the representation array; Step S33, the source language sentence and the back translation in each sample in the normalized training set are independently input into the cross-language pre-training model X-MOD, and the subword-level representation of the source language sentence and the subword-level representation of the back translation are extracted; (6); (7); Among them, H src represents the subword-level representation of the source language sentence, H bt represents the back-translated subword-level representation; Step S34, obtaining an independent enhanced representation in the source language semantic space through average pooling and interactive reasoning enhancement; (8); (9); (10); Among them, h src represents the overall representation of the source language sentence, h bt represents the back-translated overall representation, v src_bt Indicates independent enhancement of representation in the source language semantic space; AvgPool() indicates the average pooling function, "⊕" is a vector concatenation operation, which is used to integrate multiple representation information, the symbol "-" is a subtraction operation between vectors, which is used to capture the difference characteristics between sentence pairs, and the symbol "||" indicates the absolute value operation of each vector element, " " is the vector element-by-element multiplication operation, which is used to obtain the interactive features of the vectors; Step S35, concatenating the unified representation in the source language semantic space and the independently enhanced representation in the source language semantic space to obtain machine translation quality features in the source language semantic space; (11); Among them, e src_bt Represents the quality features of machine translation in the semantic space of the source language.

5. The method for machine translation quality estimation based on multi-semantic space according to claim 4, characterized in that: Step S4 extracts the quality features of the machine translation in the target language semantic space, specifically: Step S41, concatenate the machine translation and pseudo reference translation in each sample in the training set after normalization into a target language sentence pair sequence: (12); Among them, X mt_pref Represents a sequence of target language sentence pairs; Step S42, input the target language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the target language semantic space: (13); (14); Among them, H mt_pref represents the subword-level representation of the target language sentence pair sequence generated by the cross-language pre-training model X-MOD, h mt_pref It represents the unified representation in the target language semantic space, that is, the representation of the start symbol "[s]" of the subword-level representation of the source language sentence sequence; Step S43, the machine translation and pseudo reference translation in each sample in the normalized training set are independently input into the cross-language pre-training model X-MOD, and the sub-word level representation of the machine translation and the sub-word level representation of the pseudo reference translation are extracted: (15); (16); Among them, H mt represents the subword-level representation of machine translation, H pref represents the subword-level representation of the pseudo-reference translation; Step S44, obtaining an independent enhanced representation in the target language semantic space through average pooling and interactive reasoning enhancement: (17); (18); (19); Among them, h mt represents the overall representation of the machine translation, h pref represents the overall representation of the pseudo-reference translation, v mt_pref It represents independent enhancement of representation in the target language semantic space; Step S45, the unified representation in the target language semantic space and the independently enhanced representation in the target language semantic space are concatenated to obtain the machine translation quality features in the target language semantic space: (20); Among them, e mt_pref Represents the quality features of machine translation in the target language semantic space.

6. The method for machine translation quality estimation based on multi-semantic space according to claim 5, characterized in that: Step S5 extracts the machine translation quality features in the cross-language semantic space, specifically: Step S51, concatenate the source language sentence and the machine translation in each sample in the normalized training set into a cross-language sentence pair sequence: (21); Among them, X src_mt Represents a sequence of cross-language sentence pairs; Step S52, input the cross-language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the cross-language semantic space: (22); (23); Among them, H src_mt represents the subword-level representation of the cross-language sentence pair sequence generated by the cross-language pre-training model X-MOD, h src_mt It represents the unified representation in the cross-language semantic space, that is, the representation of the start symbol "[s]" of the subword-level representation of the source language sentence sequence; Step S53, the overall representation of the source language sentence and the overall representation of the machine translation are enhanced by interactive reasoning to obtain independent enhanced representations in the cross-language semantic space: (24); Among them, v src_mt Indicates independent enhancement of representations in the cross-linguistic semantic space; Step S54, the unified representation in the cross-language semantic space and the independently enhanced representation in the cross-language semantic space are concatenated to obtain the machine translation quality features in the cross-language semantic space: (25); Among them, e src_mt Representing machine translation quality features in a cross-lingual semantic space.

7. The method for machine translation quality estimation based on multi-semantic space according to claim 6, characterized in that: In step S6, the multi-semantic space machine translation quality feature vector is extracted, specifically: The machine translation quality features in the source language semantic space in step S3, the machine translation quality features in the target language semantic space in step S4, and the machine translation quality features in the cross-language semantic space in step S5 are concatenated and input into the multi-head self-attention network layer to obtain the multi-semantic space machine translation quality feature vector: (26); (27); Among them, e multi is the multi-semantic space concatenation vector, Multi-head() is the multi-head self-attention network layer function, e qe is the quality feature vector of machine translation in multi-semantic space.

8. The method for machine translation quality estimation based on multi-semantic space according to claim 7, characterized in that: In step S7, the machine translation quality score based on the multi-semantic space is predicted, specifically: The multi-semantic space machine translation quality feature vector in step S6 is input into the feed-forward neural network layer to predict the machine translation quality score based on the multi-semantic space: (28); Among them, parameters W1, W2, b1, b2 are the parameters of the feedforward neural network layer, Tanh () is the activation function of the feedforward neural network layer, and QEscore is the machine translation quality score based on multi-semantic space.

9. The method for machine translation quality estimation based on multi-semantic space according to claim 8, characterized in that: The mean square error loss in step S8 is shown in formula (29); (29); Among them, Loss represents the mean square error loss on the training set, N represents the number of samples in the training set, i represents the i-th sample in the training set, and y (i) represents the human evaluation score of the machine translation of the i-th sample in the training set, QEscore (i) represents the machine translation quality score based on multi-semantic space predicted by the i-th sample.

10. The method for machine translation quality estimation based on multi-semantic space according to claim 9, wherein the machine translation quality estimation is characterized by: The following steps are also included: Step S9, normalizing the source language sentences and the machine translations to be estimated in terms of quality; Step S10, inputting the source language sentence and the machine translation after the normalization processing in step S9 into the machine translation quality estimation model based on multi-semantic space trained in step S8 to predict the machine translation quality score.

Citation Information

Patent Citations

  • Machine translation system and method for contrasting original text through back-translated texts

    CN110069790A

  • Translation quality estimation method for Chinese and Vietnamese neural machine translation integrated with language differentiation features

    CN114004238A

  • Zhongorientation translation quality estimation method and system based on cross-language pre-training model

    CN114386437A

  • Tibetan-Chinese neural machine translation method based on cross-language pre-training model

    CN115329783A

  • Translated text quality evaluation method and system based on natural language large model

    CN116542260A

Cited By

  • Translation quality evaluation model training method, evaluation method and related equipment

    CN119940374A

  • Translation quality evaluation model training method, evaluation method, and related device

    CN119940374B

  • Multi-language deep translation method and system based on large model and electronic equipment

    CN120387462A