A method for estimating quality of machine translation based on multi semantic space

Abstract

The application discloses a machine translation quality estimation method based on multi-semantic space, which is divided into machine translation quality estimation model training and machine translation quality estimation. The model training steps are as follows: generating pseudo reference translation and back translation by using a dialogue large language model; extracting machine translation quality features in the source language semantic space; extracting machine translation quality features in the target language semantic space; extracting machine translation quality features in the cross-language semantic space; extracting a machine translation quality feature vector in the multi-semantic space; predicting a machine translation quality score based on the multi-semantic space; and training a machine translation quality estimation model based on the multi-semantic space. The machine translation quality estimation method steps are as follows: normalizing a source language sentence and machine translation to be quality estimated; inputting the normalized source language sentence and machine translation into the trained machine translation quality estimation model based on the multi-semantic space, and predicting a machine translation quality score.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a machine translation quality estimation method based on multi-semantic space. Background Art

[0002] The goal of machine translation quality estimation (QE) is to evaluate the quality of machine translations using only source language sentences, without the need for manual reference translations. Research on machine translation quality estimation methods is of great significance to the research and application of machine translation.

[0003] Based on the different feature extraction and model building methods, machine translation quality estimation can be divided into methods based on traditional machine learning, methods based on neural translation models, and methods based on pre-trained language models. Early research on machine translation quality estimation was mostly based on traditional machine learning methods, mainly using the "feature engineering + task modeling" approach to estimate machine translation quality. This method manually specifies features related to translation quality in the source language sentences and machine translations, uses computers to automatically perform linguistic analysis to extract relevant features, and uses traditional machine learning algorithms to predict machine translation quality. Methods based on neural translation models, such as the Predictor-Estimator model and the Bilingual Expert model, use transfer learning to extract machine translation quality features from some network layers of the neural network translation model. Methods based on pre-trained language models use language models pre-trained on massive text data to extract features that represent machine translation quality and build neural network models to predict machine translation quality.

[0004] Currently, machine translation quality estimation methods based on pre-trained language models only extract quality features of source language sentences and machine translations in a single semantic space, making it difficult for the extracted quality features to accurately reflect machine translation quality. To effectively alleviate this problem, this technology proposes a machine translation quality estimation method based on multiple semantic spaces. By using a large language model to generate a pseudo-reference translation of the source language sentence and a back-translation of the machine translation, this method compares three sentence pairs: source language sentence and back-translation, source language sentence and machine translation, and pseudo-reference translation and machine translation. Machine translation quality feature vectors are extracted from different semantic spaces, and these feature vectors are deeply fused into a multi-semantic space machine translation quality feature vector. This multi-semantic space machine translation quality feature vector is then used to predict machine translation quality. The starting point of this method is that the more similar the machine translation is to the pseudo-reference translation, and the more similar the source language sentence is to the back-translation, the higher the quality of the machine translation.

[0005] Machine translation quality estimation methods typically extract features from the cross-lingual space of the source and target languages ​​using the cross-lingual pre-training model XLM-R. This model employs a strategy of expanding vocabulary and increasing model parameters. Through unsupervised pre-training on large-scale monolingual datasets across 100 languages, it significantly improves its performance on tasks such as cross-lingual understanding and low-resource languages. However, as the number of languages ​​supported by the cross-lingual pre-training model increases, the resources allocated to each language decrease, resulting in a weakened representation of certain languages, particularly low-resource languages. The cross-lingual pre-training model X-MOD is a modular cross-lingual pre-training model that combines shared and language-specific parameters. Specifically, the cross-lingual pre-training model X-MOD incorporates language-specific modules in each transformer layer. These modules are bottleneck feed-forward neural network layers and are specific to the language. Although the total number of model parameters grows linearly with the number of languages, the training and inference costs do not increase because only the modules for the relevant language are used for the input of the relevant language. The cross-language pre-training model, X-MOD, introduces modular components during pre-training, effectively mitigating noise interference between different languages ​​and enabling proactive performance transfer to low-resource languages. Therefore, this patent uses the cross-language pre-training model X-MOD to extract features that characterize the quality of machine translation. Summary of the Invention

[0006] The present invention provides a machine translation quality estimation method based on multi-semantic space to improve the effect of machine translation quality estimation.

[0007] The technical solution adopted by the present invention is as follows: a method for machine translation quality estimation based on multi-semantic space, which is divided into machine translation quality estimation model training and machine translation quality estimation; wherein the machine translation quality estimation model training method steps are as follows:

[0008] Step S1, obtaining a training set, and performing normalization processing on the training set to obtain a normalized training set;

[0009] The training set consists of multiple different samples, each of which includes the source language sentence, the machine translation, and the human evaluation score of the machine translation;

[0010] Step S2: Generate pseudo-reference translation and back translation using the large conversational language model;

[0011] Step S3, extracting machine translation quality features in the source language semantic space;

[0012] Step S4, extracting machine translation quality features in the target language semantic space;

[0013] Step S5, extracting machine translation quality features in a cross-language semantic space;

[0014] Step S6, extracting multi-semantic space machine translation quality feature vectors;

[0015] The machine translation quality features in the source language semantic space in step S3, the machine translation quality features in the target language semantic space in step S4, and the machine translation quality features in the cross-language semantic space in step S5 are concatenated and input into the multi-head self-attention network layer to obtain a multi-semantic space machine translation quality feature vector;

[0016] Step S7, predicting a machine translation quality score based on the multi-semantic space;

[0017] Inputting the multi-semantic space machine translation quality feature vector in step S6 into the feedforward neural network layer to predict the machine translation quality score based on the multi-semantic space;

[0018] Step S8, training a machine translation quality estimation model based on multiple semantic spaces;

[0019] According to the machine translation quality score based on the multi-semantic space predicted in step S7 and the human evaluation score of the machine translation in the training set after normalization in step S1, the parameters of the machine translation quality estimation model based on the multi-semantic space are optimized by minimizing the mean square error loss on the training set to obtain a trained machine translation quality estimation model based on the multi-semantic space.

[0020] Furthermore, the training set in step S1 is composed of multiple different samples, each of which is specifically:

[0021] Given a sample d={src, mt, y} in the training set, d represents a training sample, src represents the source language sentence, mt represents the machine translation, and y represents the human evaluation score of the machine translation mt.

[0022] Furthermore, in step S2, the dialogue language model is used to generate a pseudo reference translation and a back translation, specifically:

[0023] Step S21: Inputting the source language sentence of each sample in the normalized training set into the dialogue language model using translation hints to generate a translation of the source language sentence into the target language.

[0024] (1);

[0025] Among them, pref represents the target language translation of the source language sentence generated by the dialogue language model, LLM() represents the output function of the dialogue language model, prompt prefIndicates the forward translation prompt, src indicates the source language sentence;

[0026] Step S22, using the target language translation pref of the source language sentence as a pseudo reference translation for machine translation;

[0027] Step S23: using translation hints, input the normalized machine translation of each sample in the training set into the conversational language model, so that the machine translation generates a translation in the corresponding source language;

[0028] (2);

[0029] Among them, bt represents the source language translation of the machine translation generated by the dialogue large language model, and mt represents the machine translation;

[0030] Step S24: Using the machine-translated source language translation bt as a back translation of the source language sentence.

[0031] Furthermore, in step S3, the quality features of the machine translation in the source language semantic space are extracted; specifically:

[0032] Step S31, concatenating the source language sentence and the back translation in each sample in the normalized training set into a source language sentence pair sequence;

[0033] (3);

[0034] Among them, X src_bt Represents a source language sentence pair sequence. The symbol “[s]” indicates the start symbol of the source language sentence pair sequence, the symbol “[sep]” indicates the separator between sentences, and the symbol “[ / s]” indicates the end symbol of the source language sentence pair sequence.

[0035] Step S32: inputting the source language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the source language semantic space;

[0036] (4);

[0037] (5);

[0038] Among them, H src_bt represents the subword-level representation of the source language sentence sequence generated by the cross-language pre-training model X-MOD, X-MOD( ) represents the output function of the cross-language pre-training model X-MOD, and h src_bt It represents the unified representation in the source language semantic space, that is, the representation of the starting symbol "[s]" of the subword-level representation of the source language sentence sequence. The symbol "[0]" represents the operation of taking the first element of the representation array;

[0039] Step S33: Input the source language sentence and back translation of each sample in the normalized training set into the cross-language pre-training model X-MOD to extract the subword-level representation of the source language sentence and the subword-level representation of the back translation;

[0040] (6);

[0041] (7);

[0042] Among them, H src represents the subword-level representation of the source language sentence, H bt represents the back-translated subword-level representation;

[0043] Step S34, obtaining an independent enhanced representation in the source language semantic space through average pooling and interactive reasoning enhancement;

[0044] (8);

[0045] (9);

[0046] (10);

[0047] Among them, h src Represents the overall representation of the source language sentence, h bt represents the back-translation of the overall representation, v src_bt Indicates independent enhancement of representation in the source language semantic space; AvgPool() represents the average pooling function, “⊕” is a vector concatenation operation, which is used to fuse multiple representation information, the symbol “-” is a subtraction operation between vectors, which is used to capture the difference characteristics between sentence pairs, and the symbol “||” represents the absolute value operation of each vector element, " is the vector element-by-element multiplication operation, which is used to obtain the interactive characteristics of the vector;

[0048] Step S35: concatenating the unified representation in the source language semantic space and the independently enhanced representation in the source language semantic space to obtain machine translation quality features in the source language semantic space;

[0049] (11);

[0050] Among them, e src_bt Represents the quality characteristics of machine translation in the semantic space of the source language.

[0051] Furthermore, in step S4, the quality features of the machine translation in the target language semantic space are extracted, specifically:

[0052] Step S41: Concatenate the machine translation and pseudo-reference translation of each sample in the normalized training set into a target language sentence pair sequence:

[0053] (12);

[0054] Among them, X mt_pref Represents a sequence of target language sentence pairs;

[0055] Step S42: Input the target language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the target language semantic space:

[0056] (13);

[0057] (14);

[0058] Among them, H mt_pref represents the subword-level representation of the target language sentence sequence generated by the cross-lingual pre-training model X-MOD, h mt_pref It represents a unified representation in the target language semantic space, that is, the representation of the start symbol "[s]" of the subword-level representation of the source language sentence sequence;

[0059] In step S43, the machine translation and pseudo-reference translation of each sample in the normalized training set are independently input into the cross-language pre-training model X-MOD to extract the sub-word-level representation of the machine translation and the pseudo-reference translation:

[0060] (15);

[0061] (16);

[0062] Among them, H mt represents the subword-level representation of machine translation, H pref represents the subword-level representation of the pseudo-reference translation;

[0063] Step S44: obtain an independent enhanced representation in the target language semantic space through average pooling and interactive reasoning enhancement:

[0064] (17);

[0065] (18);

[0066] (19);

[0067] Among them, h mt represents the overall representation of the machine translation, h pref represents the overall representation of the pseudo-reference translation, v mt_pref It represents independent enhancement of representation in the target language semantic space;

[0068] Step S45: The unified representation in the target language semantic space and the independently enhanced representation in the target language semantic space are concatenated to obtain the machine translation quality features in the target language semantic space:

[0069] (20);

[0070] Among them, e mt_pref Represents the quality features of machine translation in the target language semantic space.

[0071] Furthermore, in step S5, the quality features of the machine translation in the cross-language semantic space are extracted, specifically:

[0072] Step S51: Concatenate the source language sentence and the machine translation of each sample in the normalized training set into a cross-language sentence pair sequence:

[0073] (twenty one);

[0074] Among them, X src_mt Represents a sequence of cross-language sentence pairs;

[0075] Step S52: Input the cross-language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the cross-language semantic space:

[0076] (twenty two);

[0077] (twenty three);

[0078] Among them, H src_mt represents the subword-level representation of the cross-language sentence pair sequence generated by the cross-language pre-training model X-MOD, h src_mt It represents a unified representation in the cross-language semantic space, that is, the representation of the start symbol "[s]" of the subword-level representation of the source language sentence sequence;

[0079] Step S53: The overall representation of the source language sentence and the overall representation of the machine translation are enhanced through interactive reasoning to obtain independent enhanced representations in the cross-language semantic space:

[0080] (twenty four);

[0081] Among them, v src_mt Indicates independent enhancement of representations in the cross-lingual semantic space;

[0082] Step S54: The unified representation in the cross-language semantic space and the independently enhanced representation in the cross-language semantic space are concatenated to obtain the machine translation quality features in the cross-language semantic space:

[0083] (25);

[0084] Among them, e src_mt Representing machine translation quality features in a cross-lingual semantic space.

[0085] Furthermore, in step S6, the multi-semantic space machine translation quality feature vector is extracted, specifically:

[0086] The machine translation quality features in the source language semantic space in step S3, the machine translation quality features in the target language semantic space in step S4, and the machine translation quality features in the cross-language semantic space in step S5 are concatenated and input into the multi-head self-attention network layer to obtain the multi-semantic space machine translation quality feature vector:

[0087] (26);

[0088] (27);

[0089] Among them, e multi is the multi-semantic space concatenation vector, Multi-head( ) is the multi-head self-attention network layer function, e qe is the quality feature vector of machine translation in multi-semantic space.

[0090] Furthermore, in step S7, the machine translation quality score based on the multi-semantic space is predicted, specifically:

[0091] The multi-semantic space machine translation quality feature vector in step S6 is input into the feedforward neural network layer to predict the machine translation quality score based on the multi-semantic space:

[0092] (28);

[0093] Among them, parameters W1, W2, b1, and b2 are the parameters of the feedforward neural network layer, Tanh ( ) is the activation function of the feedforward neural network layer, and QEscore is the machine translation quality score based on the multi-semantic space.

[0094] Furthermore, the mean square error loss in step S8 is shown in formula (29);

[0095] (29);

[0096] Among them, Loss represents the mean square error loss on the training set, N represents the number of samples in the training set, i represents the i-th sample in the training set, and y (i) represents the human evaluation score of the machine translation of the i-th sample in the training set, QEscore (i) represents the machine translation quality score based on the multi-semantic space predicted by the i-th sample.

[0097] Furthermore, another technical solution adopted by the present invention is a method for estimating the quality of machine translation based on a multi-semantic space, wherein the machine translation quality estimation further includes the following steps:

[0098] Step S9, normalizing the source language sentence and the machine translation to be quality estimated;

[0099] In step S10 , the source language sentence and the machine translation after the normalization processing in step S9 are input into the machine translation quality estimation model based on the multi-semantic space trained in step S8 to predict the machine translation quality score.

[0100] The beneficial effects of the present invention are as follows: the present invention decomposes the machine translation quality estimation method into the following steps: using a large dialogue language model to generate a translation of a source language sentence and use it as a pseudo reference translation; using a large dialogue language model to generate a translation of a machine translation and use it as a back translation; extracting machine translation quality features in the source language semantic space based on the source language sentence and the back translation; extracting machine translation quality features in the target language semantic space based on the machine translation and the back translation; and extracting machine translation quality features in the cross-language semantic space based on the source language sentence and the machine translation; splicing the three aspects of machine translation quality features and using a multi-head self-attention network layer to extract a multi-semantic space machine translation quality feature vector; using a feedforward neural network layer to predict a machine translation quality score based on the multi-semantic space; and effectively improving the effect of machine translation quality estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] Figure 1 This is a flow chart of a method for training a machine translation quality estimation model based on a multi-semantic space according to the present invention;

[0102] Figure 2 This is a flow chart of a method for estimating machine translation quality based on multi-semantic space according to the present invention;

[0103] Figure 3 This is a schematic diagram of the structure of a machine translation quality estimation model based on multi-semantic space of the present invention. DETAILED DESCRIPTION

[0104] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0105] like Figure 1 As shown, this embodiment works as follows: a method for machine translation quality estimation based on a multi-semantic space, which is divided into machine translation quality estimation model training and machine translation quality estimation; wherein the machine translation quality estimation model training method steps are as follows:

[0106] In step S1, a training set is obtained, and the training set is normalized to obtain a normalized training set; the training set is composed of different samples, and each sample includes a source language sentence, a machine translation and a human evaluation score of the machine translation;

[0107] In step S2, pseudo reference translations and back translations are generated using a dialogue large language model; the source language sentence in each sample in the normalized training set is input into the dialogue large language model using a translation prompt, so that the source language sentence generates a translation corresponding to the target language, and the translation of the source language sentence in the target language is taken as a pseudo reference translation of the machine translation; the machine translation in each sample in the normalized training set is input into the dialogue large language model using a translation prompt, so that the machine translation generates a translation corresponding to the source language, and the translation of the machine translation in the source language is taken as a back translation of the source language sentence;

[0108] In step S3, a machine translation quality feature in a source language semantic space is extracted; the source language sentence and the back translation in each sample in the normalized training set are spliced into a source language sentence pair sequence, the source language sentence pair sequence is input into a cross-language pre-training model X-MOD to extract a unified representation in the source language semantic space, and the source language sentence and the back translation in each sample in the normalized training set are independently input into the cross-language pre-training model X-MOD to extract a source language sentence sub-word level representation and a back translation sub-word level representation, and an independent enhanced representation in the source language semantic space is obtained through average pooling and interactive reasoning enhancement; the unified representation in the source language semantic space and the independent enhanced representation in the source language semantic space are spliced to obtain the machine translation quality feature in the source language semantic space;

[0109] In step S4, a machine translation quality feature in a target language semantic space is extracted; the machine translation and the pseudo reference translation in each sample in the normalized training set are spliced into a target language sentence pair sequence, the target language sentence pair sequence is input into the cross-language pre-training model X-MOD to extract a unified representation in the target language semantic space, and the machine translation and the pseudo reference translation in each sample in the normalized training set are independently input into the cross-language pre-training model X-MOD to extract a machine translation sub-word level representation and a pseudo reference translation sub-word level representation, and an independent enhanced representation in the target language semantic space is obtained through average pooling and interactive reasoning enhancement; the unified representation in the target language semantic space and the independent enhanced representation in the target language semantic space are spliced to obtain the machine translation quality feature in the target language semantic space;

[0110] Step S5: extracting quality features of machine translation in a cross-language semantic space. The source language sentence and the machine translation in each sample in the normalized training set are spliced ​​into a cross-language sentence pair sequence, and the cross-language sentence pair sequence is input into the cross-language pre-training model X-MOD to extract a unified representation in the cross-language semantic space. At the same time, the overall representation of the source language sentence and the overall representation of the machine translation are enhanced through interactive reasoning to obtain independent enhanced representations in the cross-language semantic space. The unified representation in the cross-language semantic space and the independently enhanced representations in the cross-language semantic space are spliced ​​to obtain quality features of machine translation in the cross-language semantic space.

[0111] Step S6: extracting a multi-semantic space machine translation quality feature vector; concatenating the machine translation quality features in the source language semantic space in step S3, the machine translation quality features in the target language semantic space in step S4, and the machine translation quality features in the cross-language semantic space in step S5, and inputting them into a multi-head self-attention network layer to obtain a multi-semantic space machine translation quality feature vector;

[0112] Step S7, predicting a machine translation quality score based on a multi-semantic space; inputting the multi-semantic space machine translation quality feature vector in step S6 into a feedforward neural network layer to predict a machine translation quality score based on a multi-semantic space;

[0113] Step S8: training a machine translation quality estimation model based on a multi-semantic space; optimizing the parameters of the machine translation quality estimation model based on a multi-semantic space by minimizing the mean square error loss on the training set based on the machine translation quality score predicted in step S7 and the human evaluation score of the machine translation in the training set after normalization in step S1, so as to obtain a trained machine translation quality estimation model based on a multi-semantic space.

[0114] like Figure 2 As shown, a machine translation quality estimation method based on multi-semantic space also includes:

[0115] Step S9, normalizing the source language sentence and the machine translation to be quality estimated;

[0116] In step S10 , the source language sentence and the machine translation after the normalization processing in step S9 are input into the machine translation quality estimation model based on the multi-semantic space trained in step S8 to predict the machine translation quality score.

[0117] Figure 3 This is a schematic diagram of the structure of a machine translation quality estimation model based on multi-semantic space of the present invention;

[0118] Furthermore, a sample in the training set in step S1 is specifically:

[0119] Given a sample d={src, mt, y} in the training set, d represents a training sample, src represents the source language sentence, mt represents the machine translation, and y represents the human evaluation score of the machine translation mt;

[0120] Furthermore, in step S2, the dialogue language model is used to generate a pseudo reference translation and a back translation, specifically:

[0121] Step S21: Inputting the source language sentence of each sample in the normalized training set into the dialogue language model using translation hints to generate a translation of the source language sentence into the target language.

[0122] (1);

[0123] Among them, pref represents the target language translation of the source language sentence generated by the dialogue language model, LLM() represents the output function of the dialogue language model, prompt pref Indicates the forward translation prompt, src indicates the source language sentence;

[0124] Step S22, using the target language translation pref of the source language sentence as a pseudo reference translation for machine translation;

[0125] Optionally, the conversational language model can use a large language model such as ChatGPT, Gemini, or Meta-Llama-3.1-405B-Instruct;

[0126] Step S23: using translation hints, input the normalized machine translation of each sample in the training set into the conversational language model, so that the machine translation generates a translation in the corresponding source language;

[0127] (2);

[0128] Among them, bt represents the source language translation of the machine translation generated by the dialogue large language model, and mt represents the machine translation;

[0129] Step S24: Using the machine-translated source language translation bt as a back translation of the source language sentence.

[0130] Furthermore, in step S3, the quality features of the machine translation in the source language semantic space are extracted; specifically:

[0131] Step S31, concatenating the source language sentence and the back translation in each sample in the normalized training set into a source language sentence pair sequence;

[0132] (3);

[0133] Among them, Xsrc_bt Represents a source language sentence pair sequence. The symbol “[s]” indicates the start symbol of the source language sentence pair sequence, the symbol “[sep]” indicates the separator between sentences, and the symbol “[ / s]” indicates the end symbol of the source language sentence pair sequence.

[0134] Step S32: inputting the source language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the source language semantic space;

[0135] (4);

[0136] (5);

[0137] Among them, H src_bt represents the subword-level representation of the source language sentence sequence generated by the cross-language pre-training model X-MOD, X-MOD( ) represents the output function of the cross-language pre-training model X-MOD, and h src_bt It represents the unified representation in the source language semantic space, that is, the representation of the starting symbol "[s]" of the subword-level representation of the source language sentence sequence. The symbol "[0]" represents the operation of taking the first element of the representation array;

[0138] Step S33: Input the source language sentence and back translation of each sample in the normalized training set into the cross-language pre-training model X-MOD to extract the subword-level representation of the source language sentence and the subword-level representation of the back translation;

[0139] (6);

[0140] (7);

[0141] Among them, H src represents the subword-level representation of the source language sentence, H bt represents the back-translated subword-level representation;

[0142] Step S34, obtaining an independent enhanced representation in the source language semantic space through average pooling and interactive reasoning enhancement;

[0143] (8);

[0144] (9);

[0145] (10);

[0146] Among them, h src Represents the overall representation of the source language sentence, h bt represents the back-translation of the overall representation, v src_btIndicates independent enhancement of representation in the source language semantic space; AvgPool() represents the average pooling function, “⊕” is a vector concatenation operation, which is used to fuse multiple representation information, the symbol “-” is a subtraction operation between vectors, which is used to capture the difference characteristics between sentence pairs, and the symbol “||” represents the absolute value operation of each vector element, " is the vector element-by-element multiplication operation, which is used to obtain the interactive characteristics of the vector;

[0147] Step S35: concatenating the unified representation in the source language semantic space and the independently enhanced representation in the source language semantic space to obtain machine translation quality features in the source language semantic space;

[0148] (11);

[0149] Among them, e src_bt Represents the quality characteristics of machine translation in the semantic space of the source language.

[0150] Furthermore, in step S4, the quality features of the machine translation in the target language semantic space are extracted, specifically:

[0151] Step S41: Concatenate the machine translation and pseudo-reference translation of each sample in the normalized training set into a target language sentence pair sequence:

[0152] (12);

[0153] Among them, X mt_pref Represents a sequence of target language sentence pairs;

[0154] Step S42: Input the target language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the target language semantic space:

[0155] (13);

[0156] (14);

[0157] Among them, H mt_pref represents the subword-level representation of the target language sentence sequence generated by the cross-lingual pre-training model X-MOD, h mt_pref It represents a unified representation in the target language semantic space, that is, the representation of the start symbol "[s]" of the subword-level representation of the source language sentence sequence;

[0158] In step S43, the machine translation and pseudo-reference translation of each sample in the normalized training set are independently input into the cross-language pre-training model X-MOD to extract the sub-word-level representation of the machine translation and the pseudo-reference translation:

[0159] (15);

[0160] (16);

[0161] Among them, H mt represents the subword-level representation of machine translation, H pref represents the subword-level representation of the pseudo-reference translation;

[0162] Step S44: obtain an independent enhanced representation in the target language semantic space through average pooling and interactive reasoning enhancement:

[0163] (17);

[0164] (18);

[0165] (19);

[0166] Among them, h mt represents the overall representation of the machine translation, h pref represents the overall representation of the pseudo-reference translation, v mt_pref It represents independent enhancement of representation in the target language semantic space;

[0167] Step S45: The unified representation in the target language semantic space and the independently enhanced representation in the target language semantic space are concatenated to obtain the machine translation quality features in the target language semantic space:

[0168] (20);

[0169] Among them, e mt_pref Represents the quality features of machine translation in the target language semantic space.

[0170] Furthermore, in step S5, the quality features of the machine translation in the cross-language semantic space are extracted, specifically:

[0171] Step S51: Concatenate the source language sentence and the machine translation of each sample in the normalized training set into a cross-language sentence pair sequence:

[0172] (twenty one);

[0173] Among them, X src_mt Represents a sequence of cross-language sentence pairs;

[0174] Step S52: Input the cross-language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the cross-language semantic space:

[0175] (twenty two);

[0176] (twenty three);

[0177] Among them, H src_mt represents the subword-level representation of the cross-language sentence pair sequence generated by the cross-language pre-training model X-MOD, h src_mt It represents a unified representation in the cross-language semantic space, that is, the representation of the start symbol "[s]" of the subword-level representation of the source language sentence sequence;

[0178] Step S53: The overall representation of the source language sentence and the overall representation of the machine translation are enhanced through interactive reasoning to obtain independent enhanced representations in the cross-language semantic space:

[0179] (twenty four);

[0180] Among them, v src_mt Indicates independent enhancement of representations in the cross-lingual semantic space;

[0181] Step S54: The unified representation in the cross-language semantic space and the independently enhanced representation in the cross-language semantic space are concatenated to obtain the machine translation quality features in the cross-language semantic space:

[0182] (25);

[0183] Among them, e src_mt Representing machine translation quality features in a cross-lingual semantic space.

[0184] Furthermore, in step S6, the multi-semantic space machine translation quality feature vector is extracted, specifically:

[0185] The machine translation quality features in the source language semantic space in step S3, the machine translation quality features in the target language semantic space in step S4, and the machine translation quality features in the cross-language semantic space in step S5 are concatenated and input into the multi-head self-attention network layer to obtain the multi-semantic space machine translation quality feature vector:

[0186] (26);

[0187] (27);

[0188] Among them, e multi is the multi-semantic space concatenation vector, Multi-head( ) is the multi-head self-attention network layer function, e qe is the quality feature vector of machine translation in multi-semantic space.

[0189] Furthermore, in step S7, the machine translation quality score based on the multi-semantic space is predicted, specifically:

[0190] The multi-semantic space machine translation quality feature vector in step S6 is input into the feedforward neural network layer to predict the machine translation quality score based on the multi-semantic space:

[0191] (28);

[0192] Among them, parameters W1, W2, b1, and b2 are the parameters of the feedforward neural network layer, Tanh ( ) is the activation function of the feedforward neural network layer, and QEscore is the machine translation quality score based on the multi-semantic space.

[0193] Furthermore, the mean square error loss in step S8 is shown in formula (29);

[0194] (29);

[0195] Among them, Loss represents the mean square error loss on the training set, N represents the number of samples in the training set, i represents the i-th sample in the training set, and y (i) represents the human evaluation score of the machine translation of the i-th sample in the training set, QEscore (i) represents the machine translation quality score based on the multi-semantic space predicted by the i-th sample.

[0196] The proposed multi-semantic space-based machine translation quality estimation method, MSS, was tested on the Direct Assessments (DA) annotation subtask of the 8th International Conference on Machine Translation (ICMT) sentence-level machine translation quality estimation benchmark dataset. This task includes translation quality assessment for English-Marathi (En-Mr), English-Hindi (En-Hi), English-Tamil (En-Ta), English-Telugu (En-Te), and English-Gujarati (En-Gu) language pairs. Other machine translation quality estimation methods, including TransQuest, UniTE, CometKiwi, CrossQE, IOL Research, and EnsembleTQ, were used for comparison. The system performance evaluation followed the official 8th ICMT evaluation method, using the Spearman correlation coefficient as the primary metric to measure the correlation between machine translation quality estimates and human evaluation results. A larger Spearman correlation coefficient indicates a higher correlation and better model performance.

[0197] Appendix 1: Performance of different machine translation quality estimation methods on the 8th International Conference on Machine Translation Sentence-Level Machine Translation Quality Estimation DA dataset

[0198]

[0199] Appendix Table 1 shows the performance of different machine translation quality estimation methods on the 8th International Conference on Machine Translation (DA) sentence-level machine translation quality estimation dataset. MSS represents the multi-semantic space-based machine translation quality estimation method of the present invention. The data in Appendix Table 1 demonstrates that the multi-semantic space-based machine translation quality estimation method outperforms the comparative machine translation quality estimation methods TransQuest, UniTE, CometKiwi, CrossQE, IOL Research, and EnsembleTQ on average.

[0200] This also shows that the machine translation quality estimation method based on multiple semantic spaces disclosed in the present invention can fully compare three sentence pairs: source language sentences and back translations, source language sentences and machine translations, and pseudo-reference translations and machine translations. It can effectively extract machine translation quality feature vectors in different semantic spaces and can improve the effect of machine translation quality estimation on average.

[0201] Appendix 2 Correlation between machine translation quality estimation methods based on multi-semantic space and human evaluation results when different cross-language pre-training models extract translation quality features

[0202]

[0203] Appendix Table 2 shows the correlation between the machine translation quality estimation method based on multi-semantic space and human evaluation results when different cross-language pre-training models are used to extract translation quality features. The row symbolized by X-MOD indicates that the machine translation quality estimation method based on multi-semantic space of the present invention uses the cross-language pre-training model X-MOD to extract features from the multi-language space; the row symbolized by XLM-R indicates that the machine translation quality estimation method based on multi-semantic space of the present invention uses the cross-language pre-training model XLM-R to extract features from the multi-language space. The results show that the machine translation quality estimation method based on multi-semantic space, which uses the cross-language pre-training model X-MOD to extract machine translation quality features, consistently has higher correlation than the machine translation quality estimation method based on multi-semantic space, which uses the cross-language pre-training model XLM-R to extract machine translation quality features. Analysis of the machine translation quality features extracted by different cross-language pre-training models reveals that the machine translation quality features extracted using the cross-language pre-training model X-MOD more accurately reflect translation quality than those extracted using the cross-language pre-training model XLM-R, thereby improving the performance of the machine translation quality estimation method.

[0204] Because the method disclosed herein is not specifically designed for two specific languages, it has universal applicability. While this disclosure has been experimentally validated only for English-Marathi, English-Hindi, English-Tamil, English-Telugu, and English-Gujarati translation, it is also applicable to other language pairs, such as Chinese-Japanese and Chinese-Vietnamese.

[0205] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the inventive concept, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the appended claims.

Claims

1. A machine translation quality estimation method based on multi-semantic space, which is divided into machine translation quality estimation model training and machine translation quality estimation; The machine translation quality estimation model training is characterized by: The steps are as follows: Step S1, obtaining a training set, and performing normalization processing on the training set to obtain a normalized training set; The training set consists of multiple different samples, each of which includes the source language sentence, the machine translation, and the human evaluation score of the machine translation; Step S2: Generate pseudo-reference translation and back translation using the large conversational language model; Step S3, extracting machine translation quality features in the source language semantic space; Step S4, extracting machine translation quality features in the target language semantic space; Step S5, extracting machine translation quality features in a cross-language semantic space; Step S6, extracting multi-semantic space machine translation quality feature vectors; The machine translation quality features in the source language semantic space in step S3, the machine translation quality features in the target language semantic space in step S4, and the machine translation quality features in the cross-language semantic space in step S5 are concatenated and input into the multi-head self-attention network layer to obtain a multi-semantic space machine translation quality feature vector; Step S7, predicting a machine translation quality score based on the multi-semantic space; Inputting the multi-semantic space machine translation quality feature vector in step S6 into the feedforward neural network layer to predict the machine translation quality score based on the multi-semantic space; Step S8, training a machine translation quality estimation model based on multiple semantic spaces; Based on the machine translation quality scores predicted in step S7 and the human evaluation scores of the machine translations in the training set after normalization in step S1, the parameters of the machine translation quality estimation model based on the multi-semantic space are optimized by minimizing the mean square error loss on the training set, thereby obtaining a trained machine translation quality estimation model based on the multi-semantic space; The machine translation quality estimation also includes the following steps: Step S9, normalizing the source language sentence and the machine translation to be quality estimated; Step S10, inputting the source language sentence and the machine translation after the normalization processing in step S9 into the machine translation quality estimation model based on the multi-semantic space trained in step S8 to predict the machine translation quality score; In step S2, the large conversational language model is used to generate a pseudo reference translation and back translation, specifically: Step S21: Inputting the source language sentence of each sample in the normalized training set into the dialogue language model using translation hints to generate a translation of the source language sentence into the target language. （1）； Among them, pref represents the target language translation of the source language sentence generated by the dialogue language model, LLM() represents the output function of the dialogue language model, prompt pref Indicates the forward translation prompt, src indicates the source language sentence; Step S22, using the target language translation pref of the source language sentence as a pseudo reference translation for machine translation; Step S23: using translation hints, input the normalized machine translation of each sample in the training set into the conversational language model, so that the machine translation generates a translation in the corresponding source language; (2)； Among them, bt represents the source language translation of the machine translation generated by the dialogue large language model, and mt represents the machine translation; Step S24, using the source language translation bt of the machine translation as the back translation of the source language sentence; Step S3 extracts the quality features of the machine translation in the source language semantic space; specifically: Step S31, concatenating the source language sentence and the back translation in each sample in the normalized training set into a source language sentence pair sequence; (3)； Among them, X src_bt Represents a source language sentence pair sequence. The symbol "[s]" indicates the start symbol of the source language sentence pair sequence, the symbol "[sep]" indicates the separator between sentences, and the symbol "[ / s]" indicates the end symbol of the source language sentence pair sequence. Step S32: inputting the source language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the source language semantic space; (4)； (5)； Among them, H src_bt represents the subword-level representation of the source language sentence sequence generated by the cross-language pre-training model X-MOD, X-MOD( ) represents the output function of the cross-language pre-training model X-MOD, and h src_bt It represents the unified representation in the source language semantic space, that is, the representation of the starting symbol "[s]" of the subword-level representation of the source language sentence sequence. The symbol "[0]" represents the operation of taking the first element of the representation array; Step S33: Input the source language sentence and back translation of each sample in the normalized training set into the cross-language pre-training model X-MOD to extract the subword-level representation of the source language sentence and the subword-level representation of the back translation; （6）； （7）； Among them, H src represents the subword-level representation of the source language sentence, H bt represents the back-translated subword-level representation; Step S34, obtaining an independent enhanced representation in the source language semantic space through average pooling and interactive reasoning enhancement; （8）； （9）； （10）； Among them, h src Represents the overall representation of the source language sentence, h bt represents the back-translation of the overall representation, v src_bt Indicates independent enhancement of representation in the source language semantic space; AvgPool() represents the average pooling function, "⊕" is a vector concatenation operation used to fuse multiple representation information, the symbol "-" is a subtraction operation between vectors used to capture the difference characteristics between sentence pairs, and the symbol "||" represents the absolute value operation of each vector element. " is the vector element-by-element multiplication operation, which is used to obtain the interactive characteristics of the vector; Step S35: concatenating the unified representation in the source language semantic space and the independently enhanced representation in the source language semantic space to obtain machine translation quality features in the source language semantic space; (11)； Among them, e src_bt Represents the quality characteristics of machine translation in the semantic space of the source language.

2. The method for machine translation quality estimation based on multi-semantic space according to claim 1, characterized in that: The training set in step S1 consists of multiple different samples, each of which is: Given a sample d={src, mt, y} in the training set, d represents a training sample, src represents the source language sentence, mt represents the machine translation, and y represents the human evaluation score of the machine translation mt.

3. The method for machine translation quality estimation based on multi-semantic space according to claim 2, characterized in that: Step S4 extracts the quality features of the machine translation in the target language semantic space, specifically: Step S41: Concatenate the machine translation and pseudo-reference translation of each sample in the normalized training set into a target language sentence pair sequence: (12)； Among them, X mt_pref Represents a sequence of target language sentence pairs; Step S42: Input the target language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the target language semantic space: (13)； (14)； Among them, H mt_pref represents the subword-level representation of the target language sentence sequence generated by the cross-language pre-training model X-MOD, h mt_pref It represents the unified representation in the target language semantic space, that is, the representation of the start symbol "[s]" of the subword-level representation of the source language sentence sequence; In step S43, the machine translation and pseudo-reference translation of each sample in the normalized training set are independently input into the cross-language pre-training model X-MOD to extract the sub-word-level representation of the machine translation and the pseudo-reference translation: (15)； (16)； Among them, H mt represents the subword-level representation of machine translation, H pref represents the subword-level representation of the pseudo-reference translation; Step S44: obtain an independent enhanced representation in the target language semantic space through average pooling and interactive reasoning enhancement: (17)； (18)； (19)； Among them, h mt represents the overall representation of the machine translation, h pref represents the overall representation of the pseudo-reference translation, v mt_pref It represents independent enhancement of representation in the target language semantic space; Step S45: The unified representation in the target language semantic space and the independently enhanced representation in the target language semantic space are concatenated to obtain the machine translation quality features in the target language semantic space: (20)； Among them, e mt_pref Represents the quality features of machine translation in the target language semantic space.

4. The method for machine translation quality estimation based on multi-semantic space according to claim 3, characterized in that: Step S5 extracts the quality features of machine translation in the cross-language semantic space, specifically: Step S51: Concatenate the source language sentence and the machine translation of each sample in the normalized training set into a cross-language sentence pair sequence: (21)； Among them, X src_mt Represents a sequence of cross-language sentence pairs; Step S52: Input the cross-language sentence pair sequence into the cross-language pre-training model X-MOD to extract a unified representation in the cross-language semantic space: (22)； (23)； Among them, H src_mt represents the subword-level representation of the cross-language sentence pair sequence generated by the cross-language pre-training model X-MOD, h src_mt It represents a unified representation in the cross-language semantic space, that is, the representation of the start symbol "[s]" of the subword-level representation of the source language sentence sequence; Step S53: The overall representation of the source language sentence and the overall representation of the machine translation are enhanced through interactive reasoning to obtain independent enhanced representations in the cross-language semantic space: (24)； Among them, v src_mt Indicates independent enhancement of representations in the cross-lingual semantic space; Step S54: The unified representation in the cross-language semantic space and the independently enhanced representation in the cross-language semantic space are concatenated to obtain the machine translation quality features in the cross-language semantic space: (25)； Among them, e src_mt Representing machine translation quality features in a cross-lingual semantic space.

5. The method for machine translation quality estimation based on multi-semantic space according to claim 4, characterized in that: In step S6, the multi-semantic space machine translation quality feature vector is extracted, specifically: The machine translation quality features in the source language semantic space in step S3, the machine translation quality features in the target language semantic space in step S4, and the machine translation quality features in the cross-language semantic space in step S5 are concatenated and input into the multi-head self-attention network layer to obtain the multi-semantic space machine translation quality feature vector: (26)； (27)； Among them, e multi is the multi-semantic space concatenation vector, Multi-head() is the multi-head self-attention network layer function, e qe is the quality feature vector of machine translation in multi-semantic space.

6. The method for machine translation quality estimation based on multi-semantic space according to claim 5, characterized in that: In step S7, the machine translation quality score based on the multi-semantic space is predicted, specifically: The multi-semantic space machine translation quality feature vector in step S6 is input into the feedforward neural network layer to predict the machine translation quality score based on the multi-semantic space: (28)； Among them, parameters W1, W2, b1, and b2 are the parameters of the feedforward neural network layer, Tanh ( ) is the activation function of the feedforward neural network layer, and QEscore is the machine translation quality score based on the multi-semantic space.

7. The method for machine translation quality estimation based on multi-semantic space according to claim 6, characterized in that: The mean square error loss in step S8 is shown in formula (29); (29)； Among them, Loss represents the mean square error loss on the training set, N represents the number of samples in the training set, i represents the i-th sample in the training set, and y (i) represents the human evaluation score of the machine translation of the i-th sample in the training set, QEscore (i) represents the machine translation quality score based on the multi-semantic space predicted by the i-th sample.

Images