A method for Chinese-vietnamese neural machine translation based on fused bilingual aligned entities

Abstract

The application relates to a kind of Chinese and Vietnamese neural machine translation methods based on fusion bilingual alignment entity, belong to natural language processing technical field. Entity translation has been a difficulty in machine translation, because of its one-to-one translation characteristics, it is rarely in training corpus, and entity word can be excluded from the translation vocabulary, thereby leading to model unable to predict it in the derivation stage;The application incorporates the constraint position prompt information at the encoding end sentence representation level;In the decoding end, the pointer network mechanism is incorporated, so that the model can copy the output source sentence vocabulary. Experimental results show that the method effectively solves the entity word translation problem in sentence under low resource, compared with the baseline model, improves 2.84 BLEU score.

Description

Technical Field

[0001] This invention relates to a Chinese-Vietnamese neural machine translation method based on the fusion of bilingual aligned entities, belonging to the field of natural language processing technology. Background Technology

[0002] Chinese-Vietnamese machine translation is a typical low-resource task. Limited parallel corpora and significant differences in language expression lead to poor quality in Chinese-Vietnamese neural machine translation, particularly inaccurate entity word translation. Because entity words appear infrequently in the training corpus, the model cannot construct a mapping relationship between bilingual entities, making it difficult for traditional neural machine translation methods to accurately translate entity words in sentences. To improve the quality of low-resource neural machine translation, data augmentation or language models can be used to enhance the rationality and readability of the translated sentences; the accuracy of word translation within sentences can also be improved.

[0003] The performance of neural machine translation models largely depends on the size and quality of parallel corpora. However, Chinese-Vietnamese bilingual parallel sentence pairs are a scarce resource, leading to poor quality in Chinese-Vietnamese neural machine translation. To improve the quality of low-resource neural machine translation, data augmentation or language modeling methods can be employed to enhance the rationality and readability of the translated sentences. Furthermore, the accuracy of word translation can be improved to ensure that key information in the sentence is not distorted. Through methods such as data augmentation or language modeling, the number of words in the sentence can be increased, making it more readable. This will provide more context and detail, thereby improving the expressiveness and fluency of the text.

[0004] Entity translation has always been a challenge in machine translation because its one-to-one translation nature means that it is rarely found in training corpora, and entity words are likely to be excluded from the translation vocabulary, preventing the model from predicting them during the inference phase. Lexical constraints are a common method for handling out-of-vocabulary translation. The text is labeled with entity categories in the source sentence, and then the translation is obtained by querying an entity database or by using a dedicated translation module for each entity category. The translated result is then appended to the end of the source sentence as input to the model.

[0005] Lexical constraint methods can improve the accuracy of word translation in sentences, ensuring that key information is not distorted. Entity words typically contain crucial information necessary for understanding sentences, such as names of people and places, and they have a one-to-one translation characteristic, making lexical constraint methods suitable. Constructing a large bilingual entity dictionary and obtaining the entity word translations from the source language sentences in advance, then integrating these entity translations into the translation results, will significantly improve translation quality. Summary of the Invention

[0006] This invention provides a Chinese-Vietnamese neural machine translation method based on fused bilingual aligned entities to solve the problem of inaccurate entity word translation in low-resource machine translation. This invention can accurately translate entity words in bilingual dictionaries while maintaining the overall quality of the translation.

[0007] The technical solution of this invention is: a Chinese-Vietnamese neural machine translation method based on the fusion of bilingual aligned entities, the specific steps of which are as follows:

[0008] Step 1: Construct different types of translation modules;

[0009] Step 2: First, identify the named entities in the source sentence. Then, use the Chinese-Vietnamese bilingual entity dictionary to query for a corresponding translation. If a translation is found, append the entity translation to the end of the source sentence. If no translation is found, send the entity to a different type of translation module for translation based on the entity word category to obtain the translation result. Then, append the entity translation to the end of the source sentence.

[0010] Step 3: Construct and encode the encoder of the Sino-Vietnamese neural machine translation model that integrates and aligns entities: The encoder of this translation model is the traditional Transformer encoder. In the sentence representation stage, in addition to using word embedding representation and position encoding representation, additional constraint position cue information is added. The vector obtained by adding all representations is used as the final sentence representation.

[0011] Step 4: Construct and decode the decoder of the Sino-Vietnamese neural machine translation model that integrates aligned entities: The decoder of this translation model incorporates a pointer network mechanism, which enables the translation model to not only predict the output from the vocabulary, but also to copy the vocabulary in the input of the output source, thereby integrating the pre-obtained entity translation results into the output of the final translated sentence.

[0012] Furthermore, in Step 1, the construction of different types of translation modules includes the construction of a digital time translation module and a personal name translation module;

[0013] The digital time translation module is constructed using the Chinese-Vietnamese digital time correspondence rules;

[0014] The personal name translation module is used to construct pseudo-Chinese-Vietnamese personal name translation pairs by combining the correspondence between Chinese and Vietnamese characters with commonly used characters in personal names, and then collect real personal name translation pairs.

[0015] Furthermore, the specific implementation of step 2 includes:

[0016] Step 2.1: First, identify the named entities in the source sentence, then query their translations using a Chinese-Vietnamese bilingual entity dictionary to obtain the translation results. Finally, append the entity translations to the end of the source sentence using a separator. <sep>To separate each entity, and add a separator at the end of the sentence. <eos>This indicates the end; if no translation result is found, the entity word is sent to the appropriate translation module based on its category to obtain the translation result. The entity translation is then appended to the end of the source sentence using a separator. <sep>To separate each entity, and add a separator at the end of the sentence. <eos>This indicates the end.

[0017] Furthermore, the specific implementation of step 3 includes:

[0018] The encoder of the Sino-Vietnamese neural machine translation model that integrates aligned entities is the traditional Transformer encoder. In the sentence representation stage, it not only uses word embedding representation and position encoding representation, but also adds constraint position cue information, and uses the vector obtained by adding all representations as the final sentence representation.

[0019] Among them, word embedding representation and positional encoding are built into Transformer, while constraint cueing CP is a learnable representation layer that assigns the same label to the words in the sentence that are to be translated, using a separator. <sep>To separate the source sentence from the constraint word, add a separator at the end of the sentence. <eos>This indicates the end.

[0020] Furthermore, the specific implementation of step 4 includes:

[0021] The decoder of the Sino-Vietnamese neural machine translation model that integrates aligned entities introduces a pointer network mechanism based on the Transformer decoder. This enables the translation model to not only predict the output from the vocabulary but also to reproduce the vocabulary in the source input. At decoding time step t, the final output token probability is... It is a weighted sum of the token distributions in the prediction mode and the replication mode. This represents the source input containing entity translation; the SoftMax algorithm, which uses the decoder output, is used to compute the probability distribution p over the target-side vocabulary in the prediction pattern. t predict The average multi-head attention weight of the last decoding layer is used to set the token probability distribution p in the replication mode. t copy If the target word y is in the source sentence, then the probability distribution p t copy This corresponds to the attention weight of the source location; otherwise, the probability is set to zero; combined with p t copy and p t predict This yielded a new distribution on the target vocabulary, as shown in the formula:

[0022]

[0023] Where g t ∈[0,1] is the control probability distribution p t predict and p t copy The weights of the contribution at time step t; using a feedforward neural network from the context vector c t and the decoder hidden state z of the last layer t Calculations, such as the formula:

[0024] g t =FeedForward(c t ,z t )

[0025]

[0026] Among them, c t The calculation is as shown in the formula above, h s It is the hidden state of the encoder at position s in the last layer, α t,s It is the average attention weight of the last decoder layer at decoding time step t for source position s. It is the length of the source statement x.

[0027] The beneficial effects of this invention are:

[0028] 1. This invention incorporates "block information" and "entity category information" at the sentence representation level of the encoding end; and incorporates a pointer network mechanism at the decoding end, enabling the model to copy the vocabulary of the source sentence.

[0029] 2. This invention solves the problem of inaccurate entity word translation in low-resource machine translation. It obtains the translation of words in the source sentence through a Chinese-Vietnamese bilingual dictionary as a constraint condition, introduces "constraint hint information" at the encoding end, and incorporates a pointer network mechanism at the decoding end to ensure that the model can correctly output the vocabulary of the source sentence.

[0030] 3. Through case analysis, it was also found that the method of the present invention can accurately translate entity words in bilingual dictionaries while maintaining the overall quality of the translation. Attached Figure Description

[0031] Figure 1 This is a flowchart from the present invention;

[0032] Figure 2 This is a schematic diagram of the translation model used in this invention;

[0033] Figure 3 This is a schematic diagram of sentence representation in this invention. Detailed Implementation

[0034] Example 1: As Figures 1-3 As shown, a Chinese-Vietnamese neural machine translation method based on the fusion of bilingual aligned entities is described. The specific steps of the method are as follows:

[0035] Step 1: Construct different types of translation modules, including a digital time translation module and a personal name translation module. The digital time translation module is constructed using the correspondence rules between Chinese and Vietnamese digital time. The personal name translation module is used to construct pseudo Chinese-Vietnamese personal name translation pairs by combining the correspondence between Chinese and Vietnamese characters with commonly used characters in personal names, and then collect real personal name translation pairs.

[0036] This invention extracted 260,000 pairs of Sino-Vietnamese entity words through Wikipedia entity links, covering common personal and place names. Basic numerical expressions in Vietnamese use pure Vietnamese words, and the phonetic changes in numbers cause variations in their written form. When the tens digit is less than 2, it remains unchanged; for example, the number 1 is read as... The number 11 is read as When the tens digit is greater than 2, it becomes Besides the Sino-Vietnamese numerical forms, the expressions also involve complex variations such as fractions, decimals, percentages, approximate numbers, and ordinal numbers. For time expressions, Sino-Vietnamese uses "year-month-day" and "day-month-year" respectively to represent dates, but the order is similar when expressing specific times of day, such as hours and minutes. Given the diverse and complex relationships between Sino-Vietnamese numerical time expressions, this invention employs Sino-Vietnamese numerical time correspondence rules to construct a time-specific translation module.

[0037] While personal names can be directly looked up in a Chinese-Vietnamese dictionary, it cannot adequately translate names outside of official dictionaries. Like Chinese names, Vietnamese names follow the surname-first-given-last format. The difference lies in the addition of a padding character between the surname and given name to distinguish gender, clan, and generation. Therefore, this invention constructs pseudo-Chinese-Vietnamese name translation pairs by utilizing the correspondence between Chinese and Vietnamese characters and combining commonly used characters in personal names, and then collects real name translation pairs.

[0038] Step 2: First, identify the named entities in the source sentence. Then, look up their translations using a Chinese-Vietnamese bilingual entity dictionary to obtain the translation results. Finally, append the entity translations to the end of the source sentence using a separator. <sep>To separate each entity, and add a separator at the end of the sentence. <eos>This indicates the end; if no translation result is found, the entity word is sent to the appropriate translation module based on its category to obtain the translation result. The entity translation is then appended to the end of the source sentence using a separator. <sep>To separate each entity, and add a separator at the end of the sentence. <eos>This indicates the end.

[0039] Step 3: Construct and encode the encoder of the Chinese-Vietnamese neural machine translation model that integrates aligned entities: The encoder of this translation model is a traditional Transformer encoder. In the sentence representation stage, in addition to using word embedding representation and positional encoding representation, additional constraint positional cue information is added. The vector obtained by adding all representations is used as the representation of the final sentence. For example, in the sentence "Vietnam's main waterway port is located in Hanoi.", "Vietnam", "port" and "Hanoi" are all words in the bilingual dictionary. After processing, the input form of this sentence is "Vietnam's main waterway port is located in Hanoi". Its characteristics are as follows Figure 3 As shown;

[0040] Among them, token embeddings and position embeddings are built into the Transformer, while constraint hints (CP) are learnable representation layers that assign the same label to the words in the sentence that are to be translated, such as "Vietnam" and "...". In the source sentence, all are represented by A. (Separated by delimiter) <sep>To separate the source sentence from the constraint word, add a separator at the end of the sentence. <eos>This indicates the end. The final sentence embedding representation is shown in the formula.

[0041]

[0042] Step 4: Construct and decode the decoder of the Sino-Vietnamese neural machine translation model that integrates aligned entities: The decoder of this translation model incorporates a pointer network mechanism, which enables the translation model to not only predict the output from the vocabulary, but also to copy the vocabulary in the input of the output source, thereby integrating the pre-obtained entity translation results into the output of the final translated sentence.

[0043] At decoding time step t, the final output token probability is... It is a weighted sum of the token distributions in the prediction mode and the replication mode. This represents the source input containing entity translation; the SoftMax algorithm, which uses the decoder output, is used to compute the probability distribution p over the target-side vocabulary in the prediction pattern. t predict The average multi-head attention weight of the last decoding layer is used to set the token probability distribution p in the replication mode. t copy If the target word y is in the source sentence, then the probability distribution p t copy This corresponds to the attention weight of the source location; otherwise, the probability is set to zero; combined with p t copy and p t predict This yielded a new distribution on the target vocabulary, as shown in the formula:

[0044]

[0045] Where g t ∈[0,1] is the control probability distribution p t predict and p t copy The weights of the contribution at time step t; using a feedforward neural network from the context vector c t and the decoder hidden state z of the last layer t Calculations, such as the formula:

[0046] g t =FeedForward(c t ,z t )

[0047]

[0048] Among them, c t The calculation is as shown in the formula above, h s It is the hidden state of the encoder at position s in the last layer, α t,s It is the average attention weight of the last decoder layer at decoding time step t for source position s. It is the length of the source statement x.

[0049] like Figure 2 The diagram illustrates the translation model used in this invention. The translation model is based on the Transformer architecture and consists of an encoder and a decoder. The encoder is consistent with the traditional Transformer, including a multi-head self-attention layer, a normalization layer, a fully connected layer, and a residual network. Simultaneously, the decoder incorporates a pointer network mechanism, calculating the average weight of the last multi-head attention layer as the replication probability of the pointer network.

[0050] To verify the effectiveness of the proposed method, this invention was compared with different lexical constraint models. These included the soft-constraint placeholder method, Song's dictionary-based lexical constraint model LeCA (Lexical-Constraint-Aware), Chen's LeCA+Ptr (pointer network) model introducing a pointer network, and the relatively slow hard-constraint grid beam search (GBS) decoding algorithm and dynamic beam assignment (DBA) algorithm. Experimental results are shown in Table 1.

[0051] Table 1 Comparison of translation experiments with different models under lexical constraints

[0052]

[0053] Through comparative experiments, it was found that the results of the Chinese-Vietnamese and Vietnamese-Chinese translations using the lexical constraint model improved the BLE score by approximately 0.2 points compared to the hard-constrained GBS and DBA algorithms. Although the hard-constrained algorithm ensures that 100% of the constrained words appear in the target language, guaranteeing the accuracy of word translation, it compromises the model's robustness, resulting in lower BLE scores for noisy test corpora. Compared to the soft-constrained placeholder method, the lexical constraint model of this invention improved the BLE score by 1.4 points in the Chinese-Vietnamese translation direction and by 1.3 points in the Vietnamese-Chinese translation direction. Therefore, the experimental results demonstrate that the model of this invention can accurately translate words in the source sentence without compromising the contextual information of the source sentence, reflecting the effectiveness of this method.

[0054] To verify the effectiveness of the lexical constraint method on low-resource conditions, comparative experiments were conducted using several existing neural machine translation models that do not employ lexical constraints. The Transformer model with identical parameters was used as the baseline, along with ConvS2S, a fully convolutional neural network-based model, DATNMT, a Transformer machine translation model based on dual attention, and HySAN, a hybrid self-attention network. The results are shown in Table 2.

[0055] Table 2 Comparison of translation experiments using different models without lexical constraints

[0056] Model Han → Yue Vietnamese → Chinese Baseline 19.94 18.83 ConvS2S 17.36 16.84 DATNMT 20.26 18.86 HySAN 20.88 19.59 This invention model 22.78 21.36

[0057] The experimental results in Table 2 show that the model using lexical constraints produces better translation results than the traditional neural machine translation model without lexical constraints. The experimental results are optimal compared to the traditional neural machine translation model. In Chinese-to-Vietnamese translation, the lexical constraint model improves translation performance by 2.84 BLEu values ​​compared to the baseline model Transformer, and by 2.53 BLEu values ​​in Vietnamese-to-Chinese translation, demonstrating that the lexical constraint method can effectively improve translation performance for low-resource languages. Compared to the suboptimal HySAN self-attention network, the lexical constraint model of this invention improves translation performance by 1.9 BLEu values ​​in Chinese-Vietnamese reverse translation and by 1.77 BLEu values ​​in Vietnamese-Chinese reverse translation.

[0058] To investigate the impact of incorporating constraint cue information and pointer networks on the performance of the Chinese-Vietnamese translation model, this invention conducted an ablation experiment. In this experiment, the constraint cue information and pointer networks were gradually removed from the model, and the performance changes at each step were compared. The experimental results are shown in Table 3.

[0059] Table 3 Ablation Experiment Results

[0060]

[0061] Experimental results show that constraint cues do indeed improve the model's translation quality. However, when the pointer network is removed, the BLEU value decreases by 2.64 points in the Chinese-Vietnamese translation direction and by 2.18 points in the Vietnamese-Chinese translation direction, severely impacting the model's translation quality. The current model (Transformer + constraint cues) shows only a 0.29 and 0.35 BLEU improvement compared to the baseline Transformer model, respectively, less than 0.4. Therefore, removing the pointer network also reduces the impact of block and category information on the model.

[0062] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.< / eos> < / sep> < / eos> < / sep> < / eos> < / sep> < / eos> < / sep> < / eos> < / sep> < / eos> < / sep>

Claims

1. A Chinese-Vietnamese neural machine translation method based on fused bilingual aligned entities, characterized in that: The specific steps of the method are as follows: Step 1: Construct different types of translation modules; Step 2: First, identify the named entities in the source sentence. Then, use the Chinese-Vietnamese bilingual entity dictionary to query for a corresponding translation. If a translation is found, append the entity translation to the end of the source sentence. If no translation is found, send the entity to a different type of translation module for translation based on the entity word category to obtain the translation result. Then, append the entity translation to the end of the source sentence. Step 3: Construct and encode the encoder of the Sino-Vietnamese neural machine translation model that integrates and aligns entities: The encoder of this translation model is the traditional Transformer encoder. In the sentence representation stage, in addition to using word embedding representation and position encoding representation, additional constraint position cue information is added. The vector obtained by adding all representations is used as the final sentence representation. Step 4: Construct and decode the decoder of the Sino-Vietnamese neural machine translation model that integrates aligned entities: The decoder of this translation model incorporates a pointer network mechanism, which enables the translation model to not only predict the output from the vocabulary, but also to copy the vocabulary in the input of the output source, thereby integrating the pre-obtained entity translation results into the output of the final translated sentence.

2. The Chinese-Vietnamese neural machine translation method based on fused bilingual aligned entities according to claim 1, characterized in that: In Step 1, the construction of different types of translation modules includes the construction of a digital time translation module and a personal name translation module; The digital time translation module is constructed using the Chinese-Vietnamese digital time correspondence rules; The personal name translation module is used to construct pseudo-Chinese-Vietnamese personal name translation pairs by combining the correspondence between Chinese and Vietnamese characters with commonly used characters in personal names, and then collect real personal name translation pairs.

3. The Chinese-Vietnamese neural machine translation method based on fused bilingual aligned entities according to claim 1, characterized in that: The specific implementation of Step 2 includes: Step 2.1: First, identify the named entities in the source sentence, then query their translations using a Chinese-Vietnamese bilingual entity dictionary to obtain the translation results. Finally, append the entity translations to the end of the source sentence using a separator. <sep>To separate each entity, and add a separator at the end of the sentence. <eos>This indicates the end; if no translation result is found, the entity word is sent to the appropriate translation module based on its category to obtain the translation result. The entity translation is then appended to the end of the source sentence using a separator. <sep>To separate each entity, and add a separator at the end of the sentence. <eos> This indicates the end.< / eos> < / sep> < / eos> < / sep> 4. The Chinese-Vietnamese neural machine translation method based on fused bilingual aligned entities according to claim 1, characterized in that: The specific implementation of Step 3 includes: The encoder of the Sino-Vietnamese neural machine translation model that integrates aligned entities is the traditional Transformer encoder. In the sentence representation stage, it not only uses word embedding representation and position encoding representation, but also adds constraint position cue information, and uses the vector obtained by adding all representations as the final sentence representation. Among them, word embedding representation and positional encoding are built into Transformer, while constraint cueing CP is a learnable representation layer that assigns the same label to the words in the sentence that are to be translated, using a separator. <sep>To separate the source sentence from the constraint word, add a separator at the end of the sentence. <eos> This indicates the end.< / eos> < / sep> 5. The Chinese-Vietnamese neural machine translation method based on fused bilingual aligned entities according to claim 1, characterized in that: The specific implementation of step 4 includes: The decoder of the Sino-Vietnamese neural machine translation model that integrates aligned entities introduces a pointer network mechanism based on the Transformer decoder. This enables the translation model to not only predict the output from the vocabulary but also to reproduce the vocabulary in the source input. At decoding time step t, the final output token probability is... It is a weighted sum of the token distributions in the prediction mode and the replication mode. This represents the source input containing entity translation; the SoftMax algorithm, which uses the decoder output, is used to compute the probability distribution p over the target-side vocabulary in the prediction pattern. t predict The average multi-head attention weight of the last decoding layer is used to set the token probability distribution p in the replication mode. t copy If the target word y is in the source sentence, then the probability distribution p t copy This corresponds to the attention weight of the source location; otherwise, the probability is set to zero; combined with p t copy and p t predict This yielded a new distribution on the target vocabulary, as shown in the formula: Where g t ∈[0,1] is the control probability distribution p t predict and p t copy The weights of the contribution at time step t; using a feedforward neural network from the context vector c t and the decoder hidden state z of the last layer t Calculations, such as the formula: g t =FeedForward(c t ,z t ) Among them, c t The calculation is as shown in the formula above, h s It is the hidden state of the encoder at position s in the last layer, α t,s It is the average attention weight of the last decoder layer at decoding time step t for source position s. It is the length of the source statement x.

Images