Neural machine translation method integrated with field bilingual dictionary in Tali-Chinese law field

Abstract

The invention relates to a Thai-Chinese law field neural machine translation method integrated with a field bilingual dictionary, and belongs to the technical field of natural language processing. Comprising the following steps: constructing a Thai-Chinese law field parallel text corpus; mining a field bilingual dictionary from the constructed Thai-Chinese law field parallel text corpus by using a tool; using a domain bilingual dictionary to screen'pseudo domain 'data from a Tali-Chinese public data set to expand a training set; building dictionary prompt data for a field bilingual dictionary matched with the Thai source sentence; carrying out coding representation on dictionary prompt data, and fusing the dictionary prompt data into a neural machine translation model of a Transform architecture; and training a neural machine translation model integrated with dictionary prompt representation by using the expanded training set. Through instance analysis, it is found that the method can improve the translation accuracy of the field keywords.

Description

Technical Field

[0001] The present invention relates to a Thai-Chinese legal domain neural machine translation method integrated with a domain bilingual dictionary, and belongs to the technical field of natural language processing. Background Art

[0002] Domain-specific neural machine translation (NMT) is primarily based on domain adaptation, which involves migrating models from a general domain to a target domain and then fine-tuning them. In recent years, advancements in domain adaptation have enabled significant success in domain-specific neural machine translation. However, the limited size of the Thai-Chinese general domain parallel corpus makes it difficult to capture most general knowledge. Furthermore, migrating this corpus to the target low-resource legal domain presents a significant domain gap, making it difficult to fully capture contextual knowledge for domain keywords, leading to inaccurate translation.

[0003] Therefore, to train a domain baseline model, we can filter out-of-domain data containing domain keywords for data augmentation. This provides richer contextual information about the domain keywords. Furthermore, we can introduce embedded hints about domain keywords into the model to enhance its understanding of context and improve the translation accuracy of domain keywords.

[0004] Introducing bilingual dictionary constraints into neural machine translation models has always been a challenge, as dictionaries may contain noisy and redundant constraints, leading to biased model vocabulary predictions. Dictionary hints are a method for providing contextual representations. By shallowly encoding dictionary hint representations into the source encoder representation, the impact of redundant constraints can be mitigated.

[0005] The proposed Thai-Chinese neural machine translation method for legal domains operates on two levels: At the data level, it can filter out-of-domain data containing domain keywords to expand the training set. A filtering method based on coverage of infrequent Chinese terms in the legal domain offers faster filtering speeds and provides richer contextual information for low-frequency domain terms, leading to greater performance improvements. At the model level, embedded hints of domain keywords are introduced into the model's encoding representation. Additional cross-attention is used to enable interaction between the source sentence representation and the hint representation, allowing the source sentence representation to dynamically select and focus on the most relevant parts of the hint representation, reducing the interference of redundant constraints and thus enhancing the model's understanding of context. Summary of the Invention

[0006] The technical problem solved by the present invention is: the present invention provides a Thai-Chinese legal domain neural machine translation method that incorporates a domain bilingual dictionary to solve the problems of insufficient contextual information of domain keywords and noise interference in dictionary constraints in the current domain neural machine translation process. The present invention introduces more contextual information of low-frequency domain words through a screening method based on the coverage of infrequent Chinese words in the legal domain, introduces dictionary prompt representation information to the encoding end, and reduces the impact of redundant noise by adding a cross-attention module, thereby effectively improving the translation performance.

[0007] The technical solution of the present invention is: a Thai-Chinese legal domain neural machine translation method that incorporates a domain bilingual dictionary, the method comprising:

[0008] Step 1: Construct a Thai-Chinese parallel text corpus in the legal field;

[0009] Step 2: Use tools to mine bilingual dictionaries from the constructed Thai-Chinese legal field parallel text corpus;

[0010] Step 3: Use the domain bilingual dictionary to filter “pseudo-domain” data from the Thai-Chinese public dataset to expand the training set;

[0011] Step 4: Construct dictionary prompt data for the bilingual dictionary in the Thai source sentence matching domain;

[0012] Step 5: Encode the dictionary prompt data and integrate it into the neural machine translation model of the Transformer architecture;

[0013] Step 6: Use the expanded training set to train the neural machine translation model that incorporates the encoded dictionary prompt data;

[0014] Step 7: Use the trained neural machine translation model that incorporates the encoded dictionary prompt data to process the input source sentence to be translated to obtain the output target sentence.

[0015] Furthermore, the Step 1 includes:

[0016] (1) Directly crawl the Thai-Chinese legal field parallel text corpus;

[0017] (2) Download Thai legal documents, extract the text in the Thai legal documents and translate them into Chinese text to obtain Thai-Chinese parallel text corpus in the legal field;

[0018] (3) Crawl the Thai-English parallel text corpus in the legal field, and use English as the pivot to obtain the Thai-Chinese parallel text corpus in the legal field.

[0019] Furthermore, Step 2 includes:

[0020] Step 2.1. Use the fast_align tool to extract Thai-Chinese word alignments from the constructed Thai-Chinese legal field parallel text corpus, remove duplicates, and integrate them into a Thai-Chinese word alignment dictionary;

[0021] Step 2.2, use the jieba.analyse.textrank function to obtain Chinese keywords from the Chinese version of the constructed Thai-Chinese legal field parallel text corpus;

[0022] Step 2.3, use the jieba.posseg function to perform part-of-speech tagging on the Chinese side of the constructed Thai-Chinese legal field parallel text corpus, and obtain three types of POS part-of-speech words: Chinese noun n, verb v, and noun and verb nv;

[0023] Step 2.4, extracting three POS part-of-speech words, namely noun n, verb v, noun and verb nv, from Chinese keywords as Chinese domain keywords of POS part-of-speech;

[0024] Step 2.5. Extract word pairs of Chinese domain keywords whose target words are n, v and nv parts of speech from the Thai-Chinese word alignment dictionary to construct domain bilingual dictionaries of the three parts of speech: n, v and nv.

[0025] Furthermore, the Step 3 includes: in the Thai-Chinese public dataset, taking sentence pairs containing Chinese keywords in the domain bilingual dictionary of the nv part of speech on the Chinese side of the public dataset as a candidate pool, and using a screening algorithm to screen a fixed number of sentence pairs from the public dataset as an additional expanded training set.

[0026] Furthermore, the screening algorithms used in the screening process include a screening algorithm based on fuzzy matching and a screening method based on coverage of infrequent Chinese keywords in the legal field. The specific calculation process of the screening algorithm is as follows:

[0027] (1) For the fuzzy matching-based screening algorithm, calculate the first Chinese-side sentence of the legal field data against the Chinese-side sentence t in the candidate pool of the public dataset. out Based on the fuzzy matching score of EditDistance, sentence pairs in the public dataset with scores higher than the threshold are selected to join the selected pool and removed from the candidate pool. After that, each Chinese sentence in the legal field data t in Iteratively select sentence pairs in the candidate pool until the specified amount of data is selected; the screening algorithm process based on fuzzy matching is expressed as follows:

[0028]

[0029] Among them, FM is the abbreviation of fuzzy matching "FuzzyMatch", FM(t in ,t out) represents the Chinese sentence of legal field data in Compared with the Chinese sentence t in the candidate pool of the public dataset out The fuzzy matching score between EditDistance(t in ,t out ) is t in and t out The edit distance between two string types is the minimum number of operations required to convert one string into another, where the operation includes inserting a character, deleting a character, or replacing a character; max(|t in |,|t out |) means |t in | and |t out The maximum value in |; |.| refers to the length of the sentence;

[0030] (2) For the screening method based on the coverage of infrequent Chinese keywords in the legal field, the score of each candidate pool sentence pair is calculated, and a fixed number of sentence pairs with the highest sentence pair scores in the candidate pool are selected as the selected data. The specific calculation method is as follows:

[0031] Statistics of Chinese sentences in the candidate pool of public datasets t out The number of occurrences of the Chinese domain keyword dw in the sentence pair is cnt(dw), and a threshold value CT of the number of occurrences of the Chinese domain keyword is set. If the number of occurrences of the Chinese domain keyword is lower than the threshold CT, the score of the Chinese domain keyword in the sentence pair is set to the difference between the number of occurrences of the Chinese domain keyword and the threshold; otherwise, it is set to 0. The sentence pair score is the sum of the scores of the Chinese domain keywords in the Chinese end of the public dataset. The calculation process of the sentence pair score is as follows:

[0032]

[0033] Furthermore, the Step 4 includes:

[0034] Part-of-speech tagging is performed on the Thai source sentence, and the domain bilingual dictionary of the corresponding part-of-speech is matched. The part-of-speech tag and the matched Chinese domain keyword are combined to form dictionary prompt data; the dictionary prompt data consists of the part-of-speech tag and the prompt word. The prompt tag is the part-of-speech tag, and the prompt word is the target Chinese domain keyword matched by the Thai source word from the domain bilingual dictionary of the corresponding part-of-speech.

[0035] Furthermore, the specific steps of Step 4 include:

[0036] Step 4.1. Use the spaCy-Thai tool to perform part-of-speech tagging on the Thai source text to obtain the part-of-speech tagging information of the sentence. The words marked as NOUN are nouns, and the words marked as VERB are verbs.

[0037] Step 4.2. Construct dictionary prompt data based on Thai word part of speech and domain bilingual dictionary: If a Thai word belongs to the pos part of speech in the sentence, and the source word matches the target Chinese domain keyword corresponding to the pos part of speech in the domain bilingual dictionary, then the part of speech tag and the target Chinese domain keyword together constitute the dictionary prompt, and all dictionary prompts matched by the Thai source sentence together constitute the dictionary prompt data pt.

[0038] Furthermore, the Step 5 includes:

[0039] A shallow encoding representation is performed on the dictionary prompt data. A cross-attention module for the prompt encoder representation is added after the self-attention layer of the source sentence encoder, and the prompt encoder representation is concatenated with the source sentence encoder output representation.

[0040] Furthermore, the specific steps of Step 5 include:

[0041] Step 5.1: Add a prompt encoder to the neural machine translation model to perform shallow encoding on the dictionary prompt data pt, i.e., a word embedding layer plus an independent position encoding, where the prompt word shares the target word embedding and the prompt label is part of the source language vocabulary;

[0042] H PT =WprdEmbedding(pt)+PositionEncoding(pt) (3)

[0043] Among them, WordEmbedding(pt) means converting the dictionary prompt data pt into a tensor word embedding representation in the word embedding layer, and PositionEncoding(pt) means the position encoding of the dictionary prompt data pt;

[0044] Step 5.2: Input the source sentence into the source sentence encoder, and output the self-attention module at each source sentence encoder layer. An additional cross-attention module is added to the prompt encoder to represent HPT, namely the attention mechanism Prompt_Attention, the residual connection and the post-normalization layer. The cross-attention module outputs The calculation process is expressed as;

[0045] H self l =LayerNormalization(H l-1 +Self_Attention(H l-1 ,H l-1 ,H l-1 )) (4)

[0046]

[0047]

[0048] Among them, H l-1 is the output representation of the l-1th layer encoder. When l-1=0, H l-1 Refers to the output of the input layer; Self_Attention(,,) is the output of the multi-head self-attention mechanism; LayerNormalization(.) refers to layer normalization; d model Refers to the dimension of the model; the purpose of softmax(.) here is to normalize the calculation results to obtain the attention weight matrix, where each row represents the attention weight of the query on each column, the weight value is between 0 and 1, and the sum of the values of each column is 1; Prompt_Attention(,,) is used to indicate the calculation of the newly added cross attention mechanism;

[0049] Step 5.3: The output of the additional cross-attention module of each source sentence encoder layer is passed through the feed-forward network FFN to obtain the output representation H of the source sentence encoder at the lth layer. l ; Output representation H of the source sentence encoder at the first layer l Expressed as:

[0050]

[0051] FFN is the abbreviation of Feedforward Network. FFN(.) is a network that obtains an intermediate layer through a linear fully connected layer and an activation function, and then obtains the output through a linear fully connected layer.

[0052] Step 5.4: Represent the last layer output of the new source sentence encoder H L With the hint encoder representation H PT The concatenation is performed as the final encoding-end representation H. When propagating forward through the decoder's cross-attention module, in addition to calculating the attention on the source sentence encoder part, the decoder also introduces additional attention to the prompt encoder representation through cross-attention. The final encoding-end representation H is expressed as:

[0053] H=Concat(H L ,H PT )=[H L ;H PT ] (8)

[0054] Concat(.) represents a concatenation operation.

[0055] In Step 7, during the translation phase of the neural machine translation model, dictionary prompt data is matched to the source sentence to be translated, and the source sentence and dictionary prompt data are input into a new encoder end of the neural machine translation model. In each decoding step, the neural machine translation model converts the decoder output hidden state into a predicted probability distribution vector through the linear layer and the Softmax layer after the decoder, where the value of each dimension represents the probability of generating the word, and then obtains the word corresponding to a dimension of the vector in the target vocabulary as output according to the decoding algorithm. Each time a target word is generated, the generated target word is added to the generated target sequence and input into the decoder for the next decoding step. After multiple decoding steps, the neural machine translation model gradually generates the target word by autoregression until it encounters an end symbol or reaches the set maximum length, thereby obtaining the target sentence sequence.

[0056] The present invention also provides a Thai-Chinese legal domain neural machine translation system integrated with a domain bilingual dictionary. The system includes: a module for executing the Thai-Chinese legal domain neural machine translation method integrated with a domain bilingual dictionary.

[0057] The beneficial effects of the present invention are:

[0058] 1. This paper introduces more contextual information for low-frequency domain words through a screening method based on the coverage of infrequent Chinese words in the legal field. It also introduces dictionary prompt representation information to the model encoding end and reduces the impact of redundant noise by adding a cross-attention module, effectively improving model performance. Through case analysis, it is found that this method can improve the translation accuracy of domain keywords.

[0059] 2. The present invention can screen out-of-domain data containing domain keywords to expand the training set. The screening method based on the coverage of infrequent Chinese words in the legal field has a faster screening speed and can provide richer contextual information for low-frequency domain words, bringing greater performance improvement.

[0060] 3. The cross-attention module of the present invention enables the neural machine translation model to dynamically pay attention to the prompt information when processing the source sentence, so that the neural machine translation model can better understand the relationship between the source sentence and the prompt, reduce the interference of irrelevant noise information, and thus improve the performance of the neural machine translation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is the overall framework diagram of the method of the present invention;

[0062] Figure 2 Flowchart of bilingual dictionary mining in the field of the present invention. DETAILED DESCRIPTION

[0063] Example 1: Figure 1-Figure 2As shown, a Thai-Chinese legal domain neural machine translation method incorporating a domain bilingual dictionary comprises:

[0064] Step 1: Construct a Thai-Chinese parallel text corpus in the legal field;

[0065] Furthermore, the Step 1 includes:

[0066] (1) Directly crawl the Thai-Chinese legal field parallel text corpus;

[0067] (2) Download Thai legal documents, extract the text in the Thai legal documents and translate them into Chinese text to obtain Thai-Chinese parallel text corpus in the legal field;

[0068] (3) Crawl the Thai-English parallel text corpus in the legal field, and use English as the pivot to obtain the Thai-Chinese parallel text corpus in the legal field.

[0069] Step 2: Use tools to mine bilingual dictionaries from the constructed Thai-Chinese legal field parallel text corpus;

[0070] Furthermore, Step 2 includes:

[0071] Step 2.1. Use the fast_align tool to extract Thai-Chinese word alignments from the constructed Thai-Chinese legal field parallel text corpus, remove duplicates, and integrate them into a Thai-Chinese word alignment dictionary;

[0072] Step 2.2, use the jieba.analyse.textrank function to obtain Chinese keywords from the Chinese version of the constructed Thai-Chinese legal field parallel text corpus;

[0073] Step 2.3, use the jieba.posseg function to perform part-of-speech tagging on the Chinese side of the constructed Thai-Chinese legal field parallel text corpus, and obtain three types of POS part-of-speech words: Chinese noun n, verb v, and noun and verb nv;

[0074] Step 2.4, extracting three POS part-of-speech words, namely noun n, verb v, noun and verb nv, from Chinese keywords as Chinese domain keywords of POS part-of-speech;

[0075] Step 2.5. Extract word pairs of Chinese domain keywords whose target words are n, v and nv parts of speech from the Thai-Chinese word alignment dictionary to construct domain bilingual dictionaries of the three parts of speech: n, v and nv.

[0076] Step 3: Use the domain bilingual dictionary to filter “pseudo-domain” data from the Thai-Chinese public dataset to expand the training set;

[0077] Furthermore, Step 3 includes: in the Thai-Chinese public dataset, using the sentence pairs containing Chinese keywords in the nv part-of-speech domain bilingual dictionary in the Chinese side of the public dataset as a candidate pool, and using a screening algorithm to screen a fixed number of 5k sentence pairs containing Chinese keywords in the nv part-of-speech domain bilingual dictionary from the Chinese side of the public dataset as an additional expanded training set.

[0078] Furthermore, the screening algorithms used in the screening process include a screening algorithm based on fuzzy matching and a screening method based on coverage of infrequent Chinese keywords in the legal field. The specific calculation process of the screening algorithm is as follows:

[0079] (1) For the screening algorithm based on fuzzy matching, it calculates the Chinese end sentence of the first legal field data based on structural similarity to the Chinese end sentence of the candidate pool in the public data set t out Based on the fuzzy matching score of EditDistance, sentence pairs in the public dataset with scores higher than the threshold are selected to join the selected pool. The score threshold is set to 0.3 and removed from the candidate pool. After that, each Chinese sentence in the legal field data t in Iteratively select sentence pairs in the candidate pool until the specified amount of data is selected; the screening algorithm process based on fuzzy matching is expressed as follows:

[0080]

[0081] Among them, FM is the abbreviation of fuzzy matching "FuzzyMatch", FM(t in ,t out ) represents the Chinese sentence of legal field data in Compared with the Chinese sentence t in the candidate pool of the public dataset out The fuzzy matching score between EditDistance(t in ,t out ) is t in and t out The edit distance between two string types is the minimum number of operations required to convert one string into another, where the operation includes inserting a character, deleting a character, or replacing a character; max(|t in |,|t out |) means |t in | and |t out The maximum value in |; |.| refers to the length of the sentence;

[0082] (2) For the screening method based on the coverage of infrequent Chinese keywords in the legal field, the score of each candidate pool sentence pair is calculated, and a fixed number of sentence pairs with the highest sentence pair scores in the candidate pool are selected as the selected data. Due to the characteristics of its scoring function, the frequency threshold is set to 3, and the screened "pseudo-field" training set is more likely to contain richer context information of low-frequency field keywords; the specific calculation method is as follows:

[0083] Statistics of Chinese sentences in the candidate pool of public datasets t out The number of occurrences of the Chinese domain keyword dw in the sentence pair is cnt(dw), and a threshold value CT of the number of occurrences of the Chinese domain keyword is set. If the number of occurrences of the Chinese domain keyword is lower than the threshold CT, the score of the Chinese domain keyword in the sentence pair is set to the difference between the number of occurrences of the Chinese domain keyword and the threshold; otherwise, it is set to 0. The sentence pair score is the sum of the scores of the Chinese domain keywords in the Chinese end of the public dataset. The calculation process of the sentence pair score is as follows:

[0084]

[0085] Step 4: Construct dictionary prompt data for the bilingual dictionary in the Thai source sentence matching domain;

[0086] Furthermore, the Step 4 includes:

[0087] Parts of speech are tagged for the Thai source sentences, and the domain bilingual dictionary of the corresponding part of speech is matched. The part-of-speech tags and the matched Chinese domain keywords are combined to form dictionary prompt data; dictionary prompts are matched for the Thai source sentences from the domain bilingual dictionary. The dictionary prompt data consists of part-of-speech tags and prompt words. The prompt tags are the part-of-speech tags, and the prompt words are the target Chinese domain keywords matched by the Thai source words from the domain bilingual dictionary of the corresponding part of speech.

[0088] Furthermore, the specific steps of Step 4 include:

[0089] Step 4.1. Use the spaCy-Thai tool to perform part-of-speech tagging on the Thai source text to obtain the part-of-speech tagging information of the sentence. The words marked as NOUN are nouns, and the words marked as VERB are verbs.

[0090] Step 4.2: Construct dictionary prompt data based on Thai word part of speech and domain bilingual dictionary: If a Thai word in a sentence belongs to the pos part of speech, and the source word matches the target Chinese domain keyword in the domain bilingual dictionary of the pos part of speech, then the part of speech tag and the target Chinese domain keyword together constitute the dictionary prompt. All dictionary prompts matching the Thai source sentence together constitute the dictionary prompt data pt. The prompt data construction process is described in Table 1:

[0091] Table 1 shows the prompt data construction process

[0092]

[0093] Furthermore, the Step 5 includes:

[0094] A shallow encoding representation is performed on the dictionary prompt data. A cross-attention module for the prompt encoder representation is added after the self-attention layer of the source sentence encoder, and the prompt encoder representation is concatenated with the source sentence encoder output representation.

[0095] Step 5: Encode the dictionary prompt data and integrate it into the neural machine translation model of the Transformer architecture;

[0096] Furthermore, the specific steps of Step 5 include:

[0097] Step 5.1: Add a prompt encoder to the neural machine translation model to perform shallow encoding on the dictionary prompt data pt, i.e., a word embedding layer plus an independent position encoding, where the prompt word shares the target word embedding and the prompt label is part of the source language vocabulary;

[0098] H PT =WordEmbedding(pt)+PositionEncoding(pt) (3)

[0099] Among them, WordEmbedding(pt) means converting the dictionary prompt data pt into a tensor word embedding representation in the word embedding layer, and PositionEncoding(pt) means the position encoding of the dictionary prompt data pt;

[0100] Step 5.2: Input the source sentence into the source sentence encoder, and output the self-attention module at each source sentence encoder layer. Then add an additional pair of prompt encoder representation H PT The cross attention module, namely the attention mechanism Prompt_Attention and the residual connection and post-normalization layer, the cross attention module output The calculation process is expressed as;

[0101]

[0102] Among them, H l-1 is the output representation of the l-1th layer encoder. When l-1=0, H l-1 Refers to the output of the input layer; Self_Attention(,,) is the output of the multi-head self-attention mechanism; LayerNormalization(.) refers to layer normalization; d modelRefers to the dimension of the model. The default setting of Transformer-base is 512. In practice, multi-head attention calculation is adopted. The purpose of softmax(.) here is to normalize the calculation results to obtain the attention weight matrix, in which each row represents the attention weight of the query on each column. The weight value is between 0 and 1, and the sum of the values of each column is 1. Prompt_Attention(,,) is used to indicate the calculation of the newly added cross attention mechanism.

[0103] Step 5.3: The output of the additional cross-attention module of each source sentence encoder layer is passed through the feed-forward network (FFN) to obtain the output representation Hl of the source sentence encoder of the lth layer; the output representation Hl of the source sentence encoder of the lth layer l Expressed as:

[0104]

[0105] FFN is the abbreviation of Feedforward Network. FFN(.) is a network that obtains an intermediate layer through a linear fully connected layer and an activation function, and then obtains the output through a linear fully connected layer.

[0106] Step 5.4: Represent the last layer output of the new source sentence encoder H L With the hint encoder representation H PT The concatenation is performed as the final encoding-end representation H. When propagating forward through the decoder's cross-attention module, in addition to calculating the attention on the source sentence encoder part, the decoder also introduces additional attention to the prompt encoder representation through cross-attention. The final encoding-end representation H is expressed as:

[0107] H=Concat(H L ,H PT )=[H l ;H PT ] (8)

[0108] Concat(.) represents the concatenation operation;

[0109] If the number of rows in the encoder's last output matrix equals the number of tokens in the sentence and the number of columns equals the model dimension of 512, then Concat(.) concatenates the prompt representation matrix to the last row of the encoder's last output matrix, adding rows. The final encoding matrix has the number of rows equal to the number of tokens in the sentence plus the number of tokens in the prompt data. Due to the maximum row limit and padding mask, there is no need to worry about row inconsistency.

[0110] Step 6: Use the expanded training set to train the neural machine translation model that incorporates the encoded dictionary prompt data.

[0111] The corresponding model of the present invention is a neural machine translation model that integrates domain bilingual dictionary prompt information based on the Fairseq framework. It uses the default settings of Transformer-base, a 6-layer encoder and decoder, a word embedding matrix with a vector dimension of 512, a hidden layer dimension of 512, a feedforward neural network with 2048 intermediate layer units, and 8 multi-head attention heads. During model training, the batch size of each training sample is set to 512; the initial learning rate is set to 1e-4, and the warm-up strategy is used to update the learning rate with 4000 steps. Adam is used as the optimizer, with a label smoothing value of 0.1 and a dropout value of 0.1. During model inference, a beam search algorithm is used in the decoding process, and the search beam width beam is set to 5.

[0112] Shallow encoding of dictionary prompt data into the neural machine translation model: The dictionary prompt data is shallowly encoded. This encoder is different from the transformer encoder. In order to avoid introducing redundant information, only shallow encoding is performed, that is, the word embedding layer is added with independent position encoding, where the prompt word shares the target word embedding, and the prompt label is part of the source language vocabulary; a cross-attention module for the prompt representation is added after the output of the self-attention module of each source sentence encoder layer to realize the interaction between the source sentence context representation and the prompt representation, thereby reducing the impact of redundant constraints; the output representation of the last layer of the new source sentence encoder is spliced with the prompt representation as the final encoding end representation, so that when forward propagating through the cross-attention module of the decoder, in addition to calculating the attention on the source sentence encoder representation part, the cross-attention module also introduces additional attention to the prompt information.

[0113] The cross-attention mechanism allows the neural machine translation model to dynamically select the most relevant parts to focus on when processing input data; in the decoder of the neural machine translation model, the cross-attention dynamically selects the most useful information from the input text based on the currently generated context, thereby improving the quality of the generated content; the cross-attention module added by the method of the present invention enables the model to dynamically pay attention to the prompt information when processing the source sentence, so that the neural machine translation model can better understand the relationship between the source sentence and the prompt, reduce the interference of irrelevant noise information, and thus improve the performance of the neural machine translation model.

[0114] Step 7: Use the trained neural machine translation model that incorporates the encoded dictionary prompt data to process the input source sentence to be translated to obtain the output target sentence.

[0115] In Step 7, during the translation phase of the neural machine translation model, the source sentence to be translated is matched with the dictionary prompt data, and the source sentence and the dictionary prompt data are input into the new encoder end of the neural machine translation model. <bos>The start character is input to the decoder (in the training phase, it is necessary to add <bos>, add after the sentence <eos>); In each decoding step, the neural machine translation model converts the decoder output hidden state into a predicted probability distribution vector through the linear layer and Softmax layer after the decoder, where the value of each dimension represents the probability of generating the word, and then obtains the word of a corresponding dimension of the vector in the target vocabulary as the output according to the decoding algorithm; each time a target word is generated, the generated target word is added to the generated target sequence and input into the decoder for the next decoding step; after multiple decoding steps, the neural machine translation model gradually generates the target word through autoregression until it encounters the end symbol <eos>Or reach the set maximum length to obtain the target sentence sequence.

[0116] In decoding algorithms, greedy search only selects words with the largest median value in the predicted probability distribution vector for generation. This can easily lead to local optimal solutions and prevents global optimal solutions. Therefore, beam search is commonly used in translation. At each decoding step, beam-number of candidate words with the highest probability are retained to form multiple candidate sequences. These candidate sequences are then expanded until a sentence is formed. This results in multiple candidate translations, and the candidate translation with the highest sum of the logarithmic probabilities of each word is selected as the output sentence.

[0117] The present invention also provides a Thai-Chinese legal domain neural machine translation system integrated with a domain bilingual dictionary, the system comprising:

[0118] The domain parallel text corpus construction module is used to construct Thai-Chinese parallel text corpora in the legal field;

[0119] The domain bilingual dictionary mining module is used to use tools to mine domain bilingual dictionaries from the constructed Thai-Chinese legal field parallel text corpus;

[0120] The training set expansion module is used to expand the training set by filtering "pseudo-domain" data from Thai-Chinese public datasets using domain bilingual dictionaries;

[0121] A dictionary hint data construction module is used to construct dictionary hint data for a bilingual dictionary in the Thai source sentence matching domain;

[0122] The integration module is used to encode and represent the dictionary prompt data and integrate it into the neural machine translation model of the Transformer architecture;

[0123] A training module for training a neural machine translation model incorporating the encoded dictionary prompt data using the expanded training set;

[0124] The translation module is used to use the trained neural machine translation model that incorporates the encoded dictionary prompt data to process the input source sentence to be translated to obtain the output target sentence.

[0125] Experiment 1: To verify the effectiveness of the data screening method, we compared the Thai-Chinese translation performance of baseline Transformer models trained on different data. "law" is a training set based on pure legal data; "law+fuzz" is an expanded training set using a screening method based on fuzzy matching; "law+lfm" is an expanded training set using a screening method based on frequency matching of Chinese legal terms; and "law+ilr" is an expanded training set using a screening method based on coverage of infrequent legal Chinese terms. The results are shown in Table 2.

[0126] Table 2 shows the translation performance of the baseline model on different training sets.

[0127]

[0128] The experimental results in Table 2 show that the BLEU values of the three methods for filtering additional training sets have all been improved compared to using only the in-domain training set. The BLUE value of the method based on fuzzy matching only increased by 0.26. This is because the edit distance between the external data and the Chinese end of the domain data is generally relatively far, and the filtered external data contains less usable similar semantic knowledge. The BLEU value of the filtering method based on frequency matching of Chinese words in the legal field increased by 0.35. This is because the filtered data provides the target end with more domain vocabulary context, which improves the generation ability of the model. The BLEU value of the method based on the coverage of infrequent Chinese words in the legal field increased by 0.54. This is because the filtered external data contains richer context for low-frequency domain words, which improves the translation performance of some low-frequency domain words.

[0129] Experiment 2: To verify the effectiveness of the neural machine translation model incorporating dictionary hints, the present invention is compared with two flexible classical soft vocabulary-constrained neural machine translation methods, where both models are trained using "law+ilr" data and a Transformer model as the "Base". The domain bilingual dictionary target words matched by the present invention's method are used as constraint vocabulary. The "code-switched" constraint method proposed by Song et al., which adds a code-switched corpus with internal substitutions in the source sentence and adds a pointer network, is called "CS+Ptr"; the constraint method proposed by Chen et al., which performs data augmentation on the back connection constraints of the source sentence and adds segmented embedding and a pointer network, is called "LeCA+Ptr"; the main experiment of the present invention's method is called "Ours". The results are shown in Table 3:

[0130] Table 3 shows the experimental results of different vocabulary constraint methods

[0131]

[0132] Experimental comparisons in Table 3 show that the Thai-Chinese translation results using the hint model of our invention achieve a BLEU improvement of 0.61 compared to the lexically constrained model of the "CS+Ptr" method. This is because the "CS" approach, which uses target constraint words to modify the source sentence as augmented data, preserves the original source context during training but disrupts the source sentence structure during inference, leading to a loss of some source semantic information, particularly when the constraint words are noisy. The proposed model achieves a BLEU improvement of 0.34 compared to the "LeCA+Ptr" model. This is due to the unavoidable situation where a source word is matched to multiple target words in the domain bilingual dictionary or the dictionary is noisy. This complexity introduces ambiguity into the source synthetic data, leading to biased context representation and thus propagating errors throughout the model structure. Our model, which does not modify the source sentence but interacts with hint information, mitigates this problem to some extent. Compared to the two classical constraint methods, our proposed method achieves significant improvements in both BLEU and CSR, demonstrating that our method can effectively leverage domain bilingual dictionary constraints and improve the translation accuracy of domain keywords in the presence of noisy constraints.

[0133] Experiment 3: To verify the effectiveness of each module of the new model, "law+ilr" data was used for training. The traditional source encoder-decoder Transformer baseline model was called "1:Base". Based on the model "1:Base", only the shallow encoding output of the source encoder output hidden state splicing prompt data was added, and this model was called "2:1+Concat". Based on the model "1:Base", only a cross-attention module for the prompt representation was added after the self-attention layer of each encoder layer, and this model was called "3:1+Prompt_Attention". For the convenience of intuitive comparison of the models, the complete model "Ours" with the addition of the cross-attention module and splicing prompt representation is called "4:1+Concat+Pr-ompt_Attention". The model with a pointer network added to the generation end based on the model of the present invention is called "5:4+Ptr". The experimental results are shown in Table 4:

[0134] Table 4 shows the ablation experiment results to verify the effectiveness of each module

[0135]

[0136] The ablation test results in Table 4 show that Model 2 (2:1+Concat) introduces a hint representation to the encoding representation, compared to the baseline Model 1 (1:Base). This results in a high CSR of 80.0%, demonstrating that the inclusion of the hint representation maintains a high level of domain keyword translation accuracy. However, the BLEU score only improves by 0.23, indicating that relying solely on concatenated hint representations has limited performance benefits. Model 3 (3:1+Prompt_Attention) adds an additional cross-attention module for the hint representation, compared to Model 1 (1:Base), achieving a 0.49 BLEU improvement and a 3.2% CSR improvement compared to Model 2 (2:1+Concat), demonstrating the effectiveness of this cross-attention module in improving both model performance and domain keyword translation accuracy. The complete model 4 "4:1+Concat+Prompt_Attention" of the present invention has an improvement of 0.90 in BLEU value compared with model 1 "1:Base", and the CSR value is improved by 6.3% and 3.1% respectively compared with model 2 "2:1+Concat" and model 3 "3:1+Prompt_Attention", which shows that the combination of prompt representation splicing and the cross-attention module can play the greatest role and more effectively improve the model performance and domain keyword translation accuracy; it is worth noting that it has an improvement of 0.67 in BLEU value compared with model 2 "2:1+Concat" which only introduces prompt representation, further reflecting the importance of cross-attention, which may be because the absence of this attention module will introduce redundant information and greatly reduce the effect of prompt representation on the model. Compared with the model 4 "4:1+Concat+Pr-ompt_Attention" of the present invention, model 5 "5:4+Ptr" adds a pointer network, but its BLEU value is reduced by 0.33. This may be because the model overfits the inaccurate pointer network prediction probability, resulting in biased model prediction; its CSR value is reduced by 2.2% compared with model 4 "4:1+Concat+Pr-ompt_Attention", indicating that the prediction bias reduces the translation accuracy of domain keywords.

[0137] Example Analysis: To intuitively demonstrate the translation performance of our method, we selected typical sentences from a Thai-Chinese legal field test set containing a large number of domain keywords. We compared the sentences translated by our method with those translated by the baseline model. The results are shown in Table 5:

[0138] Table 5 is an analysis of translation task examples

[0139]

[0140] The results in Table 5 show that When the proposed method correctly translates the target word into the corresponding term in the domain bilingual dictionary, "regulations," while the baseline Transformer model incorrectly translates it as "order." This error may be caused by the fact that the prediction probabilities of the two terms in the generated prediction vectors are the same or similar. The proposed model incorporates the embedded representation of the target word as hint information, enabling accurate translation. This example demonstrates that the proposed method can improve the translation accuracy of domain keywords, thereby facilitating people's understanding of legal translations.

[0141] The specific embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.< / eos> < / eos> < / bos> < / bos>

Claims

1. A neural machine translation method for Thai-Chinese legal domain that incorporates a domain bilingual dictionary, characterized by: The method comprises: Step 1: Construct a Thai-Chinese parallel text corpus in the legal field; Step 2: Use tools to mine bilingual dictionaries from the constructed Thai-Chinese legal field parallel text corpus; Step 3: Use domain bilingual dictionaries to filter "pseudo-domain" data from Thai-Chinese public datasets to expand the training set; Step 4: Construct dictionary prompt data for the bilingual dictionary in the Thai source sentence matching domain; Step 5: Encode the dictionary prompt data and integrate it into the neural machine translation model of the Transformer architecture; Step 6: Use the expanded training set to train the neural machine translation model that incorporates the encoded dictionary prompt data; Step 7: Use the trained neural machine translation model that incorporates the encoded dictionary prompt data to process the input source sentence to be translated to obtain the output target sentence.

2. The Thai-Chinese legal domain neural machine translation method incorporating a domain bilingual dictionary according to claim 1 is characterized by: Step 1 includes: (1) Directly crawl the Thai-Chinese legal field parallel text corpus; (2) Download Thai legal documents, extract the text in the Thai legal documents and translate them into Chinese text to obtain Thai-Chinese parallel text corpus in the legal field; (3) Crawl the Thai-English parallel text corpus in the legal field, and use English as the pivot to obtain the Thai-Chinese parallel text corpus in the legal field.

3. The Thai-Chinese legal domain neural machine translation method incorporating a domain bilingual dictionary according to claim 1 is characterized by: Step 2 includes: Step 2.

1. Use the fast_align tool to extract Thai-Chinese word alignments from the constructed Thai-Chinese legal field parallel text corpus, remove duplicates, and integrate them into a Thai-Chinese word alignment dictionary; Step 2.2, use the jieba.analyse.textrank function to obtain Chinese keywords from the Chinese version of the constructed Thai-Chinese legal field parallel text corpus; Step 2.3, use the jieba.posseg function to perform part-of-speech tagging on the Chinese side of the constructed Thai-Chinese legal field parallel text corpus, and obtain three types of POS part-of-speech words: Chinese noun n, verb v, and noun and verb nv; Step 2.4, extracting three POS part-of-speech words, namely noun n, verb v, noun and verb nv, from Chinese keywords as Chinese domain keywords of POS part-of-speech; Step 2.

5. Extract word pairs of Chinese domain keywords whose target words are n, v and nv parts of speech from the Thai-Chinese word alignment dictionary to construct domain bilingual dictionaries of the three parts of speech: n, v and nv.

4. The Thai-Chinese legal domain neural machine translation method incorporating a domain bilingual dictionary according to claim 1 is characterized by: The step 3 includes: in the Thai-Chinese public dataset, using the sentence pairs of Chinese keywords in the domain bilingual dictionary containing the nv part of speech in the Chinese side of the public dataset as a candidate pool, and using a screening algorithm to screen a fixed number of sentence pairs from the public dataset as an additional expanded training set.

5. The Thai-Chinese legal domain neural machine translation method incorporating a domain bilingual dictionary according to claim 1 is characterized by: The screening algorithms used in the screening process include a fuzzy matching-based screening algorithm and a screening method based on the coverage of infrequent Chinese keywords in the legal field. The specific calculation process of the screening algorithm is as follows: (1) For the fuzzy matching-based screening algorithm, calculate the first Chinese-side sentence of the legal field data against the Chinese-side sentence t in the candidate pool of the public dataset. out Based on the fuzzy matching score of EditDistance, sentence pairs in the public dataset with scores higher than the threshold are selected to join the selected pool and removed from the candidate pool. After that, each Chinese sentence in the legal field data t in Iteratively select sentence pairs in the candidate pool until the specified amount of data is selected; the screening algorithm process based on fuzzy matching is expressed as follows: Among them, FM is the abbreviation of "Fuzzy Match", FM(t in ,t out ) represents the Chinese sentence of legal field data in Compared with the Chinese sentence t in the candidate pool of the public dataset out The fuzzy matching score between EditDistance(t in ,t out ) is t in and t out The edit distance between two string types is the minimum number of operations required to convert one string into another, where the operation includes inserting a character, deleting a character, or replacing a character; max(|t in |,|t out |) means |t in | and |t out The maximum value in |; |.| refers to the length of the sentence; (2) For the screening method based on the coverage of infrequent Chinese keywords in the legal field, the score of each candidate pool sentence pair is calculated, and a fixed number of sentence pairs with the highest sentence pair scores in the candidate pool are selected as the selected data. The specific calculation method is as follows: Statistics of Chinese sentences in the candidate pool of public datasets t out The number of occurrences of the Chinese domain keyword dw in the sentence pair is cnt(dw), and a threshold value CT of the number of occurrences of the Chinese domain keyword is set. If the number of occurrences of the Chinese domain keyword is lower than the threshold CT, the score of the Chinese domain keyword in the sentence pair is set to the difference between the number of occurrences of the Chinese domain keyword and the threshold; otherwise, it is set to 0. The sentence pair score is the sum of the scores of the Chinese domain keywords in the Chinese end of the public dataset. The calculation process of the sentence pair score is as follows:

6. The Thai-Chinese legal domain neural machine translation method incorporating a domain bilingual dictionary according to claim 1 is characterized by: Step 4 includes: Part-of-speech tagging is performed on the Thai source sentence, and the domain bilingual dictionary of the corresponding part-of-speech is matched. The part-of-speech tag and the matched Chinese domain keyword are combined to form dictionary prompt data; the dictionary prompt data consists of the part-of-speech tag and the prompt word. The prompt tag is the part-of-speech tag, and the prompt word is the target Chinese domain keyword matched by the Thai source word from the domain bilingual dictionary of the corresponding part-of-speech.

7. The Thai-Chinese legal domain neural machine translation method incorporating a domain bilingual dictionary according to claim 1 is characterized by: The specific steps of Step 4 include: Step 4.

1. Use the spaCy-Thai tool to perform part-of-speech tagging on the Thai source text to obtain the part-of-speech tagging information of the sentence. The words marked as NOUN are nouns, and the words marked as VERB are verbs. Step 4.

2. Construct dictionary prompt data based on Thai word part of speech and domain bilingual dictionary: If a Thai word belongs to the pos part of speech in the sentence, and the source word matches the target Chinese domain keyword corresponding to the pos part of speech in the domain bilingual dictionary, then the part of speech tag and the target Chinese domain keyword together constitute the dictionary prompt, and all dictionary prompts matched by the Thai source sentence together constitute the dictionary prompt data pt.

8. The Thai-Chinese legal domain neural machine translation method incorporating a domain bilingual dictionary according to claim 1 is characterized by: Step 5 includes: A shallow encoding representation is performed on the dictionary prompt data. A cross-attention module for the prompt encoder representation is added after the self-attention layer of the source sentence encoder, and the prompt encoder representation is concatenated with the source sentence encoder output representation.

9. The Thai-Chinese legal domain neural machine translation method incorporating a domain bilingual dictionary according to claim 1 is characterized by: The specific steps of Step 5 include: Step 5.1: Add a prompt encoder to the neural machine translation model to perform shallow encoding on the dictionary prompt data pt, i.e., a word embedding layer plus an independent position encoding, where the prompt word shares the target word embedding and the prompt label is part of the source language vocabulary; H PT =WordEmbedding(pt)+PositionEncoding(pt) (3) Among them, WordEmbedding(pt) means converting the dictionary prompt data pt into a tensor word embedding representation in the word embedding layer, and PositionEncoding(pt) means the position encoding of the dictionary prompt data pt; Step 5.2: Input the source sentence into the source sentence encoder, and output the self-attention module at each source sentence encoder layer. Then add an additional pair of prompt encoder representation H PT The cross attention module, namely the attention mechanism Prompt_Attention and the residual connection and post-normalization layer, the cross attention module output The calculation process is expressed as; Among them, H l-1 is the output representation of the l-1th layer encoder. When l-1=0, H l-1 Refers to the output of the input layer; Self_Attention(,,) is the output of the multi-head self-attention mechanism; LayerNormalization(.) refers to layer normalization; d model Refers to the dimension of the model; the purpose of softmax(.) here is to normalize the calculation results to obtain the attention weight matrix, where each row represents the attention weight of the query on each column, the weight value is between 0 and 1, and the sum of the values of each column is 1; Prompt_Attention(,,) is used to indicate the calculation of the newly added cross attention mechanism; Step 5.3: The output of the additional cross-attention module of each source sentence encoder layer is passed through the feed-forward network FFN to obtain the output representation H of the source sentence encoder at the lth layer. l ; Output representation H of the source sentence encoder at the first layer l Expressed as: FFN is the abbreviation of "Feedforward Network". FFN (.) is obtained by a linear fully connected layer and an activation function to obtain an intermediate layer, and the intermediate layer is then passed through a linear fully connected layer to obtain the output; Step 5.4: Represent the last layer output of the new source sentence encoder H L With the hint encoder representation H PT The concatenation is performed as the final encoding-end representation H. When propagating forward through the decoder's cross-attention module, in addition to calculating the attention on the source sentence encoder part, the decoder also introduces additional attention to the prompt encoder representation through cross-attention. The final encoding-end representation H is expressed as: H=Concat(H L ,H PT )=[H L ;H PT ] (8) Concat(.) represents a concatenation operation.

10. A Thai-Chinese legal domain neural machine translation system incorporating a domain bilingual dictionary, characterized by: The system includes: a module for executing the Thai-Chinese legal domain neural machine translation method incorporating a domain bilingual dictionary according to any one of claims 1 to 9.

Images