Machine translation model training method and device, and text translation method and device

Abstract

The application relates to a machine translation model training method and device, a text translation method and device, computer equipment, a storage medium and a computer program product, relates to the technical field of artificial intelligence, and can be used in the field of financial technology or other related fields.The method comprises the following steps: acquiring a training sample set; a first preset algorithm and a second preset algorithm are used for carrying out first word segmentation processing and second word segmentation processing on source language corpus respectively, so as to obtain a first subword sequence and a plurality of second subword sequences; a first subsequence of a target word in the source language corpus in the first subword sequence is compared with a second subsequence in the plurality of second subword sequences in terms of difference degree, and the second subsequence with the largest difference degree is determined as a target subsequence corresponding to the target word; and the first subsequence corresponding to the target word is replaced with the target subsequence, which is used for training a machine translation model. By using the method, the translation accuracy of the machine translation model can be improved, and the robustness of the machine translation model can be enhanced.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a machine translation model training method and apparatus, a text translation method and apparatus, computer equipment, storage medium and computer program products. Background Technology

[0002] With economic globalization and the development of artificial intelligence technology, machine translation technology has been widely used in many fields, including fintech. Among them, neural machine translation (NMT) is a technique that uses neural networks to maximize machine translation performance. That is, it uses bilingual or multilingual training corpora to train neural networks to obtain machine translation models, which are then used to translate input source language text and output target language text.

[0003] When training a machine translation model, the original training corpus needs to be segmented into words. This segmentation divides the corpus into sequences of words, where each word is the basic unit of translation. The segmented training corpus is then used to train the machine translation model. Currently, Byte Pair Encoding (BPE) is commonly used for word segmentation. However, machine translation models trained using this method have low accuracy with noisy text and exhibit poor robustness. Therefore, a better training method for machine translation models that improves translation accuracy and enhances model robustness is urgently needed. Summary of the Invention

[0004] Therefore, it is necessary to provide a machine translation model training method and apparatus, a text translation method and apparatus, a computer device, a computer-readable storage medium, and a computer program product that can improve translation accuracy and enhance model robustness in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for training a machine translation model. The method includes:

[0006] Obtain a training sample set; the training sample set includes source language corpus and target language corpus corresponding to the source language corpus;

[0007] The training sample set is processed by a first preset algorithm to obtain a first sub-word sequence corresponding to the source language corpus and a sample sub-word sequence corresponding to the target language corpus.

[0008] The source language corpus is processed by a second preset algorithm to obtain multiple second sub-word sequences corresponding to the source language corpus.

[0009] The first subsequence corresponding to the target word in the first sub-word sequence in the source language corpus is compared with the second subsequence corresponding to the target word in multiple second sub-word sequences. The second subsequence with the largest difference is determined as the target subsequence corresponding to the target word.

[0010] In the first sub-word sequence corresponding to the source language corpus, the first sub-sequence corresponding to the target word is replaced with the target sub-sequence to obtain the sample sub-word sequence corresponding to the source language corpus;

[0011] The machine translation model is trained based on the sample word sequences corresponding to the source language corpus and the sample word sequences corresponding to the target language corpus.

[0012] In one embodiment, the method further includes:

[0013] Based on preset sampling rules, target words are determined from the words contained in the source language corpus.

[0014] In one embodiment, determining the target word from the words contained in the source language corpus based on a preset sampling rule includes:

[0015] A random number is assigned to each word in the source language corpus, and the random numbers conform to a uniform distribution.

[0016] The words corresponding to random numbers less than a preset threshold are identified as target words.

[0017] In one embodiment, the step of comparing the difference between the first subsequence corresponding to the target word in the first sub-word sequence in the source language corpus and the second subsequence corresponding to the target word in multiple second sub-word sequences, and determining the second subsequence with the largest difference as the target subsequence corresponding to the target word, includes:

[0018] Determine the first subsequence corresponding to the target word in the first sub-word sequence in the source language corpus, obtain the first sub-word vector corresponding to each sub-word in the first sub-sequence, and determine the first word vector corresponding to the target word based on the first sub-word vector;

[0019] Determine the second subsequence corresponding to the target word in multiple second sub-word sequences; for each second subsequence, obtain the second sub-word vector corresponding to each word in the second subsequence; and determine the second word vector corresponding to the target word based on the second sub-word vector.

[0020] Determine the difference between each second word vector and the first word vector, and determine the second subsequence corresponding to the second word vector with the largest difference as the target subsequence corresponding to the target word.

[0021] In one embodiment, determining the difference between each second word vector and the first word vector includes:

[0022] Based on the loss function of the machine translation model, the gradient vector corresponding to each word in the first sub-sequence is calculated, and the gradient vector corresponding to the target word is determined based on the gradient vector corresponding to each word.

[0023] The difference between each second word vector and the first word vector is calculated based on the first word vector, each second word vector, and the gradient vector corresponding to the target word.

[0024] In one embodiment, obtaining the training sample set includes:

[0025] Obtain an initial training sample set and add perturbation information to the initial training sample set to obtain the training sample set.

[0026] Secondly, this application also provides a text translation method. The method includes:

[0027] Obtain the text to be translated in the source language;

[0028] The text to be translated is input into a pre-trained machine translation model for translation, resulting in a translated text expressed in the target language.

[0029] Thirdly, this application also provides a machine translation model training device. The device includes:

[0030] The acquisition module is used to acquire a training sample set; the training sample set includes source language corpus and target language corpus corresponding to the source language corpus;

[0031] The first processing module is used to perform a first word segmentation process on the training sample set using a first preset algorithm to obtain a first sub-word sequence corresponding to the source language corpus and a sample sub-word sequence corresponding to the target language corpus.

[0032] The second processing module is used to perform second word segmentation on the source language corpus using a second preset algorithm to obtain multiple second sub-word sequences corresponding to the source language corpus.

[0033] The first determining module is used to compare the difference between the first subsequence corresponding to the target word in the first sub-word sequence in the source language corpus and the second subsequence corresponding to the target word in multiple second sub-word sequences, and determine the second subsequence with the largest difference as the target subsequence corresponding to the target word;

[0034] The replacement module is used to replace the first subsequence corresponding to the target word with the target subsequence in the first subsequence corresponding to the source language corpus, so as to obtain the sample subsequence corresponding to the source language corpus;

[0035] The training module is used to train the machine translation model based on the sample word sequences corresponding to the source language corpus and the sample word sequences corresponding to the target language corpus.

[0036] In one embodiment, the apparatus further includes a second determining module for:

[0037] Based on preset sampling rules, target words are determined from the words contained in the source language corpus.

[0038] In one embodiment, the second determining module is specifically used for:

[0039] A random number is assigned to each word in the source language corpus, and each random number conforms to a uniform distribution; the word corresponding to the random number less than a preset threshold is determined as the target word.

[0040] In one embodiment, the first determining module is specifically used for:

[0041] The process involves: determining the first subsequence corresponding to the target word in the first sub-word sequence within the source language corpus; obtaining the first sub-word vector corresponding to each word in the first sub-sequence; determining the first word vector corresponding to the target word based on the first sub-word vector; determining the second subsequence corresponding to the target word in multiple second sub-word sequences; obtaining the second sub-word vector corresponding to each word in each second sub-sequence; determining the second word vector corresponding to the target word based on the second sub-word vector; determining the difference between each second word vector and the first word vector; and determining the second subsequence corresponding to the second word vector with the largest difference as the target subsequence corresponding to the target word.

[0042] In one embodiment, the first determining module is specifically used for:

[0043] Based on the loss function of the machine translation model, the gradient vector corresponding to each word in the first sub-sequence is calculated, and the gradient vector corresponding to the target word is determined based on the gradient vector corresponding to each word; based on the first word vector, each second word vector, and the gradient vector corresponding to the target word, the difference between each second word vector and the first word vector is calculated.

[0044] In one embodiment, the acquisition module is specifically used for:

[0045] Obtain an initial training sample set and add perturbation information to the initial training sample set to obtain the training sample set.

[0046] Fourthly, this application also provides a text translation device. The device includes:

[0047] The acquisition module is used to acquire the text to be translated, expressed in the source language.

[0048] The translation module is used to input the text to be translated into a pre-trained machine translation model for translation, and to obtain the translated text expressed in the target language; wherein the machine translation model is trained by the machine translation model training method described in the first aspect.

[0049] Fifthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in the first or second aspect.

[0050] Sixthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in the first or second aspect.

[0051] In a seventh aspect, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in the first or second aspect.

[0052] The aforementioned machine translation model training method and apparatus, text translation method and apparatus, computer equipment, storage medium, and computer program product perform a first word segmentation process on the training sample set to obtain a first sub-word sequence corresponding to the source language corpus and a sample sub-word sequence corresponding to the target language corpus. Then, a second word segmentation process is performed on the source language corpus to obtain multiple second sub-word sequences corresponding to the source language corpus. The second sub-sequence corresponding to the target word in the multiple second sub-word sequences in the source language corpus is then compared with the first sub-sequence corresponding to the target word in the first sub-word sequence. The second sub-sequence with the largest difference is taken as the target sub-sequence corresponding to the target word. Then, the first sub-sequence corresponding to the target word in the first sub-word sequence corresponding to the source language corpus is replaced with the target sub-sequence to obtain a sample sub-word sequence corresponding to the source language corpus. The machine translation model is then trained based on the sample sub-word sequences corresponding to the source language corpus and the sample sub-word sequences corresponding to the target language corpus. In this method, the first subsequence corresponding to the target word in the first subsequence of the source language corpus is replaced with the target subsequence with the largest difference from the first subsequence. The resulting sample subsequence has greater adversarial power. Training the machine translation model based on this sample subsequence can improve the translation accuracy of the machine translation model. In particular, for noisy texts to be translated (such as spelling errors), the machine translation model has higher translation accuracy, that is, the machine translation model has strong robustness. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating a machine translation model training method in one embodiment;

[0054] Figure 2 This is a flowchart illustrating the process of determining the target subsequence in one embodiment;

[0055] Figure 3 This is a flowchart illustrating the process of determining the degree of difference in one embodiment;

[0056] Figure 4 This is a flowchart illustrating a text translation method in one embodiment;

[0057] Figure 5 This is a structural block diagram of a machine translation model training device in one embodiment;

[0058] Figure 6 This is a structural block diagram of a text translation device in one embodiment;

[0059] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] First, before introducing the technical solutions of the embodiments of this application, we will first introduce the technical background or technical evolution on which the embodiments of this application are based. With the development of economic globalization and artificial intelligence technology, machine translation technology has been widely used in many fields, including fintech. Among them, neural machine translation (NMT) is a technology that uses neural networks to maximize machine translation performance. It uses bilingual or multilingual training corpora to train neural networks to obtain machine translation models, which are used to translate input source language text and output target language text. When training the machine translation model, it is necessary to perform word segmentation on the original training corpus. The purpose is to divide the original training corpus into a sequence of sub-words composed of multiple sub-words. Sub-words are the basic units of translation. Then, the word-segmented training corpus is used to train the machine translation model. In related technologies, word segmentation methods employ Byte Pair Encoding (BPE). However, documents requiring translation often contain noise (such as spelling errors). Machine translation models trained on corpora processed using BPE exhibit low accuracy and poor robustness when dealing with noisy text. Against this backdrop, the applicant, through long-term research and experimental verification, proposes a machine translation model training method that significantly improves translation accuracy, particularly for noisy text, demonstrating strong robustness. Furthermore, it should be noted that the applicant has invested substantial creative effort in discovering the technical problems and implementing the technical solutions described in the following embodiments.

[0062] In one embodiment, such as Figure 1 As shown, a machine translation model training method is provided. This embodiment illustrates the method by applying it to a terminal. It is understood that this method can also be applied to a server, and to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. The server can be a standalone server or a server cluster consisting of multiple servers. In this embodiment, the method includes the following steps:

[0063] Step 101: Obtain the training sample set.

[0064] The training sample set includes source language corpus and target language corpus corresponding to the source language corpus.

[0065] In implementation, a training corpus can be pre-built according to the needs of the translation scenario. For example, if the translation scenario involves translating English text in the fintech field into Chinese text, relevant English and corresponding Chinese corpora in the fintech field can be selected as training corpora, and a training corpus can be established. The terminal can obtain multiple training corpora from the training corpus to obtain a training sample set. Each sample in the training sample set includes a source language corpus and the corresponding target language corpus. Optionally, the terminal can preprocess the obtained multiple training corpora to obtain the training sample set.

[0066] Step 102: The training sample set is processed by the first preset algorithm to obtain the first sub-word sequence corresponding to the source language corpus and the sample sub-word sequence corresponding to the target language corpus.

[0067] In implementation, the terminal can use a first preset algorithm to perform a first word segmentation on the training sample set, obtaining the first sub-word sequence corresponding to each source language corpus and the sample sub-word sequence corresponding to each target language corpus in the training sample set. The first preset algorithm can be an existing word segmentation algorithm, such as Byte Pair Encoding (BPE). In one example, the terminal can call the BPE algorithm to process the training sample set. Specifically, it first constructs a source language vocabulary and a target language vocabulary based on the training sample set. The vocabulary contains sub-words and their corresponding sub-word vectors (or sub-word embeddings). Then, it performs word segmentation on each source language corpus and target language corpus in the training sample set (i.e., the first word segmentation) to divide each corpus into a sub-word sequence composed of multiple sub-words. For Chinese corpora, a subword can be a Chinese character, a phrase or word composed of several Chinese characters, etc. For corpora of Latin languages ​​such as English, a subword can be a letter, a word, or a part of a word. That is, each word can be composed of one or more subwords forming a subword sequence. In the aforementioned word segmentation results, the subword sequence corresponding to each word in the source language corpus can be obtained.

[0068] Step 103: The source language corpus is processed by the second preset algorithm to obtain multiple second sub-word sequences corresponding to the source language corpus.

[0069] In implementation, the terminal can use a second preset algorithm to perform second word segmentation on the source language corpus, obtaining multiple second sub-word sequences corresponding to the source language corpus. Since each word in the source language corpus can have different sub-word combinations, different sub-word sequences can be obtained depending on the sampling method of the sub-words in the vocabulary and the matching method between the sub-words and each word in the corpus, such as forward maximum matching, backward maximum matching, and probability-based word segmentation methods (e.g., sub-word regularization, i.e., based on a unigram language model, generating multiple segmentation results for a single word). Therefore, the terminal can use a word segmentation algorithm different from the first preset algorithm to perform word segmentation on the source language corpus to obtain multiple second sub-word sequences corresponding to the source language corpus. Based on each second sub-word sequence, the sub-word sequence corresponding to each word in the source language corpus can be determined. In one example, a word regularization method can be used to calculate the probability of each word sequence corresponding to the source language corpus. This involves multiplying the probabilities of each word in the word sequence to obtain the probability of a word sequence. The probability of each word can be obtained using the Expectation-Maximum (EM) algorithm. This yields the probabilities of several word sequences. Then, using a forward dynamic programming search and backward search algorithm, L word sequences can be searched from these probabilities to serve as multiple (e.g., L) second word sequences corresponding to the source language corpus.

[0070] Step 104: Compare the difference between the first subsequence corresponding to the target word in the first subsequence of the source language corpus and the second subsequence corresponding to the target word in multiple second subsequences, and determine the second subsequence with the largest difference as the target subsequence corresponding to the target word.

[0071] The target words are the words contained in the source language corpus. For example, it could be all the words in the source language corpus or only some of them.

[0072] For ease of distinction, the sequence of words corresponding to a statement (or corpus) is called a sub-word sequence (such as the first and second sub-word sequences corresponding to the source language corpus mentioned above), and the sequence of words corresponding to each word in the statement is called a sub-sequence (such as the first and second sub-sequences corresponding to the target word mentioned above).

[0073] For example, a sequence of words in a source language corpus (a sentence, which can be represented as X) can be represented as w = (w1, ..., w...). M ), where w1 is the first word in the corpus, w M That is, the last word in the corpus, and the corpus X contains M words.

[0074] For the j-th word w in the word sequence wj Its corresponding subsequence can be represented as in, That is, the j-th word w j The first subword, and the subsequence contains K subwords in total.

[0075] The subsequence of each word in the corpus X is combined according to the order of the word sequence to obtain the sub-word sequence corresponding to the corpus X, which can be represented as: s=(s1,…,s M In this context, s1 is the subsequence s1 corresponding to the first word w1 in the word sequence w. If different word segmentation methods are used to segment the corpus, the resulting subsequences will differ, and each word in the corpus may correspond to different subsequences in different subsequence sequences. That is, the first subsequence corresponding to the target word in the first subsequence of the source language corpus may be different from the second subsequence corresponding to the target word in the second subsequence.

[0076] In implementation, the terminal can compare the difference between the first subsequence corresponding to each target word in the source language corpus and the second subsequence corresponding to that target word in multiple second subsequences. If there are L second subsequences, then each of the L second subsequences is compared with the first subsequence. For example, the subsequences can be mapped to vectors, and the similarity between the two vectors can be calculated; the smaller the similarity, the greater the difference. Then, the terminal can determine the second subsequence with the largest difference from the first subsequence as the target subsequence corresponding to the target word.

[0077] Step 105: In the first sub-word sequence corresponding to the source language corpus, replace the first sub-sequence corresponding to the target word with the target sub-sequence to obtain the sample sub-word sequence corresponding to the source language corpus.

[0078] In implementation, the terminal can replace the first subsequence corresponding to the target word in the first subsequence of the source language corpus with the target subsequence to obtain the sample subsequence of the source language corpus. For example, for the first subsequence...

[0079]

[0080] in Let M represent the first subsequence corresponding to the j-th word in the source language corpus, and M represent that the source language corpus contains M words.

[0081] If the j-th word in the source language corpus is the target word, then the first subsequence corresponding to the target word is... Replace with the target subsequence corresponding to the target word determined in step 104 (which can be represented as...). By doing so, we can obtain the sample word sequence corresponding to the source language corpus.

[0082] Step 106: Train the machine translation model based on the sample word sequences corresponding to the source language corpus and the sample word sequences corresponding to the target language corpus.

[0083] In practice, the terminal can use the sample word sequences corresponding to the source language corpus obtained in step 105 and the sample word sequences corresponding to the target language corpus obtained in step 102 to train the machine translation model. The specific training process can adopt existing model training methods, which will not be elaborated here.

[0084] In the aforementioned machine translation model training method, the training sample set is first segmented to obtain the first sub-word sequence corresponding to the source language corpus and the sample sub-word sequence corresponding to the target language corpus. Then, the source language corpus is second segmented to obtain multiple second sub-word sequences corresponding to the source language corpus. The second sub-sequence corresponding to the target word in the source language corpus is then compared with the first sub-sequence corresponding to the target word in the first sub-word sequence. The second sub-sequence with the largest difference is taken as the target sub-sequence corresponding to the target word. Then, the first sub-sequence corresponding to the target word in the first sub-word sequence corresponding to the source language corpus is replaced with the target sub-sequence to obtain the sample sub-word sequence corresponding to the source language corpus. The machine translation model is then trained based on the sample sub-word sequences corresponding to the source language corpus and the sample sub-word sequences corresponding to the target language corpus. In this method, the first subsequence corresponding to the target word in the first subsequence of the source language corpus is replaced with the target subsequence with the largest relative difference from the first subsequence. The resulting sample subsequence has greater adversarial power. Training the machine translation model based on this sample subsequence can improve the translation accuracy of the machine translation model. In particular, for the text to be translated with noise such as spelling errors, the machine translation model has higher translation accuracy, that is, the machine translation model has strong robustness.

[0085] In one embodiment, the method further includes the step of determining the target word from the words contained in the source language corpus based on a preset sampling rule.

[0086] In implementation, the terminal can select a subset of words from the source language corpus as target words based on preset sampling rules. For example, the terminal can randomly sample and select a preset proportion of words as target words.

[0087] In this embodiment, by sampling a portion of words from the source language corpus as target words, and then comparing the subsequence differences of the selected target words, using only a portion of the words as target words can balance the efficiency of determining the sample sub-word sequences corresponding to the source language corpus and the adversarial nature of the sample sub-word sequences, thereby balancing the overall efficiency of model training and translation effect.

[0088] In one embodiment, the process of determining the target word based on a preset sampling rule specifically includes the following steps: assigning a random number to each word in the source language corpus, wherein each random number conforms to a uniform distribution; and determining the word corresponding to the random number less than a preset threshold as the target word.

[0089] In this embodiment, the terminal can assign a random number to each word in the source language corpus, and each random number conforms to a uniform distribution. For example, for a source language corpus containing M words, the terminal can generate M random numbers (denoted as r) conforming to a uniform distribution U(0,1), and each random number r corresponds one-to-one with each word. Then, the terminal can compare the random number corresponding to each word with a preset threshold (denoted as R). If the random number r corresponding to the j-th word among the M words is less than the preset threshold R, then the j-th word is determined as the target word. In one example, the preset threshold R can be set to 0.2.

[0090] In this embodiment, the target word is determined by assigning a uniformly distributed random number to each word in the source language corpus, which improves the rationality of the target word selection. Then, the first subsequence corresponding to the target word is replaced, and the resulting sample sub-word sequence is used to train the machine translation model, which can ensure the translation accuracy of the machine translation model and enhance the robustness of the model.

[0091] In one embodiment, such as Figure 2 As shown, the process of determining the target subsequence in step 104 specifically includes the following steps:

[0092] Step 201: Determine the first subsequence corresponding to the target word in the first sub-word sequence in the source language corpus, obtain the first sub-word vector corresponding to each sub-word in the first sub-sequence, and determine the first word vector corresponding to the target word based on the first sub-word vector.

[0093] In implementation, the terminal can determine the first subsequence corresponding to the target word within the first subsequence of the source language corpus. For example, the terminal can determine the first subsequence corresponding to the target word within the first subsequence of the source language corpus based on the target word's ranking number in the source language corpus (e.g., number j, meaning the target word is the j-th word). In the sequence, find the first subsequence corresponding to the sorted index. in, The first subsequence corresponding to the j-th word (the target word) The first subword in the sequence. Then, the terminal can obtain that subsequence (i.e., the first subsequence). The terminal can construct a vocabulary list during the first word segmentation process in step 102, and then search for the first word vector corresponding to each word in that vocabulary list. Then, the terminal can determine the first word vector corresponding to the target word based on each first word vector. For example, the terminal can concatenate each first word vector or calculate its mean to obtain the first word vector.

[0094] In one example, the mean of each first sub-word vector can be calculated, that is, the mean of the values ​​of the same digit in the first sub-word vector is calculated to obtain the first word vector corresponding to the target word, which can be expressed by the formula:

[0095]

[0096] in, This represents the first word vector corresponding to the j-th word (which is the target word) in the source language corpus; This represents the first subsequence corresponding to the target word. In, the first sub-word The corresponding first sub-word vector; f represents the function that calculates the mean based on the values ​​of the same digit in each first sub-word vector.

[0097] Step 202: Determine the second subsequence corresponding to the target word in multiple second sub-word sequences. For each second subsequence, obtain the second sub-word vector corresponding to each word in the second subsequence, and determine the second word vector corresponding to the target word based on the second sub-word vector.

[0098] In implementation, the terminal can determine the second subsequence corresponding to the target word from multiple second subsequences corresponding to the source language corpus. Similar to the method for determining the first word vector corresponding to the target word, for each second subsequence, the terminal can obtain the second sub-word vector corresponding to each word in the second subsequence, and determine the second word vector e(s) corresponding to the target word based on the second sub-word vector. j ).

[0099] Step 203: Determine the difference between each second word vector and the first word vector, and determine the second subsequence corresponding to the second word vector with the largest difference as the target subsequence corresponding to the target word.

[0100] In implementation, the terminal can compare the difference between each second word vector obtained in step 202 and the first word vector obtained in step 201. For example, it can calculate the similarity between the two vectors; the smaller the similarity, the greater the difference. Then, the terminal can determine the second subsequence corresponding to the second word vector with the largest difference as the target subsequence corresponding to the target word.

[0101] In this embodiment, by obtaining the first subsequence corresponding to the target word and the sub-word vectors of each word in multiple second subsequences, the first word vector and multiple second word vectors corresponding to the target word are obtained. Then, the second subsequence corresponding to the second word vector with the largest difference from the first word vector is taken as the target subsequence of the target word. The sample sub-word sequence obtained in this way is used to train the machine translation model, which can improve the translation accuracy and robustness of the machine translation model for noisy text.

[0102] In one embodiment, such as Figure 3 As shown, the process of determining the degree of difference in step 203 specifically includes the following steps:

[0103] Step 301: Calculate the gradient vector corresponding to each word in the first subsequence according to the loss function of the machine translation model, and determine the gradient vector corresponding to the target word based on the gradient vector corresponding to each word.

[0104] In practice, the loss function of the machine translation model is a pre-established loss function. In one example, the formula for the loss function L(x,y;θ) can be expressed as:

[0105]

[0106] Where X is the word sequence corresponding to a source language corpus, Y is the word sequence of the target language corpus corresponding to the source language corpus, a source language corpus and the corresponding target language corpus constitute a sample s, and D is the total number of samples; This represents the subsequence corresponding to word x in the subsequence X of the source language corpus. (as the first subword); This represents the subsequence corresponding to word y in the subsequence Y of the target language corpus. (where θ is the first subword); θ is the parameter set of the loss function; E represents the expectation; P represents the probability; P seg (·) represents a unary language model.

[0107] In another example, the loss function L(θ) can be expressed as:

[0108]

[0109] The terminal can calculate the gradient vector corresponding to each word in the first subsequence of the target word by taking the partial derivative of the loss function of the machine translation model. For example, for the j-th word (the target word) in the source language corpus, its corresponding first subsequence can be represented as follows: It contains K sub-words. For each sub-word, the partial derivative of the loss function L(x,y;θ) is taken to obtain the gradient vector corresponding to that sub-word, which can be expressed by the formula:

[0110]

[0111] in, Indicates the Kth subword The corresponding gradient vector.

[0112] Then, the terminal can calculate the gradient vector corresponding to the target word based on the gradient vector corresponding to each sub-word. For example, the gradient vector corresponding to each sub-word can be averaged to obtain the gradient vector corresponding to the target word, expressed by the formula:

[0113]

[0114] in, Let represent the gradient vector corresponding to the j-th word (which is the target word) in the source language corpus. This represents the first subsequence corresponding to the target word. In the middle, the Kth sub-word The corresponding gradient vector; f represents the function that calculates the mean based on the values ​​of the same digit in the gradient vector corresponding to each word.

[0115] Step 302: Calculate the difference between each second word vector and the first word vector based on the first word vector, each second word vector, and the gradient vector corresponding to the target word.

[0116] In implementation, the terminal can base its actions on the first word vector. Each second word vector e(s) j ), the gradient vector corresponding to the target word Calculate the vector e(s) for each second word. j ) and the first word vector The degree of difference. Specifically, the gradient vector corresponding to the target word can be used. The transpose (column vector) of the row vector, and the second word vector e(s) j ) and the first word vector The product of the differences is the degree of difference. The larger the product, the greater the degree of difference. Then, the second subsequence corresponding to the second word vector with the largest degree of difference is determined as the target subsequence corresponding to the target word, as expressed by the following formula:

[0117]

[0118] in, e(s) represents the target subsequence corresponding to the target word. j ) represents the second word vector corresponding to the target word. Ω(w) represents the first word vector corresponding to the target word. j () represents the set of multiple second subsequences corresponding to the target word; This represents the gradient vector corresponding to the target word. The transpose of .

[0119] In this embodiment, the gradient vector corresponding to the target word is calculated based on the loss function of the machine translation model. Then, based on the first word vector, each second word vector, and the gradient vector corresponding to the target word, the difference between each second word vector and the first word vector is calculated. This allows for the reasonable selection of the target subsequence corresponding to the target word. The training samples obtained by replacing the target subsequence are then used to train the machine translation model, which can improve the accuracy and robustness of the machine translation model in translating noisy text.

[0120] In one embodiment, the process of obtaining the training sample set in step 101 specifically includes the following steps: obtaining an initial training sample set and adding perturbation information to the initial training sample set to obtain the training sample set.

[0121] In practice, the terminal can add perturbation information to the acquired initial training sample set to obtain a training sample set with a certain degree of adversarial nature.

[0122] For example, the sentence-piece tool can be used to add perturbation information to the initial training sample set. Specifically, adversarial noise can be added to the initial training sample set (i.e., each initial source language corpus and target language corpus) to obtain the perturbated training sample set, which can be expressed by the following formula:

[0123]

[0124] Where E is the preset embedding matrix, x i For the initial training sample set, sample x i , For the added adversarial noise, e i ′ is a sample x in the training sample set after perturbation. i The embedding vector.

[0125] Counter noise It can be calculated using the following formula:

[0126]

[0127] Where ε represents the degree of disturbance, which is a preset value and can be set to ε = 0.01; g i This represents the gradient of the loss function l(X,Y;θ) calculated with respect to a word vector in the training sample set.

[0128] In this embodiment, by adding perturbation information to the initial training sample set (initial corpus), and then using the training sample set with added perturbation information for subsequent word segmentation processing, the resulting sample word sequences are more adversarial and can be used for model training, resulting in a more robust machine translation model.

[0129] In one embodiment, such as Figure 4 As shown, a text translation method is also provided, which includes the following steps:

[0130] Step 401: Obtain the text to be translated in the source language.

[0131] In implementation, the terminal can acquire the text to be translated in the source language. For example, a user can input the text to be translated into the terminal, or the terminal can perform image recognition to obtain the text to be translated contained in an image. The text to be translated is text expressed in the source language, which can be a word, phrase, sentence, or paragraph composed of multiple sentences, and may contain misspelled words. The source language is the language expressed in the source language corpus used in the training sample set when training the machine translation model.

[0132] Step 402: Input the text to be translated into a pre-trained machine translation model to obtain the translated text expressed in the target language.

[0133] In implementation, after acquiring the text to be translated, the terminal can input it into a pre-trained machine translation model. The machine translation model then translates the input text and outputs the translated text in the target language. The machine translation model is trained using the methods described above. The target language is the language expressed in the target language corpus used in the training sample set when training the machine translation model.

[0134] In this embodiment, the machine translation model trained by the above-mentioned machine translation model training method is used to translate the text to be translated, and a translation result text with high translation accuracy can be obtained. Since the machine translation model has been trained adversarially, it is robust. Even for the text to be translated containing noise (such as spelling errors), the machine translation model can still produce a translation result text with high translation accuracy.

[0135] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0136] Based on the same inventive concept, this application also provides a machine translation model training apparatus for implementing the machine translation model training method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more machine translation model training apparatus embodiments provided below can be found in the limitations of the machine translation model training method described above, and will not be repeated here.

[0137] In one embodiment, such as Figure 5 As shown, a machine translation model training device 500 is provided, including: an acquisition module 501, a first processing module 502, a second processing module 503, a first determination module 504, a replacement module 505, and a training module 506, wherein:

[0138] The acquisition module 501 is used to acquire the training sample set; the training sample set includes the source language corpus and the target language corpus corresponding to the source language corpus.

[0139] The first processing module 502 is used to perform first word segmentation on the training sample set using a first preset algorithm to obtain the first sub-word sequence corresponding to the source language corpus and the sample sub-word sequence corresponding to the target language corpus.

[0140] The second processing module 503 is used to perform second word segmentation on the source language corpus using a second preset algorithm to obtain multiple second sub-word sequences corresponding to the source language corpus.

[0141] The first determining module 504 is used to compare the difference between the first subsequence corresponding to the target word in the first subsequence of the source language corpus and the second subsequence corresponding to the target word in multiple second subsequences, and determine the second subsequence with the largest difference as the target subsequence corresponding to the target word.

[0142] The replacement module 505 is used to replace the first subsequence corresponding to the target word in the first subsequence corresponding to the source language corpus with the target subsequence, so as to obtain the sample subsequence corresponding to the source language corpus.

[0143] Training module 506 is used to train the machine translation model based on the sample word sequences corresponding to the source language corpus and the sample word sequences corresponding to the target language corpus.

[0144] In one embodiment, the device further includes a second determining module, configured to:

[0145] Based on preset sampling rules, target words are determined from the words contained in the source language corpus.

[0146] In one embodiment, the second determining module is specifically used for:

[0147] A random number is assigned to each word in the source language corpus, and the random numbers conform to a uniform distribution; the words corresponding to random numbers less than a preset threshold are identified as target words.

[0148] In one embodiment, the first determining module 504 is specifically used for:

[0149] Determine the first subsequence corresponding to the target word in the first subsequence of the source language corpus, obtain the first sub-word vector corresponding to each word in the first subsequence, and determine the first word vector corresponding to the target word based on the first sub-word vector; determine the second subsequence corresponding to the target word in multiple second subsequences, obtain the second sub-word vector corresponding to each word in the second subsequence for each second subsequence, and determine the second word vector corresponding to the target word based on the second sub-word vector; determine the difference between each second word vector and the first word vector, and determine the second subsequence corresponding to the second word vector with the largest difference as the target subsequence corresponding to the target word.

[0150] In one embodiment, the first determining module 504 is specifically used for:

[0151] Based on the loss function of the machine translation model, calculate the gradient vector corresponding to each word in the first subsequence, and determine the gradient vector corresponding to the target word based on the gradient vector corresponding to each word; calculate the difference between each second word vector and the first word vector based on the first word vector, each second word vector, and the gradient vector corresponding to the target word.

[0152] In one embodiment, the acquisition module 501 is specifically used for:

[0153] Obtain the initial training sample set and add perturbation information to the initial training sample set to obtain the training sample set.

[0154] Based on the same inventive concept, this application also provides a text translation apparatus for implementing the text translation method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more text translation apparatus embodiments provided below can be found in the limitations of the text translation method described above, and will not be repeated here.

[0155] In one embodiment, such as Figure 6 As shown, a text translation device 600 is provided, including: an acquisition module 601 and a translation module 602, wherein:

[0156] The acquisition module 601 is used to acquire the text to be translated in the source language.

[0157] The translation module 602 is used to input the text to be translated into a pre-trained machine translation model for translation, and to obtain the translated text expressed in the target language; wherein, the machine translation model is trained by the machine translation model training method mentioned above.

[0158] The modules in the aforementioned machine translation model training device and text translation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0159] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a machine translation model training method or a text translation method. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0160] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0161] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0162] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0163] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0164] The machine translation model training method, apparatus, computer equipment, storage medium, and computer program product provided in this application relate to the field of artificial intelligence technology and can be used in the field of fintech or other related fields. This application does not limit the application fields of the machine translation model training method, apparatus, computer equipment, storage medium, and computer program product.

[0165] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0166] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0167] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0168] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for training a machine translation model, characterized in that, The method comprises: obtaining a training sample set; the training sample set comprises a source language corpus and a target language corpus corresponding to the source language corpus; performing first word segmentation processing on the training sample set by using a first preset algorithm to obtain a first subword sequence corresponding to the source language corpus and a sample subword sequence corresponding to the target language corpus; performing second word segmentation processing on the source language corpus by using a second preset algorithm to obtain a plurality of second subword sequences corresponding to the source language corpus; comparing the first subsequence corresponding to the target word in the first subword sequence with the second subsequence corresponding to the target word in the plurality of second subword sequences in terms of difference degree, and determining the second subsequence with the largest difference degree as the target subsequence corresponding to the target word; replacing the first subsequence corresponding to the target word with the target subsequence in the first subword sequence corresponding to the source language corpus to obtain a sample subword sequence corresponding to the source language corpus; training a machine translation model based on the sample subword sequence corresponding to the source language corpus and the sample subword sequence corresponding to the target language corpus.

2. The method of claim 1, wherein, The method further comprises: determining a target word from the words included in the source language corpus based on a preset sampling rule.

3. The method of claim 2, wherein, The method of determining a target word from the words included in the source language corpus based on a preset sampling rule comprises: assigning a random number to each word included in the source language corpus, wherein each random number conforms to a uniform distribution; determining the word corresponding to the random number less than a preset threshold as the target word.

4. The method of claim 1, wherein, The method of comparing the first subsequence corresponding to the target word in the first subword sequence with the second subsequence corresponding to the target word in the plurality of second subword sequences in terms of difference degree, and determining the second subsequence with the largest difference degree as the target subsequence corresponding to the target word comprises: determining the first subsequence corresponding to the target word in the first subword sequence, obtaining a first subword vector corresponding to each subword in the first subsequence, and determining a first word vector corresponding to the target word based on the first subword vector; determining the second subsequence corresponding to the target word in the plurality of second subword sequences, obtaining a second subword vector corresponding to each subword in the second subsequence for each second subsequence, and determining a second word vector corresponding to the target word based on the second subword vector; determining the difference degree of each second word vector and the first word vector, and determining the second subsequence corresponding to the second word vector with the largest difference degree as the target subsequence corresponding to the target word.

5. The method of claim 4, wherein, The method of determining the difference degree of each second word vector and the first word vector comprises: calculating a gradient vector corresponding to each subword in the first subsequence based on a loss function of the machine translation model, and determining a gradient vector corresponding to the target word based on the gradient vector corresponding to each subword; According to the first word vector, each second word vector, and a gradient vector corresponding to the target word, a difference degree of each second word vector and the first word vector is calculated.

6. The method of claim 1, wherein, The training sample set is obtained by: An initial training sample set is obtained, and perturbation information is added to the initial training sample set to obtain the training sample set.

7. A method of translating text, characterized by, The method comprises: Obtaining a text to be translated expressed in a source language; The text to be translated is input into a pre-trained machine translation model for translation to obtain a translation result text expressed in a target language; wherein the machine translation model is trained by the machine translation model training method of any one of claims 1 to 6.

8. A machine translation model training apparatus, characterized by comprising: The device comprises: An acquisition module is configured to acquire a training sample set, wherein the training sample set comprises source language corpus and target language corpus corresponding to the source language corpus; A first processing module is configured to perform first word segmentation processing on the training sample set by using a first preset algorithm to obtain a first subword sequence corresponding to the source language corpus and a sample subword sequence corresponding to the target language corpus; A second processing module is configured to perform second word segmentation processing on the source language corpus by using a second preset algorithm to obtain a plurality of second subword sequences corresponding to the source language corpus; A first determining module is configured to compare a first subsequence corresponding to a target word in the first subword sequence in the source language corpus with a second subsequence corresponding to the target word in the plurality of second subword sequences in terms of difference degree, and determine a second subsequence with the largest difference degree as a target subsequence corresponding to the target word; A replacement module is configured to replace the first subsequence corresponding to the target word in the first subword sequence corresponding to the source language corpus with the target subsequence to obtain a sample subword sequence corresponding to the source language corpus; A training module is configured to train a machine translation model based on the sample subword sequence corresponding to the source language corpus and the sample subword sequence corresponding to the target language corpus.

9. A text translation apparatus characterized by comprising: The device comprises: An acquisition module is configured to acquire a text to be translated expressed in a source language; A translation module is configured to input the text to be translated into a pre-trained machine translation model for translation to obtain a translation result text expressed in a target language; wherein the machine translation model is trained by the machine translation model training method of any one of claims 1 to 6. 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method of any one of claims 1 to 6 or claim 7.

11. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6 or claim 7.

12. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6 or claim 7. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6 or claim 7.

Images