Translation model processing method, device and computer-readable storage medium

Abstract

The present disclosure relates to a translation model processing method, device, and computer-readable storage medium, and relates to the field of computer technology. The method of the present disclosure includes: obtaining multiple groups of training sentences, wherein each group of training sentences includes: original sentences and target translation sentences; for each group of training sentences, inputting the original sentences into the encoder of the translation model to obtain the feature vector of the original sentences, and inputting the feature vector of the original sentences into the decoder of the translation model; for each word generated by the decoder except the end of the sentence, determining the semantic similarity between the generated sentence composed of the word and the words before the word and the target translation sentence; selecting the word or the word at the same position as the word in the target translation sentence based on the semantic similarity to generate the word at the next position; and training the translation model based on the differences between the translation sentences composed of the words at each position generated by the decoder and the target translation sentence.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for processing a translation model, and a computer-readable storage medium. Background Art

[0002] Autoregressive models (ARMs) are widely used in natural language generation (NLG) tasks such as machine translation, dialogue response generation, image caption generation, and video description generation. They use an encoder-decoder framework to predict the next word conditioned on a generated short sentence.

[0003] For machine translation scenarios, during ARM training, real translations are used as generated short sentences, forcing the model to directly learn the distribution of real translations. However, during testing or use, the generated phrases come from the ARM decoder itself, which is different from the input distribution during training. Summary of the Invention

[0004] The inventors discovered that the data input into the decoder part of the translation model during training and testing or use are different, resulting in the trained translation model being unable to achieve accurate translation function during testing and use.

[0005] A technical problem to be solved by the present disclosure is to improve the accuracy of translation model training.

[0006] According to some embodiments of the present disclosure, a method for processing a translation model is provided, comprising: obtaining multiple groups of training sentences, wherein each group of training sentences comprises: an original sentence and a target translation sentence; for each group of training sentences, inputting the original sentence into the encoder of the translation model to obtain a feature vector of the original sentence, and inputting the feature vector of the original sentence into the decoder of the translation model; for the word at each position except the end of the sentence generated by the decoder, determining the semantic similarity between the generated sentence composed of the word and the words before the word and the target translation sentence; selecting the word or the word at the same position as the word in the target translation sentence according to the semantic similarity to generate the word at the next position; and training the translation model according to the difference between the translation sentence composed of the words at each position generated by the decoder and the target translation sentence.

[0007] In some embodiments, determining the semantic similarity between the generated sentence consisting of the word and the words preceding the word and the target translation sentence includes: taking the generated sentence consisting of the word and the words preceding the word and the target translation sentence as a sentence pair, inputting the bidirectional encoding representation BERT model based on the converter, and obtaining a feature vector of the output sentence pair; inputting the feature vector of the sentence pair into the activation function module to obtain the semantic similarity between the generated sentence and the target translation sentence.

[0008] In some embodiments, determining the semantic similarity between the generated sentence consisting of the word and the words preceding the word and the target translation sentence includes: generating a random number and comparing the random number with a reference value, wherein the reference value is within the value range of the random number; determining whether the random number is less than the reference value, and if it is less than the reference value, determining the semantic similarity between the generated sentence consisting of the word and the words preceding the word and the target translation sentence.

[0009] In some embodiments, the reference value increases as the number of training sessions increases.

[0010] In some embodiments, selecting the word or the word at the same position as the word in the target translation sentence based on the semantic similarity to generate the word at the next position includes: when the semantic similarity is higher than a threshold, generating the word at the next position based on the word; when the semantic similarity is lower than the threshold, generating the word at the next position based on the word at the same position as the word in the target translation sentence.

[0011] In some embodiments, the threshold increases as the number of training times increases.

[0012] In some embodiments, the decoder includes multiple decoding modules, and selecting the word or the word at the same position as the word in the target translation sentence according to semantic similarity to generate the word at the next position includes: selecting the word or the word at the same position as the word in the target translation sentence according to semantic similarity as the input word; inputting the state output by the decoding module corresponding to the word and the word vector of the input word into the decoding module corresponding to the word at the next position to obtain the output word at the next position.

[0013] In some embodiments, the method further includes: inputting the sentence to be translated into a trained translation model to obtain a corresponding translated sentence.

[0014] In some embodiments, inputting a sentence to be translated into a trained translation model to obtain a corresponding translation sentence includes: inputting the sentence to be translated into an encoder of the translation model to obtain a feature vector of the sentence to be translated, and inputting the feature vector of the sentence to be translated into a decoder of the translation model; selecting a preset number of words as multiple alternative words for each position based on the probability value of each word at each position output by the decoder; generating multiple alternative words for the next position of each position based on the multiple alternative words at each position until the end of the sentence is reached, wherein the number of alternative words at each position is the same; using the decoder to generate alternative words for each position to form multiple alternative translation sentences, wherein the generation of words in each alternative translation sentence is correlated; selecting an alternative translation sentence as the translation sentence of the sentence to be translated based on the probability value of each alternative translation sentence.

[0015] In some embodiments, based on the probability values of each word at each position output by the decoder, selecting a preset number of words as multiple alternative words for each position includes: for each word at each position output by the decoder, determining the selection probability value of the word based on the probability value of the word and the probability values of previous words associated with the generation of the word; based on the selection probability values of each word at each position output by the decoder, selecting a preset number of words as multiple alternative words for each position generated by the decoder.

[0016] In some embodiments, selecting an alternative translation sentence as the translation sentence of the sentence to be translated based on the probability values of the alternative translation sentences includes: determining the probability values of the alternative translation sentences based on the probability values of the respective words in each alternative translation sentence generated by the decoder; and selecting the alternative translation sentence with the largest probability value as the translation sentence of the sentence to be translated.

[0017] According to other embodiments of the present disclosure, a processing device for a translation model is provided, comprising an acquisition module for acquiring multiple groups of training sentences, wherein each group of training sentences comprises: an original sentence and target translation sentence input module for inputting the original sentence into the encoder of the translation model for each group of training sentences, obtaining a feature vector of the original sentence, and inputting the feature vector of the original sentence into the decoder of the translation model; a determination module for determining, for a word at each position except the end of a sentence generated by the decoder, the semantic similarity between a generated sentence consisting of the word and the words preceding the word and the target translation sentence; a generation module for selecting the word or the word at the same position as the word in the target translation sentence based on the semantic similarity to generate a word at the next position; and a training module for training the translation model based on the differences between the translation sentences consisting of the words at each position generated by the decoder and the target translation sentence.

[0018] In some embodiments, the device further includes: a translation module, configured to input the sentence to be translated into a trained translation model to obtain a corresponding translated sentence.

[0019] According to some further embodiments of the present disclosure, a translation model processing device is provided, comprising: a processor; and a memory coupled to the processor, for storing instructions, which, when executed by the processor, causes the processor to execute the translation model processing method of any of the aforementioned embodiments.

[0020] According to further embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, wherein when the program is executed by a processor, the steps of the method for processing a translation model of any of the aforementioned embodiments are implemented.

[0021] In this disclosure, when training a translation model, for each word at each position generated by the decoder, the semantic similarity between the generated sentence containing the word and the target translation sentence is used to determine whether to introduce the word into the training process to generate the next word at the next position. This allows the content generated by the decoder to be gradually integrated into the training process, reducing the differences between the training process and the testing or use process. Furthermore, by selecting words generated by the decoder based on the semantic similarity between the generated sentence and the target translation sentence, the selected words can be more realistic, reasonable, and accurate, thereby improving the efficiency and accuracy of translation model training.

[0022] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 A flowchart illustrating a method for processing a translation model according to some embodiments of the present disclosure is provided.

[0025] Figure 2 A schematic diagram illustrating the structure of a translation model according to some embodiments of the present disclosure is shown.

[0026] Figure 3 A flowchart illustrating a method for processing a translation model according to other embodiments of the present disclosure is shown.

[0027] Figure 4 A schematic diagram illustrating the structure of a translation model processing device according to some embodiments of the present disclosure.

[0028] Figure 5 A schematic structural diagram illustrating a translation model processing device according to some other embodiments of the present disclosure.

[0029] Figure 6 A schematic structural diagram of a translation model processing device according to some further embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0031] This disclosure provides a translation model processing method, which is combined with Figures 1 to 3 Provide a description.

[0032] Figure 1 Flowchart of some embodiments of the processing method of the translation model disclosed in the present invention. Figure 1 As shown, the method of this embodiment includes steps S102 to S110.

[0033] In step S102, multiple groups of training sentences are obtained.

[0034] Each set of training sentences includes: an original sentence and a target translation sentence. For example, the original sentence is a Chinese sentence, and the target translation sentence is an English sentence with the same meaning.

[0035] In step S104 , for each set of training sentences, the original sentences are input into the encoder of the translation model to obtain feature vectors of the original sentences, and the feature vectors of the original sentences are input into the decoder of the translation model.

[0036] The translation model includes an encoder and a decoder. The translation model is, for example, a Seq2Seq (sequence-to-sequence) model, a Transformer model, etc., but is not limited to the examples given. The encoder may include multiple encoding modules. For example, the various modules of the LSTM (Long Short-Term Memory) of the encoder part of the Seq2Seq model serve as encoding modules, or the multiple encoders of the Transformer model serve as encoding modules respectively. The decoder may include multiple decoding modules, for example, the various modules of the LSTM of the decoder part of the Seq2Seq model serve as decoding modules, or the multiple decoders of the Transformer model serve as decoding modules respectively. For each group of training sentences, the original sentence is input into the encoder to obtain a feature vector which is then input into the decoder part.

[0037] In step S106, for each word generated by the decoder except the end of the sentence, the semantic similarity between the generated sentence consisting of the word and the words preceding the word and the target translation sentence is determined.

[0038] The decoder can generate multiple words and the probability of each word for each position except the end of the sentence, and select a word with the highest probability as the generated word for that position.

[0039] For example, the original statement is represented as X=(x1,...,x N}, the generated translation sentence is represented as Y = {y1, ..., y T}, the target translation sentence is represented as Y′={y′1,...,y′ M}.like Figure 2 As shown, each decoding module in the decoder generates words y1, ..., y1 at different positions in the translation sentence. T The word at the current position generated by the decoder is represented as y t , then the generated sentence composed of the word and the words before it is y1,...,y t The sentence composed of y1, ..., y t The semantic similarity between the composed sentence and Y′.

[0040] In some embodiments, the generated sentence consisting of the word and the words before the word and the target translation sentence are taken as a sentence pair and input into the BERT (Bidirectional Encoder Representation from Transformers) model to obtain the feature vector of the output sentence pair; the feature vector of the sentence pair is input into the activation function module to obtain the semantic similarity between the generated sentence and the target translation sentence. Before inputting the sentence pair into the BERT model, a special identifier can be added to the header and a separator can be added between the two sentences. The vector corresponding to the special identifier in the vector output by the BERT model is taken as the feature vector of the sentence pair, and the feature vector of the sentence pair is input into the sigmoid module to obtain the semantic similarity between the generated sentence and the target translation sentence. The BERT model can be pre-trained.

[0041] In step S108, the word or the word at the same position as the word in the target translation sentence is selected based on the semantic similarity to generate the word at the next position. If the end of the sentence is reached, the steps of determining the semantic similarity and generating the word at the next position are not required.

[0042] In some embodiments, when the semantic similarity is higher than a threshold, a word at the next position is generated based on the word; when the semantic similarity is lower than the threshold, a word at the next position is generated based on the word at the same position as the word in the target translation sentence.

[0043] In some embodiments, the word or the word at the same position as the word in the target translation sentence is selected as the input word based on semantic similarity; the state output by the decoding module corresponding to the word and the word vector of the input word are input into the decoding module corresponding to the word at the next position to obtain the output word at the next position. Figure 2 As shown in the figure, the selection module is used to select from the word at the current position and the word at the same position in the target translation sentence. If the semantic similarity is higher than the threshold, y t Enter y t+1 The corresponding decoding module, otherwise, y′ t Enter y t+1 The corresponding decoding module generates y t+1 .

[0044] In some embodiments, if the target translation sentence includes multiple sentences, the semantic similarity between the generated sentence and each target translation sentence is determined, and the highest value is selected from the multiple semantic similarities as the semantic similarity corresponding to the word, and the target translation sentence corresponding to the highest value is used as the reference sentence; when the semantic similarity is higher than a threshold, the word at the next position is generated based on the word; when the semantic similarity is lower than the threshold, the word at the next position is generated based on the word at the same position as the word in the reference sentence.

[0045] In some embodiments, the threshold increases with the number of training cycles. Since the similarity between the sentences generated by the model and the target translation is relatively low at the beginning of training, setting the threshold lower allows for the inclusion of words generated by the decoder in the training process. As the number of training cycles increases, the model becomes more accurate, and the similarity between the generated sentences and the target translation increases. As the threshold increases, more accurate and reasonable words generated by the decoder can be selected for inclusion in the training process.

[0046] In some embodiments, the threshold value may be expressed using the following formula:

[0047]

[0048] In formula (1), k and γ are hyperparameters, k ≥ 1, which determine the convergence rate, and n represents the number of training times. Formula (1) can set the threshold to a specific value greater than 0 at the beginning of training to ensure that the words introduced into the training process are of high quality.

[0049] In step S110 , the translation model is trained based on the differences between the translation sentence composed of words at each position generated by the decoder and the target translation sentence.

[0050] After the decoder generates a complete translation sentence, the cross entropy loss function can be calculated based on the generated translation sentence and the target translation sentence, and the translation model can be trained based on the cross entropy loss function.

[0051] In the above-mentioned embodiment, when carrying out the training of the translation model, for the words in each position generated by the decoder, according to the semantic similarity between the generated sentence containing the words and the target translation sentence, it is determined whether the words are introduced into the training process for generating the words in the next position. In this way, the content generated by the decoder can be gradually integrated into the training process to reduce the difference between the training process and the test or use process, and the words generated by the decoder are selected according to the semantic similarity between the generated sentence and the target translation sentence, which can make the selected words more real, reasonable and accurate, thereby improving the efficiency and accuracy of the translation model training. In addition, relative to the similarity between the words in each position and the words in the target translation sentence, the words selected by the method of the above-mentioned embodiment are more reasonable and accurate.

[0052] Since the quality of the words generated at the beginning of the training process is poor, in order to reduce the introduction of poor-quality words into the training process and reduce the training convergence speed, the present disclosure also improves the training process. Figure 3 Provide a description.

[0053] Figure 3 Flowcharts of other embodiments of the processing method of the translation model disclosed herein. Figure 3 As shown, step S106 includes: executing steps S302 to S306 for each word generated by the decoder except the end of the sentence.

[0054] In step S302, a random number m is generated.

[0055] m can be a random number between 0 and 1.

[0056] In step S304 , the random number m is compared with the reference value u. If the random number m is smaller than the reference value u, step S306 is executed; otherwise, step S308 is executed.

[0057] The reference value u increases with the increase of the number of training times. The minimum value of the reference value u can be 0 and the maximum value can be 1. When u=0, the words of the target translation sentence are directly input into the decoder to train the model. When u=1, the input of the model training depends entirely on the words generated by the decoder, and the training is done like the test or use stage. If u is set too low (close to 0), the input of the decoder comes almost entirely from the target translation sentence, and it cannot process unknown words in the test or use stage. If u is set too high (close to 1) at the beginning of training, since the model has not been well trained, the quality of the generated words is poor, and as the input of the decoder, it may lead to slow convergence. The reference value u can be determined, for example, using the following formula:

[0058]

[0059] In formula (2), k ≥ 1 is a hyperparameter that determines the convergence speed, and n represents the number of training times.

[0060] In step S306, the semantic similarity between the generated sentence composed of the word and the words preceding the word and the target translation sentence is determined. Then, step S108 can be executed.

[0061] Step S308 is used to replace S108.

[0062] In step S308, a word at the next position is generated based on the word at the same position as the word in the target translation sentence.

[0063] Both the threshold and the reference value increase with the number of training sessions, and both show an accelerated rate of growth after reaching a certain number of training sessions. However, after reaching a certain number of training sessions, the threshold value increases relatively steadily with the number of training sessions, while the reference value increases more rapidly with the number of training sessions. Furthermore, the lowest threshold value is higher than the lowest reference value. This is because the maximum values of the threshold and reference value are close to 1, but the threshold range is set to a specific value above 0 at the beginning of training to reduce the probability of introducing poorly generated words into the training process. The reference value can be set to 0 initially and remains close to 0 when the number of training sessions is low, thus reducing the probability of introducing poorly generated words into the training process.

[0064] The reference value can also be set to decrease as the number of training times increases. In this case, it is necessary to determine whether the random number is greater than the reference value. If so, step S306 is executed; otherwise, step S308 is executed. The trend of the reference value is opposite to that of the above embodiment and will not be repeated here.

[0065] The method of the above embodiment, by setting random numbers and reference values, can reduce the probability of introducing inaccurate, unreasonable, and low-quality words generated by the decoder into the training process in the early stage of training, avoid reducing the speed of training convergence, and thus improve training efficiency and accuracy.

[0066] The following describes the process of testing or using the translation model after training is completed.

[0067] In some embodiments, after the translation model is trained, the sentence to be translated is input into the trained translation model to obtain a corresponding translated sentence.

[0068] In order to further expand the selection space of generated words and improve the richness and accuracy of the content of the generated translation sentences, in some embodiments, the sentence to be translated is input into the encoder of the translation model to obtain a feature vector of the sentence to be translated, and the feature vector of the sentence to be translated is input into the decoder of the translation model; based on the probability value of each word at each position output by the decoder, a preset number of words are selected as multiple alternative words for each position; based on the multiple alternative words at each position, multiple alternative words for the next position of each position are generated until the end of the sentence is reached, wherein the number of alternative words at each position is the same; the decoder is used to generate alternative words for each position to form multiple alternative translation sentences; based on the probability value of each alternative translation sentence, an alternative translation sentence is selected as the translation sentence of the sentence to be translated.

[0069] The number of candidate words for each position is the same; for example, three candidate words are generated for each position. The words in each candidate translation sentence are generated in a correlated manner. For each word in a candidate translation sentence, except for the beginning, the word is generated based on the word in the previous position, indicating that the word's generation is correlated with the word in the previous position.

[0070] In some embodiments, for each word at each position output by the decoder, the selection probability value of the word is determined based on the probability value of the word and the probability values of the previous words associated with the generation of the word; based on the selection probability value of each word at each position output by the decoder, a preset number of words are selected as multiple alternative words for each position generated by the decoder.

[0071] A preset number of words can be selected as multiple candidate words for each position in descending order of the selection probability values of the words at each position. For each position, the multiple candidate words at that position are respectively input into the decoding module corresponding to the next position to obtain multiple candidate words for the next position.

[0072] For example, the words generated by the decoder in the first position include: 11 ,y 12 ,y 13 ,y 14 ,y 15 Etc., you can sort them from large to small according to the probability value, and select a preset number of words in the front order, for example, select the three words with the largest probability value as y 11 ,y 13 ,y 15 When the decoder generates the word in the second position, it converts y 11 ,y 13 ,y 15 Input the decoding module corresponding to the second position respectively, and get y11 The corresponding word y in the second position 21 ,y 22 ,y 23 ,y 13 The corresponding word y in the second position 24 ,y 25 ,y 26 ,y 15 The corresponding word y in the second position 27 ,y 28 ,y 29 Then from y 21 ,...y 29 Select alternative words from the phrase {y 11 ,y 21}, {y 11 ,y 22}, ..., {y 15 , y 29}, select the word in the second position of the three phrases with the largest probability value as the candidate word for the second position.

[0073] In some embodiments, for each candidate translation sentence generated by the decoder, the probability value of each candidate translation sentence is determined according to the probability value of each word in the translation sentence; and the candidate translation sentence with the largest probability value is selected as the translation sentence of the sentence to be translated.

[0074] The method of the above embodiment can improve the accuracy of translation sentence generation and the richness of content, thereby improving translation quality.

[0075] The present disclosure also provides a translation model processing device, which is described below in conjunction with Figure 4 Provide a description.

[0076] Figure 4 FIG. 1 is a structural diagram of some embodiments of the processing device of the translation model disclosed in the present invention. Figure 4 As shown, the device 40 of this embodiment includes: an acquisition module 410 , an input module 420 , a determination module 430 , a generation module 440 , and a training module 450 .

[0077] The acquisition module 410 is used to acquire multiple groups of training sentences, where each group of training sentences includes: an original sentence and a target translation sentence.

[0078] The input module 420 is used to input the original sentence into the encoder of the translation model for each set of training sentences, obtain the feature vector of the original sentence, and input the feature vector of the original sentence into the decoder of the translation model.

[0079] The determination module 430 is configured to determine, for each word generated by the decoder except at the end of the sentence, the semantic similarity between the generated sentence consisting of the word and the words preceding the word and the target translation sentence.

[0080] In some embodiments, the determination module 430 is used to take the generated sentence consisting of the word and the words before the word and the target translation sentence as a sentence pair, input the bidirectional encoding representation BERT model based on the converter, and obtain the feature vector of the output sentence pair; input the feature vector of the sentence pair into the activation function module to obtain the semantic similarity between the generated sentence and the target translation sentence.

[0081] In some embodiments, the determination module 430 is used to generate a random number and compare the random number with a reference value, where the reference value is within the value range of the random number; determine whether the random number is less than the reference value, and if it is less than the reference value, determine the semantic similarity between the generated sentence consisting of the word and the words before the word and the target translation sentence.

[0082] In some embodiments, the reference value increases as the number of training sessions increases.

[0083] The generating module 440 is configured to select the word or the word at the same position as the word in the target translation sentence according to the semantic similarity to generate the word at the next position.

[0084] In some embodiments, the generation module 440 is used to generate the word at the next position based on the word when the semantic similarity is higher than a threshold; and to generate the word at the next position based on the word at the same position as the word in the target translation sentence when the semantic similarity is lower than the threshold.

[0085] In some embodiments, the threshold increases as the number of training times increases.

[0086] In some embodiments, the generation module 440 is used to select the word or the word at the same position as the word in the target translation sentence based on semantic similarity as the input word; the state of the output of the decoding module corresponding to the word and the word vector of the input word are input into the decoding module corresponding to the word at the next position to obtain the output word at the next position.

[0087] The training module 450 is used to train the translation model according to the differences between the translation sentence composed of words at each position generated by the decoder and the target translation sentence.

[0088] In some embodiments, the apparatus 40 further includes a translation module 460 configured to input the sentence to be translated into the trained translation model to obtain a corresponding translated sentence.

[0089] In some embodiments, the translation module 460 is used to input the sentence to be translated into the encoder of the translation model, obtain the feature vector of the sentence to be translated, and input the feature vector of the sentence to be translated into the decoder of the translation model; according to the probability value of each word at each position output by the decoder, a preset number of words are selected as multiple alternative words for each position; according to the multiple alternative words at each position, multiple alternative words for the next position of each position are generated until the end of the sentence is reached, wherein the number of alternative words at each position is the same; the alternative words for each position are generated by the decoder to form multiple alternative translation sentences, wherein the generation of words in each alternative translation sentence is correlated; according to the probability value of each alternative translation sentence, an alternative translation sentence is selected as the translation sentence of the sentence to be translated.

[0090] In some embodiments, the translation module 460 is used to determine the selection probability value of each word at each position output by the decoder based on the probability value of the word and the probability values of the previous words associated with the generation of the word; and select a preset number of words based on the selection probability values of each word at each position output by the decoder as multiple alternative words for each position generated by the decoder.

[0091] In some embodiments, the translation module 460 is configured to determine the probability values of each candidate translation sentence generated by the decoder based on the probability values of each word in the translation sentence; and select the candidate translation sentence with the largest probability value as the translation sentence of the sentence to be translated.

[0092] The processing device of the translation model in the embodiment of the present disclosure can be implemented by various computing devices or computer systems. Figure 5 as well as Figure 6 Provide a description.

[0093] Figure 5 FIG. 1 is a structural diagram of some embodiments of the processing device of the translation model disclosed in the present invention. Figure 5 As shown, the apparatus 50 of this embodiment includes: a memory 510 and a processor 520 coupled to the memory 510 , wherein the processor 520 is configured to execute the translation model processing method in any of the embodiments of the present disclosure based on instructions stored in the memory 510 .

[0094] The memory 510 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, a database, and other programs.

[0095] Figure 6FIG. 1 is a structural diagram of some other embodiments of the processing device of the translation model disclosed in the present invention. Figure 6 As shown, the device 60 of this embodiment includes: a memory 610 and a processor 620, which are similar to the memory 510 and the processor 520, respectively. It may also include an input / output interface 630, a network interface 640, a storage interface 650, and the like. These interfaces 630, 640, 650, as well as the memory 610 and the processor 620, may be connected, for example, via a bus 660. The input / output interface 630 provides a connection interface for input / output devices such as a display, mouse, keyboard, and touch screen. The network interface 640 provides a connection interface for various networked devices, such as a database server or a cloud storage server. The storage interface 650 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0096] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transitory storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0100] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for processing a translation model, comprising: Obtain multiple sets of training sentences, where each set of training sentences includes: an original sentence and a target translation sentence; For each set of training sentences, the original sentence is input into the encoder of the translation model to obtain the feature vector of the original sentence, and the feature vector of the original sentence is input into the decoder of the translation model; For each word generated by the decoder except the end of the sentence, determining the semantic similarity between the generated sentence consisting of the word and each word preceding the word and the target translation sentence; Selecting the word or the word at the same position as the word in the target translation sentence according to the semantic similarity to generate a word at the next position, until the word at the next position is the word at the end of the sentence; The translation model is trained according to the difference between the translation sentence composed of words at each position generated by the decoder and the target translation sentence.

2. The processing method according to claim 1, wherein Determining the semantic similarity between the generated sentence consisting of the word and the words preceding the word and the target translation sentence includes: The generated sentence consisting of the word and the words preceding it and the target translation sentence are input into the transformer-based bidirectional encoding representation BERT model as a sentence pair, and a feature vector of the sentence pair is output; The feature vector of the sentence pair is input into an activation function module to obtain the semantic similarity between the generated sentence and the target translation sentence.

3. The processing method according to claim 1, wherein: Determining the semantic similarity between the generated sentence consisting of the word and the words preceding the word and the target translation sentence includes: Generate a random number, and compare the random number with a reference value, wherein the reference value is within a value range of the random number; Determine whether the random number is less than the reference value, and if so, determine the semantic similarity between the generated sentence consisting of the word and each word preceding the word and the target translation sentence.

4. The processing method according to claim 3, wherein: The reference value increases as the number of training times increases.

5. The processing method according to claim 1, wherein: The step of selecting the word or the word at the same position as the word in the target translation sentence according to the semantic similarity to generate the word at the next position includes: When the semantic similarity is higher than a threshold, generating a word at the next position according to the word; When the semantic similarity is lower than a threshold, a word at the next position is generated according to a word at the same position as the word in the target translation sentence.

6. The processing method according to claim 5, wherein: The threshold increases as the number of training times increases.

7. The processing method according to claim 1, wherein: The decoder includes a plurality of decoding modules, and the step of selecting the word or the word at the same position as the word in the target translation sentence according to the semantic similarity to generate the word at the next position includes: Selecting the word or a word in the target translation sentence at the same position as the word as an input word according to the semantic similarity; The state of the output of the decoding module corresponding to the word is input, and the word vector of the input word is input into the decoding module corresponding to the word at the next position to obtain the output word at the next position.

8. The processing method according to any one of claims 1 to 7, further comprising: Input the sentence to be translated into the trained translation model to obtain the corresponding translation sentence.

9. The processing method according to claim 8, wherein: Inputting the sentence to be translated into the trained translation model to obtain the corresponding translated sentence includes: Inputting the sentence to be translated into an encoder of a translation model to obtain a feature vector of the sentence to be translated, and inputting the feature vector of the sentence to be translated into a decoder of the translation model; selecting a preset number of words as multiple candidate words for each position based on the probability values of each word at each position output by the decoder; generating a plurality of candidate words for the next position of each position according to the plurality of candidate words for each position, until the end of the sentence is reached, wherein the number of candidate words for each position is the same; Using the decoder to generate candidate words at each position to form a plurality of candidate translation sentences, wherein the words in each candidate translation sentence are generated in a correlated manner; An alternative translation sentence is selected according to the probability values of the alternative translation sentences as the translation sentence of the sentence to be translated.

10. The processing method according to claim 9, wherein: The step of selecting a preset number of words as the multiple candidate words for each position according to the probability values of the words at each position output by the decoder includes: For each word at each position output by the decoder, determining a selection probability value of the word based on the probability value of the word and the probability values of each previous word associated with generating the word; According to the selection probability values of the respective words at each position output by the decoder, a preset number of words are selected as the multiple candidate words for each position generated by the decoder.

11. The processing method according to claim 9, wherein: The step of selecting an alternative translation sentence according to the probability values of the alternative translation sentences as the translation sentence of the sentence to be translated comprises: For each candidate translation sentence generated by the decoder, determining a probability value of each candidate translation sentence based on the probability value of each word in the translation sentence; The candidate translation sentence with the largest probability value is selected as the translation sentence of the sentence to be translated.

12. A translation model processing device, comprising: An acquisition module is used to acquire multiple sets of training sentences, wherein each set of training sentences includes: an original sentence and a target translation sentence; An input module, for each set of training sentences, inputting the original sentence into the encoder of the translation model to obtain a feature vector of the original sentence, and inputting the feature vector of the original sentence into the decoder of the translation model; a determination module for determining, for each word generated by the decoder except at the end of the sentence, the semantic similarity between the generated sentence consisting of the word and the words preceding the word and the target translation sentence; A generation module, configured to select the word or a word at the same position as the word in the target translation sentence according to the semantic similarity to generate a word at the next position, until the word at the next position is a word at the end of the sentence; A training module is used to train the translation model according to the differences between the translation sentence composed of words at each position generated by the decoder and the target translation sentence.

13. The processing device according to claim 12, further comprising: The translation module is used to input the sentence to be translated into the trained translation model to obtain the corresponding translated sentence.

14. A translation model processing device, comprising: processor; as well as A memory coupled to the processor, for storing instructions, wherein when the instructions are executed by the processor, the processor executes the method for processing a translation model according to any one of claims 1 to 11.

15. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images