Translation model training method and device, translation method and device

Abstract

The present application provides a translation model training method and device, and a translation method and device, wherein the translation model includes a first encoder, a second encoder and a decoder; wherein the translation model training method includes: receiving a training sample, wherein the training sample includes a training sentence, a target sentence corresponding to the training sentence and a target object corresponding to the training sentence, and the target object includes a target prefix and / or a target suffix; inputting the target object into the first encoder for encoding processing to obtain a first encoding vector; inputting the training sentence and the first encoding vector into the second encoder for encoding processing to obtain a second encoding vector; inputting the target sentence and the second encoding vector into the decoder for decoding processing to obtain a decoding vector, and calculating a loss value according to the decoding vector; adjusting the parameters of the translation model according to the loss value, and continuing to train the translation model until a training stop condition is reached.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a translation model training method and apparatus, a translation method and apparatus, a computing device, and a computer-readable storage medium. Background Art

[0002] With the advancement of artificial intelligence technology, neural networks are being used more and more widely, such as building neural machine translation models to achieve the conversion of sentences to be translated into target sentences.

[0003] Neural machine translation models usually use an encoder-decoder structure to achieve modeling of variable-length input sentences. The encoder "understands" the source language sentence and forms a floating-point vector of a specific dimension. The decoder then generates the translation result of the target language word by word based on this vector. In the prior art, for translation results with specified prefixes or suffixes, traditional neural machine translation models usually use engineering means to force the first few vectors or the last few vectors of the model decoding stage to be fixed. Although translation results with fixed prefixes or suffixes can be generated, the translation results of the model will not be smooth enough.

[0004] Therefore, how to make the training effect of the translation model of the specified prefix or suffix better and the translation result more natural has become an urgent problem to be solved by technical personnel. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a translation model training method and apparatus, a translation method and apparatus, a computing device, and a computer-readable storage medium to address the technical defects in the prior art.

[0006] According to a first aspect of an embodiment of the present application, a translation model training method is provided, wherein the translation model includes a first encoder, a second encoder and a decoder;

[0007] The translation model training method comprises:

[0008] Receiving a training sample, wherein the training sample includes a training sentence, a target sentence corresponding to the training sentence, and a target object corresponding to the training sentence, wherein the target object includes a target prefix and / or a target suffix;

[0009] Inputting the target object into the first encoder for encoding to obtain a first encoding vector;

[0010] Inputting the training sentence and the first encoding vector into the second encoder for encoding to obtain a second encoding vector;

[0011] Inputting the target sentence and the second encoding vector into the decoder for decoding to obtain a decoding vector, and calculating a loss value according to the decoding vector;

[0012] The parameters of the translation model are adjusted according to the loss value, and the translation model is continuously trained until a training stop condition is reached.

[0013] Optionally, the first encoder includes a first embedding layer and x object encoding layers, where x is a positive integer greater than or equal to 1;

[0014] The step of inputting the target object into the first encoder for encoding to obtain a first encoding vector includes:

[0015] Input the target object into the first embedding layer for embedding processing to obtain a target object vector, and input the target object vector into the first object encoding layer for encoding processing to obtain a first object encoding vector;

[0016] Inputting the first object coding vector into a second object coding layer for coding to obtain a second object coding vector;

[0017] Until the x-1th object coding vector is input into the xth object coding layer for coding processing to obtain the first coding vector.

[0018] Optionally, the second encoder comprises a second embedding layer and y sentence encoding layers, where y is a positive integer greater than or equal to 1;

[0019] The step of inputting the training sentence and the first encoding vector into the second encoder for encoding to obtain the second encoding vector includes:

[0020] Input the training sentence into the second embedding layer for embedding processing to obtain a training sentence vector, and input the training sentence vector and the first encoding vector into the first sentence encoding layer for encoding processing to obtain a first sentence encoding vector;

[0021] Inputting the first sentence vector into a second sentence encoding layer for encoding to obtain a second sentence encoding vector;

[0022] Until the y-1th coding vector is input into the yth sentence coding layer for coding processing to obtain the second coding vector.

[0023] Optionally, the decoder includes a third embedding layer and z decoding layers, where z is a positive integer greater than or equal to 1;

[0024] The step of inputting the target sentence and the second encoding vector into the decoder for decoding to obtain a decoding vector includes:

[0025] Input the target sentence into the third embedding layer for embedding processing to obtain a target sentence vector, and input the target sentence vector and the second encoding vector into the first decoding layer for decoding processing to obtain a first decoding vector;

[0026] Inputting the first decoding vector into a second decoding layer for decoding to obtain a second decoding vector;

[0027] Until the z-1th decoding vector is input into the zth decoding layer for decoding processing to obtain a decoding vector.

[0028] Optionally, it is characterized in that calculating the loss value according to the decoding vector includes:

[0029] The decoded vector is compared with a preset standard vector to obtain a loss value of the decoded vector.

[0030] Optionally, it is characterized in that adjusting the parameters of the translation model according to the loss value includes:

[0031] The loss value is back-propagated to sequentially update the decoding parameters of the decoder, the encoding parameters of the second encoder, and the encoding parameters of the first encoder.

[0032] Optionally, it is characterized in that the continuing to train the translation model until a training stop condition is reached includes:

[0033] When the loss value is less than the target value, the training of the translation model is stopped.

[0034] According to a second aspect of an embodiment of the present application, a translation method is provided, comprising:

[0035] Acquire a sentence to be translated and a specified object, wherein the object includes a prefix and / or a suffix;

[0036] Inputting the object into a first encoder of a translation model for encoding to obtain a first encoding vector, wherein the translation model is trained by the above-mentioned translation model training method;

[0037] Inputting the first encoding vector and the sentence to be translated into a second encoder of a translation model for encoding to obtain a second encoding vector;

[0038] The second encoding vector is input into the decoder of the translation model for decoding to obtain the target sentence.

[0039] According to a third aspect of an embodiment of the present application, a training device for a translation model is provided, wherein the translation model includes a first encoder, a second encoder, and a decoder;

[0040] The training device of the translation model comprises:

[0041] The receiving module is configured to receive a training sample, wherein the training sample includes a training sentence, a target sentence corresponding to the training sentence, and a target object corresponding to the training sentence, and the target object includes a target prefix and / or a target suffix.

[0042] A first encoding module is configured to input the target object into the first encoder for encoding to obtain a first encoding vector;

[0043] A second encoding module is configured to input the training sentence and the first encoding vector into the second encoder for encoding to obtain a second encoding vector;

[0044] A decoding module, configured to input the target sentence and the second encoding vector into the decoder for decoding to obtain a decoding vector, and calculate a loss value according to the decoding vector;

[0045] The training module is configured to adjust the parameters of the translation model according to the loss value and continue to train the translation model until a training stop condition is reached.

[0046] According to a fourth aspect of the embodiments of the present application, a translation device is provided, including:

[0047] An acquisition module is configured to acquire a sentence to be translated and a specified object, wherein the object includes a prefix and / or a suffix;

[0048] A first encoding module is configured to input the object into a first encoder of a translation model for encoding to obtain a first encoding vector, wherein the translation model is trained by the training method according to any one of claims 1 to 7;

[0049] A second encoding module is configured to input the first encoding vector and the sentence to be translated into a second encoder of a translation model for encoding to obtain a second encoding vector;

[0050] The decoding module is configured to input the second encoding vector into the decoder of the translation model for decoding to obtain a target sentence.

[0051] According to a fifth aspect of an embodiment of the present application, a computing device is provided, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the translation model training method or the translation method when executing the instructions.

[0052] According to a sixth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer instructions, and when the instructions are executed by a processor, the steps of the translation model training method or the translation method are implemented.

[0053] According to a seventh aspect of an embodiment of the present application, a chip is provided, which stores computer instructions, and when the instructions are executed by the chip, the steps of the translation model training method or the translation method are implemented.

[0054] In an embodiment of the present application, a training sample is received, wherein the training sample includes a training sentence, a target sentence corresponding to the training sentence, and a target object corresponding to the training sentence; the target object is input into the first encoder for encoding processing to obtain a first encoding vector; the training sentence and the first encoding vector are input into the second encoder for encoding processing to obtain a second encoding vector; the target sentence and the second encoding vector are input into the decoder for decoding processing to obtain a decoding vector, and a loss value is calculated based on the decoding vector; the parameters of the translation model are adjusted based on the loss value, and the translation model is continuously trained until the training stop condition is reached. By improving the translation model and adding a first encoder specifically for encoding objects to the translation model, the auxiliary translation model can better translate results with specified objects, ensure the fluency of the translation results without losing the specified objects in the translation model, and make the translation results more natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic diagram of the structure of the translation model provided in the embodiment of the present application;

[0056] Figure 2 is a structural block diagram of a computing device provided in an embodiment of the present application;

[0057] Figure 3 is a flowchart of the translation model training method provided in an embodiment of the present application;

[0058] Figure 4A is a schematic diagram of the structure of the first encoder in the translation model provided in an embodiment of the present application;

[0059] Figure 4B is a schematic diagram of the structure of the second encoder in the translation model provided in an embodiment of the present application;

[0060] Figure 4C is a schematic diagram of the structure of a decoder in a translation model provided in an embodiment of the present application;

[0061] Figure 5 is a flowchart of the translation method provided in an embodiment of the present application;

[0062] Figure 6Schematic diagram of the structure of the training device of the translation model provided in the embodiment of the present application;

[0063] Figure 7 It is a schematic diagram of the structure of the translation device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0064] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0065] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0066] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "in response to determination".

[0067] First, for ease of understanding without limitation, the following explanations are given of the terms involved in one or more embodiments of the present invention. For machine translation tasks in the field of natural language processing:

[0068] Encoder: Convert the sentence to be translated from text into an encoding vector.

[0069] Decoder: Converts the encoded vector into the language of the translated sentence.

[0070] Training sentences: sentences to be translated for training translation models.

[0071] Target sentence: The sentence after the sentence to be translated is translated.

[0072] Prefix: refers to the first few characters or words in a sentence, such as "I am at home", "I am doing homework", "I am resting", and "I like" in "I like music", "I like swimming", and "I like my family".

[0073] Suffix: refers to the last few characters or words in a sentence, such as "beautiful as flowers", "you are so beautiful", "this dress is really beautiful", and "she is" in "What a lovely baby she is", "How lovelyshe is".

[0074] In the present application, a translation model training method and apparatus, a translation method and apparatus, a computing device, and a computer-readable storage medium are provided, which are described in detail one by one in the following embodiments.

[0075] Figure 1 A schematic diagram of the structure of a translation model provided in an embodiment of the present application is shown, wherein the translation model includes a first encoder 102 , a second encoder 104 and a decoder 106 .

[0076] The first encoder 102 includes an input port and an output port. When training the translation model, the input port is used to input the target object, and when the translation model performs sentence translation, the input port is used to input the target object or a specified object; the output port is used to output a first encoding vector obtained by encoding the target object or the specified object.

[0077] The second encoder 104 includes two input ports and one output port. When training the translation model, the first input port of the second encoder is used to input the training sentence, and when the translation model performs sentence translation, the first input port of the second encoder is used to input the sentence to be translated; the second input port of the second encoder is used to input the first encoding vector; and the output port is used to output the second encoding vector.

[0078] The decoder 106 also includes two input ports and one output port, wherein the first input port of the decoder is used to input the target sentence, the second input port of the decoder is used to input the second encoding vector, and the output port is used to output the encoding vector.

[0079] Figure 2 The structure block diagram of a computing device 200 according to an embodiment of the present application is shown. The components of the computing device 200 include but are not limited to a memory 210 and a processor 220. The processor 220 is connected to the memory 210 via a bus 230, and the database 250 is used to store data.

[0080] The computing device 200 also includes an access device 240 that enables the computing device 200 to communicate via one or more networks 260. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 240 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0081] In one embodiment of the present application, the above components of the computing device 200 and Figure 2 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 2 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0082] The computing device 200 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 200 may also be a mobile or stationary server.

[0083] The processor 220 may execute Figure 3 Steps in the translation model training method shown. Figure 3 A flowchart of a translation model training method according to an embodiment of the present application is shown. Figure 1 As shown, it includes steps 302 to 310.

[0084] Step 302: Receive a training sample, wherein the training sample includes a training sentence, a target sentence corresponding to the training sentence, and a target object corresponding to the training sentence, wherein the target object includes a target prefix and / or a target suffix.

[0085] Specifically, the training samples are samples used to train a translation model, and each training sample includes a training sentence, a target sentence corresponding to the training sentence, and a target object corresponding to the training sentence. The training sentence is a sentence to be translated, the target sentence is a sentence after the training sentence has been translated by standard, and the target object is a fixed prefix and / or suffix of the translation sentence obtained after translating the training sentence. The target object can be a target prefix, a target suffix, or a combination of a target prefix and a target suffix.

[0086] In practical applications, the language of the training sentence and the target sentence can be any natural language such as Chinese, English, French, Italian, Japanese, German, etc. In this application, the language of the training sentence and the target sentence is not limited. In addition, the language of the target object and the target sentence is the same language.

[0087] In the embodiments provided in the present application, the training sentence obtained is “I like drawing, and I also like singing”, the target sentence corresponding to the training sentence is “I like drawing and singing”, and the target prefix corresponding to the training sentence is “I like drawing”; or the training sentence obtained is “I like drawing, and I also like singing”, the target sentence corresponding to the training sentence is “I like drawing and I like singing”, and the target suffix corresponding to the training sentence is “I like singing”; or the training sentence obtained is “I like drawing, and I also like singing”, the target sentence corresponding to the training sentence is “I like drawing, I like singing, too”, and the target object corresponding to the training sentence includes the target prefix “I like drawing” and the target suffix “too”, that is, the target object is “I like drawing…too”.

[0088] Step 304: Input the target object into the first encoder for encoding to obtain a first encoding vector.

[0089] Specifically, the first encoder is specifically used to encode the target object, and the first encoding vector is used for the information of the target object, which can assist the translation model to obtain a more natural and smoother translation result when a prefix and / or suffix is ​​specified.

[0090] In the embodiment provided in the present application, when the target object corresponding to the training sentence is the target prefix "I like drawing", the target prefix "I like drawing" is input into the first encoder for encoding processing; when the target object corresponding to the training sentence is the target suffix "I like singing", the target suffix "I like singing" is input into the first encoder for encoding processing; when the target object corresponding to the training sentence is the target prefix "I like drawing" and the target suffix "too", "I like drawing...too" is input into the first encoder for encoding processing.

[0091] Further, the first encoder includes a first embedding layer and x object encoding layers, where x is a positive integer greater than or equal to 1. Specifically, the first embedding layer, the first object encoding layer to the xth object encoding layer are sequentially connected. Thus, the specific process of inputting the target object into the first encoder for encoding processing to obtain the first encoding vector may be:

[0092] Input the target object into the first embedding layer for embedding processing to obtain a target object vector, and input the target object vector into the first object encoding layer for encoding processing to obtain a first object encoding vector;

[0093] Inputting the first object coding vector into a second object coding layer for coding to obtain a second object coding vector;

[0094] Until the x-1th coding vector is input into the xth object coding layer for coding processing to obtain the first coding vector.

[0095] Specifically, the embedding processing refers to the process of representing an object with a low-dimensional vector, such as using a vector to represent a word or an object. The property of the embedded vector is that objects corresponding to vectors with similar distances have similar meanings. Embedding can encode objects with low-dimensional vectors and retain their meanings, which is very suitable for deep learning.

[0096] In practical applications, see Figure 4A , the target object is input into the first embedding layer 402 in the first encoder, and embedded to generate a target object vector, and then the target object vector is input into the first object encoding layer 404 for encoding, thereby obtaining a first object encoding vector, and then the first object encoding vector is input into the second object encoding layer 406 for encoding, thereby obtaining a second object encoding vector, and so on, on the basis of obtaining the x-1th object encoding vector, it is input into the xth object encoding layer 408 for encoding, and the first encoding vector is output.

[0097] Step 306: Input the training sentence and the first encoding vector into the second encoder for encoding to obtain a second encoding vector.

[0098] Specifically, the second encoder is used to encode the training sentence.

[0099] In the embodiment provided in this application, the training sentence "I like painting, and I also like singing"

[0100] When the target object corresponding to the training sentence is the target prefix "I like drawing", the first encoding vector corresponding to the target prefix "I like drawing" and the training sentence "I like drawing, I also like singing" are input to the second encoder for encoding processing; when the target object corresponding to the training sentence is the target suffix "I like singing", the first encoding vector corresponding to the target suffix "I like singing" and the training sentence "I like drawing, I also like singing" are input to the second encoder for encoding processing; when the target object corresponding to the training sentence is the target prefix "I like drawing" and the target suffix "too", the target object is "I like drawing...too", the first encoding vector corresponding to "I like drawing...too" and the training sentence "I like drawing, I also like singing" are input to the second encoder for encoding processing.

[0101] Further, the second encoder includes a second embedding layer and y sentence encoding layers, y is a positive integer greater than or equal to 1, and specifically, the second embedding layer, the first sentence encoding layer to the y-th sentence encoding layer are sequentially connected. Thus, the specific process of inputting the training sentence and the first encoding vector into the second encoder for encoding processing to obtain the second encoding vector may be:

[0102] Input the training sentence into the second embedding layer for embedding processing to obtain a training sentence vector, and input the training sentence vector and the first encoding vector into the first sentence encoding layer for encoding processing to obtain a first sentence encoding vector;

[0103] Inputting the first sentence vector into a second sentence encoding layer for encoding to obtain a second sentence encoding vector;

[0104] Until the y-1th coding vector is input into the yth sentence coding layer for coding processing to obtain the second coding vector.

[0105] In practical applications, see Figure 4B, input the training sentence into the second embedding layer 410 in the second encoder, perform embedding processing to generate a training sentence vector, then input the training sentence vector and the first encoding vector into the first sentence encoding layer 412 for encoding processing, thereby obtaining a first sentence encoding vector, then input the first sentence encoding vector into the second sentence encoding layer 414 for encoding processing, and obtain a second sentence encoding vector, and so on, on the basis of obtaining the y-1th sentence encoding vector, input it into the yth sentence encoding layer 416 for encoding processing, and output the second encoding vector.

[0106] Step 308: Input the target sentence and the second encoding vector into the decoder for decoding to obtain a decoding vector, and calculate the loss value based on the decoding vector.

[0107] There are many loss functions for calculating loss values, such as cross entropy loss function, L1 norm loss function, maximum loss function, mean square error loss function, logarithmic loss function, etc. In this application, there is no limitation on the choice of loss function for calculating loss values.

[0108] In the embodiment provided in the present application, the acquired training sentence is “I like drawing, and I also like singing”: when the target sentence corresponding to the training sentence is “I like drawing and singing”, and the target prefix corresponding to the training sentence is “I like drawing”, the target sentence “I like drawing and singing” and the second encoding vector corresponding to the training sentence and the target prefix “I like drawing” are input into the decoder for decoding processing; when the target sentence corresponding to the training sentence is “I like drawing and I like singing”, and the target suffix corresponding to the training sentence is “I like singing”, the target sentence “I like drawing and I like singing” and the second encoding vector corresponding to the training sentence and the target suffix “I like singing” are input into the decoder for decoding processing; when the target sentence corresponding to the training sentence is “I like drawing, I like singing, too”, and the target object corresponding to the training sentence is “I like drawing…too” (including the target prefix “I like drawing” and the target suffix “too”), the target sentence “I like drawing, I like singing, too” and the second encoding vector corresponding to the training sentence and the target object “I like drawing…too” are input into the decoder for decoding processing.

[0109] Further, the decoder includes a third embedding layer and z decoding layers, where z is a positive integer greater than or equal to 1. Specifically, the third embedding layer, the first decoding layer to the z-th decoding layer are sequentially connected. Thus, the specific process of inputting the target sentence and the second encoding vector into the decoder for decoding to obtain the decoding vector may be:

[0110] Input the target sentence into the third embedding layer for embedding processing to obtain a target sentence vector, and input the target sentence vector and the second encoding vector into the first decoding layer for decoding processing to obtain a first decoding vector;

[0111] Inputting the first decoding vector into a second decoding layer for decoding to obtain a second decoding vector;

[0112] Until the z-1th decoding vector is input into the zth decoding layer for decoding processing to obtain a decoding vector.

[0113] In practical applications, see Figure 4C , the target sentence is input into the third embedding layer 418 in the decoder, and the target sentence vector is generated by embedding processing, and then the target sentence vector and the second encoding vector are input into the first decoding layer 420 for decoding processing, thereby obtaining a first decoding vector, and then the first decoding vector is input into the second decoding layer 422 for encoding processing to obtain a second decoding vector, and so on. On the basis of obtaining the z-1th decoding vector, it is input into the zth decoding layer 424 for decoding processing, and the decoding vector is output.

[0114] Optionally, calculating the loss value according to the decoding vector includes: comparing the decoding vector with a preset standard vector to obtain the loss value of the decoding vector.

[0115] It should be noted that in this embodiment, when calculating the loss value of the decoding vector, a standard vector is introduced to compare with the obtained decoding vector to calculate the loss value, instead of directly comparing the decoding vector with the target sentence vector to calculate the loss value. This avoids overfitting when directly comparing the decoding vector with the target sentence vector to calculate the loss value, optimizes the performance of the translation model in other sentence translations, and makes the translation effect more natural and smooth.

[0116] Step 310: Adjust the parameters of the translation model according to the loss value, and continue to train the translation model until a training stop condition is reached.

[0117] Specifically, when adjusting the parameters of the translation model, it is necessary to adjust the parameters of the translation model according to the loss value. The specific implementation process is: back-propagating the loss value to sequentially update the decoding parameters of the decoder, the encoding parameters of the second encoder, and the encoding parameters of the first encoder.

[0118] In practical applications, first, the decoding parameters of the decoder are updated: for a decoder containing z decoding layers, the zth decoding layer, the z-1th decoding layer, ... the second decoding layer, and the first decoding layer are updated in sequence; secondly, the encoding parameters of the second encoder are updated: for an encoder containing y sentence coding layers, the yth sentence coding layer, the y-1th sentence coding layer, ... the second sentence coding layer, and the first sentence coding layer are updated in sequence; finally, the encoding parameters of the first encoder are updated: for an encoder containing x object coding layers, the xth object coding layer, the x-1th object coding layer, ... the second object coding layer, and the first object coding layer are updated in sequence.

[0119] Optionally, continuing to train the translation model until a training stop condition is reached includes: when the loss value is less than a target value, stopping training the translation model.

[0120] In this embodiment, a target value is set in advance. When the loss value of the translation model is lower than the preset target value, it means that the translation model has reached the translation standard or a certain translation level, that is, the training stop condition has been reached, and therefore, the training of the translation model is stopped.

[0121] The translation model training method provided in this specification receives a training sample, wherein the training sample includes a training sentence, a target sentence corresponding to the training sentence, and a target object corresponding to the training sentence, and the target object includes a target prefix and / or a target suffix; the target object is input into the first encoder for encoding processing to obtain a first encoding vector; the training sentence and the first encoding vector are input into the second encoder for encoding processing to obtain a second encoding vector; the target sentence and the second encoding vector are input into the decoder for decoding processing to obtain a decoding vector, and a loss value is calculated according to the decoding vector; the parameters of the translation model are adjusted according to the loss value, and the translation model is continuously trained until the training stop condition is reached. By improving the translation model and adding a first encoder specifically for encoding objects to the translation model, the auxiliary translation model can better translate results with specified objects, ensure the fluency of the translation results without losing the specified objects in the translation model, and make the translation results more natural.

[0122] Figure 5 A flowchart of a translation method according to an embodiment of the present application is shown, including steps 502 to 508 .

[0123] Step 502: Acquire a sentence to be translated and a specified object, wherein the object includes a prefix and / or a suffix.

[0124] In the embodiment provided in the present application, the sentence to be translated is "I love China, I love the world", and the specified prefix is ​​"I love China". Alternatively, the sentence to be translated is "I love China, I love the world", and the specified suffix is ​​"too".

[0125] Step 504: Input the object into a first encoder of a translation model for encoding to obtain a first encoding vector, wherein the translation model is trained by the above-mentioned translation model training method.

[0126] In the embodiment provided by the present application, the specified prefix "I love China" is input into the first encoder of the translation model for encoding, and the first encoding vector corresponding to the specified prefix "I love China" is obtained. Alternatively, the specified suffix "too" is input into the first encoder of the translation model for encoding, and the first encoding vector corresponding to the specified suffix "too" is obtained.

[0127] Step 506: Input the first encoding vector and the sentence to be translated into a second encoder of a translation model for encoding to obtain a second encoding vector.

[0128] In the embodiment provided in the present application, the first encoding vector corresponding to "I love China" and the sentence to be translated "I love China, I love the world" are input into the second encoder of the translation model for encoding to obtain the second encoding vector. Alternatively, the first encoding vector corresponding to "too" and the sentence to be translated "I love China, I love the world" are input into the second encoder of the translation model for encoding to obtain the second encoding vector.

[0129] Step 508: Input the second encoding vector into the decoder of the translation model for decoding to obtain the target sentence.

[0130] In the embodiment provided by the present application, when the specified object is the specified prefix "I love China", the second encoding vector is input into the decoder of the translation model for decoding, and the target sentence "I love China and the world" corresponding to the sentence to be translated "I love China, I love the world" is obtained. When the specified object is the specified suffix "too", the second encoding vector is input into the decoder of the translation model for decoding, and the target sentence "I love China, I love the world, too" corresponding to the sentence to be translated "I love China, I love the world" is obtained.

[0131] The translation method provided in this specification obtains a sentence to be translated and a specified object, wherein the object includes a prefix and / or a suffix; the object is input into a first encoder of a translation model for encoding to obtain a first encoding vector, wherein the translation model is trained by the above-mentioned translation model training method; the first encoding vector and the sentence to be translated are input into a second encoder of the translation model for encoding to obtain a second encoding vector; the second encoding vector is input into a decoder of the translation model for decoding to obtain a target sentence. The sentence to be translated and the specified object are input into a pre-trained translation model to obtain a more fluent and natural target sentence with the specified object.

[0132] Corresponding to the above-mentioned translation model training method embodiment, the present application also provides a translation model training device embodiment, Figure 6 A schematic diagram of the structure of a training device for a translation model according to an embodiment of the present application is shown, wherein the translation model includes a first encoder, a second encoder and a decoder. Figure 6 As shown, the device comprises:

[0133] The receiving module 602 is configured to receive a training sample, wherein the training sample includes a training sentence, a target sentence corresponding to the training sentence, and a target object corresponding to the training sentence, wherein the target object includes a target prefix and / or a target suffix;

[0134] A first encoding module 604 is configured to input the target object into the first encoder for encoding to obtain a first encoding vector;

[0135] A second encoding module 606 is configured to input the training sentence and the first encoding vector into the second encoder for encoding to obtain a second encoding vector;

[0136] A decoding module 608 is configured to input the target sentence and the second encoding vector into the decoder for decoding to obtain a decoding vector, and calculate a loss value according to the decoding vector;

[0137] The training module 610 is configured to adjust the parameters of the translation model according to the loss value, and continue to train the translation model until a training stop condition is reached.

[0138] Optionally, the first encoder includes a first embedding layer and x object encoding layers, where x is a positive integer greater than or equal to 1;

[0139] The first encoding module 604 is further configured to input the target object into the first embedding layer for embedding processing to obtain a target object vector, input the target object vector into the first object encoding layer for encoding processing to obtain a first object encoding vector, input the first object encoding vector into the second object encoding layer for encoding processing to obtain a second object encoding vector, until the x-1th object encoding vector is input into the xth object encoding layer for encoding processing to obtain a first encoding vector.

[0140] Optionally, the second encoder includes a second embedding layer and y sentence encoding layers, where y is a positive integer greater than or equal to 1;

[0141] The second encoding module 606 is also configured to input the training sentence into the second embedding layer for embedding processing to obtain a training sentence vector, input the training sentence vector and the first encoding vector into the first sentence encoding layer for encoding processing to obtain a first sentence encoding vector, input the first sentence vector into the second sentence encoding layer for encoding processing to obtain a second sentence encoding vector, until the y-1th sentence encoding vector is input into the yth sentence encoding layer for encoding processing to obtain a second encoding vector.

[0142] Optionally, the decoder includes a third embedding layer and z decoding layers, where z is a positive integer greater than or equal to 1;

[0143] The decoding module 608 is further configured to input the target sentence into the third embedding layer for embedding processing to obtain a target sentence vector, input the target sentence vector and the second encoding vector into the first decoding layer for decoding processing to obtain a first decoding vector, input the first decoding vector into the second decoding layer for decoding processing to obtain a second decoding vector, until the z-1th decoding vector is input into the zth decoding layer for decoding processing to obtain a decoding vector.

[0144] The decoding module 608 is further configured to compare the decoded vector with a preset standard vector to obtain a loss value of the decoded vector.

[0145] The training module 610 is further configured to back-propagate the loss value to sequentially update the decoding parameters of the decoder, the encoding parameters of the second encoder, and the encoding parameters of the first encoder.

[0146] The training module 610 is further configured to stop training the translation model when the loss value is less than a target value.

[0147] The training device of the translation model provided in this specification includes a receiving module configured to receive a training sample, wherein the training sample includes a training sentence, a target sentence corresponding to the training sentence, and a target object corresponding to the training sentence, and the target object includes a target prefix and / or a target suffix; a first encoding module configured to input the target object into the first encoder for encoding processing to obtain a first encoding vector; a second encoding module 606 configured to input the training sentence and the first encoding vector into the second encoder for encoding processing to obtain a second encoding vector; a decoding module 608 configured to input the target sentence and the second encoding vector into the decoder for decoding processing to obtain a decoding vector, and calculate a loss value based on the decoding vector; a training module 610 configured to adjust the parameters of the translation model based on the loss value, and continue to train the translation model until the training stop condition is reached. By improving the translation model and adding a first encoder specifically for encoding the target object to the translation model, the auxiliary translation model can better translate the result with the specified target object, ensure the fluency of the translation result without losing the specified target object in the translation model, and make the translation result more natural.

[0148] The above is a schematic scheme of a training device for a translation model of this embodiment. It should be noted that the technical scheme of the training device for the translation model and the technical scheme of the training method for the translation model are of the same concept, and the details not described in detail in the technical scheme of the training device for the translation model can be found in the description of the technical scheme of the training method for the translation model.

[0149] It should be noted that the components in the device claim should be understood as the functional modules that must be established to implement the steps of the program flow or the steps of the method, and the functional modules are not actually divided or separated. The device claim defined by such a group of functional modules should be understood as the functional module architecture of the solution mainly implemented by the computer program recorded in the specification, and should not be understood as the physical device that mainly implements the solution by hardware.

[0150] Corresponding to the above translation method embodiment, the present application also provides a translation device embodiment, Figure 7 FIG. 1 is a schematic diagram showing the structure of a translation device according to an embodiment of the present application. Figure 7As shown, the device comprises:

[0151] The acquisition module 702 is configured to acquire a sentence to be translated and a specified object, wherein the object includes a prefix and / or a suffix;

[0152] A first encoding module 704 is configured to input the object into a first encoder of a translation model for encoding to obtain a first encoding vector, wherein the translation model is trained by the above-mentioned translation model training method;

[0153] A second encoding module 706 is configured to input the first encoding vector and the sentence to be translated into a second encoder of a translation model for encoding to obtain a second encoding vector;

[0154] The decoding module 708 is configured to input the second encoding vector into the decoder of the translation model for decoding to obtain a target sentence.

[0155] The translation device provided in this specification includes an acquisition module configured to acquire a sentence to be translated and a specified object, wherein the object includes a prefix and / or a suffix; a first encoding module configured to input the object into a first encoder of a translation model for encoding to obtain a first encoding vector, wherein the translation model is trained by the above-mentioned translation model training method; a second encoding module configured to input the first encoding vector and the sentence to be translated into a second encoder of the translation model for encoding to obtain a second encoding vector; and a decoding module configured to input the second encoding vector into a decoder of the translation model for decoding to obtain a target sentence. The sentence to be translated and the specified object are input into a pre-trained translation model to obtain a more fluent and natural target sentence with a specified object.

[0156] The above is a schematic solution of a translation device of this embodiment. It should be noted that the technical solution of the translation device and the technical solution of the above translation method belong to the same concept, and the details of the technical solution of the translation device that are not described in detail can all be referred to the description of the technical solution of the above translation method.

[0157] It should be noted that the components in the device claim should be understood as the functional modules that must be established to implement the steps of the program flow or the steps of the method, and the functional modules are not actually divided or separated. The device claim defined by such a group of functional modules should be understood as the functional module architecture of the solution mainly implemented by the computer program recorded in the specification, and should not be understood as the physical device that mainly implements the solution by hardware.

[0158] In one embodiment of the present application, a computing device is also provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein when the processor executes the instructions, the translation model training method or the steps of the translation method are implemented.

[0159] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the translation model training method or the steps of the translation method as described above.

[0160] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned translation model training method or translation method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned translation model training method or translation method.

[0161] An embodiment of the present application discloses a chip storing computer instructions, which, when executed by a processor, implement the translation model training method or the steps of the translation method as described above.

[0162] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0163] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0164] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0165] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0166] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A translation model training method, characterized in that: The translation model includes a first encoder, a second encoder and a decoder; The translation model training method comprises: Receiving a training sample, wherein the training sample includes a training sentence, a target sentence corresponding to the training sentence, and a target object corresponding to the training sentence, wherein the target object includes a target prefix and / or a target suffix; Inputting the target object into the first encoder for encoding to obtain a first encoding vector, wherein the first encoder is used to encode the target object, and the first encoding vector is used to represent information of the target object; Inputting the training sentence and the first encoding vector into the second encoder for encoding to obtain a second encoding vector, wherein the second encoder is used to encode the training sentence; Inputting the target sentence and the second encoding vector into the decoder for decoding to obtain a decoding vector, and calculating a loss value according to the decoding vector; The parameters of the translation model are adjusted according to the loss value, and the translation model is continuously trained until a training stop condition is reached.

2. The translation model training method according to claim 1, characterized in that: The first encoder includes a first embedding layer and x object encoding layers, where x is a positive integer greater than or equal to 1; The step of inputting the target object into the first encoder for encoding to obtain a first encoding vector includes: Input the target object into the first embedding layer for embedding processing to obtain a target object vector, and input the target object vector into the first object encoding layer for encoding processing to obtain a first object encoding vector; Inputting the first object coding vector into a second object coding layer for coding to obtain a second object coding vector; Until the x-1th object coding vector is input into the xth object coding layer for coding processing to obtain the first coding vector.

3. The translation model training method according to claim 1, characterized in that: The second encoder includes a second embedding layer and y sentence encoding layers, where y is a positive integer greater than or equal to 1; The step of inputting the training sentence and the first encoding vector into the second encoder for encoding to obtain the second encoding vector includes: Input the training sentence into the second embedding layer for embedding processing to obtain a training sentence vector, and input the training sentence vector and the first encoding vector into the first sentence encoding layer for encoding processing to obtain a first sentence encoding vector; Inputting the first sentence vector into a second sentence encoding layer for encoding to obtain a second sentence encoding vector; Until the y-1th sentence encoding vector is input into the yth sentence encoding layer for encoding processing to obtain the second encoding vector.

4. The translation model training method according to claim 1, characterized in that: The decoder comprises a third embedding layer and z decoding layers, where z is a positive integer greater than or equal to 1; The step of inputting the target sentence and the second encoding vector into the decoder for decoding to obtain a decoding vector includes: Input the target sentence into the third embedding layer for embedding processing to obtain a target sentence vector, and input the target sentence vector and the second encoding vector into the first decoding layer for decoding processing to obtain a first decoding vector; Inputting the first decoding vector into a second decoding layer for decoding to obtain a second decoding vector; Until the z-1th decoding vector is input into the zth decoding layer for decoding processing to obtain a decoding vector.

5. The translation model training method according to claim 1, characterized in that: Calculating a loss value according to the decoded vector includes: The decoded vector is compared with a preset standard vector to obtain a loss value of the decoded vector.

6. The translation model training method according to claim 1, characterized in that: Adjusting the parameters of the translation model according to the loss value includes: The loss value is back-propagated to sequentially update the decoding parameters of the decoder, the encoding parameters of the second encoder, and the encoding parameters of the first encoder.

7. The translation model training method according to claim 1, characterized in that: The continuing to train the translation model until a training stop condition is reached includes: When the loss value is less than the target value, the training of the translation model is stopped.

8. A translation method, characterized in that: Acquire a sentence to be translated and a specified object, wherein the object includes a prefix and / or a suffix; Inputting the object into a first encoder of a translation model for encoding to obtain a first encoding vector, wherein the translation model is trained by the training method according to any one of claims 1 to 7; Inputting the first encoding vector and the sentence to be translated into a second encoder of a translation model for encoding to obtain a second encoding vector; The second encoding vector is input into the decoder of the translation model for decoding to obtain the target sentence.

9. A translation model training device, characterized in that: The translation model includes a first encoder, a second encoder and a decoder; The training device of the translation model comprises: A receiving module is configured to receive a training sample, wherein the training sample includes a training sentence, a target sentence corresponding to the training sentence, and a target object corresponding to the training sentence, wherein the target object includes a target prefix and / or a target suffix; A first encoding module is configured to input the target object into the first encoder for encoding to obtain a first encoding vector, wherein the first encoder is used to encode the target object, and the first encoding vector is used to represent information of the target object; A second encoding module is configured to input the training sentence and the first encoding vector into the second encoder for encoding to obtain a second encoding vector, wherein the second encoder is used to encode the training sentence; A decoding module, configured to input the target sentence and the second encoding vector into the decoder for decoding to obtain a decoding vector, and calculate a loss value according to the decoding vector; The training module is configured to adjust the parameters of the translation model according to the loss value and continue to train the translation model until a training stop condition is reached.

10. A translation device, characterized in that: An acquisition module is configured to acquire a sentence to be translated and a specified object, wherein the object includes a prefix and / or a suffix; A first encoding module is configured to input the object into a first encoder of a translation model for encoding to obtain a first encoding vector, wherein the translation model is trained by the training method according to any one of claims 1 to 7; A second encoding module is configured to input the first encoding vector and the sentence to be translated into a second encoder of a translation model for encoding to obtain a second encoding vector; The decoding module is configured to input the second encoding vector into the decoder of the translation model for decoding to obtain a target sentence.

11. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the instructions, the steps of the method described in any one of claims 1-7 or 8 are implemented.

12. A computer-readable storage medium storing computer instructions, characterized in that: When the instruction is executed by the processor, the steps of the method described in any one of claims 1 to 7 or 8 are implemented.

Images