A translation method and device

Abstract

The present invention discloses a translation method, comprising: using a source language sentence and its corresponding target language sentence as training samples to train and generate a first translation model; inputting the source language sentence into the first translation model to obtain a translation result of the source language sentence; determining mistranslated segments and their corresponding mistranslated source segments in the translation result by comparing the translation result with the corresponding target language sentence; using a source language sentence containing the mistranslated source segment and its corresponding target language sentence as training samples, with the goal of reducing the probability of generating mistranslated segments, training the first translation model to generate a second translation model. The present invention also discloses a corresponding device.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a translation method and device. Background Art

[0002] The current neural machine translation (NMT) system can translate sentences that are close to human language. However, the current training method of the neural machine translation model focuses more on the overall quality of the generated translation sentences, which makes the sentences translated by the model very fluent, with low accuracy and frequent mistranslations, that is, the semantics of the translation result deviates from the semantics of the input sentence to be translated.

[0003] In order to improve the accuracy of the translation model, the common practice is to collect mistranslated sentences during the application of the translation model, manually correct the mistranslated sentences, and then add the corrected sentences to the model's training set to retrain the translation model to strengthen the translation model's ability to translate the parts that it cannot translate, thereby improving translation accuracy. This method requires manual collection of mistranslated sentences and correction of mistranslated sentences, which has high manpower and time costs, and mistranslated sentences can only be collected after the translation model is put into use, and it is impossible to optimize the translation quality of the translation model before the translation model is put into use. Summary of the invention

[0004] To this end, the present invention provides a translation method and device to try to solve or at least alleviate the above problems.

[0005] According to a first aspect of the present invention, a translation method is provided, comprising: using a source language sentence and its corresponding target language sentence as training samples to train and generate a first translation model; inputting the source language sentence into the first translation model to obtain a translation result of the source language sentence; determining mistranslated segments in the translation result and their corresponding mistranslated source segments by comparing the translation result with the corresponding target language sentence; using a source language sentence containing the mistranslated source segment and its corresponding target language sentence as training samples, with reducing the probability of generating mistranslated segments as a training goal, training the first translation model to generate a second translation model.

[0006] According to a second aspect of the present invention, a translation device is provided, comprising a first training module adapted to train and generate a first translation model using a source language sentence and its corresponding target language sentence as training samples; a checking module adapted to input the source language sentence into the first translation model to obtain a translation result of the source language sentence; and to determine mistranslated segments in the translation result and their corresponding mistranslated source segments by comparing the translation result with the corresponding target language sentence; and a second training module adapted to train the first translation model and generate a second translation model using a source language sentence containing a mistranslated source segment and its corresponding target language sentence as training samples with the training goal of reducing the probability of generating mistranslated segments.

[0007] According to a third aspect of the present invention, there is provided a computing device, comprising: at least one processor and a memory storing program instructions; when the program instructions are read and executed by the processor, the computing device executes the translation method.

[0008] According to a fourth aspect of the present invention, a readable storage medium storing program instructions is provided, and when the program instructions are read and executed by a computing device, the computing device executes the translation method.

[0009] According to the translation method of the present invention, firstly, a source language sentence and its corresponding target language sentence are used as training samples to train and generate a first translation model. Subsequently, the source language sentence in the training set is input into the trained first translation model to obtain a translation result, and the mistranslated segment in the translation result and its corresponding source segment (i.e., mistranslated source segment) are determined by matching the translation result with the corresponding target language sentence. Subsequently, the source language sentence containing the mistranslated source segment and its corresponding target language sentence are used as training samples, and the first translation model is trained to generate a second translation model with the training goal of reducing the probability of generating mistranslated segments.

[0010] The translation method of the present invention can further model the mistranslated segments output by the first translation model, and optimize the first translation model, so that the generated second translation model can significantly improve the accuracy of the translation while maintaining the fluency of the translation result.

[0011] The translation method of the present invention obtains mistranslated segments by automatically comparing the translation results of the training set with the target language sentences. In addition, when training to generate the second translation model, the training samples used are the training samples in the original training set (i.e., the set of training samples used to train to generate the first translation model) that are mistranslated by the first translation model. The translation method of the present invention is automatically executed by a computing device, and there is no need to manually collect mistranslated sentences and correct them during the model online application stage. Compared with the prior art, the translation method of the present invention greatly reduces manpower and time costs.

[0012] In addition, the translation method of the present invention completes accuracy optimization before the model is put into online application, that is, the model finally put into online application is the optimized second translation model with high translation accuracy, avoiding presenting the first translation model with low translation accuracy to the user.

[0013] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to achieve the above and related purposes, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles disclosed herein can be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The above and other purposes, features and advantages of the present disclosure will become more apparent by reading the following detailed description in conjunction with the accompanying drawings. Throughout the present disclosure, the same reference numerals generally refer to the same parts or elements.

[0015] Figure 1 A schematic diagram of a computer system 9100 according to an embodiment of the present invention is shown;

[0016] Figure 2 A schematic diagram of a deep neural network of a machine learning model 9120 according to an embodiment of the present invention is shown;

[0017] Figure 3 A structural diagram of a first translation model 300 according to an embodiment of the present invention is shown;

[0018] Figure 4 A flowchart of a translation method 400 according to an embodiment of the present invention is shown;

[0019] Figure 5 A schematic diagram showing a process of training a first translation model to generate a second translation model according to an embodiment of the present invention;

[0020] Figure 6 A schematic diagram of a computing device 600 according to one embodiment of the present invention is shown;

[0021] Figure 7 FIG. 7 is a schematic diagram of a translation device 700 according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0023] In view of the problems existing in the prior art, the present invention provides a translation method and device to automatically improve the accuracy of the translation model and avoid wasting manpower and time costs.

[0024] The translation method of the present invention is used to optimize the first translation model to generate a second translation model with higher translation accuracy. After the second translation model is generated, the second translation model can be published online for users to use. Figure 1 A schematic diagram of a computer system 9100 for training and applying a second translation model is shown. Figure 1 As shown, system 9100 includes a user computing device 9110 , a server computing system 9130 , and a training computing system 9150 that are communicatively connected via a network 9180 .

[0025] The user computing device 9110 may be any type of computing device, including but not limited to, for example, a personal computing device (e.g., a laptop or desktop computer), a mobile computing device (a smart phone or tablet), a game console or controller, a wearable computing device, an embedded computing device, an edge computing device, or any other type of computing device. The user computing device 9110 may be deployed at the user site as a terminal smart device and interact with the user to process user input.

[0026] The user computing device 9110 may store or include one or more machine learning models 9120. The machine learning model 9120 may be designed to perform various tasks, such as machine translation, content filtering, image classification, object detection, speech recognition, and the like. The machine learning model 9120 may be a machine learning model such as a neural network (e.g., a deep neural network) or other types of machine learning models including nonlinear models and / or linear models. Examples of machine learning models 9120 include, but are not limited to, various types of deep neural networks (DNNs), such as feedforward neural networks, recurrent neural networks (RNNs, e.g., long short-term memory recurrent neural networks (LSTMs), transformer neural networks (Transformers) including or not including attention mechanisms), convolutional neural networks (CNNs), or other forms of neural networks. The machine learning model 9120 may include one machine learning model, or may be a combination of multiple machine learning models.

[0027] Figure 2 9120 according to some embodiments. A neural network has a layered architecture, with each network layer having one or more processing nodes (called neurons or filters) for processing. In a deep neural network, the output of the previous layer after processing is the input of the next layer, where the first layer in the architecture receives the network input for processing, and the output of the last layer is provided as the network output. Figure 2 As shown, the machine learning model 9120 includes network layers 9222, 9224, 9226, etc., wherein network layer 9222 receives network input and network layer 9226 provides network output.

[0028] In a deep neural network, the main processing operations within the network are interwoven linear and nonlinear transformations. These processes are distributed across various processing nodes. Figure 2 Also shown is an enlarged view of a node 9221 in the model 9120. The node 9221 receives multiple input values ​​a1, a2, a3, etc., and processes the input values ​​based on corresponding processing parameters (such as weights w1, w2, w3, etc.) to generate an output z. The node 9221 can be designed to process the input using an activation function, which can be expressed as:

[0029] z=σ(w T α+b) (1)

[0030] Where α represents the input vector of node 9221 (including elements a1, a2, a3, etc.); w represents the weight vector in the processing parameters used by node 9221 (including elements w1, w2, w3, etc.), and each weight is used to weight the corresponding input; b represents the bias term in the processing parameters used by node 9221, which is used to bias the corresponding input and the weighted result; σ() represents the activation function used by node 9221, and the activation function can be a linear function or a nonlinear function. Common activation functions in neural networks include sigmoid function, ReLu function, tanh function, maxout function, etc. The output of node 9221 can also be called an activation value. Depending on the network design, the output (i.e., activation value) of each network layer can be provided to one, multiple or all nodes of the next layer as input.

[0031] Each network layer in the machine learning model 9120 may include one or more nodes 9221. When the processing in the machine learning model 9120 is viewed in terms of network layers, the processing of each network layer can also be expressed in the form of formula (1), where α represents the input vector of the network layer and w represents the weight of the network layer.

[0032] It should be understood that Figure 2The architecture of the machine learning model shown and the number of network layers and processing nodes therein are all schematic. In different applications, the machine learning model can be designed to have other architectures as needed.

[0033] Continue to refer Figure 1 In some implementations, the user computing device 9110 may receive the machine learning model 9120 from the server computing system 9130 via the network 9180, store it in the memory of the user computing device, and use or implement it by an application in the user computing device.

[0034] In other implementations, the user computing device 9110 may call the machine learning model 9140 stored and implemented in the server computing system 9130. For example, the machine learning model 9140 may be implemented by the server computing system 9130 as part of a web service, so that the user computing device 9110 may call the machine learning model 9140 implemented as a web service, for example, over the network 9180 and according to a client-server relationship. Thus, the machine learning models that can be used at the user computing device 9110 include the machine learning model 9120 stored and implemented at the user computing device 9110 and / or the machine learning model 9140 stored and implemented at the server computing system 9130.

[0035] The user computing device 9110 may also include one or more user input components that receive user input. For example, the user input component may be a touch-sensitive component (e.g., a touch-sensitive display screen or a touchpad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component may be used to implement a virtual keyboard. Other exemplary user input components include a microphone, a conventional keyboard, a camera, or other devices through which a user can provide user input.

[0036] The server computing system 9130 may include one or more server computing devices. Where the server computing system 9130 includes multiple server computing devices, the server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination of the two.

[0037] As described above, the server computing system 9130 may store or include one or more machine learning models 9140. Similar to the machine learning model 9120, the machine learning model 9140 may be designed to perform various tasks, such as machine translation, content filtering, image classification, object detection, speech recognition, and the like. The model 9140 may include various machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recursive neural networks, and convolutional neural networks.

[0038] The user computing device 9110 and / or the server computing system 9130 may train the models 9120 and / or 9140 via interaction with a training computing system 9150 communicatively coupled via a network 9180. The training computing system 9150 may be separate from the server computing system 9130, or may be part of the server computing system 9130.

[0039] Similar to server computing system 9130, training computing system 9150 may include or otherwise be implemented by one or more server computing devices.

[0040] The training computing system 9150 may include a model trainer 9160 that trains the machine learning models 9120 and / or 9140 stored at the user computing device 9110 and / or the server computing system 9130 using various training or learning techniques such as, for example, back propagation of errors. In some implementations, performing back propagation of errors may include performing truncated backpropagation through time. The model trainer 9160 may perform a variety of generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the model being trained.

[0041] Specifically, the model trainer 9160 can train the machine learning model 9120 and / or 9140 based on the set of training data 9162. The training data 9162 may include a plurality of different training data sets, each of which, for example, respectively helps to train the machine learning model 9120 and / or 9140 to perform a plurality of different tasks. For example, the training data set includes a data set that helps the machine learning model 9120 and / or 9140 perform machine translation, content filtering, object detection, object segmentation, image classification, and / or other tasks.

[0042] In some implementations, if the user has explicitly consented, the training examples may be provided by the user computing device 9110. Thus, in such implementations, the model 9120 provided to the user computing device 9110 may be trained by the training computing system 9150 on user-specific data received from the user computing device 9110. In some cases, this process may be referred to as personalizing the model.

[0043] In addition, in some implementations, the model trainer 9160 may modify the machine learning model 9140 in the server computing system 9130 to obtain a machine learning model 9120 suitable for use in the user computing device 9110. These modifications include, for example, reducing the number of various parameters in the model, storing parameter values ​​with less precision, etc., so that the trained machine learning models 9120 and / or 9140 are suitable for running in view of the different processing performances of the server computing system 9130 and the user computing device 9110.

[0044] The model trainer 9160 includes computer logic for providing the desired functionality. The model trainer 9160 can be implemented in hardware, firmware, and / or software that controls a general-purpose processor. For example, in some implementations, the model trainer 9160 includes a program file stored on a storage device, loaded into a memory, and executed by one or more processors. In other implementations, the model trainer 9160 includes a collection of one or more computer executable instructions stored in a tangible computer-readable storage medium such as RAM, a hard disk, or an optical or magnetic medium. In some implementations, the model trainer 9160 can be copied and / or distributed across multiple different devices.

[0045] The network 9180 can be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and can include any number of wired or wireless links. In general, communications through the network 9180 can be carried via any type of wired and / or wireless connection, using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML, and JSON), and / or protection schemes (e.g., VPN, HTTPS, SSL).

[0046] In one embodiment of the present invention, the server computing system 9130 may call the training computing system 9150 to generate the first translation model 9142 and the second translation model 9144 of the present invention.

[0047] Specifically, the training data 9162 includes a plurality of parallel sentence pairs, and a parallel sentence pair refers to a pair of sentences with the same semantics in different languages, namely, a source language sentence and its corresponding target language sentence. The training computing system 9150 uses the source language sentence and its corresponding target language sentence as training samples to train and generate the first translation model 9142 of the present invention. The first translation model 9142 uses the source language sentence as input and outputs the translation result of the source language sentence in the target language. For example, if the source language is Chinese and the target language is English, a Chinese sentence is input into the first translation model 9142, and the first translation model 9142 will output the English translation result corresponding to the Chinese sentence. The first translation model 9142 generated by training can be deployed in the server computing system 9130.

[0048] Further, after training and generating the first translation model 9142, the server computing system 9130 (or the training computing system 9150) can input the source language sentences in the training set into the first translation model 9142 to obtain the translation result, and determine the mistranslated segment and its corresponding source segment (i.e., mistranslated source segment) in the translation result by matching the translation result with the corresponding target language sentence. Subsequently, the source language sentence containing the mistranslated source segment and its corresponding target language sentence are used as training samples, and the training goal is to reduce the probability of generating mistranslated segments, and the training computing system 9150 is called to train the first translation model to generate the second translation model 9144.

[0049] The training samples used when training and generating the second translation model 9144 are the training samples mistranslated by the first translation model in the original training set (i.e., the training sample set used to train and generate the first translation model). The objective function used when training and generating the second translation model 9144 is different from that used when training and generating the first translation model 9142. The objective function of the former is designed from the perspective of reducing the probability of generating mistranslated segments, and further modeling can be performed for the mistranslated segments output by the first translation model, and the first translation model can be optimized, so that the generated second translation model can significantly improve the accuracy of translation while maintaining the fluency of the translation results.

[0050] The second translation model 9144 generated by training can be deployed in the server computing system 9130. The second translation model 9144 takes the source language sentence as input and outputs the translation result of the source language sentence in the target language, which has a higher translation accuracy than the first translation model 9142.

[0051] The second translation model 9144 can be implemented as a part of the Web service by the server computing system 9130, so that the user computing device 9110 can, for example, call the second translation model 9144 implemented as a Web service through the network 9180 and according to the client-server relationship. In an embodiment of the present invention, the user computing device 9110 is a terminal device used by a user. The user can be, for example, an individual user, who can use the second translation model 9144 of the present invention through the user computing device 9110 to meet the translation needs in work, study and life. The user can also be an enterprise user, such as a translation company, a company with foreign-related business needs, etc., which can use the second translation model 9144 of the present invention through the user computing device 9110 to achieve accurate and efficient machine translation for commercial purposes, greatly reducing the manpower and time costs of translation work.

[0052] Specifically, the user may initiate a translation request to the server computing system 9130 through the user computing device 9110 and specify the source language sentence to be translated. In response to the translation request, the server computing system 9130 calls the second translation model 9144, inputs the source language sentence to be translated into the second translation model 9144, and the model 9144 outputs the translation result of the source language sentence in the target language. Subsequently, the server computing system 9130 may return the translation result to the user computing device 9110.

[0053] In other embodiments of the present invention, the training computing system 9150 may train and generate the second translation model 9144 of the present invention through certain modifications, such as quantizing the model parameter values ​​(i.e., storing the weight w, bias b and other parameter values ​​with a smaller precision, such as an 8-bit integer), and deploying the modified model in the user computing device 9110. Figure 1 As shown, the second translation model deployed in the user computing device 9110 is recorded as the second translation model 9124. Those skilled in the art will appreciate that the second translation model 9144 deployed in the server computing system 9130 has the same function as the second translation model 9124 deployed in the user computing device 9110. The user can call the second translation model 9124 in the user computing device 9110, input the source language sentence to be translated into the model, and the model will output the translation result of the source language sentence in the target language.

[0054] The structures and generation processes of the first translation model and the second translation model are described in detail below.

[0055] Figure 3 FIG. 3 shows a structural diagram of a first translation model 300 according to an embodiment of the present invention. Figure 3 As shown, the first translation model 300 includes an encoding module 310 , an attention module 320 , and a decoding module 330 .

[0056] The encoding module 310 is adapted to encode the source language sentence and generate a latent vector representation of each word in the source language sentence.

[0057] For example, the source language sentence is a word sequence consisting of n words {x1, x2, ..., x n}, the source language sentence is input into the encoding module 310, and the encoding module 310 outputs the latent vector representation of each word in the source language sentence, that is, the output latent vector representation sequence {r1, r2, ..., r n The encoding module 310 can be implemented as a combination of a word vector extraction module such as a GloVe model or a word2vec model and a bidirectional long short-term memory network (Bi-LSTM), for example, but is not limited thereto.

[0058] The attention module 320 is adapted to generate a weight matrix for the source language sentence, wherein the weight matrix includes weights of each word in the source language sentence for each word in the generated translation result.

[0059] For example, the source language sentence is a word sequence consisting of n words {x1, x2, ..., x n}, the translation result is a word sequence consisting of m words The weight matrix A generated by the attention module 320 is as follows:

[0060]

[0061] Among them, a ij Represents the jth word x in the source language sentence j (1≤j≤n) generates the i-th word in the translation result (1≤i≤m), that is, the jth word x in the source language sentence j Translated into the i-th word in the translation result For example, a 2n Represents the nth word x in the source language sentence n The second word in the translation result into the target language probability.

[0062] The decoding module 330 is adapted to decode the latent vector representation output by the encoding module 310 according to the weight matrix output by the attention module 320 to generate a translation result of the source language sentence, wherein the translation result is a word sequence (i.e., a sentence) composed of multiple words in the target language.

[0063] Specifically, the decoding module 330 is adapted to determine the latent vector representation of the translation result according to the latent vector representation sequence of the source language sentence output by the encoding module 310 and the weight matrix output by the attention module 320. Subsequently, the latent vector representation sequence of the translation result is decoded to determine the word in the target language corresponding to each latent vector representation, and generate a word sequence in the target language, that is, to obtain the translation result.

[0064] For example, the encoding module 310 outputs the source language sentence {x1, x2, ..., x n The latent vector representation sequence of} is r = {r1, r2, ..., r n}, the weight matrix A output by the attention module 320 is shown in the above formula (2). The weight matrix A is multiplied by the latent vector representation sequence r of the source language sentence to obtain the latent vector representation sequence {h1, h2, ..., h m}, as shown below:

[0065]

[0066] Then, the latent vector represents the sequence {h1, h2, ..., h m} to decode, determine the word in the target language corresponding to each latent vector representation, and obtain the word sequence of the target language This word sequence is the translation result.

[0067] It should be noted that in Figure 3 In the illustrated embodiment, the attention module 320 and the decoding module 330 are two independent modules. In other embodiments, the attention module 320 may also be implemented as an internal module of the decoding module 330.

[0068] Figure 4 FIG. 4 is a flowchart of a translation method 400 according to an embodiment of the present invention. The method 400 is executed in a computing device (such as the aforementioned server computing device). Figure 4 As shown, the method 400 starts at step S410 .

[0069] In step S410, a first translation model is generated by training using a source language sentence and its corresponding target language sentence as training samples.

[0070] In the embodiment of the present invention, the source language and the target language are different languages. The source language sentence and its corresponding target language sentence have the same semantics. In practice, the source language sentence and its corresponding target language sentence are usually recorded as a pair of parallel sentences.

[0071] Those skilled in the art will appreciate that in order to train and generate the first translation model, a plurality of training samples are required, and the plurality of training samples constitute a training set of the first translation model.

[0072] When training and generating the first translation model, the objective function used may be, for example, a cross entropy loss function, but is not limited thereto.

[0073] After the first translation model is generated through training, step S420 is performed.

[0074] In step S420, the source language sentence is input into a first translation model to obtain a translation result of the source language sentence.

[0075] In step S420, there are multiple source language sentences input into the first translation model, and accordingly, the first translation model will output translation results of the multiple source language sentences respectively. The multiple source language sentences input into the first translation model in step S420 may be all or part of the training samples used to train and generate the first translation model in step S410.

[0076] For example, the training set used to train and generate the first translation model in step S410 includes k training samples. Then, in step S420, the source language sentences in these k training samples can be input into the first translation model, or p (p<k) training samples can be selected from the k training samples and the source language sentences in these p training samples can be input into the first translation model.

[0077] After obtaining the translation results of each source language sentence output by the first translation model in step S420, step S430 is executed.

[0078] In step S430, the mistranslated segments in the translation result and their corresponding mistranslated source segments are determined by comparing the translation result with the corresponding target language sentence.

[0079] According to an embodiment, step S430 further includes: matching the words in the translation result with the words in the corresponding target language sentence, determining the mistranslated words in the translation result, and the set of mistranslated words is the mistranslated segment. Then, using a preset word alignment model, the mistranslated source words corresponding to the mistranslated words are determined from the source language sentence, and the set of mistranslated source words is the mistranslated source segment.

[0080] For example, the translation result of the first translation model outputting the source language sentence x={x1, x2, ..., x6} is The target language sentence corresponding to the source language sentence is y = {y1, y2, ..., y5}. The translation result is converted into The words in the target language sentence y are matched. A word in If there is a Matching words, then the word The translation is correct; if there is no Matching words, then the word Translation error, i.e. is a mistranslated word. The set of all mistranslated words in is called mistranslation segment y T For example, by matching the translation result with the target language sentence, the mistranslated segment can be identified.

[0081] Then, for each mistranslated word in the mistranslated segment, a preset word alignment model is used to determine the mistranslated source word corresponding to the mistranslated word from the source language sentence. The set of mistranslated source words is the mistranslated source segment x M In an embodiment of the present invention, the word alignment model is used to determine the source language word with the same semantics as the target language word. The present invention does not limit the specific structure of the word alignment model, and any model that can achieve the word alignment function is within the protection scope of the present invention. For example, by performing word alignment, the mistranslated word is determined. The corresponding mistranslated source word is x5, the mistranslated word The corresponding mistranslated source words are x3 and x4, then the mistranslated source segment x M ={x3,x4,x5}.

[0082] In step S430, the mistranslated segment y is determined. T and its corresponding mistranslated source segment x M Then, execute step S440.

[0083] In step S440, the source language sentence containing the mistranslated source segment and its corresponding target language sentence are used as training samples, and the first translation model is trained with the reduction of the probability of generating the mistranslated segment as the training goal to generate a second translation model.

[0084] In the embodiment of the present invention, the second translation model has the same structure as the first translation model, but has different training samples and training objective functions.

[0085] The training sets (training sample sets) of the first translation model and the second translation model are respectively recorded as the first training set and the second training set, and the second training set is a subset of the first training set. Specifically, the second training set consists of training samples in the first training set that are mistranslated by the first translation model.

[0086] As mentioned above, the objective function of the first translation model may be, for example, a cross entropy loss function. In addition to taking into account the objective function (fourth objective function) of the original first translation model, the objective function of the second translation model also aims to reduce the probability of generating mistranslated segments, which are the first objective function, the second objective function, and the third objective function set by the encoding module 310, the attention module 320, and the decoding module 330, respectively. That is, the objective function of the second translation model is the weighted sum of the first objective function, the second objective function, the third objective function, and the fourth objective function.

[0087] According to an embodiment, the step of training the first translation model with reducing the probability of generating mistranslated segments as the training goal to generate the second translation model in step S440 includes: setting the first objective function, the second objective function, and the third objective function corresponding to the encoding module, the attention module, and the decoding module respectively with reducing the probability of generating mistranslated segments as the goal; training the first translation model with minimizing the weighted sum of the first objective function, the second objective function, the third objective function, and the fourth objective function as the training goal to generate the second translation model, wherein the fourth objective function is the objective function used when training and generating the first translation model.

[0088] According to one embodiment, the encoding module 310 uses a masked language model training target (eg Figure 5 As shown), the first objective function L corresponding to the mask module 310 E for:

[0089]

[0090] Wherein, N is the number of words included in the mistranslated source segment, x is the source language sentence, and x M is the mistranslated source segment, x i is the i-th mistranslated source word in the mistranslated source segment, x\x M is the correct source segment, which is the set of words in the source language sentence except the mistranslated source segment, P(x i |x\x M ) is in the correct source fragment x\x M Generate mistranslated source word x under the condition i probability.

[0091] According to one embodiment, the probability of generating a mistranslated source word under the condition of a correct source segment is determined according to the following steps: replacing the mistranslated source words in the source language sentence with mask words according to a preset probability to generate a mask sentence; inputting the mask sentence into an encoding module so that the encoding module outputs a latent vector representation of the word at each mistranslated source word position; and determining the probability that the latent vector representation is the corresponding mistranslated source word based on the word vector table of the source language.

[0092] For example, in the source language sentence x = {x1, x2, ..., x6}, the mistranslated source segment is x M ={x3, x4, x5}, that is, x3, x4, x5 are mistranslated source words. According to the preset probability p, the mistranslated source words are replaced with mask words M to generate a mask sentence. For example, for the mistranslated source word x3, a random number p1 is generated. If the random number p1 is less than the preset probability p, the mistranslated source word x3 is replaced with the mask word M. Similarly, for the mistranslated source word x4, a random number p2 is generated. If the random number p2 is greater than the preset probability p, the mistranslated source word x4 is not replaced with the mask word M, but x4 is kept unchanged. For the mistranslated source word x5, a random number p3 is generated. If the random number p3 is less than the preset probability p, the mistranslated source word x5 is replaced with the mask word M. After completing the replacement of the mask words, the mask sentence generated is x'={x1, x2, M, x4, M, x6}, such as Figure 5 Shown in the lower right corner.

[0093] It should be noted that the value of the preset probability p can be set by those skilled in the art, and the present invention does not limit this. In one embodiment, for example, the preset probability can be set to 0.8. In addition, the mask word M can be any character or string, and the present invention does not limit the specific value of the mask word.

[0094] After the masked sentence x' is generated, the masked sentence x' is input into the encoding module 310, and the encoding module outputs the latent vector representation of the word at each mistranslated source word position. Referring to the above description, after the mistranslated source word is replaced by the masked word, the word at the mistranslated source word position may be the mistranslated source word itself (for example, Figure 5 As shown in the lower right corner, the word at the position of the mistranslated source word x4 is still x4 itself), or it may be a mask word (for example, Figure 5 As shown in the lower right corner, the words at the positions of the mistranslated source words x3 and x5 are mask words M). After the masked sentence x' is input into the encoding module 310, the encoding module will output the latent vector representation r3, r4, r5 of the words at the positions of each mistranslated source word.

[0095] Then, based on the word vector table of the source language, determine the probability that the latent vector represents the corresponding mistranslated source word. For example, the latent vector is represented as a column vector, and the word vector table of the source language is a matrix composed of vectors of all words in the source language, and each column in the matrix corresponds to a word in the source language. Multiplying the word vector table with the latent vector representation will result in a column vector with the same dimension as the number of words included in the source language, and each row in the column vector corresponds to a word in the source language. The element values ​​in the column vector are normalized, and the element value corresponding to the position of the corresponding mistranslated source word is the probability that the latent vector represents the corresponding mistranslated source word.

[0096] For example, Figure 5The hidden vector r3 in the representation is a column vector. Multiply the word vector table of the source language by r3 to obtain a column vector r'. The dimension of r' is the same as the number of words in the source language. Each row in r' corresponds to a word in the source language. Then, the element values ​​in the column vector r' are normalized. After normalization, the element value corresponding to word x3 is the probability P(x3|x\x M ).

[0097] According to one embodiment, the attention module 320 uses a word alignment objective (e.g. Figure 5 As shown), the second objective function L corresponding to the attention module 320 A for:

[0098]

[0099] Among them, I and J are the number of rows and columns of the weight matrix, respectively, and a i,j , b i,j are the element values ​​of the i-th row and j-th column in the current weight matrix and word alignment matrix, respectively. The word alignment matrix is ​​a matrix obtained by correcting the current weight matrix according to the correspondence between the mistranslated source word and the mistranslated word. Those skilled in the art will appreciate that the value of the weight matrix is ​​continuously iteratively changed during the training process of the second translation model, and each iteration of the training process needs to calculate the objective function value according to the above formula (5). Each iteration calculates L A The weight matrix at this time is the current weight matrix.

[0100] According to one embodiment, the word alignment matrix is ​​determined according to the following steps: for each mistranslated word, the weight of the corresponding mistranslated source word in the current weight matrix to the mistranslated word is set to 1, and the weight of other words in the source language sentence to the mistranslated word is set to 0.

[0101] For example, the source language sentence x = {x1, x2, ..., x6} to the translation result The current weight matrix of is:

[0102]

[0103] Referring to the word matching and word alignment results in the aforementioned step S430, the mistranslated word is Mistranslated words The corresponding mistranslated source word is x5, the mistranslated word The corresponding mistranslated source words are x3 and x4. Then in the word alignment matrix B, for the mistranslated word Add the corresponding mistranslated source word x5 to the mistranslated word The weight a 25 Set to 1, other words in the source language sentence to the mistranslated word The weight a21 ~a 24 、a 26 are all set to 0, such as Figure 5 The "Word Alignment Target" on the right is shown. Add the corresponding mistranslated source words x3 and x4 to the mistranslated words The weight a 33 、a 34 Set to 1, the weight a of other words in the source language sentence to the mistranslated word 31 、a 32 、a 35 、a 36 are all set to 0, that is, the word alignment matrix B is:

[0104]

[0105] According to one embodiment, the decoding module 330 uses a maximum marginal probability target (eg Figure 5 As shown), the third objective function L corresponding to the decoding module 330 D for:

[0106]

[0107] Wherein, λ is a preset constant, and its value can be set to 0.2, for example; y is the target language sentence, is the translation result, y T is the mistranslated segment, x is the source language sentence, y j , are the target language sentence and the jth word in the translation result, respectively, P(y j |y1,y2,…,y j-1 ,x) means generating the jth word y under the condition of the first (j-1) words of the target language sentence and the source language sentence x j The probability of Indicates generating the jth word based on the first (j-1) words of the translation result and the source language sentence x The probability P(y j |y1,y2,…,y j-1 ,x)、 You can refer to the aforementioned probability P(x i |x\x M ) is calculated based on the word embedding table of the target language, and the calculation process of the two will not be described in detail here.

[0108] According to an embodiment, the fourth objective function used in training and generating the first translation model is a cross entropy loss function L T .

[0109] Based on the first objective function L corresponding to the encoding module 310 E , the second objective function L corresponding to the attention module A , the third objective function L corresponding to the decoding module D And the fourth objective function L used when training and generating the first translation model T , the overall objective function L when training and generating the second translation model is the weighted sum of the above four objective functions, namely:

[0110] L=w1*L E +w2*L A +w3*L D +w4*L T (9)

[0111] Wherein, w1-w4 are weights of the first objective function to the fourth objective function respectively. The values ​​of w1-w4 can be set by those skilled in the art, and the present invention does not limit this. In one embodiment, the values ​​of w1-w4 can be set to 1, that is, L=L E +L A +L D +L T .

[0112] Based on the objective function L of the second translation model set as above, in the training process of the second translation model, the encoding module 310, the attention module 320 and the decoding module 330 are trained using different learning tasks using mistranslated segments, so that each module can strengthen the modeling of the mistranslated segments, thereby improving the accuracy of the translation. At the same time, the objective function of the second translation model also takes into account the objective function of the original first translation model (i.e., the fourth objective function), so that the second translation model can still maintain the fluency of the translation result.

[0113] The translation method of the present invention is executed in a computing device, which may be implemented as the aforementioned server computing device, for example.

[0114] Figure 6 FIG. 6 is a schematic diagram of a computing device 600 according to an embodiment of the present invention. Figure 6 As shown, the computing device 600 includes at least one processor 610 and a memory 620 storing program instructions. The memory 620 stores a translation device 700, and the device 700 includes program instructions for executing the translation method 400 of the present invention. When the program instructions are read and executed by the processor 610, the computing device 600 executes the translation method 400 of the present invention.

[0115] Figure 7 FIG. 8 is a schematic diagram of a translation device 700 according to an embodiment of the present invention. Figure 7As shown, the apparatus 700 includes a first training module 710 , a checking module 720 , and a second training module 730 .

[0116] The first training module 710 is suitable for training and generating a first translation model using a source language sentence and its corresponding target language sentence as training samples. The specific functions and processing logic of the first training module 710 can refer to the relevant description of step S410 above, which will not be repeated here.

[0117] The checking module 720 is adapted to input the source language sentence into the first translation model to obtain the translation result of the source language sentence; and to determine the mistranslated segment in the translation result and its corresponding mistranslated source segment by comparing the translation result with the corresponding target language sentence. The specific functions and processing logic of the checking module 720 can refer to the relevant description of steps S420 and S430 above, which will not be repeated here.

[0118] The second training module 730 is adapted to train the first translation model to generate a second translation model using source language sentences containing mistranslated source segments and their corresponding target language sentences as training samples and reducing the probability of generating mistranslated segments as a training goal. The specific functions and processing logic of the second training module 730 can refer to the relevant description of step S440 above, which will not be repeated here.

[0119] The present invention also provides a readable storage medium storing program instructions. When the program instructions are read and executed by a computing device, the computing device executes the translation method 400 of the present invention.

[0120] The various techniques described herein may be implemented in combination with hardware or software, or a combination thereof. Thus, the method and apparatus of the present invention, or some aspects or parts of the method and apparatus of the present invention may be in the form of program codes (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, a USB flash drive, a floppy disk, a CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into a machine such as a computer and executed by the machine, the machine becomes a device for practicing the present invention.

[0121] In the case where the program code is executed on a programmable computer, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store the program code; the processor is configured to execute the translation method of the present invention according to the instructions in the program code stored in the memory.

[0122] By way of example and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer readable instructions, data structures, program modules or other data. Communication media generally embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and include any information transfer medium. Any combination of the above is also included within the scope of readable media.

[0123] In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with examples of the present invention. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to implement the content of the present invention described herein, and the above description of specific languages ​​is for disclosing preferred embodiments of the present invention.

[0124] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0125] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed invention requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0126] Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in the devices described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or may be divided into multiple submodules.

[0127] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0128] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0129] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Therefore, a processor with necessary instructions for implementing the method or method elements forms a device for implementing the method or method elements. In addition, the elements described herein of the device embodiments are examples of devices for implementing the functions performed by the elements for the purpose of implementing the invention.

[0130] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved, and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner.

[0131] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is selected primarily for readability and teaching purposes, rather than for explaining or defining the subject matter of the present invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is illustrative and not restrictive with respect to the scope of the present invention, and the scope of the present invention is defined by the appended claims.

Claims

1. A translation method, comprising: Using source language sentences and their corresponding target language sentences as training samples, training and generating a first translation model; Inputting the source language sentence into the first translation model to obtain a translation result of the source language sentence; By comparing the translation result with the corresponding target language sentence, the mistranslated segment in the translation result and its corresponding mistranslated source segment are determined; Taking source language sentences containing mistranslated source segments and their corresponding target language sentences as training samples and taking reducing the probability of generating mistranslated segments as a training goal, the first translation model is trained to generate a second translation model, wherein a second training set corresponding to the second translation model is a subset of the first training set corresponding to the first translation model, and the second training set consists of training samples in the first training set that are mistranslated by the first translation model; The first translation model includes an encoding module, an attention module and a decoding module. The first translation model is trained with the training goal of reducing the probability of generating mistranslated segments to generate a second translation model, and the steps include: Determine a weighted summation result based on a first objective function corresponding to the encoding module, a second objective function corresponding to the attention module, a third objective function corresponding to the decoding module, and a fourth objective function used when training and generating the first translation model; The first translation model is trained with minimizing the weighted sum result as a training goal to generate the second translation model.

2. The method of claim 1, wherein: The step of determining the mistranslated segment in the translation result and its corresponding mistranslated source segment by comparing the translation result with the corresponding target language sentence comprises: Matching the words in the translation result with the words in the corresponding target language sentence to determine the mistranslated words in the translation result, wherein the set of mistranslated words is the mistranslated segment; A preset word alignment model is adopted to determine mistranslated source words corresponding to the mistranslated words from the source language sentence, and a set of mistranslated source words is the mistranslated source segment.

3. The method according to claim 1 or 2, wherein: The encoding module is adapted to encode the source language sentence and generate a latent vector representation of each word in the source language sentence; The attention module is adapted to generate a weight matrix for the source language sentence, wherein the weight matrix includes weights of each word in the source language sentence for each word in the generated translation result; The decoding module is adapted to decode the latent vector representation according to the weight matrix to generate a translation result of the source language sentence.

4. The method of claim 1, wherein: The first objective function is: Wherein, N is the number of words included in the mistranslated source segment, x is the source language sentence, and x M is the mistranslated source segment, x i is the i-th mistranslated source word in the mistranslated source segment, x\x M is the correct source segment, which is the set of words in the source language sentence except the mistranslated source segment, P(x i |x\x M ) is in the correct source fragment x\x M Generate mistranslated source word x under the condition i probability.

5. The method of claim 4, wherein: The probability of generating a mistranslated source word given a correct source segment is determined as follows: Replacing mistranslated source words in the source language sentence with mask words according to a preset probability to generate a mask sentence; Inputting the masked sentence into the encoding module so that the encoding module outputs a latent vector representation of the word at each mistranslated source word position; According to the word vector table of the source language, the probability that the latent vector represents the corresponding mistranslated source word is determined.

6. The method of claim 1, wherein: The second objective function is: Among them, I and J are the number of rows and columns of the weight matrix, respectively, and a i,j 、b i,j are the element values ​​of the i-th row and the j-th column in the current weight matrix and the word alignment matrix, respectively. The word alignment matrix is ​​a matrix obtained by correcting the current weight matrix according to the correspondence between the mistranslated source words and the mistranslated words.

7. The method of claim 6, wherein: The word alignment matrix is ​​determined according to the following steps: For each mistranslated word, the weight from the corresponding mistranslated source word in the current weight matrix to the mistranslated word is set to 1, and the weight from other words in the source language sentence to the mistranslated word is set to 0.

8. The method of claim 1, wherein: The third objective function is: Among them, λ is a preset constant, y is the target language sentence, is the translation result, y T is the mistranslated segment, x is the source language sentence, y j , are the target language sentence and the jth word in the translation result, respectively. It means generating the jth word y under the condition of the first (j-1) words of the target language sentence and the source language sentence x j The probability of Indicates generating the jth word based on the first (j-1) words of the translation result and the source language sentence x probability.

9. The method of claim 1, wherein: The fourth objective function is a cross entropy loss function.

10. A translation device, comprising: A first training module is adapted to train and generate a first translation model using a source language sentence and its corresponding target language sentence as training samples; a checking module, adapted to input the source language sentence into the first translation model to obtain a translation result of the source language sentence; as well as By comparing the translation result with the corresponding target language sentence, the mistranslated segment in the translation result and its corresponding mistranslated source segment are determined; a second training module, adapted to train the first translation model with source language sentences containing mistranslated source segments and their corresponding target language sentences as training samples and with reducing the probability of generating mistranslated segments as a training goal, to generate a second translation model, wherein a second training set corresponding to the second translation model is a subset of the first training set corresponding to the first translation model, and the second training set consists of training samples in the first training set that are mistranslated by the first translation model; The first translation model includes an encoding module, an attention module and a decoding module, and the second training module is further used for: Determine a weighted summation result based on a first objective function corresponding to the encoding module, a second objective function corresponding to the attention module, a third objective function corresponding to the decoding module, and a fourth objective function used when training and generating the first translation model; The first translation model is trained with minimizing the weighted sum result as a training goal to generate the second translation model.

11. A computing device comprising: at least one processor and a memory storing program instructions; When the program instructions are read and executed by the processor, the computing device is caused to perform the translation method according to any one of claims 1 to 9.

12. A readable storage medium storing program instructions, when the program instructions are read and executed by a computing device, the computing device executes the translation method according to any one of claims 1 to 9.

Images