Machine translation model training method, machine translation method, apparatus, and computing device

Abstract

Machine translation model training methods, machine translation methods, apparatuses, and computing devices are provided. A training method includes obtaining a first training sample set, each first training sample in the first training sample set including a source-side sample sentence in a first language as a source language and a target-side sample sentence in a second language different from the first language as a target language, obtaining a second training sample set, wherein the second training sample set is generated based on applying different types of prompt labels to at least one of the first training sample set and a public data set, wherein each prompt label is used to annotate an operation performed on a source-side sample sentence of a first training sample to which the prompt label is applied or at least a portion of public data to which the prompt label is applied, and training the machine translation model based on the first training sample set and the second training sample set.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine translation, in particular, to a machine translation model training method, a machine translation method, an apparatus, a computing device and a computer readable storage medium. BACKGROUND

[0002] With the continuous development of computer technology, users can translate through translation software, for example, translating Chinese into English. Generally speaking, after receiving a to-be-translated sentence input by a user (for example, input through an input device or voice input, etc.), the translation software can generate a translation text corresponding to the to-be-translated sentence and return it to the user (for example, in text or voice).

[0003] However, generally speaking, the training of the machine translation model in the current academic and industrial fields is based on the translation parallel sentence pairs in the training sample set, and the knowledge in the translation parallel sentence pairs is learned by the machine translation model itself set in advance, so as to achieve the purpose of model training. Taking Chinese-English machine translation as an example, the translation parallel sentence pairs in the training sample set are as follows: the source sample sentence is "He will do his best in everything and never be careless." and the target sample sentence is "He will do his best in everything and never be careless."

[0004] However, such training sample set constituted by translation parallel sentence pairs has a high requirement for the number of samples, and a large amount of manpower and financial resources are needed to complete the annotation, and other data sets that can optimize the model cannot be used, in addition, a large number of translation parallel sentence pairs need to be annotated again for some novel and very used words. Therefore, the current machine translation model training method has the problems of high sample number requirement and poor translation effect. SUMMARY

[0005] One of the purposes of the present application is to provide a machine translation model training method, which can still maintain good translation effect under the condition of limited number of translation parallel sentence pairs.

[0006] According to a first aspect of the present application, a training method of a machine translation model is provided, which can include: obtaining a first training sample set, each first training sample in the first training sample set including a source text in a first language as a source sample sentence and a translation in a second language different from the first language as a target sample sentence; obtaining a second training sample set, wherein the second training sample set is generated based on applying different types of prompt labels to at least one of the first training sample set and a public data set, wherein each prompt label is used to mark the operation of the source sample sentence of the first training sample to which it is applied or at least a part of the public data to which it is applied; and training the machine translation model based on the first training sample set and the second training sample set.

[0007] According to a second aspect of the present application, a machine translation method is provided, which can include: obtaining a sentence to be translated in a first language; inputting the sentence to be translated into a machine translation model, and outputting a predicted sentence in a second language different from the first language from the machine translation model, wherein the machine translation model is trained according to the training method of the first aspect.

[0008] According to a third aspect of the present application, a training device of a machine translation model is provided, which can include: an obtaining module and a training module, the obtaining module can be used to obtain a first training sample set, each first training sample in the first training sample set including a source text in a first language as a source sample sentence and a translation in a second language different from the first language as a target sample sentence; obtain a second training sample set, wherein the second training sample set is generated based on applying different types of prompt labels to at least one of the first training sample set and a public data set, wherein each prompt label is used to mark the operation of the source sample sentence of the first training sample to which it is applied or at least a part of the public data to which it is applied; and the training module can be used to train the machine translation model based on the first training sample set and the second training sample set.

[0009] According to a fourth aspect of the present application, a machine translation device is provided, which can include: an obtaining module and a translation module, the obtaining module can obtain a sentence to be translated in a first language, and the translation module can input the sentence to be translated into a machine translation model, and output a predicted sentence in a second language different from the first language from the machine translation model, wherein the machine translation model is trained according to the training method of the first aspect.

[0010] According to a fifth aspect of the present application, a computing device is provided, comprising: a processor; and a memory having stored thereon a computer program which, when executed by the processor, causes the processor to perform the method for training a machine translation model according to the first aspect or the method for machine translation according to the second aspect.

[0011] According to a sixth aspect of the present application, a computer readable storage medium is also provided, having stored thereon a computer program which, when executed by a processor, causes the processor to perform the steps of the method for training a machine translation model according to the first aspect or the method for machine translation according to the second aspect.

[0012] According to a seventh aspect of the present application, a computer program product is also provided, comprising a computer program which, when executed by a processor, implements the steps of the method for training a machine translation model according to the first aspect or the method for machine translation according to the second aspect.

[0013] Based on the method for training a machine translation model according to the present application, since the second training sample set is introduced during training, the training sample set is expanded by adding the prompt word label, so that the machine translation model can at least improve the understanding level and translation effect of the sentence to be translated in actual application. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application. It is to be understood that the drawings are only schematic, and that they do not necessarily represent a limiting

[0015] Figure 1 A schematic diagram of a system architecture showing an exemplary application environment of the method and apparatus for training a machine translation model and the method and apparatus for machine translation according to embodiments of the present application is shown.

[0016] Figure 2 A flowchart of the method for training a machine translation model according to embodiments of the present application is shown.

[0017] Figure 3 A schematic diagram of an exemplary way of generating the second training sample set according to embodiments of the present application is shown.

[0018] Figure 4 A schematic diagram of another exemplary way of generating the second training sample set according to embodiments of the present application is shown.

[0019] Figure 5 A schematic diagram of another exemplary way of generating the second training sample set according to embodiments of the present application is shown. Figure 2more details of the training step of the training method of the machine translation model.

[0020] Figure 6 A process diagram of the training step of the training method of the machine translation model is shown. Figure 2 A process diagram of the training step of the training method of the machine translation model is shown.

[0021] Figures 7A-7C Structural diagrams of two example machine translation models are shown.

[0022] Figure 8 A flow diagram of a machine translation method according to an embodiment of the application is shown.

[0023] Figures 9-10 A structural block diagram of a machine translation model training apparatus according to an embodiment of the application is shown.

[0024] Figure 11 A structural block diagram of a machine translation apparatus according to an embodiment of the application is shown.

[0025] Figure 12 A schematic structural block diagram of a computing device 1200 according to an embodiment of the application is shown. DETAILED DESCRIPTION

[0026] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art. The described features, structures, or characteristics can be combined in one or more implementations.

[0027] In the following description, numerous specific details are provided, such as examples of programming, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware- specific details and

[0028] Furthermore, the drawings are not necessarily drawn to scale. Like numbers refer to like elements throughout. Also, like titled or numbered blocks in the Figures can represent like or similar functions or structures unless context dictates otherwise. Some of the blocks in the drawings can be functional blocks that represent functions implemented by software, hardware, or a combination of software and hardware. Some of the blocks in the drawings can be software modules stored on a machine readable medium or computer readable medium, such as on a computer readable storage medium, for execution by a processing element or processor.

[0029] Artificial Intelligence (AI) is the theory, method, technology and application system of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0030] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc. With the research and progress of artificial intelligence technology, artificial intelligence technology is researched and applied in many fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned vehicles, autonomous vehicles, drones, robots, smart medical care, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0031] The scheme provided by the embodiments of the present application relates to machine translation technology of artificial intelligence, which is specifically explained by the following embodiments. In the context of the present application, the mentioned "source sample sentence" or "source sentence" can refer to the sentence input into the training sample of the machine translation model or the sentence to be translated, and the "target sample sentence" or "target sentence" can refer to the expected sentence and the translated text sentence output by the machine translation model. The "translation parallel sentence pair" can refer to a sentence pair composed of a source sample sentence and a corresponding target sample sentence in different languages, which is used for training of the machine translation model.

[0032] As described before, in the current machine translation technology, the machine translation model is usually trained based on the translation parallel sentence pairs in the training sample set, and such training method often has the following shortcomings.

[0033] 1) The number of available translation parallel sentence pairs is limited. For example, even for a large language pair such as Chinese-English, there are only tens of millions of sentence pairs, and for a small language pair, there are only a few tens of thousands or a few hundred thousand sentence pairs. Therefore, for the existing small number of translation parallel sentence pairs, completely relying on the machine translation model itself to model and mine the translation parallel sentence pairs may lead to insufficient model learning and low data utilization efficiency. In addition, the annotation of translation parallel sentence pairs also requires considerable human and financial resources.

[0034] 2) The large-scale monolingual (i.e., single language) data, prior knowledge topology data, and the like available on the Internet cannot be effectively used for training of the machine translation model.

[0035] 3) For the translation of specific words or phrases (such as "shenma" and "he qiliao") that are difficult to solve in specific application scenarios, the machine translation model trained based on the training sample set of translation parallel sentence pairs usually needs to be re-annotated by annotators based on a large number of different context environments for specific words or phrases in order to solve the problem without overfitting the model. Therefore, it is time-consuming and labor-intensive, and there is a risk of degradation of the translation effect of the model.

[0036] Therefore, the training method of the machine translation model proposed in the embodiments of the present application expands the traditional training sample set of translation parallel sentence pairs into a mixed training sample set of different types of N (N is an integer greater than or equal to 2) paradigms with prompt words as labels (for example, the example paradigms (also referred to as types) of prompt word labels can include "prompt sentence pair translation", "prompt word translation in sentence", "prompt word sentence generation", and "prompt word topology knowledge") by introducing a new type of sentence pair paradigm, prompt learning. Training the machine translation model based on the mixed training sample set can still maintain good translation effect under the condition of limited number of translation parallel sentence pairs.

[0037] Figure 1 A schematic diagram of a system architecture showing an exemplary application environment of the machine translation model and the training method and apparatus of the machine translation model, and the machine translation method and apparatus to which the embodiments of the present application can be applied is shown.

[0038] As Figure 1As shown, the system architecture 100 can include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc. The terminal devices 101, 102, 103 can be various electronic devices with display screens, including but not limited to desktop computers, portable computers, smart phones, and tablet computers, etc. It should be understood that Figure 1 The number of terminal devices, networks, and servers in the above description is only illustrative. Depending on the implementation needs, there can be any number of terminal devices, networks, and servers. For example, the server 105 can be a server cluster composed of multiple servers, etc.

[0039] The training method of the machine translation model and the machine translation method provided by the embodiments of the present application are generally executed by the server 105, and accordingly, the training device of the machine translation model and the machine translation device are generally arranged in the server 105. However, it is easy for those skilled in the art to understand that the training method of the machine translation model and the machine translation method provided by the embodiments of the present application can also be executed by the terminal device 101, 102, or 103, and accordingly, the training device of the machine translation model and the machine translation device can be arranged in the terminal device 101, 102, or 103, which is not specially limited in the present exemplary embodiment. For example, in an exemplary embodiment, the server 105 can obtain a first training sample set, each first training sample in the first training sample set including a source text in a first language as a source sample sentence and a translation in a second language different from the first language as a target sample sentence; obtain a second training sample set, wherein the second training sample set is generated based on applying different types of prompt word labels to at least one of the first training sample set and a public data set, wherein each prompt word label is used to mark the operation of the source sample sentence of the first training sample to which it is applied or at least a part of the public data to which it is applied; and train the machine translation model based on the first training sample set and the second training sample set.

[0040] The following will be described in detail in combination with Figures 2-8 The training scheme of the machine translation model of the embodiments of the present application will be described in detail.

[0041] Figure 2 A flowchart of a training method of a machine translation model according to an embodiment of the present application is shown. The training method of the machine translation model can be applied to the above-mentioned server 105, or one or more of the above-mentioned terminal devices 101, 102, 103, which is not specially limited in the present exemplary embodiment.

[0042] As shown Figure 2 in Figure 1, in step S210, a first training sample set is obtained. Each first training sample in the first training sample set includes the original text in a first language as the source-side sample sentence and the translation in a second language different from the first language as the target-side sample sentence.

[0043] Optionally, each training sample (first training sample) in the first training sample set can be a translation parallel sentence pair. For example, taking the first language as Chinese and the second language as English, the translation parallel sentence pair can be: the source-side sample sentence is "He will go all out in everything and never be perfunctory."; the target-side sample sentence is "He will do his best in everything and never be careless."

[0044] In step S220, a second training sample set is obtained. Among them, the second training sample set is generated by applying different types of prompting word tags to at least one of the first training sample set and the public data set. Each prompting word tag is used to label the operation performed on the source-side sample sentence of the first training sample to which it is applied or at least a part of the public data to which it is applied.

[0045] Optionally, when the second training sample set is generated by applying different types of prompting word tags to the first training sample set, by applying different prompting word tags, different-dimensional semantic understanding can be performed on each first training sample in the first training sample set, which is used to train the machine translation model, so as to improve the understanding level of the machine translation model for the sentence to be translated in actual applications.

[0046] On the other hand, the public data set mentioned in this article includes a monolingual data set and a prior knowledge topology data set that can be obtained by the public unrestrictedly or partially restricted. When the second training sample set is generated by applying different types of prompting word tags to the public data set, as mentioned above, a large amount of monolingual data and prior knowledge topology data that can be obtained from external databases (such as Wikipedia, news websites, Weibo, Zhihu, open-source websites of competitions that can provide data, etc.) can be used to generate training samples for training the machine translation model by applying different prompting word tags, which can also improve the understanding level of the machine translation model for the sentence to be translated in actual applications.

[0047] Optionally, the process of generating the second training sample set can be carried out through manual operations, that is, manually adding prompting word tags and generating the corresponding source-side sample sentence and target-side sample sentence at this time, and such accuracy is relatively high. Of course, this can also be done by referring to Figure 1The described server 105 is also executed by one or more of the terminal devices 101, 102, 103, for example, to improve efficiency and reduce labor costs, etc., for example, for the case of generating based on the first training sample set, pre-setting the word mapping manner between the source text and the translation text of each first training sample based on, for example, the grammar rules and keyword identification of the source text and the translation text, respectively, and each type of prompt word label to be added and the association relationship of the prompt word label with the source text and / or the translation text, performing word mapping between the source text and the translation text for each first training sample, and then generating the corresponding source sample sentence and target sample sentence according to each added prompt word label and the corresponding association relationship; for the case of generating based on the public data set, determining each type of prompt word label to be added and the association relationship of the prompt word label with the public data based on, for example, the grammar rules and keyword identification of the language of the public data, and for each public data, generating the corresponding source sample sentence and target sample sentence according to each added prompt word label and the corresponding association relationship. Of course, the server or the terminal device can also use other generation methods to generate the second training sample set, and the present disclosure does not limit this as long as the training sample with the prompt word label information can be generated using the translation parallel sentence pair and the public data set.

[0048] More details of generating the second training sample will be described later with reference to Figures 3-5 will be described in detail.

[0049] In step S230, the machine translation model is trained based on the first training sample set and the second training sample set.

[0050] Optionally, using each first training sample (translation parallel sentence pair) of the first training sample set to train the machine translation model can make it have translation capability from the first language to the second language, and using the second training sample set to train the machine translation model can make it have semantic understanding of different dimensions in the first training sample, have better migration and learning ability, and improve the translation effect in actual application.

[0051] Optionally, the machine translation model can be a model with an encoder-decoder framework. For example, the machine translation model can be a Transformer model, a recurrent neural network machine translation (RNN NMT) model, a Google neural machine translation (GNMT) model (for example, including a multi-layer bidirectional long short-term memory network (LSTM)). The training steps of the machine translation model will be described later with reference to Figure 5 will be described in detail.

[0052] Optionally, in order to encode the training samples used as input to the machine translation model to facilitate the training process, an additional step S220' may be included before training the machine translation model.

[0053] like Figure 2 As shown, in step S220', each training sample in the first training sample set and the second training sample set is preprocessed to obtain training samples composed of word segmentation sequences.

[0054] For example, during training, the encoder can generate a feature vector for each word segment of each training sample, and concatenate the feature vectors of all words in the training sample to obtain a feature matrix, which can easily obtain the corresponding encoding information for subsequent decoding on the decoder side.

[0055] Based on reference Figure 2 The training method for the described machine translation model, by introducing a second training sample set during training and expanding the training sample set by adding prompt word labels, can improve the machine translation model's understanding of sentences to be translated and the translation effect in practical applications.

[0056] Figure 3 A schematic diagram illustrating an example of generating a second training sample set according to an embodiment of this application is shown. This example corresponds to the case where the second training sample set is generated based on applying different types of cue word tags to the first training sample set.

[0057] For example, in the process of generating the second training sample set, prompt word tags can be added to at least a part of the source sample statement of each first training sample in the first training sample set as the source sample statement of each second training sample, and statements can be extracted from the original text or the corresponding translation of the first training sample as the target sample statement of the second training sample.

[0058] In other words, the first training sample set can be extended according to a predefined paradigm, that is, by adding different prompt word labels to each first training sample in the first training sample set, a second training sample set is obtained based on the first training sample set. This second training sample set can be used to train the machine translation model, so that the machine translation model has different dimensions of semantic understanding for each training sample in the first training sample set, thereby improving the understanding level and translation effect of the machine translation model.

[0059] Furthermore, as an example, when adding prompt word tags to each first training sample, a first prompt word tag can be added to the original text of the first training sample (e.g., corresponding to the first type of paradigm); or second prompt word tags can be added separately to each word in at least one word of the original text of the first training sample (e.g., corresponding to the second type of paradigm).

[0060] For example, the first prompt word tag can be added to the front or the end of the original text of the first training sample.

[0061] For example, when adding a first prompt word tag to the original text of each first training sample, the first prompt word tag can be used to label the translation operation of the original text of the first training sample from the first language to the second language. At this time, the first prompt word tag together with the original text serves as the source - end sample sentence of a second training sample, and the entire translation of the first training sample is extracted as the target - end sample sentence of the second training sample. As an example but not a limitation, for example, the first prompt word tag can be "Translate the Chinese sentence into English". In this way, when the source - end sample sentence of the first training sample is "他凡事必悬命,绝不打马虎眼。" and the target - end sample sentence is "He will do his best in everything and never be careless.", the source - end sample sentence with the first prompt word tag serves as the source - end sample sentence of a second training sample, that is, "<Translate the Chinese sentence into English>他凡事必悬命,绝不打马虎眼。", and the target - end sample sentence of the second training sample is still "He will do his best in everything and never be careless.".

[0062] Again, for example, for at least one word in the original text of the first training sample, the second prompt word tag can be added to the front or the end of each word. And for different words, the second prompt word tags can be different (corresponding to different second - type paradigms).

[0063] Optionally, for example, for each word, the second prompt word tag is used to label the translation operation of the word from the first language to the second language. At this time, the second prompt word tag together with the word and the original text jointly serve as the source - end sample sentence of a second training sample, and the translation corresponding to the word is extracted from the translation of the first training sample as the target - end sample sentence of the second training sample.

[0064] By way of example and not limitation, for instance, the second prompt word tag can be "The English translation of the word in the sentence is". Thus, when the source - side sample sentence of the first training sample is "He will do his best in everything and never be careless."; and the target - side sample sentence is "他凡事必悬命,绝不打马虎眼。", for the word "悬命" obtained by segmentation, the word with the second prompt word tag added and the original text can be used as the source - side sample sentence of a second training sample, such as "<句中词的英文翻译是>悬命:他凡事必悬命,绝不打马虎眼。", and the target - side sample sentence of this second training sample is "Do his best"; also, for example, for the word "打马虎眼" obtained by segmentation, the word with the second prompt word tag added can be used as the source - side sample sentence of a second training sample, such as "<句中词的英文翻译是>打马虎眼:他凡事必悬命,绝不打马虎眼。", and the target - side sample sentence of this second training sample is "Be careless".

[0065] Additively or alternatively, for example, for each word, the second prompt word tag can be used to mark sentence - making operations on the word. At this time, the second prompt word tag together with the word can be used as the source - side sample sentence of a second training sample, and at least a part of the original text of the first training sample that includes this word is extracted as the target - side sample sentence of this second training sample.

[0066] By way of example and not limitation, for instance, the second prompt word tag can be "Make a sentence with the word". Thus, for the word "悬命" obtained by segmentation, the word with the second prompt word tag added can be used as the source - side sample sentence of a second training sample, such as "<用词造句>悬命", and the target - side sample sentence of this second training sample is "他凡事必悬命,绝不打马虎眼。".

[0067] Examples of adding prompt word tags are shown in Table 1. Of course, those skilled in the art should be clear that different second training samples can be generated based on each first training sample (translation parallel sentence pair) from different dimensions according to the actual situation.

[0068]

Table 1

[0069]

[0070] The above reference Figure 3This describes the application of different types of prompt word labels to the first training sample set to generate the second training sample set. In this way, the trained machine translation model can fully understand the semantics of the sentence to be translated from different dimensions, adapt to various semantic environments, and improve the understanding level and translation effect of the machine translation model.

[0071] Figure 4 The diagram illustrates another example of generating a second training sample set according to an embodiment of this application. This example corresponds to the case where the second training sample set is generated based on applying different types of cue word labels to a public dataset.

[0072] For example, in generating the second training sample set, it can be generated based on at least one of the monolingual dataset and the prior knowledge topology dataset.

[0073] Optionally, a monolingual dataset may include a dataset consisting of multiple statements in a single language. This single language can be any language that can be obtained from the internet. For example, a monolingual dataset may include Chinese statements, English statements, German statements, and so on.

[0074] Optionally, the prior knowledge topology dataset may include knowledge that is independent of or precedes experience, such as explanations of certain words or phrases and related content (e.g., obtained based on knowledge graphs).

[0075] By generating training samples for machine translation models from at least one of monolingual datasets obtainable from external data sources and prior knowledge topology datasets, machine translation models can be made to have better transfer and learning capabilities, thereby improving their understanding of sentences to be translated in practical applications and thus improving translation results.

[0076] Optionally, for a monolingual dataset, at least one word in each monolingual data in the monolingual dataset can be added with a cue word label as the source sample sentence for each second training sample, and sentences can be extracted from the monolingual data as the target sample sentence for the second training sample.

[0077] For example, for each word in at least one word of each monolingual data, a third cue word label is added to the word as a source sample sentence of a second training sample, wherein the third cue word label is used to annotate the sentence construction operation of the word, and at least a portion of the monolingual data including the word is extracted as the target sample sentence of the second training sample.

[0078] As an example but not limitation, taking monolingual data obtained from the Internet as "The Chinese women's volleyball team is like a broken bamboo, defeated all strong teams and won the championship.", the third prompt word label can be "word making sentence", so that for the word "like a broken bamboo", the word added with the third prompt word label can be used as a source sample sentence of a second training sample, for example, "<word making sentence> like a broken bamboo", and the target sample sentence of the second training sample is "The Chinese women's volleyball team is like a broken bamboo, defeated all strong teams and won the championship.". Similarly, for the word "championship", the word added with the third prompt word label can be used as a source sample sentence of a second training sample, for example, "<word making sentence> championship", and the target sample sentence of the second training sample is "The Chinese women's volleyball team is like a broken bamboo, defeated all strong teams and won the championship.". The example is shown in Table 2 as follows.

[0079] [Table 2]

[0080]

[0081] Similarly, for the priori knowledge topology data set, at least one word of each priori knowledge topology data in the priori knowledge topology data set is added with a prompt word label as a source sample sentence of each second training sample, and a sentence is extracted from the priori knowledge topology data as a target sample sentence of the second training sample.

[0082] For example, for each word of the at least one word of each priori knowledge topology data, a fourth prompt word label is added to the word as a source sample sentence of a second training sample, wherein the fourth prompt word label is used to mark an explanation operation on the word, and at least part of the explanation of the word included in the priori knowledge topology data is extracted as a target sample sentence of the second training sample; or wherein the fourth prompt word label is used to mark a synonym expansion operation on the word, and at least part of the synonym of the word included in the priori knowledge topology data is extracted as a target sample sentence of the second training sample.

[0083] As an example but not limitation, taking the priori knowledge topology data obtained from the Internet as "Shi er po zhu:Shi:Qixi. Po zhu:Split bamboo. It means that the development of the situation like split bamboo, split the top several sections, the following several sections along the knife potential separation. Metaphor step by step victory, no obstruction. Also as "po zhu of the situation", "Shi er po zhu" for example, the fourth prompt word label can be "the word means", so for the word "Shi er po zhu", the fourth prompt word label added to this word can be used as a source sample sentence of a second training sample, for example "< make a sentence with a word >Shi er po zhu", and the target sample sentence of the second training sample is the explanation of "Shi er po zhu" extracted from the priori topology knowledge data, that is "Shi:Qixi. Po zhu:Split bamboo. It means that the development of the situation like split bamboo, split the top several sections, the following several sections along the knife potential separation. Metaphor step by step victory, no obstruction." In addition, the fourth prompt word label can be "the word synonym is", so for the word "Shi er po zhu", the fourth prompt word label added to this word can be used as a source sample sentence of a second training sample, for example "< the word synonym is >Shi er po zhu", and the target sample sentence of the second training sample is the synonym content of "Shi er po zhu" extracted from the priori topology knowledge data, that is "po zhu of the situation" or "Shi er po zhu". The example is shown in Table 3 as follows.

[0084] [Table 3]

[0085]

[0086] Of course, those skilled in the art should understand that different other prompt word labels can be added based on each monolingual data and priori knowledge topology data according to actual conditions, so as to generate other second training sample sets.

[0087] The above reference Figure 4 The example way of applying prompt word labels to the monolingual data set and the priori knowledge topology data set obtained from the external data source to generate the second training sample set is described, which can make the machine translation model have better migration and learning ability, so as to improve the understanding level of the translated sentence in actual application, and thus improve the translation effect.

[0088] In addition, for some application scenarios, if the translation of certain words or phrases is difficult to solve, in the existing machine translation method, annotators often need to re-annotate a large number of specific words, phrases, etc. based on different context environments to solve the problem under the premise that the model effect is not over-fitted, so it is time-consuming and laborious, and there is a risk of decline in the translation effect of the model. Based on the embodiments of the present application, since the machine translation model trained through the above process has good knowledge transfer learning ability, it can solve these difficult words or phrases by annotating a small amount of samples (by adding different prompt word labels) to provide the machine translation model for learning.

[0089] For example, for "Shenma", the following example training samples can be generated:

[0090] A. Source sample sentence: Shenma; target sample sentence: what;

[0091] B. Source sample sentence: <Translate the Chinese sentence into English> Shenma; target sample sentence: what;

[0092] C. Source sample sentence: <Make a sentence with the word> Shenma; target sample sentence: I don't know what this is;

[0093] D. (From monolingual data) Source sample sentence: <Make a sentence with the word> Shenma; target sample sentence: Everything is a cloud;

[0094] E. (From prior knowledge topology data) Source sample sentence: <What does this word mean> Shenma; target sample sentence: Dialect, equal to "what", such as: What did you say? Is this a what thing?

[0095] Therefore, Figure 2 The training method can also include obtaining a third training data set, wherein the third training data set is generated based on applying different types of prompt word labels to the updated sentence data set, and each prompt word label is used to annotate the operation performed on at least a part of the updated sentence data to which it is applied; and training the machine translation model using the third training data set.

[0096] That is, only a small amount of annotation of difficult words or phrases encountered in actual application scenarios is needed to obtain updated training samples, which can be used again to train the machine translation model, realize the transfer learning of the machine translation model, and ensure the efficient maintenance and low-cost advantage of the subsequent translation effect after actual application.

[0097] The following references Figure 5 The training steps of the machine translation model are described in detail. Figure 5 More details of step S230 are shown.

[0098] As shown in FIG. 2, in step S230-1, the machine translation model is constructed and the parameters of the machine translation model are initialized. Figure 5 For example, the machine translation model with an encoder-decoder framework is constructed, and the model parameters of the machine translation model are randomly initialized, i.e., randomly assigned, and then optimized in the training process. Of course, in some embodiments, in order to accelerate the convergence speed of the model, the machine translation model can also be pre-trained, for example, using the obtained partial translation parallel sentence pairs to obtain the initialization parameters.

[0099] In step S230-2, the source sample sentence and the target sample sentence of each of at least a part of the training samples in the first and second training sample sets are input into the machine translation model as inputs of the machine translation model, and a predicted sentence corresponding to the source sample sentence of each training sample is predicted.

[0100] Optionally, when the machine translation model is an encoder-decoder framework model, the vector representation of the source sample sentence of each of the at least a part of the training samples can be encoded by using the encoder of the machine translation model to obtain the encoding information of the source sample sentence, and then the decoder is used to predict a corresponding decoding sentence according to the encoding information and the vector representation of the target sample sentence as the predicted sentence for comparison with the target sample sentence of the training sample.

[0101] In addition, a part of the mixed training sample set composed of the first training sample set and the second training sample set (generated by the method as described above) can be used as a training set, and another part can be used as a test set. In this way, after the machine translation model is trained based on the training set and meets the preset requirements, the performance of the machine translation model can be tested by using the samples in the test set to ensure sufficient training.

[0102] For example, as shown in FIG. 2, the source sample sentences of a plurality of training samples (T1, T2, T3, T4) in the training set are input into the machine translation model, and the predicted sentences output by the machine translation model are compared with the corresponding target sample sentences for adjusting the model parameters of the machine translation model.

[0103] Figure 6 In step S230-3, based on the difference between the predicted sentence corresponding to the source sample sentence of each training sample and the corresponding target sample sentence, the model parameters of the machine translation model are adjusted until the machine translation model meets the preset requirements.

[0104]

[0105] ​​For example, the relevant prediction loss (e.g., cross entropy) can be calculated based on the predicted sentence corresponding to each training sample and the target-side sample sentence, and then the parameters can be updated according to the prediction loss to complete the training process. Of course, the model parameters of the machine translation model can also be adjusted by other methods known in the art, and this application places no restrictions thereon.

[0106] For example, that the machine translation model meets the preset requirements can specifically refer to that the machine translation model converges or the number of updates of the model parameters of the machine translation model during the above training has reached the preset number of times.

[0107] As an example, Figures 7A-7C The structural schematic diagrams of two example machine translation models are shown, both of which have an encoder-decoder framework and are trained based on the training method described above. Of course, those skilled in the art should understand that the structures of other machine translation models are also feasible as long as they can perform translation after being trained.

[0108] As Figure 7A shown, the schematic diagram of a recurrent neural network (RNN) neural machine translation model (NMT) is shown. In the following text, one end for inputting the sentence to be translated is called the source side, and one end for outputting the translation result is called the target side.

[0109] Given a source-side Chinese sentence "This is the meaning of life.", the encoder-decoder framework first generates word vectors for each Chinese word (the source-side sentence sequence can be obtained through a word segmentation software), and then generates a vector representation of the entire Chinese sentence from left to right through a recurrent neural network. Here, "" represents the end-of-sentence symbol. The recurrent neural network used on the source side is called the encoder, and its function is to encode the source-side Chinese sentence into a dense and continuous real-number vector representation. Thereafter, the target side uses another recurrent neural network to reversely decode the vector representation of the source-side Chinese sentence into the target-language English sentence "This is the meaning of life.". The entire decoding process generates words one by one, and when the end-of-sentence symbol "" is generated, the decoding process terminates. The recurrent neural network used on the target side is called the decoder.

[0110] As Figure 7B shown, the schematic diagram of the Transformer model is shown. In it, the left part is the encoder and the right part is the decoder. The structure and working process of the Transformer model are similar to those applied in the art, so a brief introduction to it is given below.

[0111] The encoder can be stacked by N (for example, N is 6) identical encoding layers, Figure 7B Only one layer each in the encoder and the decoder is shown. The schematic connection manner between the layers in the encoder and the decoder is asFigure 7C Each encoding layer in the encoder has two sub-layers as well, shown (for N = 6) in FIG. 2. The first sub-layer is implemented using a multi-head self-attention mechanism, and the second sub-layer is a simple fully-connected feed-forward neural network. A residual connection and layer normalization are used after each sub-layer, so the output of each sub-layer is: LayerNorm(x + Sublayer(x)), where Sublayer(x) denotes the mapping of the output of the previous sub-layer, represented as x, by the current sub-layer.

[0112] As shown in FIG. 1, the input of the source sentence sequence is received at the input side of the encoder, and each word in the source sentence sequence is formed into a corresponding word vector (i.e., a feature vector) using an embedding layer (input embedding) in the encoder, and the position of each word in the source sentence sequence is encoded to form a corresponding position vector. The position vector is added to the word vector so that the word vector has position information, and the first hidden layer vector representation of the source sentence sequence is obtained from each word vector having position information. Figure 7B Figure 7B The first hidden layer vector representation of the source sentence sequence is matrix-transformed to obtain the corresponding query (Q), key (K), and value (V) of the source sentence sequence, and Q, K, and V are input into the first multi-head attention model. After the first multi-head attention model processes Q, K, and V using a multi-head attention mechanism, the second hidden layer vector representation of the source sentence sequence is obtained.

[0113] The second hidden layer vector representation is processed using residual connection and layer normalization, for example, the first hidden layer vector representation and the second hidden layer vector are added and normalized to optimize the model.

[0114] The second hidden layer vector representation processed using residual connection and layer normalization is non-linearly transformed using a feed-forward neural network to obtain the third hidden layer vector representation.

[0115] The third hidden layer vector representation is processed using residual connection and layer normalization, for example, the second hidden layer vector representation and the third hidden layer vector representation are added and normalized to further optimize the model.

[0116] The third hidden layer vector processed using residual connection and layer normalization is matrix-transformed to obtain the processed K and V values of the source sentence sequence after the above processing, which can be used to provide to the decoder.

[0117] The third hidden layer vector processed using residual connection and layer normalization is matrix-transformed to obtain the processed K and V values of the source sentence sequence after the above processing, which can be used to provide to the decoder.

[0118] ​The decoder is also stacked by N (e.g., N is 6) identical decoding layers, where each layer is composed of 3 sub-layers. Because the decoding process is sequential, i.e., the i-th word is decoded before the i+1-th word, the masking operation at the first decoding layer prevents the knowledge of the i+1-th word and its following information when decoding the i-th word. The input of the first decoding layer is the vector representation of the target sentence sequence with a start symbol added at the beginning (the vector representation of the words after the current decoding word needs to be masked), where the vector representation is obtained in a similar way as the encoder and also embeds the position information. The first sub-layer of each decoding layer is implemented by multi-head self-attention mechanism, similar to the first sub-layer in the encoder. The second sub-layer of each decoding layer is also implemented by multi-head self-attention mechanism, but needs to be calculated based on the encoding information matrix of the encoder output (e.g., based on the matrix to obtain the required K value and V value in the self-attention mechanism) and the output vector representation of the previous decoding layer (based on which to obtain the required Q value, if it is the first decoding layer, the Q value is calculated based on the vector representation of the target sentence sequence). The third sub-layer is a fully connected feedforward neural network, which is the same as the encoder. Similar to the encoder, residual connections and layer normalization are also used on each sub-layer of the decoder.

[0119] The decoder also includes a linear layer and a Softmax layer at the end of the N decoding layers, which are used to calculate the log probability based on the vector representation output by the last layer, and finally predict the word with the maximum output probability.

[0120] Because the encoder and the decoder in the Transformer model contain many similar components, parameter sharing is performed on each component in the encoder and the decoder, so that the parameters in the encoder and the decoder will be updated at the same time during model optimization. Specifically, the parameters in the multi-head self-attention sub-layer and the parameters in the fully connected sub-layer in the encoder and the decoder, as well as the parameters in the residual connection and the layer normalization, are all set to be consistent. However, the parameters of each encoding layer in the encoder are independent and can be different; the parameters of each decoding layer in the decoder are independent and can be different.

[0121] In actual application, the encoder generates a feature encoding matrix based on the sentence to be translated, and the decoder outputs one word at each step and repeats this process until a specific symbol is output to represent the end of a translation. The output of each step is fed into the first decoding layer of the decoder as a new input in the next decoding process, and the position information is added, so as to predict the next word.

[0122] The above is combined with Figures 7A-7CThe two example machine translation models are briefly introduced, but it should be understood that the structure of other machine translation models is also feasible as long as they are trained to be able to translate, and the present application does not limit this.

[0123] By the above reference Figures 2-7C The training method of the machine translation model described above can directly and explicitly enhance the semantic understanding of the machine translation model for different dimensions of the data of the training samples in the first training sample set composed of translation parallel sentence pairs by generating a second training sample set (for example, adding prompt word labels to at least part of it), thereby improving the understanding level of the machine translation model for the sentence to be translated in actual application, to improve the translation effect. Further, because the existing super large scale monolingual data, prior knowledge topology data, etc. on the Internet can be used to constitute similar training sample sets, the migration and learning ability of the machine translation model can be enhanced, thereby further improving the understanding level of the machine translation model for the sentence to be translated in actual application, to improve the translation effect. Further, since the machine translation model already has a certain knowledge migration learning ability, when encountering difficult to solve word, phrase translation problems in specific application scenarios, it is not necessary to rely on artificial targeted annotation work of a large number of bilingual sentence pairs (hundreds of samples), but only a small number (several) of samples with special prompt word labels need to be provided to the model for migration learning.

[0124] According to another aspect of the present application, a machine translation method is also provided.

[0125] Figure 8 A flowchart of a machine translation method according to an embodiment of the present application is shown. The machine translation method can also be applied to the server 105 described above, and can also be applied to one or more of the terminal devices 101, 102, 103, which are not specially limited in the present exemplary embodiment.

[0126] As Figure 8 shown, in step S810, a sentence to be translated in a first language is obtained.

[0127] In step S820, the sentence to be translated is input into a machine translation model, and a predicted sentence in a second language different from the first language is output from the machine translation model, wherein the machine translation model is trained according to the training method described above with reference to Figures 2-7C

[0128] Further details of training the machine translation model have been described in detail in the foregoing, and therefore will not be repeated here.

[0129] Therefore, by Figure 8 ​Such a machine translation method, based on the benefits of the training method of the machine translation model described in the foregoing, can have a higher translation effect.

[0130] According to another aspect of the present application, a training apparatus of a machine translation model is also provided.

[0131] Figure 9 A structural block diagram of a training apparatus of a machine translation model according to an embodiment of the present application is shown. The training apparatus of the machine translation model can be provided in a server 105, or in one or more of the terminal devices 101, 102 or 103, for which no special limitation is made in the present exemplary embodiment.

[0132] As shown in Figure 9 The training apparatus 900 includes an acquisition module 910 and a training module 920.

[0133] The acquisition module 910 can be configured to acquire a first training sample set, each first training sample in the first training sample set including a source text in a first language as a source sample sentence and a translation in a second language different from the first language as a target sample sentence; and acquire a second training sample set, wherein the second training sample set is generated based on applying different types of prompt word labels to at least one of the first training sample set and a public data set, wherein each prompt word label is used to mark the operation of the source sample sentence of the first training sample to which it is applied or at least a part of the public data to which it is applied.

[0134] Optionally, in the case where the second training sample set is generated based on applying different types of prompt word labels to the first training sample set, a prompt word label can be added to at least a part of the source sample sentence of each first training sample in the first training sample set as the source sample sentence of each second training sample, and a sentence is extracted from the source text or the corresponding translation of the first training sample as the target sample sentence of the second training sample.

[0135] Optionally, in the case where the second training sample set is generated based on applying different types of prompt word labels to the public data set, a prompt word label can be added to at least one word in each monolingual data in the monolingual data set as the source sample sentence of each second training sample, and a sentence is extracted from the monolingual data as the target sample sentence of the second training sample.

[0136] Optionally, in a case that the second training sample set is generated based on applying different types of prompt word labels to the public data set, a prompt word label can be added to at least one word of each priori knowledge topology data in the priori knowledge topology data set as the source side sample sentence of each second training sample, and a sentence is extracted from the priori knowledge topology data as the target side sample sentence of the second training sample.

[0137] The training module 920 can be configured to train the machine translation model based on the first training sample set and the second training sample set.

[0138] Optionally, as shown in Figure 10 the second training sample set can be generated by the training apparatus 900, so the training apparatus 900 can further optionally comprise a generating module 930 configured to generate the second training sample set based on applying different types of prompt word labels to at least one of the first training sample set and the public data set, wherein each prompt word label is used to mark the operation of the source side sample sentence of the first training sample to which it is applied or at least a part of the public data to which it is applied.

[0139] In addition, the training apparatus 900 can further optionally comprise a preprocessing module 940 configured to preprocess each training sample in the obtained first training sample set and the second training sample set before training the machine translation model, to obtain a training sample composed of a word segmentation sequence.

[0140] It should be noted that although the above modules are shown by way of example in Figures 9-10 , it should be understood that the training apparatus 900 can also be divided into more or fewer modules according to different functions, or each module can be further divided into sub-modules. In some example embodiments, the modules or sub-modules can be implemented by electronic hardware (for example, general purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, etc.), computer software (for example, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable ROM (EPROM) and the like) or a combination of both.

[0141] The above reference is made to Figures 9-10The training apparatus of the machine translation model can directly and explicitly enhance the semantic understanding of the machine translation model for different dimensions of data of the training samples in the first training sample set constituted by the translation parallel sentence pairs by generating a second training sample set (for example, adding prompt word labels to at least part of the second training sample set) based on the first training sample set constituted by the translation parallel sentence pairs, so that the understanding level of the machine translation model for the sentence to be translated in actual application can be improved to improve the translation effect. Further, because the similar training sample set can be constituted by using the existing super large scale monolingual data, prior knowledge topology data and the like on the Internet, the migration and learning ability of the machine translation model can be strengthened, so that the understanding level of the machine translation model for the sentence to be translated in actual application can be further improved to improve the translation effect. Further, because the machine translation model already has a certain knowledge migration and learning ability, when encountering a translation problem of a word or a word group that is difficult to solve in a specific application scenario, a large amount of bilingual sentence pair (hundreds of samples) annotation work does not need to be performed manually and specifically, and only a small amount (several) of samples with special prompt word labels need to be provided to the model for migration and learning.

[0142] According to another aspect of the present application, a machine translation apparatus is provided.

[0143] Figure 11 A structural block diagram of a machine translation apparatus according to an embodiment of the present application is shown. The machine translation apparatus can be arranged in a server 103, or arranged in one or more of the terminal devices 101, 102 or 103, which is not specially limited in the present exemplary embodiment.

[0144] As shown in Figure 11 , the machine translation apparatus 1100 includes an acquisition module 1110 and a translation module 1120.

[0145] The acquisition module 1110 can acquire a sentence to be translated in a first language.

[0146] The translation module 1120 inputs the sentence to be translated into a machine translation model, and outputs a predicted sentence in a second language different from the first language from the machine translation model, wherein the machine translation model is trained according to the training method as previously described with reference to Figures 2-7C .

[0147] Further details of training the machine translation model have been described in detail in the foregoing, and thus will not be repeated here.

[0148] Therefore, through Figure 11 such a machine translation apparatus, based on the benefits of the training apparatus of the machine translation model as previously described, the machine translation apparatus can have a higher translation effect.

[0149] Figure 12 A schematic structural block diagram of the computing device 1200 according to an embodiment of the present application is shown.

[0150] As shown in Figure 12 the computing device 1200 includes one or more processors, one or more memories, a network interface, an input device, and a display screen connected by a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the terminal stores an operating system, and can also store a computer executable program which, when executed by the processor, can enable the processor to implement various operations described in the steps of the machine translation model training method as described above. The internal memory can also store a computer executable program which, when executed by the processor, can enable the processor to perform various operations described in the steps of the machine translation model training method and the machine translation method.

[0151] For example, the machine translation model training method according to an embodiment of the present application can include: obtaining a first training sample set, each first training sample in the first training sample set including a source sample sentence in a first language and a target sample sentence in a second language different from the first language; obtaining a second training sample set, wherein the second training sample set is generated based on applying different types of prompt word labels to at least one of the first training sample set and a public data set, wherein each prompt word label is used to mark the operation of the source sample sentence of the first training sample to which it is applied or at least a part of the public data to which it is applied; and training the machine translation model based on the first training sample set and the second training sample set. More details of each step have been described in detail in the foregoing, and therefore will not be repeated here.

[0152] For example, the machine translation method according to an embodiment of the present application can include: obtaining a sentence to be translated; inputting the sentence to be translated into a machine translation model, and outputting a target translation sentence from the machine translation model, wherein the machine translation model is trained according to the training method as described above with reference to Figures 2-7C More details of each step have been described in detail in the foregoing, and therefore will not be repeated here.

[0153] The processor can be an integrated circuit chip with a processing capability. The processor can be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The methods, steps and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can be an X84 architecture or an ARM architecture.

[0154] The non-volatile memory can be a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM) or a flash memory. It should be noted that the memory of the method described in the present application is intended to include but not limited to these and any other suitable category of memory.

[0155] The display screen of the computing device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computing device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the terminal shell, or an external keyboard, touchpad or mouse, etc.

[0156] The computing device can be a terminal or a server. The terminal can include but is not limited to a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart television, etc. Various kinds of applications (APPs) can be run in the terminal, such as a multimedia playing application, a social application, a browser application, an information flow application, an education application, etc. The terminal can be a terminal device 101, 102 or 103 described in the description. Figure 1 The server can be a server 105 described in the description, which can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and basic cloud computing services such as big data and artificial intelligence platform. Figure 1 The server can be a server 105 described in the description, which can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and basic cloud computing services such as big data and artificial intelligence platform.

[0157] According to another aspect of the present application, there is also provided a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method for training a machine translation model and the method for machine translation as described above. The computer-readable storage medium can include a hard disk or a memory of the aforementioned computing device, or an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computing device. The computer-readable storage medium can also include a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc. Further, the computer-readable storage medium can include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0158] According to yet another aspect of the present application, there is also provided a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method for training a machine translation model and the method for machine translation as described above.

[0159] It should be noted that the flowchart and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of various embodiments of the present application. In this regard, each block in the flowchart and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or combinations of hardware and software.

[0160] The example embodiments of the present application described in detail above are merely illustrative, and not restrictive. Those skilled in the art should understand that various modifications and combinations of these embodiments or features thereof can be made without departing from the principles and spirit of the present application, and such modifications should fall within the scope of the present application.

Claims

1. A method for training a machine translation model, comprising: obtaining a first training sample set, each first training sample in the first training sample set comprising a source-side sentence in a first language as a source text and a target-side sentence in a second language different from the first language as a translation text; obtaining a second training sample set, wherein the second training sample set is generated based on applying different types of prompt labels to at least one of the first training sample set and a common data set, wherein the common data set comprises at least one of a monolingual data set and a priori knowledge topology data set, each prompt label is used to annotate an operation performed on at least one of a source-side sentence of a first training sample to which the prompt label is applied or at least a portion of common data to which the prompt label is applied, the operation comprising one or more of a word translation operation in a prompt sentence, a sentence construction operation from a word in a prompt sentence, a prompt word sentence construction operation, a prompt word explanation operation, and a synonym expansion operation, different from a sentence pair translation operation of generating a translation text in the second language from a source text in the first language; and training the machine translation model based on the first training sample set and the second training sample set.

2. The training method of claim 1, wherein, in a case where the second training sample set is generated based on applying different types of prompt labels to the first training sample set, adding a prompt label to at least a portion of a source-side sentence of each first training sample in the first training sample set as a source-side sentence of each second training sample, and extracting a sentence from the source text or a corresponding translation text of the first training sample as a target-side sentence of the second training sample.

3. The method of claim 2, wherein, adding a first prompt label to the source text of each first training sample as a source-side sentence of one second training sample, wherein the first prompt label is used to annotate a translation operation from the first language to the second language performed on the source text of the first training sample, and the translation text of the first training sample as a whole is extracted as a target-side sentence of the second training sample; or for each of at least one word in the source text of each first training sample, adding a second prompt label to the word, and adding the second prompt label together with the word and the source text as a source-side sentence of one second training sample, wherein the second prompt label is used to annotate a translation operation from the first language to the second language performed on the word, and a translation text corresponding to the word in the translation text of the first training sample is extracted as a target-side sentence of the one second training sample, or for each of at least one word in the source text of each first training sample, adding a second prompt label to the word as a source-side sentence of one second training sample, the second prompt label is used to annotate a sentence construction operation performed on the word, and at least a portion of the source text including the word is extracted as a target-side sentence of the one second training sample. ​ 4. The training method of claim 1, wherein, The public dataset includes at least one of a monolingual dataset and a priori knowledge topology dataset, in a case where the second training sample set is generated based on applying different types of prompt word labels to the public dataset, adding a prompt word label to at least one word in each monolingual data in the monolingual dataset as a source sample sentence of each second training sample, and extracting a sentence from the monolingual data as a target sample sentence of the second training sample.

5. The training method of claim 4, wherein, For each word in at least one word of each monolingual data, a third prompt word label is added to the word as a source sample sentence of a second training sample, wherein the third prompt word label is used to mark a sentence construction operation on the word, and at least part of the monolingual data including the word is extracted as a target sample sentence of the second training sample.

6. The training method of claim 4, wherein, In a case where the second training sample set is generated based on applying different types of prompt word labels to the public dataset, adding a prompt word label to at least one word in each priori knowledge topology data in the priori knowledge topology dataset as a source sample sentence of each second training sample, and extracting a sentence from the priori knowledge topology data as a target sample sentence of the second training sample.

7. The training method of claim 6, wherein, For each word in at least one word of each priori knowledge topology data, a fourth prompt word label is added to the word as a source sample sentence of a second training sample, wherein the fourth prompt word label is used to mark an explanation operation on the word, and at least part of the explanation of the word in the priori knowledge topology data is extracted as a target sample sentence of the second training sample; or wherein the fourth prompt word label is used to mark a synonym expansion operation on the word, and at least part of the synonym of the word in the priori knowledge topology data is extracted as a target sample sentence of the second training sample.

8. The training method of claim 1, wherein, training the machine translation model based on the first training sample set and the second training sample set, comprising: building the machine translation model and initializing model parameters of the machine translation model; taking the source sample sentence and the target sample sentence of each of at least part of the training samples in the first and second training sample sets as input of the machine translation model, predicting a predicted sentence corresponding to the source sample sentence of each training sample; and adjusting the model parameters of the machine translation model based on the difference between the predicted sentence corresponding to the source sample sentence of each training sample and the corresponding target sample sentence until the machine translation model meets a preset requirement.

9. The training method of claim 8, wherein, The machine translation model is a model of an encoder-decoder framework, wherein taking the source sample sentence and the target sample sentence of each of at least part of the training samples in the first and second training sample sets as input of the machine translation model, predicting a predicted sentence corresponding to the source sample sentence of each training sample, comprises: For each training sample in the at least one part of the training samples, the source sample sentence of the training sample is encoded by using an encoder of the machine translation model, encoding information of the source sample sentence is extracted, and a corresponding decoding sentence is predicted by using a decoder according to the encoding information and a vector representation of the target sample sentence as the predicted sentence for comparison with the target sample sentence of the training sample.

10. The training method of claim 9, wherein, The machine translation model includes a Transformer model, a recurrent neural network machine translation (RNN NMT) model, and a Google neural machine translation (GNMT) model.

11. The training method of claim 1, further comprising: Before training the machine translation model, pre-processing each training sample in the obtained first training sample set and the second training sample set to obtain a training sample composed of a word segmentation sequence.

12. The training method of claim 1, further comprising: obtaining a third training data set, wherein the third training data set is generated based on applying different types of prompt word labels to an update sentence data set, each prompt word label being used to mark an operation performed on at least a part of the update sentence data to which it is applied; and training the machine translation model using the third training data set.

13. A machine translation method, comprising: obtaining a sentence to be translated in a first language; inputting the sentence to be translated into a machine translation model, and outputting a predicted sentence in a second language different from the first language from the machine translation model, wherein the machine translation model is trained according to the training method of any one of claims 1-12.

14. A training apparatus of a machine translation model, comprising: an obtaining module configured to: obtain a first training sample set, each first training sample in the first training sample set including a source sample sentence in a first language as a source sample sentence and a target sample sentence in a second language different from the first language as a target sample sentence; obtain a second training sample set, wherein the second training sample set is generated based on applying different types of prompt word labels to at least one of the first training sample set and a public data set, wherein the public data set includes at least one of a monolingual data set and a priori knowledge topology data set, each prompt word label being used to mark an operation performed on at least a part of the source sample sentence of the first training sample to which it is applied or the public data to which it is applied, the operation including one or more of a prompt word translation operation, a prompt word sentence generation operation, a prompt word sentence generation operation, a prompt word explanation operation, and a synonym expansion operation, which are different from a sentence pair translation operation of generating a translation in the second language from a source text in the first language, a training module configured to train the machine translation model based on the first training sample set and the second training sample set.

15. The training apparatus of claim 14, further comprising: A preprocessing module configured to preprocess each training sample in the obtained first training sample set and the second training sample set before training the machine translation model, to obtain a training sample composed of a word segmentation sequence.

16. A computing device comprising: a processor; and a memory having stored thereon a computer program which, when executed by the processor, causes the processor to perform the method of training a machine translation model according to any one of claims 1-12 or the method of machine translation according to claim 13.

17. A computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the method of training a machine translation model according to any one of claims 1-12 or the method of machine translation according to claim 13.

Images