Method, device and storage medium for determining neural machine translation model
By acquiring and superimposing cross-entropy information from multiple corpus, the neural machine translation model is determined, which solves the problem of poor training effect and improves the translation quality and efficiency of the model.
Patent Information
- Application Number
- CN202210182833.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-25
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2042-02-25
AI Technical Summary
In the prior art, when training neural machine translation models, especially in the financial field, it is difficult to obtain large-scale parallel corpus, resulting in poor model training effects and poor back-translation methods.
By obtaining the target corpus, including parallel corpus, source language monolingual corpus and target language monolingual corpus, the translation task model and dedrying task model are trained separately, multiple cross-entropy information are obtained, and superimposed to determine neural machine translation The model, which then calculates the mask rate and noise rate to adjust the training complexity.
It improves the training effect of neural machine translation models, reduces training complexity, and improves the translation quality under limited resources.
Smart Images
Figure CN114548125B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device and storage medium for determining a neural machine translation model. Background Art
[0002] Machine translation technology is widely used in many fields, such as finance or other fields. NMT models require a large number of data sets (parallel corpora) for training. However, it is difficult to obtain large-scale parallel corpora in some scenarios. For example, the data volume of parallel corpora in the financial field is not large enough to meet the training requirements of text translation models in the financial field. The cost of using manual translation to obtain parallel corpora is relatively high.
[0003] Using monolingual corpora to improve bilingual NMT (neural machine translation) models is an effective method. At present, back translation (BT) technology is one of the most effective methods to train NMT models using monolingual corpora. It uses a translation model that translates the target language into the source language to translate the monolingual corpus of the target language into the source language, thereby generating a pseudo-parallel corpus, which is then used to train the NMT model. However, using the back translation method to train the NMT model is time-consuming, and in languages with fewer corpus resources, this method leads to poor results in training the NMT model.
[0004] Currently, no effective solution has been proposed to address the problem of poor training effect of NMT models in related technologies. Summary of the invention
[0005] The main purpose of this application is to provide a method, device and storage medium for determining a neural machine translation model to solve the problem of poor training effect of NMT models in related technologies.
[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for determining a neural machine translation model is provided. The method includes: obtaining a target corpus, wherein the target corpus includes at least: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus; training the target model respectively through the target corpus to obtain multiple training results, wherein the target model includes: a translation task model, a first denoising task model, and a second denoising task model, wherein the multiple training results include: a first cross entropy information, a second cross entropy information, and a third cross entropy information; superimposing the multiple training results to obtain the target cross entropy information; determining the neural machine translation model according to the target cross entropy information.
[0007] Furthermore, the target model is trained respectively through the target corpus to obtain multiple training results including: obtaining target parallel sentences in the parallel corpus; and training the translation task model using the target parallel sentences to obtain first cross entropy information.
[0008] Furthermore, the target models are trained respectively through the target corpus to obtain multiple training results including: obtaining target words in the source language monolingual corpus; replacing the target words with target tags, wherein the target tags are used to identify the target words; and training the first denoising task model using the target tags to obtain second cross entropy information.
[0009] Furthermore, before the target models are trained separately through the target corpus to obtain multiple training results, the method also includes: obtaining target language sentences in the target language monolingual corpus; deleting or blank marking sampled words in the target language sentences according to preset sampling rules to obtain a first target sentence; and sequentially converting the sampled sentences in the target language sentences according to the preset sampling rules to obtain a second target sentence.
[0010] Furthermore, the target models are trained respectively through the target corpus to obtain multiple training results including: obtaining a first target sentence and a second target sentence; using the first target sentence and the second target sentence to train the second denoising task model to obtain third cross entropy information.
[0011] Furthermore, superimposing the multiple training results to obtain target cross entropy information includes: superimposing the first cross entropy information, the second cross entropy information, and the third cross entropy information to obtain the target cross entropy information.
[0012] Furthermore, after determining the neural machine translation model according to the target cross entropy information, the method also includes: calculating the mask rate in the first denoising task model and the text missing rate of the second denoising task model; determining the target noise rate according to the mask rate and the text missing rate; and determining the training complexity of the first denoising task model and the second denoising task model according to the target noise rate.
[0013] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a device for determining a neural machine translation model is provided. The device includes: a first acquisition unit, used to acquire a target corpus, wherein the target corpus includes at least: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus; a training unit, used to train the target model respectively through the target corpus to obtain multiple training results, wherein the target model includes: a translation task model, a first denoising task model, and a second denoising task model, wherein the multiple training results include: a first cross entropy information, a second cross entropy information, and a third cross entropy information; a superposition unit, used to superimpose multiple training results to obtain target cross entropy information; a first determination unit, used to determine the neural machine translation model according to the target cross entropy information.
[0014] Furthermore, the training unit includes: a first acquisition module, used to acquire target parallel sentences in the parallel corpus; a first training module, used to train the translation task model using the target parallel sentences to obtain first cross entropy information.
[0015] Furthermore, the training unit includes: a second acquisition module, used to acquire a target word in a source language monolingual corpus; a replacement module, used to replace the target word with a target tag, wherein the target tag is used to identify the target word; and a second training module, used to train the first denoising task model using the target tag to obtain second cross entropy information.
[0016] Furthermore, the device also includes: a second acquisition unit, used to acquire target language sentences in the target language monolingual corpus before training the target model respectively through the target corpus to obtain multiple training results; a processing unit, used to delete or blank mark the sampled words in the target language sentence according to the preset sampling rule to obtain the first target sentence; and a conversion unit, used to sequentially convert the sampled sentences in the target language sentence according to the preset sampling rule to obtain the second target sentence.
[0017] Furthermore, the training unit includes: a third acquisition module, used to acquire the first target sentence and the second target sentence; a third training module, used to train the second denoising task model using the first target sentence and the second target sentence to obtain third cross entropy information.
[0018] Furthermore, the superposition unit includes: a superposition module, which is used to superimpose the first cross entropy information, the second cross entropy information and the third cross entropy information to obtain target cross entropy information.
[0019] Furthermore, the device also includes: a calculation unit, which is used to calculate the mask rate in the first denoising task model and the text missing rate of the second denoising task model after determining the neural machine translation model according to the target cross entropy information; a second determination unit, which is used to determine the target noise rate according to the mask rate and the text missing rate; and a third determination unit, which is used to determine the training complexity of the first denoising task model and the second denoising task model according to the target noise rate.
[0020] According to another aspect of an embodiment of the present application, an electronic device is also provided, including one or more processors and a memory, and the computer program / instructions execute any of the above methods when executed by the processor.
[0021] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, any one of the above methods is performed.
[0022] Through this application, the following steps are adopted: obtaining a target corpus, wherein the target corpus includes at least: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus; training the target model through the target corpus to obtain multiple training results, wherein the target model includes: a translation task model, a first denoising task model, and a second denoising task model, wherein the multiple training results include: a first cross entropy information, a second cross entropy information, and a third cross entropy information; superimposing multiple training results to obtain target cross entropy information; determining a neural machine translation model according to the target cross entropy information. The problem of poor training effect of NMT models in related technologies is solved. The target models are trained through the target corpus respectively, and the neural machine translation model is further determined according to the training results, thereby achieving the purpose of improving the effect of training NMT models. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0024] Figure 1 is a flowchart of a method for determining a neural machine translation model provided in an embodiment of the present application;
[0025] Figure 2 is a schematic diagram of an MLM masked language model according to a method for determining a neural machine translation model provided in an embodiment of the present application;
[0026] Figure 3 It is a schematic diagram of a DAE denoising autoencoder according to a method for determining a neural machine translation model provided in an embodiment of the present application;
[0027] Figure 4 is a flowchart of a neural machine translation method provided in an embodiment of the present application;
[0028] Figure 5 is a schematic diagram of a device for determining a neural machine translation model according to an embodiment of the present application;
[0029] Figure 6 It is a schematic diagram of the network architecture of an electronic device for determining a neural machine translation model provided in an embodiment of the present application. DETAILED DESCRIPTION
[0030] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0031] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0033] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:
[0034] Machine translation: It is a technology that uses computers to achieve natural language translation.
[0035] Neural Machine Translation (NMT): A technique that uses a single neural network to maximize machine translation performance.
[0036] Masked Language Model (MLM): An undirected sequence model that can leverage the context of a text to predict missing words in a text, also known as cloze.
[0037] Denoising Autoencoder (DAE): A type of autoencoder that takes corrupted data as input and is trained to predict the original uncorrupted data as input.
[0038] According to the first embodiment of the present application, a method for determining a neural machine translation model is provided.
[0039] Figure 1 is a flow chart of a method for determining a neural machine translation model according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:
[0040] Step S101, obtaining a target corpus, wherein the target corpus at least includes: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus.
[0041] Specifically, in the NMT model, the exploration of multi-task learning is mainly to inject language knowledge through tasks such as part-of-speech tagging, dependency parsing, and semantic parsing. Therefore, it is necessary to make full use of the target corpus to train the model.
[0042] Step S102, training the target model respectively through the target corpus to obtain multiple training results, wherein the target model includes: a translation task model, a first denoising task model, and a second denoising task model, wherein the multiple training results include: first cross entropy information, second cross entropy information, and third cross entropy information.
[0043] Specifically, the multi-task learning framework mainly trains the translation task model, and obtains the first cross entropy information through training the translation task model.
[0044] Optionally, in the method for determining the neural machine translation model provided in an embodiment of the present application, the target model is trained separately through the target corpus to obtain multiple training results including: obtaining target parallel sentences in the parallel corpus; using the target parallel sentences to train the translation task model to obtain first cross-entropy information.
[0045] Specifically, we use the parallel corpus D B Perform training to obtain the target parallel sentences in the parallel corpus. For example, the target parallel sentence pair can be represented by (x, y). The first cross entropy loss L MT As shown in formula (1):
[0046]
[0047] By using target parallel sentences to train the translation task model, the training effect of the subsequent NMT model can be further improved.
[0048] Optionally, in the method for determining the neural machine translation model provided in an embodiment of the present application, the target model is trained respectively through the target corpus to obtain multiple training results including: obtaining target words in the source language monolingual corpus; replacing the target words with target tags, wherein the target tags are used to identify the target words; and using the target tags to train the first denoising task model to obtain second cross entropy information.
[0049] Specifically, in the MLM task model (corresponding to the first denoising task model in the present application), some target words in the source language monolingual corpus sentences are randomly selected and replaced with the [MASK] tag (corresponding to the target tag in the present application) and input into the first denoising task model, so that the model will try to predict the word represented by the [MASK] tag based on the context. The present application introduces the MLM task model as an auxiliary task model to improve the encoder's representation quality of the language, thereby further improving the training effect of the NMT model.
[0050] For example, Figure 2 As shown, an additional output layer is added to the encoder of the transformer model, the encoder is trained using a monolingual corpus, and the trained data is decoded, where the cross entropy loss for predicting the [MASK] tag is given by L MLM By imitating the BERT model, R is randomly sampled in the input sentence. M % of words or subwords, and then replace them with [MASK] tags, the purpose is to generate corpus containing [MASK] tags for training cloze tasks. Among them, the loss function L of the second cross entropy information of MLM MLM As shown in formula (2). Where M is the number of words replaced by [MASK], m i is the i-th word.
[0051]
[0052] Optionally, in the method for determining the neural machine translation model provided in the embodiment of the present application, before the target model is trained separately through the target corpus to obtain multiple training results, the method also includes: obtaining target language sentences in the target language monolingual corpus; deleting or blank marking the sampled words in the target language sentences according to preset sampling rules to obtain a first target sentence; and sequentially converting the sampled sentences in the target language sentences according to the preset sampling rules to obtain a second target sentence.
[0053] Specifically, before the second denoising task model is trained using the target language monolingual corpus, it is necessary to randomly (corresponding to the preset sampling rule in the present application) sample words from each input target language sentence, which are deleted or replaced by blank tokens to obtain the first target sentence, and randomly slightly change the order of words in the input target language sentence to obtain the second target sentence, wherein a random perturbation σ is used, which satisfies the condition n is the length of the input sentence, and k=3 is the maximum exchange distance.
[0054] Optionally, in the method for determining the neural machine translation model provided in an embodiment of the present application, the target model is trained respectively through the target corpus to obtain multiple training results including: obtaining a first target sentence and a second target sentence; using the first target sentence and the second target sentence to train the second denoising task model to obtain third cross entropy information.
[0055] For example, Figure 3 As shown, the monolingual corpus of the target language is used to train the model, the target language sentence is input into the second denoising task model C and encoded by the encoder and decoded by the decoder. The model attempts to restore the original sentence. This application introduces the DAE task model as the training of the denoising autoencoder (DAE), continuously learning from a large-scale monolingual corpus, and further improving the training effect of the NMT model.
[0056] Specifically, the DAE task model (corresponding to the second denoising task model in this application) uses an effective unsupervised NMT method to give a monolingual corpus D M and a random noise model C. The goal of DAE is to minimize the reconstruction loss, and the loss function of the third cross entropy information is L DAE , as shown in formula (3):
[0057]
[0058] Step S103, superimposing multiple training results to obtain target cross entropy information.
[0059] Specifically, in the method for determining the neural machine translation model provided in the embodiment of the present application, multiple training results are superimposed to obtain target cross entropy information, including: superimposing the first cross entropy information, the second cross entropy information, and the third cross entropy information to obtain the target cross entropy information L, as shown in formula (4):
[0060] L=L MT +L MLM +L DAE (4)
[0061] Specifically, by superimposing multiple training results through the above steps, the complexity of training the NMT model is reduced, and the performance of the NMT model can be improved to a certain extent.
[0062] Step S104, determining a neural machine translation model according to the target cross entropy information.
[0063] Specifically, in this application, after determining the neural machine translation model according to the target cross-entropy information, it is necessary to analyze the training difficulty of the MLM task model and the DAE task model when the monolingual corpus remains unchanged. The training difficulty mainly depends on the noise rate, and a dynamic noise rate is used for the denoising target to gradually increase the difficulty of the task.
[0064] Optionally, in the method for determining the neural machine translation model provided in an embodiment of the present application, after determining the neural machine translation model according to the target cross-entropy information, the method also includes: calculating the mask rate in the first denoising task model and the text missing rate of the second denoising task model; determining the target noise rate according to the mask rate and the text missing rate; determining the training complexity of the first denoising task model and the second denoising task model according to the target noise rate.
[0065] Specifically, the present application introduces a dynamic noise rate R(k) as a function of a training step, as shown in formula (5):
[0066]
[0067] Among them, R0 and R m is the minimum and maximum value of the noise rate, and M is the number of times the noise rate increases. The noise rate R depends on the mask rate R in the MLM task M And the text missing rate R in the DAE task D The greater the noise rate, the greater the training complexity of the first denoising task model and the second denoising task model. In the process of training the target model, the training difficulty of the MLM task model and the DAE task model is gradually increased to improve the training effect of the NMT model.
[0068] In summary, the method for determining the neural machine translation model provided in the embodiment of the present application is by obtaining a target corpus, wherein the target corpus includes at least: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus; the target model is trained separately through the target corpus to obtain multiple training results, wherein the target model includes: a translation task model, a first denoising task model, and a second denoising task model, wherein the multiple training results include: a first cross entropy information, a second cross entropy information, and a third cross entropy information; the multiple training results are superimposed to obtain the target cross entropy information; the neural machine translation model is determined according to the target cross entropy information. The problem of poor training effect of the NMT model in the related art is solved. The target model is trained separately through the target corpus, and the neural machine translation model is further determined according to the training results, thereby achieving the purpose of improving the effect of training the NMT model.
[0069] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0070] The embodiment of the present application also provides a device for determining a neural machine translation model. It should be noted that the device for determining a neural machine translation model in the embodiment of the present application can be used to execute the method for determining a neural machine translation model provided in the embodiment of the present application. The following introduces the device for determining a neural machine translation model provided in the embodiment of the present application.
[0071] According to the second embodiment of the present application, an embodiment of a neural machine translation method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0072] Figure 4 is a flowchart of a neural machine translation method provided in an embodiment of the present application, such as Figure 4 As shown, the method includes steps S401-S402, wherein:
[0073] Step S401, obtaining a target sentence to be translated from a target corpus, wherein the target sentence is at least one of the following: a target parallel sentence, a target word, and a target language sentence.
[0074] Step S402: input the target sentence into the neural machine translation model for translation to obtain a translation result corresponding to the target sentence.
[0075] Specifically, the neural machine translation model is determined by the following steps: obtaining a target corpus, wherein the target corpus includes at least: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus; training the target model through the target corpus to obtain multiple training results, wherein the target model includes: a translation task model, a first denoising task model, and a second denoising task model, wherein the multiple training results include: first cross entropy information, second cross entropy information, and third cross entropy information; superimposing the multiple training results to obtain target cross entropy information; determining the neural machine translation model according to the target cross entropy information.
[0076] Figure 5 Schematic diagram of a device for determining a neural machine translation model according to an embodiment of the present application. Figure 5 As shown, the device includes: a first acquisition unit 501, a training unit 502, a superposition unit 503, and a first determination unit 504.
[0077] Specifically, the first acquisition unit 501 is used to acquire a target corpus, wherein the target corpus at least includes: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus;
[0078] A training unit 502 is used to train the target model through the target corpus to obtain multiple training results, wherein the target model includes: a translation task model, a first denoising task model, and a second denoising task model, wherein the multiple training results include: first cross entropy information, second cross entropy information, and third cross entropy information;
[0079] A superposition unit 503 is used to superimpose multiple training results to obtain target cross entropy information;
[0080] The first determining unit 504 is used to determine the neural machine translation model according to the target cross entropy information.
[0081] In summary, the determination device of the neural machine translation model provided in the embodiment of the present application obtains the target corpus through the first acquisition unit 501, wherein the target corpus includes at least: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus; the training unit 502 trains the target model through the target corpus to obtain multiple training results, wherein the target model includes: a translation task model, a first denoising task model, and a second denoising task model, wherein the multiple training results include: first cross entropy information, second cross entropy information, and third cross entropy information; the superposition unit 503 superimposes multiple training results to obtain target cross entropy information; the first determination unit 504 determines the neural machine translation model according to the target cross entropy information, which solves the problem of poor training effect of the NMT model in the related technology. The target models are trained respectively through the target corpus, and the neural machine translation model is further determined according to the training results, thereby achieving the purpose of improving the effect of training the NMT model.
[0082] Optionally, in the determination device of the neural machine translation model provided in an embodiment of the present application, the training unit 502 includes: a first acquisition module, used to obtain target parallel sentences in a parallel corpus; and a first training module, used to train the translation task model using the target parallel sentences to obtain first cross-entropy information.
[0083] Optionally, in the determination device of the neural machine translation model provided in an embodiment of the present application, the training unit 502 includes: a second acquisition module, used to acquire a target word in a source language monolingual corpus; a replacement module, used to replace the target word with a target tag, wherein the target tag is used to identify the target word; and a second training module, used to train the first denoising task model using the target tag to obtain second cross entropy information.
[0084] Optionally, in the device for determining the neural machine translation model provided in the embodiment of the present application, the device also includes: a second acquisition unit, used to acquire target language sentences in the target language monolingual corpus before training the target model separately through the target corpus to obtain multiple training results; a processing unit, used to delete or blank mark the sampled words in the target language sentence according to a preset sampling rule to obtain a first target sentence; and a conversion unit, used to sequentially convert the sampled sentences in the target language sentence according to the preset sampling rule to obtain a second target sentence.
[0085] Optionally, in the determination device of the neural machine translation model provided in an embodiment of the present application, the training unit 502 includes: a third acquisition module, used to obtain the first target sentence and the second target sentence; a third training module, used to train the second denoising task model using the first target sentence and the second target sentence to obtain third cross entropy information.
[0086] Optionally, in the determination device of the neural machine translation model provided in an embodiment of the present application, the superposition unit 503 includes: a superposition module, used to superimpose the first cross entropy information, the second cross entropy information and the third cross entropy information to obtain target cross entropy information.
[0087] Optionally, in the device for determining the neural machine translation model provided in an embodiment of the present application, the device also includes: a calculation unit, used to calculate the mask rate in the first denoising task model and the text missing rate of the second denoising task model after determining the neural machine translation model according to the target cross entropy information; a second determination unit, used to determine the target noise rate according to the mask rate and the text missing rate; and a third determination unit, used to determine the training complexity of the first denoising task model and the second denoising task model according to the target noise rate.
[0088] The determination device of the neural machine translation model includes a processor and a memory. The above-mentioned first acquisition unit 501, training unit 502, superposition unit 503, first determination unit 504, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.
[0089] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the neural machine translation model is determined by adjusting the kernel parameters.
[0090] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.
[0091] An embodiment of the present invention provides a storage medium having a program stored thereon, which implements a method for determining a neural machine translation model when the program is executed by a processor.
[0092] An embodiment of the present invention provides a processor, which is used to run a program, wherein the method for determining a neural machine translation model is executed when the program is running.
[0093] like Figure 6As shown, an embodiment of the present invention provides an electronic device, the device includes a processor, a memory, and a program stored in the memory and executable on the processor, and the processor implements the following steps when executing the program: obtaining a target corpus, wherein the target corpus includes at least: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus; training a target model through the target corpus to obtain multiple training results, wherein the target model includes: a translation task model, a first denoising task model, and a second denoising task model, wherein the multiple training results include: a first cross entropy information, a second cross entropy information, and a third cross entropy information; superimposing the multiple training results to obtain the target cross entropy information; and determining the neural machine translation model according to the target cross entropy information.
[0094] When the processor executes the program, the following steps are also implemented: obtaining target parallel sentences in the parallel corpus; and training the translation task model using the target parallel sentences to obtain first cross entropy information.
[0095] When the processor executes the program, the following steps are also implemented: obtaining a target word in a source language monolingual corpus; replacing the target word with a target tag, wherein the target tag is used to identify the target word; and training the first denoising task model using the target tag to obtain second cross entropy information.
[0096] When the processor executes the program, the following steps are also implemented: before the target model is trained respectively through the target corpus to obtain multiple training results, a target language sentence in the target language monolingual corpus is obtained; the sampled words in the target language sentence are deleted or blank-marked according to the preset sampling rule to obtain a first target sentence; and the sampled sentences in the target language sentence are sequentially converted according to the preset sampling rule to obtain a second target sentence.
[0097] When the processor executes the program, the following steps are also implemented: obtaining a first target sentence and a second target sentence; using the first target sentence and the second target sentence to train a second denoising task model to obtain third cross entropy information.
[0098] When the processor executes the program, the following steps are also implemented: the first cross entropy information, the second cross entropy information and the third cross entropy information are superimposed to obtain target cross entropy information.
[0099] When the processor executes the program, the following steps are also implemented: after determining the neural machine translation model according to the target cross entropy information, calculating the mask rate in the first denoising task model and the text missing rate of the second denoising task model; determining the target noise rate according to the mask rate and the text missing rate; determining the training complexity of the first denoising task model and the second denoising task model according to the target noise rate.
[0100] The devices in this article can be servers, PCs, PADs, mobile phones, etc.
[0101] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: obtaining a target corpus, wherein the target corpus includes at least: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus; training a target model through the target corpus to obtain multiple training results, wherein the target model includes: a translation task model, a first denoising task model, and a second denoising task model, wherein the multiple training results include: first cross entropy information, second cross entropy information, and third cross entropy information; superimposing the multiple training results to obtain target cross entropy information; and determining a neural machine translation model based on the target cross entropy information.
[0102] When executed on a data processing device, it is also suitable for executing a program that initializes the following method steps: obtaining target parallel sentences in a parallel corpus; using the target parallel sentences to train a translation task model to obtain first cross entropy information.
[0103] When executed on a data processing device, it is also suitable for executing a program that is initialized with the following method steps: obtaining a target word in a source language monolingual corpus; replacing the target word with a target tag, wherein the target tag is used to identify the target word; and using the target tag to train a first denoising task model to obtain second cross entropy information.
[0104] When executed on a data processing device, the method is also suitable for executing a program that is initialized with the following method steps: before training the target model through the target corpus to obtain multiple training results, obtaining the target language sentence in the target language monolingual corpus; deleting or blank marking the sampled words in the target language sentence according to the preset sampling rule to obtain the first target sentence; and sequentially converting the sampled sentences in the target language sentence according to the preset sampling rule to obtain the second target sentence.
[0105] When executed on a data processing device, it is also suitable for executing a program that initializes the following method steps: obtaining a first target sentence and a second target sentence; using the first target sentence and the second target sentence to train a second denoising task model to obtain third cross entropy information.
[0106] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: superimposing the first cross entropy information, the second cross entropy information and the third cross entropy information to obtain target cross entropy information.
[0107] When executed on a data processing device, it is also suitable for executing a program that is initialized with the following method steps: after determining a neural machine translation model based on target cross entropy information, calculating the mask rate in the first denoising task model and the text missing rate of the second denoising task model; determining a target noise rate based on the mask rate and the text missing rate; and determining the training complexity of the first denoising task model and the second denoising task model based on the target noise rate.
[0108] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0109] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0110] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0112] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0113] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0114] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0115] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0116] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0117] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.
Claims
1. A method for determining a neural machine translation model, characterized in that: include: Acquire a target corpus, wherein the target corpus includes at least: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus; The target models are trained respectively by the target corpus to obtain a plurality of training results, including: the translation task model is trained by the parallel corpus to obtain first cross entropy information; the first denoising task model is trained by the source language monolingual corpus to obtain second cross entropy information; the second denoising task model is trained by the target language monolingual corpus to obtain third cross entropy information; Superimposing the multiple training results to obtain target cross entropy information; Determining a neural machine translation model according to the target cross entropy information; Wherein, after determining the neural machine translation model according to the target cross entropy information, the method further includes: Calculating the mask rate in the first denoising task model and the text missing rate in the second denoising task model; Acquire a dynamic noise rate, wherein the dynamic noise rate depends on a mask rate in the first denoising task model and a text missing rate in the second denoising task model; determining a target noise rate according to the dynamic noise rate; The training complexity of the first denoising task model and the second denoising task model is determined according to the target noise rate.
2. The method according to claim 1, characterized in that The target model is trained respectively by the target corpus, and multiple training results are obtained, including: Obtaining a target parallel sentence in the parallel corpus; The target parallel sentence is used to train the translation task model to obtain the first cross entropy information.
3. The method according to claim 1, characterized in that The target model is trained respectively by the target corpus, and multiple training results are obtained, including: Obtaining a target word from the source language monolingual corpus; Replacing the target word with a target tag, wherein the target tag is used to identify the target word; The first denoising task model is trained using the target label to obtain the second cross entropy information.
4. The method according to claim 1, characterized in that: Before respectively training the target model using the target corpus to obtain a plurality of training results, the method further includes: Acquire target language sentences from the target language monolingual corpus; Deleting or blanking the sampled words in the target language sentence according to a preset sampling rule to obtain a first target sentence; and The sampled sentences in the target language sentence are sequentially converted according to the preset sampling rule to obtain a second target sentence.
5. The method according to claim 4, characterized in that The target model is trained respectively by the target corpus, and multiple training results are obtained, including: Obtaining the first target sentence and the second target sentence; The second denoising task model is trained using the first target sentence and the second target sentence to obtain the third cross entropy information.
6. The method according to claim 1, characterized in that The target cross entropy information obtained by superimposing the multiple training results includes: The first cross entropy information, the second cross entropy information and the third cross entropy information are superimposed to obtain target cross entropy information.
7. A neural machine translation method, characterized in that: include: Acquire a target sentence to be translated from a target corpus, wherein the target sentence is at least one of the following: a target parallel sentence, a target word, or a target language sentence; The target sentence is input into a neural machine translation model for translation to obtain a translation result corresponding to the target sentence, wherein the neural machine translation model is determined by the following steps: obtaining a target corpus, wherein the target corpus includes at least: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus; training the target model with the target corpus respectively to obtain a plurality of training results, including: training the translation task model with the parallel corpus to obtain first cross entropy information; training the first denoising task model with the source language monolingual corpus to obtain second cross entropy information; training the second denoising task model with the target language monolingual corpus to obtain third cross entropy information; The multiple training results are superimposed to obtain target cross entropy information; the neural machine translation model is determined according to the target cross entropy information; wherein, after the neural machine translation model is determined according to the target cross entropy information, the method further comprises: calculating the mask rate in the first denoising task model and the text missing rate of the second denoising task model; Acquire a dynamic noise rate, wherein the dynamic noise rate depends on a mask rate in the first denoising task model and a text missing rate in the second denoising task model; A target noise rate is determined according to the dynamic noise rate; and a training complexity of the first denoising task model and the second denoising task model is determined according to the target noise rate.
8. A device for determining a neural machine translation model, characterized in that: include: A first acquisition unit is used to acquire a target corpus, wherein the target corpus at least includes: a parallel corpus, a source language monolingual corpus, and a target language monolingual corpus; A training unit is used to train the target model respectively through the target corpus to obtain multiple training results, including: training the translation task model through the parallel corpus to obtain first cross entropy information; training the first denoising task model through the source language monolingual corpus to obtain second cross entropy information; training the second denoising task model through the target language monolingual corpus to obtain third cross entropy information; A superposition unit, used for superimposing the multiple training results to obtain target cross entropy information; A first determining unit, configured to determine a neural machine translation model according to the target cross entropy information; Wherein, the determination device of the neural machine translation model further includes: A calculation unit is used to calculate the mask rate in the first denoising task model and the text missing rate of the second denoising task model after determining the neural machine translation model according to the target cross entropy information; a second acquisition unit is used to acquire a dynamic noise rate, wherein the dynamic noise rate depends on the mask rate in the first denoising task model and the text missing rate of the second denoising task model; a second determination unit is used to determine a target noise rate according to the dynamic noise rate; and a third determination unit is used to determine the training complexity of the first denoising task model and the second denoising task model according to the target noise rate.
9. A computer-readable storage medium, characterized in that: The storage medium includes a stored program, wherein the program executes the method for determining a neural machine translation model according to any one of claims 1 to 7.
10. An electronic device, characterized in that: include: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the neural machine translation model described in any one of claims 1 to 7.