Multi-domain translation model training method, multi-domain translation method and device

Abstract

The present invention provides a multi-domain translation model training method, a multi-domain translation method and an apparatus. The training method includes: determining an initial translation model; based on the initial translation model, determining the target language text of the sample source language text in the first domain; based on the sub-networks of multiple domains in the initial translation model, determining a first similarity between the output feature of the sample source language text in the first domain and the fused output feature in multiple domains, and a second similarity between the output feature of the sample source language text in the second domain and the fused output feature; based on the difference between the target language text and the sample target language text, the first similarity and the second similarity, iteratively updating the parameters of the initial translation model to obtain a trained multi-domain translation model. During the training process, the relationship between the features of the text and the domains can be learned, and the model is enabled to have the translation ability in multiple domains without relying on domain labels.

Description

Technical Field

[0001] The present invention relates to the technical field of machine translation, and in particular, to a method for training a multi-domain translation model, a multi-domain translation method, and an apparatus. Background Art

[0002] Currently, machine translation methods based on neural networks are often limited by the scale of data and perform poorly in resource-scarce domains. Therefore, domain transfer and multi-domain machine translation have received extensive attention.

[0003] In reality, machine translation problems often include two situations:

[0004] (1) General text translation. The input text often does not have an obvious domain tendency. For example, when a user uses Google Translate, no specific domain label is given.

[0005] (2) Professional domain text translation. The input text belongs to a specific domain, such as law, medicine, education, etc. Its application scenarios often lie in the specific needs of related enterprises, and a clear domain label is generally given.

[0006] Current multi-domain machine translation systems either rely too much on prior domain knowledge or do not use and cannot use prior knowledge. Therefore, they can only solve one of the above two situations well and often perform poorly in the other situation. Summary of the Invention

[0007] The present invention provides a method for training a multi-domain translation model, a multi-domain translation method, and an apparatus, which are used to solve the defect that the existing translation model cannot perform general text translation and professional domain text translation.

[0008] The present invention provides a method for training a multi-domain translation model, including:

[0009] Determine an initial translation model;

[0010] Based on the initial translation model, determine the target language text of the sample source language text in the first domain;

[0011] Based on the sub-networks of multiple domains in the initial translation model, determine a first similarity between the output feature of the sample source language text in the first domain and the fused output feature of multiple domains, and a second similarity between the output feature of the sample source language text in the second domain and the fused output feature;

[0012] Iteratively update the parameters of the initial translation model based on the differences between the target language text and the sample target language text, the first similarity, and the second similarity, to obtain a trained multi-domain translation model; the first domain is the domain of the sample source language text, and the second domain is the domain other than the first domain among the multiple domains.

[0013] According to a multi-domain translation model training method provided by the present invention, the initial translation model includes an encoder-decoder layer, and each encoder layer and each decoder layer in the encoder-decoder layer include sub-networks of the multiple domains;

[0014] The determining of the first similarity between the output features of the sample source language text in the first domain and the fused output features in the multiple domains, and the second similarity between the output features of the sample source language text in the second domain and the fused output features, based on the sub-networks of the multiple domains in the initial translation model, includes:

[0015] Based on the sub-networks of the multiple domains in any layer of the encoder-decoder layer, apply the input features of the any layer to determine the output features of the any layer in the first domain and the output features in the second domain, and based on the output features in the first domain and the output features in the second domain, determine the fused output features of the any layer;

[0016] Determine the first similarity based on the similarities between the output features in the first domain and the fused output features of each layer in the encoder-decoder layer; and determine the second similarity based on the similarities between the output features in the second domain and the fused output features of each layer in the encoder-decoder layer.

[0017] According to a multi-domain translation model training method provided by the present invention, each encoder layer and each decoder layer further include a general sub-network, including:

[0018] The determining of the target language text of the sample source language text in the first domain based on the initial translation model includes;

[0019] Based on the embedding layer in the initial translation model, determine the text embedding features of the sample source language text;

[0020] Based on the general sub-network and the sub-network of the first domain in each encoder layer, respectively encode the text embedding features to obtain text general encoding features and text domain encoding features, and based on the text general encoding features and the text domain encoding features, determine text fused encoding features;

[0021] Based on the general sub-network in each decoding layer and the sub-network in the first domain, decode the text fusion encoding features respectively to obtain text general decoding features and text domain decoding features, and determine text fusion decoding features based on the text general decoding features and the text domain decoding features;

[0022] Based on the output layer in the initial translation model, apply the text fusion decoding features to output the target language text.

[0023] According to a multi-domain translation model training method provided by the present invention, the iterative update of the parameters of the initial translation model based on the difference between the target language text and the sample target language text, the first similarity, and the second similarity includes:

[0024] Based on the training loss function, with the minimum difference between the target language text and the sample target language text, and with the maximum first similarity and the minimum second similarity as the goals, iteratively update the parameters of the initial translation model; the training loss function is constructed based on the difference between the target language text and the sample target language text, the first similarity, and the second similarity, and the formula of the training loss function is:

[0025]

[0026]

[0027]

[0028] In the formula, N is the total number of encoding layers and decoding layers in the initial translation model, and i is the i-th layer in the total number of layers N; is the output feature of the sample source language text in the first domain in the i-th layer; is the fusion output feature of the sample source language text in the i-th layer; is the output feature of the sample source language text in the second domain in the i-th layer; L MT is the translation loss, and the translation loss is determined based on the difference between the target language text and the sample target language text; is the first similarity loss; is the second similarity loss; ε is a hyperparameter.

[0029] The present invention also provides a multi-domain translation method, including:

[0030] Determine the source language text to be translated;

[0031] If the domain category of the source language text is not determined, the general sub-networks and the sub-networks of multiple domains in each encoding layer and each decoding layer of the multi-domain translation model trained based on the multi-domain translation model training method described in any of the above items are used to translate the source language text, and the target language text in the domain where the source language text is located is obtained.

[0032] According to a multi-domain translation method provided by the present invention, after determining the source language text to be translated, it further includes:

[0033] If the domain category of the source language text is determined, the general sub-networks in each encoding layer and each decoding layer of the multi-domain translation model and the sub-networks of the domain category are used to translate the source language text, and the target language text is obtained.

[0034] The present invention also provides a multi-domain translation model training device, including:

[0035] A determination module, configured to determine an initial translation model;

[0036] An output module, configured to determine the target language text of the sample source language text in the first domain based on the initial translation model;

[0037] A similarity module, configured to determine a first similarity between the output feature of the sample source language text in the first domain and the fused output feature in multiple domains, and a second similarity between the output feature of the sample source language text in the second domain and the fused output feature, based on the sub-networks of multiple domains in the initial translation model;

[0038] A training module, configured to iteratively update the parameters of the initial translation model based on the difference between the target language text and the sample target language text, the first similarity, and the second similarity, to obtain a trained multi-domain translation model; the first domain is the domain of the sample source language text, and the second domain is the domain other than the first domain among the multiple domains.

[0039] The present invention also provides a multi-domain translation device, including:

[0040] A determination module, configured to determine the source language text to be translated;

[0041] A translation module, configured to, if the domain category of the source language text is not determined, use the general sub-networks and the sub-networks of multiple domains in each encoding layer and each decoding layer of the multi-domain translation model trained based on the multi-domain translation model training method described in any of the above items to translate the source language text, and obtain the target language text in the domain where the source language text is located.

[0042] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the multi-domain translation model training method or the multi-domain translation method described in any one of the above is implemented.

[0043] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the multi-domain translation model training method or the multi-domain translation method described in any one of the above is implemented.

[0044] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the multi-domain translation model training method or the multi-domain translation method described in any one of the above is implemented.

[0045] The multi-domain translation model training method, the multi-domain translation method, and the device provided by the present invention, through the difference between the target language text of the sample source language text output by the initial translation model and the sample target language text, and the similarity between the output feature of the sample source language text in the first domain and the fusion output feature in multiple domains, and the similarity between the output feature of the sample source language text in the second domain and the fusion output feature in multiple domains output by multiple domain sub-networks in the initial translation model, iteratively update the parameters of the initial translation model, enabling the initial translation model to learn the relationship between the features of the text and the domains, and achieving that the fusion encoding feature of the source language text output by the trained multi-domain translation model is closer to the feature of the source language text in its domain, so that the model has the ability to accurately translate multiple domains without relying on domain labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0047] Figure 1 is a schematic flowchart of the multi-domain translation model training method provided by the present invention;

[0048] Figure 2 is a schematic flowchart of the method for obtaining the first similarity and the second similarity provided by the present invention;

[0049] Figure 3 is a schematic flowchart of the method for obtaining the target language text provided by the present invention;

[0050] Figure 4It is a schematic diagram of the structure of the coding layer and decoding layer framework provided by the present invention;

[0051] Figure 5 It is a data flow diagram of sub-networks in multiple fields provided by the present invention;

[0052] Figure 6 It is a flowchart of the multi-field translation method provided by the present invention;

[0053] Figure 7 Schematic diagram of the structure of the multi-domain translation model training device provided by the present invention;

[0054] Figure 8 Schematic diagram of the structure of the multi-domain translation device provided by the present invention;

[0055] Figure 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0056] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0057] At present, current multi-domain machine translation systems either rely too much on prior domain knowledge, or do not use and are unable to use prior knowledge. They can only solve one of the two situations mentioned above: general translation or specialized translation, and cannot have good general translation and specialized translation capabilities at the same time.

[0058] Therefore, how to simultaneously possess good general translation skills and professional translation skills in multiple fields is a technical problem that needs to be urgently solved by those skilled in the art.

[0059] In response to the above technical problems, an embodiment of the present invention provides a multi-domain translation model training method. Figure 1 This is a flow chart of the multi-domain translation model training method provided by the present invention. Figure 1 As shown, the method includes:

[0060] Step 110, determining an initial translation model;

[0061] It should be noted that the initial translation model is based on the existing Transformer model structure and adds sub-networks in multiple domains. Among them, the network structures of the sub-networks in each domain of the multiple domains are the same. Each sub-network in each domain may include an encoding layer and a decoding layer, or the encoding layer and the decoding layer in the initial translation model may both include sub-networks in multiple domains. The embodiments of the present invention do not limit this. Both the encoding layer and the decoding layer can be a Transformer block.

[0062] Step 120: Based on the initial translation model, determine the target language text of the sample source language text in the first domain.

[0063] Step 130: Based on the sub-networks in multiple domains in the initial translation model, determine the first similarity between the output features of the sample source language text in the first domain and the fused output features in multiple domains, and the second similarity between the output features of the sample source language text in the second domain and the fused output features.

[0064] Step 140: Based on the difference between the target language text and the sample target language text, the first similarity, and the second similarity, iteratively update the parameters of the initial translation model to obtain a trained multi-domain translation model. The first domain is the domain of the sample source language text, and the second domain is the domain other than the first domain among the multiple domains.

[0065] Generally, the translation models in multiple domains determine the corresponding domain sub-networks in the translation model according to the domain label of the source language text to be translated, and then use the domain sub-networks to complete the translation work.

[0066] Considering that the existing translation models in multiple domains have a strong dependence on domain labels, that is, the domain sub-networks corresponding to the domain labels in the translation model are trained by the sample source language text and the sample target language text corresponding to the domain labels. In order to weaken the dependence of the translation model on domain labels and enable the translation model to complete the translation of the source language text in the domain according to the characteristics of the source language text itself, the embodiments of the present invention reduce the similarity between the fused output features of each domain output by each encoding layer and each decoding layer and the features of the domain corresponding to the sample source language text output by each encoding layer and each decoding layer, and at the same time increase the similarity between the fused output features of each domain output by each encoding layer and each decoding layer and the features of other domains except the domain corresponding to the sample source language text output by each encoding layer and each decoding layer, so as to promote the iterative update of the parameters of the initial translation model.

[0067] Specifically, determine that the field where the sample source language text is located is represented as the first field, and the other fields except the first field among the multiple fields set in the initial translation model are represented as the second field. Input the sample source language text into the initial translation model to obtain the target language text of the sample source language text in the first field output by the initial translation model, and determine the difference between the target language text and the sample target language text corresponding to the sample source language text. At the same time, according to the sub-networks of multiple fields in the initial translation model, output the output features of multiple fields respectively according to the input features, and fuse the output features of multiple fields to obtain the fused output features. Then, calculate the similarity between the fused output features and the output features of the first field, denoted as the first similarity, and the similarity between the fused output features and the output features of the second field, denoted as the second similarity. Finally, calculate the similarity according to the similarity between the target language text and the sample target language text, the first distance, and the second distance, and iteratively update the parameters of the initial translation model until the loss converges to obtain the trained multi-field translation model.

[0068] It should be noted that when training the initial translation model, input the sample language pairs of multiple fields into the initial translation model in batches, and only use the sample language pairs of one field in each batch. Among them, the sample language pair includes multiple groups of sample source language texts and sample target language texts. The target language text of the sample source language text can be obtained according to the output of the sub-network of the first field of the initial translation model, or can be obtained according to the output of the sub-networks of the first field in each encoding layer and each decoding layer of the initial translation model. Or the initial translation model can also include a general sub-network, which is used to learn the text features of the field-general part in the sample source language text, and output the target language text of the sample source language text according to the target text features obtained by fusing the general features output by the general sub-network and the field-specific features output by the sub-network of the first field. The embodiments of the present invention do not limit this.

[0069] In addition, the first similarity can be the similarity between the fused output features output by the initial translation model and the output features of the first field, or can be the sum of the similarities between the fused output features output by the sub-networks of multiple fields in each encoding layer and each decoding layer of the initial translation model and the output features of the first field; the first similarity can be the similarity between the fused output features output by the initial translation model and the output features of the second field, or can be the sum of the similarities between the fused output features output by the sub-networks of multiple fields in each encoding layer and each decoding layer of the initial translation model and the output features of the second field. The embodiments of the present invention do not limit this.

[0070] The training method of the multi-domain translation model provided by the embodiment of the present invention iteratively updates the parameters of the initial translation model based on the difference between the target language text of the sample source language text output by the initial translation model and the sample target language text, and the similarity between the output features of the sample source language text in the first domain and the fused output features in multiple domains, and the similarity between the output features of the sample source language text in the second domain and the fused output features in multiple domains, enabling the initial translation model to learn the relationship between the features of the text and the domains. It is achieved that the fused encoding features of the source language text output by the trained multi-domain translation model are closer to the features of the source language text in its corresponding domain, and thus the model has the ability to accurately translate multiple domains without relying on domain labels.

[0071] Based on the above embodiment, Figure 2 is a schematic flowchart of the method for obtaining the first similarity and the second similarity provided by the present invention. As Figure 2 shown, step 130 includes:

[0072] Step 131: Based on the sub-networks of multiple domains in any layer of the encoder-decoder layer, apply the input features of this layer to determine the output features of this layer in the first domain and the output features in the second domain, and based on the output features in the first domain and the output features in the second domain, determine the fused output features of this layer;

[0073] Step 132: Determine the first similarity based on the similarity between the output features in the first domain and the fused output features of each layer in the encoder-decoder layer; and determine the second similarity based on the similarity between the output features in the second domain and the fused output features of each layer in the encoder-decoder layer.

[0074] Specifically, the initial translation model includes an encoder-decoder layer. Each encoding layer and each decoding layer in the encoder-decoder layer include sub-networks of multiple domains. According to the order of each encoding layer and each decoding layer, the input features of any layer in the encoder-decoder layer are sequentially input into the sub-networks of multiple domains in this layer for encoding or decoding respectively to obtain the output features of this layer in the first domain and the output features in the second domain. Then, the output features of this layer in the first domain and the output features in the second domain are fused to obtain the fused output features of this layer, and based on the output features of this layer in the first domain and the fused output features, the similarity between the two is determined. At the same time, based on the output features of this layer in the second domain and the fused output features, the similarity between the two is determined.

[0075] Sum the similarities between the output features of the first domain of each layer in the encoding and decoding layer obtained in step 131 and the fused output features to obtain a first similarity; and sum the similarities between the output features of the second domain of each layer in the encoding and decoding layer obtained in step 131 and the fused output features to obtain a second similarity.

[0076] It should be noted that the input features of each layer in the encoding and decoding layer can be the fused output features of multiple domains output by the previous encoding layer or decoding layer, or the features obtained by fusing the general features output by the general sub-network on the basis of the fused output features of multiple domains again, or the output features of other layers. For example, the input feature of the encoding layer of the first layer is the embedded feature of the sample source language text input by the embedding layer, and the embedding layer can obtain the embedded feature of the text by summing the randomly initialized vocabulary (Token) encoding and the position encoding.

[0077] In addition, the fusion of the output features of this layer in the first domain and the output features in the second domain is to determine the weight values of each domain in the first domain and the second domain according to the gating mechanism for weighted fusion, that is, the weight values obtained through the gating mechanism are used to perform weighted fusion on the output features of the first domain and the corresponding output features of each domain in the second domain.

[0078] Based on the above embodiments, Figure 3 It is a schematic flow chart of the method for obtaining the target language text provided by the present invention. As Figure 3 Therefore, in step 120, it includes:

[0079] Step 121, based on the embedding layer in the initial translation model, determine the text embedding feature of the sample source language text;

[0080] Step 122, based on the general sub-network and the sub-network of the first domain in each encoding layer, respectively encode the text embedding feature to obtain the text general encoding feature and the text domain encoding feature, and based on the text general encoding feature and the text domain encoding feature, determine the text fused encoding feature;

[0081] Step 123, based on the general sub-network and the sub-network of the first domain in each decoding layer, respectively decode the text fused encoding feature to obtain the text general decoding feature and the text domain decoding feature, and based on the text general decoding feature and the text domain decoding feature, determine the text fused decoding feature;

[0082] Step 124, based on the output layer in the initial translation model, apply the text fused decoding feature to output the target language text.

[0083] Specifically, each encoding layer and each decoding layer in the initial translation model further include a general sub-network. Through the embedding layer in the initial translation model, the sample source language text is embedded and position encoding is added to obtain the text embedding features of the sample source language text.

[0084] Take the first encoding layer in the initial translation model as the current encoding layer, take the text embedding features as the input features of the current encoding layer, input the input features of the current encoding layer into the general sub-network and the sub-networks in multiple domains of the current encoding layer respectively, obtain the general encoding features and domain encoding features of the current encoding layer, fuse the general encoding features and domain encoding features of the current encoding layer, and take the obtained fused encoding features of the current encoding layer as the input features of the next encoding layer. Take the next encoding layer as the current encoding layer and iterate until the encoding of the last encoding layer in the initial translation model is completed, obtaining the text fused decoding features output by the last decoding layer.

[0085] Take the first decoding layer in the initial translation model as the current decoding layer, take the text fused decoding features output by the last encoding layer as the input features of the current decoding layer, input the input features of the current decoding layer into the general sub-network and the sub-networks in multiple domains of the current decoding layer respectively, obtain the general decoding features and domain decoding features of the current decoding layer, fuse the general decoding features and domain decoding features of the current decoding layer, and take the obtained fused decoding features of the current decoding layer as the input features of the next decoding layer. Take the next decoding layer as the current decoding layer and iterate until the decoding of the last decoding layer in the initial translation model is completed, obtaining the text fused decoding features output by the last decoding layer.

[0086] The output layer in the initial translation model outputs the target language text of the sample source language text according to the text fused decoding features output by the last decoding layer.

[0087] The multi-domain translation model training method provided by the embodiments of the present invention adds a general sub-network to each encoding layer and each decoding layer, realizing the ability of the model to accurately translate texts in the general domain, and can assist in professional domain translation, making the translation result more natural and accurate.

[0088] Based on the above embodiments, Figure 4 is a schematic structural diagram of the encoding layer and decoding layer framework provided by the present invention. As Figure 4As shown, after the input features pass through the attention layer and the feedforward layer, they are respectively input into the general sub-network and the sub-networks in multiple domains for encoding or decoding, obtaining the general features output by the general sub-network and the dedicated features output by the sub-networks in multiple domains respectively. Then, through the gating mechanism, weights are obtained for weighted averaging to output the output features of this layer. Among them, the sub-networks in multiple domains sequentially pass the input features through the downsampling layer, the activation layer, and the upsampling layer to obtain the features output by the sub-networks in multiple domains. Figure 4 Where α is the weight value obtained according to the gating mechanism.

[0089] Figure 5 It is a schematic diagram of the data flow of the sub-networks in multiple domains provided by the present invention. As Figure 5 shown, during the training of the initial translation model, each encoding layer and each decoding layer will apply Figure 5 the data flow for processing. When the sub-networks in multiple domains in each encoding layer or each decoding layer in the initial translation model receive the input features, the input features will be respectively passed into the sub-network in the first domain and each sub-network in the second domain. The features output by the sub-network in the first domain are used as the standard features. At the same time, the features output by the sub-network in the first domain and the features output by each sub-network in the second domain are weighted and fused through the gating mechanism to obtain the fused output features. And during the training process, the weight information is adjusted by adjusting the gating mechanism so that the similarity between the fused output features and the standard features becomes larger and larger, and the similarity between the fused output features and the features output by each sub-network in the second domain becomes smaller and smaller. Figure 5 Where β1, β A , β B ……β N are the weight values obtained according to the gating mechanism, and the sum of β1, β A , β B ……β N is 1.

[0090] Based on any of the above embodiments, in step 140, iteratively updating the parameters of the initial translation model based on the difference, the first similarity, and the second similarity between the target language text and the sample target language text includes:

[0091] Based on the training loss function, with the goal of minimizing the difference between the target language text and the sample target language text, maximizing the first similarity, and minimizing the second similarity, iteratively updating the parameters of the initial translation model; the training loss function is constructed based on the difference, the first similarity, and the second similarity between the target language text and the sample target language text, and the formula of the training loss function is:

[0092]

[0093]

[0094]

[0095] Wherein, N is the total number of layers in the encoding layer and the decoding layer of the initial translation model, and i is the i-th layer in the total number of layers N; is the output feature of the sample source language text in the first domain in the i-th layer; is the fused output feature of the sample source language text in the i-th layer; is the output feature of the sample source language text in the second domain in the i-th layer; L MT is the translation loss, and the translation loss is determined based on the difference between the target language text and the sample target language text; is the first similarity loss; is the second similarity loss; ε is a hyperparameter.

[0096] It should be noted that the i-th layer can be either an encoding layer or a decoding layer. is the similarity between the output feature of the first domain in the i-th layer and the fused output feature; is the similarity between the output feature of the second domain in the i-th layer and the fused output feature. In particular, ε is set to 1.

[0097] The multi-domain translation model training method provided by the embodiments of the present invention trains the model with the goal of minimizing the difference between the target language text and the sample target language text, and simultaneously maximizing the first similarity and minimizing the second similarity. During the training process, the similarity between the fused output feature output by the model and the output feature of the first domain will become larger and larger, while the similarity between the fused output feature and the output feature of the second domain will become farther and farther. When the trained multi-domain translation model is used for translation, the weight of the sub-network in the domain where the source language text is located can be increased, thereby making the translation result of the source language text more accurate.

[0098] Figure 6 is a schematic flow chart of the multi-domain translation method provided by the present invention. As Figure 6 shown, the multi-domain translation method provided by the embodiments of the present invention includes:

[0099] [[ID=3**6]]Step 610, determining the source language text to be translated;

[0100] Step 620, if the domain category of the source language text is not determined, the general sub-network and the sub-networks of multiple domains in each encoding layer and each decoding layer in the multi-domain translation model trained by the multi-domain translation model training method in any of the above embodiments are used to translate the source language text to obtain the target language text in the domain where the source language text is located.

[0101] Specifically, the multi-domain translation model training method in any of the above embodiments is applied to train an initial translation model to obtain a multi-domain translation model. After determining the source language text to be translated, if there is no domain category for the source language text at this time, the general sub-networks and the sub-networks of multiple domains in each encoding layer and each decoding layer of the multi-domain translation model translate the source language text to obtain the target language text in the domain where the source language text is located.

[0102] The multi-domain translation method provided by the embodiments of the present invention generally applies the multi-domain translation model training method in any of the above embodiments to train an initial translation model to obtain a multi-domain translation model, and translates the source language text to be translated, realizing the accurate translation ability of the model for multiple domains without relying on domain labels.

[0103] Based on the above embodiments, after step 410, it further includes:

[0104] If the domain category of the source language text is determined, based on the general sub-networks and the sub-networks of the domain category in each encoding layer and each decoding layer of the multi-domain translation model, the source language text is translated to obtain the target language text.

[0105] Based on the above embodiments, the embodiments of the present invention collect data in multiple domains including law, medicine, news commentary, IT, education, and scientific and technical reports from OPUS. The data scale of each domain is shown in Table 1.

[0106] Field Data source Training set Law OPUS 487k Medical OPUS 48k News commentary OPUS 277k IT OPUS 10k Education OPUS 470k Technical report OPUS 283k

[0107] Table 1

[0108] The initial translation model is trained according to the data in Table 1 to obtain a multi-domain translation model. And the data in Table 1 is used to conduct experiments on the multi-domain translation model. The index of the experiment is the BLEU value between the translation result of the machine translation and the standard translation (the larger the value of this index, the closer the translation result of the machine translation is to the standard translation). The experimental results are shown in Table 2.

[0109]

[0110] Table 2

[0111] Next, the multi-domain translation model training device and the multi-domain translation device provided by the present invention will be described. The multi-domain translation model training device described below can be correspondingly referred to the multi-domain translation model training method described above, and the multi-domain translation device and the multi-domain translation method can be correspondingly referred to each other.

[0112] Figure 7 is a schematic structural diagram of the multi-domain translation model training device provided by the present invention. As Figure 7As shown in the figure, the device includes: a determination module 710, an output module 720, a similarity module 730, and a training module 740.

[0113] Among them,

[0114] The determination module 710 is used to determine an initial translation model;

[0115] The output module 720 is used to determine the target language text of the sample source language text in the first domain based on the initial translation model;

[0116] The similarity module 730 is used to determine a first similarity between the output feature of the sample source language text in the first domain and the fused output feature in multiple domains, and a second similarity between the output feature of the sample source language text in the second domain and the fused output feature, based on the sub-networks in multiple domains of the initial translation model;

[0117] The training module 740 is used to iteratively update the parameters of the initial translation model based on the difference between the target language text and the sample target language text, the first similarity, and the second similarity, to obtain a trained multi-domain translation model; the first domain is the domain of the sample source language text, and the second domain is the domain other than the first domain among multiple domains.

[0118] The multi-domain translation model training device provided by the embodiments of the present invention, through a determination module, is used to determine an initial translation model; an output module, is used to determine the target language text of the sample source language text in the first domain based on the initial translation model; a similarity module, is used to determine a first similarity between the output feature of the sample source language text in the first domain and the fused output feature in multiple domains, and a second similarity between the output feature of the sample source language text in the second domain and the fused output feature, based on the sub-networks in multiple domains of the initial translation model; a training module, is used to iteratively update the parameters of the initial translation model based on the difference between the target language text and the sample target language text, the first similarity, and the second similarity, to obtain a trained multi-domain translation model; the first domain is the domain of the sample source language text, and the second domain is the domain other than the first domain among multiple domains, enables the initial translation model to learn the relationship between the features of the text and the domains, realizes that the fused encoding features of the source language text output by the trained multi-domain translation model are closer to the features of the source language text in its domain, and further the model has the ability to accurately translate multiple domains without relying on domain labels.

[0119] Based on any of the above embodiments, the similarity module 730 includes:

[0120] A feature acquisition sub-module, configured to determine the output features of a layer in a first domain and the output features of a layer in a second domain by applying the input features of the layer based on the sub-networks of multiple domains in any one of the encoding layers and the decoding layers, and determine the fused output features of the layer based on the output features of the first domain and the output features of the second domain;

[0121] A similarity calculation sub-module, configured to determine a first similarity based on the similarity between the output features of the first domain and the fused output features of each layer in the encoding layers and the decoding layers; and determine a second similarity based on the similarity between the output features of the second domain and the fused output features of each layer in the encoding layers and the decoding layers.

[0122] Based on any of the above embodiments, the output module 720 includes:

[0123] An embedding sub-module, configured to determine the text embedding features of the sample source language text based on the embedding layer in the initial translation model;

[0124] An encoding sub-module, configured to encode the text embedding features respectively based on the general sub-network and the sub-network of the first domain in each encoding layer to obtain text general encoding features and text domain encoding features, and determine text fused encoding features based on the text general encoding features and the text domain encoding features;

[0125] A decoding sub-module, configured to decode the text fused encoding features respectively based on the general sub-network and the sub-network of the first domain in each decoding layer to obtain text general decoding features and text domain decoding features, and determine text fused decoding features based on the text general decoding features and the text domain decoding features;

[0126] An output sub-module, configured to output the target language text by applying the text fused decoding features based on the output layer in the initial translation model.

[0127] Based on any of the above embodiments, in the training module 740, the parameters of the initial translation model are iteratively updated based on the difference between the target language text and the sample target language text, the first similarity, and the second similarity, including:

[0128] Based on the training loss function, with the minimum difference between the target language text and the sample target language text, and with the maximum first similarity and the minimum second similarity as the goals, the parameters of the initial translation model are iteratively updated; the training loss function is constructed based on the difference between the target language text and the sample target language text, the first similarity, and the second similarity, and the formula of the training loss function is:

[0129]

[0130]

[0131]

[0132] Wherein, N is the total number of layers in the encoding layer and the decoding layer of the initial translation model, and i is the i-th layer in the total number of layers N; is the output feature of the sample source language text in the first domain in the i-th layer; is the fused output feature of the sample source language text in the i-th layer; is the output feature of the sample source language text in the second domain in the i-th layer; L MT is the translation loss, and the translation loss is determined based on the difference between the target language text and the sample target language text; is the first similarity loss, and the first similarity loss is based on; is the second similarity loss.

[0133] Figure 8 is a schematic structural diagram of the multi-domain translation device provided by the present invention. As Figure 8 shown, the device includes: a determination module 810 and a translation module 820.

[0134] Among them,

[0135] The determination module 810 is used to determine the source language text to be translated;

[0136] The translation module 820 is used to, if the domain category of the source language text is not determined, based on the general sub-network and the sub-networks of multiple domains in each encoding layer and each decoding layer of the multi-domain translation model trained by the multi-domain translation model training method in any of the above embodiments, translate the source language text to obtain the target language text in the domain where the source language text is located.

[0137] The multi-domain translation device provided by the embodiments of the present invention, through the determination module, is used to determine the source language text to be translated; the translation module, if the domain category of the source language text is not determined, based on the general sub-network and the sub-networks of multiple domains in each encoding layer and each decoding layer of the multi-domain translation model trained by the multi-domain translation model training method in any of the above embodiments, translate the source language text to obtain the target language text in the domain where the source language text is located, realizing the accurate translation ability of the model for multiple domains without relying on domain labels.

[0138] Based on the above embodiments, the multi-domain translation device further includes:

[0139] The second translation module: is used to, if the domain category of the source language text is determined, based on the general sub-network and the sub-network of the domain category in each encoding layer and each decoding layer of the multi-domain translation model, translate the source language text to obtain the target language text.

[0140] Figure 9 Illustrates a schematic diagram of the physical structure of an electronic device, as Figure 9 shown. The electronic device may include: a processor 910, a communications interface 920, a memory 930, and a communication bus 940. Among them, the processor 910, the communications interface 920, and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logical instructions in the memory 930 to execute a multi-domain translation model training method or a multi-domain translation method. Among them, the multi-domain translation model training method includes: determining an initial translation model; based on the initial translation model, determining the target language text of the sample source language text in the first domain; based on the sub-networks of multiple domains in the initial translation model, determining the first similarity between the output features of the sample source language text in the first domain and the fused output features in multiple domains, and the second similarity between the output features of the sample source language text in the second domain and the fused output features; based on the difference between the target language text and the sample target language text, the first similarity, and the second similarity, iteratively updating the parameters of the initial translation model to obtain a trained multi-domain translation model; the first domain is the domain of the sample source language text, and the second domain is the domain other than the first domain among the multiple domains.

[0141] In addition, when the logical instructions in the above-mentioned memory 930 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0142] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-domain translation model training method or the multi-domain translation method provided by the above-mentioned various methods. Among them, the multi-domain translation model training method includes: determining an initial translation model; based on the initial translation model, determining the target language text of the sample source language text in the first domain; based on the sub-networks of multiple domains in the initial translation model, determining the first similarity between the output features of the sample source language text in the first domain and the fused output features in multiple domains, and the second similarity between the output features of the sample source language text in the second domain and the fused output features; based on the difference between the target language text and the sample target language text, the first similarity and the second similarity, iteratively updating the parameters of the initial translation model to obtain a trained multi-domain translation model; the first domain is the domain of the sample source language text, and the second domain is the domain other than the first domain among multiple domains.

[0143] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the multi-domain translation model training method or the multi-domain translation method provided by the above-mentioned various methods. Among them, the multi-domain translation model training method includes: determining an initial translation model; based on the initial translation model, determining the target language text of the sample source language text in the first domain; based on the sub-networks of multiple domains in the initial translation model, determining the first similarity between the output features of the sample source language text in the first domain and the fused output features in multiple domains, and the second similarity between the output features of the sample source language text in the second domain and the fused output features; based on the difference between the target language text and the sample target language text, the first similarity and the second similarity, iteratively updating the parameters of the initial translation model to obtain a trained multi-domain translation model; the first domain is the domain of the sample source language text, and the second domain is the domain other than the first domain among multiple domains.

[0144] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0145] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a multi-domain translation model, characterized in that Including: Determine an initial translation model; Based on the initial translation model, determine the target language text of the sample source language text in the first domain; Based on the sub-networks of multiple domains in the initial translation model, determine the first similarity between the output feature of the sample source language text in the first domain and the fused output feature in multiple domains, and the second similarity between the output feature of the sample source language text in the second domain and the fused output feature; Based on the difference between the target language text and the sample target language text, the first similarity, and the second similarity, iteratively update the parameters of the initial translation model to obtain a trained multi-domain translation model; the first domain is the domain of the sample source language text, and the second domain is the domain other than the first domain among the multiple domains; The initial translation model includes an encoder-decoder layer, and each encoder layer and each decoder layer in the encoder-decoder layer include sub-networks of multiple domains; The determining the first similarity between the output feature of the sample source language text in the first domain and the fused output feature in multiple domains, and the second similarity between the output feature of the sample source language text in the second domain and the fused output feature, based on the sub-networks of multiple domains in the initial translation model, includes: Based on the sub-networks of multiple domains in any layer of the encoder-decoder layer, apply the input feature of the any layer to determine the output feature of the any layer in the first domain and the output feature in the second domain, and based on the output feature in the first domain and the output feature in the second domain, determine the fused output feature of the any layer; Based on the similarity between the output feature in the first domain and the fused output feature of each layer in the encoder-decoder layer, determine the first similarity; and based on the similarity between the output feature in the second domain and the fused output feature of each layer in the encoder-decoder layer, determine the second similarity.

2. The multi-domain translation model training method according to claim 1, wherein Each encoder layer and each decoder layer further include a general sub-network, including: The determining the target language text of the sample source language text in the first domain based on the initial translation model includes: Based on the embedding layer in the initial translation model, determine the text embedding feature of the sample source language text; Based on the general sub-network and the sub-network of the first domain in each encoder layer, respectively encode the text embedding feature to obtain a text general encoding feature and a text domain encoding feature, and based on the text general encoding feature and the text domain encoding feature, determine a text fused encoding feature; Based on the general sub-network and the sub-network of the first domain in each decoder layer, respectively decode the text fused encoding feature to obtain a text general decoding feature and a text domain decoding feature, and based on the text general decoding feature and the text domain decoding feature, determine a text fused decoding feature; Based on the output layer in the initial translation model, apply the text fused decoding feature to output the target language text.

3. The multi-domain translation model training method according to any one of claims 1 or 2, characterized in that Iteratively updating the parameters of the initial translation model based on the differences between the target language text and the sample target language text, the first similarity, and the second similarity, includes: Based on the training loss function, with the goal of minimizing the differences between the target language text and the sample target language text, maximizing the first similarity, and minimizing the second similarity, iteratively updating the parameters of the initial translation model; the training loss function is constructed based on the differences between the target language text and the sample target language text, the first similarity, and the second similarity, and the formula of the training loss function is: Where N is the total number of layers in the encoding layer and the decoding layer in the initial translation model, and i is the i-th layer in the total number of layers N; is the output feature of the sample source language text in the first domain in the i-th layer; is the fused output feature of the sample source language text in the i-th layer; is the output feature of the sample source language text in the second domain in the i-th layer; L MT is the translation loss, and the translation loss is determined based on the difference between the target language text and the sample target language text; is the first similarity loss; is the second similarity loss; ε is a hyperparameter.

4. A multi-domain translation method, characterized in that, Including: Determine the source language text to be translated; If the domain category of the source language text is not determined, translate the source language text using the general sub-networks and the sub-networks of multiple domains in the multi-domain translation model trained by the multi-domain translation model training method according to any one of claims 1 to 3 to obtain the target language text in the domain where the source language text is located.

5. The multi-domain translation method according to claim 4, characterized in that After determining the source language text to be translated, it further includes: If the domain category of the source language text is determined, translate the source language text using the general sub-networks in each encoding layer and each decoding layer of the multi-domain translation model and the sub-networks of the domain category to obtain the target language text.

6. A multi-domain translation model training device, characterized in that Including: A determination module for determining an initial translation model; An output module for determining the target language text of the sample source language text in the first domain based on the initial translation model; A similarity module for determining the first similarity between the output feature of the sample source language text in the first domain and the fused output feature in multiple domains, and the second similarity between the output feature of the sample source language text in the second domain and the fused output feature based on the sub-networks of multiple domains in the initial translation model; A training module for iteratively updating the parameters of the initial translation model based on the differences between the target language text and the sample target language text, the first similarity, and the second similarity to obtain a trained multi-domain translation model; the first domain is the domain of the sample source language text, and the second domain is the domain other than the first domain among the multiple domains; The initial translation model includes an encoding and decoding layer, and each encoding layer and each decoding layer in the encoding and decoding layer include the sub-networks of multiple domains; Determining the first similarity between the output feature of the sample source language text in the first domain and the fused output feature in multiple domains, and the second similarity between the output feature of the sample source language text in the second domain and the fused output feature based on the sub-networks of multiple domains in the initial translation model, includes: Based on the sub-networks of multiple domains in any one of the encoding and decoding layers, apply the input features of the any one layer to determine the output features of the any one layer in the first domain and the output features in the second domain, and based on the output features in the first domain and the output features in the second domain, determine the fused output features of the any one layer; Based on the similarity between the output features in the first domain and the fused output features of each layer in the encoding and decoding layers, determine the first similarity; and based on the similarity between the output features in the second domain and the fused output features of each layer in the encoding and decoding layers, determine the second similarity.

7. A multi-domain translation device, characterized in that, Comprising: A determination module, configured to determine the source language text to be translated; A translation module, configured to, if the domain category of the source language text is not determined, translate the source language text based on the general sub-networks and the sub-networks of multiple domains in each encoding layer and each decoding layer of the multi-domain translation model trained by the multi-domain translation model training method according to any one of claims 1 to 3, to obtain the target language text in the domain where the source language text is located.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the multi-domain translation model training method according to any one of claims 1 to 3 or the multi-domain translation method according to claim 4 or 5.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-domain translation model training method according to any one of claims 1 to 3 or the multi-domain translation method according to claim 4 or 5.

Images