Multilingual machine translation model training method, multilingual translation method and device

Abstract

The present invention provides a multilingual machine translation model training method, a multilingual translation method and a device. The training method comprises: obtaining a multilingual translation training corpus and a multilingual translation verification corpus; establishing and initializing a multilingual translation model with full parameter sharing, and training the multilingual translation model using the multilingual translation training corpus and the multilingual translation verification corpus to obtain a multilingual machine translation model. During the training process, the model automatically determines whether each shared parameter needs to be converted into a language-related parameter, copies the parameters that need to be converted and assigns them to the relevant languages, so that the model has both shared parameters and language-related parameters. The present invention is used to automatically determine whether parameters need to be shared and which language pairs need to be shared during the training of a multilingual machine translation model, without relying on pre-specified shared or language-related model components.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a multilingual machine translation model training method, a multilingual translation method and a device. Background Art

[0002] Machine translation is a discipline and technology that uses computers to automatically translate one natural language into another natural language. The natural language before translation is usually called the source language, and the natural language after translation is usually called the target language. In recent years, with the development of deep learning technology, neural network-based machine translation methods have significantly improved translation quality and have been widely used. Compared with traditional statistics-based machine translation methods, neural network machine translation models can directly model the mapping from one language to another in an end-to-end manner. In this mode, by expanding the language data of the source / target end, it is possible to directly map from one language set to another language set, so that a single model can handle translations between multiple language pairs, while improving the efficiency of training and deployment. Therefore, this multilingual translation model has received widespread attention and application.

[0003] In multilingual translation, a model needs to model the mapping relationship between multiple languages, and there are grammatical differences between different languages, which introduces the problem of language conflict and model bottleneck. That is, optimization on one language will affect the translation quality of another language, making the multilingual translation model less effective than the bilingual translation model. The current solution to this problem is to add some manually designed language-specific components to a model shared by all languages ​​to improve the model's modeling capabilities for specific languages. However, this method strongly relies on prior knowledge, requires linguistic features and experience to guide the design of language-specific parts, and is difficult to extend to other languages. Summary of the invention

[0004] The present invention provides a multilingual machine translation model training method, a multilingual translation method and a device, which are used to solve the defects of language conflict and model bottleneck in the multilingual translation model in the prior art, and to improve the translation accuracy of the multilingual translation model.

[0005] The present invention provides a multilingual machine translation model training method, comprising:

[0006] Acquire a multilingual translation training corpus and a multilingual translation verification corpus, wherein the multilingual translation training corpus is a plurality of language pairs including bilingual parallel data, and the multilingual translation verification corpus is a plurality of language pairs including multilingual parallel data;

[0007] A multilingual translation model with full parameter sharing is established and initialized, and the multilingual translation model is trained using the multilingual translation training corpus and the multilingual translation verification corpus to obtain a multilingual machine translation model; wherein, through the training of the multilingual translation model, the full parameter sharing of the multilingual translation model is converted into partial parameter sharing and the remaining parameters are language-related, and the language-related parameters are determined by the following steps:

[0008] During each verification in the training process, all language pairs in the multilingual translation verification corpus are clustered into two categories according to the gradient of each parameter of all language pairs in the multilingual translation verification corpus;

[0009] Select a parameter group according to the inter-class distance after two-class clustering of each parameter group, and copy the selected parameter group to obtain a parameter group copy;

[0010] The selected parameter group and the parameter group copy are respectively assigned to the language pairs included in the two categories of the selected parameter group to obtain parameters related to the languages ​​included in the language pairs.

[0011] According to a multilingual machine translation model training method provided by the present invention, the binary clustering of all language pairs in the multilingual translation verification corpus according to the gradient of each parameter of all language pairs on a parameter-by-parameter basis specifically comprises the following steps:

[0012] Obtaining the gradient of each parameter for each language pair in the multilingual translation verification corpus, dividing all parameters of the multilingual translation model into multiple parameter groups, and expanding the gradients of all language pairs in each parameter group into a gradient vector;

[0013] The cosine similarity of the gradient vectors of all the language pairs in each parameter group is calculated, and the language pairs of each parameter group are clustered into two categories according to the cosine similarity of the gradient vectors of all the language pairs.

[0014] According to a multilingual machine translation model training method provided by the present invention, the step of selecting a parameter group according to the inter-class distance after each parameter group is clustered into two classes specifically comprises the following steps:

[0015] The inter-class distances after two-class clustering of each parameter group are obtained and sorted, and the K parameter groups with the largest inter-class distances are selected, where K is a positive integer.

[0016] According to a multilingual machine translation model training method provided by the present invention, each language pair includes a source language sentence and a target language sentence, the source language and the target language of the language pair containing bilingual parallel data are both one, and the source language and the target language of the language pair containing multilingual parallel data are both one or more.

[0017] The present invention also provides a multilingual translation method, comprising:

[0018] Obtain source language sentences to be translated;

[0019] A multilingual machine translation model is obtained by training according to the multilingual machine translation model training method, and the source language sentence to be translated is translated.

[0020] The present invention also provides a multilingual machine translation model training device, comprising:

[0021] A collection unit, the collection unit is used to collect language translation training corpus and multilingual translation verification corpus, the multilingual translation training corpus is a plurality of language pairs containing bilingual parallel data, and the multilingual translation verification corpus is a plurality of language pairs containing multilingual parallel data;

[0022] A training unit, the training unit is used to establish and initialize a multilingual translation model with full parameter sharing, and train the multilingual translation model using the multilingual translation training corpus and the multilingual translation verification corpus to obtain a multilingual machine translation model; wherein, through the training of the multilingual translation model, the full parameter sharing of the multilingual translation model is converted into partial parameter sharing and the remaining parameters are language-related, and the language-related parameters are determined by the following steps:

[0023] During each verification in the training process, all language pairs in the multilingual translation verification corpus are clustered into two categories according to the gradient of each parameter of all language pairs in the multilingual translation verification corpus;

[0024] Select a parameter group according to the inter-class distance after two-class clustering of each parameter group, and copy the selected parameter group to obtain a parameter group copy;

[0025] The selected parameter group and the parameter group copy are respectively assigned to the language pairs included in the two categories of the selected parameter group to obtain parameters related to the languages ​​included in the language pairs.

[0026] The present invention also provides a multi-language translation device, comprising:

[0027] A source language acquisition unit, the source language acquisition unit is used to acquire source language sentences to be translated;

[0028] A translation unit is used to train a multilingual machine translation model using the multilingual machine translation model training method, and translate the source language sentence to be translated.

[0029] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the multilingual machine translation model training method or the multilingual translation method as described above are implemented.

[0030] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the multilingual machine translation model training methods or multilingual translation methods described above.

[0031] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any of the multilingual machine translation model training methods or multilingual translation methods described above.

[0032] The present invention provides a multilingual machine translation model training method, a multilingual translation method and a device, which automatically determines whether parameters need to be shared and which language pairs need to be shared during the multilingual machine translation model training process, without relying on pre-specified sharing or language-related model components, thereby achieving the effect of automatic learning parameter sharing. At the same time, compared with a fully shared multilingual translation model, the present invention has achieved a significant improvement in translation quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 is a flowchart of a multilingual machine translation model training method according to an embodiment of the present invention;

[0035] Figure 2 is a schematic diagram of a method for calculating the gradient of each parameter for all language pairs in an embodiment of the present invention;

[0036] Figure 3 is a schematic diagram of a method for performing two-category clustering on language pairs by parameter group according to an embodiment of the present invention;

[0037] Figure 4 A schematic diagram of a multi-language translation method according to an embodiment of the present invention;

[0038] Figure 5 A schematic diagram of the structure of a multilingual machine translation model training device according to an embodiment of the present invention;

[0039] Figure 6 A schematic diagram of the structure of a multi-language translation device according to an embodiment of the present invention;

[0040] Figure 7 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The embodiments of the present invention are aimed at the problems of language conflicts and model bottlenecks encountered in multilingual translation. In the process of training a multilingual machine translation model, the existing multilingual translation model adopts the idea of ​​full parameter sharing, so it can only achieve the effect of alleviating language conflicts by projecting or covering language pairs with larger conflicts after calculating the conflict relationship. The embodiments of the present invention can automatically determine whether parameters need to be shared and which language pairs need to be shared without relying on pre-specified shared or language-related model components. The model parameters only participate in the training of specific language pairs, thereby eliminating language conflicts. Therefore, the inventive concept of the multilingual machine translation model training method of the embodiments of the present invention is:

[0043] Acquire a multilingual translation training corpus and a multilingual translation verification corpus, wherein the multilingual translation training corpus is a plurality of language pairs including bilingual parallel data, and the multilingual translation verification corpus is a plurality of language pairs including multilingual parallel data;

[0044] A multilingual translation model with full parameter sharing is established and initialized, and the multilingual translation model is trained using the multilingual translation training corpus and the multilingual translation verification corpus to obtain a multilingual machine translation model; wherein, through the training of the multilingual translation model, the full parameter sharing of the multilingual translation model is converted into partial parameter sharing and the remaining parameters are language-related, and the language-related parameters are determined by the following steps:

[0045] During each verification in the training process, all language pairs in the multilingual translation verification corpus are clustered into two categories according to the gradient of each parameter of all language pairs in the multilingual translation verification corpus;

[0046] Select a parameter group according to the inter-class distance after two-class clustering of each parameter group, and copy the selected parameter group to obtain a parameter group copy;

[0047] The selected parameter group and the parameter group copy are respectively assigned to the language pairs included in the two categories of the selected parameter group to obtain parameters related to the languages ​​included in the language pairs.

[0048] Combine the following Figure 1 The multilingual machine translation model training method of the embodiment of the present invention is described as follows Figure 1 As shown, including:

[0049] S101, obtaining a multilingual translation training corpus and a multilingual translation verification corpus, wherein the multilingual translation training corpus is a plurality of language pairs including bilingual parallel data, and the multilingual translation verification corpus is a plurality of language pairs including multilingual parallel data;

[0050] It should be noted that each language pair includes a source language sentence and a target language sentence, the source language and the target language of the language pair containing bilingual parallel data are both one, and the source language and the target language of the language pair containing multilingual parallel data are both one or more.

[0051] The multi-language translation training corpus and the multi-language translation verification corpus are of similar size. A task label is set before each source language sentence of each language pair in the multi-language translation training corpus. For example, if English is translated into Chinese, a task label is set before the English sentence. <en2zh>Label.

[0052] S102, establishing and initializing a multilingual translation model with full parameter sharing, and training the multilingual translation model using the multilingual translation training corpus and the multilingual translation verification corpus to obtain a multilingual machine translation model; wherein, by training the multilingual translation model, the full parameter sharing of the multilingual translation model is converted into partial parameter sharing and the remaining parameters are language-related.

[0053] It should be noted that, during training in this embodiment, the multilingual translation training corpus is divided into different batches according to language pairs, each batch contains only one language pair, and verification is performed every 8,000 training steps.

[0054] The model parameters in this embodiment refer to parameters that can be updated during the training process, such as the weights and biases of the model.

[0055] The multilingual machine translation model training method of this embodiment does not rely on prior knowledge such as artificially designed unique components for different languages. During the training process, after calculating the gradient of the language pair during verification and clustering, the model automatically determines whether each shared parameter needs to be converted into a language-related parameter, copies the parameters that need to be converted and assigns them to the relevant languages, so that the model has both shared parameters and language-related parameters, greatly improving the generalization characteristics of the model.

[0056] In this embodiment, the step of performing binary clustering on all language pairs according to the gradient of each parameter of all language pairs in the multilingual translation verification corpus comprises the following steps:

[0057] S201, obtaining the gradient of each parameter for each language pair in the multilingual translation verification corpus, dividing all parameters of the multilingual translation model into multiple parameter groups, and expanding the gradients of all language pairs in each parameter group into a gradient vector;

[0058] It should be noted that the standard for dividing parameter groups in this embodiment is different parameter group granularities, and the granularity can be a layer, a module or a single parameter matrix in the model. In this embodiment, a single parameter matrix is ​​selected as the granularity of the parameter group. The gradients of all language pairs in each parameter group are collected and expanded into a one-dimensional gradient vector.

[0059] Figure 2 Schematic diagram of the gradient calculation method for each parameter for all language pairs in an embodiment of the present invention. Figure 2 As shown in the figure, the MNMT model is a multilingual translation model. When calculating the gradient, forward and reverse calculations are performed on each batch of data, and the gradient corresponding to each parameter for each language is recorded. Figure 1 middle, represents the gradient of the jth task on the ith parameter, θ i Represents the i-th parameter.

[0060] S202: Calculate the cosine similarity of the gradient vectors of all the language pairs in each parameter group, and perform two-category clustering on the language pairs of each parameter group according to the cosine similarity of the gradient vectors of all the language pairs.

[0061] It should be noted that, in this embodiment, the cosine similarity between two gradient vectors is measured by measuring the cosine value of the angle between them, and a hierarchical clustering method is used to perform two-category clustering on the tasks.

[0062] Figure 3 FIG. 1 is a schematic diagram of a method for performing two-category clustering on language pairs by parameter group according to an embodiment of the present invention. Figure 3 As shown, there are three tasks in θ i After clustering, language pairs t1 and t2 are classified into one category, and t3 is classified into another category. In the current round, let θ i =θ i′ =θ i″ , and θ i′ and θ i″ Assign tasks in two categories respectively, then θ i′ The specific language pair is t1 and t2, θ i″ The specific language pair is t3. and In this embodiment, for the selected parameter group, the parameters are copied to obtain two identical parameter groups, the two identical parameter groups are respectively assigned to tasks corresponding to the two categories, and the calculation graphs of the corresponding tasks are re-established, so that in subsequent training, each parameter only participates in the training of the corresponding task.

[0063] In this embodiment, selecting a parameter group according to the inter-class distance after each parameter group is clustered into two classes specifically includes the following steps:

[0064] Obtain the inter-class distances after two-class clustering of each parameter group and sort them, and select the K parameter groups with the largest inter-class distances, where K is a positive integer. The value of K is related to the expected parameter quantity of the model. In this embodiment, the value of K must satisfy the requirement that the model wants to obtain twice the initial parameter quantity.

[0065] Combine the following Figure 5 A multilingual machine translation model training device according to an embodiment of the present invention is described. The multilingual machine translation model training device described below and the multilingual machine translation model training method described above can be referred to each other. Figure 5 As shown, including:

[0066] A collection unit 501, the collection unit is used to collect language translation training corpus and multilingual translation verification corpus, the multilingual translation training corpus is a plurality of language pairs including bilingual parallel data, and the multilingual translation verification corpus is a plurality of language pairs including multilingual parallel data;

[0067] A training unit 502 is used to establish and initialize a multilingual translation model with full parameter sharing, and train the multilingual translation model using the multilingual translation training corpus and the multilingual translation verification corpus to obtain a multilingual machine translation model; wherein, by training the multilingual translation model, the full parameter sharing of the multilingual translation model is converted into partial parameter sharing and the remaining parameters are language-related, and the language-related parameters are determined by the following steps:

[0068] During each verification in the training process, all language pairs in the multilingual translation verification corpus are clustered into two categories according to the gradient of each parameter of all language pairs in the multilingual translation verification corpus;

[0069] Select a parameter group according to the inter-class distance after two-class clustering of each parameter group, and copy the selected parameter group to obtain a parameter group copy;

[0070] The selected parameter group and the parameter group copy are respectively assigned to the language pairs included in the two categories of the selected parameter group to obtain parameters related to the languages ​​included in the language pairs.

[0071] The multilingual machine translation model training device of this embodiment does not rely on prior knowledge such as artificially designed unique components for different languages. Instead, the model parameters of the bilingual translation model are expanded. After the gradients of the language pairs are calculated and clustered, the selected parameter group and the parameter group replica are respectively assigned to the language pairs included in the two categories of the selected parameter group, so that each parameter only participates in the training of a specific language pair, which greatly improves the generalization characteristics of the model.

[0072] Combine the following Figure 4 The multi-language translation method according to an embodiment of the present invention includes:

[0073] S401, obtaining a source language sentence to be translated;

[0074] S402: A multilingual machine translation model is obtained by training according to the multilingual machine translation model training method, and the source language sentence to be translated is translated.

[0075] The multilingual translation method of this embodiment can achieve translation from the same source language to multiple target languages, translation from multiple source languages ​​to the same target language, or translation from multiple source languages ​​to multiple target languages, and the translation accuracy is significantly improved compared with previous methods.

[0076] In this embodiment, a public multilingual translation dataset is used to verify the effect of the multilingual translation method. The datasets include the OPUS (the open parallel corpus) dataset and the WMT (Workshop on Statistical Machine Translation) dataset. Among them, the OPUS dataset contains translations from English to 12 languages ​​including French, Italian, Latvian, Lithuanian, Estonian, Finnish, Indonesian, Malay, Polish, Czech, Ukrainian, and Russian. The WMT dataset contains translations from English to 5 languages ​​including French, Czech, German, Estonian, and Romanian. In the OPUS dataset, the training data scale of each language is 1 million, and the scale of WMT training data is shown in Appendix 1.

[0077] Appendix 1 WMT training data statistics

[0078] Data Source Training set French WMT 14 About 39.03 million Czech WMT 14 About 15.65 million German WMT 14 About 4.46 million Estonian WMT 18 About 1.94 million Romanian WMT 16 About 610,000

[0079] Appendix 2-5 shows the experimental results obtained using the above data. The indicator is the BLEU value (Bilingual evaluation under study) between the machine-translated translation and the standard translation. The BLEU score ranges from 0 to 100. The closer the score is to 100, the higher the quality of the translation.

[0080] Appendix 2 OPUS one-to-many translation experimental results

[0081]

[0082] Appendix 3 Results of OPUS many-to-one translation experiment

[0083]

[0084] Appendix 2 and Appendix 3 show the results of training using OPUS data. Compared with the multilingual baseline system and the bilingual baseline system, the present invention has significantly improved the translation quality, proving the effectiveness of using other languages ​​to generate translations in the decoding stage.

[0085] Appendix 4 WMT one-to-many translation experimental results

[0086]

[0087]

[0088] Appendix 5 WMT many-to-one translation experiment results

[0089]

[0090] Appendix 4 and Appendix 5 show the results of training using WMT data, which illustrate that the present invention can also achieve good translation quality on a multilingual dataset with unbalanced training data.

[0091] Combine the following Figure 6 The multi-language translation device of the embodiment of the present invention is described. The multi-language translation device described below and the multi-language translation method described above can be referred to each other. Figure 6 As shown, including:

[0092] A source language acquisition unit 601, which is used to acquire source language sentences to be translated;

[0093] The translation unit 602 is used to train a multilingual machine translation model using the multilingual machine translation model training method to translate a source language sentence.

[0094] The multilingual translation device of this embodiment can translate the same source language into multiple target languages, and the translation accuracy is significantly improved compared with previous devices.

[0095] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the multilingual machine translation model training method, the method comprising: obtaining a multilingual translation training corpus and a multilingual translation verification corpus, the multilingual translation training corpus is a plurality of language pairs containing bilingual parallel data, and the multilingual translation verification corpus is a plurality of language pairs containing multilingual parallel data;

[0096] A multilingual translation model with full parameter sharing is established and initialized, and the multilingual translation model is trained using the multilingual translation training corpus and the multilingual translation verification corpus to obtain a multilingual machine translation model; wherein, through the training of the multilingual translation model, the full parameter sharing of the multilingual translation model is converted into partial parameter sharing and the remaining parameters are language-related, and the language-related parameters are determined by the following steps:

[0097] During each verification in the training process, all language pairs in the multilingual translation verification corpus are clustered into two categories according to the gradient of each parameter of all language pairs in the multilingual translation verification corpus;

[0098] Select a parameter group according to the inter-class distance after two-class clustering of each parameter group, and copy the selected parameter group to obtain a parameter group copy;

[0099] The selected parameter group and the parameter group copy are respectively assigned to the language pairs included in the two categories of the selected parameter group to obtain parameters related to the languages ​​included in the language pairs.

[0100] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0101] On the other hand, the present invention also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the multilingual machine translation model training method provided by the above methods, the method comprising: obtaining a multilingual translation training corpus and a multilingual translation verification corpus, the multilingual translation training corpus is a plurality of language pairs containing bilingual parallel data, the multilingual translation verification corpus is a plurality of language pairs containing multilingual parallel data;

[0102] A multilingual translation model with full parameter sharing is established and initialized, and the multilingual translation model is trained using the multilingual translation training corpus and the multilingual translation verification corpus to obtain a multilingual machine translation model; wherein, through the training of the multilingual translation model, the full parameter sharing of the multilingual translation model is converted into partial parameter sharing and the remaining parameters are language-related, and the language-related parameters are determined by the following steps:

[0103] During each verification in the training process, all language pairs in the multilingual translation verification corpus are clustered into two categories according to the gradient of each parameter of all language pairs in the multilingual translation verification corpus;

[0104] Select a parameter group according to the inter-class distance after two-class clustering of each parameter group, and copy the selected parameter group to obtain a parameter group copy;

[0105] The selected parameter group and the parameter group copy are respectively assigned to the language pairs included in the two categories of the selected parameter group to obtain parameters related to the languages ​​included in the language pairs.

[0106] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the multilingual machine translation model training method provided by the above methods, the method comprising: obtaining a multilingual translation training corpus and a multilingual translation verification corpus, the multilingual translation training corpus being a plurality of language pairs including bilingual parallel data, the multilingual translation verification corpus being a plurality of language pairs including multilingual parallel data;

[0107] A multilingual translation model with full parameter sharing is established and initialized, and the multilingual translation model is trained using the multilingual translation training corpus and the multilingual translation verification corpus to obtain a multilingual machine translation model; wherein, through the training of the multilingual translation model, the full parameter sharing of the multilingual translation model is converted into partial parameter sharing and the remaining parameters are language-related, and the language-related parameters are determined by the following steps:

[0108] During each verification in the training process, all language pairs in the multilingual translation verification corpus are clustered into two categories according to the gradient of each parameter of all language pairs in the multilingual translation verification corpus;

[0109] Select a parameter group according to the inter-class distance after two-class clustering of each parameter group, and copy the selected parameter group to obtain a parameter group copy;

[0110] The selected parameter group and the parameter group copy are respectively assigned to the language pairs included in the two categories of the selected parameter group to obtain parameters related to the languages ​​included in the language pairs.

[0111] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multilingual machine translation model training method, characterized in that: include: Acquire a multilingual translation training corpus and a multilingual translation verification corpus, wherein the multilingual translation training corpus is a plurality of language pairs including bilingual parallel data, and the multilingual translation verification corpus is a plurality of language pairs including multilingual parallel data; A multilingual translation model with full parameter sharing is established and initialized, and the multilingual translation model is trained using the multilingual translation training corpus and the multilingual translation verification corpus to obtain a multilingual machine translation model; wherein, through the training of the multilingual translation model, the full parameter sharing of the multilingual translation model is converted into partial parameter sharing and the remaining parameters are language-related, and the language-related parameters are determined by the following steps: During each verification in the training process, all language pairs in the multilingual translation verification corpus are clustered into two categories according to the gradient of each parameter of all language pairs in the multilingual translation verification corpus; Select a parameter group according to the inter-class distance after two-class clustering of each parameter group, and copy the selected parameter group to obtain a parameter group copy; Assigning the selected parameter group and the parameter group copy to the language pairs included in the two categories of the selected parameter group respectively, to obtain parameters related to the languages ​​included in the language pairs; The step of performing binary clustering on all language pairs according to the gradient of each parameter of all language pairs in the multilingual translation verification corpus comprises the following steps: Obtaining the gradient of each parameter for each language pair in the multilingual translation verification corpus, dividing all parameters of the multilingual translation model into multiple parameter groups, and expanding the gradients of all language pairs in each parameter group into a gradient vector; The cosine similarity of the gradient vectors of all the language pairs in each parameter group is calculated, and the language pairs of each parameter group are clustered into two categories according to the cosine similarity of the gradient vectors of all the language pairs.

2. The multilingual machine translation model training method according to claim 1, characterized in that: The step of selecting a parameter group according to the inter-class distance after clustering each parameter group into two classes specifically includes the following steps: The inter-class distances after two-class clustering of each parameter group are obtained and sorted, and the K parameter groups with the largest inter-class distances are selected, where K is a positive integer.

3. The multilingual machine translation model training method according to claim 1, characterized in that: The language pair includes source language sentences and target language sentences. The source language and target language of the language pair containing bilingual parallel data are both one, and the source language and target language of the language pair containing multilingual parallel data are both one or more.

4. A multilingual translation method, characterized in that: include: Obtain source language sentences to be translated; The multilingual machine translation model trained by the multilingual machine translation model training method according to any one of claims 1 to 3 is used to translate the source language sentence to be translated.

5. A multilingual machine translation model training device, characterized in that: include: A collection unit, the collection unit is used to collect language translation training corpus and multilingual translation verification corpus, the multilingual translation training corpus is a plurality of language pairs containing bilingual parallel data, and the multilingual translation verification corpus is a plurality of language pairs containing multilingual parallel data; A training unit, the training unit is used to establish and initialize a multilingual translation model with full parameter sharing, and train the multilingual translation model using the multilingual translation training corpus and the multilingual translation verification corpus to obtain a multilingual machine translation model; wherein, through the training of the multilingual translation model, the full parameter sharing of the multilingual translation model is converted into partial parameter sharing and the remaining parameters are language-related, and the language-related parameters are determined by the following steps: During each verification in the training process, all language pairs in the multilingual translation verification corpus are clustered into two categories according to the gradient of each parameter of all language pairs in the multilingual translation verification corpus; Select a parameter group according to the inter-class distance after two-class clustering of each parameter group, and copy the selected parameter group to obtain a parameter group copy; Assigning the selected parameter group and the parameter group copy to the language pairs included in the two categories of the selected parameter group respectively, to obtain parameters related to the languages ​​included in the language pairs; The step of performing binary clustering on all language pairs according to the gradient of each parameter of all language pairs in the multilingual translation verification corpus comprises the following steps: Obtaining the gradient of each parameter for each language pair in the multilingual translation verification corpus, dividing all parameters of the multilingual translation model into multiple parameter groups, and expanding the gradients of all language pairs in each parameter group into a gradient vector; The cosine similarity of the gradient vectors of all the language pairs in each parameter group is calculated, and the language pairs of each parameter group are clustered into two categories according to the cosine similarity of the gradient vectors of all the language pairs.

6. A multi-language translation device, characterized in that: include: A source language acquisition unit, the source language acquisition unit is used to acquire source language sentences to be translated; A translation unit, wherein the translation unit is used to obtain a multilingual machine translation model trained according to the multilingual machine translation model training method according to any one of claims 1 to 3, and translate the source language sentence to be translated.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the multilingual machine translation model training method according to any one of claims 1 to 3 or the multilingual translation method according to claim 4 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multilingual machine translation model training method according to any one of claims 1 to 3 or the multilingual translation method according to claim 4 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the multilingual machine translation model training method according to any one of claims 1 to 3 or the multilingual translation method according to claim 4 are implemented.

Images