Language translation model training method, electronic device, and storage medium

Abstract

The application provides a language translation model training method, an electronic device and a storage medium, and relates to the technical field of machine learning. The method inputs a pseudo-parallel training sample set equivalent to the second training sample set into a forward first translation model, that is, a high-resource translation model, while training a second translation model, that is, a low-resource translation model, using the second training sample set, and guides the training of the initial second model according to the prediction result of the pseudo-parallel source language of the forward first translation model. Since the translation performance of the high-resource translation model is better, the translation prediction result of the high-resource translation model is used to guide the translation prediction result of the low-resource translation model, so that the information migration of the high-resource translation model is more comprehensive. Moreover, the loss information is calculated by combining the prediction results of the low-resource translation model and the high-resource translation model, the amount of information contained in the loss information is richer, and the problem of overfitting in the training process of the low-resource translation model is alleviated.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and more specifically, to a method for training a language translation model, an electronic device, and a storage medium. Background Technology

[0002] Machine translation refers to the process of automatically converting one natural language (hereinafter referred to as the source language) into another natural language (hereinafter referred to as the target language) using computer technology. Currently, the translation models trained have achieved relatively satisfactory performance in high-resource translation scenarios. However, due to the scarcity of source languages ​​in low-resource translation scenarios, the performance of translation models in these scenarios is poor.

[0003] In existing technologies, the network parameters of a high-resource translation model are transferred to a low-resource translation model to enhance the translation performance of the low-resource translation model.

[0004] However, the above methods focus on the parameter transfer process, and the high-resource translation model does not provide continuous guidance for low-resource translation, resulting in poor translation performance of the low-resource translation model. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing a language translation model training method, electronic device, and storage medium, so as to facilitate the training of a low-resource translation model with better translation performance.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, embodiments of this application provide a language translation model training method, including:

[0008] A first training sample set and a second training sample set are collected respectively. The first training sample set includes multiple first corpus pairs, each first corpus pair including a first source language and a first target language. The first source language and the first target language are different languages, and both the first source language and the first target language are high-resource languages. The second training sample set includes multiple second corpus pairs, each second corpus pair including a second source language and a second target language. The second source language is a low-resource language, and the second target language is the same language as the first target language.

[0009] The first forward translation model and the first reverse translation model are trained using the first training sample set, respectively.

[0010] Based on the second target language in the second training sample set and the reverse first translation model, a pseudo-parallel training sample set is generated. The pseudo-parallel training sample set includes pseudo-parallel corpus pairs, and each pseudo-parallel corpus pair includes a pseudo-parallel source language and a second target language.

[0011] Based on the first forward translation model, an initial second model is generated;

[0012] The second training sample set is input into the initial second model to obtain the second prediction result of the initial second model. Based on the first prediction result of the positive first translation model for the pseudo-parallel corpus pair and the second prediction result, the loss information of the initial second model is determined, and a second translation model is trained and generated based on the loss information.

[0013] Optionally, the step of inputting the second training sample set into the initial second model to obtain the second prediction result of the initial second model, and determining the loss information of the initial second model based on the first prediction result of the positive first translation model for the pseudo-parallel corpus pair and the second prediction result, and training and generating the second translation model based on the loss information, includes:

[0014] Input the second training sample set into the initial second model to obtain the prediction probability of each predicted second target language corresponding to each second source language output by the initial second model;

[0015] The pseudo-parallel training sample set is input into the forward first translation model to obtain the prediction probability of each predicted target language corresponding to each pseudo-parallel source language output by the forward first translation model.

[0016] The loss information is calculated based on the prediction probability of each predicted second target language, the prediction probability of each predicted target language, and the cross-entropy loss function of the initial second model.

[0017] Based on the loss information, the initial second model is corrected and iterated until the corrected second model meets the preset conditions. The second model that meets the preset conditions is then used as the second translation model.

[0018] Optionally, the step of calculating the loss information based on the prediction probabilities of each predicted second target language, the prediction probabilities of each predicted target language, and the cross-entropy loss function of the initial second model includes:

[0019] The prediction probability distribution of the initial second model is determined based on the prediction probabilities of each predicted second target language.

[0020] The prediction probability distribution of the forward first translation model is determined based on the prediction probability of each target language.

[0021] Based on the predicted probability distribution of the initial second model and the predicted probability distribution of the forward first translation model, the divergence is calculated to obtain the consistency loss function;

[0022] The loss information is calculated based on the consistency loss function and the cross-entropy loss function of the initial second model.

[0023] Optionally, the step of training the first forward first translation model and the first backward first translation model using the first training sample set includes:

[0024] The first source language in the first training sample set is used as the source language of the forward first model, and the first target language is used as the target language of the forward first model. The forward first translation model is trained to obtain the forward first translation model. The forward first translation model is used to translate a language of the same language as the first source language into a language of the same language as the first target language.

[0025] The first source language in the first training sample set is used as the target language of the inverse first model, and the first target language is used as the source language of the inverse first model to train the inverse first translation model. The inverse first translation model is used to translate a language of the same kind as the first target language into a language of the same kind as the first source language.

[0026] Optionally, generating pseudo-parallel corpus pairs based on the second target language in the second training sample set and the reverse first translation model includes:

[0027] Each second target language in the second training sample set is input into the reverse first translation model to generate each pseudo-parallel source language that is semantically equivalent to each second source language.

[0028] The pseudo-parallel training sample set is generated based on each pseudo-parallel source language and each second target language.

[0029] Optionally, before training the first forward first translation model and the first backward first translation model using the first training sample set, the following steps are included:

[0030] The first source language and the first target language in the first training sample set are segmented into words, and the first source word list and the first target word list are generated based on the segmentation results.

[0031] The second source language and the second target language in the second training sample set are segmented into words, and the second source end word list and the second target end word list are generated based on the segmentation results.

[0032] Optionally, the forward first translation model includes: a source word embedding layer, an encoding layer, and a decoding layer; generating an initial second model based on the forward first translation model includes:

[0033] The network parameters of the encoding and decoding layers of the first forward translation model are transferred to the second model;

[0034] Based on the first source-end vocabulary, the second source-end vocabulary, and the mapping relationship between sub-words and word embedding vectors contained in the source-end word embedding layer of the first forward translation model, the source-end word embedding layer of the second model is generated.

[0035] The current second model is used as the initial second model.

[0036] Optionally, generating the source word embedding layer of the second model based on the mapping relationship between sub-words and word embedding vectors contained in the source word embedding layer of the first source word lexicon, the second source word lexicon, and the source word embedding layer in the first forward translation model includes:

[0037] Match the first source terminology and the second source terminology to determine the target sub-words that are the same in the first source terminology and the second source terminology;

[0038] Based on each target sub-word and the correspondence between the sub-words and word embedding vectors contained in the source word embedding layer of the forward first translation model, the target word embedding vector corresponding to each target sub-word is determined.

[0039] Based on the target word embedding vector corresponding to each target word, the mapping relationship between each target word and the target word embedding vector in the source word embedding layer of the second model is generated.

[0040] Secondly, embodiments of this application also provide a language translation model training device, including: a data acquisition module, a training module, and a generation module;

[0041] The acquisition module is used to acquire a first training sample set and a second training sample set respectively. The first training sample set includes multiple first corpus pairs, each first corpus pair including a first source language and a first target language. The first source language and the first target language are different languages, and both the first source language and the first target language are high-resource languages. The second training sample set includes multiple second corpus pairs, each second corpus pair including a second source language and a second target language. The second source language is a low-resource language, and the second target language is the same language as the first target language.

[0042] The training module is used to train the first forward translation model and the first reverse translation model using the first training sample set, respectively.

[0043] The generation module is used to generate a pseudo-parallel training sample set based on the second target language in the second training sample set and the reverse first translation model. The pseudo-parallel training sample set includes pseudo-parallel corpus pairs, and each pseudo-parallel corpus pair includes a pseudo-parallel source language and a second target language.

[0044] The generation module is used to generate an initial second model based on the forward first translation model;

[0045] The training module is used to input the second training sample set into the initial second model to obtain the second prediction result of the initial second model, and to determine the loss information of the initial second model based on the first prediction result of the positive first translation model for the pseudo-parallel corpus pair and the second prediction result, and to train and generate a second translation model based on the loss information.

[0046] Optionally, the training module is specifically used to input the second training sample set into the initial second model to obtain the prediction probability of each predicted second target language corresponding to each second source language output by the initial second model;

[0047] The pseudo-parallel training sample set is input into the forward first translation model to obtain the prediction probability of each predicted target language corresponding to each pseudo-parallel source language output by the forward first translation model.

[0048] The loss information is calculated based on the prediction probability of each predicted second target language, the prediction probability of each predicted target language, and the cross-entropy loss function of the initial second model.

[0049] Based on the loss information, the initial second model is corrected and iterated until the corrected second model meets the preset conditions. The second model that meets the preset conditions is then used as the second translation model.

[0050] Optionally, the training module is specifically used to determine the prediction probability distribution of the initial second model based on the prediction probabilities of each predicted second target language;

[0051] The prediction probability distribution of the forward first translation model is determined based on the prediction probability of each target language.

[0052] Based on the predicted probability distribution of the initial second model and the predicted probability distribution of the forward first translation model, the divergence is calculated to obtain the consistency loss function;

[0053] The loss information is calculated based on the consistency loss function and the cross-entropy loss function of the initial second model.

[0054] Optionally, the training module is specifically used for:

[0055] The first source language in the first training sample set is used as the source language of the forward first model, and the first target language is used as the target language of the forward first model. The forward first translation model is trained to obtain the forward first translation model. The forward first translation model is used to translate a language of the same language as the first source language into a language of the same language as the first target language.

[0056] The first source language in the first training sample set is used as the target language of the inverse first model, and the first target language is used as the source language of the inverse first model to train the inverse first translation model. The inverse first translation model is used to translate a language of the same kind as the first target language into a language of the same kind as the first source language.

[0057] Optionally, the generation module is specifically used to input each second target language in the second training sample set into the reverse first translation model to generate each pseudo-parallel source language that is semantically equivalent to each second source language.

[0058] The pseudo-parallel training sample set is generated based on each pseudo-parallel source language and each second target language.

[0059] Optionally, the generation module is further configured to perform word segmentation on each first source language and each first target language in the first training sample set, and generate a first source-end vocabulary and a first target-end vocabulary based on the word segmentation results;

[0060] The second source language and the second target language in the second training sample set are segmented into words, and the second source end word list and the second target end word list are generated based on the segmentation results.

[0061] Optionally, the generation module is specifically used to transfer the network parameters of the encoding and decoding layers of the first forward translation model to the second model;

[0062] Based on the first source-end vocabulary, the second source-end vocabulary, and the mapping relationship between sub-words and word embedding vectors contained in the source-end word embedding layer of the first forward translation model, the source-end word embedding layer of the second model is generated.

[0063] The current second model is used as the initial second model.

[0064] Optionally, the generation module is specifically used for

[0065] Match the first source terminology and the second source terminology to determine the target sub-words that are the same in the first source terminology and the second source terminology;

[0066] Based on each target sub-word and the correspondence between the sub-words and word embedding vectors contained in the source word embedding layer of the forward first translation model, the target word embedding vector corresponding to each target sub-word is determined.

[0067] Based on the target word embedding vector corresponding to each target word, the mapping relationship between each target word and the target word embedding vector in the source word embedding layer of the second model is generated.

[0068] Thirdly, embodiments of this application provide an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the language translation model training method provided in the first aspect.

[0069] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the language translation model training method provided in the first aspect.

[0070] The beneficial effects of this application are:

[0071] This application provides a language translation model training method, electronic device, and storage medium. The method trains a second translation model (i.e., a low-resource translation model) using a second training sample set, while simultaneously inputting a pseudo-parallel training sample set equivalent to the second training sample set into a forward first translation model (i.e., a high-resource translation model). The training of the initial second model is guided by the prediction results of the forward first translation model for the pseudo-parallel source languages. Since the pseudo-parallel training sample set is equivalent to the second training sample set, it effectively expands the training sample set of the low-resource translation model. Furthermore, because the high-resource translation model has better translation performance, guiding the translation prediction results of the low-resource translation model with the translation prediction results of the high-resource translation model makes the information transfer from the high-resource translation model more comprehensive. Moreover, by combining the prediction results of the low-resource and high-resource translation models to calculate loss information, the loss information contains richer information, alleviating the overfitting problem in the training process of the low-resource translation model, resulting in a low-resource translation model with better translation performance. Attached Figure Description

[0072] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0073] Figure 1 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 1 ;

[0074] Figure 2 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 2 ;

[0075] Figure 3 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 3 ;

[0076] Figure 4 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 4 ;

[0077] Figure 5 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 5 ;

[0078] Figure 6 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 6 ;

[0079] Figure 7 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 7 ;

[0080] Figure 8 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 8 ;

[0081] Figure 9 A schematic diagram of a language translation model training system provided in an embodiment of this application;

[0082] Figure 10 A schematic diagram of a language translation model training device provided in an embodiment of this application;

[0083] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0084] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0085] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0086] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0087] First, let's briefly explain high-resource languages ​​and low-resource languages. High-resource languages ​​refer to non-scarce languages, that is, languages ​​with a large number of speakers. They have a large amount of corpus, so machine translation models trained in high-resource language scenarios have better training samples, and the translation performance of the trained high-resource translation models is also better. Examples of high-resource languages ​​include English, German, and Chinese.

[0088] Conversely, low-resource languages ​​refer to scarce languages, spoken by a small number of people. Their corpora are relatively scarce and lack diversity, making them unsuitable for training low-resource translation models. Such models are prone to overfitting. Examples of low-resource languages ​​include Turkish and Tujia.

[0089] Based on this, this application uses the model parameters of the high-resource translation model obtained from training to initialize the low-resource translation model. At the same time, when training the low-resource translation model, the prediction results of the high-resource translation model are combined to continuously guide the optimization of the network parameters of the low-resource translation model, thereby achieving the goal of enhancing the translation performance of the low-resource translation model.

[0090] Figure 1 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 1 ;like Figure 1 As shown, the method may include:

[0091] S101. Collect a first training sample set and a second training sample set respectively. The first training sample set includes multiple first corpus pairs. Each first corpus pair includes a first source language and a first target language. The first source language and the first target language are different languages, and both the first source language and the first target language are high-resource languages. The second training sample set includes multiple second corpus pairs. Each second corpus pair includes a second source language and a second target language. The second source language is a low-resource language, and the second target language is the same language as the first target language.

[0092] In this embodiment, the first training sample set can refer to a high-resource language sample set, and the second training sample set refers to a low-resource language sample set. The first training sample set contains multiple first corpus pairs. Each corpus pair can include a first source language and a first target language. The source language can refer to the original language to be translated, and the target language can refer to the language to which the source language is to be translated. It is worth noting that the source language and the target language are of different languages, and both the first source language and the first target language are high-resource languages, which can be extracted from a high-resource corpus according to their language type.

[0093] The second training sample set can refer to a low-resource language sample set, which contains multiple second corpus pairs. A corpus pair can include a second source language and a second target language. Similarly, the source language and the target language are different languages. The second source language is a low-resource language, while the second target language can be a high-resource language. In one possible embodiment, the second target language is the same language as the first target language.

[0094] Since a high-resource translation model is needed to guide the training of a low-resource translation model, the translation goals of both models must be the same. Only when the high-resource translation model guides the low-resource translation model based on the same translation goal can the guidance from the high-resource translation model be meaningful. Therefore, the first and second target languages ​​mentioned above can be set to the same language.

[0095] S102. The first forward translation model and the first backward translation model are trained using the first training sample set.

[0096] Optionally, based on the collected first training samples, a forward first translation model and a reverse first translation model can be trained respectively. In this embodiment, the first translation model can be a high-resource translation model, and the forward first translation model and the reverse first translation model can be two models with the same model architecture but opposite translations. Then the model parameters of the forward first translation model and the reverse first translation model will also be different.

[0097] If the forward first translation model is: German to English, then the reverse first translation model is: English to German.

[0098] S103. Based on the second target language in the second training sample set and the reverse first translation model, generate a pseudo-parallel training sample set. The pseudo-parallel training sample set includes pseudo-parallel corpus pairs, and each pseudo-parallel corpus pair includes pseudo-parallel source language and second target language.

[0099] In some embodiments, the second target language in the second training sample set can be used as the input to the reverse first translation model to obtain the pseudo-parallel source language output by the reverse first translation model.

[0100] Optionally, as explained above, if the first training sample set is a high-resource language sample set, then it can be assumed that the first source language in the first training sample set is German and the first target language is English; if the second training sample set is a low-resource language sample set, then it can be assumed that the second source language in the second training sample set is a local language, and the second target language is English, which is the same language as the first target language.

[0101] Therefore, the forward first translation model generated using the first training sample set is used to translate German into English, while the reverse first translation model is used to translate English into German. Thus, based on the second target language, English, the pseudo-parallel source language generated using the reverse first translation model is German, which corresponds to English.

[0102] It's important to explain here that since the second training sample set is a low-resource language sample set, and the second target language is a language semantically equivalent to the second source language, the pseudo-parallel source language generated based on the second target language in the second training sample set is semantically equivalent to the second source language. The generated pseudo-parallel training sample set can be understood as a sample set semantically equivalent to the second training sample set; hence, it's called the pseudo-parallel training sample set.

[0103] For example, the corpus in the second training sample set contains a dialect-English pair. The reverse first translation model is used to translate English into German. Based on the English in the second training sample set, the pseudo-parallel source language generated by the reverse first translation model is the German corresponding to the English in the second training sample set. Since the dialect in the first training sample set is semantically equivalent to English, and the pseudo-parallel source language German generated by the reverse first translation model is semantically equivalent to English in the first training sample set, it can be considered that the dialect in the first training sample set is semantically equivalent to the pseudo-parallel source language generated by the reverse first translation model. Since the first training sample set and the pseudo-parallel training sample set generated by the reverse first translation model are equivalent sample sets.

[0104] S104. Generate the initial second model based on the forward first translation model.

[0105] Optionally, the second model can refer to the low-resource translation model to be trained. In the initial state, the network parameters of the second model are empty. Based on the model parameters of the forward first translation model obtained through training, the network parameters of the second model can be initialized, thereby generating the initial second model.

[0106] S105. Input the second training sample set into the initial second model to obtain the second prediction result of the initial second model. Based on the first and second prediction results of the positive first translation model for the pseudo-parallel corpus pairs, determine the loss information of the initial second model. Based on the loss information, train and generate the second translation model.

[0107] Optionally, the second training sample set can be input into the initial second model to train the network parameters of the initial second model and obtain the second prediction result output by the initial second model. At the same time, the pseudo-parallel training sample set can be input into the forward first translation model to obtain the first prediction result output by the forward first translation model. By combining the first prediction result and the second prediction result, the loss information of the initial second model can be calculated. Based on the loss information, the network parameters of the initial second model can be continuously trained. The initial second model at the end of training is then used as the second translation model, which can refer to the low-resource translation model.

[0108] While training the second translation model (i.e., the low-resource translation model) using the second training sample set, a pseudo-parallel training sample set equivalent to the second training sample set is input into the forward first translation model (i.e., the high-resource translation model). The training of the initial second model is guided by the prediction results of the forward first translation model for the pseudo-parallel source languages. Since the pseudo-parallel training sample set is equivalent to the second training sample set, it is equivalent to expanding the training sample set of the low-resource translation model. Furthermore, since the high-resource translation model has better translation performance, guiding the translation prediction results of the low-resource translation model with the translation prediction results of the high-resource translation model makes the information transfer of the high-resource translation model more comprehensive. Moreover, by combining the prediction results of the low-resource translation model and the high-resource translation model to calculate the loss information, the loss information contains richer information, which alleviates the overfitting problem in the training process of the low-resource translation model, resulting in better translation performance of the trained low-resource translation model.

[0109] In summary, the language translation model training method provided in this embodiment trains the second translation model (i.e., the low-resource translation model) using the second training sample set, while simultaneously inputting a pseudo-parallel training sample set equivalent to the second training sample set into the forward first translation model (i.e., the high-resource translation model). The training of the initial second model is guided by the prediction results of the forward first translation model for the pseudo-parallel source languages. Since the pseudo-parallel training sample set is equivalent to the second training sample set, it effectively expands the training sample set of the low-resource translation model. Furthermore, because the high-resource translation model has better translation performance, guiding the translation prediction results of the low-resource translation model with the translation prediction results of the high-resource translation model makes the information transfer from the high-resource translation model more comprehensive. Moreover, by combining the prediction results of the low-resource translation model and the high-resource translation model to calculate loss information, the loss information contains richer information, alleviating the overfitting problem in the training process of the low-resource translation model, resulting in better translation performance of the trained low-resource translation model.

[0110] Figure 2 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 2 In step S105 above, the second training sample set is input into the initial second model to obtain the second prediction result of the initial second model. Based on the first and second prediction results of the positive first translation model for the pseudo-parallel corpus pairs, the loss information of the initial second model is determined. Based on the loss information, the second translation model is trained and generated, which may include:

[0111] S201. Input the second training sample set into the initial second model to obtain the prediction probability of each predicted second target language corresponding to each second source language output by the initial second model.

[0112] Optionally, after the second training sample set is input into the initial second model, the initial second model can perform translation prediction for the second source language in each second corpus pair. The prediction result for the second source language can be determined based on the second target language corresponding to the second source language in each second corpus pair. The prediction result here is the prediction probability, which can be understood as the probability of predicting the target word in the second target language corresponding to each word in each second source language.

[0113] S202. Input the pseudo-parallel training sample set into the forward first translation model to obtain the prediction probability of each predicted target language corresponding to each pseudo-parallel source language output by the forward first translation model.

[0114] Similarly, the pseudo-parallel training sample set can be input into the forward first translation model. The forward first translation model can perform translation prediction for the pseudo-parallel source language in each pseudo-parallel training sample set. In this model, the probability of predicting the target word in the second target language corresponding to the pseudo-parallel source language can be determined based on the second target language corresponding to the pseudo-parallel source language in each second corpus pair.

[0115] S203. Based on the prediction probabilities of each predicted second target language, the prediction probabilities of each predicted target language, and the cross-entropy loss function of the initial second model, the loss information is calculated.

[0116] Optionally, unlike the traditional calculation of loss functions, in this embodiment, in addition to calculating the cross-entropy loss function of the initial second model, the prediction probabilities of each predicted second target language and the prediction probabilities of each predicted target language can be combined to calculate loss information, where loss information can refer to the loss function value.

[0117] S204. Based on the loss information, the initial second model is modified and iterated until the modified second model meets the preset conditions. The second model that meets the preset conditions is then used as the second translation model.

[0118] Typically, model training requires multiple rounds of iterative training. For the current round of training, the loss information calculated in the current round can be used to determine whether the loss information meets the preset accuracy threshold. If it does not meet the threshold, the network parameters of the current initial second model are modified. Based on the modified second model, the loss information of the corrected second model is recalculated in the above manner until the loss information meets the preset accuracy threshold. Then, the modification of the network parameters is stopped, and the current second model at the time of stopping the modification is used as the second translation model.

[0119] In one possible implementation, in this embodiment, when iteratively correcting network parameters, the condition for stopping the correction of network parameters is to determine whether the number of iterations meets the preset number. For example, the number of iterations is set to 200. When the number of iterations reaches 200, the correction ends.

[0120] Figure 3 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 3 In step S203, loss information is calculated based on the prediction probabilities of each predicted second target language, the prediction probabilities of each predicted target language, and the cross-entropy loss function of the initial second model. This information may include:

[0121] S301. Determine the initial prediction probability distribution of the second model based on the prediction probabilities of each predicted second target language.

[0122] In this embodiment, the initial prediction probability distribution of the second model can be a probability vector, which is calculated by combining the prediction probabilities of each predicted second target language.

[0123] S302. Determine the prediction probability distribution of the forward first translation model based on the prediction probabilities of each target language.

[0124] Similarly, the prediction probability distribution of the first forward translation model can be obtained from the prediction probabilities of each target language, which is also a probability vector.

[0125] S303. Based on the predicted probability distribution of the initial second model and the predicted probability distribution of the forward first translation model, the divergence is calculated to obtain the consistency loss function.

[0126] Assume the initial prediction probability distribution of the second model is P c (*|x a ,θ c The predicted probability distribution of the first forward translation model is: Therefore, the consistency loss function can be calculated using the following formula:

[0127]

[0128] Here, JS represents the JS divergence, which measures the difference between two distributions.

[0129] S304. Based on the consistency loss function and the initial cross-entropy loss function of the second model, the loss information is calculated.

[0130] The calculation of the cross-entropy loss function of the initial second model is the same as the calculation of the conventional damage function. Based on the predicted second target language and the second target language output by the initial second model, the cross-entropy loss function calculation formula is used to calculate the cross-entropy loss function of the initial second model.

[0131] Assume the calculated initial cross-entropy loss function of the second model is L. nll Therefore, the loss information can be calculated using the following formula:

[0132] L = L nll +αL d

[0133] Among them, L d Let α be the consistency loss function calculated above, where α is a hyperparameter that can be adjusted manually.

[0134] Figure 4 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 4 Optionally, in step S102, training the forward first translation model and the reverse first translation model using the first training sample set may include:

[0135] S401. Take each first source language in the first training sample set as the source language of the forward first model, and take each first target language as the target language of the forward first model to train the forward first translation model. The forward first translation model is used to translate a language of the same kind as the first source language into a language of the same kind as the first target language.

[0136] When training the first forward translation model, the first source language in the first training sample set is used as the source language of the first forward model, and the first target language is used as the target language of the first forward model. The model obtained at the end of training is the first forward translation model.

[0137] The forward first model can be understood as the forward first translation model when the network parameters are empty, that is, the forward first translation model to be trained. After the forward first model is successfully trained, it will be used as the forward first translation model.

[0138] S402. Take each first source language in the first training sample set as the target language of the reverse first model, and take each first target language as the source language of the reverse first model to train the reverse first translation model. The reverse first translation model is used to translate a language of the same language as the first target language into a language of the same language as the first source language.

[0139] When the first source language in the first training sample set is used as the target language of the inverse first model, and the first target language is used as the source language of the inverse first model, the resulting model is the inverse first translation model.

[0140] For example, if the first source language in the first training sample set is German and the first target language is English, then the trained forward first translation model is a German-to-English translation model used to translate German into English, while the trained reverse first translation model is an English-to-German translation model used to translate English into German.

[0141] Figure 5 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 5 In step S103, generating pseudo-parallel corpus pairs based on the second target language in the second training sample set and the reverse first translation model may include:

[0142] S501. Input each second target language in the second training sample set into the reverse first translation model to generate each pseudo-parallel source language that is semantically equivalent to each second source language.

[0143] Optionally, each second target language in the second training sample set can be used as the source language of the reverse first translation model and input into the reverse first translation model. The output of the reverse first translation model is a pseudo-parallel source language. Then, the corresponding pseudo-parallel source language can be obtained by translating each second target language.

[0144] S502. Generate a pseudo-parallel training sample set based on each pseudo-parallel source language and each second target language.

[0145] Optionally, by combining the second target language with the pseudo-parallel source language, a pseudo-parallel training sample set can be obtained. Each second target language and its corresponding pseudo-parallel source language can be combined into a pseudo-parallel corpus pair, thus obtaining all pseudo-parallel corpus pairs in the pseudo-parallel training sample set.

[0146] Figure 6 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 6 Optionally, in S102, before training the forward first translation model and the reverse first translation model using the first training sample set respectively, the following may be included:

[0147] S601. Perform word segmentation on each first source language and each first target language in the first training sample set, and generate the first source end word list and the first target end word list based on the word segmentation results.

[0148] In some embodiments, after collecting the first training sample set, preprocessing can be performed on each first source language and each first target language in the first training sample set, which may include word segmentation and sub-word segmentation.

[0149] Optionally, a corresponding word segmentation tool can be used for word segmentation based on the specific word segmentation method of each language. For example, specific word segmentation tools can be used for languages ​​with specific word segmentation methods, such as Chinese and Japanese, while other languages ​​such as English can be segmented using mosesdecoder (a phrase-based statistical machine translation system).

[0150] Based on the words obtained after word segmentation, the subword-BPE (word-pair separation algorithm) technology can be used to further segment the words into sub-words, thereby obtaining the first source word list and the first target word list.

[0151] The first source-end vocabulary is the vocabulary obtained by removing duplicate word roots from the smallest units obtained after segmenting each first source language sub-word, while the first target-end vocabulary is the vocabulary obtained by removing duplicate word roots from the smallest units obtained after segmenting each first target language sub-word.

[0152] Due to some language and grammar issues, word segmentation may only be based on punctuation marks and cannot truly separate the words in a sentence down to the smallest unit of word roots. However, sub-word segmentation can achieve more refined segmentation of the smallest unit of word roots.

[0153] S602. Perform word segmentation on each second source language and each second target language in the second training sample set, and generate a second source end word list and a second target end word list based on the word segmentation results.

[0154] Similarly, the second training sample set can be processed using the same preprocessing method as the first training sample set for word segmentation and sub-word splitting, thereby obtaining the second source-end vocabulary and the second target-end vocabulary.

[0155] For example: Suppose the first source language in the first training sample set is "about your the your", and it is segmented into "ab", "out you", "r the you", and "r". Then, the current first source terminology is: "ab", "out you", and "rthe".

[0156] In one feasible approach, before performing subword segmentation using the subword-BPE technique, a BPE model for subword segmentation on a first training sample set can be trained. In this approach, each first source language and each first target language in the first training sample set can be used as a training sample to train and obtain the BPE model for subword segmentation on the first training sample set.

[0157] Similarly, each second source language and each second target language in the second training sample set is used as a training sample to train and obtain a BPE model for word segmentation of the second training sample set.

[0158] This can be understood as follows: the training samples used to train the BPE model for word segmentation are actually the same languages ​​contained in the first or second training sample set. However, when training the BPE model, the corpus pairs are not combined again; each independent language is used as an independent training sample.

[0159] Figure 7 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 7 Optionally, the aforementioned forward first translation model may include: a source word embedding layer, an encoding layer, and a decoding layer; in step S104, generating an initial second model based on the forward first translation model may include:

[0160] S701. Transfer the network parameters of the encoding and decoding layers of the first forward translation model to the second model.

[0161] It should be noted that the source word embedding layer matches the corresponding vectors of each word in the source vocabulary of the input model, generating a semantic vector corresponding to the input source language. In other words, it converts the input text language into machine language that the computer can process. The encoding layer extracts feature vectors from the semantic vectors generated by the source word embedding layer, obtaining semantic feature vectors that can be used to represent the source language. That is, it extracts information from the semantic vectors output by the source word embedding layer. The decoding layer then transforms the semantic feature vectors output by the encoding layer to generate the translation results corresponding to the semantic feature vectors.

[0162] Optionally, for the encoding and decoding layers, the network parameters of the encoding and decoding layers of the forward first translation model can be directly copied into the second model to generate the initial second model.

[0163] S702. Based on the first source-end vocabulary, the second source-end vocabulary, and the mapping relationship between sub-words and word embedding vectors contained in the source-end word embedding layer of the forward first translation model, generate the source-end word embedding layer of the second model.

[0164] As mentioned above, the source word embedding layer is used to match the vectors corresponding to each word in the vocabulary. That is, the source word embedding layer maintains the mapping relationship between a vocabulary and vectors. The transfer of network parameters of the source word embedding layer can refer to the mapping relationship between the words in the vocabulary and vectors of the source word embedding layer of the second model.

[0165] For the source word embedding layer, the mapping relationship between sub-words and vectors in the vocabulary of the source word embedding layer of the second model can be generated based on the comparison results of the first and second source word lists and the mapping relationship between sub-words and word embedding vectors contained in the source word embedding layer of the first forward translation model.

[0166] S703, Use the current second model as the initial second model.

[0167] Optionally, based on the parameter transfer of the source word embedding layer, encoding layer, and decoding layer described above, the generated second model is the initial second model. At this point, the initial second model already has the initial parameters generated based on the network parameters of the forward first translation model, and the translation performance of the forward first translation model is transferred to the initial second model to a certain extent.

[0168] Figure 8 A flowchart illustrating the language translation model training method provided in this application embodiment. Figure 8 Optionally, in step S702, generating the source word embedding layer of the second model based on the first source word list, the second source word list, and the mapping relationship between sub-words and word embedding vectors contained in the source word embedding layer of the forward first translation model may include:

[0169] S801. Match the first source terminology and the second source terminology to determine the target sub-words that are the same in the first source terminology and the second source terminology.

[0170] This embodiment explains the parameter migration of the source word embedding layer. The first source word list and the second source word list generated above can be compared, and the sub-words that appear in both the first and second source word lists can be identified as target sub-words.

[0171] S802. Based on each target sub-word and the correspondence between the sub-words and word embedding vectors contained in the source word embedding layer of the forward first translation model, determine the target word embedding vector corresponding to each target sub-word.

[0172] For each target subword, the word embedding vector corresponding to each target subword in the forward first translation model can be determined based on the correspondence between the subwords contained in the source word embedding layer of the forward first translation model and the word embedding vector.

[0173] S803. Based on the target word embedding vectors corresponding to each target sub-word, generate the mapping relationship between each target sub-word and the target word embedding vector in the source word embedding layer of the second model.

[0174] Optionally, based on the determined target sub-words and their corresponding word embedding vectors, a mapping relationship between each target sub-word and its word embedding vector can be generated.

[0175] For words other than the target word, that is, for words other than the target word in the second source word list, the mapping relationship between other words and word embedding vectors can be randomly generated according to a normal distribution.

[0176] Based on the initial second model generated from the network parameters of the first forward translation model, the second translation model, i.e., the low-resource translation model, can be trained using the aforementioned training method for the initial second model.

[0177] In some embodiments, after training the second translation model, assuming the second translation model is used to translate the local language into English, the local language to be translated can be used as input to the second translation model to obtain the target English corresponding to the local language to be translated, which is output by the second translation model.

[0178] Figure 9 This is a schematic diagram of a language translation model training system provided in an embodiment of this application. Figure 9 This can fully demonstrate the overall process of this method. For example... Figure 9 As shown, the system may include: a pseudo-parallel training sample set generation part and a part that continuously guides the training process of the second translation model by the first forward translation model.

[0179] In the pseudo-parallel training sample set generation part, it may include a reverse first translation model. The second target language in the second training sample set is used as the input of the reverse first translation model. The reverse first translation model outputs the pseudo-parallel source language. The pseudo-parallel source language and the second target language are used as corpus pairs to obtain the pseudo-parallel training sample set.

[0180] The part where the forward first translation model continuously guides the training process of the second translation model may include a forward first translation model and a second model. The forward first translation model initializes the second model, and then the second training sample set is used as input to the initial second model to train the initial second model. At the same time, the generated pseudo-parallel training sample set is used as input to the forward first translation model to continuously guide the training process of the initial second model.

[0181] Optionally, consistency loss information can be calculated based on the prediction distribution results of the initial second model and the prediction distribution results of the forward first translation model. Cross-entropy loss information of the initial second model can be calculated based on the prediction distribution results of the initial second model. Thus, the final target loss information can be calculated based on the consistency loss information and the cross-entropy loss information. The network parameters of the initial second model can be iteratively corrected based on the target loss information to finally train the second translation model.

[0182] In summary, the language translation model training method provided in this embodiment trains the second translation model (i.e., the low-resource translation model) using the second training sample set, while simultaneously inputting a pseudo-parallel training sample set equivalent to the second training sample set into the forward first translation model (i.e., the high-resource translation model). The training of the initial second model is guided by the prediction results of the forward first translation model for the pseudo-parallel source languages. Since the pseudo-parallel training sample set is equivalent to the second training sample set, it effectively expands the training sample set of the low-resource translation model. Furthermore, because the high-resource translation model has better translation performance, guiding the translation prediction results of the low-resource translation model with the translation prediction results of the high-resource translation model makes the information transfer from the high-resource translation model more comprehensive. Moreover, by combining the prediction results of the low-resource translation model and the high-resource translation model to calculate loss information, the loss information contains richer information, alleviating the overfitting problem in the training process of the low-resource translation model, resulting in better translation performance of the trained low-resource translation model.

[0183] The following describes the apparatus, device, and storage medium used to implement the language translation model training method provided in this application. The specific implementation process and technical effects are described above and will not be repeated below.

[0184] Figure 10 This diagram illustrates a language translation model training device provided in an embodiment of this application. The functions implemented by this language translation model training device correspond to the steps executed by the above-described method. This device can be understood as a server, or a server's processor, or it can be understood as a component independent of the aforementioned server or processor, but implementing the functions of this application under the control of the server, such as... Figure 10 As shown, the device may include: a data acquisition module 110, a training module 120, and a generation module 130;

[0185] The acquisition module 110 is used to acquire a first training sample set and a second training sample set respectively. The first training sample set includes multiple first corpus pairs, each first corpus pair including a first source language and a first target language. The first source language and the first target language are different languages, and both the first source language and the first target language are high-resource languages. The second training sample set includes multiple second corpus pairs, each second corpus pair including a second source language and a second target language. The second source language is a low-resource language, and the second target language is the same language as the first target language.

[0186] Training module 120 is used to train the forward first translation model and the reverse first translation model using the first training sample set respectively;

[0187] The generation module 130 is used to generate a pseudo-parallel training sample set based on the second target language in the second training sample set and the reverse first translation model. The pseudo-parallel training sample set includes pseudo-parallel corpus pairs, and each pseudo-parallel corpus pair includes pseudo-parallel source language and second target language.

[0188] The generation module 130 is used to generate an initial second model based on the forward first translation model;

[0189] The training module 120 is used to input the second training sample set into the initial second model, obtain the second prediction result of the initial second model, and determine the loss information of the initial second model based on the first prediction result and the second prediction result of the positive first translation model for the pseudo-parallel corpus pairs, and train and generate the second translation model based on the loss information.

[0190] Optionally, the training module 120 is specifically used to input the second training sample set into the initial second model to obtain the prediction probability of each predicted second target language corresponding to each second source language output by the initial second model.

[0191] Input the pseudo-parallel training sample set into the forward first translation model to obtain the prediction probability of each predicted target language corresponding to each pseudo-parallel source language output by the forward first translation model.

[0192] The loss information is calculated based on the prediction probability of each second target language, the prediction probability of each target language, and the cross-entropy loss function of the initial second model.

[0193] Based on the loss information, the initial second model is modified and iterated until the modified second model meets the preset conditions. The second model that meets the preset conditions is then used as the second translation model.

[0194] Optionally, the training module 120 is specifically used to determine the prediction probability distribution of the initial second model based on the prediction probabilities of each predicted second target language;

[0195] Based on the prediction probabilities of each target language, determine the prediction probability distribution of the forward first translation model;

[0196] The consistency loss function is obtained by calculating the divergence based on the predicted probability distribution of the initial second model and the predicted probability distribution of the forward first translation model.

[0197] The loss information is calculated based on the consistency loss function and the cross-entropy loss function of the initial second model.

[0198] Optionally, training module 120 is specifically used for:

[0199] The first source language in the first training sample set is used as the source language of the first forward model, and the first target language is used as the target language of the first forward model. The first forward translation model is trained and used to translate a language that is the same as the first source language into a language that is the same as the first target language.

[0200] The first source language in the first training sample set is used as the target language of the reverse first model, and the first target language is used as the source language of the reverse first model. The reverse first translation model is trained to translate a language of the same language as the first target language into a language of the same language as the first source language.

[0201] Optionally, the generation module 130 is specifically used to input each second target language in the second training sample set into the reverse first translation model to generate each pseudo-parallel source language that is semantically equivalent to each second source language.

[0202] Based on each pseudo-parallel source language and each second target language, a pseudo-parallel training sample set is generated.

[0203] Optionally, the generation module 130 is further configured to perform word segmentation on each first source language and each first target language in the first training sample set, and generate a first source-end vocabulary and a first target-end vocabulary based on the word segmentation results;

[0204] The second source language and the second target language in the second training sample set are segmented into words, and the second source end word list and the second target end word list are generated based on the segmentation results.

[0205] Optionally, the generation module 130 is specifically used to transfer the network parameters of the encoding and decoding layers of the forward first translation model to the second model;

[0206] Based on the first source-end vocabulary, the second source-end vocabulary, and the mapping relationship between sub-words and word embedding vectors contained in the source-end word embedding layer of the first forward translation model, the source-end word embedding layer of the second model is generated.

[0207] Use the current second model as the initial second model.

[0208] Optionally, the generation module 130 is specifically used for

[0209] Match the first source terminology and the second source terminology to identify the target sub-words that are the same in the first source terminology and the second source terminology.

[0210] Based on the correspondence between each target word and the word embedding vector contained in the source word embedding layer of the forward first translation model, the target word embedding vector corresponding to each target word is determined.

[0211] Based on the target word embedding vectors corresponding to each target sub-word, the mapping relationship between each target sub-word and the target word embedding vector in the source word embedding layer of the second model is generated.

[0212] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0213] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more digital signal processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0214] The modules described above can be connected or communicate with each other via wired or wireless connections. Wired connections can include metal cables, optical fibers, hybrid cables, or any combination thereof. Wireless connections can include connections via LAN, WAN, Bluetooth, ZigBee, or NFC, or any combination thereof. Two or more modules can be combined into a single module, and any module can be divided into two or more units. Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here.

[0215] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The device may be a computing device with data processing capabilities.

[0216] The device includes: processor 801 and storage medium 802.

[0217] Storage medium 802 is used to store programs, and processor 801 calls the programs stored in storage medium 802 to execute the above method embodiments. The specific implementation and technical effects are similar, and will not be described in detail here.

[0218] The storage medium 802 stores program code, which, when executed by the processor 801, causes the processor 801 to perform various steps in the methods according to various exemplary embodiments of this application described in the "Exemplary Methods" section above.

[0219] The processor 801 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0220] Storage medium 802, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The storage medium can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type storage medium, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage medium, magnetic disk, optical disk, etc. The storage medium is any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, storage medium 802 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0221] Optionally, this application also provides a program product, such as a computer-readable storage medium, including a program that, when executed by a processor, performs the above-described method embodiments.

[0222] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0223] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0224] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0225] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for training a language translation model, characterized in that, include: A first training sample set and a second training sample set are collected respectively. The first training sample set includes multiple first corpus pairs, each first corpus pair including a first source language and a first target language. The first source language and the first target language are different languages, and both the first source language and the first target language are high-resource languages. The second training sample set includes multiple second corpus pairs, each second corpus pair including a second source language and a second target language. The second source language is a low-resource language, and the second target language is the same language as the first target language. The first forward translation model and the first reverse translation model are trained using the first training sample set, respectively. Based on the second target language in the second training sample set and the reverse first translation model, a pseudo-parallel training sample set is generated. The pseudo-parallel training sample set includes pseudo-parallel corpus pairs, and each pseudo-parallel corpus pair includes a pseudo-parallel source language and a second target language. Based on the first forward translation model, an initial second model is generated; the second model is a low-resource translation model to be trained, and in the initial state, the network parameters of the second model are empty; this includes: initializing the network parameters of the second model according to the model parameters of the first forward translation model obtained through training, and generating the initial second model; The second training sample set is input into the initial second model to obtain the second prediction result of the initial second model. Based on the first prediction result of the positive first translation model for the pseudo-parallel corpus pair and the second prediction result, the loss information of the initial second model is determined, and a second translation model is trained and generated based on the loss information.

2. The method according to claim 1, characterized in that, The step of inputting the second training sample set into the initial second model to obtain the second prediction result of the initial second model, and determining the loss information of the initial second model based on the first prediction result of the positive first translation model for the pseudo-parallel corpus pair and the second prediction result, and training and generating a second translation model based on the loss information, includes: Input the second training sample set into the initial second model to obtain the prediction probability of each predicted second target language corresponding to each second source language output by the initial second model; The pseudo-parallel training sample set is input into the forward first translation model to obtain the prediction probability of each predicted target language corresponding to each pseudo-parallel source language output by the forward first translation model. The loss information is calculated based on the prediction probability of each predicted second target language, the prediction probability of each predicted target language, and the cross-entropy loss function of the initial second model. Based on the loss information, the initial second model is corrected and iterated until the corrected second model meets the preset conditions. The second model that meets the preset conditions is then used as the second translation model.

3. The method according to claim 2, characterized in that, The loss information is calculated based on the prediction probabilities of each predicted second target language, the prediction probabilities of each predicted target language, and the cross-entropy loss function of the initial second model, including: The prediction probability distribution of the initial second model is determined based on the prediction probabilities of each predicted second target language. The prediction probability distribution of the forward first translation model is determined based on the prediction probability of each target language. Based on the predicted probability distribution of the initial second model and the predicted probability distribution of the forward first translation model, the divergence is calculated to obtain the consistency loss function; The loss information is calculated based on the consistency loss function and the cross-entropy loss function of the initial second model.

4. The method according to claim 1, characterized in that, The step of training a forward first translation model and a reverse first translation model using the first training sample set includes: The first source language in the first training sample set is used as the source language of the forward first model, and the first target language is used as the target language of the forward first model. The forward first translation model is trained to obtain the forward first translation model. The forward first translation model is used to translate a language of the same language as the first source language into a language of the same language as the first target language. The first source language in the first training sample set is used as the target language of the inverse first model, and the first target language is used as the source language of the inverse first model to train the inverse first translation model. The inverse first translation model is used to translate a language of the same kind as the first target language into a language of the same kind as the first source language.

5. The method according to claim 1, characterized in that, The step of generating pseudo-parallel corpus pairs based on the second target language in the second training sample set and the reverse first translation model includes: Each second target language in the second training sample set is input into the reverse first translation model to generate each pseudo-parallel source language that is semantically equivalent to each second source language. The pseudo-parallel training sample set is generated based on each pseudo-parallel source language and each second target language.

6. The method according to claim 1, characterized in that, Before training the first forward first translation model and the first backward first translation model using the first training sample set, the process includes: The first source language and the first target language in the first training sample set are segmented into words, and the first source word list and the first target word list are generated based on the segmentation results. The second source language and the second target language in the second training sample set are segmented into words, and the second source end word list and the second target end word list are generated based on the segmentation results.

7. The method according to claim 6, characterized in that, The first forward translation model includes: a source word embedding layer, an encoding layer, and a decoding layer; the generation of an initial second model based on the first forward translation model includes: The network parameters of the encoding and decoding layers of the first forward translation model are transferred to the second model; Based on the first source-end vocabulary, the second source-end vocabulary, and the mapping relationship between sub-words and word embedding vectors contained in the source-end word embedding layer of the first forward translation model, the source-end word embedding layer of the second model is generated. The current second model is used as the initial second model.

8. The method according to claim 7, characterized in that, The step of generating the source word embedding layer of the second model based on the mapping relationship between sub-words and word embedding vectors contained in the source word embedding layer of the first source word list, the second source word list, and the source word embedding layer in the first forward translation model includes: Match the first source terminology and the second source terminology to determine the target sub-words that are the same in the first source terminology and the second source terminology; Based on each target sub-word and the correspondence between the sub-words and word embedding vectors contained in the source word embedding layer of the forward first translation model, the target word embedding vector corresponding to each target sub-word is determined. Based on the target word embedding vector corresponding to each target word, the mapping relationship between each target word and the target word embedding vector in the source word embedding layer of the second model is generated.

9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus. The processor executes the program instructions to perform the steps of the language translation model training method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, performs the steps of the language translation model training method as described in any one of claims 1 to 8.

Images