Domain machine translation method based on scene course learning and related device

Abstract

The embodiment of the invention discloses a field machine translation method based on scene course learning and a related device, and belongs to the technical field of machine translation. According to the method, a scene training framework is utilized, and the model is trained by synthesizing real domain offset. By optimizing the combination of the encoder / decoder and the inempirical decoder / encoder, the encoder and the decoder can overcome the domain offset in a more robust manner. According to the invention, curriculum learning is also utilized and is inserted into a stage training framework, so that the model can be guided to better adapt. The method shows superior advantages in the field adaptation of limited power data, and can be used for low-resource field machine translation.

Description

Technical Field

[0001] The present invention relates to the technical field of machine translation, and in particular to a field machine translation method based on scenario course learning and a related device. Background Art

[0002] Domain translation is a research direction in the field of natural language processing, which aims to solve the barriers to information exchange between different fields. Traditional machine translation methods are usually based on statistical models, such as rule-based machine translation and statistical machine translation. However, these methods face some challenges when processing texts in the power field. First, there are a large number of professional terms and industry-specific expressions between different fields, and these terms may have completely different meanings in different fields. Second, texts in different fields often need to be understood and translated using different contexts and contexts.

[0003] In order to solve these problems, researchers have proposed many methods and techniques for domain translation. The more effective methods include the use of knowledge graphs, transfer learning, meta-learning, etc. Knowledge graphs integrate the knowledge of domain experts into knowledge graphs, so that machine translation systems can better understand texts in the power field and accurately translate them into the target language. Transfer learning is used to apply models trained in one or more related fields to the target field. This method can use the knowledge and patterns in the source field to solve language problems in the target field, thereby improving the accuracy and fluency of translation. Meta-learning is a machine learning technique that enables models to quickly adapt to new tasks and fields without the need for a large amount of labeled data. Meta-learning can help models better adapt to new language environments and terminology. However, these methods do not fully utilize the knowledge of a large amount of source domain data, especially in adapting to low-resource domain data, which often does not work well. Therefore, domain adaptation for low resources remains a challenging problem. Summary of the invention

[0004] The main purpose of the present invention is to provide a domain machine translation method and related devices based on scenario course learning, which can solve the problem that the domain machine translation methods in the prior art have poor domain adaptability to low-resource domain data.

[0005] To achieve the above object, the present invention provides a domain machine translation method based on scenario course learning in a first aspect, the method comprising:

[0006] Collect training corpora from the source domain and the target domain, preprocess the training corpora, and then train a basic teacher translation model on the training corpora from the source domain;

[0007] The neural machine translation model based on encoder and decoder is divided into two modules, and then a scenario training framework is constructed for training;

[0008] Incorporate curriculum learning into the model, denoise the training corpus, and then sort the remaining corpus until the data is processed;

[0009] The processed training corpus is inserted into the scenario training through the training scheduling strategy based on the difference score to simulate different difficulties;

[0010] The network is iteratively updated, and the learning rate is changed multiple times until the maximum number of iterations is reached to complete the training, and the trained network parameters are obtained as the final domain adaptation model.

[0011] To achieve the above object, the second aspect of the present invention provides a domain machine translation device based on scenario course learning, the device comprising:

[0012] Data collection module: used to collect training corpora in the source domain and the target domain, preprocess the training corpora, and then train a basic teacher translation model on the training corpora in the source domain;

[0013] The first training module is used to divide the neural machine translation model based on encoder and decoder into two modules, and then build a scenario training framework for training;

[0014] Course learning module: used to integrate course learning into the model, denoise the training corpus, and then sort the remaining corpus until the data processing is completed;

[0015] The second training module is used to insert the processed training corpus into the scenario training through the training scheduling strategy based on the difference score to simulate different difficulties;

[0016] Model determination module: used to iteratively update the network, change the learning rate multiple times until the maximum number of iterations is reached to complete the training, and obtain the trained network parameters as the final domain adaptation model.

[0017] To achieve the above-mentioned purpose, the third aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the method shown in the first aspect.

[0018] To achieve the above objectives, the fourth aspect of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method shown in the first aspect.

[0019] The embodiments of the present invention have the following beneficial effects:

[0020] The present invention provides a domain machine translation method based on scenario course learning, the method comprising: collecting training corpora of a source domain and a target domain, preprocessing the training corpora, and then training a basic teacher translation model on the training corpora of the source domain; dividing a neural machine translation model based on an encoder and a decoder into two modules, and then constructing a scenario training framework for training; integrating the course learning into the model, denoising the training corpora, and then sorting the remaining corpora until the data processing is completed; inserting the processed training corpora into the scenario training through a training scheduling strategy based on a difference score to simulate different difficulties; iteratively updating the network, changing the learning rate multiple times until the training is completed with a maximum number of iterations, and obtaining the trained network parameters as the final domain adaptation model.

[0021] By adopting the above method, the model is trained by integrating the scenario training framework and then synthesizing the real domain shift. This method can make the model better adapt to domain translation. The combination of optimizing encoder / decoder and inexperienced decoder / encoder can enhance the robustness of the model and make it more effective in overcoming the domain shift problem. In addition, by integrating knowledge into the phased training framework through curriculum learning, data denoising and sorting are performed, which can better utilize the common knowledge between the two domains and guide the model to better adapt to the new domain, thereby improving the quality of translation. This method shows superior advantages in domain adaptation of low-resource domain data such as limited power data, and is suitable for machine translation tasks in low-resource domains. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] in:

[0024] Figure 1 It is a flow chart of a domain machine translation method based on situational course learning in an embodiment of the present invention;

[0025] Figure 2 It is a schematic diagram of a scenario training framework of a domain machine translation method based on scenario course learning in an embodiment of the present invention;

[0026] Figure 3 It is a schematic diagram of a course learning of a domain machine translation method based on scenario course learning in an embodiment of the present invention;

[0027] Figure 4is another flow chart of a domain machine translation method based on scenario course learning in an embodiment of the present invention;

[0028] Figure 5 It is a structural block diagram of a domain machine translation device based on scenario course learning in an embodiment of the present invention;

[0029] Figure 6 4 is a structural block diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] It should be noted that the present invention proposes a domain machine translation method based on scenario course learning, which aims to improve the effect of domain adaptation and provide better performance for domain translation by introducing scenario training framework, course learning and other technologies. This method can effectively utilize the shared knowledge of the source domain and the target domain when the domain data is insufficient, thereby improving the effect of domain adaptation. Please refer to the following content for details.

[0032] See also Figure 1 , Figure 1 The flowchart of a method for domain machine translation based on scenario course learning in an embodiment of the present invention is shown in FIG. The method can be applied to both a terminal and a server. The terminal can be a desktop terminal or a mobile terminal. The mobile terminal can be at least one of a mobile phone, a tablet computer, and a laptop computer. The server can be implemented by an independent server or a server cluster composed of multiple servers. This embodiment is described by applying to a terminal as an example. Figure 1 The method shown comprises the following steps:

[0033] Step 1: Collect the training corpus of the source domain and the target domain, pre-process the training corpus, and then train a basic teacher translation model on the training corpus of the source domain;

[0034] It should be noted that the data of the source domain and the target domain are collected in advance, and a high-quality data set is obtained through data preprocessing, and then a basic teacher translation model is trained on the source domain data. The data collected in advance can be a training corpus, which includes several corpora, each of which includes sentences expressed in the source language and sentences expressed in the target language, wherein the target language can be the language used for the source language translation, and the sentences expressed in the source language and the sentences expressed in the target language form a contrast, which serves as a translation sample for subsequent translation model training. Among them, the source domain can be a general domain, and the target domain can be the power domain, so as to perform subsequent model training and obtain a translation model that can adapt to the power domain.

[0035] In a feasible implementation, the method collects training corpora of the source domain and the target domain, preprocesses the training corpora, and then trains a basic teacher translation model on the training corpora of the source domain, including: collecting parallel data of the source language and the target language in the general domain and the power domain as training corpora through multiple channels, and preprocessing, deduplication and filtering the parallel data; inputting the processed general domain data S into the basic neural machine translation model in batches for training until the convergence condition is reached, and obtaining the basic teacher translation model Θ base Then, we train a basic teacher language model on the general domain

[0036] That is, as a further solution of the present invention, the specific steps of Step 1 are as follows:

[0037] Step 1.1. Collect parallel data of source and target languages ​​in different fields through various channels, and pre-process the data, perform deduplication, filtering and other operations. Then, according to different sources in the power field, aggregate the corpus into D = (D1, D2, ..., D n )(1≤i≤n), where D i is the corpus of the power field from the ith source, n represents the total number of different sources, here we use 3 sources, and denote the general domain as S, which contains the source language s of the domain j and target language j , expressed as (s j ,t j )(1≤j≤m), where j represents the position in the corpus, and m represents the total number of sentences; it can be understood that the parallel data is used as training corpus.

[0038] Step 1.2, then input the processed general domain data S into the basic neural machine translation model in batches for training until the convergence condition is reached, thereby obtaining a basic teacher translation model, denoted as Θ base , and then train the basic language model, expressed as

[0039] Step 2: Divide the neural machine translation model based on encoder and decoder into two modules, and then build a scenario training framework for training;

[0040] It should be noted that the neural machine translation model (NMT model for short) based on encoder and decoder is divided into two modules: encoder g with parameters θ and decoder h with parameters φ. Then a scenario training framework is constructed for training, which consists of four parts, namely domain aggregation training, power domain training, scenario encoder training, and scenario decoder training.

[0041] In a feasible implementation, the neural machine translation model based on encoder and decoder is divided into two modules, and then a scenario training framework is constructed for training, including: dividing the neural machine translation model based on encoder and decoder into two modules: an encoder g with parameters θ and a decoder h with parameters φ; the scenario training framework includes domain aggregation training, power domain training, scenario encoder training and scenario decoder training.

[0042] That is, as a further solution of the present invention, the specific steps of Step 2 are as follows:

[0043] Step 2.1. Split the encoder and decoder parts of the transformer into two modules: the encoder g with parameters θ and the encoder g with parameters The decoder h of the NMT model f with full parameters Θ is denoted as f Θ (s) = h Φ (g θ (s));

[0044] Step 2.2, such as Figure 2 As shown, Figure 2 It is a schematic diagram of a scenario training framework of a domain machine translation method based on scenario course learning in an embodiment of the present invention. The scenario training framework specifically includes four parts, namely domain aggregation training, power domain training, scenario encoder training, and scenario decoder training.

[0045] The domain aggregation training, power domain training, scenario encoder training and scenario decoder training include:

[0046] Domain aggregation training: Through continuous training in the power and general domains, an aggregated translation model is obtained;

[0047] Power field training: fine-tune the specific power field, use the basic teacher translation model to train the power field, obtain the power field translation model for the power field, and then use the basic language model to fine-tune to obtain the power field language model The power data set is randomly divided into shards for training to improve the robustness of scenario learning;

[0048] Scenario encoder training: The encoder of the aggregate translation model and the decoder of the power domain translation model are aggregated to form a new model. Then, data with different divisions in the power domain are selected for training to obtain updated encoder parameters.

[0049] Scenario decoder training: The encoder of the power domain translation model and the decoder of the aggregated translation model are aggregated to form a new model. Then, data with different divisions from the power domain are selected for training to obtain updated decoder parameters.

[0050] As a further solution of the present invention, the specific steps of Step 2.2 are as follows:

[0051] Step 2.2.1. The purpose of domain aggregation training is to obtain an aggregate model by continuous training in the general and power domains. This aggregate translation model can benefit from the language knowledge shared in the two domains, thereby improving the performance of the model on various tasks. Its optimization function is as follows:

[0052]

[0053] Among them, argmin indicates that the optimization goal is to minimize the loss, the updated parameters are θ and φ, θ is the parameter of the encoder g, φ is the parameter of the decoder h, represents the parameters of this update, A represents the collection of general and power field corpora, represents the jth source language of the i-th domain, represents the jth target language in the i-th domain, It represents the predicted sentence obtained by the source language input model. It will perform cross entropy loss with the real sentence to update the model parameters.

[0054] Step 2.2.2 The purpose of power field training is to fine-tune the power field corpus from different sources to further improve the performance of the model in this field. When conducting power field training, the teacher translation model initially trained is used to train the power field corpus from different sources to obtain the power field translation model Θ z , which can also be considered as a fine-tuning model in the power field, and its optimization function is as follows:

[0055]

[0056] The initialization teacher translation model trained in Step 1.2 is used as the parameter initialization, and then D iThe corpus of the i-th source in the power field is represented. Generally, i ranges from 2 to 3. In order to prevent the performance from degrading due to excessive data distribution, θ i are the parameters of the encoder g for a particular source i, φ i are the parameters of decoder h for a specific source i.

[0057] Then use the basic language model trained in Step 1.2 as initialization, and then fine-tune it on the entire power field corpus to obtain the final language model in the power field, which is expressed as

[0058] Step 2.2.3, scenario encoder training involves aggregating the encoder of the aggregated translation model in Step 2.2.1 and the decoder of the translation model learned in a specific source in Step 2.2.2 to form a new model. Then select data from other sources in the power field that are different from the specific source for training. At this time, the model training process will freeze the parameters of the decoder and only update the parameters of the encoder. Finally, the trained encoder parameters are retained. The optimization function is as follows:

[0059]

[0060] The updated parameters are the parameters θ, g of the encoder g in the aggregation model. θ represents the decoder g in the aggregation model, represents the parameters of the decoder trained for a specific source k used this time, D i≠k It means selecting i source data with different specific sources k from the set of electricity sources;

[0061] Step 2.2.4, same as Step 2.2.3, scenario decoder training involves aggregating the encoder of the translation model learned in the power field in Step 2.2.2 and the decoder of the aggregated translation model in Step 2.2.1 to form a new model. Then select data from other target fields different from the power field for training. At this time, the model training process will freeze the encoder parameters and only update the decoder parameters. Finally, the trained decoder parameters are retained and combined with Step 2.2.3 for a complete scenario encoder and decoder training. The optimization function is as follows:

[0062]

[0063] The updated parameters are the parameters φ, h of the decoder h in the aggregation model φ Indicates that the decoder used is the decoder h in the aggregation model, denotes the parameters of the encoder trained for a particular source k used this time.

[0064] Step 3: Integrate course learning into the model, denoise the training corpus, and then sort the remaining corpus until the data processing is completed;

[0065] In which, the course learning is integrated into the model, the training corpus is denoised, and then the remaining corpus is sorted until the data processing is completed. In a feasible implementation, the course learning is integrated into the model, the training corpus is denoised, and then the remaining corpus is sorted until the data processing is completed, including: base and the power domain translation model Θ z Calculate the cross entropy difference for sentences in the power field and filter out negative samples; calculate the source language sentences in the power field, measure the sentence difficulty, and use the basic teacher language model and the power domain language model The cross entropy difference is used as the power domain divergence score for evaluation. The divergence score is used to evaluate the degree of adaptation of the sentence in the power domain. The divergence score of the sentence is inversely proportional to the degree of adaptation of the sentence in the power domain.

[0066] That is, as a further solution of the present invention, the specific steps of Step 3 are as follows:

[0067] Step 3.1, such as Figure 3 As shown, Figure 3 It is a schematic diagram of course learning of a domain machine translation method based on scenario course learning in an embodiment of the present invention. Course learning is mainly used to perform data denoising and measure sentence difficulty, eliminate sentences with too large domain differences, and calculate source language sentences in the power field. Data denoising is to calculate the cross entropy difference between general and power field sentences through a basic teacher model and a power field fine-tuning model.

[0068] Feed sentences from general and power domains into the base teacher translation model Θ base and power sector fine-tuning model Θ z , get the target sentences predicted by the two models, do cross entropy with the real target sentence, then we can get the difference between the two models, and finally divide it by the length of the target sentence. We will filter out the negative samples because this will have a negative impact on the adaptation of the power field. The specific calculation formula is as follows:

[0069]

[0070] Among them, q Z (s, t) represents the cross entropy difference between the two models, Θ z represents the translation model fine-tuned in the power field, Θ baserepresents the basic teacher translation model, s represents the source sentences in the general and power fields, t represents the target sentences in the general and power fields, |t| is the length of the target sentence, and its function is to normalize the cross entropy loss, logP(t|s; Θ Z ) represents the cross entropy loss between the original sentences in the general and power domains and the predicted sentences of the power domain model and the corresponding true sentences t, logP(t|s; Θ Z ) represents the cross entropy loss between the predicted sentences and the corresponding true sentences t after the original sentences in the general and power domains are translated by the basic teacher translation model.

[0071] Step 3.2. At the same time, the difficulty of the sentences in the power field is evaluated. The cross entropy difference between the basic teacher model and the power field fine-tuning model is used as the power field divergence score for evaluation. The divergence score can be used to evaluate the adaptability of the sentence in the power field. A higher divergence score means that the sentence is more divergent from the power sample and is more difficult to adapt to the power field.

[0072] Specifically, we feed sentences in the power sector into the basic language model and the language fine-tuning model in the power sector to obtain the probability distribution of the input sentence for prediction, and then do cross entropy with the real original sentence. Then we can get the difference between the two models, and finally divide it by the length of the original sentence to get the final domain divergence score. According to the score, the training samples are evenly classified and grouped into 3 shard sets, so that the samples in the same shard have similar divergence scores and the same number of samples, and then randomly combined to serve as the data for the next power sector training. The specific calculation formula is as follows:

[0073]

[0074] Among them, d Z (s) represents the domain divergence score, It is a language fine-tuning model in the power sector. is the basic language model, |s| is the length of the original sentence, which is used to normalize the divergence score, It is the distribution difference between the predicted distribution of the original sentence in the power field after the language fine-tuning model in the power field and the distribution of the real sentence. It is the distribution difference between the predicted distribution of the original sentence in the power field obtained through the basic language model and the distribution of the real sentence.

[0075] Step 4: Insert the processed training corpus into the scenario training through the training scheduling strategy based on the difference score to simulate different difficulties;

[0076] It should be noted that the processed data is inserted into the scenario training framework through a training scheduling strategy based on a difference score to simulate different difficulties to make the model more robust, and the processed data includes the training corpus in the power field after noise reduction. In a feasible implementation, the processed training corpus is inserted into the scenario training through a training scheduling strategy based on a difference score to simulate different difficulties, including: using the processed power field data, sorting the data from low to high according to the divergence score of the power field; gradually increasing the difficulty to retrain the model until the model reaches the convergence condition.

[0077] That is, as a further solution of the present invention, the specific steps of Step 4 are as follows:

[0078] Step 4.1, such as Figure 4 As shown, Figure 4 This is another flow chart of a domain machine translation method based on scenario course learning in an embodiment of the present invention. Using the power domain data processed by Step 3, some data are eliminated. We can sort the data from low to high according to the divergence score of the power domain. Then, the difficulty can be gradually increased to retrain the model. This process will be repeated until the model reaches the convergence condition.

[0079] Further, as a further solution of the present invention, the specific steps of Step 4.1 are as follows:

[0080] Step 4.1.1, after Step 1, the basic teacher translation model and basic language model are obtained, and after a scenario framework training, the aggregated translation model, the power field translation model and the language model are obtained;

[0081] Step 4.1.2, through the basic teacher translation model and the power domain translation model, we filter and delete the data of each power source, remove the sample data with negative values, and then divide and sort the remaining domain data into 3 slice data sets according to the domain divergence score dz, and use it as the power domain data set for the next scenario framework training;

[0082] Step 4.1.3. In the new round of scenario framework training, use the source domain sliced ​​data set selected in the previous round, and train them one by one in sorting order to obtain a new round of aggregated translation models, translation models in the power field, and language models. Subsequently, the sliced ​​data sets are randomly combined in pairs to replace the power data sets from different sources to ensure that the combined data sets are of the same size but not repeated in content. Next, select the encoder and decoder from the pseudo-randomly combined power field translation model and the aggregated translation model for combined training, and continue training on data from a different pseudo-random source field from the currently selected until convergence. Finally, the scenario encoder and scenario decoder are spliced ​​together as the aggregated translation model for this round. After the training is completed, a new round of data denoising and sorting selection is performed using the method in Step 4.1.2.

[0083] Step 5: Iteratively update the network, change the learning rate multiple times until the maximum number of iterations is reached to complete the training, and obtain the trained network parameters as the final domain adaptation model.

[0084] It can be understood that the network is iteratively updated by repeating Step 2 to Step 4, the learning rate is changed multiple times until the training is completed with the maximum number of iterations, and the trained network parameters are obtained as the final domain adaptation model. In a feasible implementation, the network is iteratively updated, the learning rate is changed multiple times until the training is completed with the maximum number of iterations, and the trained network parameters are obtained as the final domain adaptation model, including: each time different training expectations are selected for training according to course learning until the preset convergence conditions are reached, thereby obtaining the final domain adaptation model, that is, repeating step 2 to step 4, each time different data can be selected for training according to course learning, until the preset convergence conditions are reached, thereby obtaining the final domain adaptation model.

[0085] Wherein, as a preferred embodiment of the present invention, the specific steps of Step 5 are:

[0086] Step 5.1. In the specific implementation process, each round of training is iterated 300,000 times, the learning rate is 0.00003 for 0-80,000 times, 0.000008 for 80,000-100,000 times, 0.000006 for 100,000-200,000 times, and 0.000005 for 200,000-300,000 times. After the training is completed, the parameters of the deep neural network are saved. The final domain adaptation model can be the aggregated translation model obtained by the final training.

[0087] The present invention provides a domain machine translation method based on scenario course learning. The above method is adopted to train the model by integrating into the scenario training framework and then synthesizing the real domain shift. This method can make the model better adapt to the domain translation, and the combination of optimizing the encoder / decoder and the inexperienced decoder / encoder can enhance the robustness of the model, so that it can more effectively overcome the domain shift problem. In addition, by integrating knowledge into the phased training framework through course learning, data denoising and sorting are performed, which can better utilize the common knowledge between the two fields and guide the model to better adapt to the new field, thereby improving the quality of translation. This method shows superior advantages in the field adaptation of low-resource field data such as limited power data, and is suitable for machine translation tasks in low-resource fields.

[0088] See also Figure 5 , Figure 5 FIG. 1 is a structural block diagram of a domain machine translation device based on scenario course learning in an embodiment of the present invention. Figure 5 The device shown comprises:

[0089] Data collection module 501: used to collect training corpora in the source domain and the target domain, pre-process the training corpora, and then train a basic teacher translation model based on the training corpora in the source domain;

[0090] The first training module 502 is used to divide the neural machine translation model based on the encoder and the decoder into two modules, and then construct a scenario training framework for training;

[0091] Course learning module 503: used to integrate course learning into the model, denoise the training corpus, and then sort the remaining corpus until the data processing is completed;

[0092] The second training module 504 is used to insert the processed training corpus into the scenario training through the training scheduling strategy based on the difference score to simulate different difficulties;

[0093] Model determination module 505: used to iteratively update the network, change the learning rate multiple times until the maximum number of iterations is reached to complete the training, and obtain the trained network parameters as the final domain adaptation model.

[0094] It should be noted that if Figure 5 The functions of each module in the device shown are Figure 1 The contents of each step in the method shown are similar, so they are not described here to avoid repetition. For details, please refer to Figure 1 The content of each step in the method shown.

[0095] The present invention provides a domain machine translation device based on scenario course learning. The above device is used to train the model by integrating into the scenario training framework and then synthesizing the real domain shift. This method can make the model better adapt to the domain translation, and the combination of optimizing the encoder / decoder and the inexperienced decoder / encoder can enhance the robustness of the model, so that it can more effectively overcome the domain shift problem. In addition, by integrating knowledge into the phased training framework through course learning, data denoising and sorting are performed, which can better utilize the common knowledge between the two fields and guide the model to better adapt to the new field, thereby improving the quality of translation. This method shows superior advantages in the field adaptation of low-resource field data such as limited power data, and is suitable for machine translation tasks in low-resource fields.

[0096] Figure 6 FIG. 1 shows an internal structure diagram of a computer device in an embodiment. The computer device may be a terminal or a server. Figure 6 As shown, the computer device includes a processor, a memory and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the above method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement the above method. Those skilled in the art can understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0097] In one embodiment, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps: Figure 1 Or follow the steps shown in 4.

[0098] In one embodiment, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the processor performs the following steps: Figure 1 Or follow the steps shown in 4.

[0099] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0100] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0101] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A domain machine translation method based on scenario course learning, characterized in that: The method comprises: Collect training corpora from the source domain and the target domain, preprocess the training corpora, and then train a basic teacher translation model on the training corpora from the source domain; The neural machine translation model based on encoder and decoder is divided into two modules, and then a scenario training framework is constructed for training; Incorporate curriculum learning into the model, denoise the training corpus, and then sort the remaining corpus until the data is processed; The processed training corpus is inserted into the scenario training through the training scheduling strategy based on the difference score to simulate different difficulties; The network is iteratively updated, and the learning rate is changed multiple times until the maximum number of iterations is reached to complete the training, and the trained network parameters are obtained as the final domain adaptation model.

2. The method according to claim 1, characterized in that: The step of collecting training corpora of the source domain and the target domain, preprocessing the training corpora, and then training a basic teacher translation model on the training corpora of the source domain includes: Collect parallel data of the source language and the target language in the general field and the power field through various channels as training corpus, and perform preprocessing, deduplication and filtering operations on the parallel data; The processed general domain data is input into the basic neural machine translation model in batches for training until the convergence condition is reached to obtain the basic teacher translation model, and then the basic teacher language model is trained on the general domain.

3. The method according to claim 2, characterized in that: The neural machine translation model based on encoder and decoder is divided into two modules, and then a scenario training framework is constructed for training, including: The neural machine translation model based on encoder and decoder is divided into two modules: encoder g with parameters θ and decoder h with parameters φ; The scenario training framework includes domain aggregation training, power domain training, scenario encoder training and scenario decoder training.

4. The method according to claim 3, characterized in that: The domain aggregation training, power domain training, scenario encoder training and scenario decoder training include: Domain aggregation training: Through continuous training in the power and general domains, an aggregated translation model is obtained; Power field training: Fine-tune the specific power field, use the basic teacher translation model to train the power field, obtain the power field translation model for the power field, and then use the basic language model to fine-tune to obtain the power field language model. Randomly divide the power data set for sharding training to improve the robustness of scenario learning; Scenario encoder training: The encoder of the aggregate translation model and the decoder of the power domain translation model are aggregated to form a new model. Then, data with different divisions in the power domain are selected for training to obtain updated encoder parameters. Scenario decoder training: The encoder of the power domain translation model and the decoder of the aggregated translation model are aggregated to form a new model. Then, data with different divisions from the power domain are selected for training to obtain updated decoder parameters.

5. The method according to claim 4, characterized in that: The above mentioned process of integrating course learning into the model, denoising the training corpus, and then sorting the remaining corpus until the data processing is complete includes: The cross entropy difference of sentences in the power field is calculated by the basic teacher translation model and the power field translation model, and negative value samples are filtered out; The source language sentences in the power field are calculated to measure the sentence difficulty. The cross entropy difference between the basic teacher language model and the power field language model is used as the power field divergence score for evaluation. The divergence score is used to evaluate the adaptability of the sentence in the power field, where the divergence score of the sentence is inversely proportional to the adaptability of the sentence in the power field.

6. The method according to claim 5, characterized in that: The processed training corpus is inserted into the scenario training through a training scheduling strategy based on the difference score to simulate different difficulties, including: Using the processed power field data, sort the data from low to high according to the divergence score of the power field; gradually increase the difficulty to retrain the model until the model reaches the convergence condition.

7. The method according to claim 6, characterized in that: The network is iteratively updated, the learning rate is changed multiple times until the maximum number of iterations is reached to complete the training, and the trained network parameters are obtained as the final domain adaptation model, including: Each time, different training expectations are selected according to course learning for training until the preset convergence conditions are reached, thus obtaining the final domain adaptation model.

8. A domain machine translation device based on scenario course learning, characterized in that: The device comprises: Data collection module: used to collect training corpora in the source domain and the target domain, preprocess the training corpora, and then train a basic teacher translation model on the training corpora in the source domain; The first training module is used to divide the neural machine translation model based on encoder and decoder into two modules, and then build a scenario training framework for training; Course learning module: used to integrate course learning into the model, denoise the training corpus, and then sort the remaining corpus until the data processing is completed; The second training module is used to insert the processed training corpus into the scenario training through the training scheduling strategy based on the difference score to simulate different difficulties; Model determination module: used to iteratively update the network, change the learning rate multiple times until the maximum number of iterations is reached to complete the training, and obtain the trained network parameters as the final domain adaptation model.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

10. A computer device comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

Images