Translation model training method, paragraph-level machine translation method and device

Abstract

The present invention discloses a translation model training method, a chapter-level machine translation method and device, which are applied to the field of machine translation. For a target word in a chapter-level source corpus, a complement word of the target word is obtained from the chapter-level source corpus; according to the complement word, a source sentence in which the target word appears in the chapter-level source corpus is semantically complemented to form a complement chapter-level source corpus corresponding to the chapter-level source corpus; a target translation model is trained based on a plurality of complement chapter-level source corpora and corresponding target chapter corpora to obtain a trained chapter-level translation model. The present invention can alleviate the problem of inconsistent context translation in chapter translation to a certain extent.

Description

Technical Field

[0001] The present invention relates to the field of machine translation, and in particular to a translation model training method, a paragraph-level machine translation method and a device. Background Art

[0002] NMT (Neural machine translation) is the latest step in the continuous evolution of the translation industry. As more and more businesses go global, NMT is likely to have a huge impact on the translation industry. Unlike traditional statistical machine translation, NMT uses neural network-based technology to achieve more context-accurate translations. Since NMT can translate entire sentences at once, the output of NMT can be similar to human translation.

[0003] At present, for the translation of documents or chapters, a single sentence is usually used as the translation unit, and the translation results of each sentence are then spliced ​​together to obtain the final translation result of the chapter. However, the semantic representation of the source sentence is incomplete, which leads to poor translation effect of the chapter-level text. Summary of the invention

[0004] In view of the above-mentioned technical problems in the prior art, the embodiments of the present invention provide a translation model training method, a paragraph-level machine translation method and a device.

[0005] In a first aspect, an embodiment of the present invention provides a translation model training method, comprising:

[0006] For a target word in a passage-level source corpus, obtaining a complementary word of the target word from the passage-level source corpus;

[0007] According to the completion words, semantic completion is performed on the source sentence in which the target words appear in the chapter-level source corpus to form a completed chapter-level source corpus corresponding to the chapter-level source corpus;

[0008] The target translation model is trained according to the plurality of the completed chapter-level source corpora and the corresponding target-side chapter corpora to obtain a trained chapter-level translation model.

[0009] Optionally, the acquiring of a complement word for the target word from the passage-level source corpus includes:

[0010] Performing dependency syntactic analysis on the text-level source corpus through a dependency syntactic analysis tool to extract the complement words of the target word, or

[0011] The target translation model is used to extract the complement words of the target words from the paragraph-level source corpus.

[0012] Optionally, the acquiring of a complement word for the target word from the passage-level source corpus includes:

[0013] Extracting information from each source sentence in the chapter-level source corpus to obtain associated words for each word in the chapter-level source corpus;

[0014] Determining the associated words of the target word according to the associated words of each word in the paragraph-level source corpus;

[0015] According to the associated words of the target word, a complementary word of the target word is obtained.

[0016] Optionally, obtaining a complementary word of the target word according to the associated words of the target word includes:

[0017] Combining the associated words of the target word in different source-end sentences to obtain a set of completion words for semantically completing any source-end sentence in which the target word appears; or

[0018] The associated words of the target word are extracted from different source end sentences, and the associated words that do not exist in the source end sentence where the target word is currently located are respectively screened out to obtain a group of completion words for semantically completing the source end sentence where the target word is currently located.

[0019] Optionally, the semantically completing the source sentence in which the target word appears according to the completion word includes:

[0020] A complement word of the target word is added at an adjacent position of the target word to semantically complete the source sentence where the target word is currently located, wherein a completion tag is set for the added complement word.

[0021] Optionally, the complement words used to semantically complete the source sentence where the target word currently resides include multiple complement words, and before adding the complement words of the target word at the adjacent position of the target word, it also includes:

[0022] According to a preset sorting rule, a plurality of completion words used for semantically completing the source-end sentence where the target word is currently located are sorted to obtain a sorted completion word sequence;

[0023] Adding a complement word of the target word at an adjacent position of the target word includes:

[0024] In the source sentence where the target word is currently located, the sorted completion word sequence is added to the adjacent position of the target word.

[0025] Optionally, the plurality of completion words used for semantically completing the source-end sentence where the target word currently resides are sorted according to a preset sorting rule, including:

[0026] The plurality of completion words are sorted according to the number of occurrences in the passage-level source corpus from high to low, and / or

[0027] The multiple completion words are sorted according to the order in which they appear in the passage-level source corpus.

[0028] Optionally, the target words include identical words that appear multiple times in the passage-level source corpus.

[0029] In a second aspect, an embodiment of the present invention provides a paragraph-level machine translation method, comprising:

[0030] For a target word in an original passage to be translated, obtaining a complementary word of the target word from the original passage to be translated;

[0031] According to the completion words, semantic completion is performed on the source sentence where the target word appears, so as to form a completed chapter to be translated corresponding to the original chapter to be translated;

[0032] The completed chapter to be translated is translated by using a trained chapter-level translation model to obtain a translated chapter, wherein the trained chapter-level translation model is obtained by any of the translation model training methods described in the first aspect.

[0033] In a third aspect, an embodiment of the present invention provides a translation model training device, comprising:

[0034] A complement word acquisition unit, for acquiring a complement word of a target word in a chapter-level source-end corpus from the chapter-level source-end corpus;

[0035] A semantic completion unit, configured to perform semantic completion on the source sentence in which the target word appears in the chapter-level source-end corpus according to the completion word, so as to form a completed chapter-level source-end corpus corresponding to the chapter-level source-end corpus;

[0036] The model training unit is used to train the target translation model based on the multiple completed chapter-level source-end corpora and the corresponding target-end chapter corpora to obtain a trained chapter-level translation model.

[0037] Optionally, the completion word acquisition unit is specifically used to:

[0038] Performing dependency syntactic analysis on the text-level source corpus through a dependency syntactic analysis tool to extract the complement words of the target word, or

[0039] The target translation model is used to extract the complement words of the target words from the paragraph-level source corpus.

[0040] Optionally, the completion word acquisition unit includes:

[0041] An information extraction subunit is used to extract information from each source sentence in the chapter-level source corpus to obtain associated words of each word in the chapter-level source corpus;

[0042] A target association determination subunit, configured to determine the associated words of the target word according to the associated words of each word in the passage-level source corpus;

[0043] The complement word determination subunit is used to obtain the complement word of the target word according to the associated words of the target word.

[0044] Optionally, the target association determination subunit is specifically used to:

[0045] Combining the associated words of the target word in different source-end sentences to obtain a set of completion words for semantically completing any source-end sentence in which the target word appears; or

[0046] The associated words of the target word are extracted from different source end sentences, and the associated words that do not exist in the source end sentence where the target word is currently located are respectively screened out to obtain a group of completion words for semantically completing the source end sentence where the target word is currently located.

[0047] Optionally, the semantic completion unit includes:

[0048] The completion word adding unit is used to add the completion word of the target word at the adjacent position of the target word to semantically complete the source sentence where the target word is currently located, wherein a completion tag is set for the added completion word.

[0049] Optionally, the completion words for semantically completing the source sentence where the target word currently resides include a plurality of completion words, and the device further includes:

[0050] A word sorting unit, used for sorting a plurality of complementary words used for semantically completing the source sentence where the target word is currently located according to a preset sorting rule before adding the complementary word of the target word at the adjacent position of the target word, to obtain a sorted complementary word sequence;

[0051] The word completion adding unit is specifically used for:

[0052] In the source sentence where the target word is currently located, the sorted completion word sequence is added to the adjacent position of the target word.

[0053] Optionally, the word sorting unit is specifically used to:

[0054] The plurality of completion words are sorted according to the number of occurrences in the passage-level source corpus from high to low, and / or

[0055] The multiple completion words are sorted according to the order in which they appear in the passage-level source corpus.

[0056] Optionally, the target words include identical words that appear multiple times in the passage-level source corpus.

[0057] In a fourth aspect, an embodiment of the present invention provides a paragraph-level machine translation device, comprising:

[0058] A complement word acquisition unit is used to acquire a complement word of a target word in an original passage to be translated from the original passage to be translated;

[0059] A semantic completion unit, configured to perform semantic completion on the source sentence where the target word appears according to the completion word, so as to form a completed chapter to be translated corresponding to the original chapter to be translated;

[0060] The chapter translation unit is used to translate the completed chapter to be translated by using a trained chapter-level translation model to obtain a translated chapter, wherein the trained chapter-level translation model is obtained by any of the translation model training methods described in the first aspect.

[0061] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect, or the steps of the method described in the second aspect.

[0062] In a sixth aspect, an embodiment of the present invention provides an electronic device comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute operating instructions contained in the one or more programs for performing the method described in the first aspect or the method described in the second aspect.

[0063] The above one or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages:

[0064] By obtaining the completion words of the target words from the chapter-level source corpus; semantically completing the source sentences where the target words appear in the chapter-level source corpus, a completed chapter-level source corpus is formed; and the target translation model is trained based on multiple completed chapter-level source corpora and the corresponding target chapter corpus, so as to solve the problem of incomplete semantic representation in chapter-level translation and make the context sentence representation in the corpus used for training more complete. Therefore, the chapter-level translation model obtained after such corpus processing can alleviate the problem of inconsistent context translation in chapter translation to a certain extent. Furthermore, the trained chapter-level translation model can effectively improve the translation effect of chapter-level translation. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0066] Figure 1 A flowchart of a translation model training method provided by an embodiment of the present invention;

[0067] Figure 2 A flow chart of a paragraph-level machine translation method provided by an embodiment of the present invention;

[0068] Figure 3 A functional module diagram of a translation model training device provided by an embodiment of the present invention;

[0069] Figure 4 A functional module diagram of a paragraph-level machine translation device provided by an embodiment of the present invention;

[0070] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The embodiments of the present invention provide a translation model training method, a paragraph-level machine translation method and device, which completes the target words appearing in the paragraph-level source corpus from the paragraph-level source corpus; semantically completes the source sentences where the target words appear according to the completed words, so that the context sentence representation in the corpus used for model training is more complete. Therefore, the paragraph-level translation model obtained by training with such corpus processing can effectively improve the translation effect of paragraph-level translation.

[0072] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] In a first aspect, an embodiment of the present invention provides a translation model training method, referring to Figure 1 As shown, the translation model training method includes the following steps:

[0074] Collect multiple chapter-level source corpora for training the translation model. Each chapter-level source corpus consists of a series of sentences. For example, it can be a novel, a speech, an essay, etc., or a chapter or paragraph content in a novel, a speech, an essay, etc. Perform steps S10 to S11 for each chapter-level source corpus to generate the corresponding completed chapter-level source corpus:

[0075] S10. For a target word in the passage-level source corpus, a complementary word of the target word is obtained from the passage-level source corpus.

[0076] Among them, the chapter-level source corpus contains one or more different target words. Specifically, the target word can be the same word that appears more than twice in the chapter-level source corpus. For example, a certain chapter-level source corpus contains the following two sentences: The first sentence is "Two laptops containing the investigation data of Princess Diana's death were stolen." The second sentence is: "Reports indicate that the computer may contain confidential information led by the former Metropolitan Police Chief Sir Stevenson." Among them, the two words "data" and "computer" appear at least twice in the chapter-level source corpus, so "data" and "computer" are both target words in the chapter-level source corpus.

[0077] In the specific implementation process, the target words are not limited to the same words that appear more than twice in the chapter-level source corpus. According to the characteristics of the actual application object - the chapter-level source corpus, the target words can also include other types of words. For example, in the chapter-level source corpus of poetry, the target words also include rhyming words. Therefore, the types of target words are not exhaustively listed here.

[0078] Specifically, the target translation model itself can be used to extract the target word's complement from the text-level source corpus. Alternatively, a dependency syntactic parsing tool other than the target translation model can be used to perform dependency syntactic parsing on the text-level source corpus to extract the target word's complement. Dependency syntactic parsing can be implemented using a dependency syntactic parsing tool under the LTP (Language Technology Platform).

[0079] It should be noted that, whether the target translation model itself is used or a dependency syntactic analysis tool other than the target translation model is used to extract the target word's complement from the text-level source corpus, any of the following implementation processes can be selected according to actual needs:

[0080] Implementation process one: Step A101, extract information from each source sentence in the chapter-level source corpus respectively to obtain the associated words of each word in the chapter-level source corpus; Step A102, determine the target word and the associated words of the target word based on the associated words of each word in the chapter-level source corpus; Step A103, obtain the completion words of the target word based on the associated words of the target word.

[0081] ① In terms of the need to use dependency syntactic analysis tools other than the target translation model to extract the target word's complement from the text-level source corpus:

[0082] Step A101 specifically includes: performing word segmentation processing on each source sentence in the chapter-level source corpus to obtain the word sequence corresponding to each source sentence; performing dependency syntactic analysis on the i-th word sequence corresponding to the i-th source sentence by a dependency syntactic analysis tool to obtain the dependency relationship between different words in the i-th word sequence, where i ranges from 1 to M, and M is the number of source sentences in the chapter-level source corpus; for each word in the i-th word sequence, according to the dependency relationship between different words in the i-th word sequence, the associated word of the word in the i-th source sentence.

[0083] Since the role of dependency syntactic analysis is to extract the interdependence relationship (entity relation) between words in the same source sentence, it does not matter whether two words must be adjacent. The interdependence relationship between words in different source sentences will not be extracted. Therefore, in the embodiment of the present invention, for any word in the paragraph-level source corpus, only the associated words of the word will be extracted from the source sentence where the word is currently located based on dependency syntactic analysis.

[0084] After obtaining the associated words of each word in the passage-level source corpus based on the dependency syntax analysis tool, step A102 is specifically: for the same target word, extract the associated words of the target word from the associated words of each word in the passage-level source corpus. Among them, the target words in the same passage-level source corpus include one or more, and the specific number of target words is determined by the actual content of the passage-level source corpus, which is not limited here. For the same target word, the associated words extracted from each source sentence where the target word appears include one or more, and even the corresponding associated words cannot be extracted from some source sentences where the target word appears.

[0085] When the target word appears more than twice in the paragraph-level source corpus, the related words extracted from each source sentence where the same word appears are determined from the related words of each word in the paragraph-level source corpus; the related words extracted from each source sentence where the same word appears are combined to obtain the completion word for the same word.

[0086] Specifically, step A102 can be implemented in a variety of different ways:

[0087] The first implementation mode is: combining the associated words of the target word extracted from different source end sentences to obtain a set of completion words for semantically completing any source end sentence in which the target word appears.

[0088] In the first implementation mode, the same set of completion words is used to semantically complete different source sentences where the same target word appears. Each target word corresponds to a set of completion words, for example: for target word A "data", a set of completion words is used to semantically complete each source sentence where "data" appears; for target word B "computer", another set of completion words is used to semantically complete each source sentence where "computer" appears.

[0089] Since the associated words extracted from the source sentence where the target word is currently located already exist in the source sentence where the target word is currently located. Therefore, in order to avoid unnecessary semantic completion of the source sentence, implementation method 2 can be adopted: a set of completion words for semantic completion of the source sentence where the target word is currently located only includes words from other source sentences where the target word appears. Therefore, step A103 can also preferably adopt another implementation method: according to the associated words in different source sentences where the target word appears, the associated words that do not exist in the source sentence where the target word is currently located are respectively screened out to obtain a set of completion words for semantic completion of the source sentence where the target word is currently located.

[0090] In the second implementation mode, different completion words are used to complete the semantics of different source-end sentences in which the same target word appears.

[0091] In order to understand the above steps A101 to A103 more clearly, the following is an explanation of steps A101 to A103, taking a sentence ① "Two laptops containing information on the investigation into Princess Diana's death were stolen" and a sentence ② "Reports indicate that the computers may contain confidential information from an investigation led by former Metropolitan Police Chief Sir Stephenson" in the text-level source corpus as an example:

[0092] Among them, the target words are the same words: "data", "computer". First, through the dependency syntactic analysis of step 101, the associated words of each word in sentence ① in sentence ① are obtained, among which the associated words of the word "data" in sentence ① are "cause of death", and the associated words of the word "computer" in sentence ① are "two", "table", and "portable"; the associated words of each word in sentence ② in sentence ② are obtained, among which the associated words of the word "data" in sentence ② are "confidential", and the word "computer" does not have any associated words in sentence ②.

[0093] Next, through step A102, it is identified that the words "data" and "computer" that appear more than twice in the paragraph-level source corpus are all target words; and it is determined that the associated words of the target word "data" include "cause of death", "confidential", etc., and it is determined that the associated words of the target word "computer" include "two", "station", "laptop", etc.

[0094] Finally, step A103 selects the completion words for semantically completing sentences ① and ② from the associated words of the target word "data", such as "cause of death" and "confidential". In order to describe briefly, it is assumed that the paragraph-level source corpus only contains the above sentences ① and ②, and the completion word selected for semantically completing sentence ① where the target word "data" is currently located is "confidential", and the completion word selected for semantically completing sentence ② where the target word "data" is currently located is "cause of death". There is no completion word for semantically completing sentence ① where "computer" is located in the associated words of the target word "computer"; the completion words selected from the associated words of the target word "computer" for semantically completing sentence ② where "computer" is currently located are: "two", "table", and "portable".

[0095] Specifically, if the target word is a rhyming word, the rhyming word is generally the final word of a source sentence, and the associated words obtained for the rhyming word can be the final words of other source sentences in the same paragraph-level source corpus.

[0096] Specifically, if a dependency syntactic analysis tool other than the target translation model is used to extract the completion words of the target words from the paragraph-level source corpus, the target translation model can be a baseline translation model in the form of an encoder-decoder implemented based on a transformer framework or a recurrent neural network (RNN) framework.

[0097] ② As for the need to extract the completion words of the target words from the paragraph-level source corpus through the target translation model itself, the target translation model is a multi-fusion translation model that integrates the baseline translation model and the dependency syntactic analysis model. Specifically, it can be based on the baseline translation model in the form of an encoder-decoder implemented based on the transformer framework or the recurrent neural network (RNN) framework, and also integrates the neural network dependency syntactic analysis model.

[0098] Specifically, it can be a fusion of a graph-based or transfer-based neural network dependency syntactic analysis model. The specific implementation process of the graph-based or transfer-based neural network dependency syntactic analysis model for performing dependency syntactic analysis on the source sentence can refer to the description of step A101 under implementation mode ① above, or refer to the description of dependency syntactic analysis in the prior art.

[0099] For example, in step A101, the implementation process of the neural network dependency syntactic analysis model performing dependency syntactic analysis on the source sentence under implementation mode ② can be specifically as follows: the network dependency syntactic analysis model predicts the correct conversion operation based on the features extracted from the configuration information, and extracts the feature vector based on the configuration (state) information, wherein the feature vector is composed of a word feature vector, a part-of-speech feature vector, and a dependency arc feature vector, and contains the corresponding configuration (state) information. The goal of the network dependency syntactic analysis model is to input the feature vector and predict the corresponding conversion type. According to the predicted conversion type, the corresponding conversion operation is performed, the configuration information is updated, and then a new feature vector is obtained, which is then input into the network dependency syntactic analysis model for prediction, and this cycle is repeated until the dependency arc set in the source sentence is found, and the dependency relationship of the source sentence is obtained based on the dependency arc set.

[0100] Steps A102-A103 performed in implementation mode ② are similar to or identical to steps A102-A103 performed in implementation mode ①, and will not be described in detail for the sake of brevity.

[0101] In the second implementation mode, in order to improve efficiency, the target words can be first determined from the chapter-level source corpus, and after the target words are determined, only the associated words are extracted from the target words determined in the chapter-level source corpus. Therefore, it is necessary to first extract the target words from the chapter-level source corpus. The specific implementation mode of the second implementation mode includes the following steps B101 to B102:

[0102] Step B101, determine the target word from the chapter-level source corpus; Step B102, extract the target word's complement word from the chapter-level source corpus.

[0103] It can be seen that any of the above implementations can realize the extraction of information required for semantic completion of the paragraph-level source corpus.

[0104] S11. According to the completion words of the target word, semantic completion is performed on the source sentence in which the target word appears in the chapter-level source corpus to form a completed chapter-level source corpus corresponding to the chapter-level source corpus.

[0105] Specifically, step S11 includes: adding a complement word of the target word at an adjacent position of the target word, and performing semantic completion on the source sentence where the target word is currently located, so as to make the semantic representation of the source sentence more complete.

[0106] Since in the above step S10, the completion words obtained for each target word are a group or divided into multiple groups according to the different source sentences that need to be semantically completed, each group of completion words contains one or more. Step S11 can correspond to two different implementations, which are described below respectively:

[0107] Method 1: For each target word in the paragraph-level source corpus: add a set of completion words that match the source sentence where the target word is currently located to the position adjacent to the target word in the source sentence where the target word is currently located, so as to semantically complete the source sentence where the target word is currently located.

[0108] Method 2: For each target word in the paragraph-level source corpus, a set of completion words for the target word is added to each source sentence where the target word appears, adjacent to the target word, to semantically complete each source sentence containing the target word.

[0109] Specifically, the adjacent position of the target word may be pre-set at the front adjacent position or the back adjacent position of the target word.

[0110] Furthermore, in order to distinguish the completed words from the original words in the source sentence, a completion tag is set for the added completed words. The completion tag is a pre-set identification symbol, for example,<beg>and as a completion tag, <beg>It is used to be added before the complement word to indicate the beginning of the complement word; it is used to be added after the complement word to indicate the end of the complement word.

[0111] Taking the sentence ① "Two laptops containing information on the investigation into Princess Diana's death were stolen." as an example, the result after semantic completion is: "Two laptops containing information on the investigation into Princess Diana's death were stolen." <beg>Two laptops containing confidential information were stolen.”

[0112] Take the sentence ② above, "The report indicated that the computer may contain confidential information led by former Metropolitan Police Chief Sir Stevenson." as an example, the result after semantic completion is: "The report indicated that the theft has caused people to speculate that <beg>Loaded onto two laptops may be secrets from an investigation led by former Metropolitan Police Chief Sir Stephenson <beg>Cause of death information.

[0113] Of course, in the specific implementation, the method of marking the added supplementary words is not limited to adding the supplementary words to the supplementary tags. For example, a supplementary tag in superscript or subscript form may be added to each added supplementary word.

[0114] Specifically, for any target word, a group of completion words used to semantically complete the source sentence where the target word is currently located may include one or more. If it includes more than one, the multiple completion words used to semantically complete the source sentence where the target word is currently located are sorted according to the preset sorting rules to obtain a sorted completion word sequence. Among them, the preset sorting rules can be: sorting according to the number of occurrences of multiple completion words in the chapter-level source corpus from high to low, and / or, sorting according to the order in which multiple completion words appear in the chapter-level source corpus. In the source sentence where the target word is currently located, the sorted completion word sequence is added to the adjacent position of the target word.

[0115] For each chapter-level source corpus, the processing of steps S10 to S11 is performed to obtain the corresponding completed chapter-level source corpus.

[0116] S12. Training a target translation model based on a plurality of completed chapter-level source corpora and corresponding target-side chapter corpora to obtain a trained chapter-level translation model.

[0117] The number of completed chapter-level source corpora and corresponding target chapter corpora used in the specific training target translation model is set according to actual needs and is not limited here.

[0118] Through the translation model training method described in the first aspect, the source sentences in the chapter-level source corpus used for training are semantically completed, so that the contextual semantic representation of the source sentences is more complete. For example, for the same words, the same words in the chapter-level source corpus can have the same chapter representation in different source sentences. The model is then trained with the semantically completed corpus, so that the translation model itself can learn this ability, thereby improving the translation effect of the trained chapter-level translation model.

[0119] In a second aspect, an embodiment of the present invention provides a paragraph-level machine translation method, referring to Figure 2 As shown, the paragraph-level machine translation method includes the following steps:

[0120] S20, for a target word in the original passage to be translated, obtaining a complementary word of the target word from the original passage to be translated;

[0121] S21, semantically completing the source sentence where the target word appears according to the completion word, so as to form a completed chapter to be translated corresponding to the original chapter to be translated.

[0122] It should be noted that the specific implementation process of step S20 is similar to that of S10 described in the first aspect, and will not be repeated here for the sake of brevity. The specific implementation process of step S21 is similar to that of S11 described in the first aspect, and will not be repeated here for the sake of brevity.

[0123] S22. Translate the completed chapter to be translated using the trained chapter-level translation model to obtain a translated chapter, wherein the trained chapter-level translation model is obtained by training using the translation model training method of the first aspect.

[0124] Take the following two sentences as an example: the first sentence is "Two laptops containing information on the investigation into Princess Diana's death were stolen." The second sentence is "Reports indicate that the computers may contain confidential information from an investigation led by former Metropolitan Police Chief Sir Stephenson." If semantic completion is not performed, the translation result of the sentence-level model is:

[0125] (1)two notebook computers containing information on the cause of death of the deceased were stolen

[0126] (2)the report pointed out that confidential information might be

[0127] contained in the computer under the leadership of sir robertstevenson,former

[0128] metropolitan police chief.

[0129] The translation of the same word "computer" in the two sentences above and below will result in two different translation results. This is because the sentence-level translation takes into account the information "two" in the first sentence when translating the second sentence.

[0130] If the above two sentences to be translated are semantically completed, the completed sentences to be translated are as follows:

[0131] (1) Entry into the Inquiry into Princess Diana’s Death <beg>Two laptops containing confidential information were stolen.

[0132] (2) The report pointed out that the theft has caused people to speculate that <beg>Loaded onto two laptops may be secrets from an investigation led by former Metropolitan Police Chief Sir Stephenson <beg>Cause of death information.

[0133] The translation results obtained based on the completed sentence to be translated are as follows:

[0134] (1)two laptop computers containing information on the cause of death investigation stolen

[0135] (2)The report pointed out that the computers might contain confidential information led by former metropolitan police chief sir robertho.

[0136] After semantic completion of the above two sentences to be translated, the semantics of the word 'computer' in the context are the same, so the translation is also the same, which is more consistent with the meaning of the full text.

[0137] Through the chapter-level machine translation method described in the second aspect, semantic completion is performed on the source sentences in the original chapter to be translated, so that the contextual semantic representation of the source sentences to be translated is more complete. For example, for the same words, the same words can have the same chapter representation in different source sentences. The chapter to be translated after semantic completion is then input into the chapter translation model obtained by training in the first aspect, further improving the translation effect of the chapter-level text.

[0138] In a third aspect, an embodiment of the present invention provides a translation model training device, referring to Figure 3 As shown, including:

[0139] A complement word acquisition unit 301 is used to acquire a complement word of a target word in a chapter-level source-end corpus from the chapter-level source-end corpus;

[0140] The semantic completion unit 302 is used to semantically complete the source sentence in which the target word appears in the chapter-level source corpus according to the completion word, so as to form a completed chapter-level source corpus corresponding to the chapter-level source corpus;

[0141] The model training unit 303 is used to train the target translation model according to a plurality of completed chapter-level source-end corpora and corresponding target-end chapter corpora to obtain a trained chapter-level translation model.

[0142] In a specific implementation, the completion word acquisition unit 301 is specifically used to:

[0143] Performing dependency syntactic analysis on the text-level source corpus through a dependency syntactic analysis tool to extract the complement words of the target word, or

[0144] The target translation model is used to extract the target word's complement words from the paragraph-level source corpus.

[0145] In a specific implementation, the completion word acquisition unit 301 includes:

[0146] The information extraction subunit is used to extract information from each source sentence in the chapter-level source corpus to obtain the associated words of each word in the chapter-level source corpus;

[0147] The target association determination subunit is used to determine the associated words of the target word according to the associated words of each word in the text-level source corpus;

[0148] The complement word determination subunit is used to obtain the complement word of the target word according to the associated words of the target word.

[0149] In a specific implementation manner, the target association determination subunit is specifically used to:

[0150] Combining related words of the target word in different source sentences to obtain a set of completion words for semantically completing any source sentence in which the target word appears; or

[0151] The associated words of the target word are extracted from different source sentences, and the associated words that do not exist in the source sentence where the target word is currently located are respectively screened out to obtain a group of completion words for semantic completion of the source sentence where the target word is currently located.

[0152] In a specific implementation, the semantic completion unit 302 includes:

[0153] The completion word adding unit is used to add the completion word of the target word at the adjacent position of the target word, and perform semantic completion on the source sentence where the target word is currently located, wherein a completion tag is set for the added completion word.

[0154] In a specific implementation manner, the completion words used to semantically complete the source sentence where the target word is currently located include multiple completion words, and the device also includes:

[0155] A word sorting unit is used to sort a plurality of complementing words used to semantically complete the source sentence where the target word is currently located according to a preset sorting rule before adding the complementing words of the target word at the adjacent position of the target word to obtain a sorted complementing word sequence;

[0156] The word completion unit is used to:

[0157] In the source sentence where the target word is currently located, the sorted completion word sequence is added to the adjacent position of the target word.

[0158] In a specific implementation manner, the word sorting unit is specifically used to:

[0159] Sort multiple completion words according to their number of occurrences in the passage-level source corpus, and / or

[0160] Sort multiple completion words according to the order in which they appear in the text-level source corpus.

[0161] In a specific implementation, the target words include identical words that appear multiple times in the paragraph-level source corpus.

[0162] Regarding the above-mentioned device, the specific functions of each unit therein have been described in detail in the translation model training method embodiment provided by the present invention, and will not be elaborated here. The specific implementation process can refer to the model training method embodiment provided in the first aspect above.

[0163] In a fourth aspect, an embodiment of the present invention provides a paragraph-level machine translation device, referring to Figure 4 As shown, including:

[0164] A complement word acquisition unit 401 is used to acquire a complement word of a target word in an original passage to be translated from the original passage to be translated;

[0165] The semantic completion unit 402 is used to semantically complete the source sentence where the target word appears according to the completion word, so as to form a completed chapter to be translated corresponding to the original chapter to be translated;

[0166] The chapter translation unit 403 is used to translate the completed chapter to be translated by using a trained chapter-level translation model to obtain a translated chapter, wherein the trained chapter-level translation model is obtained by any of the translation model training methods described in the first aspect.

[0167] Regarding the above-mentioned device, the specific functions of each unit therein have been described in detail in the embodiment of the paragraph-level machine translation method provided by the present invention, and will not be elaborated in detail here. The specific implementation process can refer to the embodiment of the paragraph-level machine translation method provided in the second aspect above, and the embodiment of the model training method described in the first aspect.

[0168] In a fifth aspect, based on the same inventive concept as the aforementioned translation model training method embodiment, an embodiment of the present invention further provides an electronic device. Based on the same inventive concept as the aforementioned paragraph-level machine translation method embodiment, an embodiment of the present invention further provides an electronic device. Figure 5 1 is a block diagram of an electronic device 500 according to an exemplary embodiment. For example, the device 500 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0169] Reference Figure 5 , device 500 may include one or more of the following components: a processing component 502 , a memory 504 , a power component 506 , a multimedia component 508 , an audio component 510 , an input / output (I / O) interface 512 , a sensor component 514 , and a communication component 516 .

[0170] The processing component 502 generally controls the overall operation of the device 500, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 502 may include one or more modules to facilitate the interaction between the processing component 502 and other components. For example, the processing component 502 may include a multimedia module to facilitate the interaction between the multimedia component 508 and the processing component 502.

[0171] The memory 504 is configured to store various types of data to support operations on the device 500. Examples of such data include instructions for any application or method operating on the device 500, contact data, phone book data, messages, pictures, videos, etc. The memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0172] Power component 506 provides power to various components of device 500. Power component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for device 500.

[0173] The multimedia component 508 includes a screen that provides an output interface between the device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front camera and / or a rear camera. When the device 500 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0174] The audio component 510 is configured to output and / or input audio signals. For example, the audio component 510 includes a microphone (MIC), and when the device 500 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 504 or sent via the communication component 516. In some embodiments, the audio component 510 also includes a speaker for outputting audio signals.

[0175] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0176] The sensor assembly 514 includes one or more sensors for providing various aspects of status assessment for the device 500. For example, the sensor assembly 514 can detect the open / closed state of the device 500, the relative positioning of components, such as the display and keypad of the device 500, and the sensor assembly 514 can also detect the position change of the device 500 or a component of the device 500, the presence or absence of user contact with the device 500, the orientation or acceleration / deceleration of the device 500, and the temperature change of the device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 514 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0177] The communication component 516 is configured to facilitate wired or wireless communication between the device 500 and other devices. The device 500 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0178] In an exemplary embodiment, the device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0179] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, which can be executed by a processor 520 of the device 500 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0180] A non-temporary computer-readable storage medium is also provided. When the instructions in the storage medium are executed by the processor of the mobile terminal, the device 500 is capable of executing a translation model training method or a paragraph-level machine translation method, wherein the method includes any implementation method of the aforementioned translation model training method embodiment, or the method includes any implementation method of the aforementioned paragraph-level machine translation method embodiment.

[0181] Those skilled in the art will readily appreciate other embodiments of the invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0182] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of protection of the present invention.< / beg> < / beg> < / beg> < / beg> < / beg> < / beg> < / beg> < / beg>

Claims

1. A translation model training method, characterized in that: include: For a target word in a passage-level source corpus, obtaining a complementary word of the target word from the passage-level source corpus, including: obtaining a related word of the target word in the passage-level source corpus; obtaining a complementary word of the target word according to the related word of the target word; According to the completion words, semantic completion is performed on the source sentences in which the target words appear in the chapter-level source corpus to form a completed chapter-level source corpus corresponding to the chapter-level source corpus; a target translation model is trained based on a plurality of the completed chapter-level source corpora and the corresponding target chapter corpora to obtain a trained chapter-level translation model.

2. The method according to claim 1, characterized in that The step of obtaining a complementary word for the target word from the passage-level source corpus includes: Performing dependency syntactic analysis on the text-level source corpus through a dependency syntactic analysis tool to extract the complement words of the target word, or The target translation model is used to extract the complement words of the target words from the paragraph-level source corpus.

3. The method according to claim 1, characterized in that The step of obtaining a complementary word for the target word from the passage-level source corpus includes: Extracting information from each source sentence in the chapter-level source corpus to obtain associated words for each word in the chapter-level source corpus; Determining the associated words of the target word according to the associated words of each word in the paragraph-level source corpus; According to the associated words of the target word, a complementary word of the target word is obtained.

4. The method according to claim 3, characterized in that The extracting information from each source sentence in the chapter-level source corpus to obtain the associated words of each word in the chapter-level source corpus includes: Performing word segmentation processing on each source sentence in the passage-level source corpus to obtain a word sequence corresponding to each source sentence; Using the dependency syntax analysis tool, a dependency syntax analysis is performed on the i-th word sequence corresponding to the i-th source sentence to obtain the dependency relationship between different words in the i-th word sequence, where i ranges from 1 to M, and M is the number of source sentences in the paragraph-level source corpus; For each word in the i-th word sequence, the associated word of the word in the i-th source sentence is determined according to the dependency relationship between different words in the i-th word sequence.

5. The method according to claim 3, characterized in that The step of obtaining a complementary word of the target word according to the associated words of the target word includes: Combining the associated words of the target word in different source-end sentences to obtain a set of completion words for semantically completing any source-end sentence in which the target word appears; or The associated words of the target word are extracted from different source end sentences, and the associated words that do not exist in the source end sentence where the target word is currently located are respectively screened out to obtain a group of completion words for semantically completing the source end sentence where the target word is currently located.

6. The method according to claim 1, characterized in that The step of obtaining a complementary word for the target word from the passage-level source corpus includes: Determining target words from the text-level source corpus; Only related words are extracted for the target word to obtain the complementary words of the target word.

7. The method according to claim 1, characterized in that The semantically completing the source sentence where the target word appears according to the completion word includes: A complement word of the target word is added at an adjacent position of the target word to semantically complete the source sentence where the target word is currently located, wherein a completion tag is set for the added complement word.

8. A paragraph-level machine translation method, characterized in that: include: For a target word in an original passage to be translated, a complementary word of the target word is obtained from the original passage to be translated, including: obtaining associated words of the target word in the passage-level source corpus; obtaining a complementary word of the target word according to the associated words of the target word; According to the completion words, semantic completion is performed on the source sentence where the target word appears, so as to form a completed chapter to be translated corresponding to the original chapter to be translated; The completed chapter to be translated is translated by using a trained chapter-level translation model to obtain a translated chapter, wherein the trained chapter-level translation model is obtained by the translation model training method described in any one of claims 1-7.

9. A translation model training device, characterized in that: include: The complement word acquisition unit is used to acquire the complement words of the target word in the chapter-level source-end corpus from the chapter-level source-end corpus, including: acquiring the associated words of the target word in the chapter-level source-end corpus; and obtaining the complement words of the target word according to the associated words of the target word; A semantic completion unit, configured to perform semantic completion on the source sentence in which the target word appears in the chapter-level source-end corpus according to the completion word, so as to form a completed chapter-level source-end corpus corresponding to the chapter-level source-end corpus; The model training unit is used to train the target translation model based on the multiple completed chapter-level source-end corpora and the corresponding target-end chapter corpora to obtain a trained chapter-level translation model.

10. A paragraph-level machine translation device, characterized in that: include: A complement word acquisition unit is used to acquire the complement words of the target word in the original chapter to be translated from the original chapter to be translated, including: acquiring the associated words of the target word in the chapter-level source corpus; and obtaining the complement words of the target word according to the associated words of the target word; A semantic completion unit, configured to perform semantic completion on the source sentence where the target word appears according to the completion word, so as to form a completed chapter to be translated corresponding to the original chapter to be translated; A chapter translation unit is used to translate the completed chapter to be translated by using a trained chapter-level translation model to obtain a translated chapter, wherein the trained chapter-level translation model is obtained by the translation model training method described in any one of claims 1-7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method as claimed in any one of claims 1 to 8 are implemented.

12. An electronic device, characterized in that: The invention comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute the operation instructions contained in the one or more programs for performing the method as claimed in any one of claims 1 to 8.

Images