Natural language processing device and program

Abstract

To provide a natural language processor and a program capable of switching a method according to a frequency of appearance of a source language expression and a target language expression.SOLUTION: A frequency information storage unit stores a frequency of appearance of a pair of a source language expression and a target language expression. An input data processing unit performs processing on a translation target sentence by: reading, for the translation target sentence written in a source language, from the frequency information storage unit, a frequency of appearance of the source language expression included in the translation target sentence; performing, when the frequency of appearance is equal to or greater than a predetermined threshold, translation insertion processing of adding information of the target language expression corresponding to the source language expression into the translation target sentence; and performing, when the frequency of appearance is less than the threshold, variable substitution processing of substituting a variable for the source language expression in the translation target sentence. A translation unit inputs the translation target sentence processed by the input data processing unit as the source language sentence to perform translation.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a natural language processing device and a program. [Background technology]

[0002] Neural machine translation (NMT) is a machine translation technique that uses neural networks. In neural machine translation, a sentence written in the source language is input and a translated sentence is output.

[0003] In machine translation, there is a technique for preparing bilingual dictionary data and using the bilingual dictionary to translate. The most basic way to use a bilingual dictionary is to include the bilingual expression pairs contained in the bilingual dictionary in the training data and train the bilingual expression pairs. This technique allows the information contained in the bilingual dictionary to be incorporated into the translation model.

[0004] However, the method of training a translation model using bilingual expression pairs contained in a bilingual dictionary cannot meet the need to always prioritize output of the translation contained in the dictionary (the expression in the target language of the bilingual expression pair).

[0005] There are two methods for always giving priority to the translation registered in a bilingual dictionary (the target language expression in a bilingual expression pair) in the output (so that it is included in the translation result): (1) variable substitution method and (2) translation insertion method.

[0006] The variable substitution technique is a technique in which a predetermined expression in an input sentence (such as an expression that constitutes a bilingual expression pair registered in a bilingual dictionary) is replaced with a variable, and then the input sentence is machine translated. The translation insertion technique is a technique in which the words contained in the output sentence are controlled by including a translation in the input sentence. Translation insertion techniques include (2-a) a technique in which the relevant expression in the input is replaced with an expression in the target language (translation), and (2-b) a technique in which a tag and an expression in the target language (translation) are inserted into the relevant part of the expression in the input.

[0007] Non-Patent Document 1 describes the variable substitution technique (1) above.

[0008] Non-Patent Document 2 describes methods (2-a) and (2-b) of the above-mentioned method (2) for inserting translated words.

[0009] Non-Patent Document 3 describes the above-mentioned (2) translation insertion method (2-a) of replacing a relevant expression during input with a translation. Non-Patent Document 4 describes the above-mentioned (2) translation insertion method (2-b) of inserting a tag and a translation into a relevant part of an expression during input. [Prior art documents] [Non-patent literature]

[0010] [Non-Patent Document 1] Zi Long, Takehito Utsuro, Tomoharu Mitsuhashi, Mikio Yamamoto, Translation of patent sentences with a large vocabulary of technical terms using neural machine translation, In Proceedings of the 3rd Workshop on Asian Translation (WAT2016), pp. 47-57, Osaka, Japan, December 2016, The COLING 2016 Organizing Committee. [Non-patent document 2] Matt Post,Shuoyang Ding,Marianna Martindale,Winston Wu,An exploration of placeholding in neural machine translation,In Proceedings of Machine Translation Summit XVII Volume 1: Research Track,pp. 182-192,Dublin,Ireland,August 2019,European Association for Machine Translation. [Non-patent document 3] Kai Song,Yue Zhang,Heng Yu,Weihua Luo,Kun Wang,Min Zhang,Code-switching for enhancing NMT with pre-specified translation,In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,Volume 1 (Long and Short Papers),pp. 449-459,Minneapolis,Minnesota,June 2019,Association for Computational Linguistics. [Non-patent document 4] Tao Wang, Shaohui Kuang, Deyi Xiong, Antonio Branco, Merging external bilingual pairs into neural machine translation, arXiv:1912.00567v1, 2019. Summary of the Invention [Problem to be solved by the invention]

[0011] Even in machine translation, there are cases where it is desirable to control the translation. However, the variable substitution method described above as a conventional technique has the problem that some information is lost when natural language vocabulary is replaced with variables. This loss of information leads to a deterioration in the quality of the translation, which is a problem.

[0012] On the other hand, the method of inserting translations (2) described in the prior art above has the problem that if the target language expression (translation) included in the input data has a low frequency of appearance in the training data, the target language expression may not be translated exactly as it is. In other words, the desired translation cannot be included in the translation result.

[0013] The present invention has been made in consideration of the above circumstances, and aims to provide a natural language processing device and program that can switch methods depending on the frequency of appearance of source language expressions and target language expressions in training data. [Means for solving the problem]

[0014] [1] In order to solve the above problem, a natural language processing device according to one aspect of the present invention comprises: a dictionary storage unit that stores data of a bilingual dictionary that indicates the bilingual relationship between a source language expression, which is an expression in a source language, and a target language expression, which is an expression in a target language; and a training data processing unit that generates training data for learning a translation model based on a bilingual sentence pair that is a pair of a source language sentence, which is a sentence in the source language, and a target language sentence, which is a sentence in the target language, by performing a translation insertion process that adds information about the target language expression that corresponds to the source language expression that appears in the source language sentence of the bilingual sentence pair to the source language sentence, and by performing a variable substitution process that replaces the source language expression in the source language sentence and the target language expression in the target language sentence with a variable, thereby generating training data for learning a translation model.

[0015] [2] In addition, one aspect of the present invention is that the above-mentioned natural language processing device further includes a learning unit that controls the training of a translation model using the training data generated by the training data processing unit.

[0016] [3] Furthermore, one aspect of the present invention is the above-mentioned natural language processing apparatus, further comprising: a frequency information storage unit that stores the frequency of occurrence of pairs of the source language expression and the target language expression; a translation unit that has a translation model that has been trained using the training data generated by the training data processing unit, translates an input source language sentence, and outputs a target language sentence; and an input data processing unit that processes the sentence to be translated by reading from the frequency information storage unit the frequency of occurrence of source language expressions contained in the sentence to be translated written in the source language, and if the frequency of occurrence is equal to or greater than a predetermined threshold, performs a translation insertion process that adds information about the target language expression corresponding to the source language expression to the sentence to be translated, or if the frequency of occurrence is less than the threshold, performs a variable substitution process that replaces the source language expression in the sentence to be translated with a variable, and the translation unit inputs the sentence to be translated processed by the input data processing unit as the source language sentence and translates it.

[0017] [4] In one aspect of the present invention, the natural language processing device further includes a frequency calculation unit that calculates the frequency of occurrence of the pair of the source language expression and the target language expression in the set of bilingual sentence pairs, and writes the frequency of occurrence of the pair of the source language expression and the target language expression in the frequency information storage unit.

[0018] [5] Another aspect of the present invention is a translation model trained using: a frequency information storage unit that stores occurrence frequencies of pairs of source language expressions and target language expressions; and, based on a bilingual sentence pair that is a pair of a source language sentence that is a sentence in a source language and a target language sentence that is a sentence in a target language, training data obtained by performing a translation insertion process to add information about a target language expression corresponding to the source language expression that appears in the source language sentence of the bilingual sentence pair to the source language sentence; and training data obtained by performing a variable substitution process to replace the source language expression in the source language sentence and the target language expression in the target language sentence with a variable. and an input data processing unit that processes the sentence to be translated by reading from the frequency information storage unit the frequency of occurrence of a source language expression contained in the sentence to be translated written in the source language, and if the frequency of occurrence is equal to or greater than a predetermined threshold, performing a translation insertion process that adds information on the target language expression corresponding to the source language expression to the sentence to be translated, and if the frequency of occurrence is less than the threshold, performing a variable substitution process that replaces the source language expression in the sentence to be translated with a variable.The translation unit inputs the sentence to be translated processed by the input data processing unit as the source language sentence and translates it.

[0019] [6] Furthermore, in one aspect of the present invention, in the above-mentioned natural language processing device, when the input data processing unit performs a translation insertion process to add information about the target language expression corresponding to the source language expression to the sentence to be translated if the occurrence frequency is equal to or greater than a predetermined threshold, and when the target language expression added to the sentence to be translated in the translation insertion process is not included in the translation result output by the translation unit as a translation corresponding to the source language sentence, the input data processing unit performs the variable substitution process instead of the translation insertion process for the source language expression whose occurrence frequency is equal to or greater than the threshold, and the translation unit inputs the sentence to be translated that is the result of the variable substitution process instead of the translation insertion process as the source language sentence and translates it again.

[0020] [7] Also, one aspect of the present invention is a program for causing a computer to function as a natural language processing device, comprising: a dictionary storage unit that stores data of a bilingual dictionary that indicates the bilingual relationship between a source language expression, which is an expression in a source language, and a target language expression, which is an expression in a target language; and a training data processing unit that generates training data for training a translation model based on a bilingual sentence pair that is a pair of a source language sentence, which is a sentence in the source language, and a target language sentence, which is a sentence in the target language, by performing a translation insertion process to add information about the target language expression that corresponds to the source language expression that appears in the source language sentence of the bilingual sentence pair to the source language sentence, and by performing a variable substitution process to replace the source language expression in the source language sentence and the target language expression in the target language sentence with a variable, thereby generating training data for training a translation model.

[0021] [8] In another aspect of the present invention, a translation model is provided that is trained using a frequency information storage unit that stores the occurrence frequency of pairs of source language expressions and target language expressions, and a translation model that is trained using training data obtained by performing a translation insertion process to add information on a target language expression corresponding to a source language expression that appears in a source language sentence of a bilingual sentence pair and a target language expression that corresponds to the source language expression that appears in the target language sentence, and a variable substitution process to replace the source language expression in the source language sentence and the target language expression in the target language sentence with a variable, based on a bilingual sentence pair that is a pair of a source language sentence that is a sentence in a source language and a target language sentence that is a sentence in a target language. and outputs a target language sentence; and an input data processing unit that processes the sentence to be translated by reading from the frequency information storage unit the frequency of occurrence of source language expressions contained in the sentence to be translated written in the source language, and if the frequency of occurrence is equal to or greater than a predetermined threshold, performing a translation insertion process that adds information about the target language expression corresponding to the source language expression to the sentence to be translated, and if the frequency of occurrence is less than the threshold, performing a variable substitution process that replaces the source language expression in the sentence to be translated with a variable, wherein the translation unit inputs the sentence to be translated processed by the input data processing unit as the source language sentence and translates it. [Effects of the Invention]

[0022] According to the present invention, training data for learning a translation model can be processed by appropriately using either the translation insertion method or the variable substitution method depending on the frequency of appearance of linguistic expressions (words, etc.), which makes it easier to control translations. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a block diagram showing a schematic functional configuration of a natural language processing apparatus according to an embodiment of the present invention; [Figure 2]3 is a schematic diagram showing the configuration of data stored in a training data storage unit according to the embodiment. FIG. [Figure 3] 3 is a schematic diagram showing an example of the configuration of bilingual dictionary data stored in a dictionary storage unit according to the embodiment. FIG. [Figure 4] 3 is a schematic diagram showing the configuration of data stored in a frequency information storage unit according to the embodiment. FIG. [Figure 5] 10 is a schematic diagram showing an example of a translation insertion process performed by a training data processing unit according to the embodiment. FIG. [Figure 6] 10 is a schematic diagram showing an example of a variable substitution process performed by a training data processing unit according to the embodiment. FIG. [Figure 7] 10 is a flowchart showing a processing procedure for a frequency calculation unit according to the embodiment to calculate the frequency of a bilingual expression pair. [Figure 8] 10 is a flowchart showing the procedure of processing training data according to the embodiment. [Figure 9] 10 is a flowchart showing the procedure of a translation process according to the embodiment. [Figure 10] FIG. 2 is a block diagram showing an example of the internal configuration of the natural language processing apparatus according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0024] Next, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, in a translation process using a model trained by machine learning using training data, a number of different methods are used depending on the frequency of occurrence of bilingual expression pairs in the training data.

[0025] Specifically, for pairs of bilingual expressions (pairs of source and target language expressions) whose frequency of occurrence is equal to or greater than a predetermined threshold, the translation output is controlled by inserting information about the translation (target language expression) into the input sentence (source language sentence). Conversely, for pairs of bilingual expressions whose frequency of occurrence is less than the threshold, the corresponding expression in the input sentence is replaced with a variable, the replaced sentence is translated, and the variable appearing in the output sentence is replaced with the target language expression.

[0026] Furthermore, even if the occurrence frequency of a bilingual expression pair is above the threshold, if the target target language expression is not included in the output sentence even after translation insertion into the input sentence, the method may be further switched to variable substitution. In other words, two-stage translation using different methods may be performed. This makes it possible to take measures against the case where the target target language expression is not translated, which may occur if the frequency threshold is set too low.

[0027] The term "variable" used here corresponds to the English word "variable." The values ​​that variables in this embodiment take are linguistic expressions. Although variables do not represent numerical values, the term "variable" will be used in the following explanation. The term "variable" may also be used instead of "variable."

[0028] In this embodiment, a case will be described in which the source language (language to be translated) is Japanese and the target language (language to be translated) is English. However, the source language and the target language may each be any language. For example, the source language and the target language may each be English, German, Japanese, French, Russian, Spanish, Italian, Chinese, Korean, or another language.

[0029] FIG. 1 is a block diagram showing a schematic functional configuration of a natural language processing apparatus according to this embodiment. As shown in the figure, the natural language processing apparatus 1 includes a training data storage unit 111, a frequency calculation unit 112, a frequency information storage unit 113, a training data processing unit 114, a dictionary storage unit 16, a learning unit 17, an input data processing unit 121, a translation unit 122, and an output data processing unit 123. Each function of the natural language processing apparatus 1 can be realized, for example, by a computer and a program. Each functional unit also includes a storage unit as needed. The storage unit is, for example, a memory for storing program variables or allocated by program execution. Non-volatile storage units such as a magnetic hard disk drive or a solid-state drive (SSD) may also be used as needed. At least some of the functions of each functional unit may also be realized by a dedicated electronic circuit rather than a program.

[0030] Of the functions of the natural language processing device 1, the functions of the training data storage unit 111, the frequency calculation unit 112, and the frequency information storage unit 113 may be called the training data processing device 11. Also, of the functions of the natural language processing device 1, the functions of the input data processing unit 121, the translation unit 122, and the output data processing unit 123 may be called the translation device 12.

[0031] The training data processing device 11 processes the training data. Specifically, the training data processing device 11 processes the training data in a manner according to the frequency of appearance in the training data of pairs of source language expressions and target language expressions that appear in the training data.

[0032] The translation device 12 translates an input sentence to be translated (a sentence in a source language) and outputs a translation result sentence (a sentence in a target language) as the translation result. The translation device 12 has an internal translation model. This translation model can be trained by machine learning processing using training data processed by the training data processing device 11.

[0033] The training data storage unit 111 stores training data used to train a translation model. The training data is a collection of pairs of source language sentences and target language sentences (bilingual sentence pairs). In addition to the original bilingual sentence pairs, the training data storage unit 111 also stores bilingual sentence pairs generated by the processing of the training data processing unit 114.

[0034] The frequency calculation unit 112 calculates the frequency of occurrence of a pair of a source language expression and a target language expression, and writes the frequency of occurrence of the pair of the source language expression and the target language expression in the frequency information storage unit 113. The frequency calculation unit 112 calculates the frequency of occurrence by, for example, counting the number of times the pair of a source language expression and a target language expression appears in a set of bilingual sentence pairs stored in advance in the training data storage unit 111.

[0035] The frequency information storage unit 113 stores the occurrence frequency for pairs of source language expressions and target language expressions. Specifically, the frequency information storage unit 113 stores a list of information on pairs of source language expressions, target language expressions, and their frequency values. The occurrence frequency is calculated by the frequency calculation unit 112 based on predetermined training data, for example.

[0036] The training data processing unit 114 processes the training data stored in the training data storage unit 111. The training data processing unit 114 writes the training data resulting from the processing into the training data storage unit 111. The training data (bilingual sentence pairs) originally stored in the training data storage unit 111 is used as is for training the translation model. In addition, the training data (processed bilingual sentence pairs) generated by processing by the training data processing unit 114 is also used for training the translation model.

[0037] The training data processing unit 114 processes bilingual sentence pairs (pairs of source and target language sentences) included in the training data. The training data processing unit 114 uses a plurality of processing methods to generate (add) training data by each processing method. Specifically, as a first processing process, the training data processing unit 114 adds information on a target language expression corresponding to a source language expression to the source language sentence of the bilingual sentence pair (translation insertion process). As a second processing process, the training data processing unit 114 replaces the source language expression in the source language sentence and the target language expression in the target language sentence with a variable (variable substitution process). The translation insertion process and the variable substitution process will be further described later with reference to separate figures.

[0038] When performing the translation insertion process, the training data processing unit 114 may use tags to include information on the correspondence between the target language expression and the source language expression in the source language sentence. In other words, in the training data, information on the inserted translation is associated with the source language expression. A specific example of this method will be described later.

[0039] Furthermore, when performing the variable substitution process, the training data processing unit 114 randomly assigns variable identification information for identifying variables to pairs of source language expressions and target language expressions. A specific example of this method will be described later.

[0040] The training data processing unit 114 processes training data that has been stored in advance in the training data storage unit 111 to generate new training data, and adds this new training data to the training data storage unit 111. The added training data is also used for learning the translation model, which will be described later.

[0041] The dictionary storage unit 16 stores a bilingual dictionary. The bilingual dictionary is data that represents the correspondence between expressions in a source language and a target language. The bilingual dictionary is data that represents the bilingual relationship between source language expressions, which are expressions in the source language, and target language expressions, which are expressions in the target language.

[0042] The learning unit 17 controls the translation unit 122 to learn the translation model using the training data read from the training data storage unit 111. The training data used by the learning unit 17 for learning includes training data generated (added) by the training data processing unit 114. In other words, the learning unit 17 also uses training data processed by the above-mentioned translation term insertion process and variable substitution process to learn the translation model.

[0043] The input data processing unit 121 processes the input sentence to be translated (sentence in source language) and passes the processed sentence to be translated to the translation unit 122. Specifically, the input data processing unit 121 reads the occurrence frequency of the source language expression included in the input sentence to be translated from the frequency information storage unit 113, and performs processing according to the occurrence frequency.

[0044] If the occurrence frequency is equal to or greater than a predetermined threshold, the input data processing unit 121 performs a translation insertion process, which adds information about a target language expression corresponding to the source language expression to the sentence to be translated. The correspondence between the source language expression and the target language expression is obtained from the data in the bilingual dictionary in the dictionary storage unit 16. The translation insertion process performed by the input data processing unit 121 is the same as the translation insertion process performed by the training data processing unit 114, which has already been described. If the occurrence frequency is less than the threshold, the input data processing unit 121 performs a variable substitution process, which replaces the source language expression in the sentence to be translated with a variable.

[0045] Note that the input data processing unit 121 may perform additional processing. Specifically, if the translation term inserted by the above-described translation term insertion process is not translated by the translation unit 122, the input data processing unit 121 switches to a variable substitution processing method and processes the source language expression again. That is, (1) if the translation term insertion process is performed to add information about a target language expression corresponding to the source language expression to the translation target sentence when the occurrence frequency is equal to or greater than a predetermined threshold, and (2) if the target language expression added to the translation target sentence in the translation term insertion process is not included in the translation result output by the translation unit 122 as a translation term corresponding to the source language sentence, the input data processing unit 121 also performs variable substitution processing instead of translation term insertion processing for the source language expression whose occurrence frequency is equal to or greater than the threshold.

[0046] The translation unit 122 translates an input source language sentence and outputs a target language sentence as the translation result. Specifically, the translation unit 122 inputs a translation target sentence processed by the input data processing unit 121 as a source language sentence and translates it. The translation unit 122 is equipped with a translation model capable of machine learning. The translation model is realized using, for example, a neural network. Before the translation unit 122 performs translation processing, machine learning of this translation model is performed in advance using training data.

[0047] That is, the translation unit 122 is equipped with a trained translation model. The translation model has been trained using predetermined training data. The training data includes at least training data obtained by performing a translation insertion process, based on a bilingual sentence pair, which is a pair of a source language sentence in a source language and a target language sentence in a target language, for a pair of a source language expression appearing in the source language sentence of the bilingual sentence pair and a target language expression corresponding to the source language expression appearing in the target language sentence, in which information about the target language expression corresponding to the source language expression appears in the source language sentence is added to the source language sentence, and training data obtained by performing a variable substitution process, in which the source language expression in the source language sentence and the target language expression in the target language sentence are replaced with variables. The translation unit 122 then uses the translation model to translate the input source language sentence and output the target language sentence.

[0048] If the input data processing unit 121 performs the above-mentioned additional processing, the translation unit 122 inputs the translation target sentence resulting from variable substitution processing instead of translation word insertion processing as the source language sentence and performs translation again.

[0049] The output data processing unit 123 processes the data output by the translation unit 122. Specifically, if the translation result sentence output by the translation unit 122 includes a variable, the output data processing unit 123 replaces the variable with a target language expression. The variable is identified by variable identification information. An identifiable variable is assigned to a pair of a source language expression and a target language expression. In other words, since the variable corresponds to the target language expression, the output data processing unit 123 replaces the variable with the target language expression based on this correspondence.

[0050] 2 is a schematic diagram showing the configuration of data stored in the training data storage unit 111. As shown in the figure, the training data storage unit 111 stores training data as a set of pairs of source language sentences and target language sentences. As an example, the training data storage unit 111 stores the training data as data in a table format. Note that in addition to the original pairs of source language sentences and target language sentences, the training data storage unit 111 can also store data resulting from descent by the training data processing unit 114 in this embodiment.

[0051] 3 is a schematic diagram showing an example of the structure of bilingual dictionary data stored in the dictionary storage unit 16. The bilingual dictionary data is data that represents the relationship between source language expressions and target language expressions. i and target language expression t i The dictionary storage unit 16 stores the bilingual dictionary data as data in a table format, for example. In the illustrated example, each row of the table contains a source language expression (s i ) and the target language expression (t i ) are mutually associated. An example of an entry in the bilingual dictionary data is a source language expression "Oshamanbe" and a corresponding target language expression "Oshamanbe."

[0052] 4 is a schematic diagram showing the configuration of data stored in the frequency information storage unit 113. As shown in the figure, the frequency information storage unit 113 stores source language expressions (s i) and the target language expression (t i ) and frequency (f i ) as a set of pairs. Here, i is a numerical value (for example, i=1, 2, 3, . . .) for identifying pairs of source language expressions and target language expressions. Pairs of source language expressions and target language expressions are also associated in the dictionary storage unit 16. The frequency is information indicating the frequency with which a pair of a source language expression and a target language expression appears in the entire training data. The frequency may be, for example, a numerical value indicating the number of times a pair of a source language expression and a target language expression appears.

[0053] 5 is a schematic diagram showing an example of the translation insertion process performed by the training data processing unit 114. As shown in the figure, the training data processing unit 114 performs the translation of a source language sentence S k Source language expressions in i In the part, s i The target language expression t corresponding to i In other words, the training data processing unit 114 inserts the translated word into the source language sentence. The source language sentence with the translated word inserted is called S'. k Let's say.

[0054] In the example of Figure 5, the source sentence S k In Figure 5, the tokens that make up the sentence are separated by a full-width slash ( / ). i "Oshamanbe" and t i If there is a pair of "Oshamanbe", the training data processing unit 114 calculates S' as follows: k Create the data of the tag. <term> 、 <delim / > 、< / term> The training data processing unit 114 creates S' using k "He / is / <term> / Ten Thousand Volumes / <delim / > / Oshamanbe / < / term> "Go to / to / ." In other words, the training data processing unit 114 inserts the target language expression "Oshamanbe," which is information on the translation corresponding to the source language expression "Oshamanbe," into the source language sentence in the training data. <term> indicates the start of a pair of bilingual expressions.< / term> indicates the end of a pair of bilingual expressions. <delim / > The tag <term> and< / term>In the pair of bilingual expressions shown by and, it is the delimiter that separates the source language expression (Oshamanbe) and the target language expression (Oshamanbe). k contains information indicating the correspondence between source language expressions contained in the source language sentence and their translations.

[0055] The training data processing unit 114 generates S', which is the result of the processing of the translation insertion. k and target sentence T k The pair of and is added as training data.

[0056] 6 is a schematic diagram showing an example of variable substitution processing performed by the training data processing unit 114. As shown in the figure, the training data processing unit 114 performs variable substitution processing on the source language sentence S k Source language expressions contained in i is replaced by a tag corresponding to a variable. This variable represents a specific expression in a general way. The source language sentence after this replacement is called S´´. k For example, the tag corresponding to the variable is <term-j / > where j=1, 2, ..., N. This j is variable identification information for identifying a variable.

[0057] As shown in Figure 6, source language sentence S k is "He goes to Oshamanbe." If the source language expression "Oshamanbe" is replaced by the above variable (for example, j=5), the source language sentence S´´ after the replacement is k "He / is / <term-5 / > On the other hand, if the target sentence is T, k If "He / goes / to / Oshamanbe / .", then the source sentence S´´ after substitution k The corresponding target sentence T´´ after substitution k "He / goes / to / <term-5 / > / .". Note that in Figure 6, token separators are also represented by full-width slashes ( / ). Here, j=5 is used as an example, but the value of j can be assigned so that it is unique and can be distinguished between multiple variables in the same sentence. As an example, the value of j can be assigned randomly. The specific procedure is to randomly shuffle the integer values ​​1, 2, ..., N for each sentence, and assign the shuffled values ​​to the variables in the sentence in order, starting from the first value.

[0058] It should be noted that one variable does not necessarily correspond to only one word. One variable may correspond to two or more words. For example, in the source language sentence S k is "She is going to New York City." If we replace the source language expression "New York City" with a variable (for example, j=7), we get the source language sentence S´´ k "She / is / <term-7 / > On the other hand, if the target sentence is T, k If "She / goes / to / New / York / City / .", then the source sentence S´´ after substitution k The corresponding target sentence T´´ after substitution k is "She / goes / to / <term-7 / > / ."

[0059] By assigning the value of j randomly as described above, it becomes possible to learn about tags that represent variables evenly. Note that the value of N should be sufficiently large. In other words, N should be set to a value equal to or greater than the number of variables that can be used in a sentence (the number of variables that are expected to need to be translated using a bilingual dictionary in a sentence).

[0060] The training data processing unit 114 processes the source language sentence S'. k and target sentence T´´ k The pair of , is added to the training data.

[0061] That is, the training data processing unit 114 performs the above-mentioned process of substitution by variables for each dictionary entry (pair of bilingual expressions) and each training data (pair of bilingual sentences). When one bilingual sentence pair contains multiple bilingual expressions registered in the dictionary, these processes may be performed for each of the multiple expressions. When one bilingual sentence pair contains multiple bilingual expressions registered in the dictionary, data (bilingual sentence pairs) obtained by substituting variables for only some of the multiple expressions may be added to the training data. When substituting variables for expressions, the training data processing unit 114 may perform the process of substitution by variables for the same source language sentence S. k By changing the way the variable number (j above) is assigned, multiple parallel sentence pairs (source sentence S´´ i and target sentence T´´ i You can also create a pair of , and add it to the training data.

[0062] Next, a description will be given of the overall processing of the natural language processing apparatus 1. The processing of the training phase and the processing of the translation phase by the natural language processing apparatus 1 will be described.

[0063] [Training Phase] First, the processing in the training phase will be described.

[0064] In the training phase, the frequency calculation unit 112 first calculates the frequency with which entries in the bilingual dictionary appear in bilingual sentence pairs in the training data for neural machine translation. i and target language expression t i The frequency (number of times) that a pair of appears in the training data is f i That is, the frequency calculation unit 112 calculates a large number of triples (s i ,t i ,f i ) is created. The frequency calculation unit 112 writes the information of this list L into the frequency information storage unit 113.

[0065] Next, the training data processing unit 114 processes the training data and adds the processed data as training data. The training data processing unit 114 adds and writes the processed data to the training data storage unit 111. i and t i is a source sentence S that is a bilingual sentence pair in the training data. k and target sentence T k If k is included in the above, the training data processing unit 114 processes the data by appropriately using two methods, translation insertion processing and variable substitution processing, and adds the processed data to the training data. Note that k is the ID of the bilingual sentence pair in the training data. The translation insertion processing is as described with reference to FIG. 5. The variable substitution processing is as described with reference to FIG. 6.

[0066] Increasing the variety of expressions in the training data in the above manner makes it possible to prevent overfitting in the training of a model for neural machine translation. The training unit 17 reads the training data from the training data storage unit 111 and causes the translation model held by the translation unit 122 to perform training.

[0067] [Translation Phase] Next, the processing in the translation phase will be described. As a prerequisite for the processing in the translation phase, the translation model held by the translation unit 122 has already been trained using the training data described above.

[0068] The translation unit 122 translates an input source language sentence and outputs a target language sentence. i This source expression s i is the result of machine translation of the target language expression t i It is desirable to translate the source language expressions into i and target language expression t i are associated with each other in the bilingual dictionary stored in the dictionary storage unit 16.

[0069] In the translation process, the input data processing unit 121 adds a source language expression s to an input sentence (source language sentence) S to be translated. i If the frequency list L (frequency information storage unit 113) is searched, the pair (s i ,t i ) frequency f i Get this f i is equal to or greater than the threshold value TH, the input data processing unit 121 detects that the input sentence S contains the target language expression t i The threshold value TH may be, for example, a predetermined fixed value. The value of the threshold value TH is determined appropriately. For example, the value of TH may be set to 2 or 3.

[0070] For example, when the source language sentence S to be translated is "She went to Oshamanbe.", the input data processing unit 121 refers to the frequency list L and calculates the frequency f of (Oshamanbe, Oshamanbe), which is a pair of a source language expression and a target language expression. i The frequency f i is equal to or greater than the threshold value TH, the input data processing unit 121 inserts the translation and the tag into the source language sentence S to generate the processed source language sentence S' as "She is <term>Chang Wanbu <delim / > Oshamanbe< / term> The input data processing unit 121 passes the processed source language sentence S′ to the translation unit 122.

[0071] The translation unit 122 performs translation processing (neural machine translation) based on the processed source language sentence S' and outputs a translation result sentence (target language sentence). Since the source language sentence S' has information on the translation word ("Oshamanbe" in the above example), the translation result sentence output by the translation model of the translation unit 122 uses the translation word (the source language expression s in the dictionary). i The target language expression t corresponding to i ) is likely to be included in the source sentence S' "She / is / <term> / Ten Thousand Volumes / <delim / > / Oshamanbe / < / term> When "She went to Oshamanbe / ." is input as the sentence to be translated, the translation unit 122 is expected to output the translation result sentence "She went to Oshamanbe / .", which includes the specified translation "Oshamanbe."

[0072] Frequency f of source and target language expressions from list L i When we obtain this frequency f i If the value of is less than the threshold value TH, the input data processing unit 121 performs the source language expression f instead of inserting the translation. i is replaced with a tag representing the variable.

[0073] That is, the frequency value f i is less than TH, the input data processing unit 121 processes the source language sentence S and outputs "She / is / " as sentence S'. <term-5 / > As mentioned above, the value "5" used to identify the variables is assigned randomly. Each different variable in the sentence is uniquely identified by this value. The machine-learned model translates the above sentence S' and outputs the translation result. The tags of the variables contained in the input sentence are generated by machine learning that has been performed in advance. <term-5 / > In response to this, the translation result sentence contains the variable tag <term-5 / > Specifically, when the above source language sentence S'' is input, the translation unit 122 is expected to output the following target language sentence T''. That is, T'' is "She / went / to / <term-5 / > The output data processing unit 123 uses the tag of the variable on the output side as <term-5 / > is replaced with the target language expression “Oshamanbe.” In other words, the output data processing unit 123 outputs “She went to Oshamanbe.” as the translation result sentence.

[0074] In addition, the frequency value f i is equal to or greater than TH, and the result of translating the processed source language sentence S' by the translation unit 122 includes the source language expression s i The target language expression t corresponding to i If the target language expression t is not included, the translation result may be discarded and processed using the variable substitution technique. i This can increase the likelihood of including

[0075] Next, the processing procedure in each phase of the natural language processing apparatus 1 will be described with reference to flowcharts.

[0076] 7 is a flowchart showing the procedure of processing by the frequency calculation unit 112 to calculate the frequency of a bilingual expression pair. As shown in the figure, first, in step S11, the frequency calculation unit 112 determines whether or not there are any unprocessed bilingual expression pairs remaining in the bilingual dictionary. If there are any unprocessed bilingual expression pairs remaining (step S11: YES), the process proceeds to the next step S12 to process those bilingual expression pairs. If there are no unprocessed bilingual expression pairs remaining (step S11: NO), the entire process of this flowchart ends.

[0077] Next, in step S12, the frequency calculation unit 112 extracts one of the unprocessed pairs of bilingual expressions.

[0078] Next, in step S13, the frequency calculation unit 112 counts the frequency of the bilingual expression pair in the entire training data.

[0079] Next, in step S14, the frequency calculation unit 112 writes the frequency of the bilingual expression pair into the frequency information storage unit 113. That is, the frequency calculation unit 112 calculates the frequency of the source language expression (s i ) and the target language expression (t i ) and frequency value (f i ) is recorded as part of the list L. After the process of step S14 is completed, the process returns to step S11.

[0080] 8 is a flowchart showing the procedure for processing training data. As shown in the figure, first, in step S21, the training data processing unit 114 determines whether or not there are unprocessed bilingual text pairs in the training data. If there are unprocessed bilingual text pairs remaining (step S21: YES), the process proceeds to the next step S22 to process those bilingual text pairs. If there are no unprocessed bilingual text pairs remaining (step S21: NO), the entire process of this flowchart ends.

[0081] Next, in step S22, the training data processing unit 114 extracts one unprocessed pair of other translation sentences from the training data.

[0082] Next, in step S23, the training data processing unit 114 identifies bilingual expression pairs included in the bilingual sentence pair extracted in step S22, while referring to the bilingual dictionary stored in the dictionary storage unit 16. The number of bilingual expression pairs included in the bilingual sentence pair is 0 or more.

[0083] The processes of the next steps S24 and S25 are executed in parallel.

[0084] In step S24, the training data processing unit 114 performs a translation insertion process for each of the bilingual sentence pairs identified in step S23. The translation insertion process has already been described.

[0085] In addition, in step S25, the training data processing unit 114 performs variable substitution processing on each of the bilingual sentence pairs identified in step S23. The variable substitution processing has already been described.

[0086] That is, by the processing of steps S24 and S25, the training data processing unit 114 processes the current bilingual sentence pair in the training data using the methods of translation insertion and variable substitution as appropriate. Note that in this embodiment, the processing of steps S24 and S25 is executed in parallel, but these two processes may also be executed sequentially. In this case, if multiple bilingual expression pairs are identified for the same sentence in step S23, some of them may be processed in step S24 and the rest may be processed in step S25, thereby mixing the two processes.

[0087] Next, in step S26, the training data processing unit 114 outputs the processed bilingual sentence pair as data to be added to the training data. When the processing of this step is completed, the processing returns to step S21 to move on to processing the next bilingual sentence pair. In other words, the training data processing unit 114 adds the training data processed by each of the translation word insertion and variable substitution methods to the original training data.

[0088] 9 is a flowchart showing the procedure of the translation process. As shown in the figure, first, in step S41, the input data processing unit 121 identifies source language expressions registered in the bilingual dictionary in the sentence to be translated (source language sentence). The sentence to be translated includes zero or more registered source language expressions.

[0089] Next, in step S42, the input data processing unit 121 acquires a frequency value for each of the source language expressions identified in step S41 by referring to the frequency information storage unit 113. This frequency value is information contained in the list L described above.

[0090] Next, in step S43, the input data processing unit 121 performs a translation insertion process on source language expressions whose frequency values ​​are equal to or greater than a predetermined threshold (TH) among the source language expressions identified in step S41, while referring to the bilingual dictionary stored in the dictionary storage unit 16. The translation insertion process is the process performed on training data, as explained with reference to Fig. 5. In this step, the input data processing unit 121 performs the same translation insertion process on the sentence to be translated.

[0091] Next, in step S44, the input data processing unit 121 performs variable substitution processing on source language expressions whose frequency values ​​are less than the threshold value (TH) among the source language expressions identified in step S41. The variable substitution processing is the processing performed on training data that was explained with reference to Fig. 6. In this step, the input data processing unit 121 performs the same variable substitution processing on the sentence to be translated.

[0092] Next, in step S45, the translation unit 122 translates the source language sentence processed in the processes of steps S43 and S44. When variable substitution processing is performed, the output data processing unit 123 converts the variables included in the output sentence into target language expressions (translations) corresponding to the source language expressions.

[0093] Next, in step S46, the natural language processing apparatus 1 determines whether or not the expected target language expression has been translated as a result of the translation term insertion process. That is, for the translation term insertion process performed in step S43, it determines whether or not the translation term (target language expression) is included in the sentence of the translation result (output from the translation unit 122). If all the expected target language expressions have been translated as a result of the translation term insertion process (step S46: YES), the translation process (the entire process of this flowchart) ends. If at least a part of the expected target language expression has not been translated as a result of the translation term insertion process (step S46: NO), the process proceeds to the next step S47.

[0094] Next, when the process proceeds to step S47, in this step the natural language processing apparatus 1 switches to variable substitution processing for source language expressions that have not been translated into the expected target language expression by the translation word insertion technique. That is, for such source language expressions, variable substitution processing is performed as pre-processing for translation. That is, variable substitution processing is performed for source language expressions whose frequency is equal to or greater than the threshold value (TH). For target language expressions that have been translated as expected in the translation processing of step S45, translation word insertion processing may be performed instead of variable substitution processing. Furthermore, for source language expressions whose frequency is less than the threshold value (TH), variable substitution processing is performed in the same way as in the processing of step S44.

[0095] Next, in step S48, the translation unit 122 translates the source language sentence that has been preprocessed in step S47. For expressions that have undergone variable substitution processing, the output data processing unit 123 converts the variables included in the output sentence into target language expressions (translations) that correspond to the source language expressions. After this step is completed, the natural language processing device 1 ends the entire processing of this flowchart.

[0096] FIG. 10 is a block diagram showing an example of the internal configuration of the natural language processing apparatus 1 of the above embodiment. The natural language processing apparatus 1 can be realized using a computer. As shown in the figure, the computer includes a central processing unit 901, a RAM 902, an input / output port 903, input / output devices 904 and 905, etc., and a bus 906. The computer itself can be realized using existing technology. The central processing unit 901 executes instructions contained in a program read from the RAM 902, etc. In accordance with each instruction, the central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic and logical operations. The RAM 902 stores data and programs. Each element included in the RAM 902 has an address and can be accessed using the address. RAM is an abbreviation for "random access memory." The input / output port 903 is a port through which the central processing unit 901 exchanges data with external input / output devices, etc. The input / output devices 904 and 905 are input / output devices. Input / output devices 904 and 905 exchange data with the central processing unit 901 via an input / output port 903. A bus 906 is a common communication path used within the computer. For example, the central processing unit 901 reads and writes data from and to RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input / output port via the bus 906.

[0097] At least some of the functions of the natural language processing device 1 in the above-described embodiment can be realized by a computer and a program. In this case, the functions can be realized by recording a program for realizing the functions on a computer-readable recording medium and loading and executing the program recorded on the recording medium into a computer system. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, CD-ROMs, DVD-ROMs, and USB memory, as well as storage devices such as hard disks built into computer systems. In other words, a "computer-readable recording medium" may be a non-transitory computer-readable recording medium. Furthermore, the term "computer-readable recording medium" may also include media that temporarily and dynamically store programs, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or media that store programs for a certain period of time, such as volatile memory within a computer system that serves as a server or client in such cases. The program may be designed to realize some of the functions described above, or may be capable of realizing the functions described above in combination with a program already stored in the computer system.

[0098] As described above, according to this embodiment, the natural language processing apparatus 1 selectively uses the translation insertion method and the variable substitution method depending on the frequency of appearance of the linguistic expression, thereby controlling the translation so that the expected target language expression is included in the translation result.

[0099] When the frequency of appearance of a language expression is relatively high (above a threshold), the natural language processing apparatus 1 uses a translation insertion technique, which allows the translation model to be trained without losing information about the source language expression and the target language expression, and also allows the expected target language expression to be included in the translation result.

[0100] When the frequency of appearance of a language expression is relatively low (below a threshold), the natural language processing device 1 uses a variable substitution technique, which prioritizes the certainty that the expected translation will be included in the translation result over the loss of information about the source language expression and the target language expression.

[0101] If the translation result obtained using the word insertion method does not include the expected translation, the natural language processing device 1 performs a re-translation process for that linguistic expression using a variable substitution method, thereby increasing the possibility that the expected translation will be included in the translation result.

[0102] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Industrial Applicability]

[0103] The present invention can be used in any industry that uses translation processing of sentences written in natural languages, but the scope of use of the present invention is not limited to the examples given here. [Explanation of symbols]

[0104] 1 Natural Language Processing 11 Training data processing device 12 Translation device 16 Dictionary storage unit 17 Learning Department 111 Training data storage unit 112 Frequency Calculation Unit 113 Frequency information storage unit 114 Training Data Processing Department 121 Input Data Processing Department 122 Translation Department 123 Output Data Processing Unit 901 Central Processing Unit 902 RAM 903 Input / Output Ports 904,905 Input / Output Devices 906 Bus

Claims

1. a dictionary storage unit that stores data of a bilingual dictionary that indicates a bilingual relationship between a source language expression that is an expression in a source language and a target language expression that is an expression in a target language; a training data processing unit that generates training data for training a translation model by performing a translation insertion process to add information about a target language expression corresponding to a source language expression that appears in the source language sentence of the bilingual sentence pair and a target language expression that corresponds to the source language expression that appears in the target language sentence, based on the bilingual sentence pair that is a pair of a source language sentence that is a sentence in the source language and a target language sentence that is a sentence in the target language, into the source language sentence, and also generates training data for training a translation model by performing a variable substitution process to replace the source language expression in the source language sentence and the target language expression in the target language sentence with a variable; a frequency information storage unit that stores the frequency of occurrence of pairs of the source language expression and the target language expression; a translation unit that includes a translation model that has been trained using the training data generated by the training data processing unit, that translates an input source language sentence, and outputs a target language sentence; an input data processing unit that processes the sentence to be translated, written in the source language, by reading from the frequency information storage unit the frequency of occurrence of source language expressions contained in the sentence to be translated, and if the frequency of occurrence is equal to or greater than a predetermined threshold, performing a translation insertion process that adds information about the target language expression corresponding to the source language expression to the sentence to be translated, and if the frequency of occurrence is less than the threshold, performing a variable substitution process that replaces the source language expression in the sentence to be translated with a variable; Equipped with the translation unit inputs the translation target sentence processed by the input data processing unit as the source language sentence and performs translation; Natural language processing device.

2. A learning unit that controls learning of a translation model using the training data generated by the training data processing unit; The natural language processing apparatus according to claim 1 , further comprising:

3. a frequency calculation unit that calculates the frequency of occurrence of the pair of the source language expression and the target language expression in the set of bilingual sentence pairs and writes the frequency of occurrence of the pair of the source language expression and the target language expression in the frequency information storage unit; The natural language processing apparatus according to claim 1 or 2, further comprising:

4. a frequency information storage unit that stores the occurrence frequency of pairs of source language expressions and target language expressions; a translation unit that translates an input source language sentence and outputs a target language sentence, the translation unit including a translation model that has been trained using training data obtained by performing a translation insertion process to add information about a target language expression that corresponds to a source language expression that appears in the source language sentence of the bilingual sentence pair and a target language expression that corresponds to the source language expression that appears in the target language sentence, and a variable substitution process to replace the source language expression in the source language sentence and the target language expression in the target language sentence with a variable, based on a bilingual sentence pair that is a pair of a source language sentence that is a sentence in a source language and a target language sentence that is a sentence in a target language; an input data processing unit that processes the sentence to be translated, written in the source language, by reading from the frequency information storage unit the frequency of occurrence of source language expressions contained in the sentence to be translated, and if the frequency of occurrence is equal to or greater than a predetermined threshold, performing a translation insertion process that adds information about the target language expression corresponding to the source language expression to the sentence to be translated, and if the frequency of occurrence is less than the threshold, performing a variable substitution process that replaces the source language expression in the sentence to be translated with a variable; Equipped with the translation unit inputs the translation target sentence processed by the input data processing unit as the source language sentence and performs translation; Natural language processing device.

5. When the input data processing unit performs a translation insertion process to add information about the target language expression corresponding to the source language expression to the sentence to be translated when the frequency of occurrence is equal to or greater than a predetermined threshold, and when the target language expression added to the sentence to be translated in the translation insertion process is not included in the translation result output by the translation unit as a translation corresponding to the source language sentence, the input data processing unit also performs the variable substitution process instead of the translation insertion process for the source language expression whose frequency of occurrence is equal to or greater than the threshold, the translation unit inputs the sentence to be translated as a result of the variable substitution process instead of the translation word insertion process as the source language sentence and performs translation again; The natural language processing device according to any one of claims 1 to 4.

6. A natural language processing device according to any one of claims 1 to 5. A program that makes a computer function as a

Images