Simultaneous translation device and computer program

Abstract

A simultaneous translation system 150 that includes: an encoder 200 that encodes input word strings into an intermediate language expression; a chunk-end detection device 170 that detects the end of a chunk in a word string; a word vector reading unit 172 that enters a partial word string that comprises up to the chunk end detected by the chunk-end detection device 170, into the encoder 200; a decoder 202 and a translated word search unit 204 that use as the input therefor the intermediate language expression from the encoder 200 and output a translated word string for the partial word string; and a translated word string storage unit 178 that stores the translated word strings output from the decoder 202 and the translated word search unit 204. The decoder 202 and the translated word search unit 204 use the intermediate language expression output by the encoder, as the input therefor, and the translated word string stored in the translated word string storage unit 178, as the default, and, by searching for the next translated word string, consecutively output translated word strings in a prescribed language, for partial word strings.

Description

[Technical Field]

[0001] This invention relates to a machine translation device, and more particularly to a simultaneous translation device. This application claims priority under Japanese application No. 2021-027112, filed on 24 February 2021, and incorporates all the provisions of the said Japanese application. [Background technology]

[0002] Machine translation devices are becoming increasingly widespread. As described in Non-Patent Document 2 below, machine translation devices using neural networks (so-called "neural machine translation") can now translate even very long texts with far higher accuracy than before.

[0003] However, machine translation devices have not yet become widespread in the field of so-called simultaneous translation. In simultaneous translation, a speech recognition device is used as input. The output of a speech recognition device does not include punctuation. Therefore, neural machine translation, which is based on the premise of translating sentence by sentence, cannot directly translate the output of a speech recognition device.

[0004] To solve these problems, there is a technology disclosed in Non-Patent Document 1, which is listed below. The technology disclosed in Non-Patent Document 1 detects the end of a sentence from a sequence of words that does not contain punctuation. This technology also uses a neural network. By using this technology to divide the output of a speech recognition device into sentence units, the output of the speech recognition device can be translated sentence by sentence using neural machine translation.

[0005] However, in simultaneous translation, translating sentence by sentence presents a problem: the translation lags behind the flow of the conversation. Therefore, there is a demand for simultaneous translation devices that can translate in units shorter than one sentence, almost in real time. [Prior art documents] [Patent Documents]

[0006] [License 1] Special Announcement No. 2017-199363 [Non-licensed literature]

[0007] [Non-licensed Document 1] Xiaolin Wang, Masao Utiyama and Eiichiro Sumita. (2019) Online Sentence Segmentation for Simultaneous Interpretation using Multi-Shifted Recurrent Neural Network. MT Summit. [Non-licensed Document 2] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaier, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, UV Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 5998-6008. Curran Associates, Inc. [Non-licensed Document 3] Jiatao Gu, Graham Neubig, Kyunghyun Cho, Victor OK Li (2016) Learning to Translate in Real-time with Neural Machine Translation. https: / / arxiv.org / abs / 1610.00388 [Non-licensed Document 4] Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, Hua Wu, and Haifeng Wan. STACL: Simultaneous translation with implicit anticipation and controllable latency using prefix-to-prefix framework. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3025-3036, Florence, Italy, July 2019. Association for Computational Linguistics. [Overview of the project] [Problems that the invention aims to solve]

[0008] Non-patent documents 3 and 4 offer suggestions for solving the aforementioned problems. These documents propose using a machine translation engine that modifies the mechanism of neural machine translation itself. These modified machine translation engines divide the input word sequence into units smaller than a sentence (called "chunks") and translate at the chunk level. By performing machine translation at the chunk level, it may be possible to avoid the problem of the machine translation lagging behind the flow of the conversation.

[0009] However, the proposals disclosed in non-patent literature have a problem in that the performance of the modified machine translation engine itself is insufficient. No matter how well it can follow the flow of the conversation, if the translation accuracy is not high, it cannot serve the purpose of simultaneous translation.

[0010] Therefore, the objective of this invention is to provide a simultaneous translation device that can translate an input string of words in near real-time and with high accuracy. [Means for solving the problem]

[0011] A simultaneous translation device according to the first aspect of the present invention includes an encoder for encoding an input word sequence into a predetermined intermediate language representation; chunk end detection means for detecting the end of a chunk in the word sequence in real time; word sequence input means for inputting a partial word sequence consisting of the chunk whose end is detected by the chunk end detection means to the encoder; decoding means for outputting a translation word sequence in a predetermined language for the partial word sequence, taking the intermediate language representation output by the encoder as input; and translation word sequence storage means for storing the translation word sequence output by the decoding means. The decoding means includes sequential decoding means for sequentially outputting a translation word sequence in a predetermined language for the partial word sequence by taking the intermediate language representation output by the encoder as input and searching for a subsequent translation word sequence using the translation word sequence stored in the translation word sequence storage means as a default.

[0012] Preferably, the simultaneous translation device further includes a sentence-end detection means for detecting the end of a sentence in an input word sequence; a single-sentence translation device for outputting a translation in a predetermined language for the word sequence up to the end of the sentence in response to the detection of the sentence-end by the sentence-end detection means; and a translation replacement means for replacing the output of the decoding means with the translation from the single-sentence translation device in response to the output of the translation by the single-sentence translation device.

[0013] More preferably, the simultaneous translation apparatus further includes a clearing means for clearing the translated word sequence storage means in response to the chunk end detection means detecting the end of a word sequence chunk after the translated text has been replaced by the translated text replacement means.

[0014] More preferably, the translation sentence replacement means includes an evaluation means for evaluating the magnitude of the difference between the output of the decoding means and the translation sentence from the one-sentence translation device in response to the output of the translation sentence from the one-sentence translation device, and a replacement means for replacing the output of the decoding means with the translation sentence from the one-sentence translation device in response to the evaluation by the evaluation means that the magnitude of the difference is greater than a threshold value.

[0015] Preferably, the simultaneous translation device further includes a tagging means for attaching a first tag determined by a predetermined condition to the head of the input word sequence, and a tag insertion means for inserting a second tag at the head of the translated word sequence by storing the second tag corresponding to the first tag in the translated word sequence storage means when the translated word sequence storage means is cleared.

[0016] The computer program according to the second aspect of the present invention causes a computer to function as any of the above-described devices.

[0017] The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the present invention understood in connection with the accompanying drawings.

Brief Description of Drawings

[0018] [Figure 1] FIG. 1 is a diagram schematically showing a sequence of machine translation by the simultaneous translation device according to the first embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing an outline of the functional configuration of the simultaneous translation device according to the first embodiment of the present invention. [Figure 3] FIG. 3 is a flowchart showing the control structure of a program that causes a computer to function as the simultaneous translation device according to the first embodiment. [Figure 4] FIG. 4 is a flowchart showing the control structure regarding the details of the chunk input process in the program shown in FIG. 3. [Figure 5]Figure 5 is a flowchart showing the control structure for the details of the translated chunk retranslation process in the program shown in Figure 3. [Figure 6] Figure 6 is a flowchart showing the control structure for the details of the new chunk translation process within the program shown in Figure 3. [Figure 7] Figure 7 is a schematic diagram showing the sequence of machine translation by the simultaneous translation device according to the second embodiment of this invention. [Figure 8] Figure 8 is a block diagram illustrating the schematic functional configuration of the simultaneous translation device according to the second embodiment. [Figure 9] Figure 9 is a flowchart showing the control structure of a program that enables a computer to function as a simultaneous translation device according to the second embodiment. [Figure 10] Figure 10 is a flowchart showing the control structure for the details of the translated chunk retranslation process in the program shown in Figure 9. [Figure 11] Figure 11 is an external view of a computer system for realizing the first and second embodiments. [Figure 12] Figure 12 is a block diagram showing the hardware configuration of the computer system shown in Figure 11. [Modes for carrying out the invention]

[0019] In the following descriptions and drawings, identical parts are assigned the same reference number. Therefore, detailed descriptions of them will not be repeated.

[0020] First Embodiment 1. Outline of the translation method in the first embodiment Figure 1 shows an overview of the translation sequence by the simultaneous translation device according to the first embodiment. In this embodiment as well, it is assumed that, for example, a speech recognition device exists prior to this. Therefore, the input to the simultaneous translation device is a sequence of words from the speech recognition device and does not include punctuation. In this embodiment, by applying the technology disclosed in Non-Patent Literature 1, a sentence is divided into smaller units called chunks, and each time a new chunk is detected, chunk translation 50 is performed on the partial sequence of words from the beginning of the input sequence of words up to that chunk. This chunk translation 50 has two characteristics.

[0021] The first characteristic is that chunk translation 50 can be implemented by almost directly adapting the translation method of conventional sentence-based neural machine translation. Neural machine translation, including the one described in Non-Patent Literature 2, which is currently the mainstream, is a so-called end-to-end type consisting of an encoder and a decoder. In these machine translations, the encoder converts the input word sequence into an intermediate language representation and inputs it to the decoder. Based on this input, the decoder calculates a vector representing the probability that each translation word will be the first word in the translated sentence across the entire vocabulary of the target language. Here, we will simply call this vector the "probability distribution".

[0022] The decoder then selects several words with the highest probabilities as translation candidates based on these probability distributions, and re-inputs each of them into the decoder to obtain the next probability distribution. As a result, several more subsequent translation candidates are obtained for the translation candidate being processed, and the probability of each can be calculated. By performing this process for each selected translation candidate, a tree of translation candidate sequences is formed. By performing a probability-based beam search, the translation candidate sequence with the highest probability as the translation sequence for the sequence of words that make up the input sentence is selected and determined as the translation of the input sentence.

[0023] In this embodiment, the conventional translation format is largely followed. However, this embodiment differs from the conventional translation format in the following second characteristic.

[0024] The chunk translation 50 performs translation chunk by chunk. In this case, the word sequence to be translated is from the beginning of the input string to the last detected chunk. For example, referring to Figure 1(A), suppose that chunk 60, which contains words w1, w2, and w3, is the beginning of the input word sequence, and therefore the end of that chunk (chunk end) is detected first. Chunk 60 is translated by the chunk translation 50, and a translated word sequence 62 containing words t1 and t2 in the target language is obtained.

[0025] Next, referring to Figure 1(B), assume that chunk 64 containing words w4 and w5 is input to chunk translation 50 and the end of the chunk is detected. In this case, not only chunk 64 but also chunk 60, which has already been translated, is included in chunk sequence 66, which is the target of translation by chunk translation 50. Therefore, chunk 60 will be translated a second time as well. However, in this embodiment, the translation result of chunk 60 for the second time is determined by searching for the above-mentioned candidate translation sequence, with the translation word sequence 62, which is the result of the first translation, as the default. In other words, the translation word sequence 72, which is the translation result of chunk sequence 66, includes the translation word sequence 62 as is, and then the translation word sequence 70 (words t3, t4, t5, and t6), which is the translation result of chunk 64, is appended thereafter. This is the second feature of this embodiment.

[0026] This translation process is repeated when the end of the third chunk, 74, is detected. That is, referring to Figure 1(C), the next chunk sequence to be translated, 76, contains chunk sequence 66 and the subsequent chunk 74. The resulting translation sequence 80, output by chunk translation 50 after translating chunk sequence 76, is the same as the previous output, translation sequence 72, with translation sequence 78, the translation result of chunk 74, appended to it.

[0027] Referring to Figure 1(D), suppose this translation is repeated, and for example, the end of chunk 84, which is one chunk before the last chunk of the sentence, is detected. At this time, the chunk sequence 86 to be translated will be the chunk sequence 82 that was translated immediately before it with chunk 84 added to it. The resulting translated word sequence 92 will be the translated word sequence 88, which is the translation result of the preceding chunk sequence 82, with the translated word sequence 90, which is the translation result of chunk 84, added to it.

[0028] Referring to Figure 1(E), let's assume that chunk 94 at the end of the sentence has been detected. The chunk sequence 96 to be translated is obtained by adding chunk 94 to chunk sequence 86. The resulting translated word sequence 100 is obtained by adding the translated word sequence 98, which is the translation result of chunk 94, to the previously translated word sequence 92. This translated word sequence 100 is output as the translated sentence of the input word sequence.

[0029] Thus, in this embodiment, when the end of a chunk is detected, the portion of the input word sequence that is translated is the part from the beginning of the sequence up to that chunk. The resulting translation is obtained by adding the translated word sequence, which is the translation result of the chunk in which the end of the chunk was detected, to the translation result up to the immediately preceding chunk.

[0030] It is important to note here that just because the translation results of repeatedly translated chunk sequences are fixed as defaults does not mean that new chunks are treated as isolated targets for translation. Repeatedly translated chunks are also targets for translation, so the decoder repeatedly calculates the probability distribution for those candidate word sequences. The chunk sequences to be translated grow longer with each translation iteration. Therefore, the intermediate language representation of the chunk sequence output by the encoder changes each time. As the probability distribution changes, in a normal translation, the probability of each candidate word sequence also changes, and the selected word sequence may also change. However, in this embodiment, regardless of the calculated probability distribution, already translated words are treated as defaults in subsequent translations. This means that in the translation of a new chunk, the word sequences that have been translated up to that point are treated as its context.

[0031] 2. Configuration of the First Embodiment (1) Functional description Figure 2 shows a schematic functional configuration of the simultaneous translation system 150 according to the first embodiment in block diagram format. Referring to Figure 2, the simultaneous translation system 150 includes a simultaneous translation device 160, which plays a central role in performing simultaneous translation and translates the previously input chunk sequence in near real-time whenever a new chunk is received, and a single-sentence translation device 162, which machine-translates the previously input word sequence as a single sentence when the word sequence reaches the end of a sentence. In this embodiment, a conventional machine translation device is used as the single-sentence translation device 162. As for the simultaneous translation device 160, as will be described later, it basically uses an encoder-decoder type machine translation device similar to that described in Non-Patent Document 2. However, in the case of the simultaneous translation device 160, there is a slight difference in the method of searching for translation sequences compared to conventional machine translation devices. However, this difference is not significant, and as mentioned above, the only difference is that previously translated word sequences are treated as a predetermined context in subsequent translations, but the translation search method itself is the same.

[0032] The simultaneous translation system 150 further includes an input buffer 164 for receiving and storing word sequences from a speech recognition device (not shown), a word vector generation unit 166 for sequentially converting each word in the word sequence stored in the input buffer 164 into a word embedding vector, and a word vector storage unit 168 for sequentially storing the word embedding vectors generated by the word vector generation unit 166 from the beginning. The simultaneous translation system 150 further includes a chunk end detection device 170 for detecting a new chunk end in the word sequence stored in the input buffer 164 and outputting a chunk end detection signal, and a sentence end detection device 174 for detecting the end of a sentence in the word sequence stored in the input buffer 164 and outputting a sentence end detection signal.

[0033] In this embodiment, both the chunk end detection device 170 and the sentence end detection device 174 are implemented by neural networks trained using the same techniques as described in Non-Patent Document 1. More specifically, in the case of the chunk end detection device 170, a neural network trained with word sequences in which chunk end words are labeled to indicate that they are chunk end words is used as training data. In the case of the sentence end detection device 174, a neural network trained with word sequences in which sentence end words are labeled to indicate that they are sentence end words is used as training data.

[0034] The simultaneous translation system 150 further includes a word vector reading unit 172 that, in response to receiving a chunk end detection signal from the chunk end detection device 170, reads a sequence of word vectors stored in the word vector storage unit 168 and inputs it to the simultaneous translation device 160 to start translation, and a word vector reading unit 176 that, in response to receiving a sentence end detection signal from the sentence end detection device 174, reads a sequence of word vectors stored in the word vector storage unit 168 and inputs it to the single-sentence translation device 162 to start translation.

[0035] The simultaneous translation system 150 further includes a translation sequence storage unit 178 for storing translation sequences of the target language output by the simultaneous translation device 160, a display device 180 for displaying the translation sequences stored in the translation sequence storage unit 178, and a single-sentence translation translation storage unit 182 for storing translation sequences of a single sentence output by the single-sentence translation device 162. The simultaneous translation system 150 further includes a translation comparison unit 184 that, in response to a new translation sequence being stored in the one-sentence translation translation storage unit 182, compares the translation sequence stored in the translation sequence storage unit 178 with the translation sequence stored in the one-sentence translation translation storage unit 182, and replaces the translation sequence stored in the translation sequence storage unit 178 with the translation sequence stored in the one-sentence translation translation storage unit 182 if the magnitude of the difference exceeds a certain level; and a buffer clearing unit 186 that, after receiving a sentence end detection signal from the sentence end detection device 174, and in response to receiving the first chunk end detection signal from the chunk end detection device 170, clears the translation sequence storage unit 178. The difference in the translation sequences here can be determined by prior art, for example, the difference in character N-grams between the word sequence stored in the translation sequence storage unit 178 and the one-sentence translation result stored in the one-sentence translation translation storage unit 182.

[0036] The simultaneous translation device 160 includes an encoder 200 consisting of a pre-trained neural network that takes a sequence of word vectors read from a word vector reading unit 172 as input and outputs a vector that is an intermediate language representation corresponding to the content represented by the input, and a decoder 202 consisting of a pre-trained neural network that takes a word vector of the target language as input and outputs a probability distribution of the word that appears after the word represented by the word vector. The simultaneous translation device 160 further includes a word search unit 204 that searches for the sequence of words in the target language that gives the highest probability by repeatedly feeding the vector, which is an intermediate representation from the encoder 200, to the decoder 202 as the first input, predicting the next word from the probability distribution output by the decoder 202, and inputting it back into the decoder 202. The decoder 202 and the word search unit 204 take the intermediate language representation output by the encoder 200 as input and, using already translated word sequences as a default, search for subsequent translated word sequences to sequentially output a sequence of translated words for an input sequence of words.

[0037] The word probability distribution referred to here is a vector whose elements represent the probability that each word will appear next in the sequence of words previously input to the decoder 202, for each of the predetermined number of words selected as the vocabulary of the target language (hereinafter simply referred to as the "vocabulary of the target language"). Therefore, the output of the decoder 202 consists of the same number of words in the vocabulary selected as the target of translation in the target language, and is obtained as a result of softmax calculation performed in the output layer of the word search unit 204. In addition, the next word candidate predicted by the word search unit 204 is given as input to the decoder 202. Therefore, the decoder 202 has the same number of inputs as the number of elements in the word vector. For this reason, the intermediate representation vector output by the encoder 200 is in the same format as this word vector. However, the input to the encoder 200 must be at least the same number as the maximum number of word vectors stored in the word vector storage unit 168. In this embodiment, the encoder 200 is configured to accept up to 300 words as input. Since a predetermined sentence-ending token is appended to the end of each input word sequence, the effective maximum number of words that can be entered is 299. If the length of the input word sequence is less than 300 words, a predetermined padding string is inserted after the word sequence.

[0038] (2) Program control structure (A) Overall structure Figure 3 is a flowchart showing the overall control structure of a program that causes a computer to function as the simultaneous translation system 150 shown in Figure 2. Referring to Figure 3, this program includes a step 250 that performs initial processing for running the program. In step 250, the variables N0 representing the number of words output and N0 representing the number of words translated are... T0 is assigned to each of them. In step 250, the value "0", which indicates that it is not the end of a sentence, is assigned to both the temporary flag, which temporarily indicates whether the word sequence accumulated in the input buffer 164 shown in Figure 2 has reached the end of a sentence, and the end-of-sentence flag, which indicates that it has been confirmed that it has reached the end of a sentence. When the end of a sentence is detected, 9 is temporarily assigned to the temporary flag, and then 9 is assigned to the end-of-sentence flag when it is confirmed that it is the end of a sentence.

[0039] This program further includes step 254 (corresponding to the buffer clearing unit 186 in Figures 2 and 8) which clears the translation sequence storage unit 178 shown in Figure 2, and step 256 which repeatedly executes the following step 258 until the value of the sentence end flag becomes 9, that is, until the end of a sentence is detected and confirmed. When the processing in step 256 is completed, that is, when the processing up to the end of a sentence is completed, control returns to step 250. Note that this program itself does not have a function to terminate execution. The operating system terminates the execution of this program when it receives some signal from the user or a specific process that indicates that the program should be terminated.

[0040] If the end of a sentence is detected during the execution of step 256, the single-sentence translation process described in steps 260 to 268 will be executed. The details of this process will be described later.

[0041] (B) Chunk input processing Referring to Figure 4, the chunk input processing shown in step 252 of Figure 3 (the same applies to step 308) includes step 350, which reads the word sequence stored in the input buffer 164 in Figure 2, and step 352 (corresponding to the word vector generation unit 166 in Figures 2 and 8), which converts the words in the read word sequence into word embedding vectors to create word vectors and adds them to the end of the word vector sequence stored in the word vector reading unit 172. The chunk input processing further includes step 356, which determines whether or not a chunk end detection signal indicating the detection of a chunk end has been received from the chunk end detection device 170 shown in Figure 2, and branches the control flow according to the determination result. Note that the end of a sentence is also a type of chunk end. Therefore, the chunk end detection device 170 outputs a chunk end detection signal not only for chunks other than the end of a sentence, but also for the end of a sentence.

[0042] This program further includes step 358, which, in response to the determination in step 356 being positive, further determines whether the end of a sentence has been detected by the end-of-sentence detection signal from the end-of-sentence detection device 174 shown in Figure 2, and branches the control flow according to the determination result; and step 360, which, in response to the determination in step 358 being negative, assigns 0 to the temporary flag. This program further includes step 362, which, in response to the determination in step 358 being positive, assigns 9 to the temporary flag; and step 364, which, after step 360 and after step 362, adds a predetermined end-of-sentence token to the end of the word sequence stored in the word vector storage unit 168 shown in Figure 2, and terminates the chunk input processing (corresponding to the word vector reading unit 172 in Figures 2 and 8). When the determination in step 356 is negative, control returns to step 350.

[0043] Although the temporary flag is assigned the value 9 in step 362, no processing has yet been performed on the word sequence stored in the word vector memory unit 168. Therefore, the value of this temporary flag can be said to indicate a temporary sentence end. The sentence end token is the same as in conventional neural machine translation and indicates the end of the input. During translation, the translation of the input is completed when a predetermined token corresponding to this sentence end token is output from the decoder.

[0044] (C) Step 258 Returning to Figure 3, step 258 includes step 290, which determines whether the value of the temporary flag is 9 and branches the control flow accordingly, and step 292, which, if the determination in step 290 is positive, assigns 9 to the sentence-end flag. Step 258 further includes step 294, which starts the aforementioned sentence translation process as a separate process from the process of this program, and step 296, which, if the determination in step 290 is negative, and if the determination in step 290 is positive and the execution of steps 292 and 294 is completed, inputs the word vector sequence stored in the word vector reading unit 172 shown in Figure 2 to the encoder 200. Step 258 further includes step 298, which reads the vector, which is an intermediate language representation corresponding to the word vector sequence, output by the encoder 200 in response to this input, and continues to input it to the decoder 202.

[0045] The program further includes step 300, in which the decoder 202 and the translation search unit 204 work together to retranslate the translated word sequence stored in the translation sequence storage unit 178 shown in Figure 2 as context for the word sequence of subsequent chunks; and step 302, in response to the completion of the retranslation of the translated word sequence in step 300, in which the decoder 202 and the translation search unit 204 shown in Figure 2 work together to translate the word sequence contained in the untranslated chunk and add the translation to the end of the translated word sequence stored in the translation sequence storage unit 178.

[0046] This program further includes step 304, which, in response to the completion of translation in step 302, displays a sequence of target words stored in the sequence of target words stored in the sequence of target words storage unit 178 on the display surface of the display device 180 shown in Figure 2, and step 306, which branches the control flow depending on whether the value of the sentence-end flag is 0 or not. This program further includes step 308, which, in response to the determination in step 306 being affirmative, performs the same chunk input processing as in step 252 and terminates the execution of step 258. If the determination in step 306 is negative, the execution of step 258 terminates immediately.

[0047] (D) Contextual retranslation process The details of step 300 in Figure 3 will be explained with reference to Figure 5. Referring to Figure 5, step 300 is a step in which 0 is assigned to the variable N0, which represents the number of words output in the current translation, and the value of variable N0 is assigned to the variable N, which represents the number of words that have been translated and finalized so far. T The process includes step 402, which repeats step 404 until it equals the following:

[0048] Step 404 includes step 420, which reads the probability distribution output by the decoder 202 shown in Figure 2; step 422, which selects the N0th translation stored in the translation sequence storage unit 178 as the next word from this probability distribution; step 424, which inputs the word selected in step 422 into the decoder 202; and step 426, which adds 1 to the value of variable N0 and terminates the execution of step 404.

[0049] In other words, while normally a predetermined number of translation candidates with the highest probability are selected from the output of the decoder 202, in the contextual retranslation process, the translations stored in the translation list storage unit 178 are used, and no search for translation candidates is performed.

[0050] (E) New chunk translation process The details of step 302 in Figure 3 will be explained with reference to Figure 6. Referring to Figure 6, step 302 includes step 450, which reads the probability distribution output by decoder 202, and step 452, which searches for a translation word sequence for each input chunk, starting from a word sequence obtained by concatenating the translated word sequence stored in translation word sequence storage unit 178 with a predetermined number of translation word candidates with the highest probabilities (in this case, 5). Step 302 further includes step 454, which adds the new word sequence obtained as a result of the search in step 452 (corresponding to a new chunk) following the translated word to the end of the translated word, and the variable N T This includes step 456, which adds the number of newly added translated words in step 454 and terminates the process.

[0051] In step 452, numerous candidate translation sequences are obtained during the search. Therefore, it is desirable to shorten the processing time by, for example, pruning based on the probability of each word sequence.

[0052] (F) A program for translating one sentence On the other hand, the program executed in the process for single-sentence translation, which is started in step 294 of Figure 3, has the following control structure. Referring to Figure 3, this program includes a step 260 that executes a single-sentence translation process, and a step 262 that compares the translation obtained in step 260 with the translation obtained for the word sequence up to the end of the sentence by the process in step 258 and calculates the difference. This program further includes a step 264 that branches the control flow according to whether the magnitude of the difference calculated in step 262 is greater than a predetermined threshold, and a step 266 that, in response to the determination in step 264 being positive, replaces the word sequence of the simultaneous translation result stored in the word sequence storage unit 178 shown in Figure 2 with the translated sentence obtained by the process in step 260. This program further includes a step 268 that, in response to the word sequence stored in the word sequence storage unit 178 being replaced, updates the display on the display device 180 and terminates the single-sentence translation process. If the determination in step 264 is negative, the display on the display device 180 is not updated and the single-sentence translation process is terminated.

[0053] 3. Operation of the First Embodiment The simultaneous translation system 150 with the above configuration operates as follows:

[0054] (1) Start of translation Referring to Figures 1(A) and 2, when simultaneous translation begins, the words w1, w2, and w3 that constitute chunk 60 are stored in the input buffer 164 in that order. Note that the input buffer 164 is empty at the start of the simultaneous translation process. The chunk end detection device 170 detects the end of the chunk along with the input of word w3 and sends a chunk end detection signal to the word vector reading unit 172. In this case, word w3 is not the end of a sentence. Therefore, the sentence end detection device 174 does not output a sentence end detection signal.

[0055] The processing up to this point is achieved by steps 250 and 252 in Figure 3 (details are shown in the chunk input processing in Figure 4). Referring to Figure 3, the computer performs initialization processing in step 250. Further referring to Figure 4, the computer reads word w1 in response to the storage of a new word (word w1) in the input buffer 164 (step 350). The computer vectorizes this word w1 and stores it in the word vector storage unit 168 (step 352). Next, in step 356, it is determined whether or not the end of a chunk has been detected. For the first word w1, this determination is negative. Therefore, control returns to step 350.

[0056] As a result, the word vector memory unit 168 stores the word vector corresponding to the first word w1. The translation sequence memory unit 178 remains empty.

[0057] The same processing is performed on word w2 as on word w1. As a result, the word vector storage unit 168 stores word vector sequences corresponding to word sequences w1 and w2.

[0058] For word w3, the same processing as for words w1 and w2 is performed up to step 352 in Figure 4. As a result, at the end of step 352, the word vector storage unit 168 stores word vector sequences corresponding to word sequences w1, w2, and w3.

[0059] The word w3 is detected as the end of a chunk. Therefore, control proceeds from step 356 to step 358. Since word w3 is not the end of a sentence, control proceeds from step 358 to step 360. In step 360, the temporary flag is set to 0. This temporary flag indicates that it is not the end of a sentence.

[0060] Furthermore, in step 364, a sentence-ending token is added to the end of the word vector sequence stored in the word vector memory unit 168 in Figure 2, and the step for the first chunk is completed.

[0061] (2) Translation of the first chunk Returning to Figure 3, during the translation of the first chunk, the sentence-end flag is 0 at the start of step 256. Therefore, step 258 is executed. Step 258 is executed for each chunk.

[0062] In this chunk, the temporary flag is 0. Therefore, the determination in step 290 is negative. Control proceeds to step 296. In step 296, the word vector reading unit 172 shown in Figure 2 reads the word vector sequence stored in the word vector storage unit 168, appends a sentence-ending token to the end, and inputs it to the encoder 200. The encoder 200 converts this word vector sequence into an intermediate language representation vector and outputs it.

[0063] Returning to Figure 3, step 298 inputs the vector output by encoder 200 into decoder 202 and executes step 300.

[0064] Referring to Figure 5, in the first step 400 of step 300, 0 is assigned to the variable N0. Subsequently, in step 402, the value of variable N0 is assigned to variable N TStep 404 is executed until it becomes equal to [the given value].

[0065] Here, the variable N0 represents the number of translated words output by the processing in step 404. T This represents the total number of translated words from previous chunks. Therefore, in processing the first chunk, the variable N T It is 0. In other words, here the variables N0 and N T Both values ​​are 0, meaning they are equal. Therefore, step 404 is never executed, and step 300 terminates.

[0066] Returning to Figure 3, step 302 is executed following step 300. Referring to Figure 6, step 450 of step 302 reads the vector output by decoder 202 shown in Figure 2. This vector is a probability distribution of translation candidates. Specifically, it has as many elements as there are words in the vocabulary of the target language, and each element represents the probability that it is the translation for the source word input to decoder 202. At this time, the internal state of decoder 202 changes according to previous translations, so this probability is not simply determined by the combination of source word and translation.

[0067] In step 452, for each of the candidate words following the end of the word sequence already translated and stored in the word sequence memory unit 178 shown in Figure 2, the decoder 202 is input to find the top 5 candidates for the next word sequence. These candidate words are the top 5 from the vector read in step 450. The same processing is continued for the obtained words. This process is continued until the decoder 202 outputs a sentence-end token, the probability of the translated word sequence falls below a threshold, or the probability of the translated word sequence is no longer within the top 5, thus searching for the translated word sequence. Once the search in step 452 is complete, control proceeds to step 454.

[0068] In step 454, the word sequence with the highest probability among the searched translation word sequences is selected as the translation sequence for the input chunk. That is, such a word sequence is added to the end of the translation sequence stored in the translation sequence storage unit 178 shown in Figure 2. When processing the first chunk, the translation sequence storage unit 178 is empty, so the selected translation sequence is stored at the beginning of the translation sequence storage unit 178.

[0069] In the following step 456, the variable N T Add 1 to the value and return the control to step 304 shown in Figure 3.

[0070] In step 304, the translation sequence stored in the translation sequence storage unit 178 shown in Figure 2 is displayed on the display device 180.

[0071] Next, in step 306, it is determined whether the value of the end-of-sentence flag is 0. Here, the value of the end-of-sentence flag is 0, so step 308 is executed. That is, the steps for the second chunk (chunk 64 shown in Figure 1(B)) are executed. Step 308 is the same process as step 252 shown in Figure 3.

[0072] Referring to Figure 4, steps 350 to 356 are performed, and a word vector consisting of the word vector sequence for chunk 60 and the word vector sequence for chunk 64 shown in Figure 1(B) is stored in the word vector storage unit 168 shown in Figure 2. The subsequent determination in step 358 is negative, and in step 360, the temporary flag is set to 0. Step 364 adds a sentence-ending token to the end of the word vector sequence to be processed.

[0073] In step 296 of Figure 3, this word vector sequence is input to the encoder 200, which outputs its intermediate language representation. In step 298, this intermediate representation is input to the decoder 202.

[0074] (3) Translation of the second chunk When translating the second chunk, steps 300 and 302 are executed. Referring to FIG. 5, in step 300, the variable N in step 402 T is 3 in the example shown in FIG. 1. Therefore, step 404 is executed three times.

[0075] In the first execution, step 420 reads the probability distribution that is the output of the decoder 202. Step 422 selects the 0th translated word in the translated word sequence stored in the translated word sequence storage unit 178 from this probability distribution. Since the variable N0 = 0, the first translated word is selected. In this translated word selection, no search for the translation sentence is performed.

[0076] In the subsequent step 424, the selected word (the first translated word that has been translated) is input to the decoder 202. In step 426, 1 is added to the value of the variable N0, and the first process of step 404 ends.

[0077] In step 404 when the variable N0 is 1, the same process as the above-described process is executed. Without performing the search for the translation sentence, the second word stored in the translated word sequence storage unit 178 is selected, 1 is added to the value of the variable N0, and the second process of step 404 ends.

[0078] The same process is repeated for the third word stored in the translated word sequence storage unit 178. As a result, the execution of step 300 ends without changing the stored content of the translated word sequence storage unit 178.

[0079] For the second chunk, in step 302 shown in FIG. 3, the following process is performed. Referring to FIG. 6, step 450 reads the vector output by the decoder 202. Step 452 selects the top 5 translated word candidates from the probability distribution represented by this vector, and searches for translation word sequence candidates starting from each of them. In step 454, the word sequence with the highest probability is selected from these translation word sequence candidates and added to the end of the translated word sequence storage unit 178. In step 456, the variable N<0000​​​ In step 452, during the search for the translated word sequence, the decoder 202 repeatedly outputs a probability distribution. This probability distribution is based on the internal state of the decoder 202 when the probability distribution was calculated in step 300, using the translated word sequence as a default. Therefore, it does not necessarily yield the same result as if the search for the translated word sequence for the second chunk were performed independently without any processing in step 300. In other words, in this search for the translated word sequence for the second chunk, the translated word sequence for the first chunk stored in the translated word sequence storage unit 178 is treated as the context.

[0081] (4) Translation of the third and subsequent chunks The translation of the third and subsequent chunks is the same as the translation of the second chunk. However, it should be noted that when translating the third chunk, the translation sequences determined for the first and second chunks are treated as context, and when translating the fourth chunk, the translation sequences determined for the first to third chunks are treated as context, and so on. In other words, in these processes, the translation sequences stored in the translation sequence storage unit 178 remain unchanged, and the translation sequences for the new chunks are simply added to the end of them. However, the newly added translation sequences are not simple translations for the new chunks, but rather translation sequences searched using the translation sequences stored in the translation sequence storage unit 178 as context.

[0082] (5) Translation of the last chunk In this way, the process of step 258 is eventually performed on the second-to-last chunk. In this case, the processing from step 290 to step 306 in step 258 is the same as the processing for the third-to-last chunks. However, the processing in step 308 is different from what came before.

[0083] Referring to Figure 4, when reading the chunk following the second-to-last chunk, i.e., the last chunk, steps 350 to 356 are performed on the last chunk to word vectorize it. Subsequently, the determination in step 358 of the step is affirmative. In step 362, the temporary flag is set to 9.

[0084] As a result, in step 258, which then processes the last chunk, the determination in step 290 becomes positive, and in step 292, the sentence-end flag is set to 9. In step 294, the sentence translation process is started. The sentence translation process is performed by the sentence translation device 162 shown in Figure 2. The operation of the sentence translation device 162 will be described later.

[0085] Next, the processes from steps 296 to 304 in Figure 3 are performed in the same way as for the previous chunks. However, for the last chunk, the judgment in step 306 is negative, so step 308 is not performed and control returns to step 250. As a result, the initial processing is performed in step 250, the system waits for the next chunk in step 252, and when input for the next chunk begins, the same process described above is performed for the new chunk sequence.

[0086] (6) One sentence translation process As mentioned above, the single-sentence translation process is implemented by step 260, which is initiated in step 294. Therefore, step 260 is executed in parallel with the translation process of the last chunk in step 258. In this embodiment, step 260 basically performs translation using the same operating principle as the simultaneous translation device 160 shown in Figure 2. However, while the simultaneous translation device 160 divides a sentence into one or more chunks and translates the chunks by treating the translated words as context, the single-sentence translation device 162 differs in that it treats the entire sentence as a single chunk and performs translation, and the resulting single-sentence translation is stored in the single-sentence translation translation storage unit 182. Therefore, the details of the contents of step 260 will not be repeated here.

[0087] At step 262, when the translation of one sentence is completed in step 260, the difference between the simultaneous translation result obtained for the entire chunk sequence in step 258 and the one-sentence translation result obtained in step 260 is calculated. In Figure 2, this process is performed by the translation comparison unit 184. In this embodiment, the magnitude of this difference is determined by prior art such as the difference in character N-grams between the two translations.

[0088] In the subsequent step 264, the control flow is branched depending on whether the difference value calculated in step 262 is greater than a threshold. Normally, sentence-level translation is considered to have higher accuracy. Therefore, if the determination in step 264 is affirmative, in step 266 the contents of the translation sequence storage unit 178 shown in Figure 2 are replaced with the contents of the single-sentence translation translation storage unit 182, and in step 268 the contents are displayed on the display device 180. Normally, the simultaneous translation by the simultaneous translation device 160 is considered to be completed earlier than the single-sentence translation process. Therefore, after the translation result obtained by simultaneous translation is displayed on the display device 180, if the single-sentence translation is completed and the difference between the two is greater than a threshold, the display content of the display device 180 is replaced with the result of the single-sentence translation.

[0089] The display content of the display device 180 is maintained until step 258 is executed for the first chunk of the next sentence and the processing of step 304 is performed. In other words, whether the translation result for the input chunk sequence is simultaneous translation or single-sentence translation, the display is maintained until the translation processing of step 258 for the first chunk of the next chunk sequence is substantially completed.

[0090] 4. Effects of the First Embodiment (1) According to this first embodiment, the input word sequence is divided into chunks, and translation is performed on a chunk-by-chunk basis. Moreover, translated chunks are treated as context, and no actual translation is performed on them. Therefore, translation can be performed on a chunk-by-chunk basis in a short time, and translation of continuously input word sequences can be performed in real time. As a result, a simultaneous translation device can be provided that can translate input word sequences in near real time and with high accuracy.

[0091] (2) After the translation of preceding chunk sequences is completed, the corresponding translated word sequences are treated as context and will not be changed by translations of subsequent chunk sequences, except when they are replaced by the result of a single-sentence translation. As a result, the display of translated word sequences is stable, and when translated word sequences are displayed as subtitles, for example, the order or content of the translated word sequences may change frequently, which could confuse the user.

[0092] (3) Generally, single-sentence translation is considered to be more accurate, while simultaneous translation, which prioritizes simultaneity, is considered to be less accurate than single-sentence translation. In the above embodiment, if the difference between the result of single-sentence translation and the interpretation result by simultaneous translation exceeds a threshold, the result of simultaneous translation is replaced with the result of single-sentence translation. Therefore, even if the result of simultaneous translation is difficult to understand, the possibility of understanding it easily by using the result of single-sentence translation is greatly increased. Moreover, these displays are maintained until the simultaneous translation process for the first chunk of the next chunk sequence is substantially completed. Therefore, the possibility of reliably understanding the content of the utterance is greatly increased.

[0093] (4) Furthermore, once the simultaneous translation process for the first chunk of the next chunk sequence is effectively complete, the translation result for the preceding sentence is replaced with the simultaneous translation result for that chunk. For the user, this allows them to see the simultaneous translation result for an already started utterance early on, reducing the risk of confusion caused by the translation result for the previous utterance being displayed for an extended period even though subsequent utterances have begun.

[0094] (5) The simultaneous translation device 160 can be any conventional device (for example, the one described in Non-Patent Document 2) as long as it is in an encoder-decoder format. There is also no restriction on the single-sentence translation device 162 to being in an encoder-decoder format. Therefore, any machine translation device that can be used within this range can be used in this embodiment. There is no need to develop a new type. For this reason, for example, a device that has already been evaluated as being able to obtain high performance can be used, and a high-performance and stable simultaneous translation device can be realized without a great burden.

[0095] 5. Variations In the first embodiment described above, step 264 compares the interpretation result from simultaneous translation with the translation result of a single sentence and calculates the difference between them. However, if it is generally considered that translation on a sentence-by-sentence basis is more accurate, this comparison may be omitted, and the translation result from simultaneous interpretation may be replaced with the translation result of the single sentence once the translation of a single sentence is completed and displayed on the display device 180. In many cases, a more reliable translation result can be obtained by such a method as well.

[0096] Second second embodiment 1. Outline of the translation method in the second embodiment In the first embodiment described above, the input word sequence is divided into chunks and each chunk is translated. No special processing is performed on the input word sequence. However, as disclosed in Patent Document 1, the information that can be obtained from the input word sequence alone is limited. For example, it is not easy to determine from the input word sequence alone what field the sentence relates to, what kind of person the word sequence represents, what kind of person the speaker is, or what kind of situation the word sequence relates to. In translation, by appropriately differentiating the translation based on the context and field in which the utterance was made, the selection of translation terms can be made appropriately, and the accuracy of the translation can be further improved.

[0097] To solve these problems, Patent Document 1 discloses a method of adding a specific tag to the beginning of an input word sequence in order to input information beyond the scope of the original text into a machine translation device before translation. By applying such a technology directly to the first embodiment described above, it may be possible to further improve the accuracy of simultaneous translation.

[0098] However, if the technology disclosed in Patent Document 1 is applied directly to the first embodiment described above, the decoder output for the first tag may not correspond correctly to that tag, or its position may change. Since translated words are treated as context in subsequent chunk processing, it is desirable that the first tag remains the correct tag in the translated word sequence, and that its position is also at the beginning of the word sequence.

[0099] In this second embodiment, the tags assigned to the beginning of the input word sequence are also assigned to the beginning of the translated word sequence after simultaneous translation, as the correct tags corresponding to the assigned tags.

[0100] That is, referring to Figure 7, let's assume, for example, that the first chunk of the input word sequence is chunk 60. In the second embodiment, prior to translation, a tag 500 is added to the beginning of the first chunk. That is, the first input to the encoder is a chunk sequence 502 consisting of tag 500 and chunk 60. The translated word sequence 506 obtained by translating this chunk sequence 502 consists of tag 504 and translated word sequence 62. Tag 504 is the tag of the target language corresponding to tag 500, and translated word sequence 62 is the translated word sequence of chunk 60.

[0101] From now on, even as translation progresses, tag 504 will always remain at the beginning of the translation list, and its position will not change, nor will it be replaced by another tag or word. Furthermore, tag 504 is predetermined to correspond to tag 500 and will not become a different tag.

[0102] More specifically, referring to Figure 7(B), the next target for translation is chunk sequence 510, which consists of chunk sequence 502 and the second chunk sequence 508. The translation sequence 516 corresponding to chunk sequence 510 includes translation sequences 506 and 514. Referring to Figure 7(C), the third target for translation is chunk sequence 520, which consists of chunk sequence 510 and the third chunk 518. The translation sequence 524 corresponding to chunk sequence 520 includes translation sequences 516 and 522.

[0103] Referring to Figure 7(D), let's assume that the second to last item to be translated is chunk sequence 534, which includes chunk sequences 530 and 532. Chunk sequence 534 is the chunk that was the third to last item to be translated. The translation result corresponding to chunk sequence 534, name sequence 540, includes translation sequences 536 and 538. Name sequence 536 is the word sequence after translation of chunk sequence 530, which was the third to last item to be translated. Then, referring to Figure 7(E), the last item to be translated, chunk sequence 544, includes chunk sequence 534, which was the second to last item to be translated, and the last chunk 542. The translation result of this chunk sequence 544, name sequence 550, includes the translation result of the second to last item, name sequence 540, and the translation result of chunk 542, name sequence 548.

[0104] Figure 7 shows that tag 500, which was at the beginning of the input word sequence, is always present at the beginning of the translated word sequence as tag 504.

[0105] 2. Configuration of the second embodiment Figure 8 shows a schematic block diagram illustrating the functional configuration of the simultaneous translation system 600 according to the second embodiment. The difference between this simultaneous translation system 600 and the simultaneous translation system 150 according to the first embodiment shown in Figure 2 is: • In place of the simultaneous translation device 160 in Figure 2, the system includes a simultaneous translation device 630 that performs simultaneous translation while adding a tag corresponding to the first tag to the beginning of the translation sequence, as described above; • Instead of the sentence translation device 162 in Figure 2, it includes a sentence translation device 632 that performs sentence translation on an input word sequence in which tags are attached to the beginning of each word; The system includes a tag check unit 620 that checks the first chunk of word sequence read from the word vector storage unit 168 by the word vector reading unit 172 and word vector reading unit 176, does nothing if a tag is assigned to the beginning of the chunk, and otherwise selects a predetermined tag according to information previously instructed by the user, or information instructed by, for example, a speech recognition device that sends the word sequence to the input buffer 164, and outputs that tag; - A tag addition unit 622 is provided between the word vector reading unit 172 and the simultaneous translation device 630, and includes a tag addition unit 622 for adding tags output from the tag checking unit 620 to the beginning of the chunk sequence read by the word vector reading unit 172 from the word vector storage unit 168 and inputting it to the encoder 200; - A tag addition unit 624 is provided between the word vector reading unit 176 and the sentence translation device 632, and is used to add tags output from the tag checking unit 620 to the beginning of the chunk sequence read by the word vector reading unit 176 from the word vector storage unit 168 and input it to the sentence translation device 632; and The system is connected to the simultaneous translation device 630 and the single-sentence translation device 632, and includes a tag correspondence table 628 that stores the correspondence between source tags and target tags in a table format.

[0106] Furthermore, if the tag checking unit 620 does not output a tag, the tag adding unit 622 and the tag adding unit 624 output the output of the word vector reading unit 172 and the word vector reading unit 176 to the simultaneous translation device 630 and the single sentence translation device 632 without doing anything to them. This corresponds to the case where a tag has already been added to the beginning of the input word sequence.

[0107] The simultaneous translation device 630, instead of the translation search unit 204 of the simultaneous translation device 160 shown in Figure 2, includes a translation search unit 640 that, prior to performing a translation search on the first chunk in the same manner as the translation search unit 204, refers to the tag correspondence table 628, retrieves the tag of the target language corresponding to the tag of the source language assigned to the first chunk, and stores it at the beginning of the translation sequence storage unit 178, thereby inserting the tag of the target language at the beginning of the translation sequence. In other respects, the simultaneous translation device 630 is the same as the simultaneous translation device 160 shown in Figure 2.

[0108] The single-sentence translation device 632 is similar to the simultaneous translation device 630, but unlike the single-sentence translation device 162 in Figure 2, it has the function of inserting a tag at the beginning of the translated word sequence by retrieving the tag of the target language corresponding to the tag of the source language attached to the beginning of the input word sequence from the tag correspondence table 628 and storing it at the beginning of the single-sentence translation translated text storage unit 182 prior to translation. In other respects, the single-sentence translation device 632 is the same as the single-sentence translation device 162.

[0109] However, while the configurations of the simultaneous translation device 630 and the single-sentence translation device 632 are the same as those of the simultaneous translation device 160 and the single-sentence translation device 162 shown in Figure 2, it is important to note that the training data used to train them is different. In both cases, parallel translations consisting of a sentence in the source language and a corresponding sentence in the target language are used for training. However, when training the simultaneous translation device 160 and the single-sentence translation device 162, no tags are attached to the beginning of the input parallel translations, whereas when training the simultaneous translation device 630 and the single-sentence translation device 632, it is necessary to attach corresponding tags to the beginning of the source language sentence and the beginning of the target language sentence for each parallel translation in the input training data. The training method itself is the same between the first and second embodiments.

[0110] Figure 9 shows a flowchart illustrating the control structure of a computer program for causing a computer to function as the simultaneous translation system 600 according to this second embodiment. The flowchart in Figure 9 differs from that in Figure 3 in the following way: - Between step 254 and step 256, step 650 is included, which determines whether a tag is attached to the beginning of the input chunk, stores the tag if one is attached, and if no tag is attached, selects a predetermined tag according to information previously instructed by the user or information instructed by the speech recognition device and attaches that tag to the beginning of the word vector sequence of the first chunk; The procedure includes step 652, which repeats step 654 as described below, instead of step 256 in Figure 2.

[0111] Step 654 has a similar configuration to step 258 in Figure 2. However, step 654 differs from step 258 in that, instead of step 300 in Figure 2, it includes step 670, which performs special processing on the leading tag for the leading chunk sequence, unlike step 300.

[0112] Figure 10 shows the control structure of a program that causes a computer to execute step 670. The program for step 300 shown in Figure 10 differs from that for step 300 in that: The step 700 between step 400 and step 402 in Figure 3 includes determining whether or not there is a tag at the beginning of the chunk and branching the control flow according to the result; - Step 702 includes, in response to the determination in step 700 being positive, step 702 inserts the target language tag at the beginning of the translated word sequence by looking up the target language tag corresponding to the first tag of the input chunk from the tag correspondence table 628 shown in Figure 8 and storing it at the beginning of the one-sentence translated text storage unit 182 shown in Figure 8; and Following step 702, variable N T This includes step 704, which involves substituting 1 for the value.

[0113] If the result of step 700 is negative, the control branches to step 402.

[0114] 3. Operation of the second embodiment The operation of the simultaneous translation system 600 is the same as in the first embodiment, except for the operation of the first chunk of word sequence entered into the input buffer 164.

[0115] In other words, when a new sequence of words to be translated is entered into the input buffer 164, the processing in step 650 is performed for the first chunk. Specifically, if a tag is attached to the beginning of the first chunk, that tag is saved; if no tag is attached, a predetermined tag is attached to the beginning of the first chunk according to information previously instructed by the user. After this, in step 652, step 654 is repeatedly executed until the sentence end flag becomes 9, which is the same as in the first embodiment.

[0116] However, for the first chunk, in step 670 of step 654, processing is performed that was not performed in the first embodiment. That is, referring to Figure 10, the determination in step 700 is affirmative in this case, and in step 702, the tag corresponding to the sentence beginning tag of the source language that was attached to the beginning of the input chunk (first chunk) is read from the tag correspondence table 628 (Figure 8) and stored at the beginning of the sentence translation storage unit 182. Furthermore, in the subsequent step 704, the variable N T This is set to 1.

[0117] Variable N T Since this is set to 1, the tag stored at the beginning of the sentence translation memory unit 182 is treated as context when translating the first chunk. Therefore, strings other than the tag in the first chunk are sequentially stored in the second and subsequent positions of the sentence translation memory unit 182. This is the point in which the operation of the simultaneous translation system 600 according to this second embodiment differs from the simultaneous translation system 150 according to the first embodiment.

[0118] 4. Effects of the second embodiment In this second embodiment, by attaching a tag to the beginning of the input word sequence, more accurate translation can be performed according to the context in which the utterance was spoken, such as a specific field, the person speaking, or the person being addressed, unlike in the first embodiment. In particular, in the selection of translations, the tag attached to the beginning of the translation is treated as the context when searching for translation sequences, so it is thought that the probability of selecting an appropriate translation is higher than in the first embodiment.

[0119] As a result, we can provide a simultaneous translation device that can translate input word sequences in near real-time and with higher accuracy than the first embodiment.

[0120] Third computer implementation Figure 11 is an external view of a computer system that implements each of the above embodiments. Figure 12 is a hardware block diagram of the computer system shown in Figure 11.

[0121] Referring to Figure 11, this computer system 950 includes a computer 970 having a DVD drive 1002, and a keyboard 974, a mouse 976, and a monitor 972, all connected to the computer 970, for user interaction. Of course, these are just examples of configurations for when user interaction is required, and any general hardware and software available for user interaction (e.g., touch panels, voice input, pointing devices in general) can be used.

[0122] Referring to Figure 12, the computer 970 includes a CPU 990 and a GPU 992 in addition to the DVD drive 1002. The computer 970 further includes a bus 1010 connected to the CPU 990, GPU 992, and DVD drive 1002, and a ROM 996 connected to the bus 1010 for storing the computer 970's boot-up program, etc. The computer 970 further includes a RAM 998 connected to the bus 1010 for storing program instructions, system programs, and work data, etc., and an SSD 1000, which is a non-volatile memory connected to the bus 1010. The SSD 1000 is for storing programs executed by the CPU 990 and GPU 992, as well as data used by programs executed by the CPU 990 and GPU 992. The computer 970 further includes a network interface 1008 that provides a connection to a network 986 enabling communication with other terminals, and a USB port 1006 that allows a USB memory 984 to be attached and detached and provides communication between the USB memory 984 and various parts within the computer 970.

[0123] The computer 970 further includes an audio interface 1004 connected to the bus 1010 and connected to the microphone 982 and speaker 980. This interface reads audio signals, video signals, and text data generated by the CPU 990 and stored in RAM 998 or SSD 1000 according to the instructions of the CPU 990, performs analog conversion and amplification processing to drive the speaker 980, and digitizes the analog audio signal from the microphone 982 and stores it in RAM 998 or SSD 1000 at any address specified by the CPU 990.

[0124] In the above embodiment, the input word sequence to be translated, the word vector sequence after the input word sequence has been converted, the translated word sequence, the data displayed by the display device 180, and the parameters and programs constituting the simultaneous translation devices 160 and 630 are all stored in a storage medium of an external device (not shown) connected via network I / F 1008 and network 986, for example, as shown in Figure 12: SSD 1000, RAM 998, DVD 978, or USB memory 984. Typically, this data and parameters are written to the SSD 1000 from an external source and loaded into the RAM 998 when the computer 970 is running.

[0125] The computer programs that operate this computer system to realize the functions of the simultaneous translation system 150 shown in Figure 2, the simultaneous translation system 600 shown in Figure 8, and each of its components are stored on a DVD 978 inserted into the DVD drive 1002 and transferred from the DVD drive 1002 to the SSD 1000. Alternatively, these programs are stored on a USB memory 984, the USB memory 984 is inserted into the USB port 1006, and the programs are transferred to the SSD 1000. Alternatively, these programs may be transmitted to a computer 970 via the network 986 and stored in the SSD 1000.

[0126] The program is loaded into RAM998 at runtime. Of course, the source program may be input using the keyboard974, monitor972, and mouse976, and the compiled object program may be stored in SSD1000. In this case, both the source program and the object program correspond to this embodiment. In the case of a scripting language, the script input using the keyboard974, etc., may be stored in SSD1000. In the case of a program that runs on a virtual machine, it is necessary to install a program that functions as a virtual machine on the computer970 in advance. Since machine translation processing by the simultaneous translation systems 150 and 600, and training of the single-sentence translator 162, simultaneous translation device 630, and single-sentence translator 632 involve a large amount of computation, it is preferable to implement each part of the embodiment of the present invention as an object program consisting of the computer's native code rather than a scripting language. For similar reasons, it is desirable that the computer be equipped with a GPU992 that can perform a large amount of multiply-accumulate operations at high speed.

[0127] The CPU990 reads the program from RAM998 according to the address indicated by an internal register called the program counter (not shown), interprets the instructions, reads the data necessary for executing the instructions from RAM998, SSD1000, or other devices according to the address specified by the instructions, and executes the processing specified by the instructions. The CPU990 stores the execution result data at an address specified by the program, such as RAM998, SSD1000, or a register within the CPU990. At this time, the value of the program counter is also updated by the program. Computer programs may be loaded directly into RAM998 from DVD978, USB memory 984, or via a network. In addition, some tasks (mainly numerical calculations) within the program executed by the CPU990 are dispatched to the GPU992 according to the instructions included in the program or according to the analysis results when the CPU990 executes the instructions.

[0128] The program that implements the functions of each part according to each embodiment described above using the computer 970 includes a plurality of instructions written and arranged to operate the computer 970 to implement those functions. Some of the basic functions necessary to execute these instructions are provided by the operating system (OS) running on the computer 970, a third-party program, or modules of various toolkits installed on the computer 970. Therefore, this program does not necessarily have to include all the functions necessary to implement the system and method of this embodiment. This program only needs to include instructions that perform the operation of each of the above-described devices and their components by statically linking or dynamically calling appropriate functions or functions of the "Programming Toolkit" in a controlled manner to obtain the desired result. The method of operating the computer 970 for this purpose is well known and will not be repeated here.

[0129] Furthermore, the GPU992 is capable of parallel processing, allowing it to execute large amounts of calculations associated with machine learning concurrently, in parallel, or in a pipelined manner. For example, parallel computation elements discovered in the program during compilation, or during program execution, are dispatched from the CPU990 to the GPU992 as needed, executed, and the results are returned to the CPU990 directly or via a predetermined address in RAM998, and assigned to a predetermined variable in the program.

[0130] 4. Variation In the above embodiment, the chunk end detection device 170 and the sentence end detection device 174 are separate. However, this invention is not limited to such an embodiment. By labeling chunk end words with 1, sentence end words with 9, and other words with 0, it is also possible to train a single neural network capable of distinguishing between chunk end, sentence end, and other words. In that case, the chunk end detection device 170 and the sentence end detection device 174 can be combined into one.

[0131] Furthermore, the above embodiment makes a distinction between the end of a chunk, the end of a sentence, and everything else. However, this invention is not limited to such distinctions. It may also be possible to combine units that perform translation between chunks and sentences, or to combine units that target ranges beyond a sentence for translation.

[0132] Furthermore, in the above embodiment, both the simultaneous translation device 160 and the simultaneous translation device 630 use the same technology as disclosed in Non-Patent Document 2. However, this invention is not limited to such devices. Any method that converts an input word sequence into an intermediate language representation in some form, such as an encoder-decoder format, and then searches for the word sequence of the translated text from that intermediate language representation, can be incorporated into this embodiment. Also, the single-sentence translation devices 162 and 632 do not necessarily have to be of the same format as the simultaneous translation devices 160 and 630, and may be, for example, statistical machine translation, phrase-based machine translation, or neural machine translation.

[0133] Furthermore, in the above embodiment, the translation result is displayed on the display device 180. However, this invention is not limited to such embodiments. The simultaneous translation device may, for example, output the translation result as speech synthesized to a speaker (not shown). Furthermore, if the simultaneous translation device has two output devices, the simultaneous translation device may output the translation result to both. For example, if the simultaneous translation device has a display device and a speaker as output systems, the simultaneous translation result of the simultaneous translation device may be output from the speaker as speech synthesized, and the result of the single-sentence translation may be displayed on the display device. The reverse may also be true. Furthermore, the display device may display the translation result of the simultaneous translation device and then replace it with the single-sentence translation result, and the speech synthesis may output only the translation result of the single-sentence translation.

[0134] The embodiments disclosed herein are illustrative and not limited to those embodiments. The scope of the present invention is defined by each claim, with reference to the detailed description of the invention, and includes all modifications within the meaning and scope equivalent to the wording contained herein. [Explanation of Symbols]

[0135] 50-chunk translation 60, 64, 74, 84, 94, 518, 532, 542 chunks 62, 70, 72, 78, 80, 88, 90, 92, 98, 100, 506, 514, 516, 522, 524, 536, 538, 540, 548, 550 Translation list Chunk sequence 66, 76, 82, 86, 96, 502, 508, 510, 520, 530, 534, 544 150,600 simultaneous translation systems 160,630 simultaneous translation device 162,632 One-sentence translation device 164 input buffers 166 Word Vector Generation Unit 168 Word Vector Memory Unit 170 Chunk End Detection Device 172, 176 Word vector reading unit 174 Sentence End Detection Device 178 Translation Term Column Memory Section 180 Display device 182 1-Sentence Translation Memory Unit 184 Translation Comparison Section 186 Buffer Clearing Section 200 encoders 202 Decoder 204, 640 Translation Search Department 500, 504 tags 620 Tag Check Section 622, 624 Tag addition section 628 Tag-compatible table 990 CPU

Claims

1. An encoder for encoding an input sequence of words into a predetermined intermediate language representation, Chunk end detection means for detecting the end of a chunk in the word sequence in real time, A word sequence input means for inputting a sub-word sequence from the beginning of the input word sequence to the chunk whose end is detected by the chunk end detection means into the encoder, A decoding means for taking the intermediate language representation output by the encoder as input and outputting a sequence of translated words in a predetermined language for the sub-word sequence from the beginning, The decoding means includes a translated word sequence storage means for storing the translated word sequence output by the decoding means, The decoding means includes sequential decoding means for taking the intermediate language representation output by the encoder as input and searching for a subsequent sequence of translated words based on the probability distribution of word occurrences in the predetermined language, thereby sequentially outputting the sequence of translated words in the predetermined language for the partial sequence of words from the beginning. The sequential decoding means is configured to search for the subsequent sequence of translated words, and to sequentially select the first to last translated words of a sequence already stored in the sequence of translated words storage means without using the probability distribution, in a simultaneous translation device.

2. A sentence-end detection means for detecting the end of a sentence in an input sequence of words, A sentence translation device for outputting a translation of the word sequence up to the end of the sentence in a predetermined language in response to the detection of the end of the sentence by the sentence end detection means, The simultaneous translation apparatus according to claim 1, further comprising: a translation replacement means for replacing the output of the decoding means with the translation from the one-sentence translation device in response to the output of the translation from the one-sentence translation device.

3. The simultaneous translation apparatus according to claim 2, further comprising a clearing means for clearing the translated word sequence storage means in response to the chunk end detection means detecting the end of a chunk of the word sequence after the translated text has been replaced by the translated text replacement means.

4. The aforementioned translation replacement means is In response to the output of the translated sentence by the one-sentence translation device, an evaluation means is provided for evaluating the magnitude of the difference between the output of the decoding means and the translated sentence from the one-sentence translation device. The simultaneous translation apparatus according to claim 2 or 3, further comprising: a substitution means for replacing the output of the decoding means with the translated sentence from the one-sentence translation device in response to the evaluation means determining that the magnitude of the difference is greater than a threshold;

5. A tagging means for assigning a first tag, determined by predetermined conditions, to the beginning of the input word sequence, The simultaneous translation device according to any one of claims 1 to 4, further comprising a tag insertion means for inserting the second tag corresponding to the first tag into the beginning of the translated word sequence by storing the second tag corresponding to the first tag in the translated word sequence storage means when the translated word sequence storage means is cleared.

6. A computer program that causes a computer to function as a simultaneous translation device according to any one of claims 1 to 5.

Images