Japanese-Chinese Machine Translation Method and System Incorporating Phrase Knowledge

Abstract

The present invention provides a Japanese-Chinese machine translation method, system, storage medium and electronic device that integrates phrase knowledge, and relates to the technical field of machine translation. The present invention explicitly models the phrase information of the source language and the target language, and integrates it into the encoder and decoder of the Transformer through cross-attention. In order to further utilize the phrase translation knowledge, a word memory and a copy network are also introduced, effectively overcoming the controllability and interpretability problems brought by end-to-end learning in classical deep machine translation; at the same time, it does not increase the complexity of the decoding process. In addition, the present invention models the clause structure, encodes and translates the clause as a whole, which is beneficial to improving the translation quality of the clause; and since the information of the whole sentence translation is revealed to a certain extent after matching the phrase table, it is more beneficial to the decoding decision-making process, thereby improving the overall translation effect.

Description

Technical Field

[0001] The present invention relates to the technical field of machine translation, and particularly to a Japanese-Chinese machine translation method, system, storage medium, and electronic device that integrates phrase knowledge. Background Art

[0002] Machine translation refers to the technology of using a computer to perform automatic translation between different natural languages. For example, automatically converting Japanese into Chinese. At this time, Japanese is called the source language, and Chinese is called the target language. Since the proposal of machine translation based on the noisy channel principle, researchers have successively proposed a series of methods based on rules, examples, and statistics. In recent years, with the development of deep learning, neural machine translation (NMT) based on deep neural networks has become the mainstream technology. It adopts an end-to-end training method, avoiding the cumbersome feature engineering and error accumulation caused by modularization, and thus has achieved great success in both the industrial and academic fields.

[0003] Currently, the deep learning model Transformer provides a brand-new solution idea for sequence-to-sequence learning. It is the first model completely built based on the attention mechanism. Since the advantage of the self-attention mechanism is that it can directly model the relationship between any two units in the sequence, this enables problems such as long-distance dependencies to be better solved. In addition, the self-attention mechanism is very suitable for parallelization on the GPU, so the model training speed is very fast, and better results have also been obtained in translation tasks, becoming the most mainstream neural machine translation framework at present, and also achieving good results in various tasks of natural language processing.

[0004] However, precisely because the neural network machine based on Transformer adopts an end-to-end learning and training method, there are defects in its controllability and interpretability, which affect the translation quality. Summary of the Invention

[0005] (1) Technical Problems to be Solved

[0006] Aiming at the deficiencies of the prior art, the present invention provides a Japanese-Chinese machine translation method, system, storage medium, and electronic device that integrates phrase knowledge, and solves the technical problem that the quality of existing Japanese-Chinese machine translation needs to be improved.

[0007] (2) Technical Solutions

[0008] To achieve the above object, the present invention is realized through the following technical solutions:

[0009] A Japanese-Chinese machine translation method that integrates phrase knowledge, following the encoder-decoder structure based on Transformer, the method includes:

[0010] S1. Obtain a Japanese sentence and perform clause recognition on the Japanese sentence;

[0011] S2. According to the pre-constructed phrase table, sequentially perform phrase matching on the recognized clauses, and obtain Japanese phrases and Chinese phrases that have an alignment relationship with each other;

[0012] S3. Use a Japanese phrase encoder to encode and represent each Japanese phrase as a first vector, and use a Chinese phrase encoder to encode and identify each Chinese phrase as a second vector;

[0013] S4. Input the Japanese sentence into the encoder, and combine with the first vector to obtain a context representation that integrates Japanese phrase information;

[0014] S5. Initialize the Chinese sentence to be generated, input the Chinese word sequence translated and output at the previous moment into the decoder, and combine with the second vector and the context representation that integrates Japanese phrase information to obtain the hidden representation at the current moment;

[0015] S6. According to the hidden representation at the current moment, obtain the probability distribution of each Chinese word in the current decoding result, denoted as the generation probability;

[0016] Obtain the probability that each Chinese word in the current decoding result is a Chinese word in the word memory of the copy words; the word memory is composed of the content words of all Chinese phrases obtained in S2;

[0017] S7. According to the generation probability and the copy probability, obtain the output at the current moment;

[0018] S8. Repeat steps S5 to S7 until the end-of-sentence marker is generated, and connect the outputs at all moments to form the Chinese sentence of the machine translation.

[0019] Preferably, the construction process of the phrase table in S2 includes:

[0020] S2a. Preprocess the Japanese-Chinese parallel corpus, including performing Chinese word segmentation and Japanese word segmentation on the two languages respectively, and identifying clauses for Japanese;

[0021] S2b. Obtain word alignment based on a word alignment tool, and obtain the word alignment relationship between the two languages;

[0022] S2c. Mine candidate phrase pairs from the word alignment:

[0023] Among them, the mined candidate Japanese phrases are limited to continuous word fragments within the same clause and with a length less than or equal to a preset number; the mined candidate Chinese phrases are target sentence fragments with a length less than or equal to the same preset number and satisfying alignment consistency with the candidate Japanese phrases;

[0024] S2d, Phrase Pair Filtering:

[0025] If a candidate Japanese phrase corresponds to multiple candidate Chinese phrases, only the top preset number of target Chinese phrases with the highest scores are retained.

[0026] Preferably, the score of each phrase pair in S2d is the product of its co-occurrence frequency and similarity, where the similarity is calculated by mBERT. The specific process is as follows:

[0027] Similarity(P i , P j ) = Cosine(Enc(P i ), Enc(P j )) (1)

[0028]

[0029] c1, c2,..., c L = mBERT(P i1 , P i2 ,..., P iL ) (3)

[0030] Among them, Similarity(P i , P j ) represents the similarity between the Japanese phrase P i and the Chinese phrase P j . Cosine is the cosine distance, Enc(P) is the vector representation of the phrase, P i1 , P i2 ,..., P iL is the sub-word sequence obtained by splitting the phrase P i by mBERT, and c1, c2,..., c L is the sequence of context representation vectors output by mBERT for it.

[0031] Preferably, in S3, a Japanese phrase encoder is used to encode each Japanese phrase into a first vector, specifically including:

[0032] Assume that after segmenting the Japanese phrase, it is represented as S = s1, s2,..., s M ;

[0033] First, obtain the initial vector representation of the phrase That is:

[0034]

[0035] Among them, s m is the m-th word in the Japanese phrase, and E(s m ) is its word vector;

[0036] Secondly, obtain the final vector representation of the phrase as follows:

[0037]

[0038]

[0039] where α m is the attention distribution probability of the initial representation for the m-th word, · represents the vector inner product operator, and the softmax function in formula (6) realizes the normalization of probabilities, ensuring that the sum of attention probabilities over all words is 1;

[0040] Preferably, in S3, a Chinese phrase encoder is used to encode and identify each Chinese phrase as a second vector, specifically including:

[0041] Assume that in the phrase matching stage, the Japanese phrase is S, corresponding to I Chinese phrases, namely T 1 , T 2 ,..., T I ;

[0042] First, calculate the vector representation of each Chinese phrase T i = t1, t2,..., t N as follows: as follows:

[0043]

[0044]

[0045] where t j is the j-th word in the Chinese phrase, the softmax function in formula (8) realizes the normalization of probabilities, ensuring that the sum of attention probabilities over all words is 1; E(t j ) is its word vector; β j is the attention distribution probability of the Japanese phrase for the j-th word in the Chinese phrase; Linear represents a linear mapping, and its input and output maintain the same dimension; at this time, I Chinese phrases are obtained, namely

[0046] Secondly, aggregate them into a Chinese phrase representation as follows:

[0047]

[0048]

[0049] where γ iIt is the attention distribution probability of the i-th Chinese phrase for the Japanese phrase pair.

[0050] Preferably, in S4, in addition to self-attention, the encoder adds a layer of multi-head cross-attention for fusing the first vector; wherein, the word context representation output by the self-attention is used as the query parameter, and the first vector is input into this multi-head attention as both the key and value parameters at the same time, and finally the context representation fusing the Japanese phrase information is output.

[0051] Preferably, in S5, the decoder adds a layer of multi-head cross-attention for fusing the second vector, and fuses the output of the encoder through cross-attention to finally obtain the hidden representation at the current moment; wherein, the word context representation output by the self-attention is used as the query parameter, and the second vector is input into this multi-head attention as both the key and value parameters at the same time.

[0052] Preferably, S7 specifically includes:

[0053] p(y j ) = (1 - λ)p gen (y j ) + λp copy (y j ) (11)

[0054] λ = sigmod(W1·∑ i p copy (y j = w i )E(w i ) + W2·H j + b) (12)

[0055] p copy (y j = w i ) = softmax(Linear(H j )·Linear(E(w i ))) (13)

[0056] Among them, p gen (y j ) represents the generation probability, and p copy (y j ) represents the copy probability;

[0057] The sigmoid function makes the output value range be (0, 1), W1, is the weight parameter, and both are of the same dimension d as the word vector; is the bias term, λ is the adaptive weight coefficient, and is used to weigh the importance of the generation probability and the copy probability;

[0058] Linear represents a linear mapping whose input and output maintain the same dimension, and H j is the hidden representation output by the decoder at the current moment, and w i is the i-th word in the word memory, and E(w i ) is its word vector.

[0059] A Japanese-Chinese machine translation system that integrates phrase knowledge and follows the encoder-decoder structure based on Transformer, the system includes:

[0060] An identification module for performing S1, obtaining a Japanese sentence, and performing clause identification on the Japanese sentence;

[0061] A matching module for performing S2, sequentially performing phrase matching on the identified clauses according to a pre-constructed phrase table, and obtaining Japanese phrases and Chinese phrases that have an alignment relationship with each other;

[0062] An encoding module for performing S3, encoding each of the Japanese phrases into a first vector using a Japanese phrase encoder, and encoding and identifying each of the Chinese phrases into a second vector using a Chinese phrase encoder;

[0063] A decoding module for performing S4, inputting the Japanese sentence into the encoder, and obtaining a context representation that integrates Japanese phrase information in combination with the first vector;

[0064] An acquisition module for performing S5, initializing the Chinese sentence to be generated, inputting the Chinese word sequence translated and output at the previous moment into the decoder, and obtaining the hidden representation at the current moment in combination with the second vector and the context representation that integrates Japanese phrase information;

[0065] A probability module for performing S6, obtaining the probability distribution of each Chinese word in the current decoding result according to the hidden representation at the current moment, denoted as the generation probability;

[0066] Obtaining the probability that each Chinese word in the current decoding result is a Chinese word copied from the word memory, denoted as the copy probability; the word memory is composed of the content words of all Chinese phrases obtained in S2;

[0067] An output module for performing S7, obtaining the output at the current moment according to the generation probability and the copy probability;

[0068] An end module for performing S8, repeatedly executing S5-S7 until a sentence end marker is generated, and connecting the outputs at all moments to form the Chinese sentence of the machine translation.

[0069] A storage medium stores a computer program for Japanese-Chinese machine translation that fuses phrase knowledge, wherein the computer program causes a computer to execute the Japanese-Chinese machine translation method for fusing phrase knowledge as described above.

[0070] An electronic device includes:

[0071] One or more processors;

[0072] A memory; and

[0073] One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the Japanese-Chinese machine translation for fusing phrase knowledge as described above.

[0074] (III) Advantageous Effects

[0075] The present invention provides a Japanese-Chinese machine translation method, system, storage medium, and electronic device for fusing phrase knowledge. Compared with the prior art, the following advantageous effects are achieved:

[0076] The present invention explicitly models the phrase information of the source language and the target language, and fuses it into the encoder and decoder of the Transformer through cross-attention. In order to further utilize the phrase translation knowledge, a word memory and a copy network are also introduced, effectively overcoming the controllability and interpretability problems brought by end-to-end learning in classical deep machine translation; at the same time, it does not increase the complexity of the decoding process.

[0077] In addition, the present invention models the clause structure, encodes and translates the clause as a whole, which is beneficial to improving the translation quality of the clause; and since the information of the whole sentence translation is revealed to a certain extent after matching the phrase table, it is more beneficial to the decoding decision-making process, thereby improving the overall translation effect. Description of the Drawings

[0078] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0079] Figure 1 It is a structural block diagram of a Japanese-Chinese machine translation method for fusing phrase knowledge provided by an embodiment of the present invention;

[0080] Figure 2 It is a schematic flowchart of a Japanese-Chinese machine translation method for fusing phrase knowledge provided by an embodiment of the present invention. Detailed Implementation Manner

[0081] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described. Apparently, the described embodiments are some, rather than all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0082] By providing a Japanese-Chinese machine translation method, system, storage medium, and electronic device that integrate phrase knowledge, the embodiments of the present application solve the technical problem that the quality of existing Japanese-Chinese machine translation needs to be improved.

[0083] The overall idea of the technical solutions in the embodiments of the present application to solve the above technical problems is as follows:

[0084] 1. The embodiments of the present invention first introduce clause information into Japanese-Chinese machine translation, effectively improving the effect of Japanese-Chinese machine translation. As a special phrase structure in Japanese, a clause starts with one or more content words and is followed by zero or one or more function words. Grammatically, a clause reveals the role of words in a sentence. The segmentation of a clause is shown in the following example:

[0085] Original text: Next, the efforts of the said company, which is a member of Matsushita Electric, were outlined.

[0086] Clause sequence: Next, | Matsushita / Electric / of | a member / is | the said / company / of | efforts / were | outlined / .

[0087] Among them, different clauses are separated by vertical lines, and words are separated by slashes.

[0088] 2. The present invention proposes a method for mining bilingual phrase pairs and integrating phrase translation knowledge into the translation model of a transformer. By explicitly modeling the phrase information of the source language and the target language and fusing it into the encoder and decoder through cross-attention, in order to further utilize phrase translation knowledge, word memory and a copy network are also introduced, effectively overcoming the controllability and interpretability problems brought by end-to-end learning in classical deep machine translation. At the same time, it does not increase the complexity of the decoding process.

[0089] 3. Special processing is also performed on English, numbers, etc. in Japanese, that is, they are directly copied as the target phrase itself and added to the decoding process, which is beneficial to the translation of out-of-vocabulary words.

[0090] To better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0091] Example:

[0092] As Figure 1 shown, the embodiment of the present invention provides a Japanese-Chinese machine translation method that integrates phrase knowledge and follows an encoder-decoder structure based on Transformer. The method includes:

[0093] S1. Obtain a Japanese sentence and perform clause recognition on the Japanese sentence;

[0094] S2. According to a pre-constructed phrase table, sequentially perform phrase matching on the recognized clauses to obtain Japanese phrases and Chinese phrases that have an alignment relationship with each other;

[0095] S3. Use a Japanese phrase encoder to encode and represent each Japanese phrase as a first vector, and use a Chinese phrase encoder to encode and identify each Chinese phrase as a second vector;

[0096] S4. Input the Japanese sentence into the encoder, and combine with the first vector to obtain a context representation that integrates Japanese phrase information;

[0097] S5. Initialize the Chinese sentence to be generated, input the Chinese word sequence translated and output at the previous moment into the decoder, and combine with the second vector and the context representation that integrates Japanese phrase information to obtain the hidden representation at the current moment;

[0098] S6. According to the hidden representation at the current moment, obtain the probability distribution of each Chinese word in the current decoding result, denoted as the generation probability;

[0099] Obtain the probability that each Chinese word in the current decoding result is a Chinese word in the word memory of the copy words. The word memory is composed of the content words of all Chinese phrases obtained in S2;

[0100] S7. According to the generation probability and the copy probability, obtain the output at the current moment;

[0101] S8. Repeat S5 to S7 until the end-of-sentence marker is generated, and connect the outputs at all moments to form the Chinese sentence of the machine translation.

[0102] The embodiment of the present invention mines bilingual phrase pairs in Japanese-Chinese parallel corpora and integrates them into the encoder-decoder structure to realize guiding the translation decoding process by using bilingual phrase knowledge.

[0103] Next, each step of the above technical solution will be introduced in detail:

[0104] First of all, it needs to be introduced that the embodiment of the present invention needs to pre-construct a phrase table. The construction process of the phrase table includes:

[0105] S2a. Preprocess the Japanese-Chinese parallel corpus, including performing Chinese word segmentation and Japanese word segmentation on the two languages respectively, and using relevant tools such as the open-source tool KNP parser to identify clauses for Japanese.

[0106] S2b. Based on a word alignment tool such as the open-source tool Giza++, obtain word alignments and acquire the word alignment relationships between the two languages.

[0107] S2c. Mine candidate phrase pairs from the word alignments:

[0108] Among them, the mined candidate Japanese phrases are limited to consecutive word fragments within the same clause and with a length less than or equal to a preset number; the mined candidate Chinese phrases are target sentence fragments with a length less than or equal to the same preset number and satisfying alignment consistency with the candidate Japanese phrases.

[0109] Among them, considering that about 96% of the clauses contain less than or equal to 4 words, the above preset number can be taken as L = 4; the alignment consistency specifically means that each word in the Chinese phrase is aligned only with a certain word within the Japanese phrase or is aligned as empty; the same is true for each word within the Japanese phrase, which is aligned only with a certain word within the Chinese phrase or is aligned as empty.

[0110] S2d. Phrase pair filtering:

[0111] If a candidate Japanese phrase corresponds to multiple candidate Chinese phrases, only retain the top preset number of target Chinese phrases with the highest scores;

[0112] Among them, the score of each phrase pair is the product of its co-occurrence frequency and similarity, where the similarity is calculated by mBERT, and the specific process is as follows:

[0113] Similarity(P i , P j ) = Cosine(Enc(P i ), Enc(P j )) (1)

[0114]

[0115] c1, c2,..., c L = mBERT(P i1 , P i2 ,..., P iL ) (3)

[0116] Among them, Similarity(P i , P j ) represents the similarity between the Japanese phrase P i and the Chinese phrase P jThe similarity, Cosine is the cosine distance, Enc(P) is the vector representation of the phrase, and P i1 , P i2 ,..., P iL is the phrase P i The sub-word sequence after being segmented by mBERT, c1, c2,..., c L is the sequence of context representation vectors output by its mBERT.

[0117] In step S1, a Japanese sentence is obtained, and the clauses of the Japanese sentence are recognized.

[0118] In the embodiments of the present invention, clause information is introduced into Japanese-Chinese machine translation for the first time, effectively improving the effect of Japanese-Chinese machine translation. It is precisely because clauses preserve relatively complete syntactic and semantic information. From a linguistic perspective, modeling this language structure is beneficial to improving the effect of natural language understanding and generation of Japanese.

[0119] In addition, the embodiments of the present invention are oriented to Japanese-to-Chinese translation. As a special Japanese phrase structure, clauses are also adapted to the clause structure in phrase mining and phrase matching. For example, it is limited that a phrase is within a clause. For specific content, please refer to the subsequent steps.

[0120] In step S2, according to the pre-constructed phrase table, the recognized clauses are sequentially subjected to phrase matching to obtain Japanese phrases and Chinese phrases that have an alignment relationship with each other.

[0121] According to the aforementioned constructed phrase table, the clauses in the Japanese sentence are sequentially subjected to phrase matching:

[0122] If an entire clause completely matches a certain Japanese phrase in the phrase table, directly take its Japanese phrase and all corresponding Chinese phrases. Otherwise, the clause is segmented into several phrases at the word segmentation boundary so that each phrase segment after segmentation is a certain Japanese phrase in the phrase table or an isolated word. When there are multiple segmentation paths, select the segmentation method with the shortest path, that is, preferentially select long phrases; when the lengths of two segmentation paths are equal, preferentially select the path with the highest path score, and its score is equal to the product of the frequencies of the phrases included in the path.

[0123] For other characters in the Japanese sentence that are not Chinese characters and Japanese letters, such as English letters, punctuation marks, and numbers, they may be continuously segmented into multiple isolated words. At this time, these continuous word segments are spliced into a Japanese phrase, and its own copy is used as the target Chinese phrase because these non-Japanese characters are the same characters in Chinese. For other isolated words that do not exist in the phrase table, they are directly discarded. For English and numbers in Japanese, their own copies are directly used as target phrases and added to the decoding process, which is beneficial to the translation of out-of-vocabulary words.

[0124] In step S3, each of the Japanese phrases is encoded and represented as a first vector by using a Japanese phrase encoder, and each of the Chinese phrases is encoded and identified as a second vector by using a Chinese phrase encoder.

[0125] As Figure 2 shown, the step of encoding each of the Japanese phrases as a first vector by using a Japanese phrase encoder specifically includes:

[0126] Assume that after segmenting the Japanese phrase, it is represented as S = s1, s2,..., s M ;

[0127] First, obtain the initial vector representation of the phrase That is:

[0128]

[0129] where s m is the m-th word in the Japanese phrase, and E(s m ) is its word vector;

[0130] Second, obtain the final vector representation of the phrase as follows:

[0131]

[0132]

[0133] where α m is the attention distribution probability of the initial representation for the m-th word, · represents the vector inner product operator, and the softmax function in formula (6) realizes the normalization of probabilities, ensuring that the sum of the attention probabilities over all words is 1.

[0134] As Figure 2 shown, the step of encoding each of the Chinese phrases as a second vector by using a Chinese phrase encoder in S3 specifically includes:

[0135] Assume that in the phrase matching stage, the Japanese phrase is S, corresponding to I Chinese phrases, namely T 1 , T 2 ,..., T I ;

[0136] First, calculate the vector representation of each Chinese phrase T i = t1, t2,..., t N as follows: as follows:

[0137]

[0138]

[0139] where \(t_j\) j is the \(j\)-th word in the Chinese phrase. The softmax function in formula (8) normalizes the probabilities, ensuring that the sum of the attention probabilities over all words is 1; \(E(t_j)\) j is its word vector; \(\beta_j\) j is the attention distribution probability of the Japanese phrase to the \(j\)-th word in the Chinese phrase; Linear represents a linear mapping whose input and output have the same dimension. At this time, \(I\) Chinese phrases are obtained, namely

[0140] Secondly, they are aggregated into a Chinese phrase representation as follows:

[0141]

[0142]

[0143] where \(\gamma_i\) i is the attention distribution probability of the Japanese phrase to the \(i\)-th Chinese phrase.

[0144] In step S4, the Japanese sentence is input into the encoder, and combined with the first vector, a context representation integrating Japanese phrase information is obtained.

[0145] As Figure 2 shown, in addition to self-attention, the encoder adds a layer of multi-head cross-attention for integrating the first vector; among them, the word context representation output by self-attention is used as the query parameter, and the first vector is simultaneously used as the key and value parameters and input into this multi-head attention, and finally a context representation integrating Japanese phrase information is output.

[0146] In step S5, the Chinese sentence to be generated is initialized, and the Chinese word sequence output by the previous translation is input into the decoder, and combined with the second vector and the context representation integrating Japanese phrase information, a hidden representation at the current moment is obtained.

[0147] The Chinese sentence to be generated is initialized to contain only a start character ' <sos>’, that is, y0 = <sos>。

[0148] As Figure 2 shown, the decoder adds a layer of multi-head cross-attention to fuse the second vector, and fuses the output of the encoder through cross-attention to finally obtain the hidden representation at the current moment; among them, the word context representation output by the self-attention is used as the query parameter, and the second vector is input into the multi-head attention as both the key and value parameters at the same time.

[0149] In step S6, according to the hidden representation at the current moment, obtain the probability distribution of each Chinese word in the current decoding result, denoted as the generation probability; obtain the probability that each Chinese word in the current decoding result is a Chinese word in the copied word memory, denoted as the copy probability; the word memory is composed of the content words of all Chinese phrases obtained in S2.

[0150] The decoder uses a Softmax function to calculate the probability distribution of each word in the Chinese vocabulary based on the hidden representation at the current moment, denoted as the generation probability p gen (y j ).

[0151] Collect the content words (i.e., filter out function words such as "le", "de", etc.) in all Chinese phrases matched in S2 into a Chinese word candidate set, called the word memory. Use the copy network to calculate the probability that the current decoding result is a word in the copied word memory, denoted as the copy probability p copy (y j ).

[0152] In step S7, according to the generation probability and the copy probability, obtain the output at the current moment; specifically including:

[0153] p(y j )=(1 - λ)p gen (y j )+λp copy (y j ) (11)

[0154] λ = sigmod(W1·∑ i p copy (y j = w i )E(w i )+W2·H j +b) (12)

[0155] p copy (y j = w i )= softmax(Linear(H j )·Linear(E(w i ))) (13)

[0156] Among them, p gen (y j ) represents the generation probability, and p copy (y j ) represents the copy probability;

[0157] The sigmoid function makes the output value range be (0, 1), and W1, being the weight parameter, has the same dimension d as the word vector; being the bias term, and λ being the adaptive weight coefficient, is used to weigh the importance of the generation probability and the copy probability;

[0158] Linear represents a linear mapping, whose input and output keep the dimension unchanged, and H j is the hidden representation output by the decoder at the current moment, w i is the i-th word in the word memory, and E(w i ) is its word vector.

[0159] In step S8, S5 - S7 are repeatedly executed until the end-of-sentence marker is generated, and the outputs at all moments are concatenated to form the Chinese sentence of the machine translation.

[0160] By means of the method of integrating phrase translation knowledge, the embodiment of the present invention, while retaining the powerful representation learning ability of deep learning, adopts a method similar to using a phrase table in phrase-based statistical machine translation, so it also has the advantages of good controllability and interpretability of statistical machine translation. Specifically, before generating the translation, the present invention retrieves in advance the bilingual phrase knowledge mined from the parallel corpus to constrain and guide the translation decoding process, thereby effectively improving the controllability and interpretability of the translation.

[0161] In the previous Japanese-Chinese translation systems, usually only the word sequence was modeled, that is, only the information at the word granularity was modeled. The embodiment of the present invention models the clause structure, encodes and represents and translates the clause as a whole, which is beneficial to improving the translation quality of the clause. In addition, in the previous autoregressive decoding process, when generating the next word, only the part of the target sentence generated so far, that is, the information of the above text, can be used, and the part of the sentence that has not been generated cannot be used, so there is a lack of global information. In the embodiment of the present invention, after matching the phrase table, the information after the whole sentence translation is largely revealed, which is more beneficial to the decoding decision-making process, thereby improving the overall translation effect.

[0162] The embodiment of the present invention provides a Japanese-Chinese machine translation system integrating phrase knowledge, following the encoder-decoder structure based on Transformer. This system includes:

[0163] An identification module, configured to execute S1: obtain a Japanese sentence, and perform clause identification on the Japanese sentence;

[0164] A matching module, configured to execute S2: according to a pre-constructed phrase table, sequentially perform phrase matching on the identified clauses, and obtain Japanese phrases and Chinese phrases that have an alignment relationship with each other;

[0165] An encoding module, configured to execute S3: use a Japanese phrase encoder to encode each of the Japanese phrases as a first vector, and use a Chinese phrase encoder to encode and identify each of the Chinese phrases as a second vector;

[0166] A decoding module, configured to execute S4: input the Japanese sentence into an encoder, and obtain a context representation that fuses Japanese phrase information in combination with the first vector;

[0167] An obtaining module, configured to execute S5: initialize the Chinese sentence to be generated, input the Chinese word sequence translated and output at the previous moment into a decoder, and obtain a hidden representation at the current moment in combination with the second vector and the context representation that fuses Japanese phrase information;

[0168] A probability module, configured to execute S6: according to the hidden representation at the current moment, obtain a probability distribution of each Chinese word in the current decoding result, denoted as a generation probability;

[0169] Obtain the probability that each Chinese word in the current decoding result is a Chinese word in a word memory, denoted as a copy probability; the word memory is composed of content words of all Chinese phrases obtained in S2;

[0170] An output module, configured to execute S7: according to the generation probability and the copy probability, obtain the output at the current moment;

[0171] An end module, configured to execute S8: repeatedly execute S5 to S7 until a sentence end marker is generated, and connect the outputs at all moments to form a Chinese sentence for machine translation.

[0172] An embodiment of the present invention provides a storage medium, which stores a computer program for Japanese-Chinese machine translation that fuses phrase knowledge, wherein the computer program enables a computer to execute the Japanese-Chinese machine translation method for fusing phrase knowledge as described above.

[0173] An embodiment of the present invention provides an electronic device, including:

[0174] One or more processors;

[0175] A memory; and

[0176] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs include a Japanese-Chinese machine translation method for performing the fusion of phrase knowledge as described above.

[0177] In summary, compared with the prior art, the following beneficial effects are achieved:

[0178] 1. The embodiments of the present invention explicitly model the phrase information of the source language and the target language, and fuse it into the encoder and decoder of the Transformer through cross-attention. In order to further utilize the phrase translation knowledge, a word memory and a copy network are also introduced, effectively overcoming the controllability and interpretability problems brought by end-to-end learning in classical deep machine translation; at the same time, it does not increase the complexity of the decoding process.

[0179] 2. The embodiments of the present invention model the clause structure, encode and translate the clause as a whole, which is beneficial to improving the translation quality of the clause; and since some information about the whole sentence translation is revealed to a certain extent after matching the phrase table, it is more beneficial to the decoding decision-making process, thus improving the overall translation effect.

[0180] 3. The embodiments of the present invention directly copy English, numbers, etc. in Japanese as the target phrase itself and add them to the decoding process, which is beneficial to the translation of out-of-vocabulary words.

[0181] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0182] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.< / sos> < / sos>

Claims

1. A Japanese-Chinese machine translation method integrating phrase knowledge, characterized in that, Following the Transformer-based encoder-decoder structure, this method includes: S1. Obtain a Japanese sentence and perform clause recognition on the Japanese sentence; S2. According to the pre-constructed phrase table, sequentially perform phrase matching on the recognized clauses to obtain Japanese phrases and Chinese phrases that have an alignment relationship with each other; S3. Use a Japanese phrase encoder to encode and represent each Japanese phrase as a first vector, and use a Chinese phrase encoder to encode and identify each Chinese phrase as a second vector; S4. Input the Japanese sentence into the encoder, and combine with the first vector to obtain a context representation that fuses Japanese phrase information; S5. Initialize the Chinese sentence to be generated, input the Chinese word sequence translated and output at the previous moment into the decoder, and combine with the second vector and the context representation that fuses Japanese phrase information to obtain the hidden representation at the current moment; S6. According to the hidden representation at the current moment, obtain the probability distribution of each Chinese word in the current decoding result, denoted as the generation probability; Obtain the probability that each Chinese word in the current decoding result is a Chinese word in the word memory of the copy words, denoted as the copy probability; the word memory is composed of the content words of all Chinese phrases obtained in S2; S7. According to the generation probability and the copy probability, obtain the output at the current moment; S8. Repeat S5-S7 until the end-of-sentence marker is generated, and connect the outputs at all moments to form the Chinese sentence of the machine translation.

2. The Chinese-Japanese machine translation method according to claim 1, wherein, The construction process of the phrase table in S2 includes: S2a. Preprocess the Japanese-Chinese parallel corpus, including performing Chinese word segmentation and Japanese word segmentation on the two languages respectively, and identifying clauses for Japanese; S2b. Obtain word alignment based on a word alignment tool to obtain the word alignment relationship between the two languages; S2c. Mine candidate phrase pairs from the word alignment: Among them, the mined candidate Japanese phrases are limited to continuous word segments within the same clause and with a length less than or equal to a preset number; the mined candidate Chinese phrases are target sentence segments with a length less than or equal to the same preset number and satisfying alignment consistency with the candidate Japanese phrases; S2d. Phrase pair filtering: If a candidate Japanese phrase corresponds to multiple candidate Chinese phrases, only retain the top preset number of target Chinese phrases with the highest scores.

3. The Japanese-Chinese machine translation method according to claim 2, characterized in that, The score of each phrase pair in S2d is the product of its co-occurrence frequency and similarity, where the similarity is calculated by mBERT. The specific process is as follows: Similarity(P i ,P j ) = Cosine(Enc(P i ), Enc(P j )) (1) c1, c2,..., c L = mBERT(P i1 , P i2 ,..., P iL ) (3) Among them, Similarity(P i ,P j ) represents the similarity between the Japanese phrase P i and the Chinese phrase P j . Cosine is the cosine distance, Enc(P) is the vector representation of the phrase, P i1 ,P i2 ,...,P iL is the sub-word sequence after the phrase P i is segmented by mBERT, and c1, c2,..., c L is the sequence of context representation vectors output by its mBERT.

4. The Japanese-Chinese machine translation method according to claim 1, wherein In S3, using a Japanese phrase encoder to encode and represent each Japanese phrase as a first vector specifically includes: Assume that after segmenting a Japanese phrase, it is represented as S = s1, s2,..., s M ; First, obtain the initial vector representation of the Japanese phrase That is: where s m is the m-th word in a Japanese phrase, and E(s m ) is its word vector; Secondly, obtain the final vector representation of the Japanese phrase as follows: where α m is the attention distribution probability of the initial representation for the m-th word, · represents the vector inner product operator, and the softmax function in formula (6) realizes the normalization of probabilities, ensuring that the sum of attention probabilities over all words is 1; And / or in S3, using a Chinese phrase encoder to encode and identify each Chinese phrase as a second vector specifically includes: Assume that in the phrase matching stage, the Japanese phrase is S, corresponding to I Chinese phrases, namely T 1 ,T 2 ,...,T I ; First, calculate the vector representation of each Chinese phrase T i = t1, t2,..., t N as follows: as follows: where t j is the j-th word in the Chinese phrase, and the softmax function in formula (8) realizes the normalization of probabilities, ensuring that the sum of attention probabilities over all words is 1; E(t j ) is its word vector; β j is the attention distribution probability of the Japanese phrase for the j-th word in the Chinese phrase; Linear represents a linear mapping, whose input and output maintain the same dimension; at this time, I Chinese phrases are obtained, namely Secondly, aggregate into a Chinese phrase to represent as follows: where γ i is the attention distribution probability of the i-th Chinese phrase for the Japanese phrase pair.

5. The Japanese-Chinese machine translation method according to claim 1, characterized in that, In S4, in addition to self-attention, the encoder adds a layer of multi-head cross-attention for fusing the first vector; among them, the word context representation output by self-attention is used as the query parameter, and the first vector is input into the multi-head cross-attention of the encoder as both the key and value parameters at the same time, and finally the context representation that fuses Japanese phrase information is output.

6. The Chinese-Japanese machine translation method according to claim 5, wherein In S5, the decoder adds a layer of multi-head cross-attention to fuse the second vector, and fuses the output of the encoder through cross-attention to finally obtain the hidden representation at the current moment. Among them, the word context representation output by the self-attention is used as the query parameter, and the second vector is input into the multi-head cross-attention of the decoder as both the key and value parameters at the same time.

7. The Chinese-Japanese machine translation method according to any one of claims 1 to 6, characterized in that S7 specifically includes: p(y j ) = (1 - λ)p gen (y j ) + λp copy (y j ) (11) λ=sigmoid(W1·∑ i p copy (y j =w i )E(w i )+W2·H j +b) (12) p copy (y j = w i ) = softmax(Linear(H j )·Linear(E(w i ))) (13) Among them, p gen (y j ) represents the generation probability, and p copy (y j ) represents the copy probability; The sigmoid function makes the output range from (0, 1). are weight parameters, all having the same dimension d as the word vectors; is the bias term, and λ is the adaptive weight coefficient used to balance the importance of the generation probability and the copy probability; Linear represents a linear mapping whose input and output maintain the same dimension, and H j is the hidden representation output by the decoder at the current time step, and w i is the i-th word in the word memory, and E(w i ) is its word vector.

8. A Japanese-Chinese machine translation system that integrates phrase knowledge, characterized in that, Following the Transformer-based encoder-decoder structure, this system includes: An identification module for performing S1, obtaining a Japanese sentence, and performing clause identification on the Japanese sentence; A matching module for performing S2, sequentially performing phrase matching on the identified clauses according to a pre-constructed phrase table, and obtaining Japanese phrases and Chinese phrases with an alignment relationship therebetween; An encoding module for performing S3, encoding each of the Japanese phrases into a first vector using a Japanese phrase encoder, and encoding and identifying each of the Chinese phrases into a second vector using a Chinese phrase encoder; A decoding module for performing S4, inputting the Japanese sentence into the encoder, and obtaining a context representation that fuses Japanese phrase information in combination with the first vector; An obtaining module for performing S5, initializing the Chinese sentence to be generated, inputting the Chinese word sequence translated and output at the previous moment into the decoder, and obtaining the hidden representation at the current moment in combination with the second vector and the context representation that fuses Japanese phrase information; A probability module for performing S6, obtaining the probability distribution of each Chinese word in the current decoding result according to the hidden representation at the current moment, denoted as the generation probability; Obtaining the probability that each Chinese word in the current decoding result is a Chinese word in the word memory for copying, denoted as the copy probability; the word memory is composed of the content words of all Chinese phrases obtained in S2; An output module for performing S7, obtaining the output at the current moment according to the generation probability and the copy probability; An end module for performing S8, repeatedly executing S5 to S7 until a sentence end marker is generated, and connecting the outputs at all moments to form the Chinese sentence of the machine translation.

9. A storage medium, characterized in that, It stores a computer program for Japanese-Chinese machine translation that fuses phrase knowledge, wherein the computer program causes a computer to execute the Japanese-Chinese machine translation method for fusing phrase knowledge according to any one of claims 1 to 7.

10. An electronic device, characterized in that, Including: One or more processors; A memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a Japanese-Chinese machine translation method for fusing phrase knowledge according to any one of claims 1 to 7.

Images