A knowledge-enhanced non-autoregressive neural machine translation method and device

Abstract

A knowledge-enhanced non-autoregressive neural machine translation method and device, the method performs data preprocessing and word vector encoding on a bilingual parallel language pair; inputs the word vector representation of the source language into an encoder network, the encoder network encodes the source language document information, and obtains an encoded representation of the input word sequence information; uses a word alignment model to construct a corresponding relationship between the source language and the target language, and constructs a reproduction rate model; constructs input and output encoded representations of a decoder model; establishes dependencies between target language vocabulary through a conditional random field model, and sequentially decodes to generate a final translation result. The present invention uses a conditional random field for decoding at the decoding end; the pre-trained language model itself carries strong context information, and the conditional random field constructs a context dependency relationship, which alleviates the phenomenon of a large number of re-translations, missed translations, and inconsistencies that are more likely to occur in non-autoregressive translation, and is conducive to obtaining higher quality translation results.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine translation, and in particular relates to a knowledge-enhanced non-autoregressive neural machine translation method and device. Background Art

[0002] Neural machine translation uses an autoregressive decoding method to generate the target language from left to right. This feature makes it impossible to generate words in different positions in parallel during the decoding process. Non-autoregressive translation abandons the sequential nature of the target language generation process. Since its decoding process does not depend on the previous translation results, it achieves a high reasoning speed. During the decoding process, all target language words can be generated simultaneously, greatly speeding up the model's decoding speed.

[0003] The current non-autoregressive translation method generates target language vocabulary at all times simultaneously. Although it greatly improves the decoding speed, it abandons the dependency between vocabulary, which can easily lead to inconsistent translation content, missing translation content, or repeated translation of the same content. The translation quality is poor and cannot meet normal high-quality translation needs. Summary of the invention

[0004] To this end, the present invention provides a knowledge-enhanced non-autoregressive neural machine translation method and device to solve the problem that target language text generation cannot rely on context information in non-autoregressive neural machine translation, and is prone to repeated translation, missing translation and inconsistency in translation.

[0005] In order to achieve the above object, the present invention provides the following technical solution: a knowledge-enhanced non-autoregressive neural machine translation method, comprising the following steps:

[0006] (1) Preprocess the data and encode the word vectors of the bilingual parallel language pairs;

[0007] (2) inputting the word vector representation of the source language into the encoder network, and the encoder network encodes the source language document information to obtain the encoded representation of the input word sequence information;

[0008] (3) Use the word alignment model to build the correspondence between the source language and the target language and construct a reproduction rate model;

[0009] (4) construct the input and output encoding representations of the decoder model;

[0010] (5) The dependency between target language words is established through the conditional random field model, and the final translation result is generated by decoding them one by one.

[0011] As a preferred solution of the knowledge-enhanced non-autoregressive neural machine translation method, step (1) includes:

[0012] (11) Use the BPE algorithm to segment all sentences in the training corpus into subwords;

[0013] (12) predefine a subword sequence representing the source language, and use a pre-trained model to obtain a word vector encoding representation of the source language;

[0014] (13) Obtaining the position vector encoding of the source language input sequence;

[0015] (14) Add the word vector code and the position vector code to obtain an input code representation of the source language.

[0016] As a preferred solution of the knowledge-enhanced non-autoregressive neural machine translation method, step (2) includes:

[0017] (21) Obtaining a word sequence matrix of the source language after word vector preprocessing;

[0018] (22) Use the Transformer layer based on the self-attention mechanism to obtain the top-level encoded representation of each word after passing through the encoder network.

[0019] As a preferred solution of the knowledge-enhanced non-autoregressive neural machine translation method, step (3) includes:

[0020] (31) predefine word sequences representing different target languages, and use a word alignment model to construct a correspondence between the source language word sequence and the target language;

[0021] (32) According to the correspondence between the source language word sequence and the target language, the number of tokens in the source language corresponding to the target language is used as the reproduction rate sequence;

[0022] (33) Calculate the softmax of the top-level encoding representation of each word obtained in step (22) through the encoder network to obtain the probability distribution of the reproduction rate;

[0023] (34) Select the output corresponding to the maximum probability as the generated result of the encoder network;

[0024] Step (33) comprises:

[0025] (331) Perform a linear transformation on the output hidden state;

[0026] (332) The result of one-layer linear transformation is output through softmax to obtain the probability distribution of the reproduction rate.

[0027] As a preferred solution of the knowledge-enhanced non-autoregressive neural machine translation method, step (4) includes:

[0028] (41) constructing the input of the decoder according to the reproduction rate result obtained in step (33);

[0029] (42) Obtain decoder input encoding representation;

[0030] (43) Obtain the output encoding representation of the decoder.

[0031] As a preferred solution of the knowledge-enhanced non-autoregressive neural machine translation method, step (5) includes:

[0032] (51) Perform a linear transformation on the hidden state output by the top layer of the decoder network;

[0033] (52) The result obtained by the linear transformation of one layer is output through the CRF linear chain to output the output probability distribution at each moment;

[0034] (53) Select the word corresponding to the maximum probability as the translation result at the specified time.

[0035] The present invention also provides a knowledge-enhanced non-autoregressive neural machine translation device, comprising:

[0036] The first processing module is used to perform data preprocessing and word vector encoding on the bilingual parallel language pairs;

[0037] A second processing module is used to input the word vector representation of the source language into an encoder network, and the encoder network encodes the source language document information to obtain an encoded representation of the input word sequence information;

[0038] The third processing module is used to construct a correspondence between the source language and the target language using a word alignment model and to construct a reproduction rate model;

[0039] a fourth processing module for constructing input and output encoding representations of a decoder model;

[0040] The fifth processing module is used to establish dependencies between target language words through a conditional random field model, and decode them in sequence to generate a final translation result.

[0041] As a preferred solution of the knowledge-enhanced non-autoregressive neural machine translation device, the first processing module includes:

[0042] The sub-word segmentation submodule is used to segment all sentences in the training corpus into sub-words using the BPE algorithm;

[0043] The first acquisition submodule is used to predefine a subword sequence representing the source language and obtain a word vector encoding representation of the source language using a pre-trained model;

[0044] The second acquisition submodule is used to obtain the position vector encoding of the source language input sequence;

[0045] The input encoding representation submodule is used to add the word vector encoding and the position vector encoding to obtain the input encoding representation of the source language.

[0046] As a preferred solution of the knowledge-enhanced non-autoregressive neural machine translation device, the second processing module includes:

[0047] The word sequence matrix submodule is used to obtain the word sequence matrix of the source language after word vector preprocessing;

[0048] The top-level encoding representation submodule is used to obtain the top-level encoding representation of each word after passing through the encoder network using the Transformer layer based on the self-attention mechanism.

[0049] As a preferred solution of the knowledge-enhanced non-autoregressive neural machine translation device, the third processing module includes:

[0050] The correspondence building submodule is used to predefine word sequences representing different target languages ​​and use the word alignment model to build the correspondence between the source language word sequence and the target language;

[0051] The reproduction rate sequence submodule is used to take the number of tokens of the source language corresponding to the target language as the reproduction rate sequence according to the correspondence between the source language word sequence and the target language;

[0052] The reproduction rate probability distribution submodule is used to calculate the softmax of the top-level encoding representation of each word obtained through the encoder network to obtain the probability distribution of the reproduction rate;

[0053] The generation result submodule is used to select the output corresponding to the maximum probability as the generation result of the encoder network;

[0054] In the reproduction rate probability distribution submodule, the output hidden state is subjected to a linear transformation; the result obtained by the linear transformation is output through softmax to obtain the probability distribution of the reproduction rate.

[0055] As a preferred solution of the knowledge-enhanced non-autoregressive neural machine translation device, the fourth processing module includes:

[0056] An input construction submodule, used for constructing the input of the decoder end according to the reproduction rate result obtained by the reproduction rate probability distribution submodule;

[0057] An input encoding representation submodule, used to obtain a decoder input encoding representation;

[0058] The output encoding representation submodule is used to obtain the output encoding representation of the decoder.

[0059] As a preferred solution of the knowledge-enhanced non-autoregressive neural machine translation device, the fifth processing module includes:

[0060] Hidden state linear transformation submodule, used to perform a linear transformation on the hidden state output by the top layer of the decoder network;

[0061] The output probability distribution submodule is used to output the output probability distribution at each moment through the CRF linear chain from the result of one layer of linear transformation;

[0062] The translation result submodule is used to select the word corresponding to the maximum probability as the translation result at a specified time.

[0063] The present invention has the following advantages: data preprocessing and word vector encoding of bilingual parallel language pairs; inputting the word vector representation of the source language into the encoder network, the encoder network encodes the source language document information to obtain the encoded representation of the input word sequence information; using the word alignment model to construct the correspondence between the source language and the target language, and constructing the reproduction rate model; constructing the input and output encoded representations of the decoder model; establishing the dependency between the target language vocabulary through the conditional random field model, and decoding in sequence to generate the final translation result. The present invention utilizes the knowledge of the pre-trained language model and uses the conditional random field for decoding at the decoding end; the pre-trained language model itself carries strong context information, and the conditional random field constructs the context dependency. The present invention alleviates the phenomenon of a large number of re-translations, missed translations and inconsistencies that are more likely to occur in non-autoregressive translation, which is conducive to obtaining higher quality translation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0065] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0066] Figure 1 A schematic diagram of a flow chart of a knowledge-enhanced non-autoregressive neural machine translation method provided in Example 1 of the present invention;

[0067] Figure 2Schematic diagram of the knowledge-enhanced non-autoregressive neural machine translation device provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0068] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] Example 1

[0070] See also Figure 1 Embodiment 1 of the present invention provides a knowledge-enhanced non-autoregressive neural machine translation method, comprising the following steps:

[0071] S1, preprocessing the data and encoding the word vectors of the bilingual parallel language pairs;

[0072] S2, inputting the word vector representation of the source language into the encoder network, the encoder network encodes the source language document information to obtain the encoded representation of the input word sequence information;

[0073] S3, using the word alignment model to build the correspondence between the source language and the target language, and constructing the reproduction rate model;

[0074] S4, construct the input and output encoding representations of the decoder model;

[0075] S5. Establish the dependencies between target language words through the conditional random field model, and decode them in sequence to generate the final translation result.

[0076] In this embodiment, step S1 includes:

[0077] S11. Use the BPE algorithm to segment all sentences in the training corpus into subwords;

[0078] S12, predefine a sub-word sequence representing the source language, and use the pre-trained model to obtain a word vector encoding representation of the source language;

[0079] S13, obtaining a position vector encoding of a source language input sequence;

[0080] S14. Add the word vector code and the position vector code to obtain an input code representation of the source language.

[0081] Specifically, in step S11, in order to reduce the impact of out-of-words on translation performance, the BPE algorithm is first used to segment all sentences in the training corpus into sub-words, so that the input units of the encoder network and the output units of the decoder network are both sub-word sequences.

[0082] In step S12, first predefine x=[x1,…,x n ] represents the sub-word sequence of the source language, and the word vector encoding of the source language is obtained using the pre-trained model to represent TE x :

[0083] TE x =Bert emb (x)=[v1,…,v n ]

[0084] In step S13, the position vector encoding PE of the source language input sequence is obtained. x ;

[0085] In step S14, the word vector is encoded TE x PE with position vector encoding x Add together to get the input encoding vector representation E of the source language x :E x =TE x +PE x .

[0086] In this embodiment, step S2 includes:

[0087] S21, obtaining a word sequence matrix of the source language after word vector preprocessing;

[0088] S22. Use the Transformer layer based on the self-attention mechanism to obtain the top-level encoded representation of each word after passing through the encoder network.

[0089] Specifically, in step S21, use E x =[v1,…,v n ] represents the matrix obtained by preprocessing the input word sequence with word vectors, where v i Represents the vector of the i-th subword.

[0090] In step S22, a Transformer layer based on a self-attention mechanism is used, and the encoded representation of each word after the encoder is calculated by the following formula:

[0091] V n =SelfAttn(E x ,E x ,E x )

[0092]

[0093] Among them, E x represents the input encoding vector, V n Represents the output of the self-attention mechanism. Using the encoder, we can get the top-level encoding representation

[0094] In this embodiment, step S3 includes:

[0095] S31, pre-define word sequences representing different target languages, and use a word alignment model to build a correspondence between the source language word sequence and the target language;

[0096] S32, according to the correspondence between the source language word sequence and the target language, the number of tokens of the source language corresponding to the target language is used as a reproduction rate sequence;

[0097] S33, calculating the softmax of the top-level encoding representation of each word obtained in step S22 through the encoder network to obtain the probability distribution of the reproduction rate;

[0098] S34. Select the output corresponding to the maximum probability as the generation result of the encoder network.

[0099] Specifically, in step S31, define y=[y1,…,y n ] represents word sequences in different target languages, and the word alignment model is used to construct the corresponding relationship Map between the source language word sequence x and the target language y.

[0100] In step S32, through the correspondence Map, the number of tokens of the source language corresponding to the target language is used as the reproduction rate sequence F:

[0101] F=[f1,…,f n ]

[0102] f i Limited to natural numbers between 0 and 50.

[0103] In step S33, the top-level encoding representation of the encoder obtained in step S22 is Calculate softmax to get the probability distribution of the reproduction rate. n Perform a linear transformation: The result of linear transformation of one layer The probability distribution of the reproduction rate is output through softmax: Among them, W and b are the training parameters of the model, and the dimension of W is 51.

[0104] In this embodiment, step S4 includes:

[0105] S41, constructing the input of the decoder end according to the reproduction rate result obtained in step S33;

[0106] S42, obtaining a decoder input encoding representation;

[0107] S43, obtaining the output encoding representation of the decoder.

[0108] Specifically, in step S41, according to the reproduction rate result obtained in step 3, the input of the decoder is constructed: y = F x = f i x i ; For example, the source language x i The corresponding reproduction rate f i =3, then y i =[x i ,x i ,x i ].

[0109] In step S42, similar to steps S12 to S14, a decoder input code representation is obtained; in step S43, similar to steps S21 to S22, a decoder output code representation is obtained.

[0110] In this embodiment, step S5 includes:

[0111] S51, perform a linear transformation on the hidden state outputted by the top layer of the decoder network;

[0112] S52, the result obtained by the linear transformation of one layer is output through the CRF linear chain to output the output probability distribution at each moment;

[0113] S53: Select the word corresponding to the maximum probability as the translation result at the specified time.

[0114] Specifically, in step S51, the hidden state output by the top layer of the encoder network is After a layer of linear transformation, it looks like this:

[0115] In step S52, the linear transformation is performed to obtain Output the output probability distribution Prob at each moment through the CRF linear chain y|x :

[0116]

[0117] Among them, s represents the predicted target word y i The score (unnormalized softmax), t represents the transfer probability between words, and z(x) represents the normalization factor.

[0118] In step S53, the word corresponding to the maximum probability is selected as the translation result at time i: y = Max(Prob y|x ); According to the above steps, decode in sequence to generate the final translation result y = [y1, ..., y n ].

[0119] In summary, the present invention uses the BPE algorithm to perform sub-word segmentation on sentences in all training corpora; predefines a sub-word sequence representing the source language, and uses a pre-trained model to obtain a word vector encoding representation of the source language; obtains a position vector encoding of a source language input sequence; adds the word vector encoding to the position vector encoding to obtain an input encoding representation of the source language; uses a word alignment model to construct a correspondence between the source language and the target language, constructs a reproduction rate model, predefines word sequences representing different target languages, and uses a word alignment model to construct a correspondence between the source language word sequence and the target language; according to the correspondence between the source language word sequence and the target language, the number of tokens of the source language corresponding to the target language is used as a reproduction rate sequence; calculates softmax for each word obtained through the top-level encoding representation of the encoder network to obtain a probability distribution of the reproduction rate; selects the output corresponding to the maximum probability as the generation result of the encoder network; constructs the input and output encoding representations of the decoder model; performs a layer of linear transformation on the hidden state of the top-level output of the decoder network; outputs the output probability distribution of each moment through a CRF linear chain using the result obtained by a layer of linear transformation; and selects the word corresponding to the maximum probability as the translation result at a specified moment. The encoder of the present invention utilizes a pre-trained model to encode input text into a semantic vector with context information, and the decoder adds a conditional random field at the top layer of the decoding end, thereby establishing a contextual temporal dependency relationship; the present invention utilizes the knowledge of the pre-trained language model and uses the conditional random field for decoding at the decoding end; the pre-trained language model itself carries strong context information, and the conditional random field constructs a contextual dependency relationship. The present invention alleviates the phenomenon of a large number of repeated translations, missed translations and inconsistencies that are more likely to occur in non-autoregressive translation, and is conducive to obtaining higher quality translation results.

[0120] Example 2

[0121] See also Figure 2 Embodiment 2 of the present invention further provides a knowledge-enhanced non-autoregressive neural machine translation device, comprising:

[0122] The first processing module 1 is used for data preprocessing and word vector encoding of the bilingual parallel language pairs;

[0123] The second processing module 2 is used to input the word vector representation of the source language into the encoder network, and the encoder network encodes the source language document information to obtain the encoded representation of the input word sequence information;

[0124] The third processing module 3 is used to construct a correspondence between the source language and the target language using a word alignment model, and to construct a reproduction rate model;

[0125] A fourth processing module 4, for constructing input and output encoding representations of a decoder model;

[0126] The fifth processing module 5 is used to establish dependencies between target language words through a conditional random field model, and decode them in sequence to generate a final translation result.

[0127] In this embodiment, the first processing module 1 includes:

[0128] A sub-word segmentation submodule 11 is used to perform sub-word segmentation on all sentences in the training corpus using a BPE algorithm;

[0129] A first acquisition submodule 12 is used to predefine a subword sequence representing a source language and obtain a word vector encoding representation of the source language using a pre-trained model;

[0130] The second acquisition submodule 13 is used to acquire the position vector encoding of the source language input sequence;

[0131] The input encoding representation submodule 14 is used to add the word vector encoding and the position vector encoding to obtain the input encoding representation of the source language.

[0132] In this embodiment, the second processing module 2 includes:

[0133] The word sequence matrix submodule 21 is used to obtain the word sequence matrix of the source language after word vector preprocessing;

[0134] The top-level encoding representation submodule 22 is used to obtain the top-level encoding representation of each word after passing through the encoder network using the Transformer layer based on the self-attention mechanism.

[0135] In this embodiment, the third processing module 3 includes:

[0136] A correspondence building submodule 31 is used to predefine word sequences representing different target languages ​​and use a word alignment model to build a correspondence between a source language word sequence and a target language;

[0137] The reproduction rate sequence submodule 32 is used to use the number of tokens of the source language corresponding to the target language as the reproduction rate sequence according to the correspondence between the source language word sequence and the target language;

[0138] The reproduction rate probability distribution submodule 33 is used to calculate the softmax of the top-level encoding representation of each word obtained through the encoder network to obtain the probability distribution of the reproduction rate;

[0139] A generation result submodule 34 is used to select the output corresponding to the maximum probability as the generation result of the encoder network;

[0140] In the reproduction rate probability distribution submodule 33, the output hidden state is subjected to a linear transformation; the result obtained by the linear transformation is output as the probability distribution of the reproduction rate through softmax.

[0141] In this embodiment, the fourth processing module 4 includes:

[0142] An input construction submodule 41, used to construct an input of a decoder according to the reproduction rate result obtained by the reproduction rate probability distribution submodule;

[0143] An input code representation submodule 42, for obtaining a decoder input code representation;

[0144] The output coding representation submodule 43 is used to obtain the output coding representation of the decoder.

[0145] In this embodiment, the fifth processing module 5 includes:

[0146] Hidden state linear transformation submodule 51, used to perform a linear transformation on the hidden state outputted by the top layer of the decoder network;

[0147] The output probability distribution submodule 52 is used to output the output probability distribution at each moment through the CRF linear chain by transforming the result obtained by one layer of linear transformation;

[0148] The translation result submodule 53 is used to select the word corresponding to the maximum probability as the translation result at a specified time.

[0149] It should be noted that the information interaction, execution process, etc. between the modules / sub-modules of the above-mentioned device are based on the same concept as the method embodiment in Example 1 of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown in the previous part of the present application, and will not be repeated here.

[0150] Example 3

[0151] Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium, in which a program code of a knowledge-enhanced non-autoregressive neural machine translation method is stored, and the program code includes instructions for executing the knowledge-enhanced non-autoregressive neural machine translation method of embodiment 1 or any possible implementation thereof.

[0152] The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0153] Example 4

[0154] Embodiment 4 of the present invention provides an electronic device, including: a memory and a processor;

[0155] The processor and the memory communicate with each other via a bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the knowledge-enhanced non-autoregressive neural machine translation method of Example 1 or any possible implementation thereof.

[0156] Specifically, the processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor implemented by reading software codes stored in a memory. The memory can be integrated in the processor or can be located outside the processor and exist independently.

[0157] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0158] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0159] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A knowledge-enhanced non-autoregressive neural machine translation method, characterized in that: The following steps are involved: (1) Preprocess the data and encode the word vectors of the bilingual parallel language pairs; (2) inputting the word vector representation of the source language into the encoder network, and the encoder network encodes the source language document information to obtain the encoded representation of the input word sequence information; (3) Use the word alignment model to build the correspondence between the source language and the target language and construct a reproduction rate model; (4) construct the input and output encoding representations of the decoder model; (5) Establishing the dependencies between target language words through the conditional random field model, and decoding them sequentially to generate the final translation result; Step (2) includes: (21) obtaining a word sequence matrix of the source language after word vector preprocessing; (22) Use the Transformer layer based on the self-attention mechanism to obtain the top-level encoded representation of each word after passing through the encoder network; Step (3) includes: (31) predefining word sequences representing different target languages, and using a word alignment model to construct a correspondence between the source language and the target language; (32) According to the correspondence between the source language and the target language, the number of tokens in the source language corresponding to the target language is used as the reproduction rate sequence; (33) Calculate the softmax of the top-level encoding representation of each word obtained in step (22) through the encoder network to obtain the probability distribution of the reproduction rate; (34) Select the output corresponding to the maximum probability as the generated result of the encoder network; Step (33) comprises: (331) Perform a linear transformation on the output hidden state; (332) The result obtained by the linear transformation is output through softmax to obtain the probability distribution of the reproduction rate; Step (4) includes: (41) constructing a decoder input according to the reproduction rate result obtained in step (34); (42) Obtain decoder input encoding representation; (43) Obtain the output encoding representation of the decoder.

2. A knowledge-enhanced non-autoregressive neural machine translation method according to claim 1, characterized in that: Step (1) includes: (11) using the BPE algorithm to perform sub-word segmentation on all sentences in the training corpus; (12) predefine a subword sequence representing the source language, and use a pre-trained model to obtain a word vector encoding representation of the source language; (13) Obtaining the position vector encoding of the source language input sequence; (14) Add the word vector code and the position vector code to obtain an input code representation of the source language.

3. The knowledge-enhanced non-autoregressive neural machine translation method according to claim 1, characterized in that: Step (5) includes: (51) performing a linear transformation on the hidden state outputted by the top layer of the decoder network; (52) The result obtained by the linear transformation of one layer is output through the CRF linear chain to output the output probability distribution at each moment; (53) Select the word corresponding to the maximum probability as the translation result at the specified time.

4. A knowledge-enhanced non-autoregressive neural machine translation device, characterized in that: include: The first processing module is used to perform data preprocessing and word vector encoding on the bilingual parallel language pairs; A second processing module is used to input the word vector representation of the source language into an encoder network, and the encoder network encodes the source language document information to obtain an encoded representation of the input word sequence information; The third processing module is used to construct a correspondence between the source language and the target language using a word alignment model and to construct a reproduction rate model; a fourth processing module for constructing input and output encoding representations of a decoder model; The fifth processing module is used to establish dependencies between target language words through a conditional random field model, and decode them in sequence to generate a final translation result; The second processing module comprises: The word sequence matrix submodule is used to obtain the word sequence matrix of the source language after word vector preprocessing; The top-level encoding representation submodule is used to obtain the top-level encoding representation of each word after passing through the encoder network using the Transformer layer based on the self-attention mechanism; The third processing module includes: a correspondence construction submodule, which is used to predefine word sequences representing different target languages ​​and use a word alignment model to construct a correspondence between a source language word sequence and a target language; The reproduction rate sequence submodule is used to take the number of tokens of the source language corresponding to the target language as the reproduction rate sequence according to the correspondence between the source language word sequence and the target language; The reproduction rate probability distribution submodule is used to calculate the softmax of the top-level encoding representation of each word obtained through the encoder network to obtain the probability distribution of the reproduction rate; The generation result submodule is used to select the output corresponding to the maximum probability as the generation result of the encoder network; In the reproduction rate probability distribution submodule, the output hidden state is subjected to a linear transformation; the result obtained by the linear transformation is output through softmax to obtain the probability distribution of the reproduction rate; The fourth processing module includes: an input construction submodule, which is used to construct an input of a decoder end according to the reproduction rate result obtained by the reproduction rate probability distribution submodule; An input encoding representation submodule, used to obtain a decoder input encoding representation; The output encoding representation submodule is used to obtain the output encoding representation of the decoder.

5. A knowledge-enhanced non-autoregressive neural machine translation device according to claim 4, characterized in that: The first processing module includes: a sub-word segmentation sub-module, which is used to perform sub-word segmentation on all sentences in the training corpus using the BPE algorithm; The first acquisition submodule is used to predefine a subword sequence representing the source language and obtain a word vector encoding representation of the source language using a pre-trained model; The second acquisition submodule is used to obtain the position vector encoding of the source language input sequence; The input encoding representation submodule is used to add the word vector encoding and the position vector encoding to obtain the input encoding representation of the source language.

6. A knowledge-enhanced non-autoregressive neural machine translation device according to claim 5, characterized in that: The fifth processing module comprises: Hidden state linear transformation submodule, used to perform a linear transformation on the hidden state output by the top layer of the decoder network; The output probability distribution submodule is used to output the output probability distribution at each moment through the CRF linear chain from the result of one layer of linear transformation; The translation result submodule is used to select the word corresponding to the maximum probability as the translation result at a specified time.

Images