A machine translation method and apparatus

Abstract

The application discloses a machine translation method and device, the method comprises the following steps: receiving a source text in a source language; inputting the source text into an encoder, and generating an encoding output of the encoder by using hidden representations output by each sub-encoder in the encoder, wherein each sub-encoder comprises at least one neural network layer; and performing decoding on the encoding output by using a decoder to obtain a target text in a target language. The application adopts the mode of splitting the encoder and using multiple sub-encoders to jointly generate the encoding output, so that the encoding output can represent more text features of the source text, that is, the encoding output comprising low-level semantic information and high-level syntax information is provided to the decoder for processing, thereby more accurately translating the source text.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a machine translation method and device thereof. BACKGROUND

[0002] In the information age, computer technology is developing rapidly, and more and more human labor engaged in repetitive work can be replaced by computers, and the liberated human labor can engage in more innovative and challenging work. The rise of artificial intelligence under the background of deep learning in recent years has further accelerated the realization of this process.

[0003] Machine translation, also known as automatic translation, is a process of converting one natural language (source language) into another natural language (target language) using a computer. In practice, it is found that the deeper the depth (number of layers) of the machine translation model, the more feature information is extracted, but correspondingly, a lot of bottom layer information is lost and cannot be trained due to gradient decay.

[0004] In summary, the current machine translation method still has deficiencies. SUMMARY

[0005] Embodiments of the present application provide a machine translation method and device, at least to solve the above technical problems.

[0006] Embodiments of the present application also provide a machine translation method, the method comprising: receiving a source text in a source language; inputting the source text into an encoder, and generating an encoding output of the encoder using hidden representations output by each sub-encoder in the encoder, wherein each sub-encoder comprises at least one neural network layer; and performing decoding on the encoding output using a decoder to obtain a target text in a target language.

[0007] Embodiments of the present application also provide a machine translation device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs comprise instructions for executing the above method.

[0008] Embodiments of the present application also provide a computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by a computing device, cause the computing device to perform any of the above methods.

[0009] The above at least one technical solution adopted by the embodiments of the present application can achieve the following beneficial effects:

[0010] The application divides the encoder and uses multiple sub-encoders to jointly generate the encoding output, so that the encoding output can represent more text features of the source text, that is, the encoding output including low-level semantic information and high-level syntax information is provided to the decoder for processing, so that the source text can be more accurately translated. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0012] Figure 1 is a schematic diagram of a scene showing a machine translation system according to an example embodiment of the application;

[0013] Figure 2 is a structural diagram of a machine translation model according to an example embodiment of the application;

[0014] Figure 3 is a flowchart of a machine translation method according to an example embodiment of the application;

[0015] Figure 4 is a diagram showing the association of an encoder and a decoder according to an example embodiment of the application;

[0016] Figure 5 is a diagram showing the association of an encoder and a decoder according to an example embodiment of the application;

[0017] Figure 6 is a block diagram of a machine translation device according to an example embodiment of the application. DETAILED DESCRIPTION

[0018] To make the purposes, technical solutions and advantages of the application clearer, the technical solutions of the application will be described below in conjunction with the specific embodiments of the application and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the application.

[0019] The technical solutions provided by the embodiments of the application will be described in detail below with reference to the accompanying drawings.

[0020] Figure 1is a scene schematic diagram of a machine translation system according to an exemplary embodiment of the present application, which can include electronic terminals 11 used by users and servers 12 providing services for the electronic terminals, the electronic terminals 11 being connected with the servers 12 through the Internet or the like formed by various gateways, which will not be described herein. The number of the electronic terminals 11 and the servers 12 is not limited. Figure 1

[0021] The electronic terminals 11 include, but are not limited to, portable terminals such as mobile phones and tablets provided with instant messaging applications, and fixed terminals such as computers, inquiry machines and advertising machines, which are service ports available to users, hereinafter referred to as clients, and can be, for example, mobile phones downloaded with chat applications; in the present application, the clients provide input functions of sentences to be translated, display functions of translation results and the like;

[0022] The servers 12 provide various services for instant messaging users, provide application layer support, and include data servers, training servers, translation servers and storage servers and the like; the data servers are used for pre-processing data and the like, the training servers are used for training machine models, and the translation servers are used for translating according to machine models and the like.

[0023] In the embodiments of the present application, the electronic terminals 11 can determine source languages and target translation languages to be translated by using translation applications thereon, input sentences to be translated and send the sentences to the servers 12. The servers 12 call machine translation models trained to translate the sentences to be translated into target sentences in target translation languages after receiving the sentences to be translated.

[0024] Although the above only gives the translation by the translation application using the machine translation method of the present application, the use scenarios of the machine translation method of the present application are not limited thereto, for example, chat content can be translated into target sentences according to user instructions in a chat application, or shopping information can be translated into target sentences in a shopping application, and for another example, lyrics / subtitles can be translated into target sentences in a video and audio application.

[0025] It should be noted that, Figure 1 The system scene schematic diagram shown is only an example, the servers and scenarios described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application, and those skilled in the art can know that, with the evolution of the system and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0026] The following will be described in detail in conjunction with Figure 2 , wherein, Figure 2 a structural diagram of a machine translation model according to an exemplary embodiment of the present application is shown.​

[0027] The present application will employ a neural machine translation (NMT) model as shown in Figure 2 The NMT model completely employs a neural network to complete the translation process from a source language to a target language.

[0028] As shown in Figure 2 The NMT model can include an encoder and a decoder. Before inputting a source sentence to the encoder, the source sentence can be preprocessed, which can include word segmentation processing and word vector processing.

[0029] Word segmentation processing refers to converting a source sentence into individual words based on statistical word segmentation, where the statistical word segmentation comes from standard corpora, which are usually divided by domain, for example, there is a medical corpus for the medical field and an aviation corpus for the aviation field. For example, for a source language of Chinese, the source sentence "Beijing welcomes you" can be segmented into "Beijing", "welcomes", and "you"; for a source language of English, the source sentence "Welcome to Beijing" can be segmented into "Welcome", "to", "Bei", and "jing".

[0030] Subsequently, word vector processing can be performed on each word, that is, a word vector corresponding to each word is obtained. In the process of obtaining the word vector corresponding to each word, word embedding processing can be performed on each word to obtain a word vector corresponding to each word. Word embedding processing refers to obtaining a dense vector representation of each element using context content through a trained neural network semantic model. For example, "struggle" can be represented by the vector (0.1, 0.2, 0.3), and "don't use" can be represented by the vector (0.7, 0.3, 0.3). Since word embedding is a common technique in the art, it will not be described here, but those skilled in the art will understand that all methods that can represent words using vectors can be applied to the present application.

[0031] As shown in Figure 2 The source sentence is converted to x after preprocessing, where x = (X1, X2, X3, X4, X5). The x is input to the encoder, which converts the source sentence into a hidden information representation, and the decoder reads the hidden information and generates the target sentence y = (Y1, Y2, Y3, Y4, Y5).

[0032] In implementation, more semantic information in the source sentence can be extracted by increasing the number of neural network layers included in the encoder. For example, the bottom neural network layer can extract semantic position information and semantic detail information, while the high-level neural network layer can extract syntactic information. However, if the number of encoder layers increases, the decoder may not be able to obtain the bottom-level information extracted by the encoder, resulting in inaccurate text.

[0033] In addition, during the training process of the NMT model, a backpropagation algorithm is used. That is, the target text generated by the NMT model is matched with the human translation of the source text. At this time, the error signal during backpropagation needs to pass through the entire encoder. However, the increase in the depth of the encoder will bring about unstable gradient flow. Therefore, the difficulty of building a deep encoder lies in the inability to train it.

[0034] Based on this, the machine translation method of the present application performs a division process on the multi-layer neural network layer included in the encoder, and then uses the hidden representation of the sub-encoder including at least one layer of neural network layer as the output, and acts on the decoder together, and finally uses the decoder to perform decoding processing to obtain the target text in the target language. Figure 3 Provide a detailed description.

[0035] Figure 3 A flowchart of a machine translation method according to an exemplary embodiment of the present application is shown.

[0036] In step S310, a source text in a source language is received. For example, before utilizing the machine translation method of the present application, the source language and target language may be determined. For example, Chinese may be selected as the source language and English as the target language. The source text may be words or sentences input by the user. For example, the user may simply input words such as "goodbye" or "hello," or may input a sentence such as "where did you go today?"

[0037] In step S320, the source text is input into an encoder, and the encoded output of the encoder is generated using the hidden representations output by each sub-encoder in the encoder, wherein each sub-encoder includes at least one neural network layer.

[0038] In this application, the encoder may include multiple neural network layers, that is, the encoder is actually a deep neural network model. The layer structures of these neural network layers can be the same or different, and these neural network layers are connected according to a predetermined structure to form the encoder.

[0039] Based on this, the encoder can be divided into multiple sub-encoders according to the number of neural network layers. As an example, the division manner and the number of sub-encoders can be set according to the number of layers, wherein the division manner includes equal layer division, arithmetic layer division, and artificial division.

[0040] The equal layer division refers to a division manner in which the number of neural network layers in each sub-encoder is the same. For example, when the encoder includes 15 layers of neural network layers, the encoder can be divided into 3 sub-encoders, each including 5 layers of neural network layers, or divided into 5 sub-encoders, each including 3 layers of neural network layers.

[0041] The arithmetic layer division refers to a division manner in which the number of neural network layers in each sub-encoder is divided in an arithmetic sequence. For example, when the encoder includes 9 layers of neural network layers, the encoder can be divided into 3 sub-encoders, wherein the first encoder includes 2 layers of neural network layers, the second encoder includes 3 layers of neural network layers, and the third encoder includes 4 layers of neural network layers, or the first encoder includes 4 layers of neural network layers, the second encoder includes 3 layers of neural network layers, and the third encoder includes 2 layers of neural network layers.

[0042] The artificial division refers to a division manner in which a user (e.g., a technician) artificially divides the encoder according to the number of neural network layers included in the encoder. For example, when the encoder is 17 layers of neural network layers, it can be divided into three sub-encoders according to the user's indication, wherein the first sub-encoder includes 6 layers of neural network layers, the second sub-encoder includes 6 layers of neural network layers, and the third sub-encoder includes 5 layers of neural network layers.

[0043] Subsequently, the input word segmentation vector is processed using the multiple sub-encoders to obtain the hidden representation output by each sub-encoder. Specifically, since the sub-encoders are sequentially connected, for each sub-encoder, the input word segmentation vector input from the current sub-encoder and the output information of the previous encoder are taken as input information to perform processing, thereby obtaining multiple hidden representations corresponding to the multiple sub-encoders respectively.

[0044] Using these hidden representations, the encoding output of the encoder is generated. As an example, the encoding output of the encoder can be generated by determining the weight of each hidden representation, including: using the attention mechanism to determine the weight of each hidden representation, that is, using the decoded output of the current decoder as the query of the attention model (i.e., Attention Function), and using the hidden representation of each sub-encoder as the key of the attention model, and obtaining the weight of each sub-encoder by performing a similarity calculation. Optionally, these weights can be normalized, for example, using a softmax function to normalize these weights. Finally, the encoding output is determined based on each hidden representation and the corresponding weight.

[0045] As another example, since these sub-encoders are connected sequentially, the encoding outputs of these sub-encoders have temporal information. Based on this, the hidden representation output by each sub-encoder can be input as a single sequence data to the neural network layer for sequence data, and the output sequence data is used as the encoding output of the encoder. The neural network layer for sequence data includes a recurrent neural network (RNN) and its variants, such as a gated recurrent unit (GRU) model, a bidirectional gated recurrent unit (BiGRU) or a long short-term memory (LSTM) model. Since these models are all well-known models, they will not be described in detail here.

[0046] In step S330, a decoder is used to decode the encoded output to obtain a target text in a target language.

[0047] In the present application, the decoder may include at least one neural network layer, that is, there is no limit on the number of neural network layers of the decoder. Preferably, the decoder may include the same number of neural network layers as the encoder. In implementation, the encoded output is input to each sub-decoder in the decoder, and the output of the previous sub-decoder and the encoded output of the corresponding encoder are used as the input of the current sub-decoder, so that each sub-decoder performs processing in sequence, and the output of the last sub-decoder is used as the target text.

[0048] In the present application, after receiving the source text, the source text may be input into a trained machine translation model to obtain a target text in a target language, wherein the machine translation model includes an encoder and a decoder set in the above manner.

[0049] Therefore, when executingFigure 3 Before the method is shown, the machine translation model can be trained using a training set. The training set can include a large number of training samples, wherein each training sample includes training source text and corresponding training target text.

[0050] Then each training source text is input into the machine translation model to obtain the target text. According to the target text and the training target text, the full network parameters in the machine translation model are adjusted using a back propagation algorithm until the machine translation model meets a preset requirement, for example, the probability of the target text is maximum, thereby completing the training of the machine translation model using the training set.

[0051] During the training, since the hidden representations of each sub-encoder act on the corresponding decoder, the gradient back propagation path during the training is greatly reduced, thereby solving the unstable gradient flow caused by the increase in the depth of the neural network layer included in the encoder.

[0052] Compared with the prior art, the encoder is divided and the multiple sub-encoders are used to jointly generate the encoding output, so that the encoding output can represent more text features of the source text, that is, the encoding output including low-level semantic information and high-level syntax information is provided to the decoder for processing, thereby more accurately translating the source text.

[0053] The correspondence between the encoder and the decoder will be described below Figure 4 and Figure 5 The correspondence between the encoder and the decoder will be described below Figure 4 and Figure 5 both show a diagram of the association between the encoder and the decoder according to an example embodiment of the present application. It should be noted that Figure 4 and Figure 5 The number of sub-encoders and sub-decoders in

[0054] As shown in Figure 4 , the encoder is divided into five sub-encoders, after the second token X2 is input into the encoder, the first sub-encoder encodes the second token X2 to obtain the first hidden representation, and the first hidden representation is input into the second sub-encoder to obtain the second sub-encoding output, and so on, thereby obtaining the first to fifth hidden representations, then the attention mechanism can be used to determine the weight of each hidden output, as an example, the decoding output Y1 of the decoder for the previous token X2 can be used as the query of the five sub-encoders, and the hidden representations of the five sub-encoders are used as the key value to determine the weight of each sub-encoding output.

[0055] Preferably, the decoder can be as shown in Figure 4The shown is divided into the same number of sub-encoders as the number of sub-decoders. In this embodiment, each sub-decoder needs to utilize the output information of the previous sub-decoder and the encoding output of the encoder to perform decoding when decoding. Therefore, the output information output by the previous sub-decoder can be used as a query of the current hidden output, so as to determine the weight of each sub-encoder for the current sub-decoder. As shown in Figure 4 As shown, the output O2 of the second sub-encoder can be used as a query, the hidden outputs output by the first sub-encoder to the fifth sub-encoder can be used as key values, the weights of each hidden output are determined by using the attention mechanism, finally, the encoding output corresponding to the third sub-decoder is determined by using each hidden output and the corresponding weight, and then the third sub-decoder outputs O3 by using the encoding output and O2. The same is true for the subsequent sub-decoders, until the encoding result Y2 is output.

[0056] In summary, the machine translation method according to the example embodiment of the present application can better reflect the context information of each word by determining the weight value of the hidden representation output by each sub-encoder by using the attention mechanism, so that the translation result is more accurate.

[0057] In addition, for the association between the encoder and the decoder, the present application can also use the method as shown in Figure 5

[0058] As shown in Figure 5 The hidden representation output by each sub-encoder is sequentially output to the corresponding BiGRU as time series data. It should be noted that although Figure 5 BiGRU is shown in

[0059] Specifically, the first hidden representation C1 output by the first sub-encoder is input to the first BiGRU, and then the first BiGRU can generate an output L1 by using C1, and input the forward hidden layer state and the reverse hidden layer state to the second BiGRU, while the second BiGRU obtains the second hidden representation C2 output by the second sub-encoder. The second BiGRU generates an output L2 by using the forward hidden layer state, the reverse hidden layer state and the second hidden representation C2, and outputs the corresponding forward hidden layer state and reverse hidden layer state to the next BiGRU, and the subsequent BiGRU is executed in turn, until the last sub-encoder (the fifth sub-encoder in the figure) performs processing by using the corresponding BiGRU and outputs as the encoding output of the encoder, which is transmitted to the decoder.

[0060] ​In this process, the first sub-decoder can perform decoding using L1 of the first BiGRU output, then input the first decoding result to the second sub-decoder, the second sub-decoder performs decoding using L2 of the second BiGRU output and the first decoding result, and input the second decoding result to the next sub-decoder, and so on, until the last sub-decoder outputs the decoding result Y2.

[0061] Figure 5 It is shown in the middle that the number of sub-decoders is the same as that of sub-encoders. In implementation, if the number of sub-decoders is less than that of sub-encoders, the last sub-decoder can obtain the decoding result Y2 using the output of the corresponding BiGRU and the output of the previous sub-decoder.

[0062] In summary, according to the machine translation method of the exemplary embodiments of the present application, the hidden representation output by each sub-encoder can be input as each time series data to the corresponding RNN model or variant of the RNN model, so that the hidden representation of each sub-encoder can be mapped to the decoder end, and the corresponding gradient backpropagation path can be greatly reduced, thereby being free from the influence of gradient decay.

[0063] In implementation, the machine translation method according to the exemplary embodiments of the present application can be run by an electronic device installed with various application programs, which can include an instant messaging program for office use, an application program for shopping, etc.

[0064] For example, when the electronic device runs an instant messaging program for office use (e.g., DingTalk) and communicates with foreign colleagues, there can be difficulties. In response to a translation request for a source text issued by the user, the machine translation method according to the exemplary embodiments of the present application can be used to translate the source text and obtain and provide the target text to the user.

[0065] Similarly, when the electronic device runs an instant messaging program for shopping (e.g., Taobao or Lazada), the user can issue a translation request for a source text when facing the product introduction (e.g., an English description) of the source text, and then the application program uses the machine translation method according to the exemplary embodiments of the present application to translate the source text and obtain and provide the target text to the user.

[0066] In addition, the machine translation method according to the exemplary embodiments of the present application can be a method executed by an application server corresponding to each application, and can also be a method executed by a separate module coupled to the server, that is, the machine translation device according to the exemplary embodiments of the present application can be a module embedded in the application server or a module external to the application server, and the present application does not limit this.

[0067] Figure 6A block diagram of a machine translation device showing an exemplary embodiment of the present application. Reference is made to Figure 6 At the hardware level, the device includes a processor, an internal bus, and a computer readable storage medium, wherein the computer readable storage medium includes a volatile memory and a non-volatile memory. The processor reads the corresponding computer program from the non-volatile memory and then runs. Of course, in addition to the software implementation, the present application does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0068] Specifically, the processor performs the following operations: receiving a source text in a source language; inputting the source text into an encoder, and generating an encoding output of the encoder using hidden representations output by each sub-encoder in the encoder, wherein each sub-encoder includes at least one neural network layer; decoding the encoding output using a decoder to obtain a target text in a target language.

[0069] Optionally, the processor further includes, after receiving the source text in the source language in the implementation step: performing a word segmentation processing on the source text to obtain each input word of the source text; performing a word embedding processing on each input word to obtain an input word vector of each input word, so as to input the input word vector into the encoder.

[0070] Optionally, the processor generates the encoding output of the encoder using the hidden representations output by each sub-encoder in the implementation step of inputting the source text into the encoder includes: dividing the encoder into a plurality of sub-encoders according to the number of neural network layers; performing processing on the input word vector using the plurality of sub-encoders to obtain a hidden representation output by each sub-encoder; and generating the encoding output of the encoder using the hidden representation output by each sub-encoder.

[0071] Optionally, the processor obtains the hidden representation output by each sub-encoder by performing processing on the input word vector using the plurality of sub-encoders in the implementation step includes: for each sub-encoder, taking the input word vector input from the current sub-encoder and the output information of the previous encoder as input information to perform processing, thereby obtaining a plurality of hidden representations corresponding to the plurality of sub-encoders respectively.

[0072] Optionally, the processor generates the encoding output of the encoder using the hidden representation output by each sub-encoder in the implementation step includes: generating the encoding output of the encoder by determining the weight of each hidden representation.

[0073] Optionally, the processor, in the implementing step, generates the encoding output of the encoder by determining a weight corresponding to each hidden representation comprises: determining the weight of each hidden representation by using an attention mechanism; and determining the encoding output according to each hidden representation and the corresponding weight.

[0074] Optionally, the processor, in the implementing step, generates the encoding output of the encoder by using the hidden representations of each sub-encoder output comprises: inputting the hidden representations of each sub-encoder output as a single sequence data to a neural network layer for sequence data, and outputting the sequence data as the encoding output of the encoder.

[0075] Optionally, the neural network layer for sequence data comprises a recurrent neural network.

[0076] Optionally, the decoder comprises a plurality of sub-decoders, and the number of the plurality of sub-decoders is not constrained by the number of the sub-encoders.

[0077] Optionally, the processor, in the implementing step, obtains the target text in the target language by performing decoding on the encoding output by the decoder comprises: inputting the encoding output to each sub-decoder in the decoder, each sub-decoder performing decoding by using the output decoded by the previous sub-decoder and the corresponding encoding output and inputting the decoded output to the next sub-decoder, sequentially performing the processing, and taking the output of the last sub-decoder as the target text.

[0078] In addition, the processor performs the following operations: receiving a source text in a source language; and inputting the source text into the trained machine translation model to obtain a target text in a target language, wherein the machine translation model comprises an encoder and a decoder, the encoder comprises a plurality of sub-encoders, and an encoding output represented by a plurality of hidden representations output by the plurality of sub-encoders is used as an input of the decoder.

[0079] According to the machine translation device of the exemplary embodiments of the present application, the encoder is divided and a plurality of sub-encoders are used to jointly generate the encoding output, so that the encoding output can represent more text features of the source text, i.e., the encoding output including low-level semantic information and high-level syntax information is provided to the decoder for processing, thereby enabling more accurate translation of the source text. Furthermore, since the hidden representations of each sub-encoder will act on the corresponding decoder, the gradient backpropagation path in the training process will be greatly reduced. Furthermore, the weight values of the hidden representations output by each sub-encoder are determined by using the attention mechanism, which can better reflect the context information of each word, so that the translation result is more accurate. Furthermore, the hidden representations output by each sub-encoder can be input to the corresponding RNN model or variant of the RNN model as each time series data, so that the hidden representations of each sub-encoder can be mapped to the decoder end, and the corresponding gradient backpropagation path will be greatly reduced, thereby being not affected by the gradient decay.

[0080] Those skilled in the art will appreciate that embodiments of the application can be supplied as a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage media, etc.) embodying computer-readable program code.

[0081] The present application is described in reference to the flowcharts and / or block diagrams of the methods, apparatus (systems) and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce an apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flowcharts and / or block diagrams.

[0082] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flowcharts and / or block diagrams.

[0083] These computer program instructions can also be loaded into computer or other programmable data processing devices to cause a series of operational steps to be performed on the computer or other programmable devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable devices provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 Figure 1 Figure 1

[0084] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0085] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) about which the computer stores information about an operating system, application software, and / or the like. Memory is an example of computer readable media.

[0086] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile discs (DVD) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0087] It should also be noted that the terms "comprising", "including", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article or apparatus that includes the stated element.

[0088] ​​Those skilled in the art will appreciate that embodiments of the present application can be devised for a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code thereon for use by or in connection with an instruction execution system. For the purposes of this description, a computer-usable or computer readable storage medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0089] The above descriptions are only some embodiments of the present application and are not intended to limit the present application. Various modifications and changes can be made by those skilled in the art to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A machine translation method, characterized in that: include: receiving a source text in a source language; Performing word segmentation processing on the source text to obtain each input word segment of the source text; performing word embedding processing on each input word segment to obtain an input word segmentation vector for each input word segment; The input word segmentation vector is input into an encoder, and multiple sub-encoders process the input word segmentation vector, and the hidden representation output by each sub-encoder is input as a single sequence data into a neural network layer for sequence data, and each sequence data output by the neural network layer is used as the encoded output of the encoder, wherein each sub-encoder includes at least one neural network layer; the encoder includes multiple neural network layers and is divided to obtain multiple sequentially connected sub-encoders; the sub-encoders corresponding to the low-level neural network layers among the multiple sub-encoders are used to extract low-level semantic information; the sub-encoders corresponding to the high-level neural network layers among the multiple sub-encoders are used to extract high-level syntactic information; The encoded output is decoded by a decoder to obtain a target text in a target language.

2. The method according to claim 1, wherein Using the multiple sub-encoders to process the input word segmentation vector to obtain a hidden representation output by each sub-encoder includes: For each sub-encoder, the input word segmentation vector input from the current sub-encoder and the output information of the previous sub-encoder are used as input information for processing, thereby obtaining multiple hidden representations corresponding to multiple sub-encoders respectively.

3. The method according to claim 1, wherein The encoded output of the encoder generated by using the hidden representation of each sub-encoder output includes: The encoded output of the encoder is generated by determining the weights of each hidden representation.

4. The method according to claim 3, wherein The encoder generates the encoded output by determining the weights corresponding to each hidden element: Use the attention mechanism to determine the weight of each hidden representation; The encoded output is determined based on each hidden representation and the corresponding weight.

5. The method according to claim 1, wherein The neural network layer for sequence data includes a recurrent neural network.

6. The method according to claim 5, wherein The decoder includes a plurality of sub-decoders, wherein the number of the plurality of sub-decoders is not restricted by the number of sub-encoders.

7. The method according to claim 6, wherein Decoding the encoded output using a decoder to obtain a target text in a target language includes: The encoded output is input to each sub-decoder in the decoder. Each sub-decoder uses the output decoded by the previous sub-decoder and the corresponding encoded output to perform decoding and input the decoded output to the next sub-decoder. The processing is performed in sequence, and the output of the last sub-decoder is used as the target text.

8. A machine translation method, characterized in that: include: receiving a source text in a source language; Performing word segmentation processing on the source text to obtain each input word segment of the source text; performing word embedding processing on each input word segment to obtain an input word segmentation vector for each input word segment; Inputting the source text into a trained machine translation model to obtain a target text in a target language, wherein the machine translation model includes an encoder and a decoder, wherein the encoder includes multiple sub-encoders, and the encoding output represented by multiple hidden representations output by the multiple sub-encoders is used as the input of the decoder, and the encoder includes multiple neural network layers. The sub-encoders corresponding to the low-level neural network layers in the multiple sub-encoders are used to extract low-level semantic information; the sub-encoders corresponding to the high-level neural network layers in the multiple sub-encoders are used to extract high-level syntactic information; wherein inputting the source text into the trained machine translation model to obtain the target text in the target language includes: inputting the input word segmentation vector into the encoder, the multiple sub-encoders performing processing on the input word segmentation vector, and the hidden representation output by each sub-encoder as a single sequence data input into the neural network layer, and each sequence data output by the neural network layer is used as the encoded output of the encoder, and the decoder is used to decode the encoded output to obtain the target text in the target language.

9. A machine translation apparatus comprising: one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for executing any one of the methods according to claims 1-8.

10. A computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by a computing device, cause the computing device to perform any one of the methods according to claims 1-8.

Images