A translation method and device, a translation model training method and device

Abstract

The application provides a translation method and device, a training method and device of a translation model, and is used for a translation model, the translation model comprises an encoding layer and a decoding layer, the decoding layer comprises m second stack layer groups connected in sequence, wherein m is greater than or equal to 2; the translation method comprises the following steps: determining an output vector of the encoding layer according to an input sentence to be translated; inputting the output vector of the encoding layer into m second stack layer groups respectively to obtain an output vector of each second stack layer group; aggregating the output vectors of the m second stack layer groups to obtain a decoding vector of the decoding layer, and obtaining a translated sentence according to the decoding vector of the decoding layer, so as to realize aggregation between the output vectors of the second stack layer groups and finally improve the performance of the translation model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a translation method and apparatus, a training method and apparatus for a translation model, a computing device, and a computer-readable storage medium. Background Technology

[0002] With the improvement of computing power, neural networks are being used more and more widely, such as in building translation models to convert source language to target language. Generally, the architecture of a translation model includes an encoder layer and a decoder layer. The encoder layer encodes the source sentence to be translated into a vector, and the decoder layer decodes the vector of the source sentence to generate the corresponding target sentence.

[0003] The translation model is optimized using the training set to obtain a well-trained model for the entire training set of data. Then, the trained translation model is used to encode and decode the new input sentence to obtain the corresponding translation sentence.

[0004] Traditional translation models mostly use Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs) as the basis for their encoder-decoder layers. Google's attention-based translation model does not use any CNN or RNN structures. This model can work in high parallelism, thus improving translation performance while also being exceptionally fast to train.

[0005] See Figure 1 , Figure 1 This describes the architecture of an existing Google Translate model. The Google Translate model consists of two parts: an encoding layer and a decoding layer. The encoding layer comprises six identical encoding stacks stacked together, and the decoding layer comprises six identical decoding stacks stacked together. The bottom encoding stack receives the input sentence vector, and the input vectors of the other encoding stacks originate from the output vector of the previous encoding stack. The output vector of the last encoding stack is used as the input vector to each decoding stack.

[0006] In existing methods, the input to each encoding stack layer depends on the output vector of the previous encoding stack layer, and the output of each encoding stack layer is the encoding vector of that layer; similarly, the input to each decoding stack layer depends on the output vector of the previous decoding stack layer, and the output of each decoding stack layer is the decoding vector of that layer. This approach does not efficiently aggregate information between encoding and decoding vectors, which negatively impacts the final translation quality. Summary of the Invention

[0007] In view of this, embodiments of this application provide a translation method and apparatus, a translation model training method and apparatus, a computing device, and a computer-readable storage medium to address the technical deficiencies existing in the prior art.

[0008] This application discloses a translation method for a translation model, the translation model including an encoding layer and a decoding layer, the decoding layer including m sequentially connected second stack layer groups, where m≥2;

[0009] The method includes:

[0010] Based on the input statement to be translated, determine the output vector of the coding layer;

[0011] The output vector of the coding layer is input into m second stack layer groups respectively to obtain the output vector of each second stack layer group;

[0012] The output vectors of m second stack layer groups are aggregated to obtain the decoding vector of the decoding layer, and the translated statement is obtained based on the decoding vector of the decoding layer.

[0013] Optionally, the encoding layer includes a first hidden layer and a sequentially connected first stack layers, where a is a positive integer greater than 1;

[0014] Based on the input statement to be translated, the output vector corresponding to each first stack layer is obtained, including:

[0015] S102. Input the statement to be translated into the first hidden layer to generate a first vector;

[0016] S104. Input the first vector into the first stack layer to obtain the output vector of the first stack layer;

[0017] S106. Input the output vector of the (b-1)th first stack layer to the bth first stack layer to obtain the output vector corresponding to the bth first stack layer, where 2≤b≤a;

[0018] S108. Increment b by 1. Determine if b after incrementing by 1 is greater than a. If yes, end the process. If no, continue with step S106.

[0019] Optionally, the translation model includes: a linear layer, a fully connected layer, and a normalized layer;

[0020] The output vectors corresponding to the first stack layer are aggregated to obtain the encoding vectors corresponding to each first stack layer, including:

[0021] The output vectors of the first stack layer are aggregated through a linear layer to obtain a linear vectors of the first stack layer.

[0022] The linear vectors of the first stack layer are processed through a fully connected layer to obtain the fully connected vector corresponding to each first stack layer.

[0023] The fully connected vector corresponding to each first stack layer is processed by a normalization layer to obtain the encoding vector corresponding to each first stack layer.

[0024] Optionally, in the decoding layer, each second stack layer group includes at least two second stack layer groups, and each second stack layer group includes at least two second stack layers;

[0025] The output vector of the coding layer is input into m second stack layer groups respectively, resulting in the output vector of each second stack layer group, including:

[0026] The output vector of the coding layer is input into the at least two second stack layers respectively, and the output vector of the at least two second stack layers is obtained by the output vector of the at least two second stack layers grouped together.

[0027] The output vector of each second stack layer group is obtained by using the output vectors of the at least two second stack layer groups.

[0028] Optionally, the decoding layer includes a second hidden layer;

[0029] The output vector of the coding layer is input into m second stack layer groups to obtain the output vector of each second stack layer group, including:

[0030] S202. Input the output vector of the encoding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0031] S204. Input the output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the ith second stack layer group to obtain the output vector of the ith second stack layer group; where 2≤i≤m;

[0032] S206. Increment i by 1. Determine if the incremented i is greater than m. If yes, end the process. If no, continue with step S204.

[0033] Optionally, the output vector of the coding layer is at least two;

[0034] Step S202 includes: inputting at least one output vector of the encoding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0035] Step S204 includes: inputting at least one output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the ith second stack layer group to obtain the output vector of the ith second stack layer group.

[0036] Optionally, each second stack layer group includes n second stack layer groups;

[0037] Step S202 includes:

[0038] S302. Input the output vector of the coding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0039] S304. Input the output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group, where 2≤j≤n;

[0040] S306. Increment j by 1, and determine whether j after incrementing by 1 is greater than n. If yes, proceed to step S308; otherwise, continue to step S304.

[0041] S308. Aggregate the output vectors of the n second stack layer groups to obtain the output vector of the first second stack layer group.

[0042] Optionally, the output vector of the coding layer is at least two;

[0043] Step S302 includes: inputting at least one output vector of the encoding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0044] Step S304 includes: inputting at least one output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group.

[0045] Optionally, each second stack layer group includes n second stack layer groups;

[0046] Step S204 includes:

[0047] S402. Input the output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0048] S404. Input the output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group, where 2≤j≤n;

[0049] S406. Increment j by 1. Determine if j after incrementing by 1 is greater than n. If yes, proceed to step S408. If no, continue to step S404.

[0050] S408. Aggregate the output vectors of the n second stack layer groups to obtain the output vector of the i-th second stack layer group.

[0051] Optionally, the output vector of the coding layer is at least two;

[0052] Step S402 includes: inputting at least one output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0053] Step S404 includes: inputting at least one output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group.

[0054] Optionally, each second stack layer group includes p second stack layers; step S402 includes:

[0055] S502. Input one of the output vectors of the coding layer and the output vector of the (i-1)th second stack layer group into the first second stack layer to obtain the output vector of the first second stack layer;

[0056] S504. Input one of the output vectors of the coding layer and the output vector of the (k-1)th second stack layer into the kth second stack layer to obtain the output vector of the kth second stack layer, where 2≤k≤p;

[0057] S506. Increment k by 1, and determine whether the incremented k is greater than p. If yes, proceed to step S508; otherwise, continue to step S504.

[0058] S508. Aggregate the output vectors of the p second stack layers to obtain the output vector of the first second stack layer group.

[0059] Optionally, each second stack layer group includes p second stack layers; step S404 includes:

[0060] S602. Input one of the output vectors of the coding layer and the output vector of the (j-1)th second stack layer into the first second stack layer to obtain the output vector of the first second stack layer;

[0061] S604. Input one of the output vectors of the coding layer and the output vector of the (k-1)th second stack layer into the kth second stack layer to obtain the output vector of the kth second stack layer, where 2≤k≤p;

[0062] S606. Increment k by 1, and determine whether the incremented k is greater than p. If yes, proceed to step S608; otherwise, continue to step S604.

[0063] S608. Aggregate the output vectors of the p second stack layers to obtain the output vector of the j-th second stack layer group.

[0064] Optionally, the decoding layer may also include: a linear layer and a normalization layer;

[0065] The output vectors of the m second stack layer groups are aggregated to obtain the decoding vector of the decoding layer, including:

[0066] Aggregate the output vectors of m second stack layer groups via a linear layer;

[0067] The result obtained by aggregation through the linear layer is normalized through the normalization layer to obtain the decoding vector of the decoding layer.

[0068] This application discloses a method for training a translation model, the translation model including an encoding layer and a decoding layer, the decoding layer including m sequentially connected second stack layer groups, where m≥2;

[0069] The training method includes:

[0070] The output vector of the coding layer is determined based on the first sentence in the input target corpus.

[0071] The output vector of the encoding layer and the translated second statement corresponding to the first statement are both input into the m second stack layer groups included in the decoding layer to obtain the output vector of each second stack layer group;

[0072] The output vectors of the m second stack layer groups are aggregated to obtain the decoding vector of the decoding layer, and the error of the decoding vector is obtained based on the decoding vector;

[0073] The translation model continues to be trained based on the error of the decoded vector until the training stopping condition is met.

[0074] This application discloses a translation device disposed in a translation model, the translation model including an encoding layer and a decoding layer, the decoding layer including m second stack layer groups, where m≥2; the device includes:

[0075] The coding layer output vector acquisition module is configured to determine the output vector of the coding layer based on the input utterance to be translated.

[0076] The decoding layer output vector acquisition module is configured to input the output vector of the encoding layer into m second stack layer groups respectively, and obtain the output vector of each second stack layer group;

[0077] The aggregation module is configured to aggregate the output vectors of m second stack layer groups to obtain the decoding vector of the decoding layer, and to obtain the translated statement based on the decoding vector of the decoding layer.

[0078] This application discloses a training device for a translation model, the translation model including an encoding layer and a decoding layer, the decoding layer including m second stack layer groups, where m≥2; the training device includes:

[0079] The encoding layer output vector training module is configured to determine the output vector of the encoding layer based on the first sentence in the input target corpus.

[0080] The decoding layer output vector training module is configured to input the output vector of the encoding layer and the translated second statement corresponding to the first statement into the m second stack layer groups included in the decoding layer, so as to obtain the output vector of each second stack layer group.

[0081] The aggregation training module is configured to aggregate the output vectors of m second stack layer groups to obtain the decoding vector of the decoding layer, and obtain the error of the decoding vector based on the decoding vector;

[0082] The second training module is configured to continue training the translation model based on the error of the decoding vector until the training stopping condition is met.

[0083] This application discloses a computing device, including a memory, a processor, and computer instructions stored in the memory and executable on the processor. When the processor executes the instructions, it implements the steps of the translation method or the training method of the translation model as described above.

[0084] This application discloses a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the translation method or translation model training method as described above.

[0085] The translation method and apparatus provided in this application determine the output vector of the encoding layer based on the input sentence to be translated; input the output vector of the encoding layer into m second stack layer groups respectively to obtain the output vector of m second stack layer groups; aggregate the output vectors of the m second stack layer groups to obtain the decoding vector of the decoding layer; and obtain the translation sentence based on the decoding vector of the decoding layer, thereby realizing the aggregation between the output vectors of the second stack layer groups and ultimately improving the performance of the translation model.

[0086] Secondly, the aggregation of output vectors between the first stack layers is not a simple concatenation, but rather an aggregation of output vectors between the first stack layers using linear layers, followed by fully connected layers and normalization layers to generate the final encoded vector, thereby achieving efficient aggregation of output vectors between the first stack layers.

[0087] Furthermore, the aggregation of output vectors between the second stack layers is not a simple concatenation, but rather the final decoded vector is obtained by using fully connected layers and normalization layers, so as to achieve efficient aggregation of output vectors between the second stack layers.

[0088] The translation model training method and apparatus provided in this application determine the output vector of the encoding layer based on the first sentence in the input target corpus; input the output vector of the encoding layer and the translated second sentence corresponding to the first sentence into m second stack layer groups of the decoding layer to obtain the output vector of each second stack layer group; aggregate the output vectors of the m second stack layer groups to obtain the decoding vector of the decoding layer, and obtain the error of the decoding vector based on the decoding vector; continue to train the translation model based on the error of the decoding vector, so that the output vectors of the second stack layer can be effectively aggregated during the model training stage, which helps to improve the translation performance of the translation model during the usage stage. Attached Figure Description

[0089] Figure 1 This is a schematic diagram of the architecture of an existing Google Translate model;

[0090] Figure 2 This is a schematic diagram of the structure of a computing device according to an embodiment of this application;

[0091] Figure 3 This is a flowchart illustrating a translation method according to an embodiment of this application;

[0092] Figure 4 This is a flowchart illustrating a translation method according to an embodiment of this application;

[0093] Figure 5 This is a flowchart illustrating a translation method according to an embodiment of this application;

[0094] Figure 6 This is a flowchart illustrating a translation method according to an embodiment of this application;

[0095] Figure 7 This is a flowchart illustrating a translation method according to an embodiment of this application;

[0096] Figure 8 This is a flowchart illustrating a translation method according to an embodiment of this application;

[0097] Figure 9 This is a schematic diagram of the architecture of a translation model according to an embodiment of this application;

[0098] Figure 10 This is a schematic diagram illustrating the generation of decoding vectors in a translation model according to an embodiment of this application;

[0099] Figure 11 This is a flowchart illustrating a training method for a translation model according to another embodiment of this application;

[0100] Figure 12 This is a schematic diagram of the structure of a translation device according to an embodiment of this application;

[0101] Figure 13 This is a schematic diagram of the structure of a training device for a translation model according to an embodiment of this application. Detailed Implementation

[0102] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0103] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0104] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0105] First, the terminology used in one or more embodiments of the present invention will be explained.

[0106] Translation model: Its main idea is to encode the sentence to be translated into an encoding vector through an encoder, then decode the encoding vector using a decoder to obtain a decoded vector, and finally translate the decoded vector into the corresponding translation sentence.

[0107] Google Translate: A neural network architecture proposed by Google for machine translation.

[0108] Encoder: Converts the text to be translated into an encoded vector;

[0109] Decoder: Converts encoded vectors into the language text of translated sentences.

[0110] This application provides a translation method and apparatus, a translation model training method and apparatus, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0111] Figure 2 This is a structural block diagram of a computing device 200 according to an embodiment of this specification. The components of the computing device 200 include, but are not limited to, a memory 210 and a processor 220. The processor 220 is connected to the memory 210 via a bus 230, and a database 250 is used to store data.

[0112] The computing device 200 also includes an access device 240 that enables the computing device 200 to communicate via one or more networks 260. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 240 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0113] In one embodiment of this specification, the aforementioned components of the computing device 200 and Figure 2 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 2 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0114] The computing device 200 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 200 can also be a mobile or stationary server.

[0115] Among them, processor 220 can execute Figure 3 The steps in the method shown. Figure 3 A schematic flowchart of a translation method according to an embodiment of this application is shown for a translation model, which includes an encoding layer and a decoding layer. The encoding layer includes a first hidden layer and *a* first stack layers, which are sequentially connected. The decoding layer includes a second hidden layer and *m* groups of second stack layers, which are sequentially connected. Each second stack layer group includes at least two second stack layer subgroups, and each second stack layer subgroup includes at least two second stack layers.

[0116] One embodiment of the translation method of this application includes the following steps 302-306:

[0117] 302. Based on the input statement to be translated, obtain the output vector corresponding to each first stack layer.

[0118] Specifically, step 302 includes the following steps S3022 to S3028:

[0119] S3022. Input the statement to be translated into the first hidden layer to obtain the first vector.

[0120] S3024. Input the first vector into the first stack layer to obtain the output vector of the first stack layer.

[0121] S3026. Input the output vector of the (b-1)th first stack layer to the bth first stack layer to obtain the output vector corresponding to the bth first stack layer, where 2≤b≤a.

[0122] S3028. Increment b by 1. Determine if b after incrementing by 1 is greater than a. If yes, end the process. If no, continue with step S3026.

[0123] 304. Aggregate the output vectors corresponding to the first stack layer to obtain the encoding vector corresponding to each first stack layer, and determine the output vector of the encoding layer based on the encoding vector corresponding to each first stack layer.

[0124] The translation model includes: linear layers, fully connected layers, and normalized layers;

[0125] See Figure 4 In step 304, the output vectors corresponding to the first stack layer are aggregated to obtain the encoding vector corresponding to each first stack layer, including the following steps 402 to 408:

[0126] 402. Aggregate the output vectors of the first stack layer through a linear layer to obtain a linear vectors of the first stack layer.

[0127] Specifically, step 402 includes:

[0128] S4022. The output vector of the first stack layer is processed through a linear layer to obtain the linear vector of the first stack layer.

[0129] Optionally, step S4022 includes: multiplying the output vector corresponding to the first first stack layer by a weight coefficient to obtain the linear vector of the first first stack layer.

[0130] S4024. Input the output vector of the (b-1)th first stack layer into the bth first stack layer to obtain the output vector corresponding to the bth first stack layer. Then, aggregate the output vectors corresponding to the (b-1)th and bth first stack layers through a linear layer to obtain the linear vector of the bth first stack layer.

[0131] In this embodiment, step S4024 includes: summing the output vector corresponding to the b-th first stack layer and the output vector corresponding to the (b-1)-th first stack layer, and then multiplying it by the weight coefficient to obtain the linear vector of the b-th first stack layer.

[0132] Specifically, this can be achieved through the following formula (1):

[0133] y1=w1(x+y) (1)

[0134] Where x represents the output vector corresponding to the b-th first stack layer, y represents the output vector corresponding to the (b-1)-th first stack layer, w1 represents the weight coefficient of the linear layer, and y1 represents the linear vector of the b-th first stack layer.

[0135] 404. Process the linear vectors of the first stack layer through a fully connected layer to obtain the fully connected vector corresponding to each first stack layer.

[0136] It should be noted that each node in the fully connected layer is connected to all nodes in the linear layer. The output of each node in the fully connected layer is obtained by multiplying the outputs of all nodes in the linear layer by their corresponding weights, summing the results, and then adding the bias coefficient.

[0137] For step 404, the fully connected vector corresponding to each first stack layer is obtained by multiplying the linear vectors of the n first stack layers by weight coefficients and then adding a bias coefficient.

[0138] Specifically, the fully connected vector corresponding to each first stack layer can be implemented using the following formula (2):

[0139] y2=∑w2*y1+d (2)

[0140] Where y2 represents the fully connected vector of each first stack layer.

[0141] y1 represents the linear vectors of the first to bth stack layers.

[0142] w2 represents the weight coefficients of the fully connected layer. It should be noted that each first stack layer has a corresponding set of weight coefficients, and each linear vector of the first stack layer corresponds to one weight coefficient.

[0143] d represents the offset parameter of the fully connected layer, and each first stack layer corresponds to one offset parameter.

[0144] 406. The fully connected vector corresponding to each first stack layer is processed by a normalization layer to obtain the encoding vector corresponding to each first stack layer.

[0145] Specifically, this can be achieved through the following formula (3):

[0146] norm=(y2-mean) / variance (3)

[0147] Where norm represents the encoding vector of each first stack layer, y2 represents the fully connected vector of each first stack layer, mean represents the mean of the normalized layer, and variance represents the standard deviation of the normalized layer.

[0148] For the values ​​of the mean and variance parameters, please refer to the process of mean normalization.

[0149] As can be seen from steps S4022 and S4024, for the first first stack layer, the corresponding linear vector can be obtained directly from the output vector of the first stack layer. However, for the linear vectors corresponding to other first stack layers besides the first one, it is necessary to aggregate them based on the output vector of the first stack layer and the output vector of the previous first stack layer.

[0150] 408. Increment b by 1. Determine if b after incrementing by 1 is greater than a. If yes, end the process. If no, continue to step 406.

[0151] Step 304 yields the encoding vector corresponding to each first stack layer. After obtaining the encoding vector for each first stack layer, the output vector of the encoding layer is determined based on the encoding vector for each first stack layer. Specifically, the output vector of the encoding layer can be obtained by using the encoding vector corresponding to the last one or more first stack layers as the output vector of the encoding layer.

[0152] In this embodiment, there are several methods to input multiple encoded vectors as output vectors of the encoding layer to the decoding layer, provided that each second stack layer of the decoding layer can receive the input encoded vectors. For example, in one specific implementation, the first stack layer has 6 layers and the second stack layer has 8 layers. The encoded vectors of the last two first stack layers are input to the 8 second stack layers. Specifically, the encoded vector of the last first stack layer can be input to the last 4 second stack layers, and the encoded vector of the second-to-last first stack layer can be input to the first 4 second stack layers.

[0153] It should be explained that for multiple sequentially connected first stack layers, the earlier the position (closer to the beginning of the first stack layer), the more semantic information its encoding vector contains; conversely, the later the position (closer to the last first stack layer), the more syntactic information its encoding vector contains. This embodiment inputs the encoding vectors corresponding to the last multiple first stack layers into the decoding layer. This allows for more rational utilization of semantic and syntactic information in subsequent decoding processes, and also enhances the decoding capability of the decoding layer through the fusion of layer encoding vector information.

[0154] 306. Input the output vector of the encoding layer into the decoding layer to obtain the decoding vector of the decoding layer, and obtain the translated statement based on the decoding vector of the decoding layer.

[0155] Specifically, the decoding layer comprises m groups of second stack layers, see [link to documentation]. Figure 5 Step 306 includes the following steps 502 to 504:

[0156] 502. Input the output vector of the coding layer into m second stack layer groups respectively to obtain the output vector of each second stack layer group.

[0157] 504. Aggregate the output vectors of the m second stack layer groups to obtain the decoding vector of the decoding layer.

[0158] See Figure 6 Step 502 includes the following steps 602 to 606:

[0159] 602. Input the output vector of the encoding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group.

[0160] For cases where there are at least two output vectors of the coding layer, step 602 includes: inputting at least one output vector of the coding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group.

[0161] It should be noted that the sentence to be translated includes at least one word to be translated. During the translation process, the corresponding translation words for at least one word to be translated are obtained sequentially. Each translation word can be used as a reference vector input to the second embedding layer.

[0162] Specifically, for the first word to be translated, the reference vector is the initial value set, which can be 0; for the other words to be translated excluding the first word, the reference vector is the translation word corresponding to the previous word to be translated.

[0163] For example, for the phrase "I love China", if the word to be translated is "love", then the reference vector is the word "I" corresponding to "I".

[0164] 604. Input the output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the ith second stack layer group to obtain the output vector of the ith second stack layer group; where 2≤i≤m.

[0165] For cases where there are at least two output vectors of the coding layer, step 604 includes: inputting at least one output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the ith second stack layer group to obtain the output vector of the ith second stack layer group.

[0166] 606. Increment i by 1. Determine if the incremented i is greater than m. If yes, end the process. If no, continue to step 604.

[0167] Taking an example where each second stack layer group comprises n second stack layer groups, see [link to example]. Figure 7 Step 602 includes:

[0168] 702. Input the output vector of the encoding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group.

[0169] For cases where there are at least two output vectors of the coding layer, step 702 includes: inputting at least one output vector of the coding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group.

[0170] 704. Input the output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group, where 2≤j≤n.

[0171] For cases where there are at least two output vectors of the coding layer, step 704 includes: inputting at least one output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group.

[0172] 706. Increment j by 1, and determine whether j after incrementing by 1 is greater than n. If yes, proceed to step 708; otherwise, continue to step 704.

[0173] 708. Aggregate the output vectors of the n second stack layer groups to obtain the output vector of the first second stack layer group.

[0174] Through steps 702 to 708 above, the output vector of the first second stack layer group can be obtained.

[0175] Taking an example where each second stack layer group comprises n second stack layer groups, see [link to example]. Figure 8 Step 604 includes:

[0176] 802. Input the output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the first second stack layer group to obtain the output vector of the first second stack layer group.

[0177] For cases where there are at least two output vectors of the coding layer, step 802 includes: inputting at least one output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the first second stack layer group to obtain the output vector of the first second stack layer group.

[0178] 804. Input the output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group, where 2≤j≤n.

[0179] For cases where there are at least two output vectors of the coding layer, step 804 includes: inputting at least one output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group.

[0180] 806. Increment j by 1. Determine if j after incrementing by 1 is greater than n. If yes, proceed to step 808. If no, continue to step 804.

[0181] 808. Aggregate the output vectors of the n second stack layer groups to obtain the output vector of the i-th second stack layer group.

[0182] Through steps 802 to 808 above, the output vector of the i-th second stack layer group can be obtained.

[0183] Taking a group of p second stack layers as an example, step 802 includes:

[0184] S8022. Input one of the output vectors of the coding layer and the output vector of the (i-1)th second stack layer group into the first second stack layer to obtain the output vector of the first second stack layer.

[0185] S8024. Input one of the output vectors of the coding layer and the output vector of the (k-1)th second stack layer into the kth second stack layer to obtain the output vector of the kth second stack layer, where 2≤k≤p.

[0186] S8026. Increment k by 1. Determine if the incremented k is greater than p. If yes, proceed to step S8028. If no, continue to step S8024.

[0187] S8028. Aggregate the output vectors of the p second stack layers to obtain the output vector of the first second stack layer group.

[0188] Taking a group of p second stack layers as an example, step 804 includes:

[0189] S8042. Input one of the output vectors of the coding layer and the output vector of the (j-1)th second stack layer into the first second stack layer to obtain the output vector of the first second stack layer.

[0190] S8044. Input one of the output vectors of the coding layer and the output vector of the (k-1)th second stack layer into the kth second stack layer to obtain the output vector of the kth second stack layer, where 2≤k≤p.

[0191] S8046. Increment k by 1, and determine whether the incremented k is greater than p. If yes, proceed to step S8048; otherwise, continue to step S8044.

[0192] S8048. Aggregate the output vectors of the p second stack layers to obtain the output vector of the j-th second stack layer group.

[0193] As can be seen from the above, in this embodiment, the multiple second stack layers of the decoding layer are divided into three levels: second stack layer group, second stack layer subgroup, and second stack layer. The output vector of the second stack layer can be used to obtain the output vector of the second stack layer subgroup; the output vector of the second stack layer subgroup can be used to obtain the output vector of the second stack layer group; and the decoding vector of the decoding layer can be used to obtain the output vector of the second stack layer group.

[0194] Of course, in practical applications, it can be divided into more levels based on the number of second stack layers.

[0195] After obtaining the output vectors of m second stack layer groups, the decoding vector of the decoding layer is obtained based on the output vectors of the m second stack layer groups.

[0196] Specifically, obtaining the decoding vector of the decoding layer based on the output vectors of the m second stack layer groups includes: aggregating the output vectors of the m second stack layer groups to obtain the decoding vector of the decoding layer.

[0197] Specifically, the decoding layer also includes a linear layer and a normalization layer. The output vectors of the m second stack layer groups are aggregated through the linear layer and then normalized through the normalization layer to obtain the decoding vector of the decoding layer.

[0198] The translation method provided in this application aggregates the output vectors corresponding to the first stack layer to obtain the encoding vector corresponding to each first stack layer, and determines the output vector of the encoding layer based on the encoding vector corresponding to each first stack layer; inputs the output vector of the encoding layer to the decoding layer to obtain the decoding vector of the decoding layer, and obtains the translated sentence based on the decoding vector of the decoding layer, thereby realizing the aggregation between the output vectors of the first stack layer and ultimately improving the performance of the translation model.

[0199] Secondly, in addition, by inputting m of the n encoded vectors into the decoding layer, this embodiment can make more reasonable use of semantic and syntactic information in the subsequent decoding process, and enhance the decoding capability of the decoding layer through the information fusion of the layer encoded vectors.

[0200] Furthermore, the encoding vector for each first stack layer is not only related to the output vector of that first stack layer, but is obtained by processing the output vectors of n first stack layers through linear layers, fully connected layers, and normalization layers. This facilitates the fusion of sentence information among multiple first stack layers and helps improve the translation performance of the translation model.

[0201] To facilitate understanding of the technical solution in this embodiment, the following is combined with... Figure 9 This embodiment will be illustrated using a specific translation model.

[0202] See Figure 9 The translation model shown includes an encoding layer and a decoding layer. The encoding layer consists of six first stack layers, and the decoding layer consists of eight second stack layers. The first and second stack layers are further connected sequentially to a linear layer, a fully connected layer, and a normalization layer. The eight second stack layers are divided as follows: the decoding layer comprises two groups of second stack layers, each group of second stack layers comprises two subgroups of second stack layers, and each subgroup of second stack layers comprises two second stack layers.

[0203] Translation methods based on this translation model include:

[0204] S912. Input the statement to be translated into the first hidden layer to generate the first vector.

[0205] S914. Input the first vector into the first stack layer to obtain the output vector of the first stack layer.

[0206] S916. Input the output vector of the (b-1)th first stack layer to the bth first stack layer to obtain the output vector corresponding to the bth first stack layer, where 2≤b≤6.

[0207] S918. The output vectors of the six first stack layers are aggregated through a linear layer to obtain six linear vectors of the six first stack layers; the linear vectors of the six first stack layers are processed through a fully connected layer to obtain a fully connected vector corresponding to each first stack layer; the fully connected vector corresponding to each first stack layer is processed through a normalization layer to obtain a coding vector corresponding to each first stack layer.

[0208] S920. Input the encoding vectors corresponding to the 4th to 6th first stack layers into the decoding layer.

[0209] Specifically, the encoding vector corresponding to the 6th first stack layer is input to the 3rd to 8th second stack layers, the encoding vector corresponding to the 5th first stack layer is input to the 2nd second stack layer, and the encoding vector corresponding to the 4th first stack layer is input to the 1st second stack layer.

[0210] S922. Calculate the decoding vector output by the decoding layer, and obtain the translated statement based on the decoding vector.

[0211] Specifically, see Figure 10 , Figure 10 A schematic diagram of the decoding vector generation process is shown.

[0212] Step S920 includes:

[0213] 1) Input the reference vector corresponding to the sentence to be translated into the second hidden layer to generate the second vector.

[0214] 2) Input the second vector and the three encoded vectors into the first second stack layer group to obtain the output vector of the first second stack layer group.

[0215] The process of generating the output vector of the first second stack layer group includes:

[0216] The second vector and the encoding vectors corresponding to the 4th and 5th first stack layers are input into the first second stack layer group to obtain the output vector of the first second stack layer group.

[0217] The encoding vector corresponding to the 6th first stack layer and the output vector of the 1st second stack layer group are input into the 2nd second stack layer group to obtain the output vector of the 2nd second stack layer group.

[0218] The output vector of the first second stack layer group is obtained by aggregating the output vectors of the first and second second stack layers, and the output vector of the second second stack layer group is obtained by aggregating the output vectors of the third and fourth second stack layers.

[0219] 3) Input the encoding vector corresponding to the 6th first stack layer and the output vector of the 1st second stack layer group into the 2nd second stack layer group to obtain the output vector of the 2nd second stack layer group.

[0220] The process of generating the output vector of the second stack layer group includes:

[0221] The encoding vector corresponding to the 6th first stack layer and the output vector of the 1st second stack layer group are input into the 3rd second stack layer group to obtain the output vector of the 3rd second stack layer group.

[0222] The encoding vector corresponding to the 6th first stack layer and the output vector of the 3rd second stack layer group are input into the 4th second stack layer group to obtain the output vector of the 4th second stack layer group.

[0223] Specifically, the output vector of the third second stack layer group is obtained by aggregating the output vectors of the fifth and sixth second stack layers, and the output vector of the fourth second stack layer group is obtained by aggregating the output vectors of the seventh and eighth second stack layers.

[0224] The specific process for the output vector of each second stack layer has been described in detail in the previous embodiments, and will not be repeated here.

[0225] 4) The output vectors of the first and second second stack layers are aggregated by a linear layer and processed by a normalization layer to obtain the decoded vector.

[0226] The above embodiments are illustrative descriptions of the translation method of this application. Another embodiment of this application discloses a method for training a translation model, the translation model including an encoding layer and a decoding layer, the encoding layer including a sequentially connected first stack layers, where a is a positive integer greater than 1;

[0227] See Figure 11 The training methods include 1102 to 1108:

[0228] 1102. Based on the first sentence in the input target corpus, obtain the output vector corresponding to each first stack layer.

[0229] In this embodiment, the target corpus includes multiple first statements and multiple second statements, with a one-to-one correspondence between the first and second statements. For example, the first statement is "I love China" and the second statement is "I love china"; the first statement is "I eat apples" and the second statement is "I eat apples".

[0230] Specifically, step 1102 includes:

[0231] S1122. Input the first sentence in the target corpus into the first hidden layer to generate the first vector;

[0232] For example, if the first statement is "I love China", the first vector generated is X = (x0, x1, x2, x3), where x0 = "I", x1 = "love", x2 = "China", and x3 = "country".

[0233] S1124. Input the first vector into the first stack layer to obtain the output vector of the first stack layer;

[0234] S1126. Input the output vector of the (b-1)th first stack layer to the bth first stack layer to obtain the output vector corresponding to the bth first stack layer, where 2≤b≤a;

[0235] S1128. Increment b by 1. Determine if b after incrementing by 1 is greater than a. If yes, end the process. If no, continue with step S1126.

[0236] 1104. Aggregate the output vectors corresponding to the first stack layer to obtain the encoding vector corresponding to each first stack layer, and determine the output vector of the encoding layer based on the encoding vector corresponding to each first stack layer.

[0237] Specifically, step 1104 includes:

[0238] S1142. Aggregate the output vectors of the first stack layer through a linear layer to obtain a linear vectors of the first stack layer.

[0239] Specifically, step S1142 includes: processing the output vector of the first first stack layer through a linear layer to obtain the linear vector of the first first stack layer;

[0240] The output vector of the (b-1)th first stack layer is input to the bth first stack layer to obtain the output vector corresponding to the bth first stack layer. Then, the output vectors corresponding to the (b-1)th and bth first stack layers are aggregated through a linear layer to obtain the linear vector of the bth first stack layer.

[0241] Specifically, the output vectors corresponding to the (b-1)th and bth first stack layers are aggregated through a linear layer to obtain the linear vector of the bth first stack layer. This includes summing the output vectors corresponding to the bth first stack layer and the output vectors corresponding to the (b-1)th first stack layer, and then multiplying them by a weight coefficient to obtain the linear vector of the bth first stack layer.

[0242] S1144. Process the linear vectors of the first stack layer through a fully connected layer to obtain the fully connected vector corresponding to each first stack layer.

[0243] S1146. The fully connected vector corresponding to each first stack layer is processed by a normalization layer to obtain the encoding vector corresponding to each first stack layer.

[0244] Specifically, step 1104 determines the output vector of the coding layer based on the coding vector corresponding to each first stack layer, including: taking the coding vector corresponding to the last one or more first stack layers as the output vector of the coding layer.

[0245] 1106. Input the output vector of the encoding layer and the translated second statement corresponding to the first statement into the decoding layer to obtain the decoding vector of the decoding layer, and obtain the error of the decoding vector based on the decoding vector.

[0246] Specifically, step 1106 includes:

[0247] S1162. Input the second statement into the second hidden layer to generate the second vector;

[0248] S1164. Input the output vector and the second vector of the coding layer into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0249] S1166. Input the output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the ith second stack layer group to obtain the output vector of the ith second stack layer group; where 2≤i≤m;

[0250] S1168. Increment i by 1, and determine whether the incremented i is greater than m. If yes, proceed to step S1160; otherwise, continue to step S1166.

[0251] S1160. Aggregate the output vectors of the m second stack layer groups to obtain the decoding vector of the decoding layer.

[0252] Specifically, the acquisition of the output vector of the second stack layer group has been described in detail in the previous embodiments, and will not be repeated here.

[0253] 1108. Continue training the translation model based on the error of the decoded vector until the training stopping condition is met.

[0254] The error of the decoding vector is obtained by comparing the decoding vector with a preset vector verification set.

[0255] The training stopping condition includes: the rate of change of the error is less than the stability threshold.

[0256] It should be noted that in this embodiment, the obtained decoded vector is not directly compared with the vector corresponding to the original translated sentence to calculate the error. Instead, a vector validation set is introduced. If the decoded vector is directly compared with the vector corresponding to the original translated sentence to calculate the error, it will lead to overfitting, causing the translation model to perform worse in the translation of other sentences, and the translation effect will be counterproductive.

[0257] The training stopping condition includes: the rate of change of the error is less than the stability threshold.

[0258] The stabilization threshold can be set according to actual needs, for example, 1%. In this way, when the error tends to stabilize, the model can be considered to have completed training.

[0259] The training method for the translation model provided in this application aggregates the output vectors corresponding to the first stack layer to obtain the encoding vector corresponding to each first stack layer, and determines the output vector of the encoding layer based on the encoding vector corresponding to each first stack layer; the output vector of the encoding layer and the second sentence are both input to the decoding layer to obtain the decoding vector of the decoding layer, and the error of the decoding vector is obtained based on the decoding vector; the translation model is further trained based on the error of the decoding vector, so that the output vectors of the first stack layer can be effectively aggregated during the model training stage, which helps to improve the translation performance of the translation model during the usage stage.

[0260] The above is an exemplary description of the translation method and the training method of the translation model in this embodiment. The translation device and the training device of the translation model in this embodiment will be described exemplarily below.

[0261] This application also discloses a translation device, see [link to relevant documentation]. Figure 12 The device is configured in a translation model, which includes an encoding layer and a decoding layer.

[0262] The output vector acquisition module 1202 is configured to obtain the output vector corresponding to each first stack layer based on the input statement to be translated;

[0263] The encoding module 1204 is configured to aggregate the output vectors corresponding to the first stack layer to obtain the encoding vectors corresponding to each first stack layer, and determine the output vector of the encoding layer based on the encoding vectors corresponding to each first stack layer.

[0264] The decoding module 1206 is configured to input the output vector of the encoding layer to the decoding layer to obtain the decoding vector of the decoding layer, and to obtain the translated statement based on the decoding vector of the decoding layer.

[0265] Optionally, the coding layer includes a first hidden layer; the output vector acquisition module 1202 is specifically configured as follows:

[0266] S102. Input the statement to be translated into the first hidden layer to generate a first vector;

[0267] S104. Input the first vector into the first stack layer to obtain the output vector of the first stack layer;

[0268] S106. Input the output vector of the (b-1)th first stack layer to the bth first stack layer to obtain the output vector corresponding to the bth first stack layer, where 2≤b≤a;

[0269] S108. Increment b by 1. Determine if b after incrementing by 1 is greater than a. If yes, end the process. If no, continue with step S106.

[0270] Optionally, the translation model includes: a linear layer, a fully connected layer, and a normalization layer; the encoding module 1204 is specifically configured to: aggregate the output vectors of a first stack layer through the linear layer to obtain a linear vectors of the first stack layer;

[0271] The linear vectors of the first stack layer are processed through a fully connected layer to obtain the fully connected vector corresponding to each first stack layer.

[0272] The fully connected vector corresponding to each first stack layer is processed by a normalization layer to obtain the encoding vector corresponding to each first stack layer.

[0273] Optionally, the encoding module 1204 is specifically configured as follows:

[0274] The output vector of the first stack layer is processed through a linear layer to obtain the linear vector of the first stack layer.

[0275] The output vector of the (b-1)th first stack layer is input to the bth first stack layer to obtain the output vector corresponding to the bth first stack layer. Then, the output vectors corresponding to the (b-1)th and bth first stack layers are aggregated through a linear layer to obtain the linear vector of the bth first stack layer.

[0276] Optionally, the encoding module 1204 is specifically configured to: sum the output vector corresponding to the b-th first stack layer and the output vector corresponding to the (b-1)-th first stack layer, and then multiply by the weight coefficient to obtain the linear vector of the b-th first stack layer.

[0277] Optionally, the encoding module 1204 is specifically configured to use the encoding vector corresponding to the last one or more first stack layers as the output vector of the encoding layer.

[0278] Optionally, the decoding layer includes m second stack layer groups, where m ≥ 2; the decoding module 1206 is specifically configured to: input the output vector of the encoding layer into the m second stack layer groups respectively to obtain the output vector of each second stack layer group; and aggregate the output vectors of the m second stack layer groups to obtain the decoding vector of the decoding layer.

[0279] Optionally, the decoding module 1206 is specifically configured as follows:

[0280] S202. Input the output vector of the encoding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0281] S204. Input the output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the ith second stack layer group to obtain the output vector of the ith second stack layer group; where 2≤i≤m;

[0282] S206. Increment i by 1. Determine if the incremented i is greater than m. If yes, end the process. If no, continue with step S204.

[0283] Optionally, the output vector of the coding layer is at least two;

[0284] The decoding module 1206 is specifically configured to: input at least one output vector of the encoding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0285] The decoding module 1206 is specifically configured to input at least one output vector of the encoding layer and the output vector of the (i-1)th second stack layer group into the ith second stack layer group to obtain the output vector of the ith second stack layer group.

[0286] Optionally, each second stack layer group includes n second stack layer groups; the decoding module 1206 is specifically configured to: S302, input the output vector of the encoding layer and the reference vector corresponding to the sentence to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0287] S304. Input the output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group, where 2≤j≤n;

[0288] S306. Increment j by 1, and determine whether j after incrementing by 1 is greater than n. If yes, proceed to step S308; otherwise, continue to step S304.

[0289] S308. Aggregate the output vectors of the n second stack layer groups to obtain the output vector of the first second stack layer group.

[0290] Optionally, the output vector of the encoding layer is at least two, and the decoding module 1206 is specifically configured to: input at least one output vector of the encoding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0291] The decoding module 1206 is specifically configured to input at least one output vector of the encoding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group.

[0292] Optionally, each second stack layer group includes n second stack layer groups; the decoding module 1206 is specifically configured as follows:

[0293] S402. Input the output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0294] S404. Input the output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group, where 2≤j≤n;

[0295] S406. Increment j by 1. Determine if j after incrementing by 1 is greater than n. If yes, proceed to step S408. If no, continue to step S404.

[0296] S408. Aggregate the output vectors of the n second stack layer groups to obtain the output vector of the i-th second stack layer group.

[0297] Optionally, the output vector of the coding layer is at least two; the decoding module 1206 is specifically configured to: input at least one output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the first second stack layer group to obtain the output vector of the first second stack layer group;

[0298] The decoding module 1206 is specifically configured to input at least one output vector of the encoding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group.

[0299] Optionally, each second stack layer group includes p second stack layers; the decoding module 1206 is specifically configured as follows:

[0300] S502. Input one of the output vectors of the coding layer and the output vector of the (i-1)th second stack layer group into the first second stack layer to obtain the output vector of the first second stack layer;

[0301] S504. Input one of the output vectors of the coding layer and the output vector of the (k-1)th second stack layer into the kth second stack layer to obtain the output vector of the kth second stack layer, where 2≤k≤p;

[0302] S506. Increment k by 1, and determine whether the incremented k is greater than p. If yes, proceed to step S508; otherwise, continue to step S504.

[0303] S508. Aggregate the output vectors of the p second stack layers to obtain the output vector of the first second stack layer group.

[0304] Optionally, each second stack layer group includes p second stack layers; the decoding module 1206 is specifically configured as follows:

[0305] S602. Input one of the output vectors of the coding layer and the output vector of the (j-1)th second stack layer into the first second stack layer to obtain the output vector of the first second stack layer;

[0306] S604. Input one of the output vectors of the coding layer and the output vector of the (k-1)th second stack layer into the kth second stack layer to obtain the output vector of the kth second stack layer, where 2≤k≤p;

[0307] S606. Increment k by 1, and determine whether the incremented k is greater than p. If yes, proceed to step S608; otherwise, continue to step S604.

[0308] S608. Aggregate the output vectors of the p second stack layers to obtain the output vector of the j-th second stack layer group.

[0309] The translation apparatus provided in this application aggregates the output vectors corresponding to the first stack layer to obtain the encoding vector corresponding to each first stack layer, and determines the output vector of the encoding layer based on the encoding vector corresponding to each first stack layer; inputs the output vector of the encoding layer to the decoding layer to obtain the decoding vector of the decoding layer, and obtains the translated sentence based on the decoding vector of the decoding layer, thereby realizing the aggregation of the output vectors of the first stack layer and ultimately improving the performance of the translation model.

[0310] Secondly, the aggregation of output vectors between the first stack layers is not a simple concatenation, but rather an aggregation of output vectors between the first stack layers using linear layers, followed by fully connected layers and normalization layers to generate the final encoded vector, thereby achieving efficient aggregation of output vectors between the first stack layers.

[0311] Furthermore, the aggregation of output vectors between the second stack layers is not a simple concatenation, but rather the final encoded vector is obtained by using fully connected layers and normalization layers, so as to achieve efficient aggregation of output vectors between the second stack layers.

[0312] This application also discloses a training device for a translation model, the translation model including an encoding layer and a decoding layer, the encoding layer including a hidden layer and a first stack layers connected in sequence, wherein a is a positive integer greater than 1;

[0313] See Figure 13 The training device includes:

[0314] The output vector training module 1302 is configured to obtain the output vector corresponding to each first stack layer based on the first sentence in the input target corpus.

[0315] The encoding training module 1304 is configured to aggregate the output vectors corresponding to the first stack layer to obtain the encoding vectors corresponding to each first stack layer, and determine the output vector of the encoding layer based on the encoding vectors corresponding to each first stack layer.

[0316] The decoding training module 1306 is configured to input the output vector of the encoding layer and the translated second sentence corresponding to the first sentence into the decoding layer to obtain the decoding vector of the decoding layer, and to obtain the error of the decoding vector based on the decoding vector;

[0317] Training module 1308 is configured to continue training the translation model based on the error of the decoding vector until the training stopping condition is met.

[0318] Optionally, the decoding training module 1306 is specifically configured to: compare the decoded vector with a preset vector verification set to obtain the error of the decoded vector.

[0319] The training stopping condition includes: the rate of change of the error is less than the stability threshold.

[0320] Specifically, the output vector training module 1302 is configured as follows:

[0321] 1) Input the first sentence from the target corpus into the first hidden layer to generate the first vector;

[0322] 2) Input the first vector into the first stack layer to obtain the output vector of the first stack layer;

[0323] 3) Input the output vector of the (b-1)th first stack layer to the bth first stack layer to obtain the output vector corresponding to the bth first stack layer, where 2≤b≤a;

[0324] 4) Increment b by 1, and check if b after incrementing by 1 is greater than a. If yes, end the process; otherwise, continue with step 3.

[0325] Specifically, the encoding training module 1304 is configured to: aggregate the output vectors of the a first stack layer through a linear layer to obtain a linear vectors of the first stack layer; process the linear vectors of the a first stack layer through a fully connected layer to obtain a fully connected vector corresponding to each first stack layer; and process the fully connected vector corresponding to each first stack layer through a normalization layer to obtain the encoding vector corresponding to each first stack layer.

[0326] Specifically, the acquisition of the output vector of the second stack layer group has been described in detail in the previous embodiments, and will not be repeated here.

[0327] The training device for the translation model provided in this application aggregates the output vectors corresponding to the first stack layer to obtain the encoding vector corresponding to each first stack layer, and determines the output vector of the encoding layer based on the encoding vector corresponding to each first stack layer; the output vector of the encoding layer and the second sentence are both input to the decoding layer to obtain the decoding vector of the decoding layer, and the error of the decoding vector is obtained based on the decoding vector; the translation model is further trained based on the error of the decoding vector, so that the output vectors of the first stack layer can be effectively aggregated during the model training stage, which helps to improve the translation performance of the translation model during the usage stage.

[0328] One embodiment of this application also provides a chip that stores computer instructions, which, when executed by a processor, implement the steps of the translation method or the training method of the translation model as described above.

[0329] An embodiment of this application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the translation method or the training method of the translation model as described above.

[0330] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the above-described translation method or translation model training method. Details not described in detail in the technical solution of the storage medium can be found in the description of the technical solution of the above-described translation method or translation model training method.

[0331] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0332] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0333] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0334] The preferred embodiments disclosed above are merely illustrative of this application. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of this application, thereby enabling those skilled in the art to better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A translation method, characterized in that, The translation model includes an encoding layer and a decoding layer. The encoding layer includes a first stack layer connected in sequence, and the decoding layer includes m second stack layer groups connected in sequence, where a is a positive integer greater than 1 and m ≥ 2. The method includes: Based on the input statement to be translated, the output vector of the encoding layer is determined, wherein the output vector of the encoding layer includes the encoding vectors of the first stack layer at different sequential positions; The output vector of the encoding layer is input into m second stack layer groups respectively to obtain the output vector of each second stack layer group. This includes: selecting the encoding vectors of the last multiple first stack layers in sequence from the encoding vectors of the first stack layers in different sequential positions, and inputting the encoding vectors of the last multiple first stack layers in sequence into m second stack layer groups in different sequential positions respectively to obtain the output vector of each second stack layer group. The encoding vectors of the first stack layers in different sequential positions in the last multiple first stack layers in sequence are input into different second stack layer groups. The output vectors of m second stack layer groups are aggregated to obtain the decoding vector of the decoding layer, and the translated statement is obtained based on the decoding vector of the decoding layer.

2. The translation method as described in claim 1, characterized in that, The coding layer further includes a first hidden layer; Based on the input utterance to be translated, determine the output vector of the coding layer, including: S102. Input the statement to be translated into the first hidden layer to generate a first vector; S104. Input the first vector into the first stack layer to obtain the output vector of the first stack layer; S106. Input the output vector of the (b-1)th first stack layer to the bth first stack layer to obtain the output vector corresponding to the bth first stack layer, where 2≤b≤a; S108. Increment b by 1. Determine if b after incrementing by 1 is greater than a. If yes, end the process. If no, continue with step S106.

3. The translation method as described in claim 2, characterized in that, The translation model includes: a linear layer, a fully connected layer, and a normalization layer; Following S108, it also includes: The output vectors of the first stack layer are aggregated through a linear layer to obtain a linear vectors of the first stack layer. The linear vectors of the first stack layer are processed through a fully connected layer to obtain the fully connected vector corresponding to each first stack layer. The fully connected vector corresponding to each first stack layer is processed by a normalization layer to obtain the encoding vector corresponding to each first stack layer.

4. The translation method as described in claim 1, characterized in that, In the decoding layer, each second stack layer group includes at least two second stack layer groups, and each second stack layer group includes at least two second stack layers; The output vector of the coding layer is input into m second stack layer groups respectively, resulting in the output vector of each second stack layer group, including: The output vector of the coding layer is input into the at least two second stack layers respectively, and the output vector of the at least two second stack layers is obtained by the output vector of the at least two second stack layers grouped together. The output vector of each second stack layer group is obtained by using the output vectors of the at least two second stack layer groups.

5. The translation method as described in claim 1, characterized in that, The decoding layer includes a second hidden layer; The output vector of the coding layer is input into m second stack layer groups respectively, resulting in the output vector of each second stack layer group, including: S202. Input the output vector of the encoding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group; S204. Input the output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the ith second stack layer group to obtain the output vector of the ith second stack layer group; where 2≤i≤m; S206. Increment i by 1. Determine if the incremented i is greater than m. If yes, end the process. If no, continue with step S204.

6. The translation method as described in claim 5, characterized in that, The output vector of the coding layer is at least two; Step S202 includes: inputting at least one output vector of the encoding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group; Step S204 includes: inputting at least one output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the ith second stack layer group to obtain the output vector of the ith second stack layer group.

7. The translation method as described in claim 5, characterized in that, Each second stack layer group consists of n second stack layer groups, where n ≥ 2; Step S202 includes: S302. Input the output vector of the coding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group; S304. Input the output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group, where 2≤j≤n; S306. Increment j by 1, and determine whether j after incrementing by 1 is greater than n. If yes, proceed to step S308; otherwise, continue to step S304. S308. Aggregate the output vectors of the n second stack layer groups to obtain the output vector of the first second stack layer group.

8. The translation method as described in claim 7, characterized in that, The output vector of the coding layer is at least two; Step S302 includes: inputting at least one output vector of the encoding layer and the reference vector corresponding to the statement to be translated into the first second stack layer group to obtain the output vector of the first second stack layer group; Step S304 includes: inputting at least one output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group.

9. The translation method as described in claim 5, characterized in that, Each second stack layer group consists of n second stack layer groups, where n ≥ 2; Step S204 includes: S402. Input the output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the first second stack layer group to obtain the output vector of the first second stack layer group; S404. Input the output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group, where 2≤j≤n; S406. Increment j by 1. Determine if j after incrementing by 1 is greater than n. If yes, proceed to step S408. If no, continue to step S404. S408. Aggregate the output vectors of the n second stack layer groups to obtain the output vector of the i-th second stack layer group.

10. The translation method as described in claim 9, characterized in that, The output vector of the coding layer is at least two; Step S402 includes: inputting at least one output vector of the coding layer and the output vector of the (i-1)th second stack layer group into the first second stack layer group to obtain the output vector of the first second stack layer group; Step S404 includes: inputting at least one output vector of the coding layer and the output vector of the (j-1)th second stack layer group into the jth second stack layer group to obtain the output vector of the jth second stack layer group.

11. The translation method as described in claim 9, characterized in that, Each second stack layer group consists of p second stack layers, where p ≥ 2; Step S402 includes: S502. Input one of the output vectors of the coding layer and the output vector of the (i-1)th second stack layer group into the first second stack layer to obtain the output vector of the first second stack layer; S504. Input one of the output vectors of the coding layer and the output vector of the (k-1)th second stack layer into the kth second stack layer to obtain the output vector of the kth second stack layer, where 2≤k≤p; S506. Increment k by 1, and determine whether the incremented k is greater than p. If yes, proceed to step S508; otherwise, continue to step S504. S508. Aggregate the output vectors of the p second stack layers to obtain the output vector of the first second stack layer group.

12. The translation method as described in claim 9, characterized in that, Each second stack layer group consists of p second stack layers, where p ≥ 2; Step S404 includes: S602. Input one of the output vectors of the coding layer and the output vector of the (j-1)th second stack layer into the first second stack layer to obtain the output vector of the first second stack layer; S604. Input one of the output vectors of the coding layer and the output vector of the (k-1)th second stack layer into the kth second stack layer to obtain the output vector of the kth second stack layer, where 2≤k≤p; S606. Increment k by 1, and determine whether the incremented k is greater than p. If yes, proceed to step S608; otherwise, continue to step S604. S608. Aggregate the output vectors of the p second stack layers to obtain the output vector of the j-th second stack layer group.

13. The translation method as described in claim 1, characterized in that, The decoding layer further includes: a linear layer and a normalization layer; The output vectors of the m second stack layer groups are aggregated to obtain the decoding vector of the decoding layer, including: Aggregate the output vectors of m second stack layer groups via a linear layer; The result obtained by aggregation through the linear layer is normalized through the normalization layer to obtain the decoding vector of the decoding layer.

14. A method for training a translation model, characterized in that, The translation model includes an encoding layer and a decoding layer. The encoding layer includes a first stack layer connected in sequence, and the decoding layer includes m second stack layer groups connected in sequence, where a is a positive integer greater than 1 and m ≥ 2. The training method includes: Based on the first sentence in the input target corpus, the output vector of the coding layer is determined, wherein the output vector of the coding layer includes the coding vectors of the first stack layer at different sequential positions; The output vector of the encoding layer and the translated second statement corresponding to the first statement are both input into the m second stack layer groups included in the decoding layer to obtain the output vector of each second stack layer group. This includes: selecting the encoding vectors of the last multiple first stack layers in sequence from the encoding vectors of the first stack layers in different sequential positions, and inputting the encoding vectors of the last multiple first stack layers in sequence and the translated second statement corresponding to the first statement into the m second stack layer groups in different sequential positions to obtain the output vector of each second stack layer group. The encoding vectors of the first stack layers in different sequential positions in the last multiple first stack layers in sequence are input into different second stack layer groups. The output vectors of the m second stack layer groups are aggregated to obtain the decoding vector of the decoding layer, and the error of the decoding vector is obtained based on the decoding vector; The translation model continues to be trained based on the error of the decoded vector until the training stopping condition is met.

15. A translation device, characterized in that, The translation model is configured to include an encoding layer and a decoding layer. The encoding layer includes a sequentially connected first stack layers, and the decoding layer includes m groups of second stack layers, where m ≥ 2. The device includes: The coding layer output vector acquisition module is configured to determine the output vector of the coding layer based on the input statement to be translated, wherein the output vector of the coding layer includes the coding vectors of the first stack layer at different sequential positions; The decoding layer output vector acquisition module is configured to input the output vector of the encoding layer into m second stack layer groups respectively to obtain the output vector of each second stack layer group, including: selecting the encoding vectors of the last multiple first stack layers in sequence from the encoding vectors of the first stack layers in different sequential positions, and inputting the encoding vectors of the last multiple first stack layers in sequence into m second stack layer groups in different sequential positions respectively to obtain the output vector of each second stack layer group, wherein the encoding vectors of the first stack layers in different sequential positions in the last multiple first stack layers in sequence are input into different second stack layer groups; The aggregation module is configured to aggregate the output vectors of m second stack layer groups to obtain the decoding vector of the decoding layer, and to obtain the translated statement based on the decoding vector of the decoding layer.

16. A training device for a translation model, characterized in that, The translation model includes an encoding layer and a decoding layer. The encoding layer includes a first stack layer connected in sequence, and the decoding layer includes m groups of second stack layers, where a is a positive integer greater than 1 and m ≥ 2. The training device includes: The encoding layer output vector training module is configured to determine the output vector of the encoding layer based on the first sentence in the input target corpus, wherein the output vector of the encoding layer includes the encoding vectors of the first stack layer at different sequential positions; The decoding layer output vector training module is configured to input the output vector of the encoding layer and the translated second statement corresponding to the first statement into m second stack layer groups included in the decoding layer to obtain the output vector of each second stack layer group. This includes: selecting the encoding vectors of the last multiple first stack layers in sequence from the encoding vectors of the first stack layers in different sequential positions, and inputting the encoding vectors of the last multiple first stack layers in sequence and the translated second statement corresponding to the first statement into m second stack layer groups in different sequential positions to obtain the output vector of each second stack layer group. The encoding vectors of the first stack layers in different sequential positions in the last multiple first stack layers in sequence are input into different second stack layer groups. The aggregation training module is configured to aggregate the output vectors of m second stack layer groups to obtain the decoding vector of the decoding layer, and obtain the error of the decoding vector based on the decoding vector; The second training module is configured to continue training the translation model based on the error of the decoding vector until the training stopping condition is met.

17. A computing device, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that, When the processor executes the instructions, it implements the steps of the method according to any one of claims 1-13 or 14.

18. A computer-readable storage medium storing computer instructions, characterized in that, When executed by the processor, this instruction implements the steps of the method according to any one of claims 1-13 or 14.

Images