Translation method, device, readable medium and electronic device

Abstract

The disclosed embodiments relate to a translation method, device, readable medium and electronic device. The method comprises: obtaining a source text to be translated; inputting the source text into a pre-generated target model to obtain a target translation text output by the target model; wherein the language type corresponding to the target translation text is the target language; the target model comprises a translation model and a text vector set, the text vector set comprises a first hidden state vector, and a first target text corresponding to the first hidden state vector, the first hidden state vector is obtained by inputting a first sample text into an encoding model, the language type of the first sample text is the target language, the first target text is a subset or a complete set of the first sample text, the translation model is used to obtain a pending text vector of the source text, and translate the source text according to the pending text vector, the first hidden state vector and the first target text.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to a translation method, device, readable medium and electronic device. Background Art

[0002] With the advancement of computer technology, machine translation has become an important research topic in natural language text processing. Machine translation refers to the process of translating text in a source language into a semantically equivalent target language text through a computer or other electronic device. In recent years, neural machine translation (NMT) has become a new trend in machine translation. Compared with statistical machine translation, neural machine translation is relatively simple in terms of model. It mainly consists of two parts, an encoder and a decoder. The encoder represents the text in the source language as a high-dimensional vector after a series of neural network transformations. The decoder is responsible for re-decoding (translating) this high-dimensional vector into the target language text.

[0003] However, among related technologies, the translation quality of neural machine translation needs to be improved. Summary of the invention

[0004] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] According to a first aspect of an embodiment of the present disclosure, a translation method is provided, the method comprising:

[0006] Determine a source text to be translated; the language type corresponding to the source text is the source language;

[0007] Inputting the source text into a pre-generated target model to obtain a target translation text output by the target model; the language type corresponding to the target translation text is the target language;

[0008] The target model includes a translation model and a text vector set, the text vector set includes a first hidden state vector and a first target text corresponding to the first hidden state vector, the first hidden state vector is obtained by inputting a first sample text into an encoding model, the language type of the first sample text is the target language, the first target text is a subset or a complete set of the first sample text, and the translation model is used to obtain the text vector of the source text and translate the source text according to the text vector, the first hidden state vector and the first target text.

[0009] According to a second aspect of an embodiment of the present disclosure, a translation device is provided, the device comprising:

[0010] An acquisition module is used to acquire a source text to be translated; the language type corresponding to the source text is the source language;

[0011] A translation module, used for inputting the source text into a pre-generated target model to obtain a target translation text output by the target model; the language type corresponding to the target translation text is the target language;

[0012] The target model includes a translation model and a text vector set, the text vector set includes a first hidden state vector and a first target text corresponding to the first hidden state vector, the first hidden state vector is obtained by inputting a first sample text into an encoding model, the language type of the first sample text is the target language, the first target text is a subset or a complete set of the first sample text, the translation model is used to obtain a pending text vector of the source text, and translate the source text according to the pending text vector, the first hidden state vector and the first target text.

[0013] According to a third aspect of an embodiment of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the computer program is executed by a processing device, the steps of the method described in the first aspect of the present disclosure are implemented.

[0014] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0015] a storage device having a computer program stored thereon;

[0016] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect of the present disclosure.

[0017] The above technical solution is adopted to obtain the source text to be translated; the language type corresponding to the source text is the source language; the source text is input into the pre-generated target model to obtain the target translation text output by the target model; the language type corresponding to the target translation text is the target language; wherein the target model includes a translation model and a text vector set, the text vector set includes a first hidden state vector, and a first target text corresponding to the first hidden state vector, the first hidden state vector is obtained by inputting the first sample text into the encoding model, the language type of the first sample text is the target language, the first target text is a subset or a complete set of the first sample text, the translation model is used to obtain the pending text vector of the source text, and translate the source text according to the pending text vector, the first hidden state vector and the first target text. In this way, by constructing a text vector set through the encoding model, the accuracy of the text vector combination can be improved, the retrieval quality in the translation process can be improved, and thus the translation quality of the target model can be improved.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings:

[0020] Figure 1 The present invention is a flowchart of a translation method according to an exemplary embodiment.

[0021] Figure 2 A method for generating a text vector set is shown according to an exemplary embodiment.

[0022] Figure 3 It is a schematic diagram of a text vector set according to an exemplary embodiment.

[0023] Figure 4 The figure is a flowchart of another translation method according to an exemplary embodiment.

[0024] Figure 5 The invention is a flowchart of a method for generating a translation model according to an exemplary embodiment.

[0025] Figure 6 is a schematic diagram of a target model according to an exemplary embodiment.

[0026] Figure 7 The invention is a block diagram of a translation device according to an exemplary embodiment.

[0027] Figure 8 It is a block diagram of another translation device according to an exemplary embodiment.

[0028] Fig. 9 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0030] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0032] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise clearly indicated in the context, they should be understood as "one or more". In the description of the present disclosure, unless otherwise specified, "multiple" refers to two or more than two, and other quantifiers are similar; "at least one item", "one or more items" or similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one item a can represent any number of a; for another example, one or more items of a, b and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple; "and / or" is a kind of association relationship that describes the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural.

[0034] Although operations or steps are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood that it is required to perform these operations or steps in the specific order shown or in a serial order, or to perform all the operations or steps shown to obtain the desired results. In the embodiments of the present disclosure, these operations or steps can be performed in series; these operations or steps can also be performed in parallel; or some of these operations or steps can be performed.

[0035] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0036] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0037] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0038] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0039] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0040] At the same time, it is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0041] The present disclosure is described below in conjunction with specific embodiments.

[0042] First, the application scenario of the present disclosure is described. The present disclosure can be applied to machine translation scenarios, especially neural machine translation scenarios. Compared with statistical machine translation, neural machine translation is relatively simple in terms of model. It mainly includes two parts, one is an encoder and the other is a decoder. The encoder represents the text of the source language as a high-dimensional vector after a series of neural network transformations. The decoder is responsible for re-decoding (translating) this high-dimensional vector into the target language text.

[0043] In some embodiments of the present disclosure, in order to improve the translation quality of neural machine translation, retrieval enhanced neural machine translation (RE-NMT) can be used. RE-NMT can use an external retrieval mechanism to enhance the parameterized neural machine translation system. Since the model can access the original training corpus during reasoning, the expressive power of this type of model is greatly enhanced, which can improve the translation quality of neural machine translation to a certain extent.

[0044] Retrieval-enhanced neural machine translation needs to rely on a pre-built text vector set (such as a knowledge base) for retrieval. The quality of this text vector set directly affects the accuracy of retrieval and further affects the translation quality.

[0045] Figure 1is a flowchart of a translation method according to an exemplary embodiment. The method can be applied to electronic devices, which may include terminal devices, such as smart phones, smart wearable devices, smart speakers, smart tablets, PDAs (Personal Digital Assistants), CPEs (Customer Premise Equipment), personal computers, vehicle-mounted terminals, etc.; the electronic devices may also include servers, such as local servers or cloud servers. Figure 1 As shown, the method may include:

[0046] S101: Obtain a source text to be translated.

[0047] The language type corresponding to the source text is the source language (eg, Chinese, English, German, etc.).

[0048] It should be noted that the source text may be a word, a sentence, a paragraph or an article, and the present disclosure does not limit this.

[0049] S102: Input the source text into a pre-generated target model to obtain a target translation text output by the target model.

[0050] The language type corresponding to the target translation text is the target language (eg, English, German, Chinese, etc.). The target language is different from the source language. For example, the source text in Chinese can be translated into the target translation text in English.

[0051] The above-mentioned target model may include a translation model and a text vector set, the text vector set may include one or more first hidden state vectors, and a first target text corresponding to each first hidden state vector, the first hidden state vector is obtained by inputting a first sample text into an encoding model, the language type of the first sample text is the target language, and the first target text is a subset or a complete set of the first sample text. The translation model is used to obtain a pending text vector of a source text, and translate the source text according to the pending text vector, the first hidden state vector and the first target text.

[0052] In some embodiments, the encoding model and the translation model may have different structures.

[0053] For example, the translation model may include a neural machine translation model NMT, such as a neural machine translation model NMT based on Transformer. The encoding model may include a masked language model MLM (Masked Language Model), a denoising auto encoder DAE (Denoising Auto Encoder) or a unidirectional language model CLM (Causal Language Model).

[0054] In other embodiments, the encoding model may include an encoder in a translation model (eg, a neural machine translation model NMT), and the parameters of the encoding model are parameters learned and updated during a pre-training process, which are different from the parameters of the encoder in the translation model.

[0055] In some embodiments, the text vector set may be a knowledge base or a database. For example, the text vector set (or knowledge base) may include one or more pre-generated key-value pairs. The key-value pairs may be obtained by inputting the first sample text into the encoding model.

[0056] The above method is adopted to obtain the source text to be translated; the language type corresponding to the source text is the source language; the source text is input into the pre-generated target model to obtain the target translation text output by the target model; the language type corresponding to the target translation text is the target language; wherein the target model includes a translation model and a text vector set, the text vector set includes a first hidden state vector, and a first target text corresponding to the first hidden state vector, the first hidden state vector is obtained by inputting the first sample text into the encoding model, the language type of the first sample text is the target language, the first target text is a subset or a full set of the first sample text, the translation model is used to obtain the pending text vector of the source text, and translate the source text according to the pending text vector, the first hidden state vector and the first target text, and the encoding model is different from the translation model. In this way, the text vector set is obtained by constructing the encoding model, which can improve the accuracy of the text vector combination, improve the retrieval quality in the translation process, and thus improve the translation quality of the target model.

[0057] Figure 2 A method for generating a text vector set is shown according to an exemplary embodiment. Figure 2 As shown, the method may include:

[0058] S201: Obtain a first sample text.

[0059] The language type of the first sample text is the target language. For example, a text (such as a word, sentence, paragraph or article) whose language type is the target language can be obtained as the first sample text.

[0060] The first sample text may be one or more.

[0061] S202: Input the first sample text into an encoding model to obtain at least one first hidden state vector and a first target text corresponding to each first hidden state vector.

[0062] For example, for each first sample text, one or more decoding steps can be implemented through the encoding model, and a first hidden state vector can be generated in each decoding step. The first hidden state vector can be a hidden layer representation of the context of the first sample text. The first target text corresponding to the first hidden state vector can also be obtained in each decoding step. For example, the first target text can be the first target word that should be generated according to the context.

[0063] S203: Generate a text vector set according to the first hidden state vector and the first target text.

[0064] Figure 3 FIG. 1 is a schematic diagram showing a text vector set according to an exemplary embodiment. Figure 3 As shown, the text vector set 30 may include multiple key-value pairs, wherein the key in the key-value pair may be a first hidden state vector, for example, a hidden layer representation obtained according to the context of the first sample text; the value in the key-value pair may be a first target text corresponding to the first hidden state vector, for example, a first target word expected to be generated according to the translation context. Figure 3 In the example, the first sample text is "The sky is blue". After the first sample text is input into the coding model, it is processed through 5 decoding steps. Among them, the context of the first decoding step is " <bos> ”( <bos>can represent the beginning of a sentence), the first hidden state vector and the first target text (The) of the decoding step can be obtained through the encoding model; the context of the second decoding step is " <bos>The", through the encoding model, we can get the first hidden state vector and the first target text (sky) of the decoding step; the context of the third decoding step is " <bos>The sky", the first hidden state vector and the first target text (is) of the decoding step can be obtained through the encoding model; the context of the fourth decoding step is " <bos>Thesky is", through the encoding model, we can get the first hidden state vector and the first target text (blue) of the decoding step; the context of the 5th decoding step is" <bos>The sky is blue", the encoding model can be used to obtain the first hidden state vector and the first target text of the decoding step ( <eos>),in,

[0065] <eos>It can represent the end of a sentence. In this way, we can get Figure 3 The key-value pairs X1, X2, X3, X4 and X5 are shown. In this way, the key-value pairs can be used as a text vector set 30.

[0066] It should be noted that Figure 3 Only an illustration of obtaining a text vector set based on a first sample text is given. It can be understood that for each first sample text among multiple first sample texts, one or more key-value pairs can be obtained based on the above steps, thereby obtaining a text vector set including more key-value pairs.

[0067] The encoding model may include a masked language model MLM, a denoising autoencoder DAE or a unidirectional language model CLM. For example:

[0068] In some embodiments, when the encoding model includes an MLM model, the first sample text can be input into the MLM model to obtain the top-level context representation, and the context representation is used as the first hidden state vector, and the text input at the bottom layer is used as the first target text. For example, the first hidden state vector and the first target text corresponding to the first hidden state vector can be obtained by the following formula (1).

[0069] (k, v) = (f MLM (Y) t ,y t ) (1)

[0070] Among them, (k, v) represents the key-value pair in the text vector set, k represents the key value, that is, the first hidden state vector, v represents the first target text corresponding to the first hidden state vector, Y represents the first sample text, and f MLM (Y) t Indicates that the first sample text is input into the MLM model, and the context representation output at the tth decoding step, y t represents the undetermined text expected to be obtained at the t-th decoding step according to the input first sample text, for example, the undetermined text may be the target word in the first sample text. In this way, the undetermined text may be used as the first target text, and the context representation may be used as the first hidden state vector.

[0071] In some other embodiments, when the encoding model includes a DAE model, the first sample text can be input into the DAE model, and the first hidden state vector and the first target text corresponding to the first hidden state vector can be obtained by the following formula (2).

[0072] (k,v)=(F DAE (Y,y ＜t ), y t ) (2)

[0073] Among them, (k, v) represents the key-value pair in the text vector set, k represents the key value, that is, the first hidden state vector, v represents the first target text corresponding to the first hidden state vector, Y represents the first sample text, and f DAE (Y,Y ＜t ) represents the input of the first sample text into the DAE model, and the context representation output at the tth decoding step, y t represents the undetermined text expected to be obtained at the t-th decoding step according to the input first sample text, for example, the undetermined text may be the target word in the first sample text. In this way, the undetermined text may be used as the first target text, and the context representation may be used as the first hidden state vector.

[0074] In some other embodiments, when the encoding model includes a CLM model, the first sample text can be input into the CLM model, and the first hidden state vector and the first target text corresponding to the first hidden state vector can be obtained by the following formula (3) using Teacher Forcing Decoding.

[0075] (k, v) = (f CLM (Y ＜t ), y t ) (3)

[0076] Among them, (k, v) represents the key-value pair in the text vector set, k represents the key value, that is, the first hidden state vector, v represents the first target text corresponding to the first hidden state vector, Y represents the first sample text, and f CLM (Y ＜t ) represents the input of the first sample text into the CLM model, and the context representation output at the t-th decoding step, y t represents the undetermined text expected to be obtained at the t-th decoding step according to the input first sample text, for example, the undetermined text may be the target word in the first sample text. In this way, the undetermined text may be used as the first target text, and the context representation may be used as the first hidden state vector.

[0077] In order to improve the translation quality, the disclosed embodiment also provides a model construction and translation method for performing translation and retrieval simultaneously.

[0078] Figure 4 FIG. 1 is a flowchart of another translation method according to an exemplary embodiment. Figure 4 As shown, the method may include:

[0079] S101: Obtain a source text to be translated.

[0080] S1021. Input the source text into the translation model in the target model to obtain a pending text vector output by the translation model.

[0081] The pending text vector includes a pending translation vector and a first search vector. The pending translation vector and the first search vector may be the same or different.

[0082] In some examples, the translation model may be a model based on a Transformer structure, through which the source text may be encoded and decoded to obtain a pending translation vector, and the pending translation vector may also be used as the first retrieval vector.

[0083] S1022. Determine at least one second target text from the first target text in the text vector set according to the first retrieval vector and the first hidden state vector.

[0084] In this step, a preset number of neighbor vectors can be determined from the first hidden state vector in the text vector set based on the similarity between the first retrieval vector and the first hidden state vector, and the first target text corresponding to the neighbor vector is used as the above-mentioned second target text.

[0085] For example, the K first hidden state vectors with the highest similarity may be used as the neighbor vectors, where K is a preset number.

[0086] For another example, the K first hidden state vectors with the highest similarity can be selected from the first hidden state vectors whose similarity is greater than or equal to the preset similarity threshold as the neighbor vectors; if the number of first hidden state vectors whose similarity is greater than or equal to the preset similarity threshold is less than K, then all first hidden state vectors whose similarity is greater than or equal to the preset similarity threshold are taken as the neighbor vectors.

[0087] For example, the first search vector q t , retrieve k nearest neighbors (k1, v1), (k1, v2), ..., (k k , v k ), and retrieve V1, V2, ..., V k As the second target text mentioned above.

[0088] S1023. Encode the second target text according to the text vector matrix corresponding to the translation model to obtain a target text vector.

[0089] The text vector matrix may be a word vector matrix of a decoder of a translation model. By encoding each second target text through the text vector matrix, a target text vector may be obtained.

[0090] S1024: Determine a target translation text according to the pending translation vector and the target text vector.

[0091] In some embodiments, when there are multiple target texts, the target translation text may be determined by the following steps:

[0092] Firstly, multiple target text vectors are aggregated through the preset self-attention mechanism to obtain an aggregated text vector.

[0093] The preset self-attention mechanism can be characterized by the following preset formula. For example, the aggregated text vector can be obtained by the following formula (4):

[0094]

[0095] Among them, m represents the aggregated text vector, q t represents the first retrieval vector obtained in the t-th decoding step, and the subscript t represents the t-th decoding step (or time step), Represents the first search vector q t The vector matrix obtained after transposition, K represents the number of target text vectors, e j represents the jth target text vector, e i represents the i-th target text vector, and exp represents the exponential function with the natural constant e as the base.

[0096] Secondly, the target translation vector is obtained according to the pending translation vector and the aggregated text vector.

[0097] For example, the target translation vector can be obtained by the following formula (5) and formula (6):

[0098] g=σ(W1h t +W2m+b) (5)

[0099] z=g⊙m+(1-g)⊙h t (6)

[0100] Among them, z represents the target translation vector, g represents the target translation weight, m ​​represents the above-mentioned aggregated text vector, and h t represents the undetermined translation vector obtained at the tth decoding step after the source text is input into the translation model in the target model, m represents the above-mentioned aggregated text vector, W1 represents the first weight parameter corresponding to the undetermined translation vector, W2 represents the second weight parameter corresponding to the aggregated text vector, b represents the bias parameter, W1, W2 and b can be parameters obtained through training and learning, σ represents the sigmoid function (also called the Logistic function), which is used for the output of hidden layer neurons and can map a real number (or a value in a vector) to a value in the interval (0,1), and ⊙ represents bit-by-bit multiplication.

[0101] Finally, the target translation text is obtained according to the target translation vector.

[0102] In some embodiments, a softmax layer may be used to convert the target translation vector into a prediction probability, and the target translation text may be determined based on the prediction probability.

[0103] For example, the predicted probability of the candidate word can be calculated by the following formula:

[0104] According to the target translation vector and the text vector matrix corresponding to the translation model, the prediction probability of one or more pending texts is obtained. For example, the prediction probability can be generated by the following formula (7):

[0105] p(y t |X,Y ＜t )=softmax(z;W e ) (7)

[0106] Among them, y t represents the pending text of the tth decoding step, X represents the input source text, and Y ＜t The context representation output at the t-th decoding step, p(y t |X,Y ＜t ) represents the predicted probability, that is, the pending text y t The probability of being the target translation text at the t-th decoding step, z represents the target translation vector, W e Represents a text vector matrix (e.g. a word vector matrix).

[0107] In some embodiments, the text with the highest predicted probability among the multiple pending texts may be used as the target translation text.

[0108] In this way, through the above steps, the source text can be translated by using a text vector set to assist the translation model, thereby improving the model translation quality.

[0109] In some embodiments of the present disclosure, in order to enable a translation model (e.g., an NMT model) to retrieve reasonable text from a pre-generated text vector set (e.g., a knowledge base), the model may be trained using an alignment target based on nearest neighbor comparison during the process of generating the translation model.

[0110] Figure 5 FIG. 1 is a flowchart of a method for generating a translation model according to an exemplary embodiment. Figure 5 As shown, the method may include:

[0111] S501, obtaining a plurality of second sample texts and a second hidden state vector corresponding to each second sample text.

[0112] In some embodiments, the second sample text may be a subset or a complete set of the first sample text. The second hidden state vector corresponding to the second sample text may be obtained in the process of generating the text vector set, and no repeated calculation is required.

[0113] In some other embodiments, the second sample text may also include text other than the first sample text.

[0114] In some embodiments, the second sample text may be input into the encoding model to obtain a second hidden state vector output by the encoding model.

[0115] S502, looping through the model training steps until it is determined that the trained target neural network model meets a preset stop iteration condition, and using the trained target neural network model as a translation model.

[0116] The model training step may include:

[0117] S11. Input the second sample text into the target neural network model to obtain a second retrieval vector output by the target neural network model.

[0118] S12. According to the second search vector, determine, from the first hidden state vectors of the text vector set, a third hidden state vector having the highest similarity to the second search vector.

[0119] In some embodiments, a preset number of first hidden state vectors having the highest similarity to the second search vector may be used as the third hidden state vector. The preset number may be any pre-set positive integer, for example, the preset number may be 2, 5 or 10.

[0120] In some other embodiments, the first hidden state vector whose similarity with the second search vector is greater than or equal to a preset similarity threshold may be used as the third hidden state vector. The preset similarity threshold may be preset based on experience.

[0121] In some other embodiments, a preset number of first hidden state vectors with the highest similarity may be selected from the first hidden state vectors whose similarity with the second retrieval vector is greater than or equal to a preset similarity threshold as the third hidden state vector.

[0122] It should be noted that the third hidden state vector determined in this step may include the second hidden state vector corresponding to the second sample text, or may not include the second hidden state vector.

[0123] In some embodiments, if the third hidden state vector does not include the second hidden state vector, the second hidden state vector may be added to the third hidden state vector.

[0124] S13. According to the second retrieval vector, the second hidden state vector and the third hidden state vector, a target loss value is calculated by using a preset loss function.

[0125] In some embodiments, the second hidden state vector can be used as a positive example of the second retrieval vector, and other third hidden state vectors except the second hidden state vector can be used as negative examples of the second retrieval vector. Based on the positive examples and negative examples, the target loss value is calculated by a preset loss function.

[0126] It should be noted that the smaller the target loss value is, the higher the similarity between the second retrieval vector and the second hidden state vector is, and the lower the similarity between the second retrieval vector and other third hidden state vectors except the second hidden state vector is.

[0127] For example, the preset loss function may include the following formula (8):

[0128]

[0129] Among them, X represents the input source text, Y represents the target translation text, represents the target loss value, |Y| represents the total number of decoding steps to decode the source text to obtain the target translation text, the total number of decoding steps can be the number of words in the source text, each word corresponds to a decoding step, q t represents the first retrieval vector obtained in the t-th decoding step, and the subscript t represents the t-th decoding step (or time step), Represents the first search vector q t The vector matrix obtained after transposition, k represents the second hidden state vector (positive example), κ represents the hidden state vector set composed of the third hidden state vector and the second hidden state vector, Represents each vector in the set of hidden state vectors, τ represents a temperature parameter (also called a temperature hyperparameter, used to adjust the distribution), and the temperature parameter can be any preset value less than 1, for example, it can be 0.01 or 0.07.

[0130] S14. When it is determined according to the target loss value that the target neural network model does not satisfy the preset stop iteration condition, the parameters of the target neural network model are updated according to the target loss value to obtain a trained target neural network model, and the trained target neural network model is used as a new target neural network model.

[0131] The preset condition for stopping iteration may include that the target loss value is less than or equal to a preset loss threshold, or that the change value of the target loss value within the most recent preset number of iterations is less than a preset change threshold. It may also be a condition for stopping iteration commonly used in related technologies, and the present disclosure does not limit this.

[0132] In addition, if it is determined according to the target loss value that the target neural network model satisfies the preset stop iteration condition, the model training step can be stopped and the trained target neural network model can be used as the translation model.

[0133] In this way, through the above steps, a translation model can be generated, and the translation model can accurately retrieve the text vector set generated based on the encoding model.

[0134] Figure 6 is a schematic diagram of a target model according to an exemplary embodiment. Figure 6 As shown, the target model may include a translation model 60 and a text vector set 30, and the translation model 60 may include an encoder 61 and a decoder 62. Figure 6 As shown, the source text sample of the language type is German is "Der Himmel ist blau", and the second sample text of the speech type corresponding to the source text sample is "The sky is blue", and the second sample text has the same semantics as the source text. After the source text sample is input into the encoder 61, and the encoding vector output by the encoder is input into the decoder 62, the undetermined translation vector and the first search vector output by the decoder at each decoding step can be obtained. For example, the undetermined translation vectors output in the 1st to 4th decoding steps are respectively and The first search vectors outputted in the 1st to 4th decoding steps are q1, q2, q3 and q4 respectively.

[0135] The text vector set 30 may include key-value pairs generated according to a plurality of first sample texts. For example, the first sample texts may include "The sky is blue", "I like blue" and "The apple is red". The key-value pairs may include

[0136] According to the first search vector q4, the first hidden state vector closest to the text vector set is retrieved and determined and As the third hidden state vector. According to the second sample text, the second hidden state vector can be determined as During the training process, the preset loss function can make the first retrieval vector q4 output by the translation model and the hidden state vector The similarity of the first retrieval vector q4 and the hidden state vector q4 increases or the vector distance increases (Pull), and the first retrieval vector q4 and the hidden state vector q4 increase. and The similarity becomes smaller or the vector distance becomes larger (Push), so that the translation model can retrieve the accurate first hidden state vector from the vector text set, improve the retrieval accuracy, and thus improve the translation quality.

[0137] Figure 7 is a block diagram of a translation device 1100 according to an exemplary embodiment. Figure 7 As shown, the device 1100 may include:

[0138] The acquisition module 1101 is used to acquire a source text to be translated; the language type corresponding to the source text is the source language;

[0139] The translation module 1102 is used to input the source text into a pre-generated target model to obtain a target translation text output by the target model; the language type corresponding to the target translation text is the target language;

[0140] Among them, the target model includes a translation model and a text vector set, the text vector set includes a first hidden state vector, and a first target text corresponding to the first hidden state vector, the first hidden state vector is obtained by inputting a first sample text into an encoding model, the language type of the first sample text is the target language, the first target text is a subset or a complete set of the first sample text, the translation model is used to obtain a pending text vector of the source text, and translate the source text according to the pending text vector, the first hidden state vector and the first target text, and the encoding model is different from the translation model.

[0141] Figure 8 is a block diagram of another translation device 1100 according to an exemplary embodiment. Figure 8 As shown, the device 1100 may further include:

[0142] The generation module 1103 is used to obtain the first sample text; input the first sample text into the encoding model to obtain at least one first hidden state vector and the first target text corresponding to each first hidden state vector; and generate the text vector set according to the first hidden state vector and the first target text.

[0143] According to one or more embodiments of the present disclosure, the translation model includes a Transformer-based neural machine translation model NMT, and the encoding model includes a masked language model MLM, a denoising autoencoder DAE, or a unidirectional language model CLM.

[0144] According to one or more embodiments of the present disclosure, the translation module 1102 is used to input the source text into the translation model in the target model to obtain a pending text vector output by the translation model, wherein the pending text vector includes a pending translation vector and a first retrieval vector; determine at least one second target text from the first target text in the text vector set according to the first retrieval vector and the first hidden state vector; encode the second target text according to the text vector matrix corresponding to the translation model to obtain a target text vector; and determine the target translation text according to the pending translation vector and the target text vector.

[0145] According to one or more embodiments of the present disclosure, the translation module 1102 is used to, when there are multiple target texts, aggregate the multiple target text vectors to obtain an aggregated text vector through a preset self-attention mechanism; obtain a target translation vector according to the pending translation vector and the aggregated text vector; and generate the target translation text according to the target translation vector.

[0146] According to one or more embodiments of the present disclosure, the generating module 1103 is further used to obtain a plurality of second sample texts and a second hidden state vector corresponding to each second sample text; loop through the model training steps until it is determined that the trained target neural network model meets a preset stop iteration condition, and use the trained target neural network model as the translation model;

[0147] Wherein, the model training step includes:

[0148] Inputting the second sample text into a target neural network model to obtain a second retrieval vector output by the target neural network model;

[0149] According to the second search vector, determining a third hidden state vector having the highest similarity to the second search vector from the first hidden state vectors of the text vector set;

[0150] Obtaining a target loss value by calculating a preset loss function according to the second retrieval vector, the second hidden state vector, and the third hidden state vector;

[0151] When it is determined according to the target loss value that the target neural network model does not satisfy the preset stop iteration condition, the parameters of the target neural network model are updated according to the target loss value to obtain a trained target neural network model, and the trained target neural network model is used as a new target neural network model.

[0152] According to one or more embodiments of the present disclosure, the generating module 1103 is further configured to input the second sample text into the encoding model to obtain the second hidden state vector output by the encoding model.

[0153] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0154] Reference below Fig. 9 , which shows a schematic diagram of the structure of an electronic device 2000 (e.g., a terminal device or a server) suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The server in the embodiment of the present disclosure may include, but is not limited to, local servers, cloud servers, single servers, distributed servers, etc. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0155] like Fig. 9 As shown, the electronic device 2000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 2001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 2002 or a program loaded from a storage device 2008 into a random access memory (RAM) 2003. In the RAM 2003, various programs and data required for the operation of the electronic device 2000 are also stored. The processing device 2001, the ROM 2002, and the RAM 2003 are connected to each other via a bus 2004. An input / output (I / O) interface 2005 is also connected to the bus 2004.

[0156] Typically, the following devices may be connected to the input / output interface 2005: an input device 2006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 2007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 2008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 2009. The communication device 2009 may allow the electronic device 2000 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 2000 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0157] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 2009, or installed from the storage device 2008, or installed from the ROM 2002. When the computer program is executed by the processing device 2001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0158] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. Computer-readable storage media may include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0159] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0160] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0161] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a source text to be translated; the language type corresponding to the source text is the source language; inputs the source text into a pre-generated target model to obtain a target translation text output by the target model; the language type corresponding to the target translation text is the target language; wherein the target model includes a translation model and a text vector set, the text vector set includes a first hidden state vector, and a first target text corresponding to the first hidden state vector, the first hidden state vector is obtained by inputting a first sample text into an encoding model, the language type of the first sample text is the target language, the first target text is a subset or a full set of the first sample text, the translation model is used to obtain a pending text vector of the source text, and translate the source text according to the pending text vector, the first hidden state vector and the first target text, and the encoding model is different from the translation model.

[0162] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0163] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0164] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a module does not limit the module itself in some cases. For example, an acquisition module may also be described as a "module for acquiring a source text to be translated".

[0165] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0166] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0167] According to one or more embodiments of the present disclosure, a translation method is provided, the method comprising:

[0168] Obtaining a source text to be translated; the language type corresponding to the source text is the source language;

[0169] Inputting the source text into a pre-generated target model to obtain a target translation text output by the target model; the language type corresponding to the target translation text is the target language;

[0170] Among them, the target model includes a translation model and a text vector set, the text vector set includes a first hidden state vector, and a first target text corresponding to the first hidden state vector, the first hidden state vector is obtained by inputting a first sample text into an encoding model, the language type of the first sample text is the target language, the first target text is a subset or a complete set of the first sample text, the translation model is used to obtain a pending text vector of the source text, and translate the source text according to the pending text vector, the first hidden state vector and the first target text, and the encoding model is different from the translation model.

[0171] According to one or more embodiments of the present disclosure, the text vector set is generated in the following manner:

[0172] Obtaining the first sample text;

[0173] Inputting the first sample text into the encoding model to obtain at least one first hidden state vector and a first target text corresponding to each first hidden state vector;

[0174] The text vector set is generated according to the first hidden state vector and the first target text.

[0175] According to one or more embodiments of the present disclosure, the translation model includes a Transformer-based neural machine translation model NMT, and the encoding model includes a masked language model MLM, a denoising autoencoder DAE, or a unidirectional language model CLM.

[0176] According to one or more embodiments of the present disclosure, the step of inputting the source text into a pre-generated target model to obtain a target translation text output by the target model includes:

[0177] Inputting the source text into the translation model in the target model to obtain a pending text vector output by the translation model, wherein the pending text vector includes a pending translation vector and a first search vector;

[0178] Determine at least one second target text from the first target text in the text vector set according to the first retrieval vector and the first hidden state vector;

[0179] encoding the second target text according to the text vector matrix corresponding to the translation model to obtain a target text vector;

[0180] The target translation text is determined according to the pending translation vector and the target text vector.

[0181] According to one or more embodiments of the present disclosure, determining the target translation text according to the pending translation vector and the target text vector includes:

[0182] In the case where there are multiple target texts, aggregating the multiple target text vectors by a preset self-attention mechanism to obtain an aggregated text vector;

[0183] Acquire a target translation vector according to the pending translation vector and the aggregated text vector;

[0184] The target translation text is generated according to the target translation vector.

[0185] According to one or more embodiments of the present disclosure, the translation model is generated in the following manner:

[0186] Obtain multiple second sample texts and a second hidden state vector corresponding to each second sample text;

[0187] The model training steps are executed cyclically until it is determined that the trained target neural network model meets a preset stop iteration condition, and the trained target neural network model is used as the translation model;

[0188] Wherein, the model training step includes:

[0189] Inputting the second sample text into a target neural network model to obtain a second retrieval vector output by the target neural network model;

[0190] According to the second search vector, determining a third hidden state vector having the highest similarity to the second search vector from the first hidden state vectors of the text vector set;

[0191] Obtaining a target loss value by calculating a preset loss function according to the second retrieval vector, the second hidden state vector, and the third hidden state vector;

[0192] When it is determined according to the target loss value that the target neural network model does not satisfy the preset stop iteration condition, the parameters of the target neural network model are updated according to the target loss value to obtain a trained target neural network model, and the trained target neural network model is used as a new target neural network model.

[0193] According to one or more embodiments of the present disclosure, the second hidden state vector corresponding to the second sample text is obtained in the following manner:

[0194] The second sample text is input into the encoding model to obtain the second hidden state vector output by the encoding model.

[0195] According to one or more embodiments of the present disclosure, a translation device is provided, the device comprising:

[0196] An acquisition module is used to acquire a source text to be translated; the language type corresponding to the source text is the source language;

[0197] A translation module, used for inputting the source text into a pre-generated target model to obtain a target translation text output by the target model; the language type corresponding to the target translation text is the target language;

[0198] Among them, the target model includes a translation model and a text vector set, the text vector set includes a first hidden state vector, and a first target text corresponding to the first hidden state vector, the first hidden state vector is obtained by inputting a first sample text into an encoding model, the language type of the first sample text is the target language, the first target text is a subset or a complete set of the first sample text, the translation model is used to obtain a pending text vector of the source text, and translate the source text according to the pending text vector, the first hidden state vector and the first target text, and the encoding model is different from the translation model.

[0199] According to one or more embodiments of the present disclosure, the device further includes:

[0200] A generation module is used to obtain the first sample text; input the first sample text into the encoding model to obtain at least one of the first hidden state vectors and the first target text corresponding to each of the first hidden state vectors; and generate the text vector set according to the first hidden state vector and the first target text.

[0201] According to one or more embodiments of the present disclosure, the translation model includes a Transformer-based neural machine translation model NMT, and the encoding model includes a masked language model MLM, a denoising autoencoder DAE, or a unidirectional language model CLM.

[0202] According to one or more embodiments of the present disclosure, the translation module is used to input the source text into the translation model in the target model to obtain a pending text vector output by the translation model, wherein the pending text vector includes a pending translation vector and a first retrieval vector; determine at least one second target text from the first target text in the text vector set according to the first retrieval vector and the first hidden state vector; encode the second target text according to the text vector matrix corresponding to the translation model to obtain a target text vector; and determine the target translation text according to the pending translation vector and the target text vector.

[0203] According to one or more embodiments of the present disclosure, the translation module is used to, when there are multiple target texts, aggregate the multiple target text vectors to obtain an aggregated text vector through a preset self-attention mechanism; obtain a target translation vector based on the pending translation vector and the aggregated text vector; and generate the target translation text based on the target translation vector.

[0204] According to one or more embodiments of the present disclosure, the generation module is further used to obtain a plurality of second sample texts and a second hidden state vector corresponding to each second sample text; looping through the model training steps until it is determined that the trained target neural network model meets a preset stop iteration condition, and using the trained target neural network model as the translation model;

[0205] Wherein, the model training step includes:

[0206] Inputting the second sample text into a target neural network model to obtain a second retrieval vector output by the target neural network model;

[0207] According to the second search vector, determining a third hidden state vector having the highest similarity to the second search vector from the first hidden state vectors of the text vector set;

[0208] Obtaining a target loss value by calculating a preset loss function according to the second retrieval vector, the second hidden state vector, and the third hidden state vector;

[0209] When it is determined according to the target loss value that the target neural network model does not satisfy the preset stop iteration condition, the parameters of the target neural network model are updated according to the target loss value to obtain a trained target neural network model, and the trained target neural network model is used as a new target neural network model.

[0210] According to one or more embodiments of the present disclosure, the generating module is further used to input the second sample text into the encoding model to obtain the second hidden state vector output by the encoding model.

[0211] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0212] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0213] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.< / eos> < / eos> < / bos> < / bos> < / bos> < / bos> < / bos> < / bos>

Claims

1. A translation method, characterized in that: The method comprises: Obtaining a source text to be translated; the language type corresponding to the source text is the source language; Inputting the source text into a pre-generated target model to obtain a target translation text output by the target model; the language type corresponding to the target translation text is the target language; The target model includes a translation model and a text vector set, the text vector set includes a first hidden state vector and a first target text corresponding to the first hidden state vector, the first hidden state vector is obtained by inputting a first sample text into an encoding model, the language type of the first sample text is the target language, the first target text is a subset or a complete set of the first sample text, the translation model is used to obtain a pending text vector of the source text, and translate the source text according to the pending text vector, the first hidden state vector and the first target text; The step of inputting the source text into a pre-generated target model to obtain a target translation text output by the target model comprises: Inputting the source text into the translation model in the target model to obtain a pending text vector output by the translation model, wherein the pending text vector includes a pending translation vector and a first search vector; Determine at least one second target text from the first target text in the text vector set according to the first retrieval vector and the first hidden state vector; encoding the second target text according to the text vector matrix corresponding to the translation model to obtain a target text vector; Determining the target translation text according to the pending translation vector and the target text vector; The determining the target translation text according to the pending translation vector and the target text vector comprises: In the case where there are multiple target texts, aggregating the multiple target text vectors by a preset self-attention mechanism to obtain an aggregated text vector; Acquire a target translation vector according to the pending translation vector and the aggregated text vector; The target translation text is generated according to the target translation vector.

2. The method according to claim 1, characterized in that The text vector set is generated in the following way: Obtaining the first sample text; Inputting the first sample text into the encoding model to obtain at least one first hidden state vector and a first target text corresponding to each first hidden state vector; The text vector set is generated according to the first hidden state vector and the first target text.

3. The method according to claim 1, characterized in that The translation model includes a Transformer-based neural machine translation model NMT, and the encoding model includes a masked language model MLM, a denoising autoencoder DAE or a unidirectional language model CLM.

4. The method according to claim 1, characterized in that: The translation model is generated in the following way: Obtain multiple second sample texts and a second hidden state vector corresponding to each second sample text; The model training steps are executed cyclically until it is determined that the trained target neural network model meets a preset stop iteration condition, and the trained target neural network model is used as the translation model; Wherein, the model training step includes: Inputting the second sample text into a target neural network model to obtain a second retrieval vector output by the target neural network model; According to the second search vector, determining a third hidden state vector having the highest similarity to the second search vector from the first hidden state vectors of the text vector set; Obtaining a target loss value by calculating a preset loss function according to the second retrieval vector, the second hidden state vector, and the third hidden state vector; When it is determined according to the target loss value that the target neural network model does not satisfy the preset stop iteration condition, the parameters of the target neural network model are updated according to the target loss value to obtain a trained target neural network model, and the trained target neural network model is used as a new target neural network model.

5. The method according to claim 4, characterized in that The second hidden state vector corresponding to the second sample text is obtained in the following manner: The second sample text is input into the encoding model to obtain the second hidden state vector output by the encoding model.

6. A translation device, characterized in that: The device comprises: An acquisition module is used to acquire a source text to be translated; the language type corresponding to the source text is the source language; A translation module, used for inputting the source text into a pre-generated target model to obtain a target translation text output by the target model; the language type corresponding to the target translation text is the target language; The target model includes a translation model and a text vector set, the text vector set includes a first hidden state vector and a first target text corresponding to the first hidden state vector, the first hidden state vector is obtained by inputting a first sample text into an encoding model, the language type of the first sample text is the target language, the first target text is a subset or a complete set of the first sample text, the translation model is used to obtain a pending text vector of the source text, and translate the source text according to the pending text vector, the first hidden state vector and the first target text; The translation module is used to input the source text into the translation model in the target model to obtain a pending text vector output by the translation model, wherein the pending text vector includes a pending translation vector and a first retrieval vector; determine at least one second target text from the first target text in the text vector set according to the first retrieval vector and the first hidden state vector; encode the second target text according to the text vector matrix corresponding to the translation model to obtain a target text vector; determine the target translation text according to the pending translation vector and the target text vector; Determining the target translation text according to the pending translation vector and the target text vector includes: when there are multiple target texts, aggregating the multiple target text vectors to obtain an aggregated text vector through a preset self-attention mechanism; obtaining a target translation vector according to the pending translation vector and the aggregated text vector; and generating the target translation text according to the target translation vector.

7. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method according to any one of claims 1 to 5 are implemented.

8. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 5.

Images