A model training method, related apparatus and device

Abstract

Embodiments of the present application disclose a model training method, related device and equipment, which are used to improve the coherence and accuracy of a model generated translation. The method comprises: obtaining a source text set, a target text set and a training sample pair, taking the source text set, the target text set and an expected target text as inputs of a dialogue understanding model, outputting a first text sequence, a second text sequence and a text similarity, the first text sequence and the second text sequence corresponding to a second language, and the text similarity representing a correlation degree between the expected target text and the target text set, taking the source text set and the training sample pair as inputs of a text translation model, obtaining a third text sequence through the text translation model, the third text sequence corresponding to the second language, and updating model parameters of the text translation model according to the first text sequence, the second text sequence, the text similarity, the third text sequence and the expected target text.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a model training method, related apparatus and device. BACKGROUND

[0002] With the rapid development of Internet technology and digital technology, dialogue generation based on neural networks has attracted widespread research interest in open-domain dialogue systems. Since multi-turn dialogue is widely used in daily life, the translation of multi-turn dialogue also puts forward stricter requirements for context consistency.

[0003] Traditional dialogue translation methods usually treat the sentences of each turn in the dialogue history as independent sentences, and then perform separate semantic encoding. However, since multi-turn dialogue arises from a coherent and consistent communication process, the sentences of each turn involved are related to each other, so there is an explicit reply relationship between adjacent sentences in the dialogue history. If these sentences are encoded separately without considering their semantic relationship, the coherence relationship at the chapter level in the dialogue history may not be captured, resulting in semantic discontinuity or incoherence in the obtained dialogue translation text, thereby reducing the accuracy of multi-turn dialogue translation. SUMMARY

[0004] Embodiments of the present application provide a model training method, related apparatus and device, which is used for dialogue understanding model as an auxiliary training task of a text translation model, and performs joint training on the text translation model. The dialogue understanding model can continuously learn the characteristics of text language changes and the correlation degree between texts, thereby enhancing the coherence learning ability of the text translation model between texts, improving the training accuracy of the text translation model, and improving the coherence and accuracy of the translation text generated by the text translation model.

[0005] Therefore, in one aspect, the present application provides a model training method, comprising:

[0006] obtaining a source text set, a target text set and a training sample pair, wherein the source text set includes source text corresponding to a first language, the target text set includes target text corresponding to a second language, and the training sample pair includes initial source text corresponding to the first language and expected target text corresponding to the second language;

[0007] inputting the source text set, the target text set and the expected target text into a dialogue understanding model, and outputting a first text sequence, a second text sequence and a text similarity from the dialogue understanding model, wherein the first text sequence corresponds to the second language, the second text sequence corresponds to the second language, and the text similarity represents the correlation degree between the expected target text and the target text set;

[0008] The source text set and the training sample pair are input into the text translation model, and a third text sequence is obtained by the text translation model, wherein the third text sequence corresponds to the second language;

[0009] The model parameters of the text translation model are updated according to the first text sequence, the second text sequence, the text similarity, the third text sequence, and the expected target text.

[0010] Another aspect of the present application provides a model training device, comprising:

[0011] The obtaining unit is configured to obtain a source text set, a target text set, and a training sample pair, wherein the source text set comprises source text corresponding to a first language, the target text set comprises target text corresponding to a second language, and the training sample pair comprises initial source text corresponding to the first language and expected target text corresponding to the second language;

[0012] The processing unit is configured to input the source text set, the target text set, and the expected target text into the dialogue understanding model, and output a first text sequence, a second text sequence, and a text similarity by the dialogue understanding model, wherein the first text sequence corresponds to the second language, the second text sequence corresponds to the second language, and the text similarity represents the correlation degree between the expected target text and the target text set;

[0013] The processing unit is further configured to input the source text set and the training sample pair into the text translation model, and obtain a third text sequence by the text translation model, wherein the third text sequence corresponds to the second language;

[0014] The processing unit is further configured to update the model parameters of the text translation model according to the first text sequence, the second text sequence, the text similarity, the third text sequence, and the expected target text.

[0015] In a possible design, in an implementation manner of another aspect of the embodiment of the present application, the processing unit can be specifically configured to:

[0016] Obtain a source text embedding vector corresponding to the source text set, obtain a target text embedding vector corresponding to the target text set, and obtain an expected target embedding vector corresponding to the expected target text;

[0017] Input the target text embedding vector and the expected target embedding vector into the monolingual understanding model, and obtain the first text sequence by the monolingual understanding model;

[0018] Input the source text embedding vector and the expected target embedding vector into the cross-language understanding model, and obtain the second text sequence by the cross-language understanding model;

[0019] The target text embedding vector and an expected target embedding vector are input into a dialogue discrimination model, and the dialogue discrimination model is used to obtain the text similarity.

[0020] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit can be specifically configured to:

[0021] The target text embedding vector is input into an encoder of a single language understanding model, and the encoder of the single language understanding model is used to obtain a first target text encoding sequence, which is a sequence representation of the target text embedding vector in the second language;

[0022] The first target text encoding sequence and the expected target embedding vector are input into a decoder of the single language understanding model, and the decoder of the single language understanding model is used to obtain the first text sequence.

[0023] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit can be specifically configured to:

[0024] The source text embedding vector is input into an encoder of a cross language understanding model, and the encoder of the cross language understanding model is used to obtain a source text encoding sequence, which is a sequence representation of the source text embedding vector in the second language;

[0025] The source text encoding sequence and the expected target embedding vector are input into a decoder of the cross language understanding model, and the decoder of the cross language understanding model is used to obtain the second text sequence.

[0026] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit can be specifically configured to:

[0027] The target text embedding vector is input into an encoder of a dialogue discrimination model, and the encoder of the dialogue discrimination model is used to obtain a second target text encoding sequence, which is a sequence representation of the target text embedding vector in the second language;

[0028] The expected target embedding vector is input into the encoder of the dialogue discrimination model, and the encoder of the dialogue discrimination model is used to obtain an expected target encoding sequence, which is a sequence representation of the expected target embedding vector in the second language;

[0029] A negative sampling text embedding vector corresponding to the target text embedding vector is obtained, where the negative sampling text embedding vector is a vector representation of a text not belonging to the target text set;

[0030] The negatively sampled text embedding vector is used as the input of the dialogue discriminant model's encoder. The encoder of the dialogue discriminant model obtains the negatively sampled text encoding sequence. The negatively sampled text encoding sequence is a sequence representation of the negatively sampled text embedding vector in the second language.

[0031] The second target text encoding sequence, the expected target encoding sequence, and the negative sampling text encoding sequence are used as inputs to the classifier of the dialogue discrimination model, and the text similarity is obtained through the classifier of the dialogue discrimination model.

[0032] In one possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically configured to:

[0033] Performing word segmentation on the source text set to obtain at least two source words, wherein at least two of the source words belong to the first language;

[0034] Pass at least two source words through the word embedding layer, position embedding layer, turn embedding layer, and role embedding layer, and fuse the vector representations output by each embedding layer to obtain the source text embedding vector;

[0035] The processing unit can be used to:

[0036] Perform word segmentation on the target text set to obtain at least two target words, at least two of which belong to the second language;

[0037] Pass at least two target words through the word embedding layer, position embedding layer, turn embedding layer, and role embedding layer respectively, and fuse the vector representations output by each embedding layer to obtain the target text embedding vector;

[0038] The processing unit can be used to:

[0039] Performing word segmentation on the desired target text to obtain at least two desired end words, wherein at least two of the desired end words belong to the second language;

[0040] At least two expected words are passed through the word embedding layer, position embedding layer, turn embedding layer, and role embedding layer respectively, and the vector representation output by each embedding layer is fused to obtain the expected target embedding vector.

[0041] In one possible design, in an implementation of another aspect of the embodiment of the present application,

[0042] The processing unit is further configured to use the target text set and the training sample pair as inputs of the object discrimination model, and obtain object similarity through the object discrimination model, wherein the object similarity represents the degree of association between the target text set and the character object generating the target text;

[0043] The processing unit is further configured to update the model parameters of the text translation model according to the first text sequence, the second text sequence, the text similarity, the third text sequence, the object similarity, and the expected target text.

[0044] Another aspect of the present application provides a computer device, comprising: a memory, a transceiver, a processor, and a bus system;

[0045] The memory is configured to store a program.

[0046] The processor is configured to implement the method of any of the above aspects when executing the program stored in the memory.

[0047] The bus system is configured to connect the memory and the processor to enable communication between the memory and the processor.

[0048] Another aspect of the present application provides a computer-readable storage medium, which stores instructions, when the instructions are executed on a computer, cause the computer to perform the method of any of the above aspects.

[0049] In another aspect of the present application, a computer program product or a computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a network device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the network device to perform the method provided in any of the above aspects.

[0050] As can be seen from the above technical solutions, the embodiments of the present application have the following advantages:

[0051] The source text set, the target text set, and the training sample pair are obtained, the source text set, the target text set, and the expected target text are taken as inputs of the dialogue understanding model, first text sequences in the second language, second text sequences in the second language, and text similarities capable of representing the correlation degree between the expected target text and the target text set are output through the dialogue understanding model, the source text set and the training sample pair are taken as inputs of the text translation model, third text sequences in the second language are obtained through the text translation model, and then, the model parameters of the text translation model are updated according to the first text sequences, the second text sequences, the text similarities, the third text sequences, and the expected target text. In the foregoing manner, first text sequences in the second language, second text sequences in the second language, and text similarities capable of representing the correlation degree between the expected target text and the target text set are obtained through the dialogue understanding model, third text sequences in the second language are obtained through the text translation model, and then, the model parameters of the text translation model can be jointly and iteratively updated according to the first text sequences, the second text sequences, the text similarities, the third text sequences, and the expected target text, the dialogue understanding model can be taken as an auxiliary training task of the text translation model to jointly train the text translation model, the text translation model can continuously learn the features of text language changes and the correlation degree between texts to enhance the learning ability of the text translation model for the continuity between texts, the training precision of the text translation model can be improved, and therefore, the continuity and accuracy of the translation generated by the text translation model can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is an architecture schematic diagram of a model object control system in an embodiment of the present application;

[0053] Figure 2 is an embodiment schematic diagram of a model training method in an embodiment of the present application;

[0054] Figure 3 is another embodiment schematic diagram of a model training method in an embodiment of the present application;

[0055] Figure 4 is another embodiment schematic diagram of a model training method in an embodiment of the present application;

[0056] Figure 5 is another embodiment schematic diagram of a model training method in an embodiment of the present application;

[0057] Figure 6 is another embodiment schematic diagram of a model training method in an embodiment of the present application;

[0058] Figure 7is another embodiment schematic diagram of the method for model training in the embodiments of the present application;

[0059] Figure 8 is another embodiment schematic diagram of the method for model training in the embodiments of the present application;

[0060] Figure 9 is a principle flow schematic diagram of the method for model training in the embodiments of the present application;

[0061] Figure 10 is a dialogue sample schematic diagram of the method for model training in the embodiments of the present application;

[0062] Figure 11 is another principle flow schematic diagram of the method for model training in the embodiments of the present application;

[0063] Figure 12 is another principle flow schematic diagram of the method for model training in the embodiments of the present application;

[0064] Figure 13 is a principle schematic diagram of obtaining an embedding vector of the method for model training in the embodiments of the present application;

[0065] Figure 14 is an embodiment schematic diagram of the device for model training in the embodiments of the present application;

[0066] Figure 15 is an embodiment schematic diagram of the computer device in the embodiments of the present application. DETAILED DESCRIPTION

[0067] The embodiments of the present application provide a method for model training, related devices and equipment, which are used for jointly training a text translation model by taking a dialogue understanding model as an auxiliary training task of the text translation model, can enhance the learning ability of the text translation model for the coherence between texts by continuously learning the features of text language changes and the correlation degree between texts, can improve the training precision of the text translation model, and thus improve the coherence and accuracy of the translation generated by the text translation model.

[0068] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application and in the drawings if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms herein is to be construed as interchangeable in order to comfort with the context of the description being written and the description is to be understood to be flexible with respect to the context. Furthermore, the terms "comprising", "including", "containing", and "having" and their conjugates, as used herein, are intended to encompass the presence of one or more elements, steps, or units, without necessarily excluding the presence of one or more other elements, steps, or units. It is to be understood that the use of such terms herein is to be construed as interchangeable with respect to the context.

[0069] It should be understood that the method of model training provided by the present application can be applied to the scenario of completing the reply and translation of multi-turn dialogue text by modeling. For example, by modeling, an English dialogue is translated into Chinese and a Chinese reply is completed. As another example, by modeling, a French story dialogue is translated into an English dialogue. As still another example, by modeling, a Russian dialogue brief is translated into a French reply and translation. In the above scenarios, in order to complete the reply and translation of multi-turn dialogue text, the existing solution is to encode each turn of dialogue history as an independent sentence. However, since multi-turn dialogue is generated in a coherent communication process, the sentences of each turn involved are related to each other. If these sentences are encoded separately without considering the semantic relationship between them, the coherence relationship at the chapter level in the dialogue history may not be captured, resulting in a semantic gap or incoherence in the obtained dialogue translation text, thereby reducing the accuracy of the obtained multi-turn dialogue translation.

[0070] To solve the above problems, the present application provides a method of model training, which is applied to Figure 1 The model object control system is shown in the schematic diagram of the model object control system in the embodiment of the present application, please refer to Figure 1 , Figure 1 The schematic diagram of the architecture of the model object control system in the embodiment of the present application is shown in the schematic diagram of the architecture of the model object control system in the embodiment of the present application, please refer to Figure 1As shown, the server obtains a source text set, a target text set, and a training sample pair sent by a terminal device, and uses the source text set, the target text set, and the expected target text as inputs of a dialogue understanding model. The dialogue understanding model outputs a first text sequence belonging to the second language, a second text sequence belonging to the second language, and a text similarity that can represent the degree of association between the expected target text and the target text set. The source text set and the training sample pair are used as inputs of a text translation model, and the text translation model obtains a third text sequence belonging to the second language. Then, the model parameters of the text translation model are updated according to the first text sequence, the second text sequence, the text similarity, the third text sequence, and the expected target text. Through the above method, it is achieved that a first text sequence belonging to the second language, a second text sequence belonging to the second language, and a text similarity that can represent the degree of association between the expected target text and the target text set are obtained through the dialogue understanding model, and a third text sequence belonging to the second language is obtained through the text translation model. Then, the model parameters of the text translation model can be jointly iteratively updated according to the first text sequence, the second text sequence, the text similarity, the third text sequence and the expected target text. The dialogue understanding model can be used as an auxiliary training task of the text translation model to jointly train the text translation model. By continuously learning the characteristics of text language changes and the degree of association between texts, the text translation model's ability to learn the coherence between texts can be enhanced, the training accuracy of the text translation model can be improved, and thus the coherence and accuracy of the translations generated by the text translation model can be improved.

[0071] It is understandable that Figure 1 Only one type of terminal device is shown in the figure. In actual scenarios, more types of terminal devices may participate in the data processing process, such as personal computers (PCs). The specific number and type depend on the actual scenario and are not limited here. Figure 1 One server is shown in the figure, but in actual scenarios, multiple servers may also be involved, especially in scenarios of multi-model training interaction. The number of servers depends on the actual scenario and is not limited here.

[0072] It should be noted that in the embodiment, the server can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal device can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, and the like, but is not limited thereto. The terminal device and the server can be directly or indirectly connected through wired or wireless communication, and the terminal device and the server can be connected to form a blockchain network. The present application does not limit this.

[0073] To solve the above problems, the present application provides a model training method, which is generally executed by a server or a terminal device. Correspondingly, the device for model training is generally arranged in the server or the terminal device.

[0074] It can be understood that the model training method, related device and apparatus disclosed in the present application can be composed of a blockchain, and the server / terminal device is a node on the blockchain. In actual application, data sharing between nodes in the blockchain is required, and each node can store model training data.

[0075] The model training method of the present application will be introduced below. Please refer to Figure 2 and Figure 9 An embodiment of the model training method in the present application includes:

[0076] In step S101, a source text set, a target text set, and a training sample pair are obtained, wherein the source text set includes source text corresponding to a first language, the target text set includes target text corresponding to a second language, and the training sample pair includes initial source text corresponding to the first language and expected target text corresponding to the second language;

[0077] In the embodiment, in order to better and more accurately obtain the translation text which not only conforms to the semantic coherence of the multi-turn dialogue text, but also conforms to the original semantic of the multi-turn dialogue text, the embodiment obtains a source text set containing source texts corresponding to a first language, a target text set containing target texts corresponding to a second language, and a training sample pair containing an initial source text corresponding to the first language and an expected target text corresponding to the second language, so that the subsequent dialogue understanding model capable of obtaining more accurate semantic coherence can be trained and optimized by using the obtained source text set, target text set and training sample pair. Then, the dialogue understanding model can be used as an auxiliary training task of the text translation model for joint training of the text translation model, which can enhance the coherence learning ability of the text translation model for texts, improve the training precision of the text translation model, and thus improve the coherence and accuracy of the translation text generated by the text translation model.

[0078] The source text set contains a plurality of source texts corresponding to the first language. Specifically, the source text set can represent a certain role in a multi-turn dialogue history text, such as role A, and the dialogue question and dialogue reply received in each turn are translated to obtain the corresponding translation text. The dialogue question text and dialogue reply translation text in each turn are source texts belonging to the first language. The first language can be Chinese, English, French, etc., or other languages, which are not limited here.

[0079] For example, as shown in Figure 10 , it is assumed that the source text set is the dialogue history text of role X, the first language is English, and the source text belonging to the first language can represent the dialogue question text of role X in the first turn, such as X1: “Ohh, okey, they gave you the old time pricing”, and the dialogue reply translation text generated by role X in the first turn according to the dialogue reply text of role Y in the first turn, such as Y2: “yes”, such as X2: “yeah”, etc.

[0080] The target text set contains a plurality of target texts corresponding to the second language. Specifically, the target text set can represent a certain role in a multi-turn dialogue history text, such as role B, and the dialogue question and dialogue reply text generated in each turn according to the received dialogue question of other roles are translated to obtain the corresponding translation text. The dialogue question translation text and dialogue reply text in each turn are target texts belonging to the second language. The second language is different from the first language. For example, the first language is Chinese, and the second language can be English, Japanese or Korean, etc., or other languages, which are not limited here.

[0081] For example, as shown inFigure 10 As shown in the figure, assuming that the target text set is the dialogue history text of role Y, the partner of role X, and the source text set is the dialogue history text of role X, and the second language is Chinese, the target text belonging to the second language can specifically be that role Y generates dialogue question text in the first round according to the dialogue question text of role X in the first round, such as X1: "Ohh, okey, they gave you the old time pricing", and the dialogue question translation text generated by role Y is Y1: "They calculated according to the price of that year", and role Y generates dialogue reply text in the first round according to the dialogue question translation text in Y1, such as Y2: "Yes" and the like.

[0082] Among them, the training sample pair includes the initial source text corresponding to the first language and the expected target text corresponding to the second language, and the initial source text can specifically be the initial dialogue question text to be translated based on the summary of the source text set, for example, as shown in the figure Figure 10 As shown in the figure, the initial source text obtained from the source text set corresponding to role X can specifically be X u : "White plains. Oh, it sounds like such a magical place", and can also be other texts, which are not specifically limited here. The expected target text can specifically be the translation text expected to be obtained by translating the initial source text based on the context semantics of the source text set and the target text set, for example, as shown in the figure Figure 10 As shown in the figure, the expected target text can specifically be Y u : "White Plains. Oh, it sounds like such a magical place".

[0083] Specifically, as shown in the figures Figure 9 and Figure 11 , the source text set, the target text set and the training sample pair are obtained, which can be obtained by using network crawler technology in a big data platform according to actual application or business demand, or receiving the source text set, the target text set and the training sample pair sent by the terminal device, or obtaining the pre-stored source text set, the target text set and the training sample pair in the server database, so that the subsequent source text set, the target text set and the training sample pair can be processed accordingly, and then can be used to train and optimize the dialogue understanding model with more accurate semantic coherence, and the dialogue understanding model is used as an auxiliary training task of the text translation model, and the dialogue understanding model is used as an auxiliary training task of the text translation model, to enhance the coherence learning ability of the text translation model, improve the training precision of the text translation model, and thus improve the coherence and accuracy of the translation text generated by the text translation model.

[0084] In step S102, the source text set, the target text set, and the expected target text are input into the dialogue understanding model, and a first text sequence, a second text sequence, and a text similarity are output by the dialogue understanding model, where the first text sequence corresponds to the second language, the second text sequence corresponds to the second language, and the text similarity represents the relevance of the expected target text to the target text set.

[0085] In this embodiment, after obtaining the source text set, the target text set, and the training sample pair, in order to enable the dialogue understanding model to sufficiently learn the semantic relationship and coherence between the texts, the semantic features of the expected target text can be maximized. In this embodiment, the source text set, the target text set, and the expected target text are input into the dialogue understanding model, the first text sequence belonging to the second language, the second text sequence belonging to the second language, and the text similarity representing the relevance of the expected target text to the target text set can be obtained by the dialogue understanding model. Subsequently, the model parameters of the text translation model can be jointly and iteratively updated by the first text sequence, the second text sequence, the text similarity, the third text sequence, and the expected target text. The text translation model can continuously learn the features of the change of the language of the text and the relevance between the texts, so as to enhance the learning ability of the text translation model for the coherence between the texts, improve the training accuracy of the text translation model, and thus improve the coherence and accuracy of the translated text generated by the text translation model to a certain extent.

[0086] The first text sequence is a sequence representation of the target text set and the expected target text in the second language, the second text sequence is a sequence representation of the source text set and the expected target text in the second language, and the text similarity is used to represent the relevance of the expected target text to the target text set. The text similarity can be a score, a probability value, or a distance, and can also be other forms, which can be set according to actual application requirements, and is not limited specifically herein.

[0087] Specifically, as shown in Figure 9 and Figure 11 When the source text set, the target text set, and the training sample pair are obtained, the source text set, the target text set, and the expected target text can be input into the dialogue understanding model, and the corresponding encoding vector representations can be obtained by the encoders of the dialogue understanding model. Then, the corresponding encoding vector representations can be input into the decoders of the dialogue understanding model, so as to accurately obtain the first text sequence belonging to the second language, the second text sequence belonging to the second language, and the text similarity representing the relevance of the expected target text to the target text set.

[0088] Further, the embodiment can also update the model parameters of the dialogue understanding model based on the first text sequence, the second text sequence, the text similarity, and the expected target text. The model parameters can be updated in a gradient descent manner, or other updating manners, which do not limit specifically herein. The dialogue understanding model can learn the semantic relationship and coherence between the texts sufficiently, and can approach the semantic features of the expected target text to the greatest extent, thereby improving the accuracy of the dialogue understanding model in obtaining the translation text, and improving the semantic coherence of the translation text, and further assisting the text translation model in training and optimization.

[0089] In step S103, the source text set and the training sample pair are input into the text translation model, and a third text sequence is obtained by the text translation model, wherein the third text sequence corresponds to the second language;

[0090] In the embodiment, after obtaining the source text set and the training sample pair, the semantic relationship and semantic features between the texts learned by the text translation model can approach the semantic features of the expected target text to the greatest extent. The source text set and the training sample pair are input into the text translation model, and a third text sequence belonging to the second language is obtained by the text translation model, wherein the third text sequence is a sequence representation of the source text set and the training sample in the second language. Then, the model parameters of the text translation model can be updated jointly and iteratively based on the third text sequence, the first text sequence, the second text sequence, the text similarity, and the expected target text. The text translation model can learn the semantic relationship and semantic features between the texts sufficiently, and further learn the features of the change of the language of the text and the correlation degree between the texts, so as to enhance the learning ability of the text translation model in the semantic features and the coherence of the semantic relationship between the texts, improve the training precision of the text translation model, and thereby improve the coherence and accuracy of the translation text generated by the text translation model to a certain extent.

[0091] Specifically, as shown in Figure 12 When the source text set and the training sample are obtained, the initial source text in the source text set and the training sample pair can be input into the encoder of the text translation model, and the corresponding encoding sequence is obtained after encoding.

[0092] For example, as shown in Figure 9 and Figure 10 , it is assumed that a source text set is and an initial source text is X u The source text set and the initial source text Xu As the input of the encoder of the text translation model (NCT), the encoding can obtain the corresponding decoding sequence as

[0093] Further, as shown in Figure 12 , the encoding sequence corresponding to the encoding, the expected target text in the training sample pair, and the translation result of the (t-1) th word can be input into the decoder of the text translation model, and the expected target text and the translation result of the (t-1) th word can be compiled. Specifically, the translation result of the (t-1) th word can be encoded and learned by the decoder using a self-attention mechanism to obtain a compiled vector representation that can be used to represent the t th word, as follows (1):

[0094] H y =MultiHead(y,y,y) (1)

[0095] Where y is the vector representation corresponding to the translation result of the (t-1) th word, and MultiHead is a multi-head self-attention mechanism.

[0096] It should be noted that the self-attention mechanism used in the present embodiment can be a multi-head self-attention mechanism MultiHead, or a single-head self-attention mechanism, or other self-attention mechanisms, which are not limited here.

[0097] Further, the source text set, the encoding sequence corresponding to the initial source text, and the target expected text can be encoded and learned by another self-attention mechanism (Self-Attention) in the decoder to obtain a corresponding compiled vector representation H x , and then the compiled vector representation H x is interacted with H y to obtain an interaction representation as follows (2):

[0098]

[0099] Where FFN is a feedforward neural network.

[0100] Further, the interaction vector representation can be decoded by a classifier to accurately obtain the translation result of the t th word as follows (3), where when t is 1, the vector input into the decoder is a preset vector of fixed dimension:

[0101]

[0102] Where W o is a learning matrix parameter, and b o is a learning parameter.

[0103] Further, the translation result of the (t-1)th word is compiled repeatedly, and the compiled vector representation H y Interact with H x The interaction vector representation obtained is then decoded by the classifier to obtain the translation result of the tth word. The process continues until a special termination symbol is reached, and the third text sequence can be obtained.

[0104] Further, the loss value can be calculated according to the third text sequence, the source text set, and the initial source text based on the loss function of formula (4) below, and then the model parameters of the text translation model are updated. The update of the model parameters can be in the form of gradient descent, or other forms, which are not limited here:

[0105]

[0106] In step S104, the model parameters of the text translation model are updated according to the first text sequence, the second text sequence, the text similarity, the third text sequence, and the expected target text.

[0107] Specifically, after obtaining the first text sequence, the second text sequence, the text similarity, the third text sequence, and the expected target text, the model parameters can be updated in the form of gradient descent, or other forms, which are not limited here. The update converges stably in the gradient update direction to better update the model parameters, thereby enhancing the learning ability of the text translation model in terms of semantic and semantic relationship coherence, improving the training accuracy of the text translation model, and thus improving the coherence and accuracy of the translated text generated by the text translation model to a certain extent.

[0108] In the embodiments of the present application, a model training method is provided. Through the above method, the first text sequence in the second language, the second text sequence in the second language, and the text similarity representing the degree of association between the expected target text and the target text set are obtained by the dialogue understanding model. The third text sequence in the second language is obtained by the text translation model. Then, the model parameters of the text translation model are jointly and iteratively updated based on the first text sequence, the second text sequence, the text similarity, the third text sequence, and the expected target text. The dialogue understanding model is used as an auxiliary training task for the text translation model, and the text translation model is jointly trained. The text translation model can enhance the learning ability of the text translation model in terms of text coherence by continuously learning the characteristics of text language changes and the degree of association between texts, thereby improving the training accuracy of the text translation model and improving the coherence and accuracy of the translated text generated by the text translation model.

[0109] Optionally, in the above Figure 2 Based on the corresponding embodiments, the model training method provided by the embodiments of the present application further provides another optional embodiment, which is shown as follows. Figure 3 As shown in the figure, the source text set, the target text set and the expected target text are input into the dialogue understanding model, and the dialogue understanding model outputs the first text sequence, the second text sequence and the text similarity, which includes:

[0110] In step S301, the source text embedding vector corresponding to the source text set is obtained, and the target text embedding vector corresponding to the target text set is obtained, and the expected target embedding vector corresponding to the expected target text is obtained.

[0111] In step S302, the target text embedding vector and the expected target embedding vector are input into the monolingual understanding model, and the first text sequence is obtained through the monolingual understanding model.

[0112] In step S303, the source text embedding vector and the expected target embedding vector are input into the cross-language understanding model, and the second text sequence is obtained through the cross-language understanding model.

[0113] In step S304, the target text embedding vector and the expected target embedding vector are input into the dialogue discrimination model, and the text similarity is obtained through the dialogue discrimination model.

[0114] In this embodiment, after obtaining the source text set, the target text set and the expected target text, in order to enable the machine learning to better learn the features in the text, the source text set, the target text set and the expected target text can be converted into a form that is easy for the machine learning algorithm to use. In this embodiment, the source text set, the target text set and the expected target text can be converted into embedding vectors respectively, and the source text embedding vector corresponding to the source text set, the target text embedding vector corresponding to the target text set and the expected target embedding vector corresponding to the expected target text can be obtained. Then, the target text embedding vector and the expected target embedding vector can be input into the monolingual understanding model, and the monolingual understanding model can be used to fully learn the coherent semantic relationship between the dialogue histories of the target texts corresponding to multiple second languages, so as to obtain the first text sequence, thereby improving the coherence of the monolingual understanding model translation.

[0115] Similarly, taking the source text embedding vector and the expected target embedding vector as the input of the cross-language understanding model, the single-language understanding model can be used to fully learn the coherent semantic relationship between the dialog histories of the source texts corresponding to multiple first languages, and learn the features of the text language change based on the semantic features of the expected target text of the second language, to obtain the second text sequence, so as to improve the cross-language capability of the cross-language understanding model and improve the coherence of the translation of the model.

[0116] Similarly, taking the target text embedding vector and the expected target embedding vector as the input of the dialog discrimination model, the dialog discrimination model can be used to learn the distribution of the association degree of the expected target text and the target text set, and then the text similarity can be obtained through the distribution of the association degree, to discriminate whether the generated translation is semantically coherent with the dialog history in the target text set.

[0117] Specifically, after obtaining the source text set, the target text set and the expected target text, the source text set, the target text set and the expected target text can be respectively processed by word segmentation, and the processed words can be respectively converted into vectors through the embedding layer, so as to obtain the source text embedding vector corresponding to the source text set, the target text embedding vector corresponding to the target text set, and the expected target embedding vector corresponding to the expected target text.

[0118] Further, after obtaining the source text embedding vector corresponding to the source text set, the target text embedding vector corresponding to the target text set, and the expected target embedding vector corresponding to the expected target text, the target text embedding vector and the expected target embedding vector can be taken as the input of the single-language understanding model, the first text sequence can be obtained through the single-language understanding model, the source text embedding vector and the expected target embedding vector can be taken as the input of the cross-language understanding model, the second text sequence can be obtained through the cross-language understanding model, and the target text embedding vector and the expected target embedding vector can be taken as the input of the dialog discrimination model, the text similarity can be obtained through the dialog discrimination model, so that the model parameters of the text translation model can be jointly and iteratively updated through the first text sequence, the second text sequence, the text similarity and the expected target text, the text translation model can continuously learn the features of the text language change and the association degree between texts, to enhance the coherence learning capability of the text translation model, improve the training accuracy of the text translation model, and thus improve the coherence and accuracy of the translation generated by the text translation model to a certain extent.

[0119] Optionally, in the above Figure 3 Based on the corresponding embodiments, another optional embodiment of the model training method provided by the embodiments of the present application is as follows: Figure 4As shown, the target text embedding vector and the expected target embedding vector are input into the monolingual understanding model, and the first text sequence is obtained through the monolingual understanding model, including

[0120] In step S401, the target text embedding vector is input into the encoder of the monolingual understanding model, and the first target text encoding sequence is obtained through the encoder of the monolingual understanding model, which is a sequence representation of the target text embedding vector belonging to the second language;

[0121] In step S402, the first target text encoding sequence and the target expected embedding vector are input into the decoder of the monolingual understanding model, and the first text sequence is obtained through the decoder of the monolingual understanding model.

[0122] Specifically, as shown Figure 11 After obtaining the target text embedding vector and the expected target embedding vector, since the target text embedding vector can contain a large number of parameters representing the internal relationship between words and sentences in the target text set, such as coherent semantic relationship, the target text embedding vector is input into the encoder of the monolingual understanding model (MRG), and N encoding vectors are obtained through the encoder of the monolingual understanding model, where N is an integer greater than 1, as shown Figure 12 Specifically, the N encoding vectors can be obtained through the Transformer BottomBlock layer, and then the first target text encoding sequence can be obtained by averaging the vector dimensions. Specifically, an N*d dimensional vector matrix can be obtained by superimposing the obtained N encoding vectors, and then the N*d dimensional vector matrix can be averaged to obtain a 1*d dimensional sequence representation that can be used to represent the core content of the target text set, i.e. the first target text encoding sequence.

[0123] Further, the first target text encoding sequence and the target expected embedding vector can be input into the decoder of the monolingual understanding model, and the target expected embedding vector can be encoded and learned using a self-attention mechanism, and the first target text encoding sequence can be encoded and learned using another self-attention mechanism. Then, the encoded and learned vector representations are interacted to obtain an interaction representation and the interaction representation is decoded through a classifier to accurately obtain the decoding result of the following formula (5), i.e. the first text sequence:

[0124]

[0125] where W m is a learned matrix parameter, and b m is a learned parameter.

[0126] Furthermore, a loss value can be calculated based on the loss function of the following formula (6) according to the first text sequence and the first target text encoding sequence. Then, the model parameters of the single language understanding model are updated using the loss value. The update of the model parameters can be specifically performed by gradient descent or other methods, which are not specifically limited here:

[0127]

[0128] For example, Figure 11 As shown, the target text is embedded in the vector As the input of the encoder of the single language understanding model, the first target text encoding sequence can be obtained after encoding. Then the first target text encoding sequence can be And the target expected embedding vector Y u , as the input of the decoder of the single language understanding model, to obtain the interactive representation The interaction representation is decoded through a classifier to obtain the first text sequence.

[0129] Optionally, in the above Figure 3 On the basis of the corresponding embodiment, in another optional embodiment of the model training method provided in the embodiment of the present application, as Figure 5 As shown, the source text embedding vector and the expected target embedding vector are used as inputs of the cross-language understanding model, and the second text sequence is obtained through the cross-language understanding model, including:

[0130] In step S501, the source text embedding vector is used as the input of the encoder of the cross-language understanding model, and the encoder of the cross-language understanding model obtains the source text encoding sequence, which is a sequence representation of the source text embedding vector in the second language;

[0131] In step S502, the source text encoding sequence and the target expected embedding vector are used as inputs of a decoder of the cross-language understanding model, and a second text sequence is obtained through the decoder of the cross-language understanding model.

[0132] In this embodiment, if Figure 11 As shown in the figure, after obtaining the source text embedding vector and the expected target embedding vector, since the source text embedding vector can contain a large number of intrinsic relationships between characters, words and even sentences in the source text set, such as parameter representations of coherent semantic relationships, the source text embedding vector can be used as the input of the encoder of the cross-language understanding model (DRG), and multiple encoding vector representations can be obtained through the encoder of the cross-language understanding model, such as Figure 12As shown, it can be obtained specifically through the Transformer Bottom Block layer, and then the source text encoding sequence can be obtained by averaging the vector dimensions, wherein the method of obtaining the source text encoding sequence is similar to the method of obtaining the first target text encoding sequence in step S401, and will not be repeated here.

[0133] Furthermore, the source text encoding sequence and the target expected embedding vector can be used as the input of the decoder of the cross-language understanding model. The target expected embedding vector can be encoded and learned by a self-attention mechanism, and another self-attention mechanism can be used to encode the source text encoding sequence. Then, the encoding learning is interacted with the vector representation to obtain the interactive representation. By decoding the interaction representation through the classifier, we can accurately obtain the decoding result of the following formula (7), that is, the second text sequence:

[0134]

[0135] Among them, W c is the learning matrix parameter, b c is a learning parameter.

[0136] Furthermore, the loss value can be calculated based on the loss function of the following formula (8) according to the second text sequence and the source text encoding sequence, and then the model parameters of the cross-language understanding model are updated based on the loss value. The update of the model parameters can be specifically performed by gradient descent, or other methods, which are not specifically limited here:

[0137]

[0138] For example, Figure 11 As shown, the source text is embedded into the vector As the input of the encoder of the cross-language understanding model, the source text encoding sequence can be obtained after encoding. Then the source text encoding sequence can be and the target expected embedding vector Y u , as the input of the decoder of the cross-language understanding model, and obtain the interaction representation The interaction representation is decoded through a classifier to obtain a second text sequence.

[0139] Optionally, in the above Figure 3 On the basis of the corresponding embodiment, in another optional embodiment of the model training method provided in the embodiment of the present application, as Figure 6 As shown in the figure, the target text embedding vector and the expected target embedding vector are used as inputs to the dialogue discrimination model. The dialogue discrimination model is used to obtain text similarity, including:

[0140] In step S601, the target text embedding vector is used as input to the encoder of the dialogue discrimination model. The encoder of the dialogue discrimination model obtains a second target text encoding sequence, where the second target text encoding sequence is a sequence representation of the target text embedding vector in the second language.

[0141] In step S602, the desired target embedding vector is used as input to the encoder of the dialogue discrimination model. The encoder of the dialogue discrimination model obtains the desired target encoding sequence, which is a sequence representation of the desired target embedding vector in the second language.

[0142] In step S603, a negative sampling text embedding vector corresponding to the target text embedding vector is obtained, wherein the negative sampling text embedding vector is a vector representation corresponding to the text that does not belong to the target text set;

[0143] In step S604, the negatively sampled text embedding vector is used as input to the encoder of the dialogue discrimination model. The encoder of the dialogue discrimination model obtains a negatively sampled text encoding sequence, where the negatively sampled text encoding sequence is a sequence representation of the negatively sampled text embedding vector in the second language.

[0144] In step S605, the second target text encoding sequence, the expected target encoding sequence, and the negative sampling text encoding sequence are used as inputs of the classifier of the dialogue discrimination model, and the text similarity is obtained through the classifier of the dialogue discrimination model.

[0145] In this embodiment, if Figure 11 As shown in the figure, after obtaining the target text embedding vector and the expected target embedding vector, in order to better learn the degree of association between the target text set and the expected target text, and to accurately judge whether the generated translation is semantically consistent with the dialogue history in the target text set based on the degree of association, the target text embedding vector can be used as the input of the encoder of the dialogue discrimination model (NUD). Multiple encoding vector representations are obtained through the encoder of the dialogue discrimination model, as shown in the figure. Figure 12 As shown, the vector representation with pre-labeled CLS identifiers can be obtained through the Transformer Bottom Block layer, and then the second target text encoding sequence can be obtained by averaging the vector dimensions. The method of obtaining the second target text encoding sequence is similar to the method of obtaining the first target text encoding sequence in step S401, and will not be repeated here.

[0146] Similarly, if Figure 11As shown, the expected target embedding vector is taken as the input of the encoder of the dialogue discrimination model, and a plurality of encoding vectors are obtained through the encoder of the dialogue discrimination model, and then the expected target encoding sequence can be obtained by averaging the vector dimensions, and the manner of obtaining the expected target encoding sequence is similar to that of obtaining the first target text encoding sequence in step S401, which will not be described here.

[0147] Further, the negative sampling text embedding vector corresponding to the target text embedding vector is obtained, wherein the negative sampling text is the dialogue text of another role object interacting with the role object corresponding to the target text set, and the negative sampling text embedding vector is the vector representation of the negative sampling text, i.e., the vector representation of the interaction text corresponding to the target text set;

[0148] Further, the negative sampling text embedding vector can be taken as the input of the encoder of the dialogue discrimination model, and a plurality of encoding vectors are obtained through the encoder of the dialogue discrimination model, and then the negative sampling text encoding sequence can be obtained by averaging the vector dimensions, and the manner of obtaining the negative sampling text encoding sequence is similar to that of obtaining the first target text encoding sequence in step S401, which will not be described here.

[0149] Further, the second target text encoding sequence, the expected target encoding sequence, and the negative sampling text encoding sequence can be taken as the input of the decoder of the dialogue discrimination model, the similarity can be obtained by using the classifier, and the decoding result of the following formula (9) can be accurately obtained, i.e., the text similarity:

[0150]

[0151] wherein, W n is a learning matrix parameter.

[0152] Further, according to the text similarity, the second target text encoding sequence, and the negative sampling text encoding sequence, a loss value can be calculated based on the loss function of the following formula (10), and then the loss value is used to update the model parameters of the dialogue discrimination model, wherein the update of the model parameters can be in the form of gradient descent, or other forms, which will not be specifically limited here:

[0153]

[0154] wherein, Y u+ represents a positive example, such as a text formed by a speech once said by a speaker of a current sentence to be translated, Y u- represents a negative example, such as a text formed by a speech once said by another speaker interacting with the speaker of the current sentence to be translated.

[0155] For example, as Figure 11The target text embedding vector Y As the input of the encoder of the dialogue discrimination model, the second target text encoding sequence H can be obtained after encoding The target expectation embedding vector Y u As the input of the encoder of the dialogue discrimination model, the target expectation encoding sequence H can be obtained after encoding Yu The negative sampling embedding vector is input into the encoder of the dialogue discrimination model, and the negative sampling text encoding sequence H can be obtained after encoding, and then the second target text encoding sequence H The target expectation encoding sequence H Yu The negative sampling text encoding sequence H is calculated by the classifier to obtain the text similarity.

[0156] Optionally, in the above Figure 3 Based on the corresponding embodiment, another optional embodiment of the model training method provided by the embodiment of the present application is as follows: Figure 7 As shown, the source text embedding vector corresponding to the source text set is obtained, including

[0157] In step S701, the source text set is processed by word segmentation to obtain at least two source words, and the at least two source words belong to a first language;

[0158] In step S702, the at least two source words are respectively input into the word embedding layer, the position embedding layer, the round embedding layer and the role embedding layer, and the vector representations output by each embedding layer are fused to obtain the source text embedding vector;

[0159] In this embodiment, after obtaining the source text set, in order to enable the machine learning to better learn the features in the text of the source text set, the source text set can be converted into a form that is easy for the machine learning algorithm to use, that is, the obtained source text set can be converted into N source text embedding vectors, where N is an integer greater than 1, so that subsequent processing of the N source text embedding vectors can accurately obtain the first text sequence belonging to the second language.

[0160] Specifically, as shown in Figure 13As shown, after obtaining the source text set, this embodiment can obtain N source text embedding vectors corresponding to the source text set. Specifically, the source text set can be segmented first to obtain two source words, and then each source word or character obtained by processing can be mapped to an array or list consisting of a total number of words or a total number of characters based on a one-hot encoding method, wherein each character or word can be mapped to a uniquely corresponding array or list, and the uniqueness of the array or list is represented by 1, and each text can be integrated into a sparse matrix, that is, the sparse matrix corresponding to the text to be processed.

[0161] Furthermore, in order to avoid the situation where the text operation processing volume becomes large and the operation complexity increases due to the excessive number of words in the text, and at the same time, in order to better learn the coherence between the conversation contexts of each round in the text and generate the role characteristics of the role objects of the conversation, this embodiment can pass the sparse matrix corresponding to the source text set through the embedding layer, specifically, by Figure 13 The word embeddings layer, positional embeddings layer, turn embeddings layer, and role embeddings layer are shown, and the vector representations output by each embedding layer are fused. Specifically, dimension addition can be performed, or other fusion methods, such as dimension splicing, can be used. There is no specific limitation here. The sparse matrix can be converted into a fused vector representation, and then the fused vector representation can be mapped into a dense matrix. Specifically, some linear transformations (such as table lookup operations) can be used to convert the sparse matrix into a dense matrix. The dense matrix can be used to represent all texts with N features. It can be understood that since the dense matrix represents a one-to-one correspondence between the dense matrix and a single word, it actually contains a large number of intrinsic relationships between words, words, and even sentences, such as coherent semantic relationships. These relationships can be represented by the parameters learned by the embedding layer. By performing dimensionality reduction on the sparse matrix, the computational complexity can be effectively reduced, thereby improving the efficiency of text processing to a certain extent.

[0162] Furthermore, after obtaining the dense matrix corresponding to the source text set, the dense matrix can be encoded by a word encoder (Encoder). It can be understood that it is based on the mapping relationship between the vocabulary and the matrix to accurately obtain the vector representation corresponding to each word or character in the source text set, and obtain N source text embedding vectors corresponding to the source text set.

[0163] Get the target text embedding vector corresponding to the target text set, including:

[0164] In step S703, the target text set is subjected to word segmentation processing to obtain at least two target words, and the at least two target words belong to the second language;

[0165] In step S704, the at least two target words are respectively subjected to word embedding layer, position embedding layer, round embedding layer and role embedding layer, and vector fusion is performed on the vector representations output by each embedding layer to obtain a target text embedding vector.

[0166] Specifically, the target text embedding vector corresponding to the target text set is obtained in a manner similar to the source text embedding vector corresponding to the source text set obtained in steps S701 to S702, which will not be described here.

[0167] The expected target text corresponding to the expected target embedding vector is obtained, comprising:

[0168] In step S705, the expected target text is subjected to word segmentation processing to obtain at least two expected words, and the at least two expected words belong to the second language;

[0169] In step S706, the at least two expected words are respectively subjected to word embedding layer, position embedding layer, round embedding layer and role embedding layer, and vector fusion is performed on the vector representations output by each embedding layer to obtain an expected target embedding vector.

[0170] Specifically, the expected target embedding vector corresponding to the expected target text is obtained in a manner similar to the source text embedding vector corresponding to the source text set obtained in steps S701 to S702, which will not be described here.

[0171] Optionally, in the above Figure 3 Based on the corresponding embodiments, another optional embodiment of the model training method provided by the embodiments of the present application is as shown in the method further comprising: Figure 8

[0172] In step S801, the target text set and the training sample pair are taken as the input of the object discrimination model, and the object similarity is obtained by the object discrimination model, wherein the object similarity represents the correlation degree between the target text set and the role object generating the target text;

[0173] In step S802, the model parameters of the text translation model are updated according to the first text sequence, the second text sequence, the text similarity, the third text sequence, the object similarity and the expected target text.

[0174] ​In the embodiment, since the speaker preference is generated in the multi-turn dialogue, if the sentences of the multi-turn dialogue are encoded separately without considering the speaker preference information carried by the sentences, the role object generating the dialogue in the dialogue history cannot be well captured, resulting in a case that the obtained dialogue translation text does not conform to the language style of the role object speaking, thereby causing the accuracy of obtaining the multi-turn dialogue translation to be reduced. Therefore, in the embodiment, the target text set and the training sample pair are taken as inputs of the object discrimination model, to accurately obtain an object similarity capable of representing a correlation degree between the target text set and the role object generating the target text, where the object similarity can be specifically a score, a probability value, a distance, or other forms, and can be specifically set according to actual application requirements, which is not limited here. Then, the model parameters of the text translation model can be jointly iteratively updated according to the first text sequence, the second text sequence, the text similarity, the third text sequence, the object similarity, and the expected target text, the dialogue understanding model and the object discrimination model can be taken as auxiliary training tasks of the text translation model to jointly train the text translation model, the text translation model can continuously learn the features of the text language change and the correlation degree between the texts, and simultaneously learn the correlation degree between the text and the role object generating the text, to enhance the learning ability of the text translation model on the continuity between the texts, and enhance the learning ability of the text translation model on determining the preference of the role object generating the text, thereby improving the training precision of the text translation model, and improving the continuity and accuracy of the translation generated by the text translation model, and the translation can maintain the preference information of the role object.

[0175] Specifically, after obtaining the target text set and the training sample pair, a target text embedding vector corresponding to the target text set can be obtained, an expected target embedding vector corresponding to the expected target text can be obtained, and an initial source text embedding vector corresponding to the initial source text can be obtained.

[0176] Further, as shown in Figure 11 To better learn the correlation degree between the target text set and the role object generating the target text, and to accurately determine whether the generated translation text is consistent with the role object generating the target text, i.e., the original language speaker, the target text embedding vector can be taken as an input of an encoder (Encoder) of the object discrimination model (SI), and a plurality of encoding vectors are obtained through the encoder of the object discrimination model, as shown in Figure 12 The encoding sequence can be obtained by averaging the vector dimensions, and the manner of obtaining the encoding sequence is similar to the manner of obtaining the first target text encoding sequence in step S401, which is not described here.

[0177] Similarly, as shown in Figure 11 , the expected target embedding vector and the text formed by the historical dialogue said by the role object corresponding to the expected target embedding vector are taken as the input of the encoder of the dialogue discrimination model, and a plurality of encoding vectors are obtained through the encoder of the dialogue discrimination model, and then the corresponding target encoding sequence can be obtained by averaging the vector dimensions, and the way of obtaining the corresponding target encoding sequence is similar to the way of obtaining the first target text encoding sequence in step S401, which will not be described here.

[0178] Similarly, the initial source embedding vector and the text formed by the historical dialogue said by the role object corresponding to the initial source embedding vector can be taken as the input of the encoder of the dialogue discrimination model, and a plurality of encoding vectors are obtained through the encoder of the dialogue discrimination model, and then the corresponding source encoding sequence can be obtained by averaging the vector dimensions, and the way of obtaining the corresponding source encoding sequence is similar to the way of obtaining the first target text encoding sequence in step S401, which will not be described here.

[0179] Further, the obtained encoding sequence and the target encoding sequence can be taken as the input of the decoder of the object discrimination model, the similarity can be obtained by using the classifier, and the decoding result of the following formula (11) can be accurately obtained, that is, the text similarity:

[0180]

[0181] Wherein, W s is a learning matrix parameter.

[0182] Further, the object similarity and the obtained encoding sequence can be used to calculate the loss value based on the loss function of the following formula (12), and then the model parameters of the object discrimination model are updated, and the updating of the model parameters can be in the form of gradient descent, or other forms, which will not be specifically limited here:

[0183]

[0184] Wherein, represents the text formed by the historical dialogue said by the role object corresponding to the initial source embedding vector, represents the text formed by the historical dialogue said by the role object corresponding to the expected target embedding vector.

[0185] For example, as shown in Figure 11 , the target text embedding vector is taken as the input of the encoder of the object discrimination model, and the encoding sequence such as Embed the target expectation into vector Y u As the input of the encoder of the dialogue discrimination model, the target expected encoding sequence can be obtained after encoding, such as H Yu , and then the coding sequence can be Target expected coding sequence such as H Yu The object similarity is obtained by calculation through the classifier.

[0186] Furthermore, the loss value can be calculated based on the loss function of the following formula (13) according to the first text sequence, the second text sequence, the text similarity, the third text sequence, the object similarity and the expected target text. Then, the loss value is used to update the model parameters of the text translation model. The update of the model parameters can be specifically performed by gradient descent or other methods, which are not specifically limited here:

[0187] L=L NCT +α(L MRG +L CRG +L NUD )+βL SI (13)

[0188] Among them, α and β represent balance factors.

[0189] The following is a detailed description of the device for model training in this application. Figure 14 , Figure 14 This is a schematic diagram of an embodiment of a model training device in an embodiment of the present application. The model training device 20 includes:

[0190] An acquisition unit 201 is configured to acquire a source text set, a target text set, and training sample pairs, wherein the source text set includes source texts corresponding to a first language, the target text set includes target texts corresponding to a second language, and the training sample pairs include initial source texts corresponding to the first language and expected target texts corresponding to the second language;

[0191] Processing unit 202 is configured to use the source text set, the target text set, and the expected target text as inputs to a dialogue understanding model, and output, through the dialogue understanding model, a first text sequence, a second text sequence, and text similarity, wherein the first text sequence corresponds to the second language, the second text sequence corresponds to the second language, and the text similarity indicates the degree of association between the expected target text and the target text set;

[0192] The processing unit 202 is further configured to use the source text set and the training sample pair as inputs of the text translation model, and obtain a third text sequence through the text translation model, wherein the third text sequence corresponds to the second language;

[0193] The processing unit 202 is also configured to update the model parameters of the text translation model according to the first text sequence, the second text sequence, the text similarity, the third text sequence, and the expected target text.

[0194] Optionally, in the above Figure 14 Based on the corresponding embodiments, in another embodiment of the model training apparatus provided by the present embodiment, the processing unit 202 can be specifically configured to:

[0195] obtain a source text embedding vector corresponding to the source text set, obtain a target text embedding vector corresponding to the target text set, and obtain an expected target embedding vector corresponding to the expected target text;

[0196] input the target text embedding vector and the expected target embedding vector into the monolingual understanding model, and obtain the first text sequence by the monolingual understanding model;

[0197] input the source text embedding vector and the expected target embedding vector into the cross-language understanding model, and obtain the second text sequence by the cross-language understanding model;

[0198] input the target text embedding vector and the expected target embedding vector into the dialogue discrimination model, and obtain the text similarity by the dialogue discrimination model.

[0199] Optionally, in the above Figure 14 Based on the corresponding embodiments, in another embodiment of the model training apparatus provided by the present embodiment, the processing unit 202 can be specifically configured to:

[0200] input the target text embedding vector into the encoder of the monolingual understanding model, obtain a first target text encoding sequence by the encoder of the monolingual understanding model, the first target text encoding sequence being a sequence representation of the target text embedding vector in the second language;

[0201] input the first target text encoding sequence and the target expected embedding vector into the decoder of the monolingual understanding model, and obtain the first text sequence by the decoder of the monolingual understanding model.

[0202] Optionally, in the above Figure 14 Based on the corresponding embodiments, in another embodiment of the model training apparatus provided by the present embodiment, the processing unit 202 can be specifically configured to:

[0203] input the source text embedding vector into the encoder of the cross-language understanding model, obtain a source text encoding sequence by the encoder of the cross-language understanding model, the source text encoding sequence being a sequence representation of the source text embedding vector in the second language;

[0204] The source text encoding sequence and the target expected embedding vector are input into a decoder of the cross-language understanding model, and a second text sequence is obtained through the decoder of the cross-language understanding model.

[0205] Optionally, in the above Figure 14 Based on the corresponding embodiments, the processing unit 202 can be specifically configured to, in another embodiment of the model training apparatus provided by the present embodiment:

[0206] The target text embedding vector is input into an encoder of the dialogue discrimination model, and a second target text encoding sequence is obtained through the encoder of the dialogue discrimination model, the second target text encoding sequence being a sequence representation of the target text embedding vector belonging to the second language;

[0207] The expected target embedding vector is input into the encoder of the dialogue discrimination model, and an expected target encoding sequence is obtained through the encoder of the dialogue discrimination model, the expected target encoding sequence being a sequence representation of the expected target embedding vector belonging to the second language;

[0208] The target text embedding vector corresponds to a negative sampling text embedding vector, wherein the negative sampling text embedding vector is a vector representation of a text not belonging to the target text set;

[0209] The negative sampling text embedding vector is input into the encoder of the dialogue discrimination model, and a negative sampling text encoding sequence is obtained through the encoder of the dialogue discrimination model, the negative sampling text encoding sequence being a sequence representation of the negative sampling text embedding vector belonging to the second language;

[0210] The second target text encoding sequence, the expected target encoding sequence, and the negative sampling text encoding sequence are input into a classifier of the dialogue discrimination model, and a text similarity is obtained through the classifier of the dialogue discrimination model.

[0211] Optionally, in the above Figure 14 Based on the corresponding embodiments, the processing unit 202 can be specifically configured to, in another embodiment of the model training apparatus provided by the present embodiment:

[0212] The source text set is subjected to word segmentation processing to obtain at least two source words, the at least two source words belonging to the first language;

[0213] The at least two source words are respectively input into a word embedding layer, a position embedding layer, a turn embedding layer, and a role embedding layer, and vector fusion is performed on vector representations output by each embedding layer to obtain a source text embedding vector;

[0214] The processing unit 202 can be specifically configured to:

[0215] The target text set is processed by word segmentation to obtain at least two target words, and the at least two target words belong to a second language;

[0216] The at least two target words are respectively input into a word embedding layer, a position embedding layer, a round embedding layer and a role embedding layer, and vector fusion is performed on vector representations output by each embedding layer to obtain a target text embedding vector;

[0217] The processing unit 202 can be specifically used for:

[0218] The target text set is processed by word segmentation to obtain at least two target words, and the at least two target words belong to a second language;

[0219] The at least two target words are respectively input into a word embedding layer, a position embedding layer, a round embedding layer and a role embedding layer, and vector fusion is performed on vector representations output by each embedding layer to obtain a target text embedding vector.

[0220] Optionally, in the above Figure 14 Based on the corresponding embodiment, another embodiment of the model training device provided by the present application is provided,

[0221] The processing unit 202 is further configured to input the target text set and the training sample pair into an object discrimination model as an input, and obtain an object similarity through the object discrimination model, wherein the object similarity represents a degree of association between the target text set and a role object generating the target text;

[0222] The processing unit 202 is further configured to update model parameters of the text translation model according to the first text sequence, the second text sequence, the text similarity, the third text sequence, the object similarity and the expected target text.

[0223] Another aspect of the present application provides another computer device schematic diagram, as shown in Figure 15 as shown, Figure 15is a computer device structure schematic diagram provided by an embodiment of the present application. The computer device 300 can have great differences due to different configurations or performances, and can include one or more central processing units (CPUs) 310 (for example, one or more processors) and a memory 320, one or more storage media 330 (for example, one or more mass storage devices) storing application programs 331 or data 332. The memory 320 and the storage media 330 can be temporary storage or persistent storage. The programs stored in the storage media 330 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the computer device 300. Further, the central processing unit 310 can be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the computer device 300.

[0224] The computer device 300 can further include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 333, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM , and the like.

[0225] The computer device 300 described above is further configured to perform the steps in the embodiments. Figures 2 to 8

[0226] Another aspect of the present application provides a computer readable storage medium, which stores instructions when executed on a computer, causes the computer to perform the steps in the method described in the embodiments. Figures 2 to 8

[0227] Another aspect of the present application provides a computer program product containing instructions, which when executed on a computer or processor, causes the computer or processor to perform the steps in the method described in the embodiments. Figures 2 to 8

[0228] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0229] ​​​In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0230] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0231] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0232] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present application essentially, or the part that makes a contribution to the prior art, or all or a part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and various other media that can store program codes.

Claims

1. A model training method, characterized in that: include: Obtaining a source text set, a target text set, and training sample pairs, wherein the source text set includes source texts corresponding to a first language, the target text set includes target texts corresponding to a second language, and the training sample pairs include initial source texts corresponding to the first language and expected target texts corresponding to the second language; The source text set, the target text set, and the expected target text are used as inputs of a dialogue understanding model, and the dialogue understanding model outputs a first text sequence, a second text sequence, and text similarity, wherein the first text sequence corresponds to the second language, the second text sequence corresponds to the second language, and the text similarity indicates the degree of association between the expected target text and the target text set. The dialogue understanding model includes a monolingual understanding model, a cross-lingual understanding model, and a dialogue discrimination model. Using the source text set and the training sample pair as inputs to a text translation model, and obtaining a third text sequence through the text translation model, wherein the third text sequence corresponds to the second language; updating model parameters of the text translation model according to the first text sequence, the second text sequence, the text similarity, the third text sequence, and the expected target text; The method of using the source text set, the target text set, and the expected target text as inputs of a dialogue understanding model and outputting a first text sequence, a second text sequence, and text similarity through the dialogue understanding model includes: Obtaining a source text embedding vector corresponding to the source text set, obtaining a target text embedding vector corresponding to the target text set, and obtaining an expected target embedding vector corresponding to the expected target text; Using the target text embedding vector and the expected target embedding vector as inputs to the single language understanding model, and obtaining the first text sequence through the single language understanding model; Using the source text embedding vector and the expected target embedding vector as inputs to the cross-language understanding model, and obtaining the second text sequence through the cross-language understanding model; The target text embedding vector and the expected target embedding vector are used as inputs of the dialogue discrimination model, and the text similarity is obtained through the dialogue discrimination model.

2. The method according to claim 1, characterized in that The step of using the target text embedding vector and the expected target embedding vector as inputs to the single language understanding model and obtaining the first text sequence through the single language understanding model includes: Using the target text embedding vector as input to an encoder of the single language understanding model, and obtaining a first target text encoding sequence through the encoder of the single language understanding model, where the first target text encoding sequence is a sequence representation of the target text embedding vector in the second language; The first target text encoding sequence and the expected target embedding vector are used as inputs of a decoder of the single language understanding model, and the first text sequence is obtained through the decoder of the single language understanding model.

3. The method according to claim 1, characterized in that The step of using the source text embedding vector and the expected target embedding vector as inputs to the cross-language understanding model and obtaining the second text sequence through the cross-language understanding model includes: Using the source text embedding vector as input to an encoder of the cross-language understanding model, and obtaining a source text encoding sequence through the encoder of the cross-language understanding model, wherein the source text encoding sequence is a sequence representation of the source text embedding vector in the second language; The source text encoding sequence and the expected target embedding vector are used as inputs of a decoder of the cross-language understanding model, and the second text sequence is obtained through the decoder of the cross-language understanding model.

4. The method according to claim 1, wherein The step of using the target text embedding vector and the expected target embedding vector as inputs to the dialogue discrimination model and obtaining the text similarity through the dialogue discrimination model includes: Using the target text embedding vector as input to an encoder of the dialogue discriminant model, and obtaining a second target text encoding sequence through the encoder of the dialogue discriminant model, where the second target text encoding sequence is a sequence representation of the target text embedding vector in a second language; Using the desired target embedding vector as input to an encoder of the dialogue discriminant model, and obtaining a desired target encoding sequence through the encoder of the dialogue discriminant model, where the desired target encoding sequence is a sequence representation of the desired target embedding vector in the second language; Obtaining a negative sampling text embedding vector corresponding to the target text embedding vector, wherein the negative sampling text embedding vector is a vector representation corresponding to a text that does not belong to the target text set; Using the negatively sampled text embedding vector as input to an encoder of the conversational discrimination model, and obtaining a negatively sampled text encoding sequence through the encoder of the conversational discrimination model, where the negatively sampled text encoding sequence is a sequence representation of the negatively sampled text embedding vector in the second language; The second target text encoding sequence, the expected target encoding sequence, and the negative sampling text encoding sequence are used as inputs of a classifier of the dialogue discrimination model, and the text similarity is obtained through the classifier of the dialogue discrimination model.

5. The method according to claim 1, wherein The obtaining of the source-end text embedding vector corresponding to the source-end text set includes: Performing word segmentation on the source text set to obtain at least two source words, where the at least two source words belong to the first language; Passing the at least two source-end words through a word embedding layer, a position embedding layer, a turn embedding layer, and a role embedding layer, respectively, and performing vector fusion on the vector representation output by each embedding layer to obtain the source-end text embedding vector; The obtaining of the target text embedding vector corresponding to the target text set includes: Performing word segmentation on the target text set to obtain at least two target words, wherein the at least two target words belong to the second language; Passing the at least two target words through a word embedding layer, a position embedding layer, a turn embedding layer, and a role embedding layer, respectively, and performing vector fusion on the vector representation output by each embedding layer to obtain the target text embedding vector; The obtaining of the expected target embedding vector corresponding to the expected target text includes: Performing word segmentation on the expected target text to obtain at least two expected end words, wherein the at least two expected end words belong to the second language; The at least two expected end words are respectively passed through the word embedding layer, the position embedding layer, the turn embedding layer and the role embedding layer, and the vector representation output by each embedding layer is vector fused to obtain the expected target embedding vector.

6. The method according to claim 1, characterized in that Before updating the model parameters of the text translation model according to the first text sequence, the second text sequence, the text similarity, the third text sequence, and the expected target text, the method further includes: Using the target text set and the training sample pairs as inputs of an object discrimination model, and obtaining object similarity through the object discrimination model, wherein the object similarity represents the degree of association between the target text set and the character object used to generate the target text; The updating of the model parameters of the text translation model according to the first text sequence, the second text sequence, the text similarity, the third text sequence, and the expected target text includes: Model parameters of the text translation model are updated according to the first text sequence, the second text sequence, the text similarity, the third text sequence, the object similarity, and the expected target text.

7. A model training device, characterized in that: include: an acquisition unit, configured to acquire a source text set, a target text set, and training sample pairs, wherein the source text set includes source texts corresponding to a first language, the target text set includes target texts corresponding to a second language, and the training sample pairs include initial source texts corresponding to the first language and expected target texts corresponding to the second language; a processing unit configured to use the source text set, the target text set, and the expected target text as inputs of a dialogue understanding model, and output a first text sequence, a second text sequence, and text similarity through the dialogue understanding model, wherein the first text sequence corresponds to the second language, the second text sequence corresponds to the second language, and the text similarity indicates the degree of association between the expected target text and the target text set, wherein the dialogue understanding model includes a monolingual understanding model, a cross-lingual understanding model, and a dialogue discrimination model; The processing unit is further configured to use the source text set and the training sample pair as inputs of a text translation model, and obtain a third text sequence through the text translation model, wherein the third text sequence corresponds to the second language; The processing unit is further configured to update model parameters of the text translation model based on the first text sequence, the second text sequence, the text similarity, the third text sequence, and the expected target text; The processing unit can be used to: Obtaining a source text embedding vector corresponding to the source text set, obtaining a target text embedding vector corresponding to the target text set, and obtaining an expected target embedding vector corresponding to the expected target text; Using the target text embedding vector and the expected target embedding vector as inputs to the single language understanding model, and obtaining the first text sequence through the single language understanding model; Using the source text embedding vector and the expected target embedding vector as inputs to the cross-language understanding model, and obtaining the second text sequence through the cross-language understanding model; The target text embedding vector and the expected target embedding vector are used as inputs of the dialogue discrimination model, and the text similarity is obtained through the dialogue discrimination model.

8. The device according to claim 7, characterized in that The processing unit is specifically configured to: Using the target text embedding vector as input to an encoder of the single language understanding model, and obtaining a first target text encoding sequence through the encoder of the single language understanding model, where the first target text encoding sequence is a sequence representation of the target text embedding vector in the second language; The first target text encoding sequence and the expected target embedding vector are used as inputs of a decoder of the single language understanding model, and the first text sequence is obtained through the decoder of the single language understanding model.

9. The device according to claim 7, characterized in that The processing unit is specifically configured to: Using the source text embedding vector as input to an encoder of the cross-language understanding model, and obtaining a source text encoding sequence through the encoder of the cross-language understanding model, wherein the source text encoding sequence is a sequence representation of the source text embedding vector in the second language; The source text encoding sequence and the expected target embedding vector are used as inputs of a decoder of the cross-language understanding model, and the second text sequence is obtained through the decoder of the cross-language understanding model.

10. The device according to claim 7, characterized in that The processing unit is specifically configured to: Using the target text embedding vector as input to an encoder of the dialogue discriminant model, and obtaining a second target text encoding sequence through the encoder of the dialogue discriminant model, where the second target text encoding sequence is a sequence representation of the target text embedding vector in a second language; Using the desired target embedding vector as input to an encoder of the dialogue discriminant model, and obtaining a desired target encoding sequence through the encoder of the dialogue discriminant model, where the desired target encoding sequence is a sequence representation of the desired target embedding vector in the second language; Obtaining a negative sampling text embedding vector corresponding to the target text embedding vector, wherein the negative sampling text embedding vector is a vector representation corresponding to a text that does not belong to the target text set; Using the negatively sampled text embedding vector as input to an encoder of the conversational discrimination model, and obtaining a negatively sampled text encoding sequence through the encoder of the conversational discrimination model, where the negatively sampled text encoding sequence is a sequence representation of the negatively sampled text embedding vector in the second language; The second target text encoding sequence, the expected target encoding sequence, and the negative sampling text encoding sequence are used as inputs of a classifier of the dialogue discrimination model, and the text similarity is obtained through the classifier of the dialogue discrimination model.

11. The device according to claim 7, characterized in that The processing unit is specifically configured to: Performing word segmentation on the source text set to obtain at least two source words, where the at least two source words belong to the first language; Passing the at least two source-end words through a word embedding layer, a position embedding layer, a turn embedding layer, and a role embedding layer, respectively, and performing vector fusion on the vector representation output by each embedding layer to obtain the source-end text embedding vector; The processing unit is specifically used to: Performing word segmentation on the target text set to obtain at least two target words, wherein the at least two target words belong to the second language; Passing the at least two target words through a word embedding layer, a position embedding layer, a turn embedding layer, and a role embedding layer, respectively, and performing vector fusion on the vector representation output by each embedding layer to obtain the target text embedding vector; The processing unit can be used to: Performing word segmentation on the expected target text to obtain at least two expected end words, wherein the at least two expected end words belong to the second language; The at least two expected end words are respectively passed through the word embedding layer, the position embedding layer, the turn embedding layer and the role embedding layer, and the vector representation output by each embedding layer is vector fused to obtain the expected target embedding vector.

12. The device according to claim 7, characterized in that The processing unit is further configured to use the target text set and the training sample pairs as inputs of an object discrimination model, and obtain object similarity through the object discrimination model, wherein the object similarity represents the degree of association between the target text set and the character object used to generate the target text; The processing unit is further configured to update model parameters of the text translation model according to the first text sequence, the second text sequence, the text similarity, the third text sequence, the object similarity, and the expected target text.

13. A computer device, characterized in that: include: memories, transceivers, processors, and bus systems; Wherein, the memory is used to store programs; The processor is configured to implement the method according to any one of claims 1 to 6 when executing the program in the memory; The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

14. A computer-readable storage medium comprising instructions, which, when executed on a computer, causes the computer to perform the method according to any one of claims 1 to 6.

15. A computer program product, characterized in that The computer program product comprises instructions, and when the instructions are executed on a computer device, the computer is caused to execute the method according to any one of claims 1 to 6 .

Images