A multi-round dialogue rewriting method and device

Abstract

The embodiment of the present invention provides a method and device for rewriting a multi-round dialogue, wherein the method comprises: obtaining a target multi-round dialogue rewriting model, a dialogue to be rewritten, and a historical dialogue corresponding to the dialogue to be rewritten; inputting the dialogue to be rewritten and the historical dialogue into the target multi-round dialogue rewriting model, so as to predict the sequence label corresponding to each sequence in the dialogue to be rewritten through the target multi-round dialogue rewriting model, and rewriting the dialogue to be rewritten according to the sequence label to obtain the target dialogue. This allows the advantages of sequence labeling and autoregressive generation to be combined to complete the multi-round dialogue rewriting, effectively improving the quality of rewriting and the speed of generation.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a multi-round dialogue rewriting method and a multi-round dialogue rewriting device. Background Art

[0002] With the development of deep learning and big data technology, open-domain human-computer dialogue systems have become an important research work in the field of artificial intelligence, and have been widely used in intelligent customer service, chatbots, virtual assistants, etc. According to the number of rounds of human-computer interaction, open-domain dialogue systems are divided into: single-round open-domain dialogue systems and multi-round open-domain dialogue systems. Through massive single-round social dialogue data and retrieval-based or generative modeling methods based on deep learning, single-round open-domain dialogue systems have produced some good results; however, in multi-round dialogue systems, there are generally only two people in the dialogue. In order to make the communication concise, smooth and efficient, some omissions or references are usually made according to the context, so the practice of the single-round dialogue system cannot be directly applied, for example:

[0003] “What’s your favorite movie?”

[0004] “Titanic.”

[0005] "Why?"

[0006] If it is a semantic dialogue for completion, it should be like this:

[0007] “What’s your favorite movie?”

[0008] “My favorite movie is Titanic.”

[0009] “Why is Titanic your favorite movie?”

[0010] Since there are common incompleteness in the multi-round dialogue system, such as missing, omitting, and referring, there is still great potential for the development of multi-round open domain dialogue systems. The general approach is to break down the multi-round dialogue generation into a multi-round dialogue rewriting task + a single-round dialogue task, that is, first rewrite the sentences with missing semantics into independent and complete semantic sentences based on the contextual semantic information, and then do it according to the single-round dialogue. Therefore, multi-round dialogue rewriting has gradually become an important branch of research on multi-round dialogue systems.

[0011] Currently, there are two main solutions for multi-round dialogue rewriting:

[0012] Solution 1: Combine the transformer version of seq2seq and the pointer generation network to create a rewriting model for sentences with missing semantics (T-Ptr-λ), and perform attention decoding operations separately to distinguish the importance of the context and the current missing sentence. However, since the rewriting model (T-Ptr-λ) only uses 6 layers of transformers as encoder and decoder, the decoder is performed twice to distinguish the importance of the context and the current missing semantics sentence. This results in the model being unable to load some pre-trained BERT-like weights, and it is also very time-consuming to run a total of 18 layers of transformer-blocks in series to decode and generate from scratch.

[0013] Solution 2: Use the cascade model (PAC) for rewriting. First, use the BERT model to find the words that need to be modified, and then use the seq2seq model to generate the rewritten sentence. Although the cascade model (PAC) uses the pre-trained BERT to improve the quality of rewriting to a certain extent, as a cascade method, it is necessary to complete the stage of finding and modifying words first, and then proceed to the rewriting stage; this will inevitably lead to the error in the first stage being amplified in the second stage, and it is also time-consuming to generate the entire model from scratch in the form of BERT+seq2seq. Summary of the invention

[0014] In view of the above problems, embodiments of the present invention are proposed to provide a multi-round dialogue rewriting method and a corresponding multi-round dialogue rewriting device that overcome the above problems or at least partially solve the above problems.

[0015] The embodiment of the present invention discloses a multi-round dialogue rewriting method, comprising:

[0016] Obtain a target multi-round dialogue rewriting model, a dialogue to be rewritten, and a historical dialogue corresponding to the dialogue to be rewritten;

[0017] The dialogue to be rewritten and the historical dialogue are input into the target multi-round dialogue rewriting model, so as to predict the sequence label corresponding to each sequence in the dialogue to be rewritten through the target multi-round dialogue rewriting model, and the dialogue to be rewritten is rewritten according to the sequence label to obtain the target dialogue.

[0018] Optionally, the sequence tags include a deletion tag, a replacement tag, and a retention tag, and rewriting the dialogue to be rewritten according to the sequence tags to obtain a target dialogue includes:

[0019] The sequence corresponding to the deletion tag is deleted, the sequence corresponding to the retention tag is retained, and a replacement is generated for the sequence corresponding to the replacement tag to obtain a target dialogue.

[0020] Optionally, before the step of obtaining the target multi-round dialogue rewriting model, the dialogue to be rewritten and the historical dialogue corresponding to the dialogue to be rewritten, the step further includes:

[0021] Get the original training data;

[0022] Labeling the original training data to obtain target training data with labels;

[0023] The initial multi-round dialogue rewriting model is trained according to the target training data to obtain the target multi-round dialogue rewriting model.

[0024] Optionally, the training the initial multi-round dialogue rewriting model according to the target training data to obtain the target multi-round dialogue rewriting model includes:

[0025] Encoding the target training data to obtain encoded data, wherein the encoded data includes encoding information of the historical sentences and encoding information of the current rewritten sentence;

[0026] Learning the sequence labels corresponding to each sequence in the current rewritten sentence according to the encoding information of the current rewritten sentence, and calculating the first loss value of the learning label;

[0027] Decoding the current rewritten sentence to obtain a rewritten sentence according to the sequence label and the encoded data, and calculating a second loss value of the encoding;

[0028] According to the first loss value and the second loss value, the parameters of the original multi-round dialogue rewriting model are updated to obtain the target multi-round dialogue rewriting model.

[0029] Optionally, decoding the current rewritten sentence to obtain a rewritten sentence according to the sequence label and the encoded data to obtain decoded data includes:

[0030] Determining a target dictionary from the full dictionary and the historical information according to the encoded data;

[0031] Determining, from the target dictionary, a generated word corresponding to each sequence in the current rewritten sentence;

[0032] A rewritten sentence is obtained according to the sequence labels and the generated words.

[0033] Optionally, the original training data includes a current rewritten sentence and a historical sentence corresponding to the current rewritten sentence; and the step of labeling the original training data to obtain target training data with labels includes:

[0034] Defining a rewritten sentence according to the current rewritten sentence and the historical sentence;

[0035] Calculating the longest common subsequence between the current rewritten sentence and the rewritten sentence;

[0036] Aligning the current rewritten sentence, the rewritten sentence and the longest common subsequence to obtain an alignment result;

[0037] The current rewritten sentence is annotated according to the rewritten sentence and the alignment result to obtain target training data with labels.

[0038] Optionally, before the step of calculating the longest common subsequence between the sentence to be rewritten and the modified sentence, the step further includes:

[0039] A placeholder is added between every two words in the current rewritten sentence.

[0040] The embodiment of the present invention also discloses a multi-round dialogue rewriting device, comprising:

[0041] A dialogue acquisition module, used to acquire a target multi-round dialogue rewriting model, a dialogue to be rewritten, and a historical dialogue corresponding to the dialogue to be rewritten;

[0042] A dialogue rewriting module is used to input the dialogue to be rewritten and the historical dialogue into the target multi-round dialogue rewriting model, so as to predict the sequence label corresponding to each sequence in the dialogue to be rewritten through the target multi-round dialogue rewriting model, and rewrite the dialogue to be rewritten according to the sequence label to obtain the target dialogue.

[0043] The embodiment of the present invention further discloses an electronic device, including:

[0044] A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform a method as described in any one of the embodiments of the present invention.

[0045] An embodiment of the present invention further discloses a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in any one of the embodiments of the present invention is executed.

[0046] The embodiments of the present invention include the following advantages:

[0047] In the embodiment of the present invention, by obtaining the target multi-round dialogue rewriting model, the dialogue to be rewritten and the historical dialogue corresponding to the dialogue to be rewritten; the dialogue to be rewritten and the historical dialogue are input into the target multi-round dialogue rewriting model, so as to predict the sequence label corresponding to each sequence in the dialogue to be rewritten by the target multi-round dialogue rewriting model, and rewrite the dialogue to be rewritten according to the sequence label to obtain the target dialogue. This allows the advantages of sequence labeling and autoregressive generation to be combined to complete the multi-round dialogue rewriting, effectively improving the quality of rewriting and the speed of generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flowchart of a multi-round dialogue rewriting method provided by an embodiment of the present invention;

[0049] Figure 2 is a structural diagram of a multi-round dialogue rewriting model provided by an embodiment of the present invention;

[0050] Figure 3 is a structural block diagram of a multi-round dialogue rewriting device provided by an embodiment of the present invention;

[0051] Figure 4 is a structural block diagram of an electronic device provided by an embodiment of the present invention;

[0052] Figure 5 It is a structural block diagram of a storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] In one embodiment of the present invention, the multi-round dialogue rewriting method can be run on a terminal device or a server. The terminal device can be a local terminal device. When the multi-round dialogue rewriting method is run on a server, the multi-round dialogue rewriting method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0055] In an optional implementation, various cloud applications can be run under the cloud interaction system, such as cloud games. Taking cloud games as an example, cloud games refer to a game mode based on cloud computing. In the operation mode of cloud games, the operation subject of the game program and the game screen presentation subject are separated, the storage and operation of the multi-round dialogue rewriting method are completed on the cloud game server, and the role of the client device is used for receiving and sending data and presenting the game screen. For example, the client device can be a display device with data transmission function close to the user side, such as a first terminal device, a TV, a computer, a handheld computer, etc.; but the terminal device for performing the multi-round dialogue rewriting method is a cloud game server in the cloud. When playing the game, the player operates the client device to send an operation instruction to the cloud game server. The cloud game server runs the game according to the operation instruction, encodes and compresses the game screen and other data, and returns it to the client device through the network. Finally, the client device decodes and outputs the game screen.

[0056] In an optional embodiment, the terminal device may be a local terminal device. Taking a game as an example, the local terminal device stores a game program and is used to present a game screen. The local terminal device is used to interact with the player through a graphical user interface, that is, the game program is downloaded and installed by an electronic device and run conventionally. The local terminal device may provide the graphical user interface to the player in a variety of ways, for example, it may be rendered and displayed on a display screen of the terminal, or provided to the player through a holographic projection. For example, the local terminal device may include a display screen and a processor, the display screen is used to present a graphical user interface, the graphical user interface includes a game screen, and the processor is used to run the game, generate a graphical user interface, and control the display of the graphical user interface on the display screen.

[0057] Reference Figure 1 , shows a flowchart of a multi-round dialogue rewriting method embodiment of the present invention, which may specifically include the following steps:

[0058] Step 101, obtaining a target multi-round dialogue rewriting model, a dialogue to be rewritten, and a historical dialogue corresponding to the dialogue to be rewritten;

[0059] The target multi-round dialogue rewriting model is a pre-trained neural network model, which is used to rewrite the dialogue to be rewritten and output the required rewritten dialogue. The dialogue to be rewritten is the dialogue content that needs to be rewritten, and the dialogue to be rewritten can usually contain content of multiple characters. In order to better rewrite the dialogue to be rewritten, the historical dialogue corresponding to the dialogue to be rewritten is usually obtained, wherein the obtained historical dialogue can be one or more dialogue contents. For example, the dialogue to be rewritten is "Andy Lau's songs are good", and the historical dialogues corresponding to the dialogue to be rewritten are "Do you listen to Andy Lau's songs" and "His songs are good."

[0060] Step 102: input the dialogue to be rewritten and the historical dialogue into the target multi-round dialogue rewriting model, so as to predict the sequence label corresponding to each sequence in the dialogue to be rewritten through the target multi-round dialogue rewriting model, and rewrite the dialogue to be rewritten according to the sequence label to obtain the target dialogue.

[0061] In an embodiment of the present invention, the dialogue to be rewritten and the historical dialogue can be input into the target multi-round dialogue rewriting model, so as to predict the sequence label corresponding to each sequence in the dialogue to be rewritten through the target multi-round dialogue rewriting model, and rewrite the dialogue to be rewritten according to the sequence label to obtain the target dialogue. Specifically, the target multi-round dialogue rewriting model can include a sequence labeling algorithm and an autoregressive algorithm. First, the dialogue to be rewritten is labeled by the sequence labeling algorithm to obtain the sequence label corresponding to each sequence in the dialogue to be rewritten, and then the content corresponding to each sequence label is parsed by the autoregressive algorithm to generate new content, and finally the required target dialogue is generated according to the parsed content.

[0062] Specifically, the sequence tag is used to indicate the tag of the content corresponding to each sequence in the dialogue to be rewritten, and the sequence tag may include a deletion tag DELETE, a retention tag KEEP and a replacement tag CHANGE, wherein the deletion tag is used to indicate that the content corresponding to the sequence needs to be deleted, the retention tag is used to indicate that the content corresponding to the sequence needs to be retained, and the replacement tag is used to indicate that the content corresponding to the sequence needs to be replaced.

[0063] In a preferred embodiment of the present invention, rewriting the dialogue to be rewritten according to the sequence tag to obtain a target dialogue includes:

[0064] The sequence corresponding to the deletion tag is deleted, the sequence corresponding to the retention tag is retained, and replacement content is generated for the sequence corresponding to the replacement tag to obtain the target dialogue.

[0065] When parsing the conversation content through the autoregressive algorithm, only the content with the replacement label as the sequence label is parsed, and the content with the deleted label and the retained label position is filled with meaningless placeholders. Finally, the target conversation is generated based on the parsed content and the sequence label. Specifically, the sequence corresponding to the deleted label is deleted, the sequence corresponding to the retained label is retained, and the sequence corresponding to the replaced label is replaced with the content generated at the position. Finally, the target conversation can be obtained by sorting the content of each sequence according to its position.

[0066] In a preferred embodiment of the present invention, before step 101, the following steps may also be included:

[0067] Acquire original training data; annotate the original training data to obtain target training data with labels; train an initial multi-round dialogue rewriting model according to the target training data to obtain the target multi-round dialogue rewriting model.

[0068] The original training data may be used as dialogue data for training, and the original training data may include the current rewritten sentence and the historical sentences corresponding to the current rewritten sentence.

[0069] In a specific implementation, the original training data may be labeled to obtain target training data with labels, and then the initial multi-round dialogue rewriting model may be trained based on the target training data to obtain the target multi-round dialogue rewriting model.

[0070] In a preferred embodiment of the present invention, the step of labeling the original training data to obtain target training data with labels includes:

[0071] A rewritten sentence is defined according to the current rewritten sentence and the historical sentence; the longest common subsequence between the current rewritten sentence and the rewritten sentence is calculated; the current rewritten sentence, the rewritten sentence and the longest common subsequence are aligned to obtain an alignment result; the current rewritten sentence is labeled according to the rewritten sentence and the alignment result to obtain target training data with labels.

[0072] Specifically, the problem of multi-round rewriting tasks can be defined as follows:

[0073] f(H,U)=R.

[0074] in, It is a historical sentence. is the current rewritten sentence, and R is the rewritten sentence.

[0075] After defining multiple rounds of rewriting tasks, first, calculate the longest common subsequence between the current rewritten sentence and the rewritten sentence. In the specific implementation, the longest common subsequence can be calculated by the LCS algorithm (Longest Common Sequence). After calculating the longest common subsequence, align the current rewritten sentence, the rewritten sentence, and the longest common subsequence to obtain an alignment result. In the specific implementation, a greedy algorithm can be used to align the current rewritten sentence, the rewritten sentence, and the longest common subsequence. Finally, the current rewritten sentence is annotated according to the rewritten sentence and the alignment result to obtain target training data with labels. Specifically, for the words deleted after the rewriting, a deletion label is added, for the words replaced after the rewriting, a replacement label is added, and for the words that have not changed after the rewriting, a retention label is added.

[0076] It should be noted that in order to increase the speed of training the model, after the original training data is labeled, the labeled target training data can be stored in the database, so that when the model needs to be trained later, the labeled target training data can be directly obtained from the database.

[0077] In a preferred embodiment of the present invention, before the step of calculating the longest common subsequence between the sentence to be rewritten and the modified sentence, the method further comprises:

[0078] A placeholder is added between every two words in the current rewritten sentence.

[0079] In the embodiment of the present invention, in order to avoid the ambiguity of multiple sequence tags, a placeholder token (u k ), so at this time

[0080] In a preferred embodiment of the present invention, the training of the initial multi-round dialogue rewriting model according to the target training data to obtain the target multi-round dialogue rewriting model may include the following sub-steps:

[0081] Sub-step S11, encoding the target training data to obtain encoded data, wherein the encoded data includes encoding information of the historical sentences and encoding information of the current rewritten sentence;

[0082] Specifically, the standard transformer-block can be used as the Encoder, such as Figure 2 As shown in the figure, Embedder is a word embedding module, Multi-headAttention is a head-to-head attention module, Layer Norm is a normalization module, and FeedForward is a feedforward network module. The historical sentence H and the current rewritten sentence U are concatenated together as input, and three Embedding layers are used to represent each word, namely Word Embedding (WE) representing word information, Position Embedding (PE) representing the position information of the word in the sentence, and Turn Embedding (TE) representing the number of turns of the sentence in multiple rounds of chat. Therefore, for the input word w i The input Embedding representation can be obtained as follows:

[0083]

[0084] Among them, LN is the Layer Normalize normalization layer. After the input word is stacked by the transformer-block of the normalization layer, the representation of the normalization layer is:

[0085] E (l) = TransformerBlock(E (l-1) ).

[0086] Put E (L) Split into two parts, namely the encoding representation of historical sentences E h and the current rewritten sentence encoding representation E u , where E h and E u They are respectively expressed as follows:

[0087] E h ={h1,h2,…,h m};

[0088] E u ={u1,u2,…,u 2n+1}.

[0089] Sub-step S12, learning to obtain a sequence label corresponding to each sequence in the current rewritten sentence according to the encoding information of the current rewritten sentence, and calculating a first loss value of the learned label;

[0090] Specifically, the E obtained by encoding in the above sub-step S11 can be u To learn sequence labels, such as Figure 2 As shown in , TagFC is a fully connected layer. If the i-th label is y i , then the probability of Tagger learning is:

[0091] p(y i |u i )=softmax(W t ·u i +b t )

[0092] Among them, W t and b t are the parameters that need to be learned.

[0093] The probability calculation loss of Tagger learning obtained from the above calculation is:

[0094] loss tag =-∑ i logp(y i |u i )

[0095] Sub-step S13, decoding the current rewritten sentence to obtain a rewritten sentence according to the sequence label and the encoded data, and calculating a second loss value of the encoding;

[0096] In the embodiment of the present invention, only those words whose sequences are marked as CHANGE are decoded and generated, such as Figure 2 As shown, LSTM is a unidirectional LSTM module, and VocabFC is a fully connected layer mapped to the dictionary. Because the multi-round dialogue rewriting model in the embodiment of the present invention is a joint learning model, a sequence will be generated in parallel at each word position of U, but only the position marked with CHANGE is meaningful, and other label positions are meaningless placeholders.

[0097] In a preferred embodiment of the present invention, the sub-step S13 may include the following sub-steps:

[0098] According to the encoded data, a target dictionary is determined from the full dictionary and historical information; a generated word corresponding to each sequence in the current rewritten sentence is determined from the target dictionary; and a rewritten sentence is obtained according to the sequence label and the generated word.

[0099] In the embodiment of the present invention, a unidirectional LSTM is used as a decoder, and the initial state s0 of the decoding LSTM at the i-th character position in U is set to be u i ∈E u , the state of LSTM at the tth moment is s t =LSTM(WE(x t ), s t-1 ), where x t is the word generated by the previous decoding. In order to dynamically choose whether to generate from the full dictionary or from historical information, a pointer generation network mechanism is introduced. In the decoding at time t, s is used t Collect historical information, the formula is as follows:

[0100]

[0101] Among them, j is the index of historical information, c t is the coverage vector at time t, which is initialized from 0, c t Satisfies the following formula:

[0102]

[0103] Among them, a t Satisfy the formula: a t =softmax(e t ). In the calculation of a tAfter that, the attention can be further calculated as:

[0104]

[0105] Then calculate the probability of generating a word as:

[0106]

[0107]

[0108]

[0109] Among them, σ is the sigmoid function, g is the weight of weighing whether to generate from the full dictionary or copy from historical information, and p(x t+1 ) is the probability distribution of the final generated word.

[0110] In order to prevent copying of the same words, a duplicate penalty loss is introduced:

[0111]

[0112] Finally, the decoding loss is:

[0113]

[0114] Among them, i corresponds to the i-th character in U, and λ is a hyperparameter.

[0115] Sub-step S14: updating the parameters of the original multi-round dialogue rewriting model according to the first loss value and the second loss value to obtain the target multi-round dialogue rewriting model.

[0116] In the embodiment of the present invention, the loss of the encoding operation and the decoding operation can be jointly learned, which is expressed as:

[0117] loss = αloss tag +loss dec .

[0118] Among them, α is a hyperparameter that adjusts the encoding and decoding operations.

[0119] In an embodiment of the present invention, a target multi-round dialogue rewriting model, a dialogue to be rewritten, and a historical dialogue corresponding to the dialogue to be rewritten are obtained; the dialogue to be rewritten and the historical dialogue are input into the target multi-round dialogue rewriting model, so that the sequence label corresponding to each sequence in the dialogue to be rewritten is predicted by the target multi-round dialogue rewriting model, and the dialogue to be rewritten is rewritten according to the sequence label to obtain the target dialogue. This allows the advantages of sequence labeling and autoregressive generation to be combined to complete multi-round dialogue rewriting, and uses a standard transformer stacking structure to complete sequence labeling and decoding generation, which can easily load the weights of the pre-trained model and effectively improve the quality of rewriting and the speed of generation.

[0120] Different from the traditional seq2seq methods of T-Ptr-λ and PAC that generate rewritten text from scratch, the multi-round dialogue rewriting model (hereinafter referred to as SARG) in the embodiment of the present invention proposes a method of combining semi-autoregression with semi-sequence labeling, which effectively combines the advantages of sequence labeling and autoregression to complete multi-round rewriting. A comparative experiment was conducted on the public dataset Restoration-200k[2] using the same training set / validation set / test set, and the results are as follows.

[0121] Model f1 f2 f3 bleu1 bleu2 rouge1 rouge2 Average time PAC 63.7 49.7 40.4 89.9 86.3 91.6 82.8 — T-Ptr-λ 51.0 40.4 33.3 90.3 87.4 90.1 83.0 602s SARG 62.4 52.5 46.3 92.2 89.6 92.1 86.0 50s

[0122] in:

[0123]

[0124]

[0125]

[0126] It can be seen that, except for the f1 index, the other indexes of the SARG of the embodiment of the present invention are better than those of the other two algorithms. Specifically, compared with the PAC method, the f2 of SARG is improved by 2.8 points, the f3 is improved by 5.9 points, the bleu1 is improved by 2.3 points, the bleu2 is improved by 3.3 points, the rouge1 is improved by 0.5 points, and the rouge2 is improved by 3.2 points; compared with T-Ptr-λ, the f1 of SARG is improved by 11.4 points, the f2 is improved by 12.1 points, the f3 is improved by 13 points, the bleu1 is improved by 1.9 points, the bleu2 is improved by 2.2 points, the rouge1 is improved by 2 points, and the rouge2 is improved by 3 points. Moreover, the speed of SARG is more than 10 times faster than that of T-Ptr-λ, and the speed of PAC is slower because it is a two-stage method.

[0127] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0128] Reference Figure 3 , shows a structural block diagram of an embodiment of a multi-round dialogue rewriting device of the present invention, which may specifically include the following modules:

[0129] A dialogue acquisition module 301 is used to acquire a target multi-round dialogue rewriting model, a dialogue to be rewritten, and a historical dialogue corresponding to the dialogue to be rewritten;

[0130] The dialogue rewriting module 302 is used to input the dialogue to be rewritten and the historical dialogue into the target multi-round dialogue rewriting model, so as to predict the sequence label corresponding to each sequence in the dialogue to be rewritten through the target multi-round dialogue rewriting model, and rewrite the dialogue to be rewritten according to the sequence label to obtain the target dialogue.

[0131] In a preferred embodiment of the present invention, the sequence tags include a deletion tag, a replacement tag and a retention tag, and the dialogue rewriting module 302 includes:

[0132] The dialogue rewriting submodule is used to delete the sequence corresponding to the deletion tag, retain the sequence corresponding to the retention tag, and generate a replacement for the sequence corresponding to the replacement tag to obtain a target dialogue.

[0133] In a preferred embodiment of the present invention, the device further comprises:

[0134] A data acquisition module, used to acquire original training data;

[0135] A data labeling module, used to label the original training data to obtain target training data with labels;

[0136] A training module is used to train the initial multi-round dialogue rewriting model according to the target training data to obtain the target multi-round dialogue rewriting model.

[0137] In a preferred embodiment of the present invention, the training module includes:

[0138] An encoding submodule, used for encoding the target training data to obtain encoded data, wherein the encoded data includes encoding information of the historical sentences and encoding information of the current rewritten sentence;

[0139] A label learning submodule, used for learning the sequence label corresponding to each sequence in the current rewritten sentence according to the encoding information of the current rewritten sentence, and calculating a first loss value of the learning label;

[0140] A decoding submodule, configured to decode the current rewritten sentence according to the sequence label and the encoded data to obtain a rewritten sentence, and calculate a second loss value of the encoding;

[0141] A parameter updating submodule is used to update the parameters of the original multi-round dialogue rewriting model according to the first loss value and the second loss value to obtain the target multi-round dialogue rewriting model.

[0142] In a preferred embodiment of the present invention, the decoding submodule includes:

[0143] a target dictionary determining unit, configured to determine a target dictionary from the full dictionary and the historical information according to the encoded data;

[0144] a character generation unit, used to determine, from the target dictionary, a generated character corresponding to each sequence in the current rewritten sentence;

[0145] The rewriting unit is used to obtain a rewritten sentence according to the sequence label and the generated word.

[0146] In a preferred embodiment of the present invention, the original training data includes the current rewritten sentence and the historical sentences corresponding to the current rewritten sentence; the data annotation module includes:

[0147] A definition submodule, used for defining a rewritten sentence according to the current rewritten sentence and the historical sentence;

[0148] A sequence calculation submodule, used for calculating the longest common subsequence between the current rewritten sentence and the rewritten sentence;

[0149] An alignment submodule, used for aligning the current rewritten sentence, the rewritten sentence and the longest common subsequence to obtain an alignment result;

[0150] The annotation submodule is used to annotate the current rewritten sentence according to the rewritten sentence and the alignment result to obtain target training data with labels.

[0151] In a preferred embodiment of the present invention, the data annotation module further includes:

[0152] The placeholder adding submodule is used to add a placeholder between every two words in the current rewritten sentence.

[0153] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0154] The embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, including:

[0155] Processor 401, storage medium 402 and bus 403, the storage medium 402 stores machine-readable instructions executable by the processor 401, when the electronic device is running, the processor 401 communicates with the storage medium 402 through the bus 403, and the processor 401 executes the machine-readable instructions to execute the method as described in any one of the embodiments of the present invention. The specific implementation method and technical effect are similar to those in the method embodiment, and will not be repeated here.

[0156] The embodiment of the present invention also provides a storage medium, such as Figure 5 As shown, the storage medium stores a computer program 501, and the computer program 501 is executed by a processor to execute the method as described in any one of the embodiments of the present invention. The specific implementation method and technical effect are similar to those of the method embodiment, and will not be repeated here.

[0157] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0158] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0159] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0160] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0161] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0162] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0163] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0164] The above is a detailed introduction to a multi-round conversation rewriting method and a multi-round conversation rewriting device provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A multi-round dialogue rewriting method, characterized in that: include: Acquire original training data, wherein the original training data includes a current rewritten sentence and a historical sentence corresponding to the current rewritten sentence; Defining a rewritten sentence according to the current rewritten sentence and the historical sentence; Calculating the longest common subsequence between the current rewritten sentence and the rewritten sentence; Aligning the current rewritten sentence, the rewritten sentence, and the longest common subsequence to obtain an alignment result; Annotating the current rewritten sentence according to the rewritten sentence and the alignment result to obtain target training data with labels; Training the initial multi-round dialogue rewriting model according to the target training data to obtain a target multi-round dialogue rewriting model; Obtaining the target multi-round dialogue rewriting model, the dialogue to be rewritten, and the historical dialogue corresponding to the dialogue to be rewritten; The dialogue to be rewritten and the historical dialogue are input into the target multi-round dialogue rewriting model, so as to predict the sequence label corresponding to each sequence in the dialogue to be rewritten through the target multi-round dialogue rewriting model, and the dialogue to be rewritten is rewritten according to the sequence label to obtain the target dialogue.

2. The method according to claim 1, characterized in that The sequence tags include a deletion tag, a replacement tag, and a retention tag, and rewriting the dialogue to be rewritten according to the sequence tags to obtain a target dialogue includes: The sequence corresponding to the deletion tag is deleted, the sequence corresponding to the retention tag is retained, and a replacement is generated for the sequence corresponding to the replacement tag to obtain a target dialogue.

3. The method according to claim 1, characterized in that: The training of the initial multi-round dialogue rewriting model according to the target training data to obtain the target multi-round dialogue rewriting model includes: Encoding the target training data to obtain encoded data, wherein the encoded data includes encoding information of the historical sentences and encoding information of the current rewritten sentence; Learning the sequence labels corresponding to each sequence in the current rewritten sentence according to the encoding information of the current rewritten sentence, and calculating the first loss value of the learning label; Decoding the current rewritten sentence to obtain a rewritten sentence according to the sequence label and the encoded data, and calculating a second loss value of the encoding; According to the first loss value and the second loss value, the parameters of the original multi-round dialogue rewriting model are updated to obtain the target multi-round dialogue rewriting model.

4. The method according to claim 3, characterized in that The decoding of the current rewritten sentence to obtain a rewritten sentence according to the sequence tag and the encoded data includes: Determining a target dictionary from the full dictionary and the historical information according to the encoded data; Determining, from the target dictionary, a generated word corresponding to each sequence in the current rewritten sentence; A rewritten sentence is obtained according to the sequence labels and the generated words.

5. The method according to claim 1, characterized in that Before calculating the longest common subsequence between the current rewritten sentence and the rewritten sentence, the method further includes: A placeholder is added between every two words in the current rewritten sentence.

6. A multi-round dialogue rewriting device, characterized in that: include: A data acquisition module, used to acquire original training data, wherein the original training data includes a current rewritten sentence and a historical sentence corresponding to the current rewritten sentence; A definition submodule, used for defining a rewritten sentence according to the current rewritten sentence and the historical sentence; A sequence calculation submodule, used for calculating the longest common subsequence between the current rewritten sentence and the rewritten sentence; An alignment submodule, used for aligning the current rewritten sentence, the rewritten sentence and the longest common subsequence to obtain an alignment result; A labeling submodule, used for labeling the current rewritten sentence according to the rewritten sentence and the alignment result to obtain target training data with labels; A training module, used to train the initial multi-round dialogue rewriting model according to the target training data to obtain a target multi-round dialogue rewriting model; A dialogue acquisition module, used to acquire the target multi-round dialogue rewriting model, the dialogue to be rewritten, and the historical dialogue corresponding to the dialogue to be rewritten; A dialogue rewriting module is used to input the dialogue to be rewritten and the historical dialogue into the target multi-round dialogue rewriting model, so as to predict the sequence label corresponding to each sequence in the dialogue to be rewritten through the target multi-round dialogue rewriting model, and rewrite the dialogue to be rewritten according to the sequence label to obtain the target dialogue.

7. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to perform the method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is executed.

Images