A method and device for multi-turn dialogue domain recognition based on sentence completion

Abstract

The present invention discloses a multi-turn dialogue domain recognition method and device based on sentence completion. The method includes: converting historical dialogue corpus and each domain into semantic vectors; sequentially considering the Nth domain, inputting the semantic vector of this domain and the semantic vector of the historical corpus into a sentence completion model to obtain a completed sentence; inputting the completed sentence of the Nth domain into a validity determination model. If it is determined to be valid, the Nth domain is the recognition result; otherwise, jump back to step two and continue to consider the (N + 1)th domain. The present invention combines the multi-turn dialogue domain recognition task and the sentence completion task, and can well recognize the situation where the user replies with an elliptical sentence in a multi-turn dialogue. In addition, the present invention not only utilizes the historical information of the dialogue, but also ensures that the historical information will not interfere with the domain switching, thereby improving the recognition accuracy.

Description

Technical Field

[0001] The present invention relates to the technical fields of natural language processing and deep learning, and particularly relates to a multi-turn dialogue domain recognition method and device based on sentence completion. Background Art

[0002] Since the early days of artificial intelligence research, people have been committed to developing highly intelligent human-computer dialogue systems. Alan Turing proposed the Turing Test in 1950, believing that if humans cannot distinguish whether the entity they are conversing with is a machine or a human, then it can be said that the machine has passed the Turing Test and has a high level of intelligence. With the continuous development of information technology and natural language processing technology, human-computer dialogue systems have been initially applied in scenarios such as intelligent customer service and intelligent speakers, and people's expectations have gradually changed from single-domain single-turn dialogue systems to multi-domain multi-turn dialogue systems.

[0003] In a multi-domain multi-turn dialogue system, the first step is domain recognition. The chat computer obtains the user's input, correctly classifies it into the corresponding domain, and then conducts subsequent tasks based on that domain. Therefore, the accuracy of domain recognition directly determines the service quality of the entire dialogue system.

[0004] One approach is to identify the domain only based on the current turn of the sentence. Earlier methods used prior knowledge or templates of the domain for identification, and later classification models based on statistical methods emerged. With the development of deep learning technology, classification models such as convolutional neural networks, recurrent neural networks, and fastText have also appeared. However, these methods do not utilize historical information and cannot handle the situation where the user's reply is an ellipsis sentence.

[0005] Another approach is to fuse the features of historical sentences and the current turn of the sentence as input to train a classification model. However, when the domains of historical sentences and the current turn of the sentence are inconsistent, the historical sentences completely become interfering information, greatly affecting the classification accuracy. In this case, it may be necessary for the user to repeatedly emphasize to successfully switch the domain, which greatly affects the user experience.

[0006] In summary, the existing domain recognition models have the following problems:

[0007] (1) The single-turn based domain recognition model only utilizes the current turn of information and does not utilize historical information. The flexibility of the model is not high, and it cannot handle the situation where users often reply with ellipsis sentences in actual conversations.

[0008] (2) Some domain recognition models that utilize historical information simply combine historical information and the current turn of information without considering that historical information contains interfering information during domain switching, resulting in a low accuracy rate of the model. Summary of the Invention

[0009] The present invention provides a multi-turn dialogue domain recognition method and device based on sentence completion to solve the above technical problems.

[0010] The technical solution adopted by the present invention is: to provide a multi-turn dialogue domain recognition method based on sentence completion, including:

[0011] Step 1, convert historical dialogue corpus and each domain into semantic vectors;

[0012] Step 2, consider the Nth domain in sequence, input the semantic vector of this domain and the semantic vector of the historical corpus into the sentence completion model to obtain the completed sentence;

[0013] Step 3, input the completed sentence of the Nth domain into the validity determination model. If it is determined to be valid, the Nth domain is the recognition result; otherwise, jump back to Step 2 and continue to consider the (N + 1)th domain.

[0014] Further, the method for obtaining semantic vectors includes: a semantic encoding method based on deep learning.

[0015] Further, in Step 2, the method for considering domains in sequence includes, for non-first-round sentences, taking the domain of the previous-round sentence as the primary consideration domain.

[0016] Further, the method for constructing the sentence completion model includes: a Seq2Seq model or a Pointer-Generator Networks model.

[0017] Further, after constructing the sentence completion model, it also includes setting a loss function and setting parameters for iteratively updating the sentence completion model.

[0018] Further, the training method of the sentence completion model includes: obtaining corpus data, processing the text in the corpus according to actual needs, converting the processed text into semantic vectors, and inputting the semantic vectors into the sentence completion model for training to obtain a trained sentence completion model.

[0019] Further, the method for constructing the validity determination model includes: a classification model method based on deep learning.

[0020] Further, after constructing the validity determination model, it also includes setting a loss function and setting parameters for iteratively updating the validity determination model.

[0021] Further, the training method of the validity determination model includes: obtaining corpus data, processing the text in the corpus according to actual needs, converting the processed text into semantic vectors, and inputting the semantic vectors into the validity determination model for training to obtain a trained validity determination model.

[0022] The present invention also provides a multi-turn dialogue domain recognition device based on sentence completion, including:

[0023] A semantic encoding unit for converting historical dialogue corpus and each domain into semantic vectors;

[0024] A sentence completion unit for inputting the semantic vectors of the Nth domain and the semantic vectors of the historical corpus into a sentence completion model to obtain a completed sentence;

[0025] A validity determination unit for inputting the completed sentence of the Nth domain into a validity determination model. If it is determined to be valid, the Nth domain is the recognition result; otherwise, it jumps back to the sentence completion unit to continue considering the (N + 1)th domain.

[0026] The beneficial effects of the present invention are:

[0027] (1) The present invention combines the multi-turn dialogue domain recognition task and the sentence completion task, and can well recognize the situation where the user replies with an elliptical sentence in a multi-turn dialogue.

[0028] (2) The present invention not only utilizes the historical information of the dialogue, but also ensures that the historical information will not interfere with the domain switching, thereby improving the recognition accuracy.

[0029] (3) The present invention can obtain the completed sentence, which is convenient for subsequent tasks of multi-turn dialogue. Description of the Drawings

[0030] Figure 1 It is a schematic flowchart of a multi-turn dialogue domain recognition method based on sentence completion according to the present invention.

[0031] Figure 2 It is a block diagram of the composition of a multi-turn dialogue domain recognition device based on sentence completion according to the present invention. Detailed Embodiments

[0032] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings, but the embodiments of the present invention are not limited thereto.

[0033] Embodiment 1:

[0034] As Figure 1 shown, it is a schematic flowchart in Embodiment 1 of a multi-turn dialogue domain recognition method based on sentence completion according to the present invention, including:

[0035] Step 1: Convert the historical dialogue corpus and each domain into semantic vectors.

[0036] In a specific embodiment, the method for obtaining a semantic vector includes: semantic encoding methods based on deep learning, such as RNN models, LSTM models, BERT models, etc.

[0037] Step 2: Consider the Nth domain in sequence, input the semantic vector of this domain and the semantic vector of the historical corpus into the sentence completion model to obtain the completed sentence.

[0038] In a specific embodiment, the method for considering domains in sequence includes: for non-first-round sentences, preferentially consider the domain of the previous-round sentence.

[0039] In a specific embodiment, before inputting the semantic vector into the sentence completion model, it further includes a method for constructing the sentence completion model.

[0040] The method for constructing the sentence completion model includes: adopting a Seq2Seq model, which includes an Encoder, a Decoder, and an intermediate state vector C connecting the two. The Encoder encodes the input sequence into a fixed-size state vector C through learning, and then passes C to the Decoder. The Decoder then outputs the corresponding sequence through learning the state vector C.

[0041] The method for constructing the sentence completion model further includes: adopting a Pointer-generator network, adding an attention mechanism on the basis of the Seq2Seq model, using the hidden state of the previous moment in the Encoder stage to calculate attention to obtain a context vector, and then using it as the input of the unit at this moment.

[0042] In a specific embodiment, the sentence completion model uses a cross-entropy loss function to calculate the difference between the true value and the predicted value.

[0043] In a specific embodiment, the training method of the sentence completion model includes: obtaining corpus data, performing special processing on the text in the corpus according to actual needs, converting the processed text into a semantic vector, inputting the semantic vector into the sentence completion model for training to obtain a trained sentence completion model.

[0044] Step 3: Input the completed sentence into the validity determination model. If it is determined to be valid, the Nth domain is the recognition result; otherwise, continue to consider the (N + 1)th domain.

[0045] In a specific embodiment, before inputting the semantic vector into the validity determination model, it further includes a method for constructing the validity determination model.

[0046] The method for constructing the validity determination model includes: classification model methods based on deep learning, such as CNN models, fastText models, BERT models, etc.

[0047] In a specific embodiment, the validity determination model uses a cross-entropy loss function to calculate the difference between the true value and the predicted value.

[0048] In a specific embodiment, the training method of the validity determination model includes: obtaining corpus data, performing special processing on the text in the corpus according to actual needs, converting the processed text into semantic vectors, and inputting the semantic vectors into the validity determination model for training to obtain a trained validity determination model.

[0049] According to the method provided by the present invention, combining the multi-turn dialogue domain recognition task and the sentence completion task can not only well identify the situation where the user replies with an ellipsis sentence in the multi-turn dialogue, but also obtain the completed sentence, facilitating the subsequent tasks of the multi-turn dialogue.

[0050] Embodiment 2

[0051] As Figure 2 shown, it is an architecture diagram of a multi-turn dialogue domain recognition device based on sentence completion according to the present invention, including:

[0052] A semantic encoding unit 100, configured to convert the historical dialogue corpus and each domain into semantic vectors;

[0053] A sentence completion unit 200, configured to input the semantic vectors of the Nth domain and the semantic vectors of the historical corpus into the sentence completion model to obtain a completed sentence;

[0054] A validity determination unit 300, configured to input the completed sentence of the Nth domain into the validity determination model. If it is determined to be valid, the Nth domain is the recognition result; otherwise, it jumps back to the sentence completion unit to continue considering the (N + 1)th domain.

[0055] It should be noted that each unit in this embodiment is logical. In the specific implementation process, one unit can be split into multiple units, and multiple units can also be combined into one unit.

[0056] According to a multi-turn dialogue domain recognition device based on sentence completion provided by Embodiment 2 of the present invention, combining the multi-turn dialogue domain recognition task and the sentence completion task can not only well identify the situation where the user replies with an ellipsis sentence in the multi-turn dialogue, but also obtain the completed sentence, facilitating the subsequent tasks of the multi-turn dialogue. The structure of the multi-task combination not only utilizes the historical information of the dialogue but also ensures that the historical information will not interfere with the domain switching, thereby improving the recognition accuracy.

[0057] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for identifying multi-turn dialogue domains based on sentence completion, characterized in that, Including: Step 1: Convert the historical dialogue corpus and each domain into semantic vectors; Step 2: Consider the Nth domain in sequence, input the semantic vector of this domain and the semantic vector of the historical corpus into the sentence completion model to obtain the completed sentence; Step 3: Input the sentence completed for the Nth domain into the validity determination model. If it is determined to be valid, the Nth domain is the recognition result; otherwise, jump back to Step 2 and continue to consider the N+1th domain; The training method of the validity determination model includes: obtaining corpus data, processing the text in the corpus according to actual needs, converting the processed text into semantic vectors, and inputting the semantic vectors into the validity determination model for training to obtain a trained validity determination model.

2. The method for identifying multi-round dialogue fields based on sentence completion according to claim 1, wherein The method for obtaining semantic vectors includes: a semantic encoding method based on deep learning.

3. The method for identifying the multi-round dialogue field based on sentence completion according to claim 1, wherein In Step 2, the method for considering domains in sequence includes, for non-first-round sentences, taking the domain of the previous-round sentence as the primary consideration domain.

4. The method for identifying multi-turn dialogue fields based on sentence completion according to claim 1, characterized in that, The method for constructing the sentence completion model includes: the Seq2Seq model or the Pointer-Generator Networks model.

5. The method for identifying multi-turn dialogue fields based on sentence completion according to claim 1, characterized in that After constructing the sentence completion model, it further includes setting a loss function and setting parameters for iteratively updating the sentence completion model.

6. The method for identifying multi-turn dialogue fields based on sentence completion according to claim 1, characterized in that The training method of the sentence completion model includes: obtaining corpus data, processing the text in the corpus according to actual needs, converting the processed text into semantic vectors, and inputting the semantic vectors into the sentence completion model for training to obtain a trained sentence completion model.

7. The method for identifying multi-turn dialogue fields based on sentence completion according to claim 1, wherein The method for constructing the validity determination model includes: a classification model method based on deep learning.

8. The method for identifying multi-turn dialogue fields based on sentence completion according to claim 1, characterized in that, After constructing the validity determination model, it further includes setting a loss function and setting parameters for iteratively updating the validity determination model.

9. A multi-turn dialogue domain recognition device based on sentence completion, characterized in that, Including: A semantic encoding unit for converting the historical dialogue corpus and each domain into semantic vectors; A sentence completion unit for inputting the semantic vector of the Nth domain and the semantic vector of the historical corpus into the sentence completion model to obtain the completed sentence; A validity determination unit for inputting the sentence completed for the Nth domain into the validity determination model. If it is determined to be valid, the Nth domain is the recognition result; otherwise, jump back to the sentence completion unit and continue to consider the N+1th domain; The validity determination unit is further used for obtaining corpus data, processing the text in the corpus according to actual needs, converting the processed text into semantic vectors, and inputting the semantic vectors into the validity determination model for training to obtain a trained validity determination model.

Images