Dialogue processing methods, training methods and devices for question rewriting models

Abstract

This disclosure proposes a dialogue processing method, a training method for a question rewriting model, and a device thereof. The dialogue processing method includes: when information is missing in the user question, inputting the user question and historical dialogue content into the encoding layer of the question rewriting model to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user question, as well as the keyword scores of each word in the historical dialogue content; then, the decoding layer accurately selects keywords semantically related to the user question from the historical dialogue content based on the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores, and explicitly generates a fully rewritten user question based on these keywords. This accurately supplements the missing information in the user question, achieving completeness and readability of the user question content and improving the user experience.

Description

Technical Field

[0001] This disclosure relates to the field of natural language processing technology, and in particular to a dialogue processing method, a training method for a question rewriting model, and an apparatus thereof. Background Technology

[0002] Currently, in multi-turn dialogue systems, user questions often contain missing information (e.g., ellipsis and pronoun references). When key components of a question are missing, the system cannot respond accurately. For example, the user's question "What does this mean?" falls under the category of missing information; the system cannot determine the specific issue the user is asking based solely on the content of this question.

[0003] In related technologies, content is obtained from chat content and either directly appended to the end of the sentence or inserted into the user's question to make the question complete. However, the content obtained from the chat content may be unrelated to the user's question. Inserting unrelated content into the user's question can lead to significant deviations in the question and affect the user experience. Summary of the Invention

[0004] This disclosure aims to at least partially address one of the technical problems in the aforementioned technologies.

[0005] To address this, this disclosure proposes a dialogue processing method. When a user's question contains missing information, the method inputs the user's question and historical dialogue content into the encoding layer of a question rewriting model. This yields semantic representation vectors for each word in the historical dialogue content and the user's question, along with keyword scores for each word in the historical dialogue content. Then, the decoding layer of the question rewriting model accurately selects keywords semantically relevant to the user's question from the historical dialogue content based on these semantic representation vectors and keyword scores. The method then explicitly generates a fully rewritten user question based on these keywords, accurately supplementing the missing information in the user's question. This achieves completeness and readability of the user's question content, improving the user experience.

[0006] This disclosure provides a dialogue processing method in a dialogue system. The dialogue system includes a question rewriting model, comprising an encoding layer and a decoding layer. The encoding layer includes a semantic representation layer and a keyword analysis layer. The method includes: acquiring a user question to be responded to; if it is determined that the user question has missing information, acquiring the historical dialogue content of the user question, wherein the round difference between the round in which the user question is located and the round in which the historical dialogue content is located is less than a preset round threshold; and inputting the historical dialogue content and the user question into the dialogue system. A semantic representation layer is used to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user question; the semantic representation vectors of each word in the historical dialogue content are input into the keyword analysis layer to obtain the keyword scores of each word in the historical dialogue content; the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores are input into the decoding layer to obtain the rewritten user question with complete information; the response information to the rewritten user question is obtained and output.

[0007] This disclosure discloses a dialogue processing method applied in a dialogue system. The dialogue system includes a question rewriting model, comprising an encoding layer and a decoding layer. The encoding layer includes a semantic representation layer and a keyword analysis layer. The method involves: acquiring the user question to be answered; if information is missing in the user question, acquiring the historical dialogue content of the user question, wherein the round difference between the round in which the user question is located and the round in which the historical dialogue content is located is less than a preset round threshold; inputting the historical dialogue content and the user question into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user question; inputting the semantic representation vectors of each word in the historical dialogue content into the keyword analysis layer to obtain the keyword scores of each word in the historical dialogue content; inputting the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores into the decoding layer to obtain a rewritten user question with complete information; acquiring and outputting the response information to the rewritten user question. This method addresses the issue of missing information in user questions by inputting the user question and historical dialogue content into the encoding layer of a question rewriting model. This yields semantic representation vectors for each word in the historical dialogue and the user question, along with keyword scores for each word in the historical dialogue. Subsequently, the decoding layer of the question rewriting model accurately selects keywords semantically relevant to the user question from the historical dialogue based on these semantic representation vectors and keyword scores. It then explicitly generates a fully rewritten user question based on these keywords, accurately supplementing the missing information in the user question and improving its completeness and readability, thus enhancing the user experience.

[0008] Optionally, the step of inputting the semantic representation vectors of each word in the historical dialogue content into the keyword analysis layer to obtain the keyword score of each word in the historical dialogue content includes: determining the semantic relevance between each word based on the semantic representation vectors of each word in the plurality of words; and determining the keyword score of each word based on the semantic relevance between each word.

[0009] Optionally, determining the keyword score of each word based on the semantic relevance between the words includes: determining the importance score of each word based on the semantic relevance between the words; and determining the keyword score of each word based on the importance score of each word.

[0010] Optionally, the step of inputting the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores into the decoding layer to obtain a completely rewritten user question includes: decoding the semantic representation vectors of each word in the user question to obtain the user question; determining the initial word probability of each word in the historical dialogue content based on the semantic representation vectors of each word in the historical dialogue content; determining the target word probability of each word in the historical dialogue content based on the keyword scores and initial word probabilities of the semantic representation vectors of each word in the historical dialogue content; selecting the word with the highest target word probability as the target word based on the target word; and filling in the missing information in the user question based on the target word to obtain a completely rewritten user question.

[0011] Optionally, before inputting the historical dialogue content and the user question into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user question, the method further includes: segmenting the historical dialogue content to obtain each word in the historical dialogue content; matching each word in the historical dialogue content with noise words in a noise vocabulary; and deleting words in the historical dialogue content that match the noise words.

[0012] A second aspect of this disclosure proposes a training method for a question rewriting model. The question rewriting model includes an encoding layer and a decoding layer. The encoding layer includes a semantic representation layer and a keyword analysis layer. The method includes: acquiring training data, wherein the training data includes sample user questions, sample historical dialogue content of the sample user questions, and rewritten sample user questions, wherein the sample user questions have missing information, and the round difference between the round in which the sample user questions are located and the round in which the sample historical dialogue content is located is less than a preset round threshold; inputting the sample historical dialogue content and the sample user questions into the semantic representation layer. A representation layer is used to obtain the semantic representation vectors of each word in the sample historical dialogue content and the semantic representation vectors of each word in the sample user question. The semantic representation vectors of each word in the sample historical dialogue content are input into a keyword analysis layer to obtain the keyword scores of each word in the sample historical dialogue content. The semantic representation vectors of each word in the sample user question, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores are input into a decoding layer to obtain a rewritten sample user question with complete information. The question rewriting model is trained based on the rewritten sample user question and the rewritten user question.

[0013] The training method for the question rewriting model in this embodiment involves acquiring training data, including sample user questions, sample historical dialogue content of the sample user questions, and rewritten sample user questions. The sample user questions contain missing information, and the round difference between the round in which the sample user question is located and the round in which the sample historical dialogue content is located is less than a preset round threshold. The sample historical dialogue content and the sample user questions are input into a semantic representation layer to obtain semantic representation vectors for each word in the sample historical dialogue content and for each word in the sample user questions. The semantic representation vectors for each word in the sample historical dialogue content are input into a keyword analysis layer to obtain keyword scores for each word in the sample historical dialogue content. The semantic representation vectors for each word in the sample user questions, the semantic representation vectors for each word in the sample historical dialogue content, and the keyword scores are input into a decoding layer to obtain a rewritten sample user question with complete information. The question rewriting model is then trained based on the rewritten sample user questions and the rewritten user questions. This method inputs sample historical dialogue content and sample user questions into the encoding layer of a question rewriting model to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user questions, as well as the keyword scores of each word in the historical dialogue content. Then, the semantic representation vectors of each word in the sample user questions, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores are input into the decoding layer to obtain a second sample user question with complete information. The question rewriting model is trained based on the rewritten first and second sample user questions, enabling the trained model to output a rewritten user question with complete information, thus addressing the issue of information in user questions. In cases where information is missing, the user's question and historical dialogue content are input into the encoding layer of the question rewriting model. This yields semantic representation vectors for each word in the historical dialogue content and the user's question, as well as keyword scores for each word in the historical dialogue content. Subsequently, the decoding layer of the question rewriting model accurately selects keywords semantically relevant to the user's question from the historical dialogue content based on the semantic representation vectors of each word in the user's question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores. Based on these keywords, a fully rewritten user question is explicitly generated, accurately supplementing the missing information in the user's question. This achieves completeness and readability of the user's question content, improving the user experience.

[0014] Optionally, the semantic representation vectors of each word in the sample historical dialogue content are input into the keyword analysis layer to obtain the keyword score of each word in the sample historical dialogue content, including: determining the semantic relevance between each word based on the semantic representation vectors of each word in the multiple words; and determining the keyword score of each word based on the semantic relevance between each word.

[0015] Optionally, the keyword score of each word is determined based on the semantic relevance between the words, including: determining the importance score of each word based on the semantic relevance between the words; and determining the keyword score of each word based on the importance score of each word.

[0016] A third aspect of this disclosure provides a dialogue processing apparatus applied in a dialogue system. The dialogue system includes a question rewriting model, comprising an encoding layer and a decoding layer. The encoding layer includes a semantic representation layer and a keyword analysis layer. The apparatus includes: a first acquisition module for acquiring a user question to be answered; a second acquisition module for acquiring historical dialogue content of the user question when it is determined that the user question has missing information, wherein the round difference between the round in which the user question is located and the round in which the historical dialogue content is located is less than a preset round threshold; and a third acquisition module for inputting the historical dialogue content and the user question into a... The semantic representation layer obtains the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user question; the fourth acquisition module is used to input the semantic representation vectors of each word in the historical dialogue content into the keyword analysis layer to obtain the keyword score of each word in the historical dialogue content; the fifth acquisition module is used to input the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword score into the decoding layer to obtain the user question with complete information rewritten; the output module is used to acquire the response information of the rewritten user question and output the response information.

[0017] This disclosure provides a training device for a question rewriting model, comprising an encoding layer and a decoding layer. The encoding layer includes a semantic representation layer and a keyword analysis layer. The device comprises: a first acquisition module for acquiring training data, wherein the training data includes sample user questions, sample historical dialogue content of the sample user questions, and a rewritten first sample user question, wherein the sample user questions have missing information, and the round difference between the round in which the sample user question is located and the round in which the sample historical dialogue content is located is less than a preset round threshold; and a second acquisition module for inputting the sample historical dialogue content and the sample user questions into the semantic representation layer to obtain... The system comprises the following modules: a first module for obtaining the semantic representation vectors of each word in the sample historical dialogue content and the second module for obtaining the keyword scores of each word in the sample user question; a third module for obtaining the semantic representation vectors of each word in the sample historical dialogue content and the second module for obtaining the keyword scores of each word in the sample historical dialogue content; a fourth module for obtaining the semantic representation vectors of each word in the sample user question, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores of each word in the sample user question and the second module for obtaining the second sample user question with complete information rewritten; and a training module for training the question rewriting model based on the rewritten first sample user question and the rewritten second sample user question.

[0018] A fifth aspect of this disclosure provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect of the present invention, or to implement the method described in the second aspect of the present invention.

[0019] A sixth aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method described in the first aspect of the present invention, or implements the method described in the second aspect of the present invention.

[0020] A seventh aspect of this disclosure provides a computer program product that, when executed by an instruction processor, performs the method described in the first aspect embodiment or the method described in the second aspect embodiment.

[0021] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0022] Figure 1This is a flowchart illustrating the dialogue processing method provided in Embodiment 1 of this disclosure;

[0023] Figure 2 This is a schematic flowchart of the dialogue processing method provided in Embodiment 2 of this disclosure;

[0024] Figure 3 This is a flowchart illustrating the dialogue processing method provided in Embodiment 3 of this disclosure;

[0025] Figure 4 This is a flowchart illustrating the dialogue processing method provided in Embodiment 4 of this disclosure;

[0026] Figure 5 This is a flowchart illustrating the dialogue processing method according to an embodiment of the present disclosure;

[0027] Figure 6 This is a flowchart illustrating the training method for the question rewriting model provided in Embodiment 5 of this disclosure;

[0028] Figure 7 This is a flowchart illustrating the training method for the question rewriting model provided in Embodiment Six of this disclosure;

[0029] Figure 8 This is a schematic diagram of the question rewriting model according to an embodiment of the present disclosure;

[0030] Figure 9 This is a schematic diagram of the structure of a dialogue processing device provided in Embodiment 7 of this disclosure;

[0031] Figure 10 This is a schematic diagram of the structure of a training device for a question rewriting model provided in Embodiment 8 of this disclosure;

[0032] Figure 11 This is a block diagram of an electronic device for training a dialogue processing method or a question rewriting model, according to an exemplary embodiment. Detailed Implementation

[0033] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0034] Currently, in multi-turn dialogue systems, user questions often contain missing information (e.g., ellipsis and pronoun references). When key components of a question are missing, the system cannot provide an accurate answer. For example, the user's question "What does this mean?" falls under the category of missing information; the system cannot determine the specific issue the user is asking based solely on the content of this question.

[0035] In related technologies, the following two methods are mainly used to achieve the completeness of user questions. The first method is to extract the content from the historical chat through a word extraction algorithm and insert the extracted content into the user question by directly concatenating it to the end of the sentence. The second method is to use a deep generative model to encode the semantic information of the historical chat and the user question, and then decode a semantically complete sentence.

[0036] However, in the first method, the word extraction algorithm identifies a lot of noise, and some content unrelated to the user's question is inserted into the user's question; in addition, the form of word insertion cannot guarantee the fluency of the sentence, resulting in poor readability of the user's question; the second method uses the semantic representation capability of deep generative models to automatically generate semantically complete sentences and implicitly identifies key components in historical chat content, but without explicit information guidance, the generated sentence content has little variation.

[0037] To address the aforementioned issues, this disclosure proposes a dialogue processing method, a training method for a question rewriting model, and a device thereof.

[0038] Figure 1 This is a flowchart illustrating the dialogue processing method provided in Embodiment 1 of this disclosure. It should be noted that this dialogue processing method can be applied to a dialogue processing device. This device can be configured in an electronic device. The electronic device can be a mobile terminal, such as a mobile phone, tablet computer, personal digital assistant, or other hardware device with various operating systems. As an example, the dialogue processing method of this disclosure is applicable to dialogue systems.

[0039] like Figure 1 As shown, the dialogue processing method includes the following steps:

[0040] Step 101: Obtain the user's question to be answered.

[0041] In this embodiment of the disclosure, sentences in the form of questions raised by users can be obtained from multi-turn dialogues and used as user questions to be answered. For example, if a user asks "What does this mean?", this question can be used as a user question to be answered.

[0042] Step 102: If it is determined that there is missing information in the user's question, obtain the historical dialogue content of the user's question, wherein the difference between the round in which the user's question is located and the round in which the historical dialogue content is located is less than a preset round threshold.

[0043] Furthermore, natural language processing can be applied to the user question to be answered to determine if any information is missing. For example, word segmentation can be performed on the user question to be answered, obtaining each word corresponding to the question, and then part-of-speech tagging can be performed on each word to determine if any information is missing. Alternatively, semantic recognition can be performed on the user question to be answered, and the results can be used to determine if any information is missing.

[0044] Furthermore, if it is determined that there is missing information in the user's question, the historical dialogue content of the user's question can be obtained, wherein the difference between the round in which the user's question is located and the round in which the historical dialogue content is located is less than a preset round threshold.

[0045] It is important to understand that historical dialogue content can save multiple rounds of chat between the user and the system. The further the historical dialogue content is from the user's question, the lower their relevance and the more noise there is in the historical dialogue content; conversely, the closer the historical dialogue content is to the user's question, the higher their relevance and the less noise there is. To reduce noise in historical dialogue content and improve the relevance between historical dialogue content and user's question, this embodiment of the disclosure imposes a dialogue round limit on the historical dialogue content. The round difference between the round in which the user's question is located and the round in which the historical dialogue content is located is less than a preset round threshold, such as a round difference of less than 4.

[0046] Step 103: Input the historical dialogue content and user questions into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user questions.

[0047] In this embodiment, the question rewriting model may include an encoding layer and a decoding layer. The encoding layer may include a semantic representation layer and a keyword analysis layer. Optionally, historical dialogue content and user questions are input into the semantic representation layer. The semantic representation layer (which may be a Transformer) has an interactive function. The semantic representation layer can interactively calculate the semantic information by comparing each word in the historical dialogue content with each word in the user question, thereby learning more explicit semantic information. This results in the output of a sequence of semantic representation vectors for each word in the historical dialogue content and a sequence of semantic representation vectors for each word in the user question, specifically expressed as the following formula:

[0048] ;

[0049] in, It can represent the semantic representation vector sequence (or semantic vector matrix) of each word in the historical dialogue content. It can represent the semantic representation vector sequence (or semantic vector matrix) of each word in a user's question.

[0050] Step 104: Input the semantic representation vectors of each word in the historical dialogue content into the keyword analysis layer to obtain the keyword scores of each word in the historical dialogue content.

[0051] Furthermore, the semantic representation vectors of each word in the historical dialogue content can be input into the keyword analysis layer, which can then output the keyword scores of each word in the historical dialogue content.

[0052] Step 105: Input the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores into the decoding layer to obtain the user question with complete information rewritten.

[0053] Then, the semantic representation vectors of each word in the user's question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores are simultaneously input into the decoding layer. The decoding layer decodes the user's question based on the semantic representation vectors of each word in the user's question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores, to obtain the rewritten user question with complete information.

[0054] Step 106: Obtain and output the response information for the rewritten user question.

[0055] Furthermore, the user's question can be determined based on the content of the rewritten user question, and a response can be provided and output.

[0056] In summary, the process involves: acquiring the user's question to be answered; if information is missing from the user's question, acquiring the historical dialogue content of the user's question, where the difference between the round number of the user's question and the round number of the historical dialogue content is less than a preset round number threshold; inputting the historical dialogue content and the user's question into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user's question; inputting the semantic representation vectors of each word in the historical dialogue content into the keyword analysis layer to obtain the keyword scores of each word in the historical dialogue content; inputting the semantic representation vectors of each word in the user's question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores into the decoding layer to obtain the rewritten user's question with complete information; acquiring and outputting the response information for the rewritten user's question. Therefore, when user questions contain missing information, the user question and historical dialogue content are input into the encoding layer of the question rewriting model. This yields the semantic representation vectors of each word in the historical dialogue content and the user question, as well as the keyword scores of each word in the historical dialogue content. Subsequently, the decoding layer of the question rewriting model accurately selects keywords semantically related to the user question from the historical dialogue content based on the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores. Based on these keywords, a fully rewritten user question is explicitly generated, accurately supplementing the missing information in the user question, achieving completeness and readability of the user question content, and improving the user experience.

[0057] To accurately obtain keyword scores for each word in the historical dialogue content, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating the dialogue processing method provided in Embodiment 2 of this disclosure. In this embodiment, the semantic relevance between words can be determined based on the semantic representation vectors of each word in the historical dialogue content. Furthermore, the keyword score of each word can be determined based on the semantic relevance between them. Figure 2 The illustrated embodiment may include the following steps:

[0058] Step 201: Obtain the user's question to be answered.

[0059] Step 202: If it is determined that there is missing information in the user's question, obtain the historical dialogue content of the user's question, wherein the difference between the round in which the user's question is located and the round in which the historical dialogue content is located is less than a preset round threshold.

[0060] Step 203: Input the historical dialogue content and user questions into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user questions.

[0061] Step 204: Determine the semantic relevance between words based on the semantic representation vectors of each word in the multiple words.

[0062] In this embodiment, the semantic representation vectors of each word in the historical dialogue content are input into the keyword analysis layer. The keyword analysis layer can use self-attention weights to calculate the semantic relevance between each word, and then construct a weighted fully connected graph based on the semantic relevance between each word. The nodes in this fully connected graph represent each word in the historical dialogue content, and the edges in the fully connected graph represent the semantic relevance between each word. Specifically, this can be expressed as the following formula:

[0063] ;

[0064] in, This represents the semantic representation vector of each word in the historical dialogue content.

[0065] Step 205: Determine the keyword score for each word based on the semantic relevance between them.

[0066] Optionally, the importance score of each word can be determined based on the semantic relevance between them; and the keyword score of each word can be determined based on its importance score.

[0067] In other words, the importance score of each word can be calculated based on the semantic relevance between them, and the importance score of each word can be used as the keyword score of each word.

[0068] In this embodiment of the disclosure, keywords can be identified from a weighted fully connected graph constructed based on the semantic relevance between words in historical dialogue content. For example, relevant keyword algorithms (such as the TextRank algorithm) can be used to iteratively calculate the fully connected graph multiple times, continuously updating the importance scores of nodes. In the initial state, the node scores are all... This indicates that all words in the historical dialogue have equal importance. At the end of the iteration, the node importance score represents the final importance of each word in the historical dialogue; a higher score indicates greater word importance. This can be represented by the following formula:

[0069] ;

[0070] ;

[0071] in, It is a list of keyword scores. This represents the number of words in the historical dialogue. The length of the list is the same as the number of words in the context, and each value represents the importance score of a word.

[0072] Step 206: Input the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores into the decoding layer to obtain the user question with complete information rewritten.

[0073] Step 207: Obtain and output the response information for the rewritten user question.

[0074] In this embodiment of the disclosure, steps 201-203 and 206 can be implemented in any of the ways described in the various embodiments of the disclosure. This embodiment of the disclosure does not limit these methods and will not elaborate further.

[0075] In summary, by analyzing the semantic representation vectors of each word within a multi-word context, the semantic relevance between words is determined; based on this semantic relevance, the keyword score for each word is then calculated. This allows for the accurate acquisition of keyword scores for each word within the historical dialogue content.

[0076] To accurately obtain the complete rewritten user question, such as Figure 3 As shown, Figure 3 This is a flowchart illustrating the dialogue processing method provided in Embodiment 3 of this disclosure. In this embodiment, target words can be obtained from historical dialogue content, and missing information in the user's question can be filled in based on the target words to obtain a rewritten user question with complete information. Figure 3 The illustrated embodiment may include the following steps:

[0077] Step 301: Obtain the user's question to be answered.

[0078] Step 302: If it is determined that there is missing information in the user's question, obtain the historical dialogue content of the user's question, wherein the difference between the round in which the user's question is located and the round in which the historical dialogue content is located is less than a preset round threshold.

[0079] Step 303: Input the historical dialogue content and user questions into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user questions.

[0080] Step 304: Input the semantic representation vectors of each word in the historical dialogue content into the keyword analysis layer to obtain the keyword scores of each word in the historical dialogue content.

[0081] Step 305: Decode the semantic representation vectors of each word in the user's question to obtain the user's question.

[0082] In this embodiment of the disclosure, the decoding layer decodes the semantic representation vectors of each word in the user question to obtain the user question.

[0083] Step 306: Determine the initial lexical probability of each word in the historical dialogue content based on the semantic representation vector of each word in the historical dialogue content.

[0084] It should be noted that when user question information is missing, the words corresponding to the missing information are derived from historical dialogue content, while the remaining words are derived from the user question; when user question information is complete, the words are derived solely from the user question. In this embodiment, a gating switch can be set via a copy mechanism. The source of words is controlled. During each decoding step, the decoding layer calculates the initial lexical probability of each word in the historical dialogue content based on the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user's question. Initial lexical probabilities of each word in the user's question Then through The final word probability is obtained by weighting the word probabilities in historical chats and user questions. The decoding layer then selects the word with the highest probability for generation. This can be represented by the following formula:

[0085] ;

[0086] Where i represents the i-th decoding step. This indicates a door control switch, provided that no information is missing in the user's question. When user questions contain missing information, .

[0087] In this embodiment of the disclosure, when it is determined that there is missing information in the user's question, the initial lexical probability of each word in the historical dialogue content can be expressed as: .

[0088] Step 307: Determine the target word probability of each word in the historical dialogue content based on the keyword score of the semantic representation vector of each word in the historical dialogue content and the initial word probability.

[0089] To ensure that the rewritten user question contains more key information, in this embodiment of the disclosure, the keyword score can be added to the initial word probability of each word in the historical dialogue content. The target word probability of each word in the historical dialogue content is then determined based on the addition result. Thus, the target word probability of each word in the historical dialogue content... This can highlight keywords in historical dialogue content, increasing the probability of those keywords being generated. Specifically, it can be represented by the following formula:

[0090] ;

[0091] in, This represents the initial lexical probability of each word in the historical dialogue content. Keyword scores represent the semantic representation vectors of each word in the historical dialogue content.

[0092] Step 308: Based on the target word probability of each word in the historical dialogue content, select the word with the highest target word probability as the target word.

[0093] As one possible implementation of this disclosure, the word with the highest probability of target vocabulary in each word in the historical dialogue content can be used as the target word.

[0094] Step 309: Fill in the missing information in the user question based on the target words to obtain a rewritten user question with complete information.

[0095] Furthermore, the target words are added to the user's question to fill in the missing information, resulting in a rewritten user question with complete information.

[0096] Step 310: Obtain and output the response information for the rewritten user question.

[0097] In this embodiment of the disclosure, steps 301-304 and step 310 can be implemented in any of the ways described in the various embodiments of the disclosure. This embodiment of the disclosure does not limit these methods and will not elaborate further.

[0098] In summary, the user question is obtained by decoding the semantic representation vectors of each word in the user question; the initial lexical probability of each word in the historical dialogue content is determined based on the semantic representation vectors of each word in the historical dialogue content; the target lexical probability of each word in the historical dialogue content is determined based on the keyword score and initial lexical probability of the semantic representation vectors of each word in the historical dialogue content; the word with the highest target lexical probability is selected as the target word; and the missing information in the user question is filled in based on the target word to obtain a complete rewritten user question. Therefore, keywords semantically related to the user question can be accurately selected from the historical dialogue content, and a complete rewritten user question can be explicitly generated based on these keywords, accurately supplementing the missing information in the user question, achieving completeness and readability of the user question content, and improving the user experience.

[0099] To improve the accuracy of extracting target vocabulary from historical dialogue content, such as Figure 4As shown, Figure 4 This is a flowchart illustrating the dialogue processing method provided in Embodiment 4 of this disclosure. In this embodiment, since historical dialogue content may contain noise words, such as words inviting comments, urging payment, and small talk, these noise words do not provide any effective information for the question rewriting model and instead interfere with the model's identification of important information. Therefore, before inputting the historical dialogue content and user questions into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user questions, noise words in the historical dialogue content can be deleted. Figure 4 The illustrated embodiment may include the following steps:

[0100] Step 401: Obtain the user's question to be answered.

[0101] Step 402: If it is determined that there is missing information in the user's question, obtain the historical dialogue content of the user's question, wherein the difference between the round in which the user's question is located and the round in which the historical dialogue content is located is less than a preset round threshold.

[0102] Step 403: Segment the historical dialogue content to obtain the individual words in the historical dialogue content.

[0103] In this embodiment of the disclosure, the historical dialogue content can be segmented using a relevant word segmentation algorithm to obtain individual words from the historical dialogue content. It should be noted that the relevant word segmentation algorithm may include: dictionary-based word segmentation algorithms, machine learning-based word segmentation algorithms, or neural network-based word segmentation algorithms, etc.

[0104] Step 404: Match each word in the historical dialogue content with the noise words in the noise word list.

[0105] Furthermore, each word in the historical dialogue can be matched with noise words in a preset noise word list.

[0106] Step 405: Delete words in the historical dialogue that match the noise words.

[0107] Furthermore, words matching the noise words in the historical dialogue can be deleted; for example, the phrase "welcome" can be deleted.

[0108] Step 406: Input the historical dialogue content and user questions into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user questions.

[0109] Step 407: Input the semantic representation vectors of each word in the historical dialogue content into the keyword analysis layer to obtain the keyword scores of each word in the historical dialogue content.

[0110] Step 408: Input the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores into the decoding layer to obtain the user question with complete information rewritten.

[0111] Step 409: Obtain and output the response information for the rewritten user question.

[0112] In this embodiment of the disclosure, steps 401-402 and 409 can be implemented in any of the ways described in the various embodiments of the disclosure. This embodiment of the disclosure does not limit these methods and will not elaborate further.

[0113] In summary, by matching each word in the historical dialogue content with noise words in the noise word list; segmenting the historical dialogue content to obtain each word in the historical dialogue content; and deleting words in the historical dialogue content that match noise words, the accuracy of obtaining target vocabulary from historical dialogue content can be improved.

[0114] To illustrate the above embodiments more clearly, examples are given below.

[0115] like Figure 5 As shown, the dialogue processing method mainly includes the following four aspects:

[0116] 1. Noise Reduction of Historical Chat (Historical Dialogue Content): In order to improve the accuracy of target words (keywords) extracted from historical chat content, noise filtering can be performed on historical chat content to delete sentences that are irrelevant to user questions or have low relevance.

[0117] 2. Encoding: Low-noise historical chat content and user questions are concatenated and input into the encoder for simultaneous encoding to obtain semantic representation vectors of words;

[0118] 3. Keyword Recognition: The self-attention keyword detection module identifies keywords and calculates keyword scores (keyword importance scores) based on the semantic vectors of words in historical chats.

[0119] 4. Decoding: Keyword scores are used to calculate the probability of word generation, guiding the decoder to generate semantically complete sentences.

[0120] This disclosure presents a dialogue processing method applied in a dialogue system. The dialogue system includes a question rewriting model, comprising an encoding layer and a decoding layer. The encoding layer includes a semantic representation layer and a keyword analysis layer. The method involves: acquiring a user question to be answered; if information is missing in the user question, acquiring the historical dialogue content of the user question, wherein the round difference between the round in which the user question is located and the round in which the historical dialogue content is located is less than a preset round threshold; inputting the historical dialogue content and the user question into the semantic representation layer to obtain semantic representation vectors of each word in the historical dialogue content and the user question; inputting the semantic representation vectors of each word in the historical dialogue content into the keyword analysis layer to obtain keyword scores for each word in the historical dialogue content; inputting the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores into the decoding layer to obtain a rewritten user question with complete information; acquiring and outputting response information related to the rewritten user question. This method addresses the issue of missing information in user questions by inputting the user question and historical dialogue content into the encoding layer of a question rewriting model. This yields semantic representation vectors for each word in the historical dialogue and the user question, along with keyword scores for each word in the historical dialogue. Subsequently, the decoding layer of the question rewriting model accurately selects keywords semantically relevant to the user question from the historical dialogue based on these semantic representation vectors and keyword scores. It then explicitly generates a fully rewritten user question based on these keywords, accurately supplementing the missing information in the user question and improving its completeness and readability, thus enhancing the user experience.

[0121] To achieve the above embodiments, this disclosure also proposes a training method for a question rewriting model.

[0122] Figure 6 This is a flowchart illustrating the training method for the question rewriting model provided in Embodiment 5 of this disclosure. It should be noted that this training method for the question rewriting model can be applied to a training device for the question rewriting model. This device can be configured in an electronic device. The electronic device can be a mobile terminal, such as a mobile phone, tablet computer, personal digital assistant, or other hardware device with various operating systems.

[0123] like Figure 6 As shown, the training method for this question rewriting model may include the following steps:

[0124] Step 601: Obtain training data, which includes sample user questions, sample historical dialogue content of sample user questions, and the first sample user question after rewriting the sample user questions. The sample user questions have missing information, and the round difference between the round in which the sample user question is located and the round in which the sample historical dialogue content is located is less than a preset round threshold.

[0125] In this embodiment, user questions with missing information in multi-turn dialogues can be used as sample user questions. The rewritten sample user questions with complete information are then used as the first rewritten sample user question. Next, historical dialogue content of the sample user questions is obtained from the multi-turn dialogues. To obtain historical dialogue content with high relevance to the sample user questions, this embodiment imposes a dialogue turn limit on the historical dialogue content. The turn difference between the turn containing the sample user question and the turn containing the historical dialogue content is less than a preset turn threshold; for example, the turn difference between the turn containing the sample user question and the turn containing the historical dialogue content is less than 4.

[0126] Step 602: Input the sample historical dialogue content and sample user questions into the semantic representation layer to obtain the semantic representation vectors of each word in the sample historical dialogue content and the semantic representation vectors of each word in the sample user questions.

[0127] In this embodiment, the question rewriting model may include an encoding layer and a decoding layer. The encoding layer includes a semantic representation layer and a keyword analysis layer. Sample historical dialogue content and sample user questions are input into the semantic representation layer. The semantic representation layer has an interactive function, which can interactively calculate the semantic information by comparing each word in the sample historical dialogue content with each word in the user question. This allows it to learn more explicit semantic information and output the semantic representation vector sequences of each word in the sample historical dialogue content and the semantic representation vector sequences of each word in the sample user question.

[0128] Step 603: Input the semantic representation vectors of each word in the sample historical dialogue content into the keyword analysis layer to obtain the keyword scores of each word in the sample historical dialogue content.

[0129] Furthermore, the semantic representation vectors of each word in the sample historical dialogue content can be input into the keyword analysis layer. The keyword analysis layer can output the keyword score of each word in the sample historical dialogue content based on the semantic relevance of each word.

[0130] Step 604: Input the semantic representation vectors of each word in the sample user question, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores into the decoding layer to obtain the second sample user question with complete information rewritten.

[0131] Then, the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores are simultaneously input into the decoding layer. The decoding layer decodes the second sample user question based on the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores, to obtain the second sample user question with complete information rewritten.

[0132] Optionally, the semantic representation vectors of each word in the sample user's question are decoded to obtain the sample user's question; the initial word probability of each word in the sample historical dialogue content is determined based on the semantic representation vectors of each word in the sample historical dialogue content; the target word probability of each word in the sample historical dialogue content is determined based on the keyword score and initial word probability of the semantic representation vectors of each word in the sample historical dialogue content; the word with the highest target word probability is selected as the target word based on the target word; and the missing information in the sample user's question is filled in based on the target word to obtain a second sample user's question with complete information after rewriting.

[0133] Step 605: Train the question rewriting model based on the rewritten first sample user question and the rewritten second sample user question.

[0134] Furthermore, based on the differences between the rewritten first sample user question and the rewritten second sample user question, and combined with a preset loss function, the value of the loss function is determined. The parameters of the question rewriting model are then adjusted according to this loss function value to train the question rewriting model. The preset loss function is:

[0135] ;

[0136] Where N is the length of the generated, rewritten second sample user question, and KL is the relative entropy distance. The target word probability is denoted as ______ for each word in the sample's historical dialogue content. This represents the initial lexical probability of each word in the sample's historical dialogue content. The keyword score represents the semantic representation vector of each word in the historical dialogue content of the sample.

[0137] In summary, by acquiring training data, including sample user questions, sample historical dialogue content of the sample user questions, and rewritten sample user questions, where the sample user questions contain missing information and the round difference between the round in which the sample user question is located and the round in which the sample historical dialogue content is located is less than a preset round threshold; the sample historical dialogue content and the sample user questions are input into a semantic representation layer to obtain the semantic representation vectors of each word in the sample historical dialogue content and the semantic representation vectors of each word in the sample user questions; the semantic representation vectors of each word in the sample historical dialogue content are input into a keyword analysis layer to obtain the keyword scores of each word in the sample historical dialogue content; the semantic representation vectors of each word in the sample user questions, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores are input into a decoding layer to obtain the rewritten sample user questions with complete information; and the question rewriting model is trained based on the rewritten sample user questions and the rewritten user questions. Therefore, the sample historical dialogue content and sample user questions are input into the encoding layer of the question rewriting model to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user questions, as well as the keyword scores of each word in the historical dialogue content. Then, the semantic representation vectors of each word in the sample user questions, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores are input into the decoding layer to obtain a second sample user question with complete information. The question rewriting model is trained based on the rewritten first and second sample user questions, enabling the trained model to output a rewritten user question with complete information, thus addressing the issue of information in user questions. In cases where information is missing, the user's question and historical dialogue content are input into the encoding layer of the question rewriting model. This yields semantic representation vectors for each word in the historical dialogue content and the user's question, as well as keyword scores for each word in the historical dialogue content. Subsequently, the decoding layer of the question rewriting model accurately selects keywords semantically relevant to the user's question from the historical dialogue content based on the semantic representation vectors of each word in the user's question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores. Based on these keywords, a fully rewritten user question is explicitly generated, accurately supplementing the missing information in the user's question. This achieves completeness and readability of the user's question content, improving the user experience.

[0138] To accurately obtain keyword scores for each word in the sample's historical dialogue content, such as Figure 7 As shown, Figure 7This is a flowchart illustrating the training method for the question rewriting model provided in Embodiment Six of this disclosure. In this embodiment, the semantic relevance between words can be determined based on the semantic representation vectors of each word in the sample historical dialogue content. Furthermore, the keyword score of each word can be determined based on the semantic relevance between them. Figure 7 The illustrated embodiment may include the following steps:

[0139] Step 701: Obtain training data, which includes sample user questions, sample historical dialogue content of sample user questions, and the first sample user question after rewriting the sample user questions. The sample user questions have missing information, and the round difference between the round in which the sample user question is located and the round in which the sample historical dialogue content is located is less than a preset round threshold.

[0140] Step 702: Input the sample historical dialogue content and sample user questions into the semantic representation layer to obtain the semantic representation vectors of each word in the sample historical dialogue content and the semantic representation vectors of each word in the sample user questions.

[0141] Step 703: Determine the semantic relevance between words based on the semantic representation vectors of each word in the multiple words.

[0142] In this embodiment, the semantic representation vectors of each word in the sample historical dialogue content are input into the keyword analysis layer. The keyword analysis layer can use self-attention weights to calculate the semantic relevance between each word, and then construct a weighted fully connected graph based on the semantic relevance between each word. The nodes in this fully connected graph represent each word in the historical dialogue content, and the edges in the fully connected graph represent the semantic relevance between each word. Specifically, this can be expressed as the following formula:

[0143] ;

[0144] in, This represents the semantic representation vector of each word in the historical dialogue content.

[0145] Step 704: Determine the keyword score for each word based on the semantic relevance between them.

[0146] Optionally, the importance score of each word can be determined based on the semantic relevance between them; and the keyword score of each word can be determined based on its importance score.

[0147] In other words, the importance score of each word can be calculated based on the semantic relevance between them, and the importance score of each word can be used as the keyword score of each word.

[0148] In this embodiment of the disclosure, keywords can be identified from a weighted fully connected graph constructed based on the semantic relevance between words in historical dialogue content. For example, relevant keyword algorithms (such as the TextRank algorithm) can be used to iteratively calculate the fully connected graph multiple times, continuously updating the importance scores of nodes. In the initial state, the node scores are all... This indicates that all words in the historical dialogue have equal importance. At the end of the iteration, the node importance score represents the final importance of each word in the historical dialogue; a higher score indicates greater word importance. This can be represented by the following formula:

[0149] ;

[0150] ;

[0151] in, It is a list of scores. This represents the number of words in the historical dialogue. The length of the list is the same as the number of words in the context, and each value represents the importance score of a word.

[0152] Step 705: Input the semantic representation vectors of each word in the sample user question, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores into the decoding layer to obtain the second sample user question after complete rewriting of the information.

[0153] Step 706: Train the question rewriting model based on the rewritten first sample user question and the rewritten second sample user question.

[0154] In this embodiment of the disclosure, steps 701-702 and 705-706 can be implemented in any of the embodiments of this disclosure. This embodiment of the disclosure does not limit these steps and will not elaborate further.

[0155] In summary, by analyzing the semantic representation vectors of each word within a multi-word context, the semantic relevance between words is determined; and based on this semantic relevance, the keyword score for each word is determined. This allows for the accurate selection of keywords semantically relevant to the user's question from historical dialogue content.

[0156] To illustrate the above embodiments more clearly, examples are given below.

[0157] like Figure 8 As shown, Figure 8The detailed structure of the question rewriting model is as follows: it consists of an encoder and a decoder. A self-attention keyword detection module is added to the encoder to identify keywords in the historical dialogue content. Then, the identified keyword information is combined with the decoder's copy mechanism to set the gating switch. Control the source of words. Guide the model to decode step by step.

[0158] The training method for the question rewriting model in this embodiment involves acquiring training data, including sample user questions, sample historical dialogue content of the sample user questions, and rewritten sample user questions. The sample user questions contain missing information, and the round difference between the round in which the sample user question is located and the round in which the sample historical dialogue content is located is less than a preset round threshold. The sample historical dialogue content and the sample user questions are input into a semantic representation layer to obtain semantic representation vectors for each word in the sample historical dialogue content and for each word in the sample user questions. The semantic representation vectors for each word in the sample historical dialogue content are input into a keyword analysis layer to obtain keyword scores for each word in the sample historical dialogue content. The semantic representation vectors for each word in the sample user questions, the semantic representation vectors for each word in the sample historical dialogue content, and the keyword scores are input into a decoding layer to obtain a rewritten sample user question with complete information. The question rewriting model is then trained based on the rewritten sample user questions and the rewritten user questions. This method inputs sample historical dialogue content and sample user questions into the encoding layer of a question rewriting model to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user questions, as well as the keyword scores of each word in the historical dialogue content. Then, the semantic representation vectors of each word in the sample user questions, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores are input into the decoding layer to obtain a second sample user question with complete information. The question rewriting model is trained based on the rewritten first and second sample user questions, enabling the trained model to output a rewritten user question with complete information, thus addressing the issue of information in user questions. In cases where information is missing, the user's question and historical dialogue content are input into the encoding layer of the question rewriting model. This yields semantic representation vectors for each word in the historical dialogue content and the user's question, as well as keyword scores for each word in the historical dialogue content. Subsequently, the decoding layer of the question rewriting model accurately selects keywords semantically relevant to the user's question from the historical dialogue content based on the semantic representation vectors of each word in the user's question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores. Based on these keywords, a fully rewritten user question is explicitly generated, accurately supplementing the missing information in the user's question. This achieves completeness and readability of the user's question content, improving the user experience.

[0159] In order to achieve Figures 1 to 5 In the embodiments described herein, this disclosure also proposes a dialogue processing apparatus. Figure 9 This is a schematic diagram of the structure of a dialogue processing device provided in Embodiment 7 of this disclosure.

[0160] like Figure 9 As shown, the dialogue processing device 900 includes: a first acquisition module 910, a second acquisition module 920, a third acquisition module 930, a fourth acquisition module 940, a fifth acquisition module 950, and an output module 960.

[0161] The system comprises the following modules: a first acquisition module 910, used to acquire the user question to be answered; a second acquisition module 920, used to acquire the historical dialogue content of the user question when it is determined that the user question has missing information, wherein the difference between the round number of the user question and the round number of the historical dialogue content is less than a preset round number threshold; a third acquisition module 930, used to input the historical dialogue content and the user question into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user question; a fourth acquisition module 940, used to input the semantic representation vectors of each word in the historical dialogue content into the keyword analysis layer to obtain the keyword scores of each word in the historical dialogue content; a fifth acquisition module 950, used to input the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores into the decoding layer to obtain the rewritten user question with complete information; and an output module 960, used to acquire and output the reply information of the rewritten user question.

[0162] As one possible implementation of this disclosure, the fourth acquisition module is specifically used to: determine the semantic relevance between words based on the semantic representation vectors of each word in the plurality of words; and determine the keyword score of each word based on the semantic relevance between words.

[0163] As one possible implementation of this disclosure, the fourth acquisition module is further configured to: determine the importance score of each word based on the semantic relevance between each word; and determine the keyword score of each word based on the importance score of each word.

[0164] As one possible implementation of this disclosure, the fifth acquisition module is specifically used for: decoding the semantic representation vectors of each word in the user question to obtain the user question; determining the initial word probability of each word in the historical dialogue content based on the semantic representation vectors of each word in the historical dialogue content; determining the target word probability of each word in the historical dialogue content based on the keyword score and the initial word probability of the semantic representation vectors of each word in the historical dialogue content; selecting the word with the highest target word probability as the target word based on the target word; and filling in the missing information in the user question based on the target word to obtain a rewritten user question with complete information.

[0165] As one possible implementation of this disclosure, the dialogue processing device 900 further includes: a word segmentation module, a matching module, and a deletion module.

[0166] The system includes a word segmentation module for segmenting historical dialogue content into individual words; a matching module for matching individual words in historical dialogue content with noise words in a noise word list; and a deletion module for deleting words in historical dialogue content that match noise words.

[0167] This embodiment of the dialog processing apparatus is applied in a dialog system. The dialog system has a question rewriting model, which includes an encoding layer and a decoding layer. The encoding layer includes a semantic representation layer and a keyword analysis layer. The apparatus obtains the user question to be replied to; if it is determined that the user question has missing information, it obtains the historical dialogue content of the user question, wherein the difference between the round number of the user question and the round number of the historical dialogue content is less than a preset round number threshold; it inputs the historical dialogue content and the user question into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user question; it inputs the semantic representation vectors of each word in the historical dialogue content into the keyword analysis layer to obtain the keyword scores of each word in the historical dialogue content; it inputs the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores into the decoding layer to obtain a rewritten user question with complete information; it obtains and outputs the reply information to the rewritten user question. This device enables the input of user questions and historical dialogue content into the encoding layer of a question rewriting model when information is missing from the user question. This yields semantic representation vectors for each word in the historical dialogue content and the user question, along with keyword scores for each word in the historical dialogue content. Then, the decoding layer of the question rewriting model accurately selects keywords semantically relevant to the user question from the historical dialogue content based on these semantic representation vectors and keyword scores. Using these keywords, it explicitly generates a fully rewritten user question, accurately supplementing the missing information in the user question, thus improving the completeness and readability of the user question content and enhancing the user experience.

[0168] In order to achieve Figures 6 to 8 In the embodiments described herein, this disclosure also proposes a training device for a question rewriting model. Figure 10 This is a schematic diagram of the structure of a training device for a question rewriting model provided in Embodiment 8 of this disclosure.

[0169] like Figure 10As shown, the training device 1000 for the question rewriting model includes: a first acquisition module 1010, a second acquisition module 1020, a third acquisition module 1030, a fourth acquisition module 1040, and a training module 1050.

[0170] The first acquisition module 1010 is used to acquire training data, which includes sample user questions, sample historical dialogue content of the sample user questions, and the rewritten first sample user questions. The sample user questions contain missing information, and the round difference between the round in which the sample user question is located and the round in which the sample historical dialogue content is located is less than a preset round threshold. The second acquisition module 1020 is used to input the sample historical dialogue content and the sample user questions into the semantic representation layer to obtain the semantic representation vectors of each word in the sample historical dialogue content and the semantic tables of each word in the sample user questions. The first sample user question is given by a first sample user question and the second sample user question. The second sample user question is given by a third acquisition module 1030, which is used to input the semantic representation vectors of each word in the sample historical dialogue content into the keyword analysis layer to obtain the keyword scores of each word in the sample historical dialogue content. The third acquisition module 1040 is used to input the semantic representation vectors of each word in the sample user question, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores into the decoding layer to obtain the second sample user question after complete information rewriting. The fourth acquisition module 1040 is used to input the semantic representation vectors of each word in the sample user question, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores into the decoding layer to obtain the second sample user question after complete information rewriting. The fifth acquisition module 1050 is used to train the question rewriting model based on the rewritten first sample user question and the rewritten second sample user question.

[0171] As one possible implementation of this disclosure, the third acquisition module 1030 is specifically used to: determine the semantic relevance between words based on the semantic representation vectors of each word in a plurality of words; and determine the keyword score of each word based on the semantic relevance between words.

[0172] As one possible implementation of this disclosure, the third acquisition module 1030 is further configured to: determine the importance score of each word based on the semantic relevance between each word; and determine the keyword score of each word based on the importance score of each word.

[0173] The training apparatus for the question rewriting model in this embodiment acquires training data, including sample user questions, sample historical dialogue content of the sample user questions, and rewritten sample user questions. The sample user questions contain missing information, and the round difference between the round in which the sample user question is located and the round in which the sample historical dialogue content is located is less than a preset round threshold. The sample historical dialogue content and the sample user questions are input into a semantic representation layer to obtain semantic representation vectors for each word in the sample historical dialogue content and for each word in the sample user questions. The semantic representation vectors for each word in the sample historical dialogue content are input into a keyword analysis layer to obtain keyword scores for each word in the sample historical dialogue content. The semantic representation vectors for each word in the sample user questions, the semantic representation vectors for each word in the sample historical dialogue content, and the keyword scores are input into a decoding layer to obtain a rewritten sample user question with complete information. The question rewriting model is trained based on the rewritten sample user questions and the rewritten user questions. This device can input sample historical dialogue content and sample user questions into the encoding layer of a question rewriting model to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user questions, as well as the keyword scores of each word in the historical dialogue content. Then, the semantic representation vectors of each word in the sample user questions, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores are input into the decoding layer to obtain a second sample user question with complete information. The question rewriting model is trained based on the rewritten first and second sample user questions, enabling the trained model to output a rewritten user question with complete information, thus addressing the issue of information leakage in user questions. In cases of missing information, the user's question and historical dialogue content are input into the encoding layer of the question rewriting model. This yields semantic representation vectors for each word in the historical dialogue content and the user's question, as well as keyword scores for each word in the historical dialogue content. Subsequently, the decoding layer of the question rewriting model accurately selects keywords semantically relevant to the user's question from the historical dialogue content based on the semantic representation vectors of each word in the user's question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores. Based on these keywords, a fully rewritten user question is explicitly generated, accurately supplementing the missing information in the user's question. This achieves completeness and readability of the user's question content, improving the user experience.

[0174] To achieve the above embodiments, this application also proposes an electronic device, such as... Figure 11 As shown, Figure 11 This is a block diagram of an electronic device for training a dialogue processing method or a question rewriting model, according to an exemplary embodiment.

[0175] like Figure 11 As shown, the above-mentioned electronic device 1100 includes:

[0176] The memory 1110 and processor 1120 are connected by a bus 1130, which connects different components (including the memory 1110 and the processor 1120). The memory 1110 stores a computer program, which, when executed by the processor 1120, implements the dialogue processing method or question rewriting model training method described in the embodiments of this disclosure.

[0177] Bus 1130 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0178] Electronic device 1100 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by electronic device 1100, including volatile and non-volatile media, removable and non-removable media.

[0179] Memory 1110 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 1140 and / or cache memory 1150. Electronic device 1100 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 1160 may be used to read and write non-removable, non-volatile magnetic media (… Figure 11 Not shown; usually referred to as a "hard drive"). Although Figure 11 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 1130 via one or more data media interfaces. Memory 1110 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0180] A program / utility 1180 having a set (at least one) of program modules 1170 may be stored, for example, in memory 1110. Such program modules 1170 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 1170 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0181] Electronic device 1100 can also communicate with one or more external devices 1190 (e.g., keyboard, pointing device, display, etc.), and with one or more devices that enable a user to interact with the electronic device 1100, and / or with any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 1192. Furthermore, electronic device 1100 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1193. Figure 11 As shown, network adapter 1093 communicates with other modules of electronic device 1100 via bus 1130. It should be understood that, although... Figure 11 As not shown, other hardware and / or software modules can be used in conjunction with electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0182] The processor 1120 performs various functional applications and data processing by running programs stored in the memory 1110.

[0183] It should be noted that the implementation process and technical principles of the electronic device in this embodiment are described in [reference needed]. Figures 1 to 5 Explanation of the dialogue processing in the embodiments of this disclosure, or Figures 6 to 8 The explanation of training the question rewriting model according to the embodiments of this disclosure will not be repeated here.

[0184] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing computer instructions, which are used to cause a computer to perform the above-described actions. Figures 1 to 5 The dialogue processing method described in the embodiments, or Figures 6 to 8 The training method for the question rewriting model described in the embodiment.

[0185] To implement the above embodiments, this disclosure also provides a computer program product, which, when executed by an instruction processor, performs the above-described... Figures 1 to 5The dialogue processing method described in the embodiments, or Figures 6 to 8 The training method for the question rewriting model described in the embodiment.

[0186] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0187] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0188] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0189] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0190] It should be understood that various parts of this disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0191] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0192] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0193] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A dialogue processing method, characterized in that, The method is applied in a dialogue system, which has a question rewriting model. The question rewriting model includes an encoding layer and a decoding layer. The encoding layer includes a semantic representation layer and a keyword analysis layer. The method includes: Retrieve user questions that need to be answered; If it is determined that the user question has missing information, the historical dialogue content of the user question is obtained, wherein the difference between the round in which the user question is located and the round in which the historical dialogue content is located is less than a preset round threshold. The historical dialogue content and the user question are simultaneously input into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user question. The semantic representation vectors of each word in the historical dialogue content are input into the keyword analysis layer. The keyword analysis layer uses self-attention weights to calculate the semantic relevance between each word in order to obtain the keyword score of each word in the historical dialogue content. The semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores are simultaneously input into the decoding layer to obtain a completely rewritten user question. Obtain the response information to the rewritten user question, and output the response information; The step of simultaneously inputting the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores into the decoding layer to obtain a completely rewritten user question includes: The semantic representation vectors of each word in the user question are decoded to obtain the user question. Based on the semantic representation vectors of each word in the historical dialogue content, the initial word probability of each word in the historical dialogue content is determined. Specifically, based on the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user's question, the initial word probability of each word in the historical dialogue content and the initial word probability of each word in the user's question are calculated respectively. The final word probability is obtained by weighting the word probabilities of the historical chat and the user's question. Based on the keyword scores and initial word probabilities of the semantic representation vectors of each word in the historical dialogue content, the target word probability of each word in the historical dialogue content is determined. Based on the target word probability of each word in the historical dialogue content, the word with the highest target word probability is selected as the target word; Based on the target words, the missing information in the user question is filled in to obtain a rewritten user question with complete information.

2. The method as described in claim 1, characterized in that, The semantic representation vectors of each word in the historical dialogue content are input into the keyword analysis layer. The keyword analysis layer uses self-attention weights to calculate the semantic relevance between each word to obtain the keyword score of each word in the historical dialogue content, including: Based on the semantic representation vectors of each word in a multi-word set, determine the semantic relevance between the words. The keyword score for each word is determined based on the semantic relevance between them.

3. The method as described in claim 2, characterized in that, The process of determining the keyword score for each word based on the semantic relevance between them includes: The importance score of each word is determined based on the semantic relevance between them. Based on the importance score of each word, the keyword score of each word is determined.

4. The method according to any one of claims 1-3, characterized in that, Before inputting the historical dialogue content and the user question into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user question, the method further includes: The historical dialogue content is segmented into words to obtain the individual words in the historical dialogue content; Match each word in the historical dialogue content with the noise words in the noise word list; Delete words in the historical dialogue content that match the noise word.

5. A training method for a question rewriting model, characterized in that, The question rewriting model includes an encoding layer and a decoding layer. The encoding layer includes a semantic representation layer and a keyword analysis layer. The method includes: Acquire training data, wherein the training data includes sample user questions, sample historical dialogue content of the sample user questions, and a rewritten first sample user question, wherein the sample user questions have missing information, and the round difference between the round in which the sample user questions are located and the round in which the sample historical dialogue content is located is less than a preset round threshold. The sample historical dialogue content and the sample user question are simultaneously input into the semantic representation layer to obtain the semantic representation vectors of each word in the sample historical dialogue content and the semantic representation vectors of each word in the sample user question. The semantic representation vectors of each word in the sample historical dialogue content are input into the keyword analysis layer. The keyword analysis layer uses self-attention weights to calculate the semantic relevance between each word in order to obtain the keyword score of each word in the sample historical dialogue content. The semantic representation vectors of each word in the sample user question, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores are input into the decoding layer to obtain the second sample user question after complete rewriting of the information. The question rewriting model is trained based on the rewritten first sample user question and the rewritten second sample user question; Specifically, the semantic representation vectors of each word in the sample user question, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores are input into the decoding layer to obtain a second sample user question with complete information rewriting: The semantic representation vectors of each word in the sample user question are decoded to obtain the sample user question; Based on the semantic representation vectors of each word in the sample historical dialogue content, the initial word probability of each word in the sample historical dialogue content is determined. Specifically, based on the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user question, the initial word probability of each word in the historical dialogue content and the initial word probability of each word in the user question are calculated respectively. The final word probability is obtained by weighting the word probabilities of the historical chat and the user question. Based on the keyword scores and initial word probabilities of the semantic representation vectors of each word in the sample historical dialogue content, the target word probability of each word in the sample historical dialogue content is determined. Based on the target word probability of each word in the sample historical dialogue content, the word with the highest target word probability is selected as the target word; Based on the target words, the missing information in the sample user question is filled in to obtain a second sample user question with complete information after rewriting.

6. The method as described in claim 5, characterized in that, The semantic representation vectors of each word in the sample historical dialogue content are input into the keyword analysis layer. The keyword analysis layer uses self-attention weights to calculate the semantic relevance between each word to obtain the keyword score of each word in the sample historical dialogue content, including: Based on the semantic representation vectors of each word in a multi-word set, determine the semantic relevance between the words. The keyword score for each word is determined based on the semantic relevance between them.

7. The method as described in claim 6, characterized in that, The process of determining the keyword score for each word based on the semantic relevance between them includes: The importance score of each word is determined based on the semantic relevance between them. Based on the importance score of each word, the keyword score of each word is determined.

8. A dialogue processing device, characterized in that, The device is used in a dialogue system, which has a question rewriting model. The question rewriting model includes an encoding layer and a decoding layer. The encoding layer includes a semantic representation layer and a keyword analysis layer. The device includes: The first acquisition module is used to acquire user questions that need to be answered. The second acquisition module is used to acquire the historical dialogue content of the user question when it is determined that there is missing information in the user question, wherein the round difference between the round in which the user question is located and the round in which the historical dialogue content is located is less than a preset round threshold. The third acquisition module is used to simultaneously input the historical dialogue content and the user question into the semantic representation layer to obtain the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user question. The fourth acquisition module is used to input the semantic representation vector of each word in the historical dialogue content into the keyword analysis layer. The keyword analysis layer uses self-attention weights to calculate the semantic relevance between each word in order to obtain the keyword score of each word in the historical dialogue content. The fifth acquisition module is used to input the semantic representation vectors of each word in the user question, the semantic representation vectors of each word in the historical dialogue content, and the keyword scores into the decoding layer to obtain the user question with complete information rewritten. The output module is used to obtain the response information to the rewritten user question and output the response information. The fifth acquisition module is specifically used for: The semantic representation vectors of each word in the user question are decoded to obtain the user question. Based on the semantic representation vectors of each word in the historical dialogue content, the initial word probability of each word in the historical dialogue content is determined. Specifically, based on the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user's question, the initial word probability of each word in the historical dialogue content and the initial word probability of each word in the user's question are calculated respectively. The final word probability is obtained by weighting the word probabilities of the historical chat and the user's question. Based on the keyword scores and initial word probabilities of the semantic representation vectors of each word in the historical dialogue content, the target word probability of each word in the historical dialogue content is determined. Based on the target word probability of each word in the historical dialogue content, the word with the highest target word probability is selected as the target word; Based on the target words, the missing information in the user question is filled in to obtain a rewritten user question with complete information.

9. The apparatus as claimed in claim 8, characterized in that, The fourth acquisition module is specifically used for: Based on the semantic representation vectors of each word in a multi-word set, determine the semantic relevance between the words. The keyword score for each word is determined based on the semantic relevance between them.

10. The apparatus as claimed in claim 9, characterized in that, The fourth acquisition module is also used for: The importance score of each word is determined based on the semantic relevance between them. Based on the importance score of each word, the keyword score of each word is determined.

11. The apparatus according to any one of claims 8-10, characterized in that, The device further includes: The word segmentation module is used to segment the historical dialogue content into words to obtain the individual words in the historical dialogue content; The matching module is used to match each word in the historical dialogue content with noise words in the noise word list; The deletion module is used to delete words in the historical dialogue content that match the noise words.

12. A training device for a question rewriting model, characterized in that, The question rewriting model includes an encoding layer and a decoding layer, the encoding layer includes a semantic representation layer and a keyword analysis layer, and the device includes: The first acquisition module is used to acquire training data, wherein the training data includes sample user questions, sample historical dialogue content of the sample user questions, and a rewritten first sample user question, wherein the sample user questions have missing information, and the round difference between the round in which the sample user questions are located and the round in which the sample historical dialogue content is located is less than a preset round threshold. The second acquisition module is used to simultaneously input the sample historical dialogue content and the sample user question into the semantic representation layer to obtain the semantic representation vector of each word in the sample historical dialogue content and the semantic representation vector of each word in the sample user question. The third acquisition module is used to input the semantic representation vector of each word in the sample historical dialogue content into the keyword analysis layer. The keyword analysis layer uses self-attention weights to calculate the semantic relevance between each word in order to obtain the keyword score of each word in the sample historical dialogue content. The fourth acquisition module is used to input the semantic representation vectors of each word in the sample user question, the semantic representation vectors of each word in the sample historical dialogue content, and the keyword scores into the decoding layer to obtain the second sample user question after complete rewriting of information. The training module is used to train the question rewriting model based on the rewritten first sample user question and the rewritten second sample user question; The fourth acquisition module is specifically used for: The semantic representation vectors of each word in the sample user question are decoded to obtain the sample user question; Based on the semantic representation vectors of each word in the sample historical dialogue content, the initial word probability of each word in the sample historical dialogue content is determined. Specifically, based on the semantic representation vectors of each word in the historical dialogue content and the semantic representation vectors of each word in the user question, the initial word probability of each word in the historical dialogue content and the initial word probability of each word in the user question are calculated respectively. The final word probability is obtained by weighting the word probabilities of the historical chat and the user question. Based on the keyword scores and initial word probabilities of the semantic representation vectors of each word in the sample historical dialogue content, the target word probability of each word in the sample historical dialogue content is determined. Based on the target word probability of each word in the sample historical dialogue content, the word with the highest target word probability is selected as the target word; Based on the target words, the missing information in the sample user question is filled in to obtain a second sample user question with complete information after rewriting.

13. The apparatus as claimed in claim 12, characterized in that, The third acquisition module is specifically used for: Based on the semantic representation vectors of each word in a multi-word set, determine the semantic relevance between the words. The keyword score for each word is determined based on the semantic relevance between them.

14. The apparatus as claimed in claim 13, characterized in that, The third acquisition module is also used for: The importance score of each word is determined based on the semantic relevance between them. Based on the importance score of each word, the keyword score of each word is determined.

15. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements the method as described in any one of claims 1-4, or implements the method as described in any one of claims 5-7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-4, or implements the method as described in any one of claims 5-7.

Images