Question-answer pair construction method, device, electronic device, and storage medium

Abstract

The present disclosure relates to the field of computer technology and discloses a question-answer pair construction method, device, electronic device and storage medium. The method comprises: determining the criticality of each user question in a customer service conversation based on the semantic similarity between each user question in the customer service conversation; selecting a target user question from the user questions in the customer service conversation based on the criticality of each user question; matching each candidate customer service reply sentence corresponding to the target user question with the target user question, and determining a standard user question and a standard reply sentence in the question-answer pair based on the matching results; wherein the candidate customer service reply sentence is the customer service reply sentence corresponding to the target user question in the customer service conversation. This method can efficiently and accurately construct a question-answer pair.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method, device, electronic device, and storage medium for constructing question-answer pairs. Background Art

[0002] Frequently Asked Questions (FAQ) systems automatically provide responses to user questions without human intervention, significantly improving customer service efficiency. The FAQ system compares the question being processed with the questions in each question-and-answer pair in the knowledge base. After identifying questions with semantic similarities to the question being processed, the response from that question-and-answer pair is returned to the user. Therefore, it is crucial to construct a complete and rich set of question-and-answer pairs within the FAQ system's knowledge base.

[0003] In the related art, customer service personnel manually determine question-answer pairs and add them to the knowledge base. However, this method of constructing question-answer pairs is time-consuming and labor-intensive. Summary of the Invention

[0004] The present disclosure provides a question-answer pair construction method, device, electronic device, and storage medium for efficiently and accurately constructing question-answer pairs.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for constructing a question-answer pair, the method comprising:

[0006] Determining the criticality of each user question in the customer service conversation based on the semantic similarity between the user questions in the customer service conversation;

[0007] Selecting target user questions from the user questions in the customer service session based on the criticality of each user question;

[0008] Match each candidate customer service reply sentence corresponding to the target user question with the target user question respectively, and determine the standard user question and standard reply sentence in the question-answer pair based on the matching results; wherein the candidate customer service reply sentence is the customer service reply sentence corresponding to the target user question in the customer service conversation.

[0009] In some optional implementations, determining the criticality of each user question in the customer service session based on the semantic similarity between the user questions in the customer service session includes:

[0010] The criticality of the first user question in the customer service session is determined based on the semantic similarity between the first user question and each second user question, the criticality of each second user question in the customer service session, and the number of second user questions in the customer service session.

[0011] In some optional implementations, selecting target user questions from the user questions in the customer service session based on the criticality of each user question includes:

[0012] Based on the criticality of each user question, selecting a first preset number of target user questions from the customer service conversation; or

[0013] A user question sentence in the customer service conversation whose criticality is greater than a preset criticality is determined as the target user question sentence.

[0014] In some optional implementations, matching each candidate customer service reply sentence corresponding to the target user question with the target user question includes:

[0015] Determine whether the number of identical keywords in the target user question and each candidate customer service reply sentence is greater than a second preset number, and / or

[0016] Determine whether the semantic similarity between the target user question and each candidate customer service reply sentence is greater than a first preset similarity.

[0017] In some optional implementations, determining a standard user question and a standard answer in a question-answer pair based on the matching results includes:

[0018] If the semantic similarity between the target user question and the candidate customer service reply sentence is greater than a first preset similarity, and / or the number of identical keywords in the target user question and the candidate customer service reply sentence is greater than a second preset number, then the target user question is determined as the standard user question in the question-answer pair, and the candidate customer service reply sentence is determined as the standard reply sentence in the question-answer pair.

[0019] In some optional embodiments, the method further includes:

[0020] Determine the semantic similarity between each non-target user question and the target user question, and determine the non-target user question whose semantic similarity to the target user question is greater than a second preset similarity as a similar user question in the question-answer pair; wherein the similar user question is associated with the standard answer sentence in the question-answer pair.

[0021] In a second aspect, an embodiment of the present disclosure provides a question-answer pair construction device, comprising:

[0022] A criticality determination module, configured to determine the criticality of each user question in the customer service session based on the semantic similarity between the user questions in the customer service session;

[0023] a question selection module, configured to select target user questions from the user questions in the customer service session based on the criticality of each user question;

[0024] The question-answer pair processing module is used to match each candidate customer service reply sentence corresponding to the target user question with the target user question respectively, and determine the standard user question and standard reply sentence in the question-answer pair based on the matching results; wherein the candidate customer service reply sentence is the customer service reply sentence corresponding to the target user question in the customer service conversation.

[0025] In some optional implementations, the criticality determination module is specifically configured to:

[0026] The criticality of the first user question in the customer service session is determined based on the semantic similarity between the first user question and each second user question, the criticality of each second user question in the customer service session, and the number of second user questions in the customer service session.

[0027] In some optional implementations, the question selection module is specifically configured to:

[0028] Based on the criticality of each user question, selecting a first preset number of target user questions from the customer service conversation; or

[0029] A user question sentence in the customer service conversation whose criticality is greater than a preset criticality is determined as the target user question sentence.

[0030] In some optional implementations, the question-answer pair processing module is specifically configured to:

[0031] Determine whether the number of identical keywords in the target user question and each candidate customer service reply sentence is greater than a second preset number, and / or

[0032] Determine whether the semantic similarity between the target user question and each candidate customer service reply sentence is greater than a first preset similarity.

[0033] In some optional implementations, the question-answer pair processing module is specifically configured to:

[0034] If the semantic similarity between the target user question and the candidate customer service reply sentence is greater than a first preset similarity, and / or the number of identical keywords in the target user question and the candidate customer service reply sentence is greater than a second preset number, then the target user question is determined as the standard user question in the question-answer pair, and the candidate customer service reply sentence is determined as the standard reply sentence in the question-answer pair.

[0035] In some optional implementations, the question-answer pair processing module is further configured to:

[0036] Determine the semantic similarity between each non-target user question and the target user question, and determine the non-target user question whose semantic similarity to the target user question is greater than a second preset similarity as a similar user question in the question-answer pair; wherein the similar user question is associated with the standard answer sentence in the question-answer pair.

[0037] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, the processor executes the question-answer pair construction method as described in any one of the first aspects above.

[0038] In a fourth aspect, an embodiment of the present disclosure provides a storage medium storing a computer program executable by an electronic device. When the program runs on the electronic device, the electronic device executes the question-answer pair construction method as described in any one of the first aspects above.

[0039] The question-answer pair construction method, device, electronic device, and storage medium provided by the embodiments of the present disclosure have the following beneficial effects:

[0040] By measuring the semantic similarity between user questions in a customer service conversation, we can determine the criticality that represents the importance of each user question in the customer service conversation. Since there are usually customer service reply sentences corresponding to important user questions in a customer service conversation, we select important target user questions from the customer service conversation, and then match the target user questions with their corresponding candidate customer service reply sentences. When the match is successful, it means that the candidate customer service reply sentence is the standard reply to the target user question, thereby efficiently and accurately constructing a pair of standard user questions and standard reply sentences in a question-answer pair.

[0041] Other features and advantages of the present disclosure will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present disclosure. The purposes and other advantages of the present disclosure can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 This is a schematic diagram of the first question-answer pair provided by the embodiment of the present disclosure;

[0044] Figure 2 This is a flowchart of the first question-answer pair construction method provided by the embodiment of the present disclosure;

[0045] Figure 3 is a schematic diagram of semantic similarity between user questions provided by an embodiment of the present disclosure;

[0046] Figure 4 This is a flow chart of a second question-answer pair construction method provided by an embodiment of the present disclosure;

[0047] Figure 5 This is a schematic diagram of the second question-answer pair provided by the embodiment of the present disclosure;

[0048] Figure 6 Schematic diagram of a question-answer pair construction device provided by an embodiment of the present disclosure;

[0049] Figure 7 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;

[0050] Figure 8 Schematic diagram of a storage medium provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0051] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. Rather, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0052] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software.

[0053] In the embodiments of the present disclosure, the term "and / or" describes the association relationship between associated objects, indicating that three possible relationships exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0054] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of such features. Throughout this disclosure, unless otherwise specified, "plurality" means two or more.

[0055] It should be understood herein that any number of elements in the drawings is for illustration only and not for limitation, and any naming is only for distinction and does not have any limiting meaning.

[0056] The principles and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure. SUMMARY OF THE INVENTION

[0058] By building question-answer pairs in the FAQ system's knowledge base, the FAQ system compares the question to be processed with the questions in each question-answer pair in the knowledge base. After determining the question with similar semantics to the question to be processed, the system returns the answer sentence in the question-answer pair to the user, thus achieving intelligent question answering. Figure 1 As shown, if question 1 in question-answer pair 1 is semantically similar to the question to be processed, answer sentence 1 in the question-answer pair is returned to the user.

[0059] It can be seen that it is very important to construct complete and rich question-answer pairs in the knowledge base of the FAQ system.

[0060] In the related art, customer service personnel manually determine question-and-answer pairs and add them to the knowledge base. However, this method of constructing question-and-answer pairs is not only time-consuming and labor-intensive, but also prone to errors. If an incorrect question-and-answer pair is constructed, an incorrect response sentence will be returned to the user.

[0061] In view of this, an embodiment of the present disclosure provides a question-answer pair construction method, device, electronic device and storage medium, the method comprising: determining the criticality of each user question in the customer service session based on the semantic similarity between each user question in the customer service session; selecting a target user question from the user questions in the customer service session based on the criticality of each user question; matching each candidate customer service reply sentence corresponding to the target user question with the target user question, and determining a standard user question and a standard reply sentence in the question-answer pair based on the matching results; wherein the candidate customer service reply sentence is the customer service reply sentence corresponding to the target user question in the customer service session.

[0062] The above embodiment can determine the criticality that represents the importance of each user question in the customer service conversation through the semantic similarity between each user question in the customer service conversation; since there are usually customer service reply sentences corresponding to important user questions in the customer service conversation, important target user questions are selected from the customer service conversation, and then the target user questions are matched with their corresponding candidate customer service reply sentences respectively. When the match is successful, it means that the candidate customer service reply sentence is the standard reply to the target user question, thereby efficiently and accurately constructing a pair of standard user questions and standard reply sentences in the question-answer pair.

[0063] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0064] Exemplary Methods

[0065] Figure 2 FIG. 1 is a flow chart illustrating a first method for constructing question-answer pairs according to an exemplary embodiment. The method includes the following steps:

[0066] Step S201: determining the criticality of each user question in the customer service session based on the semantic similarity between the user questions in the customer service session.

[0067] Since customer service conversations usually have customer service responses corresponding to important user questions, it is necessary to first determine the criticality that represents the importance of each user question in the customer service conversation. The semantic similarity between user questions in the customer service conversation reflects their criticality in the customer service conversation.

[0068] Based on this, this embodiment needs to first determine the criticality of each user question in the customer service session based on the semantic similarity between each user question in the customer service session.

[0069] The semantic similarity between user questions in the above customer service conversation is the semantic similarity between every two user questions.

[0070] For example, each user question is regarded as a node, and the semantic similarity is used as the weight between nodes to construct an undirected connected graph. Figure 3 As shown, the customer service conversation includes user questions A, B, C, and D. Among them, the semantic similarity between user questions A and B is 0.47, the semantic similarity between user questions A and C is 0.08, the semantic similarity between user questions A and D is 0.17, the semantic similarity between user questions B and C is 0.02, the semantic similarity between user questions B and D is 0.21, and the semantic similarity between user questions C and D is 0.32.

[0071] The above examples are only intended to more clearly illustrate the semantic similarity between user questions in a customer service conversation, but this embodiment is not limited thereto.

[0072] Step S202: Based on the criticality of each user question, target user questions are selected from the user questions in the customer service conversation.

[0073] In this embodiment, since there are usually customer service reply sentences corresponding to important user questions in the customer service conversation, it is necessary to select important target user questions from the customer service conversation.

[0074] Step S203: Match each candidate customer service answer sentence corresponding to the target user question with the target user question, and determine the standard user question and standard answer sentence in the question-answer pair based on the matching results.

[0075] The candidate customer service reply sentence is the customer service reply sentence corresponding to the target user question in the customer service conversation.

[0076] The candidate customer service response sentences can be one or more customer service response sentences corresponding to the target user question. For example, the customer service response sentence usually comes after the user asks the corresponding question. Therefore, the customer service response sentence after the target user question in the customer service conversation can be used as the candidate customer service response sentence.

[0077] The above scheme can determine the criticality that represents the importance of each user question in the customer service conversation through the semantic similarity between each user question in the customer service conversation; since there are usually customer service reply sentences corresponding to important user questions in the customer service conversation, therefore, important target user questions are selected from the customer service conversation, and then the target user questions are matched with their corresponding candidate customer service reply sentences respectively. When the match is successful, it means that the candidate customer service reply sentence is the standard reply to the target user question, thereby efficiently and accurately constructing a pair of standard user questions and standard reply sentences in the question-answer pair.

[0078] In some optional implementations, before the above step S201, the following steps are further included:

[0079] All sentences in the customer service conversation are vectorized. This embodiment does not limit the specific implementation method of the vectorization process. For example, Sentence-BERT technology is used. Sentence-BERT is a sentence semantic similarity judgment model trained using a twin-tower network of bidirectional encoders (BERT) based on a self-attention mechanism. BERT is a bidirectional encoder based on a self-attention mechanism that converts text into vector representations in real-number space.

[0080] Use the trained sentence classification model to identify the sentence patterns of each sentence in the customer service conversation, including questions and responses.

[0081] Sentences sent by users with question patterns are determined as user questions; and sentences sent by customer service with reply patterns are determined as customer service reply sentences.

[0082] Correspondingly, the semantic similarity between user questions can be determined based on the distance (such as cosine value) between the vectorized user questions. The greater the distance between the vectorized user questions, the smaller the corresponding semantic similarity.

[0083] The above is merely an exemplary description of an optional implementation method for determining the semantic similarity between user questions. This embodiment may also adopt other methods to determine the semantic similarity between user questions.

[0084] In some optional implementations, the above step S201 may be implemented by, but not limited to, the following methods:

[0085] The criticality of the first user question in the customer service session is determined based on the semantic similarity between the first user question and each second user question, the criticality of each second user question in the customer service session, and the number of second user questions in the customer service session.

[0086] The first user question is any user question in the customer service session, and the second user question is any user question other than the first user question in the customer service session.

[0087] For example, the key degree of the first user question can be calculated by the following formula (1):

[0088]

[0089] Among them, S(V i ) is the first user question V i The criticality, S(V j ) is any second user question V jThe criticality of Sim ij V i With V j The semantic similarity between Link(V j ) is the number of second user questions.

[0090] Let’s take the above four user questions as an example:

[0091] When determining the criticality of user question A, the first user question is user question A, and the second user question is user question B, user question C, and user question D;

[0092] When determining the criticality of user question B, the first user question is user question B, and the second user question is user question A, user question C, and user question D;

[0093] When determining the criticality of user question C, the first user question is user question C, and the second user question is user question A, user question B, and user question D;

[0094] When determining the criticality of user question D, the first user question is user question D, and the second user question is user question A, user question B, and user question C.

[0095] Substituting the similarity between each user's questions and the number of second user questions into the above formula (1), we can obtain formulas (2)-(5):

[0096]

[0097]

[0098]

[0099]

[0100] Based on the above formulas (2)-(5), the criticality S(A) of user question A, the criticality S(B) of user question B, the criticality S(C) of user question C, and the criticality S(D) of user question D can be solved.

[0101] For example:

[0102] 1) In the above equations (2)-(5), there are four unknowns: S(A), S(B), S(C), and S(D). Through the system of simultaneous equations, S(A), S(B), S(C), and S(D) can be directly solved;

[0103] 2) S(A), S(B), S(C) and S(D) can be set as initial values, and multiple iterations can be performed based on the above equations (2)-(5) until S(A), S(B), S(C) and S(D) basically no longer change, and the four unknowns can also be solved.

[0104] The above method of determining the criticality of a user's question is merely an example and is not intended to limit the present disclosure.

[0105] In some optional implementations, the above step S202 may be implemented by, but not limited to, the following methods:

[0106] 1) Based on the criticality of each user question, selecting a first preset number of target user questions from the customer service conversation;

[0107] Exemplarily, all user questions in the customer service session are sorted from most critical to least critical, and a first preset number of target user questions are selected according to the sorting, that is, N (N is the above-mentioned first preset number) target user questions with the greatest criticality are selected from the customer service session.

[0108] The first preset number can be set according to actual application scenarios, such as being set to 1.

[0109] 2) Determine the user questions in the customer service conversation whose key level is greater than a preset key level as the target user questions.

[0110] Exemplarily, the criticality of each user question in the customer service conversation is compared with a preset criticality. User questions with a criticality greater than the preset criticality are target user questions, and user questions with a criticality not greater than the preset criticality are non-target user questions.

[0111] In this embodiment, the target user question can be selected by any of the above methods to meet the needs of different application scenarios.

[0112] Figure 4 FIG. 1 is a flow chart illustrating a second method for constructing question-answer pairs according to an exemplary embodiment, the method comprising the following steps:

[0113] Step S401: Based on the semantic similarity between the user questions in the customer service session, determine the criticality of each user question in the customer service session.

[0114] Step S402: Based on the criticality of each user question, target user questions are selected from the user questions in the customer service conversation.

[0115] The specific implementation of steps S401 to S402 can refer to the above embodiment and will not be repeated here.

[0116] Step S403: Determine whether the number of identical keywords in the target user question and each candidate customer service reply sentence is greater than a second preset number, and / or determine whether the semantic similarity between the target user question and each candidate customer service reply sentence is greater than a first preset similarity.

[0117] Illustratively, this embodiment performs at least one of the following steps:

[0118] 1) Determine the number of identical keywords in the target user's question and each candidate customer service reply, and then determine whether the number of identical keywords is greater than a second preset number;

[0119] 2) Determine the semantic similarity between the target user question and each candidate customer service reply sentence, and then determine whether the semantic similarity is greater than a first preset similarity.

[0120] Among them, the above keywords are important segments in the corresponding sentences (user questions / customer service response sentences). This embodiment does not specifically limit the implementation method of determining the keywords of each sentence, such as extracting a certain number of keywords from the sentence through TFIDF.

[0121] The first preset similarity can be set according to the actual application scenario, such as 0.25;

[0122] The second preset number may also be set according to actual application scenarios, such as being set to 2.

[0123] Step S404: If the semantic similarity between the target user question and the candidate customer service reply sentence is greater than a first preset similarity, and / or the number of identical keywords in the target user question and the candidate customer service reply sentence is greater than a second preset number, then the target user question is determined as the standard user question in the question-answer pair, and the candidate customer service reply sentence is determined as the standard reply sentence in the question-answer pair.

[0124] In some optional implementations, step S404 may be implemented in the following ways, but not limited to:

[0125] The first method

[0126] If the semantic similarity between the target user question and the first target customer service reply sentence is greater than a first preset similarity, the target user question is determined as the standard user question in the question-answer pair, and the first target customer service reply sentence is determined as the standard reply sentence in the question-answer pair;

[0127] The first target customer service reply sentence is a candidate customer service reply sentence having the most identical keywords with the target user question sentence.

[0128] Exemplarily, the target user question among the above four user questions is user question C, corresponding to candidate customer service reply sentence c and candidate customer service reply sentence d; there is 1 identical keyword in user question C and candidate customer service reply sentence c, and there are 0 identical keywords in user question C and candidate customer service reply sentence d, and candidate customer service reply sentence c is the first target customer service reply sentence; if the semantic similarity between user question C and candidate customer service reply sentence c is greater than the first preset similarity, the match is successful, and user question C is determined as the standard user question in the question and answer pair, and candidate customer service reply sentence c is determined as the standard reply sentence in the same question and answer pair; if the semantic similarity between user question C and candidate customer service reply sentence c is not greater than the first preset similarity, the match fails, and a reliable question and answer pair cannot be constructed based on this customer service session.

[0129] Second method

[0130] If the number of identical keywords in the target user question and the second target customer service reply is greater than a second preset number, the target user question is determined as the standard user question in the question-answer pair, and the second target customer service reply is determined as the standard reply in the question-answer pair;

[0131] The second target customer service reply sentence is a candidate customer service reply sentence having the greatest semantic similarity to the target user question sentence.

[0132] For example, the target user question among the above four user questions is user question C, corresponding to candidate customer service reply sentence c and candidate customer service reply sentence d; the semantic similarity between user question C and candidate customer service reply sentence c is recorded as the first similarity, and the semantic similarity between user question C and candidate customer service reply sentence d is recorded as the second similarity, and the first similarity is greater than the second similarity, and candidate customer service reply sentence c is the second target customer service reply sentence; if the number of identical keywords in user question C and candidate customer service reply sentence c is greater than the second preset number, the match is successful, and user question C is determined as the standard user question in the question and answer pair, and candidate customer service reply sentence c is determined as the standard reply sentence in the same question and answer pair; if the number of identical keywords in user question C and candidate customer service reply sentence is not greater than the second preset number, the match fails, and a reliable question and answer pair cannot be constructed based on this customer service session.

[0133] The above two methods of determining the standard user question and the standard answer in the question-answer pair are merely exemplary descriptions, but the present disclosure is not limited thereto.

[0134] In the above scheme, the semantic similarity between the target user question and the candidate customer service reply sentences, as well as the number of identical keywords in the target user question and the candidate customer service reply sentences, both represent the degree of matching between the target user question and the candidate customer service reply sentences. Therefore, when the semantic similarity between the target user question and the candidate customer service reply sentences is greater than the first preset similarity, and the number of identical keywords in the target user question and the candidate customer service reply sentences is greater than the second preset number, it indicates that the degree of matching is high. Based on this, a reliable question-answer pair can be constructed.

[0135] See Figure 5 As shown, in some optional implementations, the question-answer pair also includes similar user questions. After the target user question is determined as the standard user question in the question-answer pair, the following steps are further included:

[0136] Determining the semantic similarity between each non-target user question and the target user question, and determining a non-target user question whose semantic similarity to the target user question is greater than a second preset similarity as a similar user question in the question-answer pair;

[0137] The similar user questions are associated with the standard answer sentences in the question-answer pair.

[0138] The non-target user questions include non-target questions in this customer service session, and may also include non-target questions in other customer service sessions that are not associated with standard answer sentences in other question-answer pairs.

[0139] The second preset similarity can be set according to actual application scenarios, such as being set to 0.8.

[0140] Still taking the above four user questions as an example, the target user question is user question C. After user question C is determined as the standard user question in question-answer pair 1, the semantic similarity between user question A and user question C, the semantic similarity between user question B and user question C, and the semantic similarity between user question D and user question C are determined respectively; if the semantic similarity between user question A and user question C is not greater than the second preset similarity, the semantic similarity between user question B and user question C is greater than the second preset similarity, and the semantic similarity between user question D and user question C is greater than the second preset similarity, user question B is determined as the similar user question in question-answer pair 1. 11 , user question D is determined to be a similar user question in question-answer pair 1 12 .

[0141] After determining the standard user questions in other question-answer pairs, similar methods are used to determine similar user questions in corresponding question-answer pairs. Examples will not be given here one by one.

[0142] In addition, the above examples are only intended to more clearly illustrate how to determine similar user questions in a question-answer pair, but the present disclosure is not limited thereto.

[0143] In the above scheme, if the semantic similarity between a non-target user question and a target user question is greater than the second preset similarity, it means that the semantics of the non-target user question and the target user question are relatively similar. By taking the non-target user question as a similar user question in the question-answer pair, it is associated with the standard answer sentence in the question-answer pair, thereby constructing a richer question-answer pair.

[0144] Exemplary devices

[0145] Based on the same inventive concept, the embodiment of the present disclosure also provides a question-answer pair construction device, and the embodiment of the question-answer pair construction device can inherit the content described in the above method embodiment. Based on the above embodiment, Figure 6 FIG. 1 is a schematic diagram of a structure of a question-answer pair construction device provided by an embodiment of the present disclosure. The question-answer pair construction device 600 specifically includes:

[0146] A criticality determination module 601 is configured to determine the criticality of each user question in the customer service session based on the semantic similarity between the user questions in the customer service session;

[0147] A question selection module 602 is configured to select target user questions from the user questions in the customer service session based on the criticality of each user question;

[0148] The question-answer pair processing module 603 is used to match each candidate customer service reply sentence corresponding to the target user question with the target user question respectively, and determine the standard user question and standard reply sentence in the question-answer pair based on the matching results; wherein the candidate customer service reply sentence is the customer service reply sentence corresponding to the target user question in the customer service conversation.

[0149] In some optional implementations, the criticality determination module 601 is specifically configured to:

[0150] The criticality of the first user question in the customer service session is determined based on the semantic similarity between the first user question and each second user question, the criticality of each second user question in the customer service session, and the number of second user questions in the customer service session.

[0151] In some optional implementations, the question selection module 602 is specifically configured to:

[0152] Based on the criticality of each user question, selecting a first preset number of target user questions from the customer service conversation; or

[0153] A user question sentence in the customer service conversation whose criticality is greater than a preset criticality is determined as the target user question sentence.

[0154] In some optional implementations, the question-answer pair processing module 603 is specifically configured to:

[0155] Determine whether the number of identical keywords in the target user question and each candidate customer service reply sentence is greater than a second preset number, and / or

[0156] Determine whether the semantic similarity between the target user question and each candidate customer service reply sentence is greater than a first preset similarity.

[0157] In some optional implementations, the question-answer pair processing module 603 is specifically configured to:

[0158] If the semantic similarity between the target user question and the candidate customer service reply sentence is greater than a first preset similarity, and / or the number of identical keywords in the target user question and the candidate customer service reply sentence is greater than a second preset number, then the target user question is determined as the standard user question in the question-answer pair, and the candidate customer service reply sentence is determined as the standard reply sentence in the question-answer pair.

[0159] In some optional implementations, the question-answer pair processing module 603 is further configured to:

[0160] Determine the semantic similarity between each non-target user question and the target user question, and determine the non-target user question whose semantic similarity to the target user question is greater than a second preset similarity as a similar user question in the question-answer pair; wherein the similar user question is associated with the standard answer sentence in the question-answer pair.

[0161] Since the question-answer pair construction device is the question-answer pair construction device in the method in the embodiment of the present disclosure, and the principle of solving the problem by the question-answer pair construction device is similar to that of the method, the implementation of the question-answer pair construction device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0162] Refer to the following Figure 7 700 according to this embodiment of the present disclosure will be described. Figure 7 The electronic device shown is only an example and does not limit the functions and scope of use of the embodiments of the present disclosure.

[0163] like Figure 7 As shown, electronic device 700 is implemented as a general-purpose computing device. Components of electronic device 700 may include, but are not limited to, at least one processor 701, at least one memory 702, and a bus 703 connecting different system components (including memory 702 and processor 701).

[0164] Bus 703 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a processor or local bus using any of a variety of bus architectures.

[0165] The memory 702 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 7021 and / or a cache memory 7022 , and may further include a read-only memory (ROM) 7023 .

[0166] The memory 702 may also include a program / utility 7025 having a set (at least one) of program modules 7024, such program modules 7024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0167] The electronic device 700 may also communicate with one or more external devices 704 (e.g., a keyboard, pointing device, etc.), one or more devices that enable a user to interact with the electronic device 700, and / or any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., a router, a modem, etc.). Such communication may occur via an input / output (I / O) interface 705. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 706. As shown, the network adapter 706 communicates with other modules of the electronic device 700 via a bus 703. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0168] In an embodiment of the present disclosure, the memory 702 stores a computer program. When the program is executed by the processor 701, the processor 701 executes the method in any of the above embodiments.

[0169] Since the electronic device is the electronic device in the method in the embodiment of the present disclosure, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0170] Exemplary Program Products

[0171] In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product is run on an electronic device, the program code is used to enable the processor of the electronic device to execute the steps of any of the above-mentioned question-answer pair construction methods.

[0172] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0173] As mentioned above, the program product may be in the form of a storage medium, see Figure 8 , a storage medium 800 according to an embodiment of the present disclosure is described, which may be a portable compact disk read-only memory (CD-ROM) and includes program code and can be run on an electronic device. However, the storage medium of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0174] A readable signal medium may include a data signal transmitted in baseband or as part of a carrier wave, which carries readable program code. Such a transmitted data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0175] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0176] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the electronic device, partially on the electronic device, as a stand-alone software package, partially on the electronic device and partially on a remote device, or entirely on the remote device. In cases involving a remote device, the remote device may be connected to the electronic device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0177] It should be noted that although several modules or submodules of the system are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules.

[0178] Furthermore, although the operations of the modules of the disclosed system are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some operations may be omitted, multiple operations may be combined into one operation, and / or one operation may be decomposed into multiple operations.

[0179] Although the spirit and principles of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the specific embodiments disclosed, and the division into various aspects does not mean that the features in these aspects cannot be combined to benefit. Such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the appended claims.

Claims

1. A question-answer pair construction method, characterized in that: The method comprises: Determining the criticality of each user question in the customer service conversation based on the semantic similarity between the user questions in the customer service conversation; Selecting target user questions from the user questions in the customer service session based on the criticality of each user question; Match each candidate customer service answer sentence corresponding to the target user question with the target user question, and determine the standard user question and standard answer sentence in the question-answer pair based on the matching results; wherein the candidate customer service answer sentence is the customer service answer sentence corresponding to the target user question in the customer service conversation; Determining the criticality of each user question in the customer service session based on the semantic similarity between each user question in the customer service session includes: determining the criticality of the first user question in the customer service session based on the semantic similarity between the first user question and each second user question, the criticality of each second user question in the customer service session, and the number of second user questions in the customer service session; The criticality of the first user question is determined by the following formula: ； Among them, V i is the first user question, S(V i ) is the criticality of the first user question, V j For any second user question, S(V j ) is the criticality of the second user question, Sim ij is the semantic similarity between the first user question and the second user question, Link(V j ) are all second user questions, Link(V j ) The number of questions asked by the second user; Based on the matching results, the standard user question and standard answer sentences in the question-answer pair are determined, including: If the semantic similarity between the target user question and the candidate customer service reply sentence is greater than a first preset similarity, and / or the number of identical keywords in the target user question and the candidate customer service reply sentence is greater than a second preset number, then the target user question is determined as the standard user question in the question-answer pair, and the candidate customer service reply sentence is determined as the standard reply sentence in the question-answer pair.

2. The method according to claim 1, characterized in that Selecting target user questions from the user questions in the customer service session based on the criticality of each user question includes: Based on the criticality of each user question, selecting a first preset number of target user questions from the customer service conversation; or A user question sentence in the customer service conversation whose criticality is greater than a preset criticality is determined as the target user question sentence.

3. The method according to claim 1, characterized in that Matching each candidate customer service answer sentence corresponding to the target user question with the target user question, including: Determine whether the number of identical keywords in the target user question and each candidate customer service reply sentence is greater than a second preset number, and / or Determine whether the semantic similarity between the target user question and each candidate customer service reply sentence is greater than a first preset similarity.

4. The method according to claim 1, wherein Also includes: Determine the semantic similarity between each non-target user question and the target user question, and determine the non-target user question whose semantic similarity to the target user question is greater than a second preset similarity as a similar user question in the question-answer pair; wherein the similar user question is associated with the standard answer sentence in the question-answer pair.

5. A question-answer pair construction device, characterized in that: The device comprises: A criticality determination module, configured to determine the criticality of each user question in the customer service session based on the semantic similarity between the user questions in the customer service session; a question selection module, configured to select target user questions from the user questions in the customer service session based on the criticality of each user question; a question-answer pair processing module, configured to match each candidate customer service answer sentence corresponding to the target user question with the target user question, and determine a standard user question and a standard answer sentence in the question-answer pair based on the matching results; wherein the candidate customer service answer sentence is the customer service answer sentence corresponding to the target user question in the customer service conversation; The criticality determination module is specifically used for: determining the criticality of the first user question in the customer service session based on the semantic similarity between the first user question and each second user question, the criticality of each second user question in the customer service session, and the number of second user questions in the customer service session; The criticality determination module is specifically configured to determine the criticality of the first user question using the following formula: ； Among them, V i is the first user question, S(V i ) is the criticality of the first user question, V j For any second user question, S(V j ) is the criticality of the second user question, Sim ij is the semantic similarity between the first user question and the second user question, Link(V j ) are all second user questions, Link(V j ) The number of questions asked by the second user; The question-answer pair processing module is specifically used to: If the semantic similarity between the target user question and the candidate customer service reply sentence is greater than a first preset similarity, and / or the number of identical keywords in the target user question and the candidate customer service reply sentence is greater than a second preset number, then the target user question is determined as the standard user question in the question-answer pair, and the candidate customer service reply sentence is determined as the standard reply sentence in the question-answer pair.

6. The device according to claim 5, characterized in that The question selection module is specifically used for: Based on the criticality of each user question, selecting a first preset number of target user questions from the customer service conversation; or A user question sentence in the customer service conversation whose criticality is greater than a preset criticality is determined as the target user question sentence.

7. The device according to claim 5, characterized in that The question-answer pair processing module is specifically used to: Determine whether the number of identical keywords in the target user question and each candidate customer service reply sentence is greater than a second preset number, and / or Determine whether the semantic similarity between the target user question and each candidate customer service reply sentence is greater than a first preset similarity.

8. The device according to claim 5, characterized in that The question-answer pair processing module is further configured to: Determine the semantic similarity between each non-target user question and the target user question, and determine the non-target user question whose semantic similarity to the target user question is greater than a second preset similarity as a similar user question in the question-answer pair; wherein the similar user question is associated with the standard answer sentence in the question-answer pair.

9. An electronic device, characterized in that: The system comprises at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, the processor executes the method according to any one of claims 1 to 4.

10. A storage medium, characterized in that: It stores a computer program that can be executed by an electronic device. When the program runs on the electronic device, the electronic device executes the method according to any one of claims 1 to 4.

Images