Question and answer method, device and equipment

Abstract

The present application provides a question-and-answer method, device, and equipment. The method of the present application determines the entity information contained in the question by performing named entity recognition on the question input by the user, so as to determine the user's consulting object; matches the entity information contained in the question with the entity information of the candidate question in the question-and-answer knowledge base, calculates the semantic similarity between the question and the candidate question, and determines a first candidate question that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold, wherein the first candidate question has a high semantic similarity with the question and matches the entity information of the question; according to the semantic similarity between the first candidate question and the question, selects a preset answer of the first candidate question as the answer information of the question, and outputs the answer information of the question, which can avoid the situation where the answer information is incorrect due to the high similarity of the questions but different consulting objects, thereby improving the accuracy of the answer information.

Description

Technical Field

[0001] The present application relates to computer technology, and more particularly to a question-answering method, device and apparatus. Background Art

[0002] As the first consulting object many consumers encounter when they have questions about products, if the intelligent customer service robot can accurately and efficiently solve consumers' problems in the shortest possible time, it will greatly enhance consumers' shopping experience.

[0003] At present, in the FAQ (Frequently Asked Questions) question-and-answer scheme, a commonly used question-and-answer method is to use an online text matching algorithm to calculate the similarity between the question entered by the consumer and the question configured by the merchant. After the similarity reaches a certain threshold, the answer to the question configured by the merchant will be displayed to the consumer. However, if two questions are too similar, but the objects of consultation are different, for example, the two questions "Can a one-year-old child use it" and "Can a two-year-old child use it" are very similar, but the objects of consultation are different, it is easy to produce wrong answers. Summary of the invention

[0004] The present application provides a question-answering method, device and apparatus for solving the problem that two questions are very similar, but the objects of inquiry are different, which easily leads to wrong answers.

[0005] On the one hand, the present application provides a question-answering method, comprising:

[0006] Get the user's question;

[0007] Performing named entity recognition on the question to determine entity information contained in the question;

[0008] Matching the entity information contained in the question with the entity information of the question in the question-answering knowledge base, and calculating the semantic similarity between the question and the question in the question-answering knowledge base;

[0009] Determine, according to the matching result and the calculated semantic similarity, a first candidate question sentence that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold;

[0010] According to the semantic similarity between the first candidate question and the question, selecting a preset answer to the first candidate question as answer information to the question;

[0011] Output the answer information of the question.

[0012] On the other hand, the present application provides a question-answering device, comprising:

[0013] Question acquisition module, used to obtain user questions;

[0014] An entity recognition module, used to perform named entity recognition on the question to determine entity information contained in the question;

[0015] A matching module, used to match the entity information contained in the question with the entity information of the question in the question-answering knowledge base, and calculate the semantic similarity between the question and the question in the question-answering knowledge base; according to the matching result and the calculated semantic similarity, determine a first candidate question that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold;

[0016] an answer determination module, configured to select a preset answer to the first candidate question as answer information of the question according to the semantic similarity between the first candidate question and the question;

[0017] The output module is used to output the answer information of the question.

[0018] On the other hand, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0019] The memory stores computer-executable instructions;

[0020] The processor executes the computer-executable instructions stored in the memory to implement the above-mentioned method.

[0021] On the other hand, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement the above-described method when executed by a processor.

[0022] The question-and-answer method, apparatus, and device provided by the present application perform named entity recognition on the user's question to determine the entity information contained in the question, thereby identifying the object that the user wants to consult; match the entity information contained in the question with the entity information of the question in the question-and-answer knowledge base, and calculate the semantic similarity between the question and the question in the question-and-answer knowledge base; determine a first candidate question that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold based on the matching result and the calculated semantic similarity; select a preset answer to the first candidate question as answer information for the question based on the semantic similarity between the first candidate question and the question; output the answer information for the question, the first candidate question not only has a high semantic similarity with the question, but also has matching entity information with the input question, and the answer information of the first candidate question can be used as the answer information for the input question, which can avoid the situation where the answer information is incorrect due to the high similarity of the questions but different consultation objects, thereby improving the accuracy of the answer information. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0024] Figure 1 An example diagram of a system architecture to which the question-answering method provided in this application is applicable;

[0025] Figure 2 A flow chart of a question-and-answer method provided for an exemplary embodiment of the present application;

[0026] Figure 3 A flow chart of a question-and-answer method provided for another exemplary embodiment of the present application;

[0027] Figure 4 An example diagram of additionally displaying recommended questions provided by an exemplary embodiment of the present application;

[0028] Figure 5 An example diagram of a recommendation question with a small difference provided by an exemplary embodiment of the present application;

[0029] Figure 6 A flow chart of a method for automatically mining question and answer information provided by an exemplary embodiment of the present application;

[0030] Figure 7 An example diagram of clustering results of semantic categories provided by an exemplary embodiment of the present application;

[0031] Figure 8 An overall framework diagram of a question-answering system provided for an exemplary embodiment of the present application;

[0032] Fig. 9A schematic diagram of the structure of a question-answering device provided by an exemplary embodiment of the present application;

[0033] Fig.10 A schematic diagram of the structure of a question-and-answer device provided for another exemplary embodiment of the present application;

[0034] Fig.11 A schematic diagram of the structure of an electronic device provided in an exemplary embodiment of the present application.

[0035] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0036] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0037] In the field of intelligent customer service, a common implementation method is to configure answers for scenarios in the question-and-answer knowledge base configured by the merchant based on the abstracted scenarios. For example, scenarios related to product use such as "applicable age", "applicable population", "scope of use", "usage method", and "product function" can be configured, and answers can be configured for the configured scenarios for a product. At present, the configuration of answers to "product use" questions accounts for a large proportion. After online consumers ask questions, the scenario corresponding to the question raised by the consumer is determined based on the scenario classification algorithm, and the configured answer is output according to the scenario corresponding to the question. This method has the following technical defects: 1) Due to the accuracy of the scenario classification algorithm, once the scenario classification is wrong, an incorrect answer will be given; 2) The answer for each scenario corresponding to each product needs to be manually configured, and the efficiency of question configuration is low; 3) The granularity of the answer configuration is too coarse for the scenario configuration, and a large part of the consumer's questions are not covered. In addition, the answers configured for the scenario are relatively general, and the answers are usually too long, which increases the cost of understanding for consumers, and there is a problem of inaccurate answers in the correct scenario. If such problems cannot be solved by intelligent customer service robots, consumers will turn to manual customer service for solutions. If it is difficult to turn to manual or it takes too much time, satisfaction will decrease.

[0038] In addition to the question-and-answer solution based on scene recognition, there is another way to trigger FAQ online, which is to use a text matching algorithm online to calculate the similarity between the question entered by the consumer and the question configured by the merchant. After the similarity reaches a certain threshold, the answer to the question configured by the merchant will be displayed to the consumer. If two questions are too similar, but the objects of consultation are different, for example, the two questions "Can a one-year-old child use it" and "Can a two-year-old child use it" are very similar, but the objects of consultation are different, it is easy to produce wrong answers. In addition, this question-and-answer solution requires more questions to be manually configured.

[0039] In view of the problem that when the two questions in the above question-and-answer scheme are very similar but the objects of consultation are different, it is easy to produce incorrect answers. The present application provides a question-and-answer method, which performs named entity recognition on the user's question to determine the entity information contained in the question; matches the entity information contained in the question with the entity information of the question in the question-and-answer knowledge base, and calculates the semantic similarity between the question and the question in the question-and-answer knowledge base; determines a first candidate question that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold based on the matching result and the calculated semantic similarity; selects a preset answer to the first candidate question as the answer information of the question based on the semantic similarity between the first candidate question and the question; and outputs the answer information of the question. The method of the present application combines entity recognition and text matching as online trigger conditions, and outputs the corresponding answer information when the entity information of the question matches and the semantic similarity is high, thereby ensuring the correctness of the consultation object (entity) and improving the accuracy of the output answer.

[0040] For example, the question-answering method provided in this application can be applied to Figure 1 The system architecture is shown in Figure 1. Figure 1 As shown, the system architecture includes: a terminal and a server.

[0041] The server may be a server where the question-and-answer system is located, and specifically may be a server cluster deployed in the cloud. The server stores a question-and-answer knowledge base and configured named entity information. The question-and-answer knowledge base includes configured candidate questions and preset answers and entity information for the candidate questions. The entity combination includes pre-configured entity information that needs to be identified. The entity information corresponds to the objects that appear in the consulting questions, such as "one year old", "two years old", etc. Through the preset operation logic in the server, the server realizes the automatic configuration and update of the question-and-answer knowledge base, as well as the online question-and-answer functions.

[0042] The system architecture includes a first terminal used by online users (such as consumers) and a second terminal used by product providers (such as merchants). The first terminal and the second terminal can be hardware devices with network communication functions, computing functions, and information display functions, including but not limited to smartphones, tablet computers, desktop computers, Internet of Things devices, etc.

[0043] Through communication interaction with the server, when the user inputs a question through the first terminal, the first terminal sends the question to the server. The server obtains the user's question, performs named entity recognition on the question to determine the entity information contained in the question; matches the entity information contained in the question with the entity information of the question sentence in the question-answering knowledge base, and calculates the semantic similarity between the question and the question sentence in the question-answering knowledge base; determines a first candidate question sentence that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold value based on the matching result and the calculated semantic similarity; selects a preset answer of the first candidate question sentence as the answer information of the question based on the semantic similarity between the first candidate question sentence and the question, and outputs the answer information of the question to the first terminal.

[0044] In addition, the server can also automatically mine question-answer pairs based on historical data, the question-answer pairs include sample questions with product objects and answer information of the sample questions; according to the entity information of the sample questions, the sample questions with matching entity information are clustered into the same entity category to form multiple entity categories; in the same entity category, the sample questions are clustered based on the semantic similarity to form at least one semantic category, and the sample questions and answer information are output to the second terminal, and the second terminal classifies and outputs the sample questions and answer information, at least one answer information corresponding to each semantic category, so that the product provider (such as a merchant) can review the answer information of the sample questions of each semantic category, and configure the general answer information corresponding to each semantic category through simple modification, deletion, adoption and other operations, and submit the sample questions and answer information of the configured semantic category to the server, and the server uses the sample questions that have passed the review as candidate questions, and adds the sample questions and the entity information and answer information of the sample questions to the question-answer knowledge base. In addition, the question-answer knowledge base can also store information such as semantic categories and entity categories corresponding to the candidate questions.

[0045] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0046] Figure 2 A flowchart of a question-answering method provided by an exemplary embodiment of the present application. The execution subject of this embodiment may be the server mentioned above, such as Figure 2 As shown, the specific steps of this method are as follows:

[0047] Step S201: Obtain the user's question.

[0048] The user's question may be a question input by the user online, specifically a text message input by the user through text input, or a voice input by the user through voice input or the like, and the input question may be obtained through voice recognition. This embodiment does not limit the way in which the user inputs the question.

[0049] Step S202: Perform named entity recognition on the question to determine the entity information contained in the question.

[0050] In this embodiment, according to the objects that often appear in the consultation questions, each object is configured as an entity information, and a large amount of object corresponding entity information is obtained. The server stores pre-configured named entity information, and the named entity information includes object corresponding entity information.

[0051] After obtaining the user's question, the server uses the named entity recognition (NER) model to perform named entity recognition on the input question to identify the entity information contained in the question.

[0052] Exemplarily, each entity information may be a key-value pair, where the key represents the category, field or scenario information of the entity, and the value represents the value of the entity, which may be the name of the entity. For any two entity information, as long as any one of the key and value is different, it means that the two entity information are different entity information. For example, "Brand-Apple" and "Food-Apple" are two different entity information.

[0053] Step S203: Match the entity information contained in the question with the entity information of the question in the question-answering knowledge base, and calculate the semantic similarity between the question and the question in the question-answering knowledge base.

[0054] After identifying the entity information contained in the input question, the entity information contained in the question is matched with the entity information of the question sentence in the question-answering knowledge base to determine a candidate question sentence that matches the entity information of the question. At the same time, the server may also calculate the semantic similarity between the input question and the candidate question sentence in the question-answering knowledge base, thereby matching the input question with the candidate question sentence in the question-answering knowledge base by semantic similarity to determine a candidate question sentence whose semantic similarity to the question is greater than or equal to a first threshold.

[0055] Step S204: Determine, based on the matching result and the calculated semantic similarity, a first candidate question sentence that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold.

[0056] Based on the matching results and the calculated semantic similarity, a first candidate question that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold can be determined. The first candidate question not only has a high semantic similarity with the question, but also has matching entity information. The answer information of the first candidate question can be used as the answer information of the input question, thereby avoiding the situation where the answer information is incorrect due to the high similarity of the questions but different consulting objects, thereby improving the accuracy of the answer information.

[0057] Step S205: According to the semantic similarity between the first candidate question and the question, a preset answer of the first candidate question is selected as answer information of the question.

[0058] After determining the first candidate question, a first candidate question can be selected from the first candidate questions, and the preset answer of the selected first candidate question in the question-answering knowledge base can be used as the answer information of the input question. Since the first candidate question not only has a high semantic similarity with the question but also has matching entity information, the answer information of the first candidate question can be used as the answer information of the input question, which can avoid the situation where the answer information is incorrect due to the high similarity of the questions but different consulting objects, and improve the accuracy of the answer information.

[0059] Optionally, according to the semantic similarity between the first candidate question and the question, a preset answer of a first candidate question having a higher semantic similarity between the first candidate question and the question may be used as the answer information of the input question.

[0060] Exemplarily, a preset answer to a first candidate question sentence having the highest semantic similarity between the first candidate question sentence and the question is used as the answer information of the input question.

[0061] In addition, in this step, a first candidate question sentence may be randomly selected from the first candidate question sentences, and a preset answer to the randomly selected first candidate question sentence may be used as the answer information of the input question.

[0062] Step S206: output the answer information of the question.

[0063] In this embodiment, named entity recognition is performed on the user's question to determine the entity information contained in the question, thereby identifying the object that the user wants to consult; the entity information contained in the question is matched with the entity information of the question in the question and answer knowledge base, and the semantic similarity between the question and the question in the question and answer knowledge base is calculated; based on the matching result and the calculated semantic similarity, a first candidate question is determined that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold; based on the semantic similarity between the first candidate question and the question, a preset answer to the first candidate question is selected as the answer information of the question; the answer information of the question is output, the first candidate question not only has a high semantic similarity with the question, but also has matching entity information with the input question, the answer information of the first candidate question can be used as the answer information of the input question, which can avoid the situation where the answer information is incorrect due to the high similarity of the questions but different consultation objects, thereby improving the accuracy of the answer information.

[0064] Figure 3 A flowchart of a question-and-answer method provided for another exemplary embodiment of the present application. Based on the above embodiment, in this embodiment, when there is no first candidate question sentence in the question-and-answer knowledge base that matches the entity information of the question input by the user and whose semantic similarity with the question is greater than or equal to a first threshold, the output answer information and the additional display of recommended questions are displayed. If the output answer information fails to solve the user's problem, the user can click on the additionally displayed recommended questions to standardize the questions submitted by the user and obtain accurate answers, so as to improve the ability of the intelligent customer service robot to solve user problems.

[0065] like Figure 3 As shown, the specific steps of this method are as follows:

[0066] Step S301: Obtain the user's question.

[0067] The user's question may be a question input by the user online, specifically a text message input by the user through text input, or a voice input by the user through voice input or the like, and the input question may be obtained through voice recognition. This embodiment does not limit the way in which the user inputs the question.

[0068] Step S302: Perform named entity recognition on the question to determine the entity information contained in the question.

[0069] In this embodiment, according to the objects that often appear in the consultation questions, each object is configured as an entity information, and a large amount of object corresponding entity information is obtained. The server stores pre-configured named entity information, and the named entity information includes object corresponding entity information.

[0070] After obtaining the user's question, the server uses the named entity recognition (NER) model to perform named entity recognition on the input question to identify the entity information contained in the question.

[0071] Step S303: Match the entity information contained in the question with the entity information of the question in the question-answering knowledge base, and calculate the semantic similarity between the question and the question in the question-answering knowledge base.

[0072] After identifying the entity information contained in the input question, the entity information contained in the question is matched with the entity information of the question sentence in the question-answering knowledge base to determine a candidate question sentence that matches the entity information of the question. At the same time, the server may also calculate the semantic similarity between the input question and the candidate question sentence in the question-answering knowledge base, thereby matching the input question with the candidate question sentence in the question-answering knowledge base by semantic similarity to determine a candidate question sentence whose semantic similarity to the question is greater than or equal to a first threshold.

[0073] For example, for example, "Can a one-year-old baby use it?", "Can a four-year-old baby use it?", "Can an infant use it?", different merchants and customer service staff have different understandings. In the case of different products and different scenarios, it is difficult to give a consistent conclusion on whether the text matches. In order to minimize the impact of subjective understanding on the text matching algorithm, in the construction of the question and answer knowledge base for questions related to "product usage", less attention is paid to semantic details, and the focus is placed on the consulting object. That is, "Can it be used?", "Can it be used?", "Can it be used?" are considered to be matched, and consulting objects such as "one year old", "four years old", "infant" are configured as entity information, and the entity information in the question / question is identified based on the NER model to match the entity information.

[0074] Optionally, the question-answer knowledge base may be implemented using an ES (Elastic search) library to improve the efficiency of question-answer matching.

[0075] In an optional implementation, for each question in the question-and-answer knowledge base, based on the entity information contained in the question and the entity information contained in the question, if it is determined that the first entity contained in the question corresponds one-to-one to the second entity contained in the question, and each first entity is the same as the corresponding second entity, then it is determined that the entity information of the question matches the entity information of the question.

[0076] In another optional implementation, considering that different users may call the same consulting object differently in actual applications, the same consulting object may have different entity information, and some entity information may be different, but the corresponding consulting objects are very different, and the same answers can be used for similar questions of these consulting objects. Therefore, one or more similar entity sets can be configured and stored on the server, each similar entity set contains at least two entity information, and all entity information in the same similar entity set belongs to similar entities.

[0077] In this step, the entity information contained in the question is matched with the entity information of the question in the question-answering knowledge base. This can be achieved in the following ways:

[0078] For each question in the question-answering knowledge base, based on the entity information contained in the question and the entity information contained in the question, if it is determined that the first entity contained in the question corresponds to the second entity contained in the question, and each first entity is the same as the corresponding second entity or belongs to the same set of similar entities, then it is determined that the entity information of the question matches the entity information of the question.

[0079] In this implementation, by configuring a similar entity set, when matching the entity information of the questions, if the two entities with a corresponding relationship in the two questions are consistent, then the two entities are confirmed to match; if the two entities with a corresponding relationship in the two questions are inconsistent, but belong to the same similar entity set, then the two entities can also be confirmed to match, which is closer to the user's usage habits in actual application scenarios and improves the accuracy of the output answer information.

[0080] Optionally, the semantic similarity between questions may be the Euclidean distance between sentence vectors corresponding to the questions, or other commonly used information that can measure the semantic similarity of questions, which is not specifically limited here.

[0081] In another optional implementation, the entity information contained in the input question constitutes an entity information set. For each question in the question-answering knowledge base, the entity information contained in the question constitutes another entity information set. According to the N-gram similarity between the entity information set corresponding to the question and the entity information set corresponding to each question, N can be set and adjusted according to the specific application scenario. If the N-gram similarity between the entity information set corresponding to the question and the entity information set corresponding to the question is greater than or equal to the third threshold, it is determined that the question matches the entity information of the input question. Among them, the third threshold can be set and adjusted according to the specific application scenario.

[0082] Step S304: Determine whether the first candidate question exists based on the matching result and the calculated semantic similarity.

[0083] The first candidate question sentence matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold. The first threshold can be set and adjusted according to the needs of the actual application scenario and is not specifically limited here.

[0084] In this step, based on the matching result and the calculated semantic similarity, it is determined whether there is a first candidate question sentence that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold.

[0085] If it is determined that there is a first candidate question sentence that matches the entity information of the question and has a semantic similarity with the question greater than or equal to the first threshold, steps S305-S306 are executed to directly output answer information.

[0086] If it is determined that there is no first candidate question sentence that matches the entity information of the question and has a semantic similarity with the question greater than or equal to the first threshold, step S307 is executed to additionally display recommended questions with a higher semantic similarity with the question input by the user while outputting the answer information.

[0087] Step S305: According to the semantic similarity between the first candidate question and the question, a preset answer of the first candidate question is selected as answer information of the question.

[0088] This step is implemented in the same manner as the above-mentioned step S205. For details, please refer to the relevant description of step S205, which will not be repeated here.

[0089] Step S306: output the answer information of the question.

[0090] Step S307: Determine whether there is a second candidate question sentence according to the calculated semantic similarity.

[0091] The semantic similarity between the second candidate question and the question is greater than or equal to a second threshold. The second threshold is less than the first threshold, and the second threshold can be set and adjusted according to the needs of the actual application scenario, and is not specifically limited here.

[0092] In the above step S304, if it is determined that there is no first candidate question sentence that matches the entity information of the question and whose semantic similarity with the question is greater than or equal to the first threshold, then in this step, it is determined whether there is a second candidate question sentence whose semantic similarity with the question is greater than or equal to the second threshold based on the calculated semantic similarity.

[0093] If it is determined that there is a second candidate question, steps S308-S309 are executed to output a preset answer of the second candidate question with a high semantic similarity to the input question, and at least one second candidate question is additionally displayed as a recommended question.

[0094] If it is determined that there is no second candidate question, it is determined that the answer information of the user input question cannot be found, and then step S311 is executed to output a rejection message. The rejection message is used to prompt the user that the answer information of the user input question cannot be found.

[0095] Step S308: Select a preset answer to the second candidate question as answer information of the question according to the semantic similarity between the second candidate question and the question.

[0096] If it is determined that there is a second candidate question whose semantic similarity with the question is greater than or equal to the second threshold, then in this step, based on the semantic similarity between the second candidate question and the question, a second candidate question with a higher semantic similarity with the question is selected, and the preset answer of the selected second candidate question is used as the answer information of the question.

[0097] Step S309: output the answer information of the question, and take at least one second candidate question sentence as a recommended question, and output the recommended question.

[0098] Since the semantic similarity between the second candidate question selected in step S308 and the question input by the user reaches the second threshold but does not reach the first threshold, the preset answer of the second candidate question may not be able to answer the question input by the user well. Therefore, in this embodiment, at least one second candidate question with a high semantic similarity to the question input by the user is selected as a recommended question, and the recommended questions are displayed while outputting the answer information. Among these recommended questions, there may be questions that the user really wants to ask, so that the user can select the recommended questions to ask, which can standardize the user's questions and solve the user's problems as much as possible, thereby improving the ability of the intelligent customer service robot to solve user problems.

[0099] Exemplarily, the recommended questions may be displayed in a clickable list, and the user may quickly submit the recommended questions to the server by clicking on the recommended questions to obtain preset answers to the recommended questions, thereby improving the convenience and efficiency of the question-and-answer system operation.

[0100] In addition, the recommended questions may be displayed in the area below the output answer information or in other areas that are convenient for users to browse and operate. The display position of the recommended questions is not specifically limited here.

[0101] For example, Figure 4 This is an example diagram of an additional display of recommended questions provided in this embodiment, such as Figure 4As shown, for the user's input question "for single use", the matched answer information is output, and a recommended question "Is it for single use? Dear" is additionally displayed below the output answer information. When the user clicks on the recommended question "Is it for single use? Dear", the recommended question is submitted to the server through the terminal. The server matches the preset answer to the recommended question in the question and answer knowledge base and outputs the preset answer. The questions and preset answers in the question and answer knowledge base can be obtained by mining the question and answer data of manual customer service and manually reviewing them, and they come from the answers of manual customer service to the question about the product.

[0102] Optionally, in this step, based on the semantic similarity between the second candidate question and the input question, at least one second candidate question with a higher semantic similarity may be selected as the recommended question.

[0103] In actual applications, the recommended questions to be displayed additionally cannot be increased indefinitely. Optionally, an upper limit on the number of recommended questions can be set, and the number of recommended questions selected is less than or equal to the upper limit.

[0104] In addition, if the differences between the recommendation questions are small, then multiple recommendation questions may be essentially different ways of asking the same question, such as Figure 5 As shown, the three additional recommended questions "What material is it made of?", "What material?", and "What material?" are three different ways of asking the same question, and essentially only cover one valid question.

[0105] In order to increase the coverage of valid questions as much as possible, the multiple recommended questions displayed additionally need to form a certain degree of difference, and it is expected that questions with similar questions will only appear once. Optionally, in this step, according to the semantic category of the second candidate question, a second candidate question can be selected from each semantic category as a representative question of the semantic category; according to the semantic similarity between each representative question and the question, at least one representative question is selected as a recommended question.

[0106] Exemplarily, according to the semantic similarity between each representative question and the question, at least one representative question with a high similarity may be selected as a recommended question.

[0107] By selecting a second candidate question as a representative question in each semantic category, selecting recommended questions from representative questions of different semantic categories, and displaying only one recommended question in each semantic category, the additionally displayed recommended questions have a certain degree of difference, thereby increasing the coverage of the recommended questions and the possibility that the recommended questions cover the questions that users really want to ask.

[0108] Step S310: If the user triggers any recommended question, the preset answer to the recommended question is output.

[0109] After the recommended questions are displayed, the user clicks on any of the recommended questions, and the triggered recommended question is submitted to the server. The server matches the preset answers to the recommended questions from the question-and-answer knowledge base based on the recommended questions triggered by the user, and outputs the preset answers to the recommended questions, thereby solving the user's questions as much as possible and improving the ability of the intelligent customer service robot to solve user problems.

[0110] In actual applications, when matching input questions with candidate questions in the question-answering knowledge base to determine the answer information, stronger constraints can indeed make the answer information more accurate, but this will inevitably reduce the generalization ability of the question-answering data in the question-answering knowledge base. In a more extreme case, a very accurate answer can be found by matching consumer questions with candidate questions in the question-answering knowledge base word by word, but such an accurate match is very rare, so the number of replies with this accurate answer is also very small.

[0111] In this embodiment, the threshold for semantic similarity matching is further refined, and two semantic similarity thresholds, a first threshold and a second threshold, are set. When a first candidate question that matches the entity of the input question and has a semantic similarity with the input question greater than the first threshold is matched, a preset answer to the first candidate question is directly output. When the first candidate question is not matched, and a second candidate question that has a semantic similarity with the input question greater than the second threshold is matched, the answer information of the second candidate question is output, and at least one second candidate question that may be used as a reference question is displayed in the form of a recommended question click list below the output answer information, expanding the scope of the review of high-quality question and answer pairs in the question and answer knowledge base, thereby improving the answer penetration rate of the question and answer system to reveal valid answers, thereby solving user problems as much as possible, reducing the number of failed question answers, and improving the ability of intelligent customer service robots to solve user problems.

[0112] Step S311: output recognition rejection information.

[0113] In this embodiment, if there is no first candidate question sentence in the question-answering knowledge base that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold, and there is no second candidate question sentence with a semantic similarity with the question greater than or equal to a second threshold, then it is determined that the answer information of the user input question cannot be found, and a rejection message is output. The rejection message is used to prompt the user that the answer information of the user input question has not been found.

[0114] This embodiment outputs answer information and displays additional recommended questions when the semantic similarity between the question input by the user and the candidate question in the question-answering knowledge base is high. If the output answer information fails to solve the user's problem, the user can click on the additional recommended question to standardize the question submitted by the user and obtain an accurate answer, thereby improving the ability of the intelligent customer service robot to solve user problems.

[0115] Figure 6 A flow chart of a method for automatically mining question and answer information provided for an exemplary embodiment of the present application. Based on any of the above method embodiments, in this embodiment, the server can mine question and answer pairs (sample questions and answer information) related to product usage through a question and answer pair mining algorithm based on historical data such as historical manual customer service conversation records and / or records of intelligent assistance recommending answers to manual customer service, identify the entity information contained in the sample questions through the NER model, cluster the sample questions according to the entity information, and cluster the sample questions with matching entity information into the same entity category to form multiple entity categories. For entity categories containing multiple sample questions, for sample questions in the same entity category, cluster the sample questions according to their semantic similarity to form one or more semantic categories, and output the sample sentences and answer information according to their classification for review (modification, deletion or adoption) by the product provider (merchant). For the sample sentences and corresponding answer information that have been reviewed and adopted as candidate questions and corresponding preset answers, the sample sentences and corresponding answer information, semantic categories, entity categories and other information are stored in the question and answer knowledge base, so as to realize the rapid configuration of questions and answers in the question and answer knowledge base, reduce the workload of users to manually configure question answers, and improve the efficiency of question and answer configuration.

[0116] like Figure 6 As shown, the specific steps of this method are as follows:

[0117] Step S601: Acquire conversation log data within a historical period, and filter out conversation logs with product objects.

[0118] The length of the historical period can be set and adjusted according to the needs of the actual application scenario and is not specifically limited here.

[0119] The conversation log data in the historical period may be manual conversation log data in the historical period, including the conversation log of manual customer service, specifically including questions raised by users, answers given by manual customer service to the questions of users, and the answers given by manual customer service to users may be answers generated by manual customer service based on their own knowledge, or answers recommended by the intelligent assistance module to the manual customer service, or answers determined by the manual customer service after modifying the answers recommended by the intelligent assistance module, etc. In this embodiment, the manual conversation log data in the historical period is screened, conversation logs with product objects are screened out, and conversation logs without product objects are filtered out, so that the question-answer pairs mined based on the screened conversation logs are of product granularity, and the questions and preset answers in the question-answer knowledge base are of product granularity, and the preset answers come from the replies of manual customer service, which are more concise and accurate than the general replies based on scenario configuration.

[0120] Optionally, the manual conversation log data may also be subjected to conventional data cleaning processing such as cleaning greetings and meaningless information.

[0121] In addition, in other embodiments, the conversation log data within the historical period may include non-artificial conversation log data with relatively accurate answers.

[0122] Step S602: Question-answer pair mining is performed on the conversation logs with product objects by using a question-answer pair mining algorithm to obtain sample questions with product objects and answer information of the sample questions.

[0123] Among them, the question-answer pair mining algorithm can be any existing algorithm implementation for mining question-answer pairs from historical conversation data, such as a question-answer pair mining algorithm based on machine reading comprehension (MRC), etc., and is not specifically limited here.

[0124] Optionally, after mining out question-answer pairs consisting of sample questions and answer information of the sample questions, sensitive information is identified on the question-answer pairs, and question-answer pairs containing sensitive information in the sample questions or answer information are filtered out, while question-answer pairs that do not contain sensitive information in either the sample questions or the answer information are retained.

[0125] Optionally, after mining out the question-answer pairs consisting of the sample questions and the answer information of the sample questions, the repeated contents in the question-answer pairs are filtered out.

[0126] Step S603: clustering the sample questions according to the entity information contained in the sample questions, so as to cluster the sample questions with matching entity information into the same entity category, thereby forming a plurality of entity categories.

[0127] After obtaining the sample questions and their answer information, the NER model is used to perform named entity recognition on the sample questions to identify the entity information contained in the sample questions.

[0128] Furthermore, based on the entity information contained in the sample questions, the sample questions are clustered (referred to as entity clustering in this application), and the sample questions with matching entity information are clustered into the same entity category to form multiple entity categories. Each entity category includes at least one sample question.

[0129] Step S604: clustering the sample questions in the same entity category according to the semantic similarity of the sample questions to form at least one semantic category.

[0130] After entity clustering is performed to determine multiple entity categories, if any entity category includes multiple sample questions, for the entity category including multiple sample questions, the sample questions in the same entity category are clustered according to the semantic similarity of the sample questions to form at least one semantic category, thereby clustering the multiple sample questions in the same entity category into one or more semantic categories, each semantic category including one or more sample questions with matching entity information and similar semantics.

[0131] For semantic categories including multiple sample questions, the entity information of multiple sample questions in each semantic category matches and has similar semantics, and unified answer information can be configured for each semantic category, thereby greatly reducing the number of answers that need to be reviewed and configured.

[0132] Step S605: Classify and output the sample question according to the entity category and semantic category of the sample question, and output at least one answer information corresponding to each semantic category.

[0133] After determining the entity category and semantic category of each sample question, the sample question is classified and output, and at least one answer information corresponding to each semantic category is output, so that the product provider can configure unified answer information for the sample questions of each semantic category.

[0134] For example, Figure 7 This is an example diagram of the clustering results of the semantic categories provided in this embodiment. Figure 7 Only three semantic categories are taken as examples to exemplify sample questions in the semantic categories and the configured unified preset answers.

[0135] The product provider may adopt (or adopt after modification) one of the at least one answer information corresponding to each semantic category output as the unified answer information for the sample question of the semantic category.

[0136] Optionally, one answer information may be selected from the answer information of the sample questions in each semantic category as the default preset answer corresponding to each semantic category, and when the sample questions are classified and output, the default preset answer corresponding to each semantic category is output. The product provider (merchant) may adopt (or adopt after modification) the default preset answer, or adopt (or adopt after modification) other answer information as the preset answer information corresponding to the semantic category.

[0137] Optionally, the server may use the backtest module to obtain the online predicted hit count of the sample question. For each semantic category, the answer information of a sample question may be selected as the default preset answer corresponding to the semantic category according to the online predicted hit count of the sample question in the semantic category; when the sample question is classified and output according to the entity category and semantic category of the sample question, the default preset answer corresponding to each semantic category is output.

[0138] Exemplarily, when the sample questions are classified and output according to their entity categories and semantic categories, the default preset answers corresponding to each semantic category are output, and other answer information of the semantic category is output at the same time, and the default preset answers in each semantic category are displayed first.

[0139] By estimating the number of online hits of sample questions, and based on the number of online hits of sample questions in the semantic category, the answer information of a sample question with a higher number of online hits (for example, the highest or second highest) is selected as the default preset answer corresponding to the semantic category. This allows product providers to give priority to answer information of sample questions that have a higher impact on online Q&A and are more popular as preset answers during the review, and to add these sample questions and preset answers to the Q&A knowledge base in a timely manner.

[0140] In addition, the selection of the default preset answer may also be selected based on the quality of the answer information, or may be randomly selected, or may be selected in other ways, which are not specifically limited here.

[0141] Optionally, in this step, the server may utilize the backtest module to obtain the estimated online hit counts of the sample questions, sort the semantic categories according to the entity categories and semantic categories of the sample questions and the estimated online hit counts of the sample questions in each semantic category, and classify and output the sample questions according to the sorting results.

[0142] Exemplarily, the online predicted hit count of the sample questions in each semantic category may be the sum or average of the online predicted hit counts of all the sample questions in each semantic category.

[0143] Optionally, the sample questions in the same semantic category may be sorted according to the online estimated hit counts and output according to the sorting results. In addition, the sample questions in the same semantic category may also be output in a random order, which is not specifically limited here.

[0144] The backtest module may estimate the number of online hits of the mined sample questions at intervals of a certain period of time (such as 1 day, several hours, several days, etc.) to determine the expected number of online hits of the sample questions.

[0145] By estimating the expected online hit times of sample questions, sorting the semantic categories and sample questions within the semantic categories according to the expected online hit times, and classifying and outputting the sample questions based on the sorting results, it is possible to prioritize sample questions and answer information that will have a greater impact and are more popular on online Q&A, so that product providers can prioritize the review of sample questions and answer information that will have a greater impact and are more popular on online Q&A, and promptly add these sample questions and preset answers to the Q&A knowledge base.

[0146] Furthermore, after the above step S605, the server may perform corresponding processing according to the review operation of the product provider on any sample question, so as to realize the review function of the product provider on the sample question and answer information.

[0147] Exemplarily, after the above step S605, the product provider may modify any answer information corresponding to any semantic category. The server may update the answer information according to the modification operation on any answer information corresponding to any semantic category. The updated answer information may be adopted by the product provider as the preset answer for the semantic category, which can reduce the workload of the user manually configuring the answer to the question and improve the efficiency of the configuration of the answer to the question.

[0148] For example, after the above step S605, the product provider can delete any sample question in any semantic category. The server deletes the sample question according to the deletion operation on any sample question in any semantic category, so as to delete unreasonable sample questions or sample questions that will no longer appear, thereby improving the validity of the question and answer data in the question and answer knowledge base.

[0149] Exemplarily, after the above step S605, the product provider can delete all sample sentences of any semantic category. The server can delete the sample questions of any semantic category according to the deletion operation of the semantic category, so as to quickly and batch delete unreasonable sample questions or sample questions that will no longer appear, thereby improving the validity of the question and answer data in the question and answer knowledge base.

[0150] Exemplarily, after the above step S605, the product provider can adopt any answer information corresponding to any semantic category, so that the adopted answer information is used as a unified preset answer for the sample questions of the semantic category. According to the adoption operation of any answer information corresponding to any semantic category, the server uses the adopted answer information as the preset answer for each sample question in the semantic category, and each sample question in the semantic category as a candidate question, and stores the sample questions and the preset answers of the sample questions, the entity information contained therein, and the semantic category in the question-answering knowledge base, so that the preset answers for the sample questions can be configured quickly and in batches, reducing the workload of users manually configuring the answers to questions and improving the efficiency of the configuration of the answers to questions.

[0151] The candidate questions and corresponding preset answers added to the question-and-answer knowledge base will be displayed through the entity information matching of the NER recognition results and the semantic similarity of the questions. Depending on the entity information matching and the semantic similarity threshold, online answer response operations such as direct answer display, additional display of recommended questions, and output of rejection information can be adopted.

[0152] In an optional embodiment, the question-answer information can be mined offline every once in a while based on the newly generated log data during this period, and candidate questions and their preset answers, entity information, and semantic category information can be added to the question-answer knowledge base. The above step S603 can be implemented in the following way:

[0153] The answer information and entity information contained in the sample questions mined this time are obtained. According to the entity information contained in the sample questions, the sample questions mined this time and the candidate questions added to the question-answering knowledge base are clustered together to cluster the sample questions with matching entity information into the same entity category to form multiple entity categories.

[0154] Furthermore, before the above step S605, if any sample question included in any semantic category is a third candidate question that has been added to the question-answering knowledge base (i.e., a question that has been reviewed), then the first question in the semantic category other than the third candidate question (a sample question that has not yet been added to the question-answering knowledge base) is added to the question-answering knowledge base as a candidate question, the first question is added to the semantic category where the third candidate question is located (the semantic category of the first question is updated), the preset answer of the third candidate question is stored in the question-answering knowledge base as the preset answer of the first question, and the entity information of the first question is stored in the question-answering knowledge base. The sample questions in this semantic category do not need to be reviewed by the product provider and can be directly added to the question-answering knowledge base as candidate questions, thereby further reducing the workload of users manually configuring question answers and improving the efficiency of question answer configuration.

[0155] In this embodiment, sample questions and answer information corresponding to the sample questions are mined from the conversation records between consumers and manual customer service, entity information contained in the sample questions is identified through the NER model, and the sample questions are clustered based on the contained entity information, and the sample questions with matching entity information are clustered into the same entity category; for multiple sample questions in the same entity category, semantic categories are formed by clustering based on the semantic similarity of the sample questions, so that the entity information of the sample sentences in each semantic category is consistent or very similar, and the semantic similarity of the sample sentences is high, the sample sentences in each semantic category can be answered using the same answer information, the sample sentences are classified and output, and the answer information corresponding to each semantic category is output, so that the product provider can configure a unified preset answer for the sample questions in each semantic category through simple modification and adoption operations, which can reduce the workload of users to manually configure question answers and improve the efficiency of question answer configuration, thereby solving the problem that a large number of product-related consulting questions require product providers to analyze the conversation consultation records and manually configure them themselves. At the same time, the questions mined from the conversation records of manual customer service are more granular, and the answer information is usually concise and clear, making it possible for the intelligent question-and-answer robot to automatically reply with direct answers such as "yes" and "suitable" that most consumers want to see. This eliminates the need for consumers to parse long general answers configured by product providers in the general knowledge base, greatly reducing the user's understanding cost.

[0156] By combining with any of the above method embodiments, a complete set of question and answer data is implemented, which is mined from the conversation records between consumers and manual customer service, and the question and answer data is classified and displayed on the review page for review by product providers. The reviewed sample questions and answer information, semantic categories and other information are then added to the knowledge question and answer library, and then the complete process is carried out online to match the refined answer information in the question and answer knowledge library according to the user input questions.

[0157] For example, Figure 8 The overall framework diagram of the question-answering system provided in the embodiment of the present application is as follows: Figure 8 As shown, the overall framework of the question-answering system can include an offline mining module and an online answer disclosure module.

[0158] Among them, the offline mining module specifically includes a question-answer pair mining unit, a filtering unit and a storage unit. The question-answer pair mining unit mines question-answer pairs on historical data based on the MRC question-answer pair mining algorithm to obtain multiple question-answer pairs, each of which contains sample questions and corresponding answer information. The filtering unit has the functions of NER recognition and text matching. It recognizes the entity information contained in the sample questions in the question-answer pair through the NER model, and performs the first clustering of the question-answer pair according to the entity information to form multiple entity categories; it performs text matching on multiple sample questions in each entity category, and performs the second clustering based on the semantic similarity between the sample questions to form one or more semantic categories. The storage unit is used to store the question-answer pair and the information of the entity category and semantic category of the question-answer pair. The question-answer pair will be classified and displayed on the front-end page for the product provider to adopt, modify, delete and other review operations on the question-answer pair, so as to realize the configuration of a unified preset answer corresponding to each semantic category. The reviewed sample questions, adopted preset answers and entity information are stored in the question-answer knowledge base. The question-and-answer knowledge base can be implemented using the ES (Elastic search) library to improve the efficiency of question-and-answer matching.

[0159] The online answer disclosure module specifically includes a refined sorting algorithm and an ES recall module. The ES recall module is used to recall candidate questions in the question-answering knowledge base according to the user's question. The refined sorting algorithm has the functions of NER recognition and semantic similarity matching, and can identify the entity information contained in the input question, and according to the matching of the entity information of the input question and the recalled candidate question and the size of the semantic similarity, it can directly output the answer, output the answer and add the recommended question, and reject the recognition. Specifically, if there is a first candidate question that matches the entity information of the input question and the semantic similarity is greater than or equal to the first threshold, the answer information of the first candidate question is directly disclosed. If there is no first candidate question that matches the entity information of the input question and the semantic similarity is greater than or equal to the first threshold, and there is a second candidate question that has a semantic similarity greater than or equal to the second threshold with the input question, the answer information of the second candidate question is disclosed, and at least one second candidate question is displayed in addition for the user to click to trigger. If there is no first candidate question sentence that matches the entity information of the input question and has a semantic similarity greater than or equal to a first threshold, and there is no second candidate question sentence that has a semantic similarity greater than or equal to a second threshold, then a rejection message is output.

[0160] In this embodiment, sample questions and answer information corresponding to the sample questions are mined from the conversation records between consumers and manual customer service, and the sample questions are clustered by combining entity information matching and semantic similarity matching, and the sample sentences are classified and output, and the answer information corresponding to each semantic category is output, so that the product provider can configure the unified preset answer of the sample questions in each semantic category through simple modification and adoption operations, which can reduce the workload of users to manually configure the answers to questions and improve the efficiency of question answer configuration, thereby solving the problem that a large number of product-related consulting questions require product providers to analyze the conversation consultation records and manually configure them. At the same time, the questions mined based on the conversation records of manual customer service are more granular, and the answer information is usually concise and clear, making it possible for the intelligent question-answering robot to automatically reply to the direct answers that most consumers want to see, such as "yes" and "suitable", so that consumers do not need to parse the long paragraphs of general answers configured by the product provider in the general knowledge base, which greatly reduces the user's understanding cost. When answering questions online, combining the strong constraint of directly outputting answers through entity recognition results with the weak constraint of online triggering through the semantic similarity of questions can ensure the correctness of the consumer's consultation object, while the problem-solving ability of the question-answering system can be increased by additionally displaying recommended questions.

[0161] Fig. 9 This is a schematic diagram of the structure of a question-answering device provided by an exemplary embodiment of the present application. The device provided by this embodiment is applied to the above-mentioned server and may be an intelligent question-answering robot, an intelligent question-answering system, etc. Fig. 9 As shown, the question-answering device 90 includes: a question acquisition module 91, an entity recognition module 92, a matching module 93, an answer determination module 94 and an output module 95.

[0162] The question acquisition module 91 is used to acquire the user's question.

[0163] The entity recognition module 92 is used to perform named entity recognition on the question and determine the entity information contained in the question.

[0164] The matching module 93 is used to match the entity information contained in the question with the entity information of the question in the question-answering knowledge base, and calculate the semantic similarity between the question and the question in the question-answering knowledge base; based on the matching result and the calculated semantic similarity, determine a first candidate question that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold.

[0165] The answer determination module 94 is used to select a preset answer to the first candidate question as answer information of the question according to the semantic similarity between the first candidate question and the question.

[0166] The output module 95 is used to output the answer information of the question.

[0167] The device provided in this embodiment can be specifically used to perform the above Figure 2 The solutions, specific functions and technical effects that can be achieved by the corresponding method embodiments are not described in detail here.

[0168] In an optional embodiment, the matching module further includes:

[0169] If the first candidate question does not exist in the question-answering knowledge base, a second candidate question in the question-answering knowledge base is obtained, and the semantic similarity between the second candidate question and the question is greater than or equal to a second threshold.

[0170] The answer determination module is also used to select a preset answer to a second candidate question as answer information of the question according to the semantic similarity between the second candidate question and the question.

[0171] The output module is also used to: output answer information of the question, and take at least one second candidate question as a recommended question, and output the recommended question.

[0172] In an optional embodiment, after taking at least one second candidate question as a recommendation question and outputting the recommendation question, the output module is further configured to:

[0173] If the user triggers any recommended question, the preset answer to the recommended question is output.

[0174] In an optional embodiment, when implementing taking at least one second candidate question as a recommendation question, the output module is further configured to:

[0175] According to the semantic category of the second candidate question, a second candidate question is selected from each semantic category as a representative question of the semantic category; according to the semantic similarity between each representative question and the question, at least one representative question is selected as a recommended question.

[0176] In an optional embodiment, when matching the entity information contained in the question with the entity information of the question sentence in the question-answering knowledge base, the matching module is further used to:

[0177] For each question in the question-answering knowledge base, based on the entity information contained in the question and the entity information contained in the question, if it is determined that the first entity contained in the question corresponds to the second entity contained in the question, and each first entity is the same as the corresponding second entity or belongs to the same set of similar entities, then it is determined that the entity information of the question matches the entity information of the question.

[0178] In an optional embodiment, Fig.10 As shown, the question-answering device also includes:

[0179] The offline mining module 96 is used to: obtain the conversation log data within the historical period and filter out the conversation logs with product objects; perform question-answer pair mining on the conversation logs with product objects through a question-answer pair mining algorithm to obtain sample questions with product objects and answer information of the sample questions; cluster the sample questions according to the entity information contained in the sample questions, so as to cluster the sample questions with matching entity information into the same entity category to form multiple entity categories; cluster the sample questions in the same entity category according to the semantic similarity of the sample questions to form at least one semantic category; classify and output the sample questions according to the entity category and semantic category of the sample questions, and output at least one answer information corresponding to each semantic category.

[0180] In an optional embodiment, after classifying and outputting the sample questions according to the entity categories and semantic categories of the sample questions and outputting at least one answer information corresponding to each semantic category, the offline mining module is further used to perform at least one of the following processes:

[0181] According to a modification operation on any answer information corresponding to any semantic category, the answer information is updated;

[0182] According to a deletion operation on any sample question in any semantic category, the sample question is deleted;

[0183] According to the deletion operation of any semantic category, the sample questions of the semantic category are deleted;

[0184] According to the adoption operation of any answer information corresponding to any semantic category, the adopted answer information is used as the preset answer for each sample question in the semantic category, and each sample question in the semantic category is used as a candidate question. The sample questions and the preset answers to the sample questions, the entity information contained therein and the semantic category to which they belong are stored in the question-answering knowledge base.

[0185] In an optional embodiment, when clustering the sample questions according to the entity information contained in the sample questions to cluster the sample questions with matching entity information into the same entity category to form multiple entity categories, the offline mining module is further used to:

[0186] The answer information and entity information contained in the sample question are obtained; based on the entity information contained in the sample question, the sample question mined this time and the candidate question that has been added to the question-answering knowledge base are clustered together to cluster the questions with matching entity information into the same entity category to form multiple entity categories.

[0187] Before classifying and outputting the sample questions according to the entity categories and semantic categories of the sample questions and outputting at least one answer information corresponding to each semantic category, the offline mining module is further used to:

[0188] If any sample question included in any semantic category is a third candidate question that has been added to the question-answering knowledge base, then the first question in the semantic category except the third candidate question is added as a candidate question to the question-answering knowledge base, the first question is added to the semantic category where the third candidate question is located, the preset answer of the third candidate question is stored in the question-answering knowledge base as the preset answer of the first question, and the entity information of the first question is stored in the question-answering knowledge base.

[0189] In an optional embodiment, when implementing classification and output of sample questions according to entity categories and semantic categories of the sample questions, the offline mining module is further used to:

[0190] Obtain the estimated online hit count of the sample questions; for each semantic category, select the answer information of a sample question as the default preset answer corresponding to the semantic category according to the estimated online hit count of the sample questions in the semantic category, and sort the semantic categories according to the entity category and semantic category of the sample questions, as well as the estimated online hit count of the sample questions in each semantic category; classify and output the sample questions according to the sorting results, and output the default preset answer corresponding to each semantic category.

[0191] The device provided in this embodiment can be specifically used to execute the solution provided by any of the above method embodiments, and the specific functions and technical effects that can be achieved are not repeated here.

[0192] Fig.11 This is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. Fig.11 As shown, the electronic device 110 includes: a processor 1101, and a memory 1102 communicatively connected to the processor 1101, and the memory 1102 stores computer-executable instructions.

[0193] Among them, the processor executes the computer execution instructions stored in the memory to implement the solution provided by any of the above method embodiments, and the specific functions and technical effects that can be achieved are not repeated here.

[0194] An embodiment of the present application also provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement the solution provided by any of the above method embodiments. The specific functions and technical effects that can be achieved are not repeated here.

[0195] An embodiment of the present application also provides a computer program product, which includes: a computer program, which is stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above method embodiments. The specific functions and technical effects that can be achieved are not repeated here.

[0196] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and are only used to distinguish between different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit "first" and "second" to different types. The meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0197] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0198] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A question-answering method, characterized in that: include: Get the user's question; Performing named entity recognition on the question to determine entity information contained in the question; Matching the entity information contained in the question with the entity information of the question in the question-answering knowledge base, and calculating the semantic similarity between the question and the question in the question-answering knowledge base; Determine, according to the matching result and the calculated semantic similarity, a first candidate question sentence that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold; According to the semantic similarity between the first candidate question and the question, selecting a preset answer to the first candidate question as answer information to the question; Output the answer information of the question; If the first candidate question does not exist in the question-answering knowledge base, obtaining a second candidate question in the question-answering knowledge base, wherein the semantic similarity between the second candidate question and the question is greater than or equal to a second threshold; According to the semantic similarity between the second candidate question and the question, selecting a preset answer to the second candidate question as answer information to the question; Outputting answer information of the question, and taking at least one second candidate question sentence as a recommended question, and outputting the recommended question for selection by the user; The taking at least one of the second candidate questions as a recommendation question includes: According to the semantic category of the second candidate question, selecting a second candidate question from each of the semantic categories as a representative question of the semantic category; At least one of the representative questions is selected as a recommended question according to the semantic similarity between each of the representative questions and the question.

2. The method according to claim 1, characterized in that After taking at least one of the second candidate questions as a recommended question and outputting the recommended question, the method further includes: If the user triggers any of the recommended questions, a preset answer to the recommended question is output.

3. The method according to claim 1 or 2, characterized in that: The matching of the entity information contained in the question with the entity information of the question sentence in the question-answering knowledge base includes: For each question in the question-and-answer knowledge base, based on the entity information contained in the question and the entity information contained in the question, if it is determined that the first entity contained in the question corresponds one-to-one to the second entity contained in the question, and each of the first entities is the same as the corresponding second entity or belongs to the same set of similar entities, then it is determined that the entity information of the question matches the entity information of the question.

4. The method according to claim 1 or 2, characterized in that: Also includes: Obtain the conversation log data within the historical period and filter out the conversation logs with product objects; Using a question-answer pair mining algorithm, the conversation log with the product object is mined for question-answer pairs to obtain sample questions with the product object and answer information of the sample questions; Clustering the sample questions according to the entity information contained in the sample questions, so as to cluster the sample questions with matching entity information into the same entity category to form multiple entity categories; Clustering the sample questions in the same entity category according to the semantic similarity of the sample questions to form at least one semantic category; According to the entity category and semantic category of the sample question, the sample question is classified and output, and at least one answer information corresponding to each semantic category is output.

5. The method according to claim 4, characterized in that After classifying and outputting the sample question according to the entity category and the semantic category of the sample question and outputting at least one answer information corresponding to each semantic category, the method further includes at least one of the following: According to a modification operation on any answer information corresponding to any semantic category, updating the answer information; According to a deletion operation on any sample question in any semantic category, the sample question is deleted; According to the deletion operation of any semantic category, the sample questions of the semantic category are deleted; According to the adoption operation of any answer information corresponding to any semantic category, the adopted answer information is used as the preset answer for each sample question in the semantic category, and each sample question in the semantic category is used as a candidate question. The sample questions and the preset answers to the sample questions, the entity information contained therein and the semantic category to which they belong are stored in the question-answering knowledge base.

6. The method according to claim 4, characterized in that The clustering of the sample questions according to the entity information contained in the sample questions, so as to cluster the sample questions with matching entity information into the same entity category to form a plurality of entity categories, includes: Obtain the answer information and entity information of the sample question; According to the entity information contained in the sample questions, the sample questions mined this time and the candidate questions added to the question-answering knowledge base are clustered together to cluster the questions with matching entity information into the same entity category to form multiple entity categories; Before classifying and outputting the sample question according to the entity category and the semantic category of the sample question and outputting at least one answer information corresponding to each semantic category, the method further includes: If any sample question included in any semantic category is a third candidate question that has been added to the question-answering knowledge base, then the first question in the semantic category except the third candidate question is added as a candidate question to the question-answering knowledge base, the first question is added to the semantic category where the third candidate question is located, the preset answer of the third candidate question is stored in the question-answering knowledge base as the preset answer of the first question, and the entity information of the first question is stored in the question-answering knowledge base.

7. The method according to claim 4, characterized in that The classifying and outputting the sample question according to the entity category and the semantic category of the sample question includes: Obtaining the estimated online hit count of the sample question; For each semantic category, according to the number of online predicted hits of the sample questions in the semantic category, select the answer information of a sample question as the default preset answer corresponding to the semantic category, and sort the semantic categories according to the entity category and semantic category of the sample question and the number of online predicted hits of the sample questions in each semantic category; The sample questions are classified and output according to the sorting results, and a default preset answer corresponding to each semantic category is output.

8. A question-answering device, characterized in that: include: Question acquisition module, used to obtain user questions; An entity recognition module, used to perform named entity recognition on the question to determine entity information contained in the question; A matching module, used to match the entity information contained in the question with the entity information of the question in the question-answering knowledge base, and calculate the semantic similarity between the question and the question in the question-answering knowledge base; according to the matching result and the calculated semantic similarity, determine a first candidate question that matches the entity information of the question and has a semantic similarity with the question greater than or equal to a first threshold; an answer determination module, configured to select a preset answer to the first candidate question as answer information of the question according to the semantic similarity between the first candidate question and the question; An output module, used for outputting answer information of the question; The matching module is further configured to obtain a second candidate question in the question-answering knowledge base if the first candidate question does not exist in the question-answering knowledge base, wherein the semantic similarity between the second candidate question and the question is greater than or equal to a second threshold; The answer determination module is further configured to select a preset answer to the second candidate question as answer information to the question according to the semantic similarity between the second candidate question and the question; The output module is also used to output the answer information of the question, and select a second candidate question from each semantic category as a representative question of the semantic category according to the semantic category of the second candidate question, select at least one representative question as a recommended question according to the semantic similarity between each representative question and the question, and output the recommended question for user selection.

9. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images