Intelligent question answering method, system, device and computer-readable storage medium

Abstract

The present application discloses an intelligent question-answering method, system, device and computer-readable storage medium, which obtains a target question to be answered; classifies the answer intent of the target question to obtain the intent classification result; if the intent classification result represents a corrective answer, the target question is corrected to obtain the target answer information; if the intent classification result represents a graph question and answer, the target question is searched based on a preset knowledge graph to obtain the target answer information; if the intent classification result represents a search question and answer, the target question is searched based on a preset question and answer library to obtain the target answer information. In the present application, the target question can be given a corrective answer, a knowledge graph search or a question and answer library search as needed, that is, a suitable search method can be selected to answer the target question with high accuracy. The intelligent question-answering system, device and computer-readable storage medium provided in the present application also solve the corresponding technical problems.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and more specifically, to an intelligent question-answering method, system, device, and computer-readable storage medium. Background Art

[0002] Human-computer interaction is a science that studies the interactive relationship between systems and users. Systems can be various machines or computerized systems and software. Intelligent question-answering systems are an artificial intelligence system developed based on human-computer interaction technology, such as intelligent customer service systems, voice control systems, etc.

[0003] The existing intelligent question-answering method is to judge the business relevance of the customer's consultation input, and call the business guidance module when it is found that the user input is not relevant to the business. When the business is relevant, the business content and activity content in the user input are obtained through the business identification module and the activity identification module respectively, and then the graph information of the knowledge semantic network is retrieved with reference to the knowledge base. The knowledge content obtained by the graph retrieval engine is organized and represented as business knowledge output, and the partial path loss is fed back to the business guidance through the loss prompt module for user reference. However, the accuracy of the intelligent question-answering results output by this method is not high, which affects the user experience.

[0004] In summary, how to improve the accuracy of intelligent question answering methods is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0005] The purpose of this application is to provide an intelligent question-answering method, which can solve the technical problem of how to improve the accuracy of the intelligent question-answering method to a certain extent. This application also provides an intelligent question-answering system, device and computer-readable storage medium.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] An intelligent question-answering method, comprising:

[0008] Get the target question to be answered;

[0009] Performing answer intent classification on the target question to obtain an intent classification result;

[0010] If the intention classification result represents a corrective answer, the target question is corrected to obtain target answer information;

[0011] If the intent classification result represents graph question answering, the target question is searched based on the preset knowledge graph to obtain target answer information;

[0012] If the intent classification result represents a retrieval question and answer, the target question is retrieved based on a preset question and answer library to obtain target answer information.

[0013] Preferably, the performing classification of the answer intent of the target question to obtain the intent classification result includes:

[0014] Processing the target problem to obtain a target processing problem;

[0015] Determine whether the target processing problem satisfies the preset rules;

[0016] If the target processing question does not satisfy the preset rule, determining that the intent classification result represents a corrective answer;

[0017] If the target processing problem meets the preset rules, the target processing problem is classified for answering intent based on the pre-trained model Albert and the bidirectional GRU model of the attention mechanism to obtain the intent classification result, and the intent classification result represents graph question answering or retrieval question answering.

[0018] Preferably, the searching the target question based on a preset knowledge graph to obtain target answer information includes:

[0019] Matching the target question based on the dictionary tree corresponding to the knowledge graph to obtain a matching result;

[0020] Classifying the matching results to obtain question-answer trigger words and question-answer intention words;

[0021] Matching the word set in the knowledge graph with the question and answer trigger words and the question and answer intention words to obtain an initial candidate word set;

[0022] Performing triple matching on the initial candidate word set to obtain a target candidate word set;

[0023] Performing graph matching on the target candidate word set to obtain initial answer information;

[0024] The target answer information is determined based on the initial answer information.

[0025] Preferably, determining the target answer information based on the initial answer information includes:

[0026] In the network diagram of the knowledge graph, performing path query retrieval on the initial answer information;

[0027] If no path corresponding to the initial answer information is retrieved, the initial answer information is corrected based on the specifications of the knowledge and relationships in the knowledge graph to obtain the target answer information;

[0028] If a path corresponding to the initial answer information is found, the initial answer information is used as the target answer information.

[0029] Preferably, the step of searching the target question based on a preset question-answer library to obtain target answer information includes:

[0030] Obtaining a keyword dictionary corresponding to the question and answer database;

[0031] Matching the target question based on the keyword dictionary;

[0032] If a target keyword is matched, the question-answer database is recalled based on the target keyword to obtain candidate answer information, and the candidate answer information is sorted according to the information length to obtain the target answer information;

[0033] If two or more target keywords are matched, the question-answer database is recalled based on the target keywords to obtain candidate answer information, and the similarity value between the target question and each candidate answer information is calculated, and the candidate answer information is sorted based on the similarity value to obtain the target answer information;

[0034] If the target keyword is not matched, semantic matching is performed on the target question based on the question-answer library to obtain the target answer information.

[0035] Preferably, the obtaining of a keyword dictionary corresponding to the question and answer library includes:

[0036] Performing word segmentation and part-of-speech tagging on the question-answer database to obtain a processing result;

[0037] Extracting keywords from the processing results to obtain key fragment words;

[0038] According to the preset keyword restriction rules, the key fragment words are merged to obtain the key phrase;

[0039] Calculating weight values ​​of the keyword phrases, and determining a keyword set based on the weight values;

[0040] Convert and expand the keyword set based on the synonym table to obtain the keyword dictionary;

[0041] Among them, the keyword limitation rules include: the token length of the phrase does not exceed the first preset value; and / or, the number of function words in the phrase does not exceed the second preset value; and / or, the tokens at both ends of the phrase are not function words and stop words; and / or, the number of stop words in the phrase does not exceed the third preset value; and / or, the MMR value is calculated by carrying repetition in the phrase; and / or, the phrase is a noun.

[0042] Preferably, performing semantic matching on the target question based on the question-answer database to obtain the target answer information includes:

[0043] Performing semantic matching on the target question based on the question-answer library through a semantic matching template to obtain the target answer information;

[0044] Among them, the basic model of the semantic matching template is the DSSM double-tower model; the semantic representation layer of the semantic matching template is a bidirectional GRU model that combines the pre-trained model Albert and the attention mechanism; the matching layer of the semantic matching template uses cosine similarity to calculate similarity.

[0045] An intelligent question-answering system, comprising:

[0046] A first acquisition module is used to acquire a target question to be answered;

[0047] The first classification module is used to classify the answer intent of the target question and obtain the intent classification result;

[0048] A first correction module, configured to correct the target question to obtain target answer information if the intention classification result represents a corrective answer;

[0049] A first retrieval module is used to retrieve the target question based on a preset knowledge graph to obtain target answer information if the intent classification result represents a graph question answer;

[0050] The second retrieval module is used to search the target question based on a preset question and answer library to obtain target answer information if the intention classification result represents a retrieval question and answer.

[0051] An intelligent question-answering device, comprising:

[0052] Memory for storing computer programs;

[0053] A processor is used to implement the steps of any of the above-mentioned intelligent question-answering methods when executing the computer program.

[0054] A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by a processor, implements the steps of any of the above-described intelligent question-answering methods.

[0055] The present application provides an intelligent question-and-answer method, which obtains a target question to be answered; classifies the answer intent of the target question to obtain the intent classification result; if the intent classification result represents a corrective answer, the target question is corrected to obtain the target answer information; if the intent classification result represents a graph question and answer, the target question is searched based on a preset knowledge graph to obtain the target answer information; if the intent classification result represents a retrieval question and answer, the target question is searched based on a preset question and answer library to obtain the target answer information. In the present application, the target question can be given a corrective answer, a knowledge graph search, or a question and answer library search as needed, that is, a suitable retrieval method can be selected to answer the target question with high accuracy. The present application provides an intelligent question-and-answer system, device, and computer-readable storage medium that also solve corresponding technical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0057] Figure 1 A first flow chart of an intelligent question-answering method provided in an embodiment of the present application;

[0058] Figure 2 A second flow chart of an intelligent question-answering method provided in an embodiment of the present application;

[0059] Figure 3 This is a schematic diagram of the operation of the BERT model;

[0060] Figure 4 This is a connection diagram of the BERT model;

[0061] Figure 5 A flowchart for retrieving the target question based on the preset knowledge graph in this application to obtain the target answer information;

[0062] Figure 6 A flowchart for searching the target question based on the preset question-answer library in this application to obtain the target answer information;

[0063] Figure 7 This is a schematic diagram of the DSSM double-tower model structure;

[0064] Figure 8 A flowchart of an intelligent question-answering method provided in an embodiment of the present application;

[0065] Fig. 9A schematic diagram of the structure of an intelligent question-answering system provided in an embodiment of the present application;

[0066] Fig.10 A schematic diagram of the structure of an intelligent question-answering device provided in an embodiment of the present application;

[0067] Fig.11 Another structural schematic diagram of an intelligent question-and-answer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0069] See also Figure 1 , Figure 1 A first flow chart of an intelligent question-answering method provided in an embodiment of the present application.

[0070] An intelligent question-answering method provided in an embodiment of the present application may include the following steps:

[0071] Step S101: Obtain a target question to be answered.

[0072] In practical applications, the target question to be answered can be obtained first. The type and content of the target question can be determined according to actual needs. For example, the target question can be a question in the tax field, a question in the automobile maintenance field, etc. This application does not make specific limitations here.

[0073] Step S102: classify the target question's answer intent to obtain an intent classification result.

[0074] In practical applications, after obtaining the target problem to be answered, the answer intent of the target problem can be classified to obtain the intent classification result, so as to determine the answer method for the target problem with the help of the intent classification result.

[0075] Step S103: If the intention classification result represents a corrective answer, the target question is corrected to obtain target answer information.

[0076] Step S104: If the intent classification result represents graph question and answer, the target question is searched based on the preset knowledge graph to obtain the target answer information.

[0077] Step S105: If the intent classification result represents a retrieval question and answer, the target question is searched based on a preset question and answer library to obtain target answer information.

[0078] In practical applications, after classifying the intent of answering the target question and obtaining the intent classification result, if the intent classification result represents a corrective answer, the target question can be corrected to obtain the target answer information; if the intent classification result represents a graph question and answer, the target question can be searched based on the preset knowledge graph to obtain the target answer information; if the intent classification result represents a search question and answer, the target question can be searched based on the preset question and answer library to obtain the target answer information. That is, the present application can comprehensively consider the correction method, knowledge graph search, and question and answer library search to answer the target question with good accuracy.

[0079] The present application provides an intelligent question-answering method, which obtains a target question to be answered; classifies the answer intent of the target question to obtain the intent classification result; if the intent classification result represents a corrective answer, the target question is corrected to obtain the target answer information; if the intent classification result represents a graph question and answer, the target question is searched based on a preset knowledge graph to obtain the target answer information; if the intent classification result represents a search question and answer, the target question is searched based on a preset question and answer library to obtain the target answer information. In the present application, the target question can be given a corrective answer, a knowledge graph search, or a question and answer library search as needed, that is, a suitable search method can be selected to answer the target question with high accuracy.

[0080] See also Figure 2 , Figure 2 A second flow chart of an intelligent question-answering method provided in an embodiment of the present application.

[0081] An intelligent question-answering method provided in an embodiment of the present application may include the following steps:

[0082] Step S201: Obtain a target question to be answered.

[0083] Step S202: Process the target problem to obtain a target processed problem.

[0084] In practical applications, when classifying the answer intent of the target question and obtaining the intent classification result, the target question can be processed first to obtain the target processed question. For example, the target question can be replaced with synonyms, splicing verification can be performed, non-standard colloquial questions can be converted, and words with splicing errors can be corrected, etc., so as to make the target question more accurate and facilitate the subsequent accurate classification of the answer intent of the target question.

[0085] Step S203: Determine whether the target processing problem meets the preset rules; if the target processing problem does not meet the preset rules, execute step S204; if the target processing problem meets the preset rules, execute step S205.

[0086] Step S204: Determine whether the intention classification result represents a corrective answer.

[0087] In practical applications, after processing the target problem and obtaining the target processed problem, because the answer content corresponding to the target problem satisfies the corresponding rules, we can first determine whether the target processed problem satisfies the preset rules. If the target processed problem does not satisfy the preset rules, we can determine that the intention classification result represents a corrective answer.

[0088] To facilitate understanding, assume that the target question in the tax field is "a self-employed business owner issues a special VAT invoice on his own", but in actual business scenarios, self-employed business owners do not have the authority to issue special invoices on their own. Therefore, this target question does not conform to normal tax logic, and a corrective answer is required for this target question. For example, the corresponding target answer information can be "You do not have the authority to issue a special VAT invoice".

[0089] Step S205: Based on the pre-trained model Albert and the bidirectional GRU model of the attention mechanism, the target processing question is classified by answering intent to obtain an intent classification result, which represents graph question answering or retrieval question answering.

[0090] In actual application scenarios, if the target processing problem meets the preset rules, the target problem can be searched through a knowledge graph or a question and answer library. However, the specific search method still needs to be determined at this time. To achieve this goal, the present application can classify the answer intent of the target processing problem based on the pre-trained model Albert and the bidirectional GRU model of the attention mechanism to obtain the intent classification result. The intent classification result represents the graph question and answer or the retrieval question and answer, that is, with the help of the pre-trained model Albert and the bidirectional GRU model of the attention mechanism, the intent classification result of the target problem is output as a graph question and answer or a retrieval question and answer.

[0091] It should be noted that in the training process of the pre-trained model Albert and the bidirectional GRU model of the attention mechanism, taking the tax field as an example, the public Q&A customer service data, question and answer pairs on the 12366 of the State Administration of Taxation, the search log data on the tax house website, the triple knowledge in the knowledge graph, including entities, attributes, events, etc., and other related tax field proprietary word lists, synonym lists, keyword lists, etc. are obtained, and the pre-trained model Albert and the bidirectional GRU model of the attention mechanism are trained based on the obtained data. Specifically, there are more than 30,000 public Q&A customer service data sets on 12366, and the data is relatively complete. It can be obtained through crawler technology and stored in the MySQL database, and this part of the data is used as a training set for semantic matching of the Q&A library retrieval, which belongs to semi-supervised learning. The 110,000 search log data on the tax website is the search questions raised by users. As a training data set for intent classification, different types of search sentences are divided into different answer methods. If the knowledge graph can be retrieved, the knowledge graph is used for accurate Q&A. If the graph cannot be used for accurate Q&A, it will enter the Q&A library for matching. The triplet knowledge of the knowledge graph is retrieved based on the entity and attribute content in the user's question. Other vocabulary tables are used as supplements and optimizations in the model training and system operation process. According to the test, the accuracy of the bidirectional GRU model based on the pre-trained model Albert and the attention mechanism is 96.50%. Compared with the existing intent classification model, the accuracy has increased by 8% and the recall rate has increased by 3.5%, which is a significant effect.

[0092] It should be noted that Albert is a variant of the BERT pre-training model. The following introduces the relevant content of the BERT pre-training model: BERT adopts the structure of the encoder in the Transformer. For the input of the model, BERT directly designed the data of the model in the pre-training stage. The first is to add a special character [CLS] at the beginning of each sequence. This character is mainly used to store the semantic information of the entire input sequence, similar to doc2vec, training a vector for the sentence separately. In this way, for some classification tasks, the output of CLS can be directly used for prediction. Another trick is to allow the model to recognize whether the input is one sentence or multiple sentences. The author used two methods. One is to use the special character "[SEP]" to separate the sentences, and the other is to add a sentence-level embedding to distinguish whether the vocabulary belongs to sentence A or sentence B. Therefore, the final input of each vocabulary should contain three parts of information, namely vocabulary embedding, sentence embedding, and position embedding. As shown in the following example: Figure 3As shown in Figure 1, the text semantic representation after embedding enters the bidirectional GRU and attention layers for further feature extraction and representation. Figure 4 As shown in the figure, the input layer is the embedding vector after Albert performs semantic representation, the hidden layer is Bigru, followed by the attention mechanism layer, and finally the fully connected dense layer inputs the probability of classification.

[0093] Step S206: If the intention classification result represents a corrective answer, the target question is corrected to obtain target answer information.

[0094] Step S207: If the intent classification result represents graph question and answer, the target question is searched based on the preset knowledge graph to obtain the target answer information.

[0095] Step S208: If the intent classification result represents a retrieval question and answer, the target question is searched based on a preset question and answer library to obtain target answer information.

[0096] See also Figure 5 , Figure 5 This is a flowchart for searching the target question based on the preset knowledge graph in this application to obtain the target answer information.

[0097] In an intelligent question-answering method provided in an embodiment of the present application, in the above embodiment, the process of searching the target question based on a preset knowledge graph and obtaining the target answer information may include the following steps:

[0098] Step S301: Match the target question based on the dictionary tree corresponding to the knowledge graph to obtain a matching result.

[0099] In practical applications, when searching for a target question based on a preset knowledge graph and obtaining target answer information, the target question can be matched based on the dictionary tree corresponding to the knowledge graph to obtain a matching result. Specifically, the dictionary tree corresponding to the knowledge graph can be composed of existing concept words, entity words, attribute words, relationship words, object words, event words, synonyms, aliases, etc. in the knowledge graph, and this application does not make specific limitations here.

[0100] Step S302: Classify the matching results to obtain question and answer trigger words and question and answer intention words.

[0101] In practical applications, after matching the target question based on the dictionary tree corresponding to the knowledge graph and obtaining the matching results, the matching results can be classified to obtain question-answering trigger words and question-answering intention words. Question-answering trigger words are words that represent the triggering of questions and answers, which can be the core concept words, entity words, and attribute words in the target question. Question-answering intention words represent what the target question is asking, whether it is asking about attributes, relationships, events, etc.

[0102] To facilitate understanding, assuming the target question is "I am a small-scale taxpayer, what VAT benefits should I enjoy?", the Q&A trigger words are "small-scale taxpayer" and "VAT", and the Q&A intention words are "benefits". Through the Q&A trigger words, we can determine that the question the user is asking is related to small-scale taxpayers and VAT, and through the Q&A intention words, we can determine that the question the user is asking is about VAT-related benefits for small-scale taxpayers.

[0103] Step S303: Match the word set in the knowledge graph with the question and answer trigger words and question and answer intention words to obtain an initial candidate word set.

[0104] Step S304: Perform triple matching on the initial candidate word set to obtain a target candidate word set.

[0105] Step S305: Perform graph matching on the target candidate word set to obtain initial answer information.

[0106] In practical applications, after classifying the matching results and obtaining the question-answering trigger words and question-answering intention words, three matching algorithms can be used to retrieve elements: subset matching, triple matching, and graph matching. Among them, subset matching is at the string level, matching the word set in the graph with the trigger word set in the user's search question to find the initial candidate word set; the candidate word set after subset matching is then matched at the triple level, that is, data containing entities and attributes, entities and events, entities and concepts, entities and relationships, etc. are matched with the candidate triples in the graph to further narrow the candidate word set and obtain the target candidate word set; after determining the result of the triple matching, the third step of graph matching is required. Graph matching is the knowledge of some specific rules. For example, whether an invoice can be issued, first of all, one must be qualified to receive the invoice. These two kinds of knowledge are in a sequential relationship, that is, first it is necessary to determine whether the invoice can be received, and then retrieve the invoice. After graph matching, the knowledge point corresponding to the question can be basically determined, that is, the initial answer information, to ensure the accuracy of the question-answering results.

[0107] Step S306: Determine target answer information based on the initial answer information.

[0108] In practical applications, after graph matching is performed on the target candidate word set to obtain initial answer information, the target answer information can be determined based on the initial answer information.

[0109] In specific application scenarios, the initial answer information may not conform to the specifications of knowledge and relationships in the knowledge graph. In this case, it is necessary to use path query retrieval to locate whether it is a problem with the knowledge construction itself or a problem with the user's search terms, and accordingly adjust the initial answer information to ensure the accuracy of the target answer information. That is, in the process of determining the target answer information based on the initial answer information, the initial answer information can be searched for by path query in the network diagram of the knowledge graph; if the path corresponding to the initial answer information is not retrieved, the initial answer information is corrected based on the specifications of knowledge and relationships in the knowledge graph to obtain the target answer information; if the path corresponding to the initial answer information is retrieved, the initial answer information is used as the target answer information. In this way, the verification quality of knowledge graph retrieval is controlled.

[0110] For ease of understanding, assume that the target question is "Small-scale taxpayers issue special VAT invoices", and the node of small-scale taxpayers is queried in the knowledge graph. Then the attributes of the node are queried, and it is found that small-scale taxpayers can only issue special VAT invoices on behalf of others (with the attribute of issuing special invoices on behalf of others), but do not have the attribute of issuing special invoices. Therefore, it will be judged based on the results that the answer to the user's question is not standardized, or the question is asked incorrectly. If it is not standardized, the question sentence will be corrected and recommended. For example, the above sentence will be corrected to small-scale taxpayers issuing special VAT invoices on behalf of others, or general taxpayers issuing special VAT invoices, because the general taxpayer node has the attribute of issuing special VAT invoices.

[0111] See also Figure 6 , Figure 6 This is a flowchart for searching the target question based on the preset question and answer library in this application to obtain the target answer information.

[0112] In an intelligent question-answering method provided in an embodiment of the present application, in the above embodiment, the process of searching the target question based on a preset question-answering library to obtain the target answer information may include the following steps:

[0113] Step S401: Obtain a keyword dictionary corresponding to the question and answer database.

[0114] In actual applications, when searching for a target question based on a preset question and answer library to obtain target answer information, a keyword dictionary corresponding to the question and answer library can be obtained first, so that the question and answer library can be searched for the target question based on the keyword dictionary later.

[0115] In a specific application scenario, in the process of obtaining the keyword dictionary corresponding to the question and answer library, the question and answer library can be segmented and POS tagged to obtain a processing result. Specifically, the segmentation and POS tagged can be performed based on the pkuseg tool; keywords are extracted from the processing result to obtain key fragment words. Specifically, tfidf can be used to calculate the keyword weight of the text, and the fragmented keywords are found through the keyword extraction algorithm; according to the preset keyword limitation rules, the key fragment words are merged to obtain keyword phrases; the weight values ​​of the keyword phrases are calculated, and the keyword set is determined based on the weight values. Specifically, a pre-trained LDA model can be used to calculate the topic probability distribution of the text and the topic probability distribution of each candidate phrase to obtain the final weight; the keyword set is transformed and expanded based on the synonym table to obtain a keyword dictionary; wherein the keyword limitation rules include: the token length of the phrase does not exceed a first preset value; and / or, the number of function words in the phrase does not exceed a second preset value; and / or, the tokens at both ends of the phrase It is not a function word or a stop word; and / or, the number of stop words in the phrase does not exceed a third preset value; and / or, the phrase carries a repetition value for calculating the MMR value; and / or, the phrase is a noun.

[0116] Step S402: Match the target question based on the keyword dictionary.

[0117] In actual applications, after obtaining the keyword dictionary corresponding to the question-answer library, the target question can be matched based on the keyword dictionary, so that the target question can be answered based on the matching result.

[0118] In specific application scenarios, in the process of matching the target question based on the keyword dictionary, if the target question completely matches the keyword dictionary, that is, only one target keyword is matched, then the matching result is matching one target keyword; if the target question does not completely match the keyword dictionary, for example, the target question includes a keyword in the keyword dictionary, but also includes other non-keyword parts, then the other parts can be segmented to remove stop words, and then the remaining parts can be matched with the keyword dictionary for word set matching, and the matching results are recalled to obtain other target keywords. At this time, the matching result is matching two or more target keywords; if the target question does not match the keyword dictionary at all, the matching result is that the target keyword is not matched.

[0119] Step S403: If a target keyword is matched, the question-answer database is recalled based on the target keyword to obtain candidate answer information, and the candidate answer information is sorted according to the information length to obtain the target answer information.

[0120] In practical applications, after matching the target question based on the keyword dictionary, if a target keyword is matched, the question-answer database can be recalled based on the target keyword to obtain candidate answer information, and the candidate answer information can be sorted according to the length of the information to obtain the target answer information.

[0121] Step S404: If two or more target keywords are matched, the question and answer database is recalled based on the target keywords to obtain candidate answer information, and the similarity value between the target question and each candidate answer information is calculated, and the candidate answer information is sorted based on the similarity value to obtain the target answer information.

[0122] In actual applications, after matching the target question based on the keyword dictionary, if two or more target keywords are matched, the question and answer database can be recalled based on the target keywords to obtain candidate answer information, and the similarity value between the target question and each candidate answer information can be calculated. The candidate answer information is sorted based on the similarity value to obtain the target answer information.

[0123] In specific application scenarios, in the process of calculating the similarity value between the target question and each candidate answer information, the similarity value between the target question and each candidate answer information can be calculated based on the Jaccard coefficient, and the candidate answer information can be sorted according to the size of the Jaccard coefficient. The larger the Jaccard coefficient, the higher the correlation, and the higher the ranking of the corresponding candidate answer information.

[0124] It should be noted that given two sets , , the Jaccard coefficient is defined as and The size of the intersection is and The ratio of the sizes of the unions is defined as follows:

[0125] ;

[0126] When the collection , When both are empty, Defined as 1.

[0127] The metric related to the Jaccard coefficient is called the Jaccard distance, which is used to describe the dissimilarity between sets. The larger the Jaccard distance, the lower the sample similarity. The formula is defined as follows:

[0128] ;

[0129] The symmetric difference is:

[0130] Step S405: If the target keyword is not matched, semantic matching is performed on the target question based on the question-answer database to obtain target answer information.

[0131] In actual applications, after matching the target question based on the keyword dictionary, if the target keyword is not matched, the target question is semantically matched based on the question and answer database to obtain the target answer information.

[0132] In specific application scenarios, after obtaining the target answer information, the target answer information can also be restricted by rules so that the target answer information meets the corresponding specification restrictions. For example, the target question is "Can small and micro-profit enterprises not enjoy corporate tax incentives?" and the sentence in the library "Can small and micro-profit enterprises enjoy corporate tax incentives?" has a high score in similarity calculation, but one is asking affirmatively and the other is asking negatively. It is necessary to strongly restrict the result rules and eliminate the results of this situation. For example, "What is the definition of a small and micro-profit enterprise?" and "What is the definition of a small and micro-enterprise?" are semantically close to small and micro-profit enterprises, and the similarity score will be relatively high, but "small and micro-profit enterprises" and "small and micro-enterprises" are not the same in terms of tax regulations. Therefore, it is necessary to eliminate such questions that do not comply with tax regulations but have a high similarity based on business.

[0133] In specific application scenarios, in the process of semantically matching the target question based on the question and answer library to obtain the target answer information, the semantic matching template can be used to semantically match the target question based on the question and answer library to obtain the target answer information; among them, the basic model of the semantic matching template is the DSSM double-tower model; the semantic representation layer of the semantic matching template is a bidirectional GRU model that combines the pre-trained model Albert and the attention mechanism; the matching layer of the semantic matching template uses cosine similarity to calculate the similarity, and ranks are sorted according to the similarity scores. The top3 recall accuracy of the matching model is 82.25%, and the top10 recall accuracy is 91.83%. The accuracy has increased by 6%, and the recall has increased by 2%, which meets the online requirements.

[0134] In specific application scenarios, when using cosine similarity to calculate similarity, the method of calculating the similarity scores of the original single question and multiple questions in the library (for example, 10,000 questions) one by one can be changed to using matrix operations to generate a numerical matrix for the user's question and a 10,000-dimensional matrix for the 10,000 questions in the library. The matrices are calculated simultaneously to generate a 10,000-dimensional result vector. Each value in the vector represents the similarity score calculated between the question and the sentence in the same position in the library. In this way, the original calculation time of 2 minutes is improved to 40ms to complete the entire calculation.

[0135] It should be noted that the principle of the DSSM model is relatively simple. Through the massive click exposure logs of the query and document in the search engine, the DNN deep network is used to express the query and document as low-dimensional semantic vectors, and the distance between the two semantic vectors is calculated by cosine similarity, and finally a semantic similarity model is trained. This model can be used to predict the semantic similarity of two sentences and obtain the low-dimensional semantic embedding vector of a sentence. The DSSM twin tower model structure is as follows Figure 7 As shown in the figure, Q represents query information, D represents document information; Term Vector: represents the embedding vector of the text; Word Hashing technology: to solve the problem of too large Term Vector, the dimension of the bag-of-word vector is reduced; Multi-layer nonlinear projection: represents the hidden layer of the deep learning network; Semantic feature: represents the final embedding vector of the query and document; Relevance measured by cosinesimilarity: represents the calculation of the cosine similarity between the query and the document; Posterior probabilitycomputed by softmax: represents the conversion of the semantic similarity between the query and the positive sample document into a posterior probability through the Softmax function. In addition, in order to increase the effect of the model, improve the accuracy of semantic matching, and make full use of existing data, because in the 12366 customer service data, there are two parts of corpus for user questions, one is the complete question content, which is a longer text; the other is the abbreviation of the question, as a descriptive short sentence; in order to use the above two parts of information, according to actual user feedback, in terms of process, you can also design a way to embed the two texts separately at the same time: one is descriptive phrase matching, and the other is question text matching, generating two embedding candidate sets, after semantically representing the questions asked by users in real time, semantically matching the two candidate sets respectively to calculate the cos similarity, and finally weighting the cos values ​​of the two parts, with initial weights of 0.6 and 0.4, such as Figure 8 As shown, the final comprehensive score is obtained, and then the comprehensive scores are sorted and the search results are returned.

[0136] See also Fig. 9 , Fig. 9 A schematic diagram of the structure of an intelligent question-answering system provided in an embodiment of the present application.

[0137] An intelligent question-answering system provided in an embodiment of the present application may include:

[0138] A first acquisition module 101 is used to acquire a target question to be answered;

[0139] The first classification module 102 is used to classify the answer intent of the target question and obtain the intent classification result;

[0140] A first correction module 103 is used to correct the target question and obtain target answer information if the intention classification result represents a corrective answer;

[0141] The first retrieval module 104 is used to retrieve the target question based on the preset knowledge graph to obtain the target answer information if the intent classification result represents the graph question answer;

[0142] The second retrieval module 105 is used to search the target question based on a preset question and answer library to obtain target answer information if the intention classification result represents a retrieval question and answer.

[0143] In an intelligent question-answering system provided in an embodiment of the present application, the first classification module may include:

[0144] A first processing unit is used to process the target problem to obtain a target processing problem;

[0145] The first judgment unit is used to judge whether the target processing problem meets the preset rules; if the target processing problem does not meet the preset rules, it is determined that the intent classification result represents the corrective answer; if the target processing problem meets the preset rules, the target processing problem is classified based on the pre-trained model Albert and the bidirectional GRU model of the attention mechanism to obtain the intent classification result, and the intent classification result represents the graph question and answer or the retrieval question and answer.

[0146] In an intelligent question-answering system provided in an embodiment of the present application, the first search module may include:

[0147] The first matching unit is used to match the target question based on the dictionary tree corresponding to the knowledge graph to obtain a matching result;

[0148] The first classification unit is used to classify the matching results to obtain question-answer trigger words and question-answer intention words;

[0149] The second matching unit is used to match the word set in the knowledge graph with the question and answer trigger words and question and answer intention words to obtain an initial candidate word set;

[0150] A third matching unit is used to perform triple matching on the initial candidate word set to obtain a target candidate word set;

[0151] The fourth matching unit is used to perform graph matching on the target candidate word set to obtain initial answer information;

[0152] The first determining unit is configured to determine target answer information based on the initial answer information.

[0153] In an intelligent question-answering system provided by an embodiment of the present application, the first determination unit can be specifically used to: perform path query retrieval on initial answer information in a network diagram of a knowledge graph; if a path corresponding to the initial answer information is not retrieved, correct the initial answer information based on the specifications of knowledge and relationships in the knowledge graph to obtain target answer information; if a path corresponding to the initial answer information is retrieved, use the initial answer information as the target answer information.

[0154] In an intelligent question-answering system provided in an embodiment of the present application, the second search module may include:

[0155] A first acquisition unit is used to acquire a keyword dictionary corresponding to the question and answer database;

[0156] The fifth matching unit is used to match the target question based on the keyword dictionary; if a target keyword is matched, the question and answer database is recalled based on the target keyword to obtain candidate answer information, and the candidate answer information is sorted according to the information length to obtain the target answer information; if two or more target keywords are matched, the question and answer database is recalled based on the target keyword to obtain candidate answer information, and the similarity value between the target question and each candidate answer information is calculated, and the candidate answer information is sorted based on the similarity value to obtain the target answer information; if the target keyword is not matched, the target question is semantically matched based on the question and answer database to obtain the target answer information.

[0157] In an intelligent question-and-answer system provided by an embodiment of the present application, a first acquisition unit can be specifically used to: perform word segmentation and part-of-speech tagging on a question-and-answer library to obtain a processing result; perform keyword extraction on the processing result to obtain key fragment words; fuse the key fragment words according to a preset keyword limitation rule to obtain a keyword phrase; calculate a weight value of the keyword phrase, and determine a keyword set based on the weight value; convert and expand the keyword set based on a synonym table to obtain a keyword dictionary; wherein the keyword limitation rule includes: the token length of the phrase does not exceed a first preset value; and / or, the number of function words in the phrase does not exceed a second preset value; and / or, the tokens at both ends of the phrase are not function words and stop words; and / or, the number of stop words in the phrase does not exceed a third preset value; and / or, the MMR value is calculated based on the repetition in the phrase; and / or, the phrase is a noun.

[0158] An intelligent question and answer system provided by an embodiment of the present application, the fifth matching unit can be specifically used to: perform semantic matching on a target question based on a question and answer library through a semantic matching template to obtain target answer information; wherein, the basic model of the semantic matching template is a DSSM double-tower model; the semantic representation layer of the semantic matching template is a bidirectional GRU model that combines the pre-trained model Albert and the attention mechanism; the matching layer of the semantic matching template uses cosine similarity to calculate similarity.

[0159] The present application also provides an intelligent question-answering device and a computer-readable storage medium, both of which have the corresponding effects of the intelligent question-answering method provided in the embodiments of the present application. Fig.10 , Fig.10 A schematic diagram of the structure of an intelligent question-answering device provided in an embodiment of the present application.

[0160] An intelligent question-answering device provided in an embodiment of the present application includes a memory 201 and a processor 202. The memory 201 stores a computer program. When the processor 202 executes the computer program, the steps of the intelligent question-answering method described in any of the above embodiments are implemented.

[0161] See also Fig.11 , another intelligent question-and-answer device provided in the embodiment of the present application may also include: an input port 203 connected to the processor 202, used to transmit commands input from the outside to the processor 202; a display unit 204 connected to the processor 202, used to display the processing results of the processor 202 to the outside; a communication module 205 connected to the processor 202, used to realize the communication between the intelligent question-and-answer device and the outside. The display unit 204 can be a display panel, a laser scanning display, etc.; the communication mode adopted by the communication module 205 includes but is not limited to mobile high-definition link technology (HML), universal serial bus (USB), high-definition multimedia interface (HDMI), wireless connection: wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, and communication technology based on IEEE802.11s.

[0162] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the intelligent question-answering method described in any of the above embodiments are implemented.

[0163] The computer-readable storage medium involved in the present application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.

[0164] For the description of the relevant parts of the intelligent question-answering system, device, and computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the intelligent question-answering method provided in the embodiments of the present application, which will not be repeated here. In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0165] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0166] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent question-answering method, characterized in that: include: Get the target question to be answered; Performing answer intent classification on the target question to obtain an intent classification result; If the intention classification result represents a corrective answer, the target question is corrected to obtain target answer information; If the intent classification result represents graph question answering, the target question is searched based on the preset knowledge graph to obtain target answer information; If the intent classification result represents a search question and answer, the target question is searched based on a preset question and answer library to obtain target answer information; The step of searching the target question based on the preset knowledge graph to obtain target answer information includes: Matching the target question based on the dictionary tree corresponding to the knowledge graph to obtain a matching result; Classifying the matching results to obtain question-answer trigger words and question-answer intention words; Matching the word set in the knowledge graph with the question and answer trigger words and the question and answer intention words to obtain an initial candidate word set; Performing triple matching on the initial candidate word set to obtain a target candidate word set; Performing graph matching on the target candidate word set to obtain initial answer information; Determining the target answer information based on the initial answer information; Wherein, determining the target answer information based on the initial answer information includes: In the network diagram of the knowledge graph, performing path query retrieval on the initial answer information; If no path corresponding to the initial answer information is retrieved, the initial answer information is corrected based on the specifications of the knowledge and relationships in the knowledge graph to obtain the target answer information; If a path corresponding to the initial answer information is retrieved, the initial answer information is used as the target answer information; The step of searching the target question based on a preset question-answer database to obtain target answer information includes: Obtaining a keyword dictionary corresponding to the question and answer database; Matching the target question based on the keyword dictionary; If a target keyword is matched, the question-answer database is recalled based on the target keyword to obtain candidate answer information, and the candidate answer information is sorted according to the information length to obtain the target answer information; If two or more target keywords are matched, the question-answer database is recalled based on the target keywords to obtain candidate answer information, and the similarity value between the target question and each candidate answer information is calculated, and the candidate answer information is sorted based on the similarity value to obtain the target answer information; If the target keyword is not matched, semantic matching is performed on the target question based on the question-answer database to obtain the target answer information; The step of obtaining a keyword dictionary corresponding to the question and answer library includes: Performing word segmentation and part-of-speech tagging on the question-answer database to obtain a processing result; Extracting keywords from the processing results to obtain key fragment words; According to the preset keyword restriction rules, the key fragment words are merged to obtain the keyword phrase; Calculating weight values ​​of the keyword phrases, and determining a keyword set based on the weight values; Convert and expand the keyword set based on the synonym table to obtain the keyword dictionary; The keyword restriction rules include: the token length of the phrase does not exceed the first preset value; and / or, the number of function words in the phrase does not exceed the second preset value; and / or, the tokens at both ends of the phrase are not function words and stop words; and / or, the number of stop words in the phrase does not exceed the third preset value; and / or, the MMR value is calculated by carrying repetition in the phrase; and / or, the phrase is a noun; The step of performing semantic matching on the target question based on the question-answer database to obtain the target answer information includes: Performing semantic matching on the target question based on the question-answer library through a semantic matching template to obtain the target answer information; Among them, the basic model of the semantic matching template is the DSSM double-tower model; the semantic representation layer of the semantic matching template is a bidirectional GRU model that combines the pre-trained model Albert and the attention mechanism; the matching layer of the semantic matching template uses cosine similarity to calculate similarity.

2. The method according to claim 1, characterized in that: The performing of answering intent classification on the target question to obtain an intent classification result includes: Processing the target problem to obtain a target processing problem; Determine whether the target processing problem satisfies the preset rules; If the target processing question does not satisfy the preset rule, determining that the intent classification result represents a corrective answer; If the target processing problem meets the preset rules, the target processing problem is classified for answering intent based on the pre-trained model Albert and the bidirectional GRU model of the attention mechanism to obtain the intent classification result, and the intent classification result represents graph question answering or retrieval question answering.

3. An intelligent question-answering system, characterized in that: include: A first acquisition module is used to acquire a target question to be answered; The first classification module is used to classify the answer intent of the target question and obtain the intent classification result; A first correction module, configured to correct the target question to obtain target answer information if the intention classification result represents a corrective answer; A first retrieval module is used to retrieve the target question based on a preset knowledge graph to obtain target answer information if the intent classification result represents a graph question answer; A second retrieval module is used to retrieve the target question based on a preset question and answer library to obtain target answer information if the intention classification result represents a retrieval question and answer; Wherein, the first retrieval module includes: The first matching unit is used to match the target question based on the dictionary tree corresponding to the knowledge graph to obtain a matching result; The first classification unit is used to classify the matching results to obtain question-answer trigger words and question-answer intention words; The second matching unit is used to match the word set in the knowledge graph with the question and answer trigger words and question and answer intention words to obtain an initial candidate word set; A third matching unit is used to perform triple matching on the initial candidate word set to obtain a target candidate word set; The fourth matching unit is used to perform graph matching on the target candidate word set to obtain initial answer information; A first determining unit, configured to determine target answer information based on the initial answer information; The first determination unit is used to: perform a path query and search for the initial answer information in the network diagram of the knowledge graph; if the path corresponding to the initial answer information is not retrieved, correct the initial answer information based on the specifications of the knowledge and relations in the knowledge graph to obtain the target answer information; if the path corresponding to the initial answer information is retrieved, use the initial answer information as the target answer information; Wherein, the second retrieval module includes: A first acquisition unit is used to acquire a keyword dictionary corresponding to the question and answer database; The fifth matching unit is used to match the target question based on the keyword dictionary; if a target keyword is matched, the question and answer database is recalled based on the target keyword to obtain candidate answer information, and the candidate answer information is sorted according to the information length to obtain the target answer information; if two or more target keywords are matched, the question and answer database is recalled based on the target keyword to obtain candidate answer information, and the similarity value between the target question and each candidate answer information is calculated, and the candidate answer information is sorted based on the similarity value to obtain the target answer information; if the target keyword is not matched, the target question is semantically matched based on the question and answer database to obtain the target answer information; Wherein, the first acquisition unit is used to: perform word segmentation and part-of-speech tagging on the question-answer library to obtain a processing result; perform keyword extraction on the processing result to obtain key fragment words; fuse the key fragment words according to the preset keyword limitation rules to obtain a keyword phrase; calculate the weight value of the keyword phrase, and determine the keyword set based on the weight value; convert and expand the keyword set based on the synonym table to obtain a keyword dictionary; wherein the keyword limitation rules include: the token length of the phrase does not exceed the first preset value; and / or, the number of function words in the phrase does not exceed the second preset value; and / or, the tokens at both ends of the phrase are not function words and stop words; and / or, the number of stop words in the phrase does not exceed the third preset value; and / or, the MMR value is calculated by carrying repetition in the phrase; and / or, the phrase is a noun; Among them, the fifth matching unit is used to: perform semantic matching on the target question based on the question and answer library through the semantic matching template to obtain the target answer information; wherein, the basic model of the semantic matching template is the DSSM double-tower model; the semantic representation layer of the semantic matching template is a bidirectional GRU model that combines the pre-trained model Albert and the attention mechanism; the matching layer of the semantic matching template uses cosine similarity to calculate the similarity.

4. An intelligent question-answering device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of the intelligent question-answering method as claimed in claim 1 or 2 when executing the computer program.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the intelligent question-answering method as claimed in claim 1 or 2 are implemented.

Images