Intelligent question answering method and device, equipment and storage medium

Abstract

The application provides a smart question answering method and device, equipment and a storage medium, which are suitable for the field of natural language processing. The method comprises the following steps: obtaining a user question; determining a target candidate business that meets the user question from a plurality of candidate businesses; inputting the user question into a multi-label model, and determining a target candidate intention from a plurality of candidate intentions according to an output result of the multi-label model; and taking target data corresponding to the target candidate business and the target candidate intention in a knowledge graph as a query result.

Description

Technical Field

[0001] This application relates to the field of natural language processing, and in particular to an intelligent question-answering method, apparatus, device, and storage medium. Background Technology

[0002] Knowledge graph-based intelligent question answering is a hot research area in the field of natural language processing.

[0003] Currently, knowledge graph-based intelligent question answering methods mainly include semantic parsing-based methods, information retrieval-based methods, and joint extraction-based methods. Intelligent question answering systems retrieve answers by querying pre-built knowledge graphs based on natural language questions input by users and using reasoning.

[0004] The methods described above all employ a single strategy to extract information from all valid entities and select the most likely candidate path from multiple candidate paths. However, due to the diversity of information contained in the problem, it is easy to find incorrect candidate paths, leading to erroneous final query results. Summary of the Invention

[0005] This application provides an intelligent question answering method, apparatus, device, and storage medium, which solves the problem that existing knowledge graph-based intelligent question answering methods are prone to errors in query results.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] Firstly, this application provides an intelligent question answering method. The method includes: acquiring a user question; determining a target candidate service that matches the user question from multiple candidate services; inputting the user question into a multi-label model, and determining a target candidate intent from multiple candidate intents based on the output of the multi-label model; and using the target data in the knowledge graph corresponding to the target candidate service and the target candidate intent as the query result.

[0008] Secondly, this application provides an intelligent question-answering device, which includes an acquisition module, a determination module, and a processing module. The acquisition module is used to acquire user questions; the determination module is used to determine target candidate services that match the user questions from multiple candidate services; the processing module is used to input the user questions into a multi-label model, and determine the target candidate intent from multiple candidate intents based on the output of the multi-label model; and to use the target data in the knowledge graph corresponding to the target candidate services and the target candidate intents as query results.

[0009] Thirdly, this application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intelligent question-answering method as described in the first aspect.

[0010] Fourthly, this application provides a storage medium storing a computer program that is loaded by a processor to execute the intelligent question-answering method as described in the first aspect.

[0011] The intelligent question-answering method, apparatus, device, and storage medium provided in this application, after obtaining a user's question, can first determine the target candidate service that matches the user's question from multiple candidate services; then, input the user's question into a multi-label model, and determine the target candidate intent from multiple candidate intents based on the output of the multi-label model; finally, use the target data in the knowledge graph corresponding to the target candidate service and the target candidate intent as the query result. Through this scheme, based on the natural language question given by the user, by understanding the entities, question intent, and candidate services contained in the natural language question, a more accurate answer can be retrieved from the knowledge graph, thereby improving the accuracy and recall rate of intelligent question-answering queries. Attached Figure Description

[0012] Figure 1 A flowchart illustrating the process of mining synonyms for a given business, provided as an embodiment of this application;

[0013] Figure 2 A schematic diagram illustrating the process of creating a multi-label model provided in an embodiment of this application;

[0014] Figure 3 Example diagram of the multi-label model provided in the embodiments of this application;

[0015] Figure 4 A flowchart illustrating the intelligent question-answering method provided in this application embodiment;

[0016] Figure 5 This is a schematic diagram of the structure of the intelligent question-answering device provided in the embodiments of this application;

[0017] Figure 6 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0018] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0019] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0020] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0021] In the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0022] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0023] The intelligent question-answering method provided in this application can be used in systems that interact with humans using natural language, such as automated question-answering systems, automated customer service systems, intelligent voice systems, and chatbot systems, based on knowledge base retrieval. Taking a mobile operator scenario as an example, data containing four categories—"question," "question intent," "related service," and "answer"—can be pre-collected.

[0024] Based on the answers users might want, the question intent is divided into 18 types, including: "Activation Method", "Cancellation Method", "Activation Conditions", "Cancellation Conditions", "Business Introduction", "Price", "Business Introduction", "Launch Time", "Usage Method", "Scope of Use", "Applicable Apps", "Data Usage", "Voice Duration", "Bandwidth", "Included Other Services", "Validity Period Rules", "Effectiveness Rules", "Stacking Rules", "Carry-over Rules", and "Capping Rules".

[0025] Based on the services and packages offered by the operator, all categories of the relevant services are listed. For example, the operator offers 131 services, including "Car Owner Services," "Voice Mail," "Nationwide Family Network," and "Enjoy Worry-Free Service Packages," among others.

[0026] Examples of the above four types of content are as follows:

[0027] Example 1

[0028] Question: Cancel the family number service

[0029] Question Intent: Cancellation Method

[0030] Business Area: Nationwide Family Network

[0031] Answer: Cancellation method_115

[0032] Example 2

[0033] Question: How much data does the unlimited data plan include, and how do I cancel it?

[0034] Question Intent: Cancellation Method | Traffic

[0035] Related Service: Enjoy Worry-Free Package

[0036] Answer: Cancellation method_17|30720

[0037] After completing the statistical analysis of the four categories of content—question, question intent, related business, and answer—synonyms for the related business can be pre-mined, and a multi-label model can be created. After the user enters a question, the business to which the question belongs can be predicted. Then, by querying tail entities related to the related business, the possible intent of the question can be limited. Finally, the question intent is generated based on the prediction results of the created multi-label model.

[0038] To illustrate the technical solution of this application, the following embodiments, in conjunction with the accompanying drawings, provide examples of mining synonyms for the relevant business, creating multi-label models, multi-strategy prediction, and knowledge graph querying.

[0039] I. Identify synonyms for the relevant business.

[0040] After a user inputs a question, the identification of the service to which the question belongs may contain issues such as aliases and typos. For example, for the "Beijing Mobile Plus Membership" service, the user question might mention "Plus," "puls," or "Member Plus," all containing typos. Similarly, for the "New Enjoy Data Coupon Activity" service, the user question might mention different terms such as "20 Yuan 20G," "Enjoy Coupon," or "Smart Data Coupon." To address this, this application provides a synonym mining algorithm that can automatically mine different expressions for the same service based on historical labeled data.

[0041] For certain business categories, we can first provide constraints including business attributes and aliases. For example, we can count synonyms for "Internet TV" as "mobile TV box," "set-top box," and "magic box," etc. Then, we can start synonym mining using the training set data, as follows:

[0042] S101. Obtain training set data.

[0043] The training set data mentioned above may include multiple statements, each of which is a user question.

[0044] For example, the training set data mentioned above can be a large number of natural language-based questions collected by the server based on big data, such as "How to upgrade the data package", "How to unsubscribe from the game service", "How to change the basic plan", etc. It can be understood that the more sentences included in the training set data, the more synonyms of the relevant business can be mined from the training set data.

[0045] S102. Divide each statement in the training set into multiple clauses according to the statement length.

[0046] For each statement in the training set data, the statement is split by length. For each statement, clauses are split according to a sliding window with a length ranging from 2 to the maximum statement length.

[0047] For example, for the user question "unsubscribe from message box", a set S = s1, s2, ..., s can be generated, consisting of multiple sentences with lengths ranging from 2 to 5. n This set can include: "Unsubscribe", "Leave a Subscription", "Message Box" with a length of 2; "Unsubscribe with a Subscription", "Leave a Subscription", "Message Box" with a length of 3; "Unsubscribe with a Subscription", "Leave a Subscription", "Message Box" with a length of 4; and "Unsubscribe with a Subscription" with a length of 5.

[0048] S103. Obtain multiple synonyms for the business to which each statement belongs.

[0049] The training data above will be pre-labeled with the business category to which each statement belongs. Assuming "unsubscribe message box" is labeled as belonging to the "voicemail" business, the thesaurus Y = y1, y2, ..., y... for "voicemail" will be obtained. m The thesaurus includes "voicemail", "mailbox" and "message service".

[0050] S104. If the target clause of the target statement meets the filtering criteria with the target synonym among the multiple synonyms of the business to which the target statement belongs, then the target clause is added to the multiple synonyms of the business to which the target statement belongs.

[0051] After obtaining sets S and Y, for each sub-sentence in set S, calculate the edit distance ratio with the synonyms in set Y. The formula for calculating the edit distance is as follows:

[0052]

[0053] Where i represents the length of the clause in set S, and j represents the length of the synonym in set Y.

[0054] If the length of the common substring between strings is 0, then lev a,b (|a|,|b|) equals the maximum of |a| and |b|. If the length of the common substring between the strings is not 0, proceed to the otherwise branch. The three formulas in the otherwise branch represent the insertion, deletion, and replacement operations for the corresponding strings, respectively, and each operation increases the distance by 1.

[0055] Since each clause has a different length, an edit distance ratio needs to be defined as a unified similarity metric. Assuming the edit distance ratio ranges from [0,1], the calculation formula is as follows:

[0056]

[0057] Among them, lev a,b (|a|,|b|) represents the edit distance, and max(|a|,|b|) represents the maximum value of the lengths of a and b.

[0058] After obtaining multiple edit distances, the edit distance with the largest ratio among the multiple edit distances is selected, as shown in the formula below:

[0059]

[0060] Suppose that the clause with the largest edit distance ratio in set S is "message box", and the edit distance ratio between "message box" and "message mailbox" is 0.86.

[0061] Furthermore, this application pre-sets a threshold, for example, threshold = 0.4. If the maximum edit distance ratio of the current word is greater than the threshold, and the current word is not in the thesaurus, then the current word is determined to be a new synonym. Therefore, the synonym "message box" is added to the thesaurus of the "voicemail" service.

[0062] It should be noted that the above synonym mining process is performed for each statement in the training set data until all statements have completed synonym mining, and finally the expanded synonym dictionary Y′ is obtained.

[0063] In this application embodiment, a novel synonym mining algorithm is proposed, which automatically discovers potential business synonyms based on sentence sliding window and edit distance to identify the business to which a sentence belongs. Compared with traditional classification algorithms, it is more interpretable and more accurate.

[0064] II. Creating a Multi-Label Model

[0065] Since a user question may belong to multiple intents, traditional text classification algorithms cannot solve the problem of intent recognition. This application treats the multi-label task as multiple binary classification tasks, with each binary classification model operating independently to determine whether the output matrix belongs to the current category.

[0066] This application improves the output layer of the multi-classification algorithm, making it suitable for multi-label models. For example... Figure 2 As shown, the process of creating a multi-label model is as follows:

[0067] S201. Obtain the preprocessed training data.

[0068] For example, such as Figure 3 The diagram shown is an example of a multi-label model provided in an embodiment of this application. The multi-label model may include: an encoding network, multiple fully connected layers connected to the encoding network, and a classifier (e.g., a Softmax classifier) ​​connected to each fully connected layer. A binary classification task includes: a fully connected layer and a classifier connected to that fully connected layer. One binary classification task corresponds to one label.

[0069] The encoding network can be used to encode the input text, generating a multi-dimensional sentence vector from the input text, which consists of questions from the training set.

[0070] Unlike traditional classification algorithms, which have a fixed number of fully connected layers, the number of fully connected layers in multi-label algorithms is determined by the number of labels. In the question intent recognition scenario for telecom operators, there are 18 intents, corresponding to 18 fully connected layers. Therefore, after obtaining the preprocessed training data, it is necessary to determine whether the number of fully connected layers is equal to 18. If it is equal to 18, then proceed with the following steps; if it is less than 18, then create a new connected layer and continue with the following steps.

[0071] S202. When the number of fully connected layers is equal to the preset number, the preprocessed training data is input into the classifier connected to each fully connected layer through the encoding network.

[0072] S203. Take the average of the errors of the outputs of each classifier as the target error.

[0073] Each fully connected layer is followed by a classifier, and each classifier's output dimension is 2, including one dimension indicating whether it belongs to the current intent and one dimension indicating whether it does not belong to the current intent. The output probabilities of the classifiers can be used to calculate the cross-entropy error. Specifically, the error between the classifier output and the manually labeled intent can be calculated, with 18 classifiers corresponding to 18 errors. Finally, the average of the 18 errors is calculated and used as the final error, i.e., the target error.

[0074] S204. If the target error is less than or equal to the preset error, save the multi-label model.

[0075] S205. If the target error is greater than the preset error, backpropagate the target error using manually labeled training data until the training of the multi-label model is completed.

[0076] In this embodiment, the output layer and error calculation method of the traditional multi-classification model are improved, making it applicable to multi-label classification algorithms and solving the problem of inaccurate and incomplete question intent recognition.

[0077] III. Multi-strategy prediction and knowledge graph query

[0078] After mining synonyms for the relevant business and creating multi-label models, users can input natural language questions via voice or text. For example, "Beijing Mobile Plus Membership" or "What is Kuaishou data traffic and how do I activate it?", which allows for multi-strategy prediction and knowledge graph queries.

[0079] like Figure 4 The diagram shown is a flowchart of an intelligent question-answering method provided in an embodiment of this application. The method includes steps S401 to S404 described below.

[0080] S401, Obtain the user's question.

[0081] S402. Identify the target candidate service that matches the user's question from multiple candidate services.

[0082] In multi-strategy prediction, it is necessary to first predict the business to which the question belongs, then limit the possible intent of the question by querying tail entities related to the business, and finally generate the question intent based on the prediction results of the multi-label model. The following example illustrates the reasoning question-answering of the user question q, "What is Kuaishou traffic and how do I activate it?"

[0083] First, obtain the expanded thesaurus Y′=y1′,y2′,...,y n Each of the candidate services determines whether any synonym of any segment of any word is contained in the user's sentence.

[0084] The first scenario is that if at least one synonym of a candidate service is included in the user's question, the candidate service is directly identified as the target candidate service.

[0085] The second scenario is that, if none of the synonyms of each candidate service are included in the user's question, the candidate service with the highest matching degree with the user's question is determined as the target candidate service based on the user's question and synonyms.

[0086] In the second scenario, if no match is found in the thesaurus for any of the business terms, fuzzy matching is performed using the edit distance ratio described in the previous embodiment. First, based on the formula for the edit distance ratio in the previous embodiment, the maximum and average edit distance ratios for each word in the sentence and the expanded thesaurus are calculated:

[0087]

[0088]

[0089] Next, the maximum and average values ​​are reconciled to ensure that the maximum value accounts for a larger proportion and the average value accounts for a smaller proportion, thereby obtaining the final matching degree between the question and each business function:

[0090] sim=λ×sim_max+(1-λ)×sim_avg(λ>0.5)

[0091] Next, the business with the highest sim score is selected as the final predicted business. For example, if a user's question 'q' cannot be completely matched with any business in the thesaurus, after calculating the fuzzy matching score, if the user's question 'q' has the highest matching score with 'Kuaishou Targeted Traffic Package', then the business to which the user's question 'q' belongs can be predicted to be 'Kuaishou Targeted Traffic Package'.

[0092] S403. Input the user's question into the multi-label model, and determine the target candidate intent from multiple candidate intents based on the output of the multi-label model.

[0093] For multi-label models, please refer to the above embodiments for a description of multi-label models.

[0094] Since the multi-label model corresponds to multiple labels, and each label corresponds to a candidate intent, the above S403 may specifically include: when the probability of the candidate intent corresponding to the label to which the user question belongs is greater than or equal to a preset probability, the candidate intent corresponding to the label to which the user question belongs is determined as the target candidate intent.

[0095] Optionally, a user's question may belong to one or more tags.

[0096] In one scenario, if the probability that a user's question belongs to a single candidate intent corresponding to a single tag is greater than or equal to a preset probability, then that single candidate intent corresponding to a single tag is identified as a target candidate intent. In another scenario, if the probability that a user's question belongs to multiple candidate intents corresponding to multiple tags is greater than or equal to a preset probability, then all candidate intents corresponding to multiple tags are identified as target candidate intents.

[0097] Continuing with the mobile operator scenario mentioned above, let's assume a preset probability of 0.5. After predicting that the user's question 'q' belongs to the "Kuaishou targeted data package" service, the question 'q' can be used as input to a multi-label model. The multi-label model outputs the probability of whether the current question belongs to each of the 18 intents. Then, the categories whose output of the multi-label model belongs to the category with a probability value greater than 0.5 are obtained as the final target candidate intents. For example, after the user's question 'q' is input into the multi-label model, the final target candidate intents obtained are "service introduction" and "activation method".

[0098] Furthermore, the intent is restricted by business logic. The knowledge graph is queried to determine if the entity "Kuaishou Targeted Traffic Package" contains the relationships "Business Introduction" and "Activation Method". If either of these relationships is not present, the current intent is removed from the candidate intents to ensure that the final query does not return no results. For example, consider the question "What is Kuaishou traffic, and how do I activate it?". The multi-strategy prediction determines the question intent as "Business Introduction | Activation Method" and the relevant business as "Kuaishou Targeted Traffic Package".

[0099] It should be noted that a knowledge graph can also be called a graph knowledge graph, a knowledge base, or any other possible name, and the embodiments of this application do not make specific limitations.

[0100] S404. Use the target data in the knowledge graph that corresponds to the target candidate business and the target candidate intent as the query results.

[0101] For user-input questions, multiple strategies are used for semantic understanding to determine target candidate businesses and target candidate intentions, and a structured query statement is generated; then, the final answer is derived by reasoning based on multiple entities and attributes retrieved from the knowledge graph.

[0102] Specifically, it includes the following steps 1 and 2.

[0103] Step 1: Generate a query statement that includes the target candidate business and the target candidate intent.

[0104] Typically, a user's question belongs to only one business function, but the question's intent may be one or more.

[0105] For example, if there is only one question intent, query statement 1 can be:

[0106] MATCH(n: business) - [r: intent] -> (p) WHERE n.name = "business" and r.name = "problem intent" RETURN p

[0107] In query statement 1, the question intent and the business to which it belongs can be replaced based on the actual query data.

[0108] For example, if there are two question intentions, query statement 2 can be:

[0109] MATCH(n: business) - [r: intent] -> (p) WHERE n.name = "business to which" and (r.name = "problem intent") Figure 1 "or r.name = "The problem means Figure 2 RETURN p

[0110] The problem in query statement 2 is... Figure 1 Problem Figure 2 The corresponding business function can be replaced based on the actual query data.

[0111] Step 2: Obtain the target data corresponding to the query statement from the knowledge graph, and use the target data as the query result.

[0112] Optionally, the knowledge graph mentioned above can be a Neo4J knowledge graph, and the Neo4J knowledge graph stores the original data in the format of triples. For example, <139 email, effective rule, effective rule_69>, <139 email, price, 20>, <5G family package, launch time, 20191031>, etc., with the left and right sides being the head entity and the tail entity, and the middle being the relationship between entities.

[0113] After generating the query statement, the query results can be retrieved from the Neo4J knowledge graph. For example, the Cypher query language can be used to perform queries within the Neo4J knowledge graph.

[0114] Optionally, the target data mentioned above can be a target identifier, which consists of at least one of letters, numbers, or symbols. The target identifier is used to indicate the query results. For example, the target data could be "Cancellation Method_17|30720" or "Cancellation Method_115". It is understood that using a target identifier to represent the query results can prevent the leakage of commercial information.

[0115] For example, let's take the question "What is Kuaishou traffic and how do I activate it?" as an example. If the intent of the question is determined to be "Business Introduction | Activation Method" and the relevant business is "Kuaishou Targeted Traffic Package" using the method described in the above embodiment, then a query statement containing "Business Introduction | Activation Method" and "Kuaishou Targeted Traffic Package" can be generated and queried in the Neo4J knowledge graph. The final query result is "Business Introduction_100|Activation Method_89".

[0116] The intelligent question-answering method provided in this application, after obtaining the user's question, first identifies the target candidate service that matches the user's question from multiple candidate services; then, it inputs the user's question into a multi-label model, and determines the target candidate intent from multiple candidate intents based on the output of the multi-label model; finally, it uses the target data in the knowledge graph corresponding to the target candidate service and the target candidate intent as the query result. Through this approach, based on the natural language question given by the user, and by understanding the entities, question intent, and candidate services contained in the natural language question, a more accurate answer can be retrieved from the knowledge graph, thereby improving the accuracy and recall rate of intelligent question-answering queries.

[0117] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0118] Corresponding to the intelligent question-answering method provided in the above embodiments, Figure 5 This is a structural block diagram of the intelligent question-answering device provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0119] like Figure 5 As shown in the figure, this application embodiment provides an intelligent question-answering device 50. The device includes an acquisition module 51, a determination module 52, and a processing module 53.

[0120] The acquisition module 51 is used to acquire user questions. The determination module 52 is used to determine the target candidate business that matches the user question from multiple candidate businesses. The processing module 53 is used to input the user question into a multi-label model, determine the target candidate intent from multiple candidate intents based on the output of the multi-label model, and use the target data in the knowledge graph corresponding to the target candidate business and the target candidate intent as the query result.

[0121] Optionally, the processing module 53 can be used to: determine a candidate service as the target candidate service when at least one synonym of the segmentation of a candidate service is included in the user question; or, when none of the synonyms of the segmentation of each candidate service are included in the user question, determine the candidate service with the highest matching degree with the user question as the target candidate service based on the user question and synonyms.

[0122] Optionally, the acquisition module 51 can also be used to acquire training set data before acquiring user questions. The processing module 53 can also be used to divide each statement in the training set data into multiple clauses according to the statement length. The acquisition module 51 can also be used to acquire multiple synonyms of the business to which each statement belongs. The processing module 53 can also be used to add the target clause to the multiple synonyms of the business to which the target statement belongs if the target clause of the target statement meets the filtering criteria.

[0123] Optionally, the multi-label model corresponds to multiple labels, with each label corresponding to a candidate intent. The processing module 53 can specifically be used to: determine the candidate intent corresponding to the label to which the user's question belongs as the target candidate intent if the probability of that candidate intent is greater than or equal to a preset probability.

[0124] Optionally, the multi-label model includes: an encoding network, multiple fully connected layers connected to the encoding network, and a classifier connected to each fully connected layer. Each fully connected layer corresponds to one label, and each classifier includes a dimension indicating whether the user belongs to the current intent and a dimension indicating whether the user does not belong to the current intent. The acquisition module 51 can also be used to acquire preprocessed training data before acquiring the user's question. The processing module 53 can also be used, if the number of fully connected layers equals a preset number, to input the preprocessed training data through the encoding network into the classifier connected to each fully connected layer; to use the average error of the output of each classifier as the target error; and to save the multi-label model if the target error is less than or equal to a preset error.

[0125] Optionally, the processing module 53 can also be used to backpropagate the target error using manually labeled training data when the target error is greater than the preset error, until the training of the multi-label model is completed.

[0126] Optionally, the processing module 53 can be used to generate a query statement that includes target candidate business and target candidate intent; obtain target data corresponding to the query statement from the knowledge graph; and use the target data as the query result.

[0127] Optionally, the target data is a target identifier, which consists of at least one of letters, numbers, or symbols, and can be used to indicate query results.

[0128] The intelligent question-answering device provided in this application, after acquiring a user's question, can first determine the target candidate service that matches the user's question from multiple candidate services; then, it inputs the user's question into a multi-label model, and determines the target candidate intent from multiple candidate intents based on the output of the multi-label model; finally, it uses the target data in the knowledge graph corresponding to the target candidate service and the target candidate intent as the query result. Through this scheme, based on the natural language question given by the user, and by understanding the entities, question intent, and candidate services contained in the natural language question, a more accurate answer can be retrieved from the knowledge graph, thereby improving the accuracy and recall rate of intelligent question-answering queries.

[0129] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0130] in addition, Figure 5 The device shown can be a software unit, hardware unit, or a combination of software and hardware built into an existing terminal device, or it can be integrated into the terminal device as an independent component, or it can exist as an independent terminal device.

[0131] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0132] Figure 6 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 6 As shown, the terminal device includes: at least one processor 60 ( Figure 6 (Only one is shown) a processor, a memory 61, and a computer program 62 stored in the memory 61 and executable on at least one processor 60, which executes the steps in any of the above method embodiments when the processor 60 executes the computer program 62.

[0133] The terminal device can be a desktop computer, laptop, handheld computer, or cloud server, etc. This terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 6 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0134] The processor 60 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0135] In some embodiments, memory 61 may be an internal storage unit of the terminal device, such as a hard disk or RAM. In other embodiments, memory 61 may be an external storage device of the terminal device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the terminal device. Furthermore, memory 61 may include both internal and external storage units of the terminal device. Memory 61 is used to store operating systems, applications, bootloaders, data, and other programs, such as program code for computer programs. Memory 61 can also be used to temporarily store data that has been output or will be output.

[0136] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps described in the various method embodiments above.

[0137] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0138] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0139] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0140] In the embodiments provided in this application, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0141] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An intelligent question-answering method, characterized in that, The method includes: Get user questions; Based on a thesaurus, a target candidate service matching the user's question is determined from multiple candidate services, wherein the thesaurus includes synonyms of the multiple candidate services; The user question is input into a multi-label model, and the target candidate intent is determined from multiple candidate intents based on the output of the multi-label model. The target data in the knowledge graph that corresponds to the target candidate business and the target candidate intent are used as the query results; The thesaurus is mined as follows: Training set data is acquired; each statement in the training set data is divided into multiple clauses according to its length to obtain a first set; multiple synonyms for the business to which each statement belongs are acquired to obtain a second set; for all clauses in the first set, the edit distance ratio with the synonyms in the second set is calculated; the maximum edit distance among multiple edit distance ratios is selected; if the maximum edit distance ratio of the target clause of the target statement is greater than a preset threshold, and the target clause is not in the thesaurus, then the target clause is added to the multiple synonyms of the business to which the target statement belongs. The step of determining the target candidate service that matches the user question from multiple candidate services includes: if at least one synonym of the segmentation of a candidate service is included in the user question, then the candidate service is determined as the target candidate service; or, if none of the synonyms of the segmentation of each candidate service are included in the user question, then the maximum and average values ​​of the edit distance ratio between the user question and each word in the thesaurus are determined, the proportions of the maximum and average values ​​are adjusted, the matching degree between the user question and each candidate service is obtained, and the candidate service with the highest matching degree with the user question is determined as the target candidate service, wherein the proportion of the maximum value is greater than the proportion of the average value.

2. The method according to claim 1, characterized in that, The edit distance ratio is determined by the following formula: ； ； Where i represents the length of the user question or the clause in the first set, and j represents the length of the synonyms in the thesaurus or the second set. Represents edit distance, This represents the maximum value of the lengths of a and b. This represents the edit distance ratio.

3. The method according to claim 1, characterized in that, The multi-label model corresponds to multiple labels, and each label corresponds to a candidate intent. The step of determining the target candidate intent from multiple candidate intents based on the output of the multi-label model includes: If the probability of the candidate intent corresponding to the tag to which the user question belongs is greater than or equal to a preset probability, the candidate intent corresponding to the tag to which the user question belongs is determined as the target candidate intent.

4. The method according to claim 1, characterized in that, The multi-label model includes: an encoding network, multiple fully connected layers connected to the encoding network, and a classifier connected to each fully connected layer; in the multi-label model, the number of fully connected layers is determined by the number of labels, one label corresponds to one candidate intent, and each classifier includes one dimension indicating that it belongs to the current intent and one dimension indicating that it does not belong to the current intent; Before obtaining the user's question, the method further includes: Obtain the preprocessed training data; If the number of fully connected layers is less than a preset number, connective layers are created such that the number of fully connected layers equals the number of labels; if the number of fully connected layers equals the preset number, the preprocessed training data is input into the classifier connected to each fully connected layer through the encoding network; the preset number is the number of labels. The average of the errors output by each classifier is used as the target error; If the target error is less than or equal to the preset error, the multi-label model is saved.

5. The method according to claim 4, characterized in that, The method further includes: If the target error is greater than the preset error, the target error is backpropagated using manually labeled training data until the training of the multi-label model is completed.

6. The method according to any one of claims 1 to 5, characterized in that, The step of using the target data in the knowledge graph corresponding to the target candidate service and the target candidate intent as the query result includes: Generate a query statement that includes the target candidate service and the target candidate intent; Obtain the target data corresponding to the query statement from the knowledge graph, and use the target data as the query result.

7. The method according to claim 6, characterized in that, The target data is a target identifier, which consists of at least one of letters, numbers, or symbols, and is used to indicate the query result.

8. An intelligent question-and-answer device, characterized in that, The device includes an acquisition module, a determination module, and a processing module; The acquisition module is used to acquire user questions; The determining module is configured to, based on a thesaurus, determine a candidate service as a target candidate service if at least one synonym of the segmented word of a candidate service is included in the user question; or, if none of the synonyms of the segmented word of each candidate service are included in the user question, determine the maximum and average values ​​of the edit distance ratio between the user question and each word in the thesaurus, adjust the proportion of the maximum and average values, obtain the matching degree between the user question and each candidate service, and determine the candidate service with the highest matching degree with the user question as the target candidate service, wherein the proportion of the maximum value is greater than the proportion of the average value; The processing module is used to input the user question into a multi-label model and determine the target candidate intent from multiple candidate intents based on the output of the multi-label model. And the target data in the knowledge graph corresponding to the target candidate business and the target candidate intent will be used as the query results; The thesaurus includes synonyms for the multiple candidate services; The thesaurus is mined as follows: Training set data is acquired; each statement in the training set data is divided into multiple clauses according to its length to obtain a first set; multiple synonyms for the business to which each statement belongs are acquired to obtain a second set; for all clauses in the first set, the edit distance ratio with the synonyms in the second set is calculated; the maximum edit distance among multiple edit distance ratios is selected; if the maximum edit distance ratio of the target clause of the target statement is greater than a preset threshold, and the target clause is not in the thesaurus, then the target clause is added to the multiple synonyms of the business to which the target statement belongs.

9. A terminal device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the intelligent question-answering method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a computer program, which is loaded by a processor to execute the intelligent question-answering method according to any one of claims 1 to 7.

Images