Answer determination method and apparatus, and storage medium

Abstract

The application discloses an answer determination method and device and a storage medium, relates to the technical field of artificial intelligence, and is applied to an intelligent customer service scene. The method comprises the following steps: determining a target theme tree corresponding to a problem to be replied; the target theme tree is constructed based on a target theme and subthemes corresponding to the target theme; determining a target standard question corresponding to the problem to be replied based on the similarity between a standard question represented by a leaf node in the target theme tree and the problem to be replied; determining a target standard answer corresponding to the target standard question based on a standard question and answer pair corresponding to the target standard question; and determining the target standard answer as a standard answer of the problem to be replied. The application can quickly and accurately determine a standard question corresponding to the problem to be replied.

Description

Answer determination method, device and storage medium Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus and storage medium for determining an answer. Background Technology

[0002] In the field of intelligent customer service question-and-answer technology, user questions are typically answered by similarity matching based on models. These models store a large number of questions, and for a standard question, a large number of similar questions are set. During the answer determination process, a large number of similar questions may be identified that match the user's question. These similar questions need to be sorted to determine the question with the highest similarity before the answer can be determined. On the one hand, the model requires a large amount of manually labeled data during training, resulting in high initial human and time costs. On the other hand, the model needs to sort numerous similar questions to obtain a matching answer. Clearly, existing technologies are slow and inefficient in determining the answer to a user's question, severely impacting the real-time performance of intelligent question-and-answer products.

[0003] Therefore, it is necessary to provide a method, apparatus, and storage medium for determining the answer, so as to quickly determine the standard question corresponding to the question to be answered. Summary of the Invention

[0004] This application provides a method, apparatus, and storage medium for determining answers, which can quickly determine the standard question corresponding to the question to be answered.

[0005] On the one hand, this application provides a method for determining an answer, the method comprising:

[0006] A target topic tree corresponding to the question to be answered is determined; the target topic tree is constructed based on the target topic and the subtopics corresponding to the target topic; the root node of the target topic tree represents the standard question-answer pair corresponding to the target topic, and the nodes in the target topic tree other than the root node represent the standard question-answer pairs corresponding to the subtopics; the branches of the target topic tree represent the relationship between different topics, and the standard question-answer pair includes a standard question and a standard answer;

[0007] Based on the similarity between the standard questions represented by the leaf nodes in the target topic tree and the question to be answered, the target standard question corresponding to the question to be answered is determined;

[0008] Based on the standard question-answer pairs corresponding to the target standard question, determine the target standard answer corresponding to the target standard question;

[0009] The target standard answer is determined as the standard answer to the question to be answered.

[0010] On the other hand, an answer-determining apparatus is provided, the apparatus comprising:

[0011] The target topic tree determination module is used to determine the target topic tree corresponding to the question to be answered; the target topic tree is constructed based on the target topic and the subtopics corresponding to the target topic; the root node of the target topic tree represents the standard question-answer pair corresponding to the target topic, and the nodes in the target topic tree other than the root node represent the standard question-answer pairs corresponding to the subtopics; the branches of the target topic tree represent the relationship between different topics, and the standard question-answer pair includes a standard question and a standard answer;

[0012] The target standard question determination module is used to determine the target standard question corresponding to the question to be answered based on the similarity between the standard questions represented by the leaf nodes in the target topic tree and the question to be answered.

[0013] The target standard answer determination module is used to determine the target standard answer corresponding to the target standard question based on the standard question-answer pair corresponding to the target standard question;

[0014] The answer determination module is used to determine the target standard answer as the standard answer to the question to be answered.

[0015] On the other hand, an answer determination device is provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the answer determination method as described above.

[0016] On the other hand, a computer storage medium is provided that stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the answer determination method as described above.

[0017] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, executes the computer instructions, and causes the computer device to perform the answer determination method as described above.

[0018] The answer determination method, apparatus, and storage medium provided in this application have the following technical advantages:

[0019] This application determines the target topic tree corresponding to the question to be answered; based on the similarity between the standard questions represented by the leaf nodes in the target topic tree and the question to be answered, it determines the target standard question corresponding to the question to be answered; based on the standard question-answer pairs corresponding to the target standard questions, it determines the target standard answer corresponding to the target standard questions; and it determines the target standard answer as the standard answer to the question to be answered. The target topic tree of this application includes multiple levels of topics, and each topic corresponds to a standard question-answer pair, thus enabling the matching of corresponding standard questions to the question to be answered and the determination of the standard answer. Since each topic in the target topic tree of this application corresponds to a standard question, and the similarity between the standard questions corresponding to different topics is very low and they are dissimilar questions, it is possible to quickly and accurately determine the standard question corresponding to the question to be answered. Attached Figure Description

[0020] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 is a schematic diagram of an answer determination system provided in an embodiment of this application;

[0022] Figure 2 is a flowchart illustrating an answer determination method provided in an embodiment of this application;

[0023] Figure 3 is a flowchart illustrating a method for determining the target standard question corresponding to a question to be answered, according to an embodiment of this application.

[0024] Figure 4 is a flowchart illustrating a method for determining the similarity between a standard question represented by a leaf node and a question to be answered, according to an embodiment of this application.

[0025] Figure 5 is a flowchart illustrating the method for determining the target standard question corresponding to the question to be answered when the similarity is less than or equal to the preset threshold, as provided in an embodiment of this application.

[0026] Figure 6 is a flowchart illustrating a method for updating a target topic tree according to an embodiment of this application;

[0027] Figure 7 is a flowchart illustrating a method for determining the target standard question corresponding to a question to be answered, provided in an embodiment of this application.

[0028] Figure 8 is a target topic tree corresponding to the subject tutoring application provided in the embodiments of this application;

[0029] Figure 9 is a schematic diagram of the user's real question corpus provided in the embodiments of this application;

[0030] Figure 10 is a schematic diagram of the training process of a similarity determination model provided in an embodiment of this application;

[0031] Figure 11 is a flowchart illustrating another method for determining the answer provided in an embodiment of this application;

[0032] Figure 12 is a schematic diagram of an answer determination device provided in an embodiment of this application;

[0033] Figure 13 is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0034] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, and mechatronics.

[0035] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0036] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, smart customer service, vehicle networking, and intelligent transportation. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0037] The solutions provided in this application relate to natural language processing technology in artificial intelligence. The technical solutions in this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0038] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0039] Please refer to Figure 1, which is a schematic diagram of an answer determination system provided in an embodiment of this application. As shown in Figure 1, the answer determination system may include at least a server 01 and a client 02.

[0040] Specifically, in this embodiment, server 01 may include a standalone server, a distributed server, or a server cluster composed of multiple servers. It may also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 01 may include network communication units, processors, and memory, etc. Specifically, server 01 can be used to determine the target topic tree corresponding to the question to be answered; based on the similarity between the standard questions represented by the leaf nodes in the target topic tree and the question to be answered, determine the target standard question corresponding to the question to be answered; based on the standard question-answer pair corresponding to the target standard question, determine the target standard answer corresponding to the target standard question; and determine the target standard answer as the standard answer to the question to be answered.

[0041] Specifically, in this embodiment, the client 02 may include physical devices such as smartphones, desktop computers, tablets, laptops, digital assistants, smart wearable devices, smart speakers, in-vehicle terminals, and smart TVs. It may also include software running on the physical device, such as web pages provided to users by service providers, or applications provided to users by those service providers. Specifically, the client 02 can be used to query standard answers to questions awaiting replies online.

[0042] The following describes a method for determining an answer according to this application. Figure 2 is a flowchart illustrating a method for determining an answer provided in an embodiment of this application. This specification provides the operational steps of the method as described in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many steps and does not represent the only execution order. In actual system or server product execution, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiments or figures. Specifically, as shown in Figure 2, the method may include:

[0043] S201: Determine the target topic tree corresponding to the question to be answered; the target topic tree is constructed based on the target topic and the subtopics corresponding to the target topic; the root node of the target topic tree represents the standard question-answer pair corresponding to the target topic, and the nodes in the target topic tree other than the root node represent the standard question-answer pairs corresponding to the subtopics; the branches of the target topic tree represent the relationship between different topics, and the standard question-answer pair includes a standard question and a standard answer.

[0044] In this embodiment, different target topic trees can be constructed in different technical fields and application scenarios. Each target topic tree corresponds to a broad target topic, and each target topic has at least one subtopic. Each subtopic can also have at least one corresponding subtopic. The root node of the target topic tree can use the target topic as its identifier, and then other nodes can be set according to the correspondence between the target topic and its subtopics. Each topic in the target topic tree corresponds to a standard question-answer pair, that is, each node is used to store a standard question-answer pair. The leaf nodes of the target topic tree can be one or more.

[0045] In this embodiment, a standard question refers to a question that corresponds to multiple similar questions but whose corresponding similar questions are not in the target topic tree; a standard answer refers to the unique answer corresponding to a standard question. The standard questions between different topics differ significantly and have low similarity; there are no similar standard questions in the target topic tree.

[0046] In this embodiment, a question topic framework incorporating domain expert knowledge is first constructed based on the experience of industry experts. Using deep learning and Natural Language Processing (NLP) techniques, user sample questions are matched with topics in the topic framework to obtain a large number of real question samples that conform to the framework. These question samples are then rewritten into standard sentences (standard questions). Standard sentences are the simplified versions of the question samples.

[0047] In this application embodiment, the method for determining the standard problem includes:

[0048] Identify the topics in the target topic tree and the relationships between them;

[0049] Determine the standard questions corresponding to each topic.

[0050] In this application embodiment, the corresponding target topic can be determined first based on the technical field to which the target topic tree belongs, and then the sub-topics can be determined based on the actual application scenario; in actual application, each sub-topic can be determined based on the collected user feedback issues.

[0051] In a specific embodiment, a target topic tree can be constructed for the subject tutoring application (App), as shown in Figure 8. Figure 8 is the target topic tree corresponding to the subject tutoring application. The target topic of this target topic tree is the subject tutoring application, and its corresponding sub-topics include learning status, reasons for dropping out of class, use of reading pen, express delivery and logistics, App user experience, course content issues, pronunciation issues, leave application issues, and parental accompaniment issues. Each sub-topic corresponds to the next level of topic. For example, express delivery and logistics corresponds to two sub-topics: not receiving the package and logistics information not being updated.

[0052] In this embodiment of the application, the standard questions for determining the corresponding topics may include:

[0053] Based on the historical dialogue data corresponding to each topic, obtain the original question set corresponding to each topic;

[0054] In this embodiment, meaningful corpus can be collected from users' real historical dialogue data. As shown in Figure 9, the corpus in chat scenarios is relatively colloquial, so it is necessary to remove idle chat corpus and extract as much corpus as possible from the corpus when users are actually asking customer service questions. That is, idle chat corpus and unrelated corpus in historical dialogue data are deleted to obtain the original question set.

[0055] Based on the original question set corresponding to each topic, the standard questions corresponding to each topic are determined.

[0056] In this embodiment of the application, a second similarity between any two original questions in the original question set can be calculated. If the second similarity is greater than a preset value, the two original questions are determined to be similar questions. By analogy, all similar questions corresponding to the same topic are obtained, and these similar questions are set to the same standard question, thereby converting multiple questions into one standard question. In the subsequent process of determining the matching of questions to be answered, the amount of calculation is reduced, so that the standard question corresponding to the question to be answered can be determined quickly.

[0057] S203: Based on the similarity between the standard questions represented by the leaf nodes in the target topic tree and the question to be answered, determine the target standard question corresponding to the question to be answered.

[0058] In this embodiment of the application, as shown in Figure 3, determining the target standard question corresponding to the question to be answered based on the similarity between the standard question represented by the leaf node in the target topic tree and the question to be answered includes:

[0059] S2031: Calculate the similarity between the standard questions represented by the leaf nodes in the target topic tree and the question to be answered;

[0060] In this embodiment of the application, as shown in Figure 4, calculating the similarity between the standard question represented by the leaf node in the target topic tree and the question to be answered includes:

[0061] S203101: Based on the similarity determination model, the standard question represented by the leaf node is converted into a first vector, and the question to be answered is converted into a second vector;

[0062] S203103: Based on the similarity determination model, calculate the vector similarity between the first vector and the second vector;

[0063] In some embodiments, the dot product between the first vector and the second vector can be used as the vector similarity.

[0064] S203105: Based on the similarity determination model, the vector similarity is determined as the similarity between the standard problem represented by the leaf node and the problem to be answered.

[0065] In this embodiment, a similarity determination model can be obtained by training the original question set based on a natural language processing (NLP) model. The NLP model can be the simBERT model, which is based on the BERT model and integrates text generation and text retrieval. As shown in Figure 10, the BERT model needs to be trained before constructing simBERT. The BERT model training process is supervised training, but it does not require manually labeled data; only user-generated chat corpora that have undergone simple processing are needed. The simBERT training process is supervised training, and the training corpus consists of a small number of pairs of similar sentences (i.e., similar question pairs in the original question set). The Seq2Seq part is constructed by generating similar sentences by predicting one sentence from another. Another output of the training process is a vector that can represent the input sentence, so this vector can be used to complete the retrieval task. A part of the simBERT model can be used as a text vector extraction model to extract the vectors of the user's question to be answered and the standard question, and then the similarity between the two vectors is used to determine the similarity between the questions.

[0066] In some embodiments, the similarity between the standard question represented by the leaf node in the target topic tree and the question to be answered can be determined according to the specific application scenario; for example, the similarity can be determined based on the keywords in the question. If the standard question represented by the leaf node in the target topic tree and the question to be answered include one or more of the same keywords, then the standard question represented by the leaf node in the target topic tree is determined as the target standard question of the question to be answered.

[0067] In this embodiment of the application, as shown in Figure 5, after calculating the similarity between the standard question represented by the leaf node in the target topic tree and the question to be answered, the method further includes:

[0068] S20321: If the similarity is less than or equal to a preset threshold, the parent node corresponding to the leaf node shall be taken as the current node;

[0069] In this embodiment, a preset threshold for similarity can be determined based on the actual application scenario. Based on this threshold, similar questions corresponding to the question to be answered can be identified. During the question matching process, since the topics corresponding to leaf nodes represent the most detailed topics, matching begins with the leaf nodes at the bottom of the target topic tree. If no match is found, matching proceeds upwards layer by layer. This ensures rapid matching of the standard question corresponding to the question to be answered and improves the accuracy of the matched standard questions, thereby increasing the accuracy of the standard answer to the question to be answered.

[0070] S20323: Calculate the similarity between the standard problem represented by the current node and the problem to be answered, and obtain the current similarity;

[0071] In this embodiment of the application, after calculating the similarity between the standard question represented by the current node and the question to be answered, and obtaining the current similarity, the method further includes:

[0072] If the current similarity is greater than the preset threshold, the standard question represented by the current node is determined as the target standard question corresponding to the question to be answered.

[0073] In this embodiment of the application, when the lowest-level subtopic in the target topic tree cannot match the standard answer, the parent topic of each subtopic can continue to be matched, thereby matching layer by layer upwards until the standard answer is matched.

[0074] In this embodiment of the application, if there are multiple current nodes, and if the current similarity is greater than the preset threshold, determining the standard question represented by the current node as the target standard question corresponding to the question to be answered includes:

[0075] The current node whose current similarity is greater than the preset threshold is determined as a candidate current node;

[0076] If there are at least two candidate current nodes, sort the at least two candidate current nodes based on the similarity corresponding to each candidate current node to obtain the sorting result of the candidate current nodes;

[0077] Based on the sorting results of the candidate current nodes, the target current node is determined;

[0078] The standard problem represented by the current target node is determined as the target standard problem corresponding to the problem to be answered.

[0079] In this embodiment of the application, the above-mentioned sorting of at least two candidate current nodes based on the similarity corresponding to each candidate current node to obtain the sorting result of the candidate current nodes includes:

[0080] Sort at least two candidate current nodes in descending order of similarity to obtain the sorting result;

[0081] In this embodiment of the application, determining the target current node based on the sorting results of the candidate current nodes may include:

[0082] The candidate current node that ranks first in the sorting is determined as the target current node.

[0083] In this embodiment, candidate current nodes can also be sorted in ascending order of similarity; however, this application does not limit the specific sorting method. The target current node determined in this embodiment is the candidate current node with the highest similarity.

[0084] S20325: If the current similarity is less than or equal to the preset threshold, determine whether the current node has a parent node;

[0085] S20327: If the current node has a parent node, the parent node is used as the current node again, and the calculation of the similarity between the standard problem represented by the current node and the problem to be answered is repeated to obtain the current similarity, until the step of using the parent node as the current node again if the current node has a parent node is repeated.

[0086] In this embodiment of the application, if there are multiple leaf nodes, it is necessary to determine whether the standard question corresponding to the parent node of each leaf node matches the question to be answered; if the similarity between the standard question corresponding to the leaf node and the question to be answered is low, the similarity between the standard question of the parent node and the question to be answered is calculated; the standard answer is found by matching upwards layer by layer.

[0087] In this embodiment of the application, if the current node does not have a parent node, the method further includes:

[0088] Send a standard answer retrieval request to the terminal; the standard answer retrieval request carries the question to be answered;

[0089] The terminal sends a question-and-answer pair to be answered, constructed based on the standard answer retrieval request; the question-and-answer pair to be answered carries the question to be answered and the corresponding target standard answer.

[0090] Update the target topic tree based on the question-and-answer pairs to be answered.

[0091] In this embodiment, if the current node has no parent node, it means that the current node is the root node and cannot continue traversing to the next level. In this case, it means that all nodes in the target topic tree do not match the question to be answered, that is, there is no answer to the question to be answered in the target topic tree. At this time, an answer retrieval request can be sent to the terminal. The user can input the corresponding standard answer to construct the target topic tree and update the target topic tree to facilitate the next matching.

[0092] In this embodiment of the application, as shown in FIG6, updating the target topic tree based on the question-and-answer pair to be answered includes:

[0093] S601: Determine the target subtopic corresponding to the question-and-answer pair to be answered;

[0094] S603: In the target topic tree, determine the target parent topic corresponding to the target subtopic;

[0095] S605: Under the node corresponding to the target parent topic, add a target node corresponding to the target subtopic; the target node represents the question-and-answer pair to be answered corresponding to the target subtopic.

[0096] In this embodiment of the application, if there is no matching answer in the leaf node, the answer can be manually set and the target topic tree can be updated to add the corresponding node.

[0097] S2033: If the similarity is greater than a preset threshold, the standard problem represented by the leaf node is determined as the target standard problem corresponding to the problem to be answered.

[0098] In this embodiment of the application, when the target topic tree has a two-layer structure, that is, the target topic tree only includes a root node and leaf nodes; if the similarity is less than or equal to a preset threshold, the above method further includes:

[0099] S2035: Calculate the first similarity between the standard problem represented by the root node and the problem to be answered;

[0100] S2037: If the first similarity is greater than a preset threshold, the standard problem represented by the root node is determined as the target standard problem corresponding to the problem to be answered.

[0101] In this embodiment of the application, if the first similarity is less than or equal to a preset threshold, it is determined that there is no standard answer in the target topic tree that matches the question to be answered.

[0102] In this embodiment of the application, when the first similarity is less than or equal to a preset threshold, the user can be prompted to update the target topic tree. Specifically, a new standard question-answer pair can be constructed based on the question to be answered, thereby improving the matching accuracy of the target topic tree.

[0103] In this embodiment of the application, as shown in FIG7, there are at least two leaf nodes. The step of determining the target standard question corresponding to the question to be answered based on the similarity between the standard question represented by the leaf node in the target topic tree and the question to be answered includes:

[0104] S20301: Calculate the similarity between the standard question represented by each leaf node in the target topic tree and the question to be answered, and obtain the question similarity corresponding to each leaf node;

[0105] In this embodiment of the application, the similarity between two questions can be calculated after the question is converted into a vector.

[0106] S20303: Based on the question similarity corresponding to each leaf node, determine candidate leaf nodes whose question similarity is greater than a preset threshold;

[0107] S20305: If there are at least two candidate leaf nodes, sort the at least two candidate leaf nodes based on the question similarity corresponding to each candidate leaf node to obtain the sorting result;

[0108] In this embodiment of the application, if there is only one candidate leaf node, the standard problem represented by the candidate leaf node is determined as the target standard problem corresponding to the problem to be answered.

[0109] S20307: Based on the sorting results, determine the target leaf node;

[0110] S20309: The standard problem represented by the target leaf node is determined as the target standard problem corresponding to the problem to be answered.

[0111] In this embodiment of the application, the candidate leaf node with the highest similarity can be determined as the target leaf node.

[0112] S205: Based on the standard question-answer pair corresponding to the target standard question, determine the target standard answer corresponding to the target standard question.

[0113] In this embodiment of the application, the corresponding standard question-answer pair can be determined based on the target standard question, thereby determining the target standard answer.

[0114] S207: The target standard answer is determined as the standard answer to the question to be answered.

[0115] In this embodiment of the application, the target standard question is the question with the highest similarity to the question to be answered, and the standard answer corresponding to the target standard question is the answer corresponding to the question to be answered.

[0116] In a specific embodiment, as shown in Figure 11, which is a flowchart of an answer determination method, including:

[0117] S1101: Collect real user data and remove casual conversation data; that is, delete casual conversation data and unrelated data from the user's historical dialogue data to obtain the original question set.

[0118] S1103: Train a natural language processing model based on the original question set to obtain a similarity determination model;

[0119] S1105: Construct a user question topic framework and standard question-answer pairs (including standard sentences, i.e. standard questions) under each topic; the question topic framework is the target topic tree;

[0120] S1107: Input the standard sentences under each topic into the similarity determination model to obtain the first vector;

[0121] S1109: Input the user's question into the similarity determination model to obtain the second vector;

[0122] S11011: Based on the first vector and the second vector, calculate the similarity between the standard sentences and user questions under each topic, and determine whether the similarity is greater than the threshold.

[0123] S11013: If the similarity of each standard sentence is greater than the threshold, sort the standard sentences that are greater than the threshold according to their similarity values, and determine the standard sentence with the highest similarity as the standard question corresponding to the user's question, thereby further determining the standard answer;

[0124] S11015: If the similarity of each standard sentence is less than or equal to the threshold, the user question is determined to be an issue not covered in the topic framework.

[0125] S11017: Determine the topic categories corresponding to the uncovered questions, construct user input instructions based on the framework, determine the corresponding standard questions and standard answers, and update the topic framework.

[0126] Specifically, in this embodiment of the application, if no answer matching the user's question is found in the topic framework, relevant prompts can be displayed to remind the framework builder to update the topic framework.

[0127] It is understood that in the specific implementation of this application, user information such as user chat data is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0128] As can be seen from the technical solutions provided in the above embodiments of this application, the embodiments of this application determine the target topic tree corresponding to the question to be answered; based on the similarity between the standard questions represented by the leaf nodes in the target topic tree and the question to be answered, the target standard question corresponding to the question to be answered is determined; based on the standard question-answer pair corresponding to the target standard question, the target standard answer corresponding to the target standard question is determined; and the target standard answer is determined as the standard answer to the question to be answered. The target topic tree of this application includes multiple levels of topics, and each topic corresponds to a standard question-answer pair, thereby matching the corresponding standard question to the question to be answered and determining the standard answer. Since each topic in the target topic tree of this application corresponds to a standard question, and the similarity between the standard questions corresponding to each topic is very low and they are dissimilar questions, the standard question corresponding to the question to be answered can be determined quickly and accurately.

[0129] This application embodiment also provides an answer determination device, as shown in FIG12, the device comprising:

[0130] The target topic tree determination module 1210 is used to determine the target topic tree corresponding to the question to be answered; the target topic tree is constructed based on the target topic and the subtopics corresponding to the target topic; the root node of the target topic tree represents the standard question-answer pair corresponding to the target topic, and the nodes in the target topic tree other than the root node represent the standard question-answer pairs corresponding to the subtopics; the branches of the target topic tree represent the relationship between different topics, and the standard question-answer pair includes a standard question and a standard answer;

[0131] The target standard question determination module 1220 is used to determine the target standard question corresponding to the question to be answered based on the similarity between the standard questions represented by the leaf nodes in the target topic tree and the question to be answered.

[0132] The target standard answer determination module 1230 is used to determine the target standard answer corresponding to the target standard question based on the standard question-answer pair corresponding to the target standard question;

[0133] The answer determination module 1240 is used to determine the target standard answer as the standard answer to the question to be answered.

[0134] In some embodiments, the target standard answer determination module may include:

[0135] A similarity calculation unit is used to calculate the similarity between the standard question represented by the leaf node in the target topic tree and the question to be answered.

[0136] The first target standard problem determination unit is used to determine the standard problem represented by the leaf node as the target standard problem corresponding to the problem to be answered if the similarity is greater than a preset threshold.

[0137] In some embodiments, the apparatus may further include:

[0138] The current node determination module is used to determine the parent node corresponding to the leaf node as the current node if the similarity is less than or equal to the preset threshold.

[0139] The current similarity determination module is used to calculate the similarity between the standard question represented by the current node and the question to be answered, and obtain the current similarity.

[0140] The parent node determination module is used to determine whether the current node has a parent node if the current similarity is less than or equal to the preset threshold.

[0141] The repeat module is used to, if the current node has a parent node, re-establish the parent node as the current node, and repeat the process of calculating the similarity between the standard question represented by the current node and the question to be answered to obtain the current similarity, up to the step of re-establishing the parent node as the current node if the current node has a parent node.

[0142] In some embodiments, the apparatus may further include:

[0143] The current standard question determination module is used to determine the standard question represented by the current node as the target standard question corresponding to the question to be answered if the current similarity is greater than the preset threshold.

[0144] In some embodiments, the leaf nodes are at least two, and the device may further include:

[0145] In some embodiments, the target standard problem determination module may include:

[0146] The question similarity determination unit is used to calculate the similarity between the standard question represented by each leaf node in the target topic tree and the question to be answered, so as to obtain the question similarity corresponding to each leaf node;

[0147] The candidate leaf node determination unit is used to determine candidate leaf nodes whose question similarity is greater than a preset threshold based on the question similarity corresponding to each leaf node.

[0148] The sorting result determination unit is used to sort at least two candidate leaf nodes based on the question similarity corresponding to each candidate leaf node if there are at least two candidate leaf nodes, and obtain a sorting result.

[0149] The target leaf node determination unit is used to determine the target leaf node based on the sorting result;

[0150] The second target standard problem determination unit is used to determine the standard problem represented by the target leaf node as the target standard problem corresponding to the problem to be answered.

[0151] In some embodiments, if the current node does not have a parent node, the apparatus may further include:

[0152] The answer retrieval request sending module is used to send a standard answer retrieval request to the terminal; the standard answer retrieval request carries the question to be answered;

[0153] The pending question-and-answer pair receiving module is used to receive a pending question-and-answer pair sent by the terminal, which is constructed based on the standard answer acquisition request; the pending question-and-answer pair carries the question to be answered and the corresponding target standard answer.

[0154] The topic tree update module is used to update the target topic tree based on the question-and-answer pairs to be answered.

[0155] In some embodiments, the topic tree update module may include:

[0156] The target subtopic determination unit is used to determine the target subtopic corresponding to the question-and-answer pair to be answered;

[0157] The target parent topic determination unit is used to determine the target parent topic corresponding to the target subtopic in the target topic tree;

[0158] The target node adding unit is used to add a target node corresponding to the target subtopic under the node corresponding to the target parent topic; the target node represents the question-and-answer pair to be answered corresponding to the target subtopic.

[0159] In some embodiments, the similarity calculation unit may include:

[0160] The vector transformation subunit is used to determine the model based on similarity, transforming the standard question represented by the leaf node into a first vector and the question to be answered into a second vector.

[0161] A vector similarity determination subunit is used to calculate the vector similarity between the first vector and the second vector based on the similarity determination model;

[0162] The similarity determination subunit is used to determine the vector similarity as the similarity between the standard question represented by the leaf node and the question to be answered, based on the similarity determination model.

[0163] The apparatus and method embodiments described herein are based on the same inventive concept.

[0164] This application provides an answer determination device, which includes a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the answer determination method provided in the above method embodiments.

[0165] Embodiments of this application also provide a computer storage medium, which can be disposed in a terminal to store at least one instruction or at least one program related to implementing an answer determination method in the method embodiments. The at least one instruction or at least one program is loaded and executed by the processor to implement the answer determination method provided in the above method embodiments.

[0166] Embodiments of this application also provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the answer determination method provided in the above-described method embodiments.

[0167] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0168] The memory described in this application embodiment can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for the functions, etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0169] The answer determination method embodiments provided in this application can be executed in mobile terminals, computer terminals, servers, or similar computing devices. Taking a server as an example, Figure 13 is a hardware structure block diagram of a server for an answer determination method provided in this application embodiment. As shown in Figure 13, the server 1300 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 1310 (CPUs 1310 may include, but are not limited to, microprocessors MCUs or programmable logic devices FPGAs), a memory 1330 for storing data, and one or more storage media 1320 (e.g., one or more mass storage devices) for storing application programs 1323 or data 1322. The memory 1330 and storage media 1320 may be temporary or persistent storage. The program stored in the storage media 1320 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1310 can be configured to communicate with the storage medium 1320 and execute a series of instructions stored in the storage medium 1320 on the server 1300. The server 1300 may also include one or more power supplies 1360, one or more wired or wireless network interfaces 1350, one or more input / output interfaces 1340, and / or one or more operating systems 1321, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0170] The input / output interface 1340 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 1300. In one example, the input / output interface 1340 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 1340 may be a radio frequency (RF) module used for wireless communication with the Internet.

[0171] Those skilled in the art will understand that the structure shown in FIG13 is merely illustrative and does not limit the structure of the electronic device described above. For example, server 1300 may also include more or fewer components than shown in FIG13, or have a different configuration than shown in FIG13.

[0172] As can be seen from the embodiments of the answer determination method, apparatus, device, or storage medium provided in this application, this application determines the target topic tree corresponding to the question to be answered; based on the similarity between the standard questions represented by the leaf nodes in the target topic tree and the question to be answered, it determines the target standard question corresponding to the question to be answered; based on the standard question-answer pair corresponding to the target standard question, it determines the target standard answer corresponding to the target standard question; and it determines the target standard answer as the standard answer to the question to be answered. The target topic tree of this application includes multiple levels of topics, and each topic corresponds to a standard question-answer pair, thereby matching the corresponding standard question to the question to be answered and determining the standard answer. Since each topic in the target topic tree of this application corresponds to a standard question, and the similarity between the standard questions corresponding to each topic is very low and they are dissimilar questions, the standard question corresponding to the question to be answered can be determined quickly and accurately.

[0173] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0174] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0175] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer storage medium, such as a read-only memory, a disk, or an optical disk.

[0176] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for determining an answer, characterized in that, The method is applied to an intelligent customer service scenario. The method includes: determining a target topic tree corresponding to the question to be answered; the target topic tree is constructed based on a target topic and its corresponding subtopics; the root node of the target topic tree represents a standard question-and-answer pair corresponding to the target topic, and the nodes in the target topic tree other than the root node represent standard question-and-answer pairs corresponding to the subtopics; the branches of the target topic tree represent the relationships between different topics, and the standard question-and-answer pair includes a standard question and a standard answer; calculating the similarity between the standard question represented by the leaf node in the target topic tree and the question to be answered; if the similarity is less than or equal to a preset threshold, taking the parent node of the leaf node as the current node, calculating the similarity between the standard question represented by the current node and the question to be answered, obtaining the current similarity, and matching upwards layer by layer to determine the target standard question corresponding to the question to be answered; determining the target standard answer corresponding to the target standard question based on the standard question-and-answer pair corresponding to the target standard question; and determining the target standard answer as the standard answer to the question to be answered.

2. The method according to claim 1, characterized in that, The method further includes: if the similarity is greater than a preset threshold, determining the standard problem represented by the leaf node as the target standard problem corresponding to the question to be answered.

3. The method according to claim 1, characterized in that, The method further includes: if the current similarity is less than or equal to the preset threshold, determining whether the current node has a parent node; if the current node has a parent node, taking the parent node as the current node again, and repeating the calculation of the similarity between the standard question represented by the current node and the question to be answered to obtain the current similarity, up to the step of taking the parent node as the current node again if the current node has a parent node.

4. The method according to claim 1, characterized in that, After calculating the similarity between the standard problem represented by the current node and the question to be answered, and obtaining the current similarity, the method further includes: if the current similarity is greater than the preset threshold, determining the standard problem represented by the current node as the target standard problem corresponding to the question to be answered.

5. The method according to any one of claims 1-4, characterized in that, The leaf nodes are at least two. The step of determining the target standard question corresponding to the question to be answered based on the similarity between the standard question represented by the leaf node in the target topic tree and the question to be answered includes: calculating the similarity between the standard question represented by each leaf node in the target topic tree and the question to be answered, obtaining a question similarity corresponding to each leaf node; determining candidate leaf nodes whose question similarity is greater than a preset threshold based on the question similarity corresponding to each leaf node; if there are at least two candidate leaf nodes, sorting the at least two candidate leaf nodes based on the question similarity corresponding to each candidate leaf node, obtaining a sorting result; determining the target leaf node based on the sorting result; and determining the standard question represented by the target leaf node as the target standard question corresponding to the question to be answered.

6. The method according to claim 3, characterized in that, If the current node does not have a parent node, the method further includes: sending a standard answer retrieval request to the terminal; the standard answer retrieval request carrying the question to be answered; receiving a question-and-answer pair to be answered, constructed based on the standard answer retrieval request, sent by the terminal; the question-and-answer pair to be answered carrying the question to be answered and the corresponding target standard answer; and updating the target topic tree based on the question-and-answer pair to be answered.

7. The method according to claim 6, characterized in that, The step of updating the target topic tree based on the question-and-answer pair to be answered includes: determining the target subtopic corresponding to the question-and-answer pair to be answered; determining the target parent topic corresponding to the target subtopic in the target topic tree; adding a target node corresponding to the target subtopic under the node corresponding to the target parent topic; the target node represents the question-and-answer pair to be answered corresponding to the target subtopic.

8. The method according to claim 1, characterized in that, The step of calculating the similarity between the standard question represented by the leaf node in the target topic tree and the question to be answered includes: based on a similarity determination model, converting the standard question represented by the leaf node into a first vector and the question to be answered into a second vector; based on the similarity determination model, calculating the vector similarity between the first vector and the second vector; and based on the similarity determination model, determining the vector similarity as the similarity between the standard question represented by the leaf node and the question to be answered.

9. An answer-determining device, characterized in that, The device is applied in an intelligent customer service scenario. The device includes: a target topic tree determination module, used to determine the target topic tree corresponding to the question to be answered; the target topic tree is constructed based on a target topic and its corresponding subtopics; the root node of the target topic tree represents a standard question-and-answer pair corresponding to the target topic, and the nodes in the target topic tree other than the root node represent standard question-and-answer pairs corresponding to the subtopics; the branches of the target topic tree represent the relationships between different topics, and the standard question-and-answer pair includes a standard question and a standard answer; a target standard question determination module, used to calculate the similarity between the standard question represented by the leaf node in the target topic tree and the question to be answered; if the similarity is less than or equal to a preset threshold, the parent node corresponding to the leaf node is taken as the current node, and the similarity between the standard question represented by the current node and the question to be answered is calculated to obtain the current similarity, and the process is repeated layer by layer upwards to determine the target standard question corresponding to the question to be answered; a target standard answer determination module, used to determine the target standard answer corresponding to the target standard question based on the standard question-and-answer pair corresponding to the target standard question; and an answer determination module, used to determine the target standard answer as the standard answer to the question to be answered.

10. The apparatus according to claim 9, characterized in that, The target standard answer determination module includes: a first target standard question determination unit, used to determine the standard question represented by the leaf node as the target standard question corresponding to the question to be answered if the similarity is greater than a preset threshold.

11. The apparatus according to claim 9, characterized in that, The device further includes: a parent node determination module, used to determine whether the current node has a parent node if the current similarity is less than or equal to the preset threshold; and a repetition module, used to, if the current node has a parent node, re-use the parent node as the current node, and repeat the steps of calculating the similarity between the standard question represented by the current node and the question to be answered to obtain the current similarity, up to the step of re-using the parent node as the current node if the current node has a parent node.

12. The apparatus according to claim 9, characterized in that, The device further includes a current standard problem determination module, used to determine the standard problem represented by the current node as the target standard problem corresponding to the question to be answered if the current similarity is greater than the preset threshold.

13. The apparatus according to any one of claims 9-12, characterized in that, The leaf nodes are at least two, and the target standard question determination module includes: a question similarity determination unit, used to calculate the similarity between the standard question represented by each leaf node in the target topic tree and the question to be answered, to obtain the question similarity corresponding to each leaf node; a candidate leaf node determination unit, used to determine candidate leaf nodes whose question similarity is greater than a preset threshold based on the question similarity corresponding to each leaf node; a ranking result determination unit, used to rank at least two candidate leaf nodes based on the question similarity corresponding to each candidate leaf node if there are at least two candidate leaf nodes, to obtain a ranking result; a target leaf node determination unit, used to determine the target leaf node based on the ranking result; and a second target standard question determination unit, used to determine the standard question represented by the target leaf node as the target standard question corresponding to the question to be answered.

14. The apparatus according to claim 11, characterized in that, If the current node does not have a parent node, the device further includes: an answer retrieval request sending module, used to send a standard answer retrieval request to the terminal; the standard answer retrieval request carries the question to be answered; a question-and-answer pair receiving module, used to receive a question-and-answer pair to be answered sent by the terminal, constructed based on the standard answer retrieval request; the question-and-answer pair to be answered carries the question to be answered and the corresponding target standard answer; and a topic tree updating module, used to update the target topic tree based on the question-and-answer pair to be answered.

15. The apparatus according to claim 14, characterized in that, The topic tree update module includes: a target subtopic determination unit, used to determine the target subtopic corresponding to the question-and-answer pair to be answered; a target parent topic determination unit, used to determine the target parent topic corresponding to the target subtopic in the target topic tree; and a target node addition unit, used to add a target node corresponding to the target subtopic under the node corresponding to the target parent topic; the target node represents the question-and-answer pair to be answered corresponding to the target subtopic.

16. The apparatus according to claim 9, characterized in that, The similarity calculation unit includes: a vector transformation subunit, used to transform the standard question represented by the leaf node into a first vector and the question to be answered into a second vector based on the similarity determination model; a vector similarity determination subunit, used to calculate the vector similarity between the first vector and the second vector based on the similarity determination model; and a similarity determination subunit, used to determine the vector similarity as the similarity between the standard question represented by the leaf node and the question to be answered based on the similarity determination model.

17. An answer-determining device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the answer determination method as described in any one of claims 1-8.

18. A computer storage medium, characterized in that, The computer storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the answer determination method as described in any one of claims 1-8.

19. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the answer determination method as described in any one of claims 1-8.

Images