Question answering method, apparatus and device, and medium

Abstract

The embodiment of the invention discloses a question answering method and device, equipment and a medium. According to the scheme, the method comprises the steps that after target problem information provided by a user is obtained, intention recognition is conducted on the target problem information, and an intention recognition result is obtained; then, determining a target node corresponding to the intention recognition result; wherein the target node is a node in a standard operation program, and is configured with a verbal skill library for solving the target problem information; therefore, the target reply content corresponding to the target question information can be further matched from the verbal skill library based on the target question information.

Description

Technical Field

[0001] This application relates to the technical field of intelligent customer service, and particularly to a question answering method, device, equipment and medium. Background Art

[0002] Existing customer service systems usually adopt human customer service. However, with the increase in the number of users and the frequency of user questions, it may lead to a shortage of human customer service. Moreover, using human customer service also requires training of human customer service, which consumes a large amount of human and material resources.

[0003] In the prior art, there are also intelligent customer service robots implemented based on Retrieval-augmented Generation (RAG) technology to answer questions raised by users. However, the question answering results generated using the retrieval-augmented generation technology may have the problem of model hallucination, resulting in insufficient accuracy of the generated question answering results.

[0004] In view of this, there is a need to provide a question answering solution with low labor cost and high accuracy. Summary of the Invention

[0005] In view of this, embodiments of this application provide a question answering method, device, equipment and medium to provide a question answering solution with low labor cost and high accuracy.

[0006] To solve the above technical problems, embodiments of this specification provide a question answering method. The method is applied to a question answering system, and the question answering system includes a standard operation procedure for processing question answering services. The method includes: Obtain the target question information provided by the user; Perform intent recognition on the target question information to obtain an intent recognition result; Determine the target node corresponding to the intent recognition result; the target node is a node in the standard operation procedure and is configured with a speech library for solving the target question information; Based on the target question information, match the target reply content corresponding to the target question information from the speech library.

[0007] Embodiments of this specification also provide a question answering device. The device is applied to a question answering system, and the question answering system includes a standard operation procedure for processing question answering services. The device includes: An information acquisition module, configured to obtain the target question information provided by the user; An intent recognition module, configured to perform intent recognition on the target question information to obtain an intent recognition result; A node determination module, configured to determine a target node corresponding to the intention recognition result; the target node is a node in the standard operation procedure, and is configured with a speech library for solving the target problem information. A reply determination module, configured to match, based on the target problem information, a target reply content corresponding to the target problem information from the speech library.

[0008] An embodiment of this specification further provides a question answering device, which is applied to a question answering system, and the question answering system includes a standard operation procedure for processing question answering services. The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can: Obtain target problem information provided by a user; Perform intention recognition on the target problem information to obtain an intention recognition result; Determine a target node corresponding to the intention recognition result; the target node is a node in the standard operation procedure, and is configured with a speech library for solving the target problem information; Based on the target problem information, match a target reply content corresponding to the target problem information from the speech library.

[0009] An embodiment of this specification further provides a computer-readable storage medium, which stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the above question answering method is implemented.

[0010] At least one embodiment provided in this specification can achieve the following beneficial effects: In the embodiment of this specification, after the question answering system obtains the target problem information provided by the user, it can perform intention recognition on the target problem information to obtain an intention recognition result, and determine a target node corresponding to the intention recognition result; wherein, the target node is a node in the standard operation procedure, and is configured with a speech library for solving the target problem information; finally, based on the target problem information, match a target reply content corresponding to the target problem information from the speech library. Thus, instead of using a human customer service to reply according to the user's question, the question answering system including the standard operation procedure for processing question answering services performs intention recognition on the user's question, and matches a reply content corresponding to the user's question from the speech library, which is beneficial to reducing the consumption of human resources and reducing labor costs.

[0011] On the other hand, by matching the reply content corresponding to the user's question from the conversation library at the target node corresponding to the intent recognition result, the answer to the question can be determined more pertinently, and the result generated directly by the model is also avoided, which is beneficial to improving the accuracy of the question reply. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0013] Figure 1 Schematic diagram of an application scenario of a question reply method provided by an embodiment of this specification; Figure 2 Schematic flowchart of a question reply method provided by an embodiment of this specification; Figure 3 Schematic diagram of the structure of a standard operating procedure provided by an embodiment of this specification; Figure 4 Swimlane diagram of a question reply method provided by an embodiment of this specification; Figure 5 Corresponding to an embodiment of this specification Figure 2 Schematic diagram of the structure of a question reply device; Figure 6 Corresponding to an embodiment of this specification Figure 2 Schematic diagram of the structure of a question reply device. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0014] Many specific details are set forth in the following description in order to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of this application. Therefore, this application is not limited by the specific embodiments disclosed below.

[0015] The terms used in one or more embodiments of this application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this application. The singular forms "a", "said", and "the" used in one or more embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this application refers to and includes any or all possible combinations of one or more related listed items.

[0016] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of the present application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0017] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards in the relevant region, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0018] In the prior art, there are usually two ways for the customer service system to answer questions raised by users. The first is to use human customer service to answer and reply to the questions raised by users. However, with the increase in the number of users and the frequency of user questions, it may lead to a shortage of human customer service. Moreover, using human customer service also requires pre-training of human customer service, which consumes a large amount of human and material resources. The second is to use an intelligent customer service robot implemented based on the Retrieval-augmented Generation (RAG) technology to answer the questions raised by users. This method uses a model to retrieve information on the Internet based on the questions raised by users and generates a reply content for the user's questions based on the retrieval results. However, the reply content generated by the model may have the problem of model hallucination, resulting in insufficient accuracy of the reply content generated for the user's questions.

[0019] To solve the defects in the related art, the following embodiments are given in this solution.

[0020] Figure 1 It is a schematic diagram of an application scenario of a question reply method provided in an embodiment of this specification.

[0021] As Figure 1As shown, after the client 101 of the question-and-answer system obtains the target question information provided by the user, it can send the target question information to the server 102 of the question-and-answer system. After the server 102 of the question-and-answer system obtains the target question information, it can perform intent recognition on the target question information to obtain an intent recognition result, and based on the intent recognition result, determine a target node corresponding to the intent recognition result. Among them, the target node is a node in the standard operating procedure for processing question-and-answer services included in the question-and-answer system, and is configured with a speech library for solving the target question information.

[0022] After the server 102 of the question-and-answer system determines the target node, it can match the target reply content corresponding to the target question information from the speech library configured at the target node based on the target question information. Further, the server 102 of the question-and-answer system can feedback the determined target reply content corresponding to the target question information to the client 101 of the question-and-answer system, so that the client 101 of the question-and-answer system can display the target reply content to the user.

[0023] Specifically, the client 101 of the question-and-answer system can be carried on the user's electronic device, such as a smart watch, a smart phone, a handheld computer, a virtual reality terminal, or an augmented reality terminal, etc. Specifically, the server 102 of the question-and-answer system can be an independent physical server, or a server cluster or a distributed file system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0024] In practical applications, data transmission between the client 101 of the question-and-answer system and the server 102 of the question-and-answer system can be carried out through a local area network connection, a wide area network connection, an Internet connection, or other types of data network connections, or through other means of data transmission, and no specific limitation is made thereto.

[0025] Figure 2 It is a schematic flowchart of a question reply method provided by an embodiment of this specification. From a hardware perspective, the execution subject of this process can be a question-and-answer system. From a program perspective, the execution subject of this process can be an application program installed in the question-and-answer system. The method is applied to a question-and-answer system, and the question-and-answer system includes a standard operating procedure for processing question-and-answer services. As Figure 2 shown, this process can include the following steps: Step 202: Obtain the target question information provided by the user.

[0026] In the embodiments of this specification, a question-and-answer system may be a system for processing and answering user questions. The question-and-answer system may include a standard operating procedure for processing question-and-answer services. Among them, a standard operating procedure (SOP) is a standardized method for describing the execution of specific tasks or processes in a specific organization or industry. The main purpose of the standard operating procedure is to ensure consistency, improve efficiency, reduce errors, and ensure compliance, especially in fields with strict quality management and regulatory requirements (such as the pharmaceutical, food safety, and medical fields).

[0027] In the embodiments of this specification, the standard operating procedure set in the question-and-answer system may be a standard operating procedure set based on expert experience for processing question-and-answer services. Since users may ask questions based on various services, corresponding standard operating procedures may be set in the question-and-answer system for various services that may be involved. Among them, services may include but are not limited to: payment services, fund services, insurance services, logistics services, commodity purchase services, rental services, ticket purchase services, home improvement services, claim settlement services, etc.

[0028] In the embodiments of this specification, a standard operating procedure may include multiple nodes, and a service may involve one or more nodes. For example: For the logistics service, it may involve "logistics order number query node", "logistics progress query node", "logistics personnel information query node", etc.

[0029] In the embodiments of this specification, the target question information provided by the user may be question information for business consultation by the user for a specific service. Among them, services may include but are not limited to: payment services, fund services, insurance services, logistics services, commodity purchase services, rental services, ticket purchase services, home improvement services, claim settlement services, etc.

[0030] In practical applications, the user may input the question information that they want to consult at the client of the question-and-answer system. Specifically, the user may directly input text to ask questions, or may also ask questions by voice, and no specific limitation is made thereto. For example: After a user purchases a certain commodity online and is not satisfied, and wants to consult how to handle the return, they may input the target question information: "How to handle the return?" Another example: When the user sends a package and accidentally fills in the wrong logistics order number, they may input the target question information: "What should I do if I filled in the wrong logistics order number?" In practical applications, after the client of the Q&A system obtains the target question information provided by the user, it can provide the target question information to the server of the Q&A system for processing. After the server of the Q&A system obtains the target answer content for the target question information, it can feedback the target answer content to the client of the Q&A system, so that the target answer content can be displayed to the user at the client of the Q&A system. If the user is not satisfied with the answer result, they can continue to ask questions and provide new target question information.

[0031] It should be noted that using the client to obtain the user's question information is only one embodiment of this application. In other embodiments, the user's terminal device can directly report the target question information provided by the user to the Q&A system, or other methods can be used to obtain the target question information provided by the user, and no specific limitation is made in this regard.

[0032] Step 204: Perform intent recognition on the target question information to obtain an intent recognition result.

[0033] In the embodiments of this specification, after the Q&A system obtains the target question information of the user, it can perform intent recognition on the target question information to identify the specific needs of the user. By performing intent recognition on the target question information, the business category that the user wants to consult and the specific requirements can be determined.

[0034] In practical applications, the Q&A system can use a large language model to perform intent recognition on the target question information, or can perform intent recognition on the target question information by extracting keywords, or other recognition methods can be adopted to perform intent recognition on the target question information, and no specific limitation is made in this regard. Since the subsequent embodiments in the embodiments of this specification will explain in detail performing intent recognition on the target question information using a large language model and performing intent recognition on the target question information by extracting keywords, no further elaboration will be made here.

[0035] Step 206: Determine the target node corresponding to the intent recognition result; the target node is a node in the standard operating procedure, and is configured with a speech library for solving the target question information.

[0036] In practical applications, the node configurator can pre-configure the nodes in the standard operating procedure. Specifically, the node name, node number, node function description, speech library at the node, etc. can be configured.

[0037] In practical applications, not all nodes in the standard operation procedure are necessarily set with a speech library for solving problem information. Some nodes may be intermediate nodes, and these nodes may not answer specific questions raised by users, but provide some further options for users to select to refine the specific needs of users. In this case, a speech library may not be set at these intermediate nodes, but question options for users to select are set.

[0038] In the embodiments of this specification, the speech library for solving the target problem information may include at least one speech for answering the target problem information. These speeches may be artificially preset or the speech content generated by the model. For the speech generated by the model, after being manually reviewed and passed, it can be added to the speech library.

[0039] Figure 3 It is a schematic structural diagram of a standard operation procedure provided by the embodiments of this specification. As Figure 3As shown, the nodes in the standard operating procedure can be set in the form of a tree diagram, and some nodes for claims problems and logistics problems are shown in the figure. Taking the problem raised by the user at the client side of the Q&A system (intelligent customer service) as a logistics problem as an example, nodes 2, 21, 22, 211, 212, 221, and 222 are nodes for handling logistics problems. Among them, nodes 21 and 22 are child nodes of node 2, nodes 211 and 212 are child nodes of node 21, and nodes 221 and 222 are child nodes of node 22. Node function description information corresponding to the nodes and a speech library corresponding to the nodes are preset at nodes 211, 212, 221, and 222. For example, the node function description information of node 211 is: "Solve the problem of how to query the logistics order number", and the speech in the corresponding speech library can be the speech related to "how to solve the problem of querying the logistics order number". Suppose the problem raised by the user is: "Excuse me, how can I query the logistics order number of the goods I purchased?", after identifying the intention of the target problem information, the corresponding target node can be determined as: node 211, and then the target reply content corresponding to the target problem information can be matched from the speech library of node 211. Nodes 2, 21, and 22 can correspond to intermediate nodes, and question options for users to select can be pre-configured at these nodes. Taking node 2 as an example, the question options configured at node 2 can be: "Do you want to consult the logistics order number problem or the logistics progress problem?" When the user asks the intelligent customer service: "I want to consult a logistics problem", the intelligent customer service can advance from the start node in the standard operating procedure to node 2, and based on the question options configured at node 2, ask the user the question: "Do you want to consult the logistics order number problem or the logistics progress problem?" Suppose the user answers: "I want to consult the logistics progress problem", then the intelligent customer service can advance from node 2 in the standard operating procedure to node 22, and based on the question options configured at node 22, ask the user the question again: "Do you want to consult 'how to query the logistics progress' or 'how to handle the situation where the logistics progress has not been updated for a long time'?" Suppose the user answers: "How to query the logistics progress", then the intelligent customer service can advance from node 22 in the standard operating procedure to node 221, and based on the speech library configured at node 221, match the reply content corresponding to the specific problem of "how to query the logistics progress" and feedback it to the user. If the user's initial question is "How to query the logistics progress?", then the intelligent customer service can advance continuously from the start node to node 221 in the node advancement order of "start node - node 2 - node 22 - node 221" in the standard operating procedure, and based on the speech library configured at node 221, match the reply content corresponding to the specific problem of "how to query the logistics progress" and feedback it to the user.

[0040] Step 208: Based on the target question information, match the target answer content corresponding to the target question information from the speech library.

[0041] In the embodiments of this specification, the target answer corresponding to the target question information can be a solution to the target question information, or a query result based on the appeal of the target question information. Based on the intent recognition results obtained by performing intent recognition on the target question information provided by the user, the question-answering system determines the target node and can then match the target answer corresponding to the target question information from the pre-configured speech library at the target node. The question-answering system client can then display the target answer content to the user.

[0042] In actual applications, the target answer content corresponding to the target question information is matched from the speech library configured at the target node. Specifically, it can be done by calling a large language model to directly filter out preset known speech from the speech library as the target answer content, or it can be done by using a large language model to generate new speech based on the known speech in the speech library and the target question information as the target answer content. In actual implementation, the large language model can calculate the matching degree between the target question information and each known speech in the speech library. If the calculated matching degrees are all less than a preset threshold, the large language model can generate a new speech as the target answer content; if at least one of the calculated matching degrees is not less than a preset threshold, the known speech in the speech library with the highest matching degree with the target question information can be used as the target answer content.

[0043] Figure 2 In the method, after the question-answering system obtains the target question information provided by the user, it can perform intent recognition on the target question information, obtain the intent recognition result, and determine the target node corresponding to the intent recognition result; wherein, the target node is a node in the standard operating procedure, and is configured with a speech library for solving the target question information; finally, based on the target question information, the target reply content corresponding to the target question information is matched from the speech library. Thus, there is no need to use manual customer service to reply to the user's question. Instead, the question-answering system including the standard operating procedure for handling question-answering business will perform intent recognition on the user's question and match the reply content corresponding to the user's question from the speech library, which is conducive to reducing the consumption of human resources and reducing labor costs. On the other hand, by matching the reply content corresponding to the user's question from the speech library at the target node corresponding to the intent recognition result, the answer to the question can be determined more specifically, and the direct use of the result generated by the model is avoided, which is conducive to improving the accuracy of the answer to the question.

[0044] based on Figure 2For the method in [the above], the embodiments of this specification also provide some specific implementation schemes of this method, which will be described below.

[0045] Optionally, Figure 2 For the method in [the above], step 204: Perform intent recognition on the target problem information to obtain an intent recognition result, which may specifically include: Use a large language model to perform intent recognition on the target problem information to obtain an intent recognition result.

[0046] In the embodiments of this specification, a large language model (LLM) refers to a deep learning model trained using a large amount of text data. The large language model can generate natural language text or understand the meaning of language text. The large language model can provide relevant knowledge about various topics by training on a huge dataset. The core idea of the large language model is to learn the patterns and structures of natural language through large-scale unsupervised training, and to simulate the language cognition and generation process of humans to a certain extent. The large language model performs well in a variety of application scenarios. It can not only perform simple language tasks such as spelling check and grammar correction, but also handle complex tasks such as text summarization, machine translation, sentiment analysis, dialogue generation, and content recommendation. Through pre-training on a large-scale dataset, the large language model has obtained powerful general modeling capabilities and generalization capabilities.

[0047] In practical applications, the large language model can be trained or fine-tuned so that the large language model has the ability to recognize the intent of the question information proposed by the user. Specifically, the large language model can be trained or fine-tuned using training sample data containing "question information - intent recognition result" sample pairs so that the large language model has the ability to recognize the intent of the question information proposed by the user, or other methods can also be adopted to make the large language model have the ability to recognize the intent of the question information proposed by the user, and no specific limitation is made in this regard.

[0048] In practical applications, by using the large language model to perform intent recognition on the target problem information, the powerful data analysis and data processing capabilities of the large language model can be utilized to accurately and quickly determine the intent recognition result for the target problem information.

[0049] Optionally, Figure 2 For the method in [the above], the question-answering system may further include function description information for describing the functions of each node in the standard operating procedure; correspondingly, step 206: Determine the target node corresponding to the intent recognition result, which may specifically include: Determine the matching degree between the intent recognition result and each piece of the function description information; Determine the node corresponding to the function description information with the highest matching degree to the intention recognition result as the target node.

[0050] In the embodiments of this specification, the question-answering system may further include function description information for describing the functions of each node in the standard operation procedure. Specifically, a node function information library may be set up in the question-answering system. In this node function information library, the node numbers of each node in the standard operation procedure may be associated and stored with the function description information corresponding to the nodes. One node may be stored corresponding to one or more function description information. After determining the intention recognition result, the intention recognition result may be matched with the function description information corresponding to each node in the node function information library, and the node corresponding to the function description information with the highest matching degree to the intention recognition result is determined as the target node.

[0051] In practical applications, the information pre-configured at each node in the standard operation procedure may include the function description information of the node. After determining the intention recognition result, the intention recognition result may be respectively matched with the function description information configured at each node in the standard operation procedure, and the node corresponding to the function description information with the highest matching degree to the intention recognition result is determined as the target node.

[0052] In the embodiments of this specification, the function description information for describing the functions of each node in the standard operation procedure may be the function description information preset for each node. For example, still taking Figure 3 as an example, Figure 3 in, the function description information set for node 221 may be: "Solve the problem of how to query the logistics progress". Then, if the intention recognition result indicates that the question the user wants to ask is "How to query the logistics progress?", the node corresponding to the function description information with the highest matching degree to this intention recognition result is node 221.

[0053] In practical applications, the matching degree between the intention recognition result and the function description information may be calculated using a pre-trained language model, or the matching degree between the intention recognition result and the function description information may be calculated using a deep learning model. Other methods may also be used to calculate the matching degree between the intention recognition result and the function description information, and no specific limitation is made thereto.

[0054] In practical applications, the intention recognition result of the target question information may also be determined by extracting the keywords included in the target question information, and the corresponding target node may be determined according to the keywords extracted from the target question information.

[0055] Based on this, Figure 2 in the method of Extract keywords from the target problem information to obtain the target keywords included in the target problem information.

[0056] Correspondingly, Figure 2 In the method of [reference], step 206: Determine the target node corresponding to the intent recognition result, which may specifically include: Judge whether the target keyword is included in the node keyword library to obtain a first judgment result; the node keyword library includes the node keywords corresponding to each node in the standard operating procedure; If the first judgment result indicates that the target keyword is included in the node keyword library, determine the node with the same node keyword and the target keyword as the target node.

[0057] In practical applications, a large language model with keyword extraction capabilities can be used to extract keywords from the target problem information, or a keyword extraction model can be used to extract keywords from the target problem information, or other keyword extraction methods can be used to extract keywords from the target problem information, which is not specifically limited. Among them, the keyword extraction model may include, but is not limited to: Bidirectional Encoder Representations from Transformers (BERT) model, Bidirectional Long Short-Term Memory (BiLSTM) model, Graph Neural Network (GNN) model, etc.

[0058] In the embodiments of this specification, a node keyword library can be correspondingly set for the standard operating procedure, and the node keyword library can include the node keywords corresponding to each node in the standard operating procedure. Specifically, the node keyword library can associate and store the node numbers of each node with the corresponding node keywords, and one node can be stored corresponding to one or more node keywords.

[0059] In practical applications, after extracting keywords from the target problem information, the target keywords included in the target problem information can be obtained, and then it can be judged whether the target keyword is included in the node keyword library. If the target keyword is included in the node keyword library, the node number corresponding to the target keyword in the node keyword library can be determined, and then the target node can be determined; if the target keyword is not included in the node keyword library, the large language model can be used to perform intent recognition on the target problem information.

[0060] In the embodiments of this specification, if the target node can be determined by extracting keywords, the large language model may not be used to perform intent recognition on the target problem information, and the target node can be determined more conveniently and quickly, which is beneficial to improving the efficiency of determining the target node, and can also reduce the invocation of the large language model, which is beneficial to reducing the burden on the large language model.

[0061] Optionally, Figure 2 In the method of [reference], step 208: Based on the target problem information, match the target reply content corresponding to the target problem information from the conversation template library, which may specifically include: Generate a first prompt word for instructing the large language model to screen the answer conversation corresponding to the target problem information from the conversation template library based on the target problem information; Provide the first prompt word to the large language model to obtain the target reply content corresponding to the target problem information provided by the large language model.

[0062] In practical applications, the question and answer system can call the large language model and match the target reply content corresponding to the target problem information from the conversation template library at the target node based on the target problem information. The large language model can calculate the matching degree between the target problem information and each known conversation in the conversation template library. If at least one of the calculated matching degrees is not less than the preset threshold, it means that the target reply content can be directly screened from the conversation template library without the large language model generating the reply content itself. In this case, the large language model can feedback the matching result to the question and answer system. The question and answer system can generate a first prompt word for instructing the large language model to screen the answer conversation corresponding to the target problem information from the conversation template library based on the matching result, and provide the first prompt word to the large language model; after receiving the first prompt word, the large language model can determine the known conversation with the highest matching degree with the target problem information in the conversation template library as the target reply content corresponding to the target problem information, and feedback the target reply content to the question and answer system.

[0063] In other embodiments, after the large language model calculates the matching degree between the target problem information and each known conversation in the conversation template library, if at least one of the calculated matching degrees is not less than the preset threshold, it can also directly determine the known conversation with the highest matching degree with the target problem information in the conversation template library as the target reply content corresponding to the target problem information, and feedback the target reply content to the question and answer system to improve the efficiency of determining the target reply content.

[0064] In the embodiments of this specification, when the matching degree requirement is met, the known phrase in the phrase library with the highest matching degree to the target question information is directly determined as the target reply content corresponding to the target question information, without using a model to generate the reply content, avoiding the inaccurate situation of the model generation result caused by model hallucination, which is not only beneficial to improving the accuracy of the generated reply content, but also beneficial to reducing the burden on the model.

[0065] Optionally, Figure 2 In the method of, step 208: Based on the target question information, matching the target reply content corresponding to the target question information from the phrase library may specifically include: Generating a second prompt word based on the target question information for instructing the large language model to derive the answer information for answering the target question information based on the known phrases included in the phrase library; Providing the second prompt word to the large language model to obtain the target reply content corresponding to the target question information provided by the large language model.

[0066] In practical applications, the large language model can be trained or fine-tuned so that the large language model has the ability to generate the target reply content for the target question information based on some known phrases. Specifically, the large language model can be trained or fine-tuned using the training sample data containing the sample pairs of "known phrase - target question information - target reply content" so that the large language model has the ability to generate the target reply content for the target question information based on some known phrases, or other methods can also be adopted to make the large language model have the ability to generate the target reply content for the target question information based on some known phrases, and no specific limitation is made thereto.

[0067] [[ID=ConfigOption]]In practical applications, after the large language model calculates the matching degree between the target question information and each known phrase in the phrase library, if the calculated matching degrees are all less than the preset threshold, it means that the target reply content cannot be directly screened out from the phrase library, and the large language model needs to generate the reply content by itself. In this case, the large language model can feedback the matching result to the question and answer system, and the question and answer system can generate a second prompt word for instructing the large language model to derive the answer information for answering the target question information based on the known phrases included in the phrase library based on the matching result, and provide the second prompt word to the large language model; after receiving the second prompt word, the large language model can generate the target reply content corresponding to the target question information based on the known phrases included in the phrase library and the target question information, and feedback the target reply content to the question and answer system.

[0068] In other embodiments, after the large language model calculates the matching degree between the target question information and each known speech in the speech library, when the calculated matching degree is less than a preset threshold, it can also directly generate the target reply content corresponding to the target question information based on the known speech contained in the speech library and the target question information, so as to improve the efficiency of determining the target reply content.

[0069] In actual applications, considering that the accuracy of the target response content generated by the large language model cannot be guaranteed, the target response content generated by the large language model can be manually reviewed. The target response content after manual review can be fed back to the user to ensure that the target response content fed back to the user is accurate.

[0070] Based on this, after obtaining the target answer content corresponding to the target question information provided by the large language model by providing the second prompt word to the large language model using the large language model, the method may further include: The target answer content corresponding to the target question information generated by the large language model is sent to a manual verification module; the manual verification module is used to provide the target answer content to a manual reviewer and receive the verification result provided by the manual reviewer; Obtaining a verification result of the target response content fed back by the manual verification module; If the verification result indicates that the verification is passed, the target reply content is provided to the user.

[0071] In actual applications, the manual verification module can provide the target answer content corresponding to the target question information generated by the large language model to manual reviewers for verification. If the verification is passed, the target answer content can be provided to the user; if the verification fails, the manual reviewer can modify it before providing it to the user, or it can be transferred to manual customer service to handle the target question information. There is no specific limitation on this.

[0072] Correspondingly, Figure 2 The method may further include: The target reply content that has passed the verification result is added to the speech library.

[0073] In actual applications, the target answer content corresponding to the target question information generated by the large language model, after manual review and verification, indicates that the target answer content generated by the large language model can accurately answer the target question information raised by the user. Therefore, the target answer content generated by the verified large language model can be added to the speech library at the target node, which is conducive to enriching the speech library.

[0074] In practical applications, when a user asks questions to an intelligent customer service, the user may ask multiple questions consecutively. In this case, the latest question asked by the user is usually the question the user wants to ask. Therefore, it is possible to only reply to the latest question asked by the user.

[0075] Optionally, Figure 2 in the method of , the method may further include: If multiple question messages continuously sent by the user are obtained, the question message sent last by the user among the multiple question messages is determined as the target question message.

[0076] In practical applications, if a question-and-answer system obtains multiple question messages continuously sent by a user within a preset time period, the question message sent last by the user among the multiple question messages can be determined as the target question message for processing. Among them, the preset time period can be a preset time period with a relatively short time, such as: 1 second, 5 seconds, 10 seconds, etc. The preset time period can be set and adjusted according to actual needs, and no specific limitation is made thereto. For example: A certain user continuously sent question message A, question message B, and question message C within 10 seconds. When the user sent question message C, the user may no longer be concerned about question message A and question message B. Therefore, question message C can be determined as the target question message, and only question message C is processed. On the one hand, this can avoid disturbing the user by generating question replies to all the user's questions, and it can also reduce the burden on the question-and-answer system. Moreover, if question message A, question message B, and question message C are processed simultaneously based on the standard operating procedure, it is also easy to cause processing failure due to inconsistent states of the standard operating procedure. Among them, inconsistent states of the standard operating procedure can mean that when processing different questions, the node advancement paths corresponding to different questions on the standard operating procedure are different. For example: To solve question message A, the advancement path on the standard operating procedure is "node 1 - node 11 - node 111", and to solve question message B, the advancement path on the standard operating procedure is "node 1 - node 12 - node 121", which causes the problem of inconsistent states of the standard operating procedure. Therefore, in this embodiment of this specification, the problem of inconsistent states of the standard operating procedure can also be avoided, which is beneficial to ensuring the smooth progress of answering the user's questions.

[0077] Of course, in other embodiments, the question-and-answer system can also reply to all the questions asked by the user, and no specific limitation is made thereto.

[0078] In practical applications, to ensure that the question message being processed is the latest question message sent by the user, the question-and-answer system can repeatedly determine during the processing whether the question message currently being processed is the latest question message sent by the user. If so, the processing continues; if not, the latest question message sent by the user can be obtained for processing.

[0079] Based on this, Figure 2 in the method in Figure 2 , step 208: after matching the target response content corresponding to the target question information from the conversation library based on the target question information, it may further include: judging whether the user provides other question information after providing the target question information to obtain a second judgment result; if the second judgment result indicates that the user does not provide other question information after providing the target question information, providing the target response content to the user; if the second judgment result indicates that the user provides other question information after providing the target question information, using the other question information as the target question information and executing the response steps for the other question information.

[0080] In the embodiments of this specification, if the second judgment result indicates that the user does not provide other question information after providing the target question information, it may indicate that the target question information is the latest question information sent by the user. Thus, the question-answering system can provide the target response content corresponding to the determined target question information to the user; if the second judgment result indicates that the user provides other question information after providing the target question information, it may indicate that the just-processed target question information is not the latest question information sent by the user. Thus, the question-answering system can use the latest other question information sent by the user as the new target question information and execute the response steps for the other question information.

[0081] In practical applications, the question-answering system can judge whether the currently processed question information is the latest question information sent by the user multiple times during the question processing. Specifically, it can also be at Figure 2 before step 204, after step 204, before step 206, after step 206, and before step 208 in Figure 2 , judge whether the currently processed question information is the latest question information sent by the user. If so, continue to process; if not, the latest question information sent by the user can be obtained for processing. Or, it can also be judged once every preset time period, where the preset time period can be set and adjusted according to actual needs. For example: judge once every 2 seconds whether the currently processed question information is the latest question information sent by the user. If so, continue to process; if not, the latest question information sent by the user can be obtained for processing.

[0082] Optionally, Figure 2 in the method in Figure 2 , step 202: after obtaining the target question information provided by the user, it may further include: adding an execution lock identifier to the target question information; the execution lock identifier is used to indicate that the question information carrying the execution lock identifier is the question information that needs to be replied and processed.

[0083] Correspondingly, Figure 2 In the method of Figure 2 , step 208: After matching the target response content corresponding to the target question information from the conversation library based on the target question information, it may further include: Providing the target response content to the user; Deleting the execution lock flag on the target question information.

[0084] In the embodiments of this specification, the execution lock flag can be used to indicate that the question information carrying the execution lock flag is the question information that needs to be replied and processed. Therefore, after the question and answer system determines the target question information that needs to be processed, it can add the execution lock flag to the target question information. The question and answer system can perform reply processing on the target question information carrying the execution lock flag, and can not process the question information without the execution lock flag, thereby avoiding the situation where it is impossible to quickly determine which question information is the question information that needs to be processed when there is too much question information, which is beneficial to improving the accuracy and efficiency of question information processing.

[0085] In practical applications, after the question and answer system determines the target response content corresponding to the target question information and provides the target response content to the user, the execution lock flag on the target question information can be deleted to release the execution lock and avoid the question and answer system from repeatedly processing the target question information.

[0086] In practical applications, when the question and answer system (intelligent customer service) has problems processing the user's target question information, it can also be taken over by the human customer service to ensure that the questions raised by the user can be answered accurately and quickly.

[0087] Based on this, Figure 2 In the method of Figure 2 , the method may further include: Judging whether a target condition is satisfied during the execution of the method to obtain a third judgment result; the target condition includes at least one of a first judgment condition, a second judgment condition, a third judgment condition, and a fourth judgment condition; The first judgment condition is that at a moment greater than a first preset duration from a first moment, new question information provided by the user is obtained; the first moment is the moment when the target question information provided by the user is obtained; the second judgment condition is that no target response content for the target question information is obtained within a second preset duration after the target question information provided by the user is obtained; the third judgment condition is that the target node is a preset node that needs to be taken over by the human customer service; the fourth judgment condition is that the recognized dissatisfaction emotion value of the user exceeds a preset emotion threshold; If the third judgment result indicates that the target condition is satisfied during the execution of the method, the target problem information will be provided to the human customer service so that the human customer service can process the target problem information.

[0088] In practical applications, when at least one of the first judgment condition, the second judgment condition, the third judgment condition, and the fourth judgment condition is satisfied, the target problem information proposed by the user can be provided to the human customer service, and the human customer service will process the target problem information, that is: when the target condition is satisfied during the execution of the method, the human customer service will take over.

[0089] Among them, the first judgment condition is that at a moment greater than the first preset duration from the first moment, new problem information provided by the user is obtained; the first moment is the moment when the target problem information provided by the user is obtained. Among them, the first preset duration can be a relatively long duration preset by humans, such as: 10 minutes, 20 minutes, 30 minutes, etc.

[0090] In practical applications, when the question-answering system obtains the target problem information provided by the user, the moment of obtaining the target problem information can be recorded as the first moment. When the question-answering system obtains new problem information provided by the user, the moment of obtaining the new problem information can be recorded as the second moment, and calculate whether the duration between the second moment and the first moment is greater than the first preset duration. If the duration between the second moment and the first moment is greater than the first preset duration, the target problem information will be provided to the human customer service so that the human customer service can take over and process the target problem information. For example: The user provided the target problem information at 8:15, and 20 minutes later (assuming the first preset duration is 15 minutes), the user asked a new question about this problem, which indicates that the reply content fed back by the intelligent customer service cannot satisfy the user. At this time, the human customer service can take over and handle it.

[0091] Among them, the second judgment condition is that no target reply content for the target problem information is obtained within the second preset duration after obtaining the target problem information provided by the user. The second preset duration can be set and adjusted by humans according to actual needs, such as: the second preset duration can be set to 5 minutes, 10 minutes, 20 minutes, etc. The situation where the second judgment condition is satisfied may be due to network reasons, timeout in calling the large language model, or other reasons. At this time, the human customer service can take over and handle it.

[0092] In practical applications, when the question-and-answer system obtains the target question information provided by the user, it can record the moment when the target question information is obtained as the first moment. After a second preset duration, it determines whether the target reply content for the target question information is obtained at the third moment (the time difference between the third moment and the first moment is the first preset duration); if the target reply content for the target question information is not obtained at the third moment, the target question information is provided to the human customer service so that the human customer service can take over and process the target question information.

[0093] Among them, the third judgment condition is that the target node is a node that needs to be taken over by the human customer service as preset. In practical applications, for relatively important nodes, these nodes can be preset as nodes that need to be taken over by the human customer service. When advancing to the preset nodes that need to be taken over by the human customer service, the target question information can be provided to the human customer service, and the human customer service processes the target question information.

[0094] In practical applications, after the question-and-answer system performs intent recognition on the target question information provided by the user to obtain an intent recognition result and determines the target node based on the intent recognition result, it can judge whether the target node is a node that needs to be taken over by the human customer service as preset. If the target node is a node that needs to be taken over by the human customer service as preset, the target question information is provided to the human customer service so that the human customer service can take over and process the target question information.

[0095] Among them, the fourth judgment condition is that the recognized dissatisfaction emotion value of the user exceeds the preset emotion threshold. Specifically, a large language model or an emotion recognition model with emotion recognition ability can be used to recognize the emotion of the user based on the target question information provided by the user or the text information or voice information input during the communication between the user and the intelligent customer service. When the dissatisfaction emotion of the user exceeds the preset emotion threshold, it can be transferred to the human customer service for takeover and processing to avoid reducing the user experience.

[0096] Figure 4 This is a swimlane diagram of a question reply method provided by an embodiment of this specification.

[0097] As Figure 4 shown, the execution subjects involved in the process of this question reply method may include the client of the question-and-answer system and the server of the question-and-answer system. Figure 4 The method process may include a question acquisition stage, a target node determination stage, and a reply content determination and feedback stage, and may specifically include the following steps: In the question acquisition stage, an optional implementation manner may be as Figure 4 shown in steps 402 to 408.

[0098] Step 402: The client of the question-and-answer system obtains the target question information provided by the user.

[0099] Step 404: The client of the Q&A system sends the target question information to the server of the Q&A system.

[0100] Step 406: The server of the Q&A system receives the target question information.

[0101] Step 408: The server of the Q&A system adds an execution lock flag to the target question information. Wherein, the execution lock flag is used to indicate that the question information carrying the execution lock flag is the question information that needs to be replied and processed.

[0102] In the target node determination phase, an optional implementation manner can be as shown in Figure 4 Steps 410 to 414 below.

[0103] Step 410: The server of the Q&A system uses a large language model to perform intent recognition on the target question information to obtain an intent recognition result.

[0104] Step 412: The server of the Q&A system determines the matching degree between the intent recognition result and each piece of function description information used to describe the functions of each node in the standard operating procedure.

[0105] Step 414: The server of the Q&A system determines the node corresponding to the function description information with the highest matching degree with the intent recognition result as the target node.

[0106] In the reply content determination and feedback phase, an optional implementation manner can be as shown in Figure 4 Steps 416 to 430 below.

[0107] Step 416: The server of the Q&A system generates a second prompt word based on the target question information, which is used to instruct the large language model to derive the answer information for answering the target question information based on the known words included in the word library.

[0108] Step 418: The server of the Q&A system provides the second prompt word to the large language model to obtain the target reply content corresponding to the target question information provided by the large language model.

[0109] Step 420: The server of the Q&A system sends the target reply content corresponding to the target question information generated by the large language model to the manual verification module.

[0110] Step 422: The server of the Q&A system obtains the verification result for the target reply content fed back by the manual verification module.

[0111] Step 424: The server of the Q&A system sends the target reply content indicating that the verification result is passed to the client of the Q&A system.

[0112] Step 426: The server of the Q&A system deletes the execution lock flag on the target question information and adds the verified target reply content to the conversation template library.

[0113] Step 428: The client of the Q&A system obtains the target reply content.

[0114] Step 430: The client of the Q&A system displays the target reply content.

[0115] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method. Figure 5 For the embodiments of this specification, the corresponding Figure 2 is a schematic structural diagram of a question answering device. This device can be applied to a Q&A system, and the Q&A system may include a standard operating procedure for processing Q&A services, such as Figure 5 shown, this device may include: An information acquisition module 502, configured to acquire target question information provided by a user.

[0116] An intention recognition module 504, configured to perform intention recognition on the target question information to obtain an intention recognition result.

[0117] A node determination module 506, configured to determine a target node corresponding to the intention recognition result; the target node is a node in the standard operating procedure, and is configured with a conversation template library for solving the target question information.

[0118] A reply determination module 508, configured to match the target reply content corresponding to the target question information from the conversation template library based on the target question information.

[0119] Based on Figure 5 the device, the embodiments of this specification also provide some specific implementation solutions of this device, which are described below.

[0120] Optionally, the intention recognition module 504 may specifically include: An intention recognition unit, configured to use a large language model to perform intention recognition on the target question information to obtain an intention recognition result.

[0121] Optionally, the intention recognition module 504 may specifically further include: A keyword extraction unit, configured to extract keywords from the target question information to obtain target keywords included in the target question information.

[0122] Correspondingly, the node determination module 506 may specifically include: A judgment unit, configured to judge whether the target keyword is included in the node keyword library, and obtain a first judgment result; the node keyword library includes node keywords corresponding to each node in the standard operation procedure.

[0123] A target node determination unit, configured to, if the first judgment result indicates that the target keyword is included in the node keyword library, determine the node whose node keyword is the same as the target keyword as the target node.

[0124] Optionally, the question-answering system may further include function description information for describing the functions of each node in the standard operation procedure; correspondingly, the node determination module 506 may specifically further include: A matching degree determination unit, configured to determine the matching degree between the intention recognition result and each piece of function description information.

[0125] The target node determination unit is further configured to determine the node corresponding to the function description information with the highest matching degree with the intention recognition result as the target node.

[0126] Optionally, the reply determination module 508 may specifically include: A first prompt word generation unit, configured to generate a first prompt word based on the target question information, for instructing the large language model to screen the answer words corresponding to the target question information from the word set.

[0127] A target reply content determination module, configured to provide the first prompt word to the large language model, and obtain the target reply content corresponding to the target question information provided by the large language model.

[0128] Optionally, the reply determination module 508 may specifically further include: A second prompt word generation unit, configured to generate a second prompt word based on the target question information, for instructing the large language model to deduce the answer information for answering the target question information based on the known words included in the word set.

[0129] The target reply content determination module is further configured to provide the second prompt word to the large language model, and obtain the target reply content corresponding to the target question information provided by the large language model.

[0130] Optionally, the device may further include: A reply content sending module, configured to send the target reply content corresponding to the target question information generated by the large language model to the manual verification module; the manual verification module is configured to provide the target reply content to the manual reviewer and receive the verification result provided by the manual reviewer.

[0131] A verification result acquisition module, configured to acquire the verification result of the target response content fed back by the manual verification module.

[0132] A response content feedback module, configured to provide the target response content to the user if the verification result indicates a passed verification result.

[0133] Optionally, the device may further include: A response content addition module, configured to add the target response content with a passed verification result to the conversation template library.

[0134] Optionally, the device may further include: A target question information determination module, configured to determine the last question information sent by the user among the multiple question information as the target question information if multiple question information continuously sent by the user is acquired.

[0135] Optionally, the device may further include: A first judgment module, configured to judge whether the user provides other question information after providing the target question information, and obtain a second judgment result.

[0136] The response content feedback module is further configured to provide the target response content to the user if the second judgment result indicates that the user does not provide other question information after providing the target question information.

[0137] A target question information change module, configured to use the other question information as the target question information and execute the response step for the other question information if the second judgment result indicates that the user provides other question information after providing the target question information.

[0138] Optionally, the device may further include: An execution lock identifier addition module, configured to add an execution lock identifier to the target question information; the execution lock identifier is used to indicate that the question information carrying the execution lock identifier is the question information that needs to be replied and processed.

[0139] The response content feedback module is further configured to provide the target response content to the user.

[0140] An execution lock identifier deletion module, configured to delete the execution lock identifier on the target question information.

[0141] Optionally, the device may further include: A second judgment module, configured to judge whether a target condition is satisfied during the execution of the method, and obtain a third judgment result; the target condition includes at least one of a first judgment condition, a second judgment condition, a third judgment condition, and a fourth judgment condition.

[0142] Wherein, the first judgment condition is that at a moment greater than a first preset duration from a first moment, new question information provided by a user is obtained; the first moment is the moment when the target question information provided by the user is obtained; the second judgment condition is that within a second preset duration after the target question information provided by the user is obtained, no target reply content for the target question information is obtained; the third judgment condition is that the target node is a preset node that requires a human customer service to take over; the fourth judgment condition is that it is recognized that the dissatisfaction emotion value of the user exceeds a preset emotion threshold.

[0143] A human customer service takeover module, configured to, if the third judgment result indicates that the target condition is satisfied during the execution of the method, provide the target question information to a human customer service, so that the human customer service processes the target question information.

[0144] It can be understood that the above-mentioned modules refer to computer programs or program segments for executing one or more specific functions. In addition, the distinction of the above-mentioned modules does not mean that the actual program codes must also be separated.

[0145] Based on the same idea, an embodiment of this specification also provides a device corresponding to the above method.

[0146] Figure 6 For the embodiment of this specification, it provides Figure 2 a structural schematic diagram of a question reply device. As Figure 6 shown, the device 600 may include: at least one processor 610; and a memory 630 communicatively connected to the at least one processor; wherein, the memory 630 stores instructions 620 executable by the at least one processor 610, and the instructions are executed by the at least one processor 610, so that the at least one processor 610 can: Obtain target question information provided by a user.

[0147] Perform intent recognition on the target question information to obtain an intent recognition result.

[0148] Determine a target node corresponding to the intent recognition result; the target node is a node in the standard operation procedure, and is configured with a speech library for solving the target question information.

[0149] Based on the target question information, match the target reply content corresponding to the target question information from the speech library.

[0150] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for Figure 6 the device shown, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiment.

[0151] The specific embodiments of this specification are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous. In the 1990s, it was obvious to distinguish whether an improvement in a technology was a hardware improvement (e.g., improvement in circuit structures such as diodes, transistors, switches, etc.) or a software improvement (improvement in method flows). However, with the development of technology, many method flow improvements today can be regarded as direct improvements in hardware circuit structures. Almost all designers obtain the corresponding hardware circuit structures by programming the improved method flows into the hardware circuits. Therefore, it cannot be said that an improvement in a method flow cannot be implemented with a hardware entity module. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can program by themselves to "integrate" a digital system on a piece of PLD without having to ask a chip manufacturer to design and fabricate a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, which is similar to the software compiler used in program development and writing. The original code before compilation also has to be written in a specific programming language, which is called a Hardware Description Language (HDL). And there is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones currently are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be clear that only by slightly logically programming the method flow with the above-mentioned several hardware description languages and programming it into the integrated circuit can the hardware circuit implementing the logical method flow be easily obtained.

[0152] The controller can be implemented in any suitable manner. For example, the controller can take the form of, for example, a microprocessor or a processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of the controller include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, it is entirely possible to logically program the method steps to enable the controller to be implemented in the form of logic gates, switches, ASICs, programmable logic controllers, embedded microcontrollers, etc. to achieve the same function. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as either software modules for implementing the method or structures within the hardware component.

[0153] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0154] For the convenience of description, when describing the above devices, they are described separately as various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0155] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0156] The present invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each flow and / or block in the flowchart illustrations and / or block diagrams, and combinations of flows and / or blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing apparatus create means for implementing the functions specified in the flowchart Figure 1 one flow or more flows and / or blocks Figure 1 or means for implementing the functions specified in one block or more blocks.

[0157] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement the functions specified in the flowchart Figure 1 one flow or more flows and / or blocks Figure 1 or means for implementing the functions specified in one block or more blocks.

[0158] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart Figure 1 one flow or more flows and / or blocks Figure 1 or means for implementing the functions specified in one block or more blocks.

[0159] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0160] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0161] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0162] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0163] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0164] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A question answering method, applied to a question answering system, which includes a standard operating procedure for processing question answering services. The method includes: Obtain target question information provided by the user; Perform intent recognition on the target question information to obtain an intent recognition result; Determine a target node corresponding to the intent recognition result; The target node is a node in the standard operating procedure and is configured with a speech library for solving the target question information; Based on the target question information, match the target reply content corresponding to the target question information from the speech library.

2. The method according to claim 1, wherein the performing intent recognition on the target question information to obtain an intent recognition result specifically includes: Using a large language model to perform intent recognition on the target question information to obtain an intent recognition result.

3. The method according to claim 1, wherein the performing intent recognition on the target question information to obtain an intent recognition result specifically includes: Extract keywords from the target question information to obtain target keywords included in the target question information; The determining the target node corresponding to the intent recognition result specifically includes: Judge whether the node keyword library contains the target keyword to obtain a first judgment result; the node keyword library contains node keywords corresponding to each node in the standard operating procedure; If the first judgment result indicates that the node keyword library contains the target keyword, determine the node with the same node keyword and the target keyword as the target node.

4. The method according to claim 1, wherein the question answering system further includes function description information for describing the functions of each node in the standard operating procedure; the determining the target node corresponding to the intent recognition result specifically includes: Determine the matching degree between the intent recognition result and each function description information; Determine the node corresponding to the function description information with the highest matching degree with the intent recognition result as the target node.

5. The method according to claim 1, wherein the matching the target reply content corresponding to the target question information from the speech library specifically includes: Based on the target question information, generate a first prompt word for instructing the large language model to screen the answer speech corresponding to the target question information from the speech library; Provide the first prompt word to the large language model to obtain the target reply content corresponding to the target question information provided by the large language model.

6. The method according to claim 1, wherein the matching the target reply content corresponding to the target question information from the speech library specifically includes: Based on the target question information, generate a second prompt word for instructing the large language model to deduce the answer information for answering the target question information based on the known speeches included in the speech library; Provide the second prompt word to the large language model to obtain the target reply content corresponding to the target question information provided by the large language model.

7. For the method according to claim 6, after using the large language model to obtain the target reply content corresponding to the target question information provided by the large language model according to the second prompt word, the method further includes: Sending the target reply content corresponding to the target question information generated by the large language model to the manual verification module; The manual verification module is used to provide the target reply content to the manual reviewer and receive the verification result provided by the manual reviewer; Obtaining the verification result of the target reply content fed back by the manual verification module; If the verification result is a result indicating verification passed, providing the target reply content to the user.

8. For the method according to claim 7, the method further includes: Adding the target reply content with the verification result indicating verification passed to the conversation template library.

9. For the method according to claim 1, the method further includes: If multiple question information continuously sent by the user is obtained, determining the question information last sent by the user among the multiple question information as the target question information.

10. For the method according to claim 1, after matching the target reply content corresponding to the target question information from the conversation template library based on the target question information, the method further includes: Judging whether the user provides other question information after providing the target question information to obtain a second judgment result; If the second judgment result indicates that the user does not provide other question information after providing the target question information, providing the target reply content to the user; If the second judgment result indicates that the user provides other question information after providing the target question information, using the other question information as the target question information and performing the reply step for the other question information.

11. For the method according to claim 1, after obtaining the target question information provided by the user, the method further includes: Adding an execution lock identifier to the target question information; The execution lock identifier is used to indicate that the question information carrying the execution lock identifier is the question information that needs to be replied and processed; After matching the target reply content corresponding to the target question information from the conversation template library based on the target question information, the method further includes: Providing the target reply content to the user; Deleting the execution lock identifier on the target question information.

12. For the method according to claim 1, the method further includes: Judging whether the target condition is satisfied during the execution of the method to obtain a third judgment result; The target condition includes at least one of a first judgment condition, a second judgment condition, a third judgment condition, and a fourth judgment condition; The first judgment condition is that at a moment greater than the first preset duration from the first moment, new question information provided by the user is obtained; the first moment is the moment when the target question information provided by the user is obtained; the second judgment condition is that within the second preset duration after the target question information provided by the user is obtained, no target reply content for the target question information is obtained; the third judgment condition is that the target node is a preset node that requires manual customer service takeover; The fourth judgment condition is that it is recognized that the dissatisfaction emotion value of the user exceeds the preset emotion threshold; If the third judgment result indicates that the target condition is satisfied during the execution of the method, the target question information is provided to the manual customer service so that the manual customer service can process the target question information.

13. A question reply device, which is applied to a question and answer system. The question and answer system includes a standard operation procedure for processing question and answer services. The device includes: An information acquisition module, configured to acquire target question information provided by the user; An intention recognition module, configured to perform intention recognition on the target question information to obtain an intention recognition result; A node determination module, configured to determine a target node corresponding to the intention recognition result; The target node is a node in the standard operation procedure, and is configured with a speech library for solving the target question information; A reply determination module, configured to match, based on the target question information, a target reply content corresponding to the target question information from the speech library.

14. A question reply device, which is applied to a question and answer system. The question and answer system includes a standard operation procedure for processing question and answer services. The device includes: At least one processor; And, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can: Acquire target question information provided by the user; Perform intention recognition on the target question information to obtain an intention recognition result; Determine a target node corresponding to the intention recognition result; the target node is a node in the standard operation procedure, and is configured with a speech library for solving the target question information; Based on the target question information, match a target reply content corresponding to the target question information from the speech library.

15. A computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the question reply method according to any one of claims 1 to 12 is implemented.

Images