Question and answer method and device based on intelligent question and answer agent, electronic equipment, storage medium and computer program product

Abstract

The embodiment of the invention discloses a question answering method and device based on an intelligent question answering agent, electronic equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a trained action strategy model and a target question proposed by a target object, and inputting the target question into the action strategy model to obtain an action strategy output by the action strategy model; under the condition that the action strategy is that a determined action is not executed, a target answer of the target question is determined according to the target question and an intelligent question and answer agent, and the determined action is an action for determining related questions and answers of the target question. According to the technical scheme provided by the embodiment of the invention, the computing resource consumption and the response time are reduced.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a question-answering method, device, electronic device, storage medium, and computer program product based on an intelligent question-answering agent. Background Art

[0002] In the field of medical services, patients and their families usually raise various questions in hospitals, such as appointment-related, triage-related, clinical department business-related, disease-related and drug-related questions, which need to be answered by medical staff.

[0003] In recent years, with the development of models, in order to improve the service quality of hospitals and reduce service costs, models are increasingly being used to answer target questions raised by target objects.

[0004] However, the currently used model-based question-answering method based on intelligent question-answering agents has the problems of large consumption of computing resources and long response time, which need to be solved urgently. Summary of the invention

[0005] The embodiments of the present invention provide a question-answering method, device, electronic device, storage medium and computer program product based on an intelligent question-answering agent, which reduces computing resource consumption and response time.

[0006] According to one aspect of the present invention, a question-answering method based on an intelligent question-answering agent is provided, which may include: obtaining a trained action strategy model and a target question raised by a target object, and inputting the target question into the action strategy model to obtain an action strategy output by the action strategy model; when the action strategy is not to perform a determined action, determining a target answer to the target question based on the target question and the intelligent question-answering agent, wherein the determined action is an action to determine the relevant questions and answers of the target question.

[0007] According to another aspect of the present invention, a question-and-answer device based on an intelligent question-and-answer agent is provided, which may include: an action strategy acquisition module, used to obtain a trained action strategy model and a target question raised by a target object, and input the target question into the action strategy model to obtain an action strategy output by the action strategy model; a first target answer determination module, used to determine a target answer to the target question based on the target question and the intelligent question-and-answer agent when the action strategy is not to perform a determined action, wherein the determined action is an action to determine a related question and answer of the target question.

[0008] According to another aspect of the present invention, an electronic device is provided, which may include: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that when the at least one processor executes, the question-and-answer method based on the intelligent question-and-answer agent provided in any embodiment of the present invention is implemented.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, on which computer instructions are stored, and the computer instructions are used to enable a processor to implement the question-answering method based on an intelligent question-answering agent provided by any embodiment of the present invention when executed.

[0010] According to another aspect of the present invention, a computer program product is provided, including a computer program, which, when executed by a processor, implements the question-answering method based on an intelligent question-answering agent provided by any embodiment of the present invention.

[0011] The technical solution of the embodiment of the present invention obtains the trained action strategy model and the target question raised by the target object, and inputs the target question into the action strategy model to obtain the action strategy output by the action strategy model, so as to determine whether to execute the action of determining the relevant questions and answers of the target question through the action strategy; when the action strategy is not to execute the determined action, the target answer of the target question is determined according to the target question and the intelligent question-answering agent, wherein the determined action is the action of determining the relevant questions and answers of the target question, so as to achieve the answer to the target question. The above technical solution determines the action strategy, and when the action strategy is not to execute the determined action, the action of determining the relevant questions and answers of the target question is not executed, and the target answer of the target question is determined directly according to the target question and the intelligent question-answering agent. In order to improve the accuracy of the target answer, there is no need to determine the relevant questions and answers every time the target question is answered, thereby reducing the consumption of computing resources and the response time while ensuring the accuracy of the target answer.

[0012] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0014] Figure 1 is a flow chart of a question-answering method based on an intelligent question-answering agent provided according to an embodiment of the present invention;

[0015] Figure 2 is a flowchart of another question-answering method based on an intelligent question-answering agent provided according to an embodiment of the present invention;

[0016] Figure 3 is a flowchart of another question-answering method based on an intelligent question-answering agent provided according to an embodiment of the present invention;

[0017] Figure 4 is a flowchart of an optional example of another question-answering method based on an intelligent question-answering agent provided according to an embodiment of the present invention;

[0018] Figure 5 is a flowchart of another question-answering method based on an intelligent question-answering agent provided according to an embodiment of the present invention;

[0019] Figure 6 is a flowchart of an optional example of another question-answering method based on an intelligent question-answering agent provided according to an embodiment of the present invention;

[0020] Figure 7 is a structural block diagram of a question-answering device based on an intelligent question-answering agent provided according to an embodiment of the present invention;

[0021] Figure 8 It is a structural diagram of an electronic device for implementing the question-answering method based on an intelligent question-answering agent according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The situations of "target", "original", etc. are similar and will not be repeated here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] Before introducing the embodiments of the present invention, the implementation process of the currently adopted model-based question-answering solution and the reasons for the problems of large computing resource consumption and long response time are first exemplified to better understand why the solution proposed in the embodiments of the present invention reduces computing resource consumption and response time.

[0025] In the field of medical services, a large amount of question-and-answer data has been accumulated through the various questions raised by patients and their families in the hospital and the answers given by medical staff. If this accumulated knowledge can be applied to daily patient services, especially patient questions and answers, it will be of great value to improving the hospital's service quality and reducing service costs.

[0026] Traditional question-and-answer methods mainly directly answer questions based on the semantic matching results of the knowledge base composed of the question and question-and-answer data. For example, an intelligent question-and-answer system based on natural language processing first obtains the user input text and medical knowledge items and preprocesses them, extracts feature words and constructs feature vectors respectively, and then matches the feature vectors. The text is pushed to the user based on the matching results. In addition, the user's score for the push result during the management cycle is obtained, and the process of pushing text to the user in the next management cycle is optimized based on the score. However, this traditional method has limited context understanding ability and lacks a deep understanding of natural language. Therefore, it is easy to lose context in multiple rounds of dialogue and cannot provide coherent answers. In addition, the flexibility of answer generation is insufficient, and fixed answers can only be provided based on the preset answer library. If the user's question changes slightly or the question is asked in a different way, the correct answer may not be matched.

[0027] In recent years, with the development of generative artificial intelligence and large language models (LLMs), the retrieval-augmented generation (RAG) method that combines the powerful generation ability of large prediction models and the domain knowledge of knowledge bases has been increasingly used in question answering. For example, a medical-assisted question answering method based on knowledge calibration and retrieval enhancement combines medical expertise and hospital information to create a knowledge base, extracts features from each document fragment and node in the knowledge base through a fine-tuned Chinese semantic vector model, and then builds a vector database to obtain user questions and extract industry keywords. The user questions and industry keywords are integrated with the vector database for similarity matching, and the knowledge graph is combined to recall related knowledge data. The user questions and their industry keywords and related knowledge data are integrated to build a vector database. The prompt template is input into the large language model to obtain the answer to the user's question. However, this method generally includes modules such as question and answer library and vector library construction, related question retrieval, and large model generation of answers by integrating related question and answer information. However, the existing RAG-based question and answer method needs to frequently retrieve relevant context from the knowledge base when facing a large number of queries in order to generate accurate answers. However, this method has the problem of high computational cost, that is, each query requires context retrieval, resulting in a large consumption of computing resources. This method also has the problem of long response time, that is, frequent context retrieval increases the response time of the system and affects the user experience. This method also has the problem of fixed retrieval process, that is, the traditional retrieval process lacks flexibility and cannot dynamically adjust the retrieval strategy according to actual conditions, resulting in unnecessary context retrieval in some cases.

[0028] In summary, the currently used model-based question-answering method has the problems of large consumption of computing resources and long response time. It is difficult to reduce the consumption of computing resources and response time without affecting the accuracy of the answer.

[0029] In view of this, the embodiment of the present invention determines an action strategy, and when the action strategy is not to perform a determined action, does not perform an action to determine the relevant questions and answers of the target question, and directly determines the target answer of the target question based on the target question and the intelligent question-answering agent, so that it is not necessary to determine the relevant questions and answers every time the target question is answered in order to improve the accuracy of the target answer, thereby reducing computing resource consumption and response time while ensuring the accuracy of the target answer. This will be explained in detail below.

[0030] Figure 1It is a flow chart of a question-answering method based on an intelligent question-answering agent provided in an embodiment of the present invention. This embodiment can be applied to question-answering situations, and is particularly applicable to situations where it is required to answer the target questions raised. The method can be executed by a question-answering device based on an intelligent question-answering agent provided in an embodiment of the present invention, and the device can be implemented in software and / or hardware, and the device can be integrated on an electronic device, and the electronic device can be various user terminals or servers.

[0031] See also Figure 1 The method of the embodiment of the present invention specifically includes the following steps:

[0032] S110, obtaining a trained action strategy model and a target question raised by a target object, and inputting the target question into the action strategy model to obtain an action strategy output by the action strategy model.

[0033] The action strategy model is a model for determining whether to execute a certain action. The target object is the object that raises the target question. The target question is the question that needs to be answered based on the requirements raised by the target object. The action strategy is a strategy that characterizes whether to execute a certain action for the target question.

[0034] It is understandable that in the question-answering process, when processing consecutive target questions, if the intelligent question-answering agent has already obtained contextual questions and answers related to the target question as relevant questions and answers in the previous questions and answers, it means that the relevant questions and answers of these contexts have been passed to the intelligent question-answering agent. Even if the relevant questions and answers are no longer retrieved in the subsequent questions and answers, the intelligent question-answering agent can still provide the correct answer based on the previous data. There may also be a situation where the target object asks several similar target questions in succession, and the relevant questions and answers retrieved for these target questions are the same, and it is only necessary to retrieve the relevant questions and answers when asking the question for the first time. There may also be a situation where the target question inputs a question and answer related to the diagnosis and treatment service theme, etc., which is different from the embodiment of the present invention. For questions that are irrelevant to the topic involved, or questions that are irrelevant to the topic of at least one alternative question and answer used to determine the relevant question and answer, there is no need to retrieve the relevant questions and answers. In each of the above cases, it may not be necessary to obtain the relevant questions and answers, and the intelligent question and answer agent can directly generate the target answer that meets the requirements. Therefore, a policy-based agent method is constructed to optimize the question and answer process, that is, to obtain an action policy model located outside the intelligent question and answer agent, so as to determine whether to execute the action of determining the relevant question and answer of the target question based on the action policy output by the policy model, thereby optimizing the minimum number of units (tokens) input into the intelligent question and answer agent, that is, to reduce the use of tokens of the intelligent question and answer agent and improve the accuracy of the answer.

[0035] In the embodiment of the present invention, the target question can be processed by a prompt word, and the obtained first prompt word processing result is input into the action strategy model, and the action strategy model generates a corresponding action strategy according to the input first prompt word processing result. In the embodiment of the present invention, the method of inputting the target question into the action strategy model and obtaining the action strategy output by the action strategy model is not specifically limited.

[0036] S120. When the action strategy is not to perform a determination action, determine a target answer to the target question based on the target question and the intelligent question-answering agent, wherein the determination action is an action to determine questions and answers related to the target question.

[0037] Among them, the intelligent question-answering agent is an agent used to answer the target question; the intelligent question-answering agent can also be called a question-answering intelligent agent (Agent) intelligent body; the intelligent question-answering agent can include a question-answering model. The question-answering model can be understood as a pre-trained model that can answer the target question; the question-answering model can be, for example, a large model such as a large language model. In the embodiment of the present invention, the type, structure and parameters of the question-answering model are not specifically limited. The target answer is the answer obtained by solving the target question. Related questions and answers are a combination of related questions related to the target question and related answers to the related questions.

[0038] In an embodiment of the present invention, the action strategy model can determine whether to execute the action strategy of the Frequently Asked Questions (FAQ) retrieval action of the related questions and answers of the determined target question based on the input content. When the action strategy is to execute the determined action, the RAG process can be executed. When the action strategy is not to execute the determined action, the action of retrieving FAQ related questions and answers is not performed. The target question can be directly input into the intelligent question and answer agent, and the intelligent question and answer agent determines the target answer.

[0039] In an embodiment of the present invention, the target question can be processed by a prompt word, and the obtained second prompt word processing result is input into the trained question and answer model in the intelligent question and answer agent, and the question and answer model generates a corresponding answer according to the input second prompt word processing result, that is, the target answer to the target question can be determined according to the first output result of the question and answer model. In an embodiment of the present invention, there is no specific limitation on the method of determining the target answer to the target question according to the target question and the intelligent question and answer agent. It should be noted that since the embodiment of the present invention can be oriented to the question and answer of patients visiting the hospital, it is necessary to try to avoid the intelligent question and answer agent from generating "hallucination" content. Therefore, when constructing the prompt word, additional instructions can be added for prompting so that the intelligent question and answer agent can only answer using the provided data. For example, when constructing the prompt word, relevant content such as "please consult manual" can be added to output the output that cannot be answered. In an embodiment of the present invention, when the action strategy is not to perform a determined action, the first historical question and answer data can be obtained, and the target answer to the target question can be determined according to the first historical question and answer data, the target question and the intelligent question and answer agent.

[0040] It can be understood that the question-answering method based on the intelligent question-answering agent mentioned in the embodiment of the present invention can be applied to a question-answering system, for example, it can be applied to a diagnosis and treatment service question-answering system.

[0041] The technical solution of the embodiment of the present invention obtains the trained action strategy model and the target question raised by the target object, and inputs the target question into the action strategy model to obtain the action strategy output by the action strategy model, so as to determine whether to execute the action of determining the relevant questions and answers of the target question through the action strategy; when the action strategy is not to execute the determined action, the target answer of the target question is determined according to the target question and the intelligent question-answering agent, wherein the determined action is the action of determining the relevant questions and answers of the target question, so as to achieve the answer to the target question. The above technical solution determines the action strategy, and when the action strategy is not to execute the determined action, the action of determining the relevant questions and answers of the target question is not executed, and the target answer of the target question is determined directly according to the target question and the intelligent question-answering agent. In order to improve the accuracy of the target answer, there is no need to determine the relevant questions and answers every time the target question is answered, thereby reducing the consumption of computing resources and the response time while ensuring the accuracy of the target answer.

[0042] An optional technical solution, a question-answering method based on an intelligent question-answering agent, also includes: obtaining second historical question-answering data and historical strategies corresponding to the second historical question-answering data; inputting the target question into an action strategy model to obtain an action strategy output by the action strategy model, including: inputting the second historical question-answering data, historical strategies and the target question into the action strategy model to obtain the action strategy output by the action strategy model.

[0043] Among them, the second historical question and answer data are other questions and answers that have been asked before the target question; the second historical question and answer data may include, for example, other questions and answers before the target question in the conversation where the target question is located, and the second historical question and answer data may include, for example, a second preset number of other questions and answers before the target question in the conversation where the target question is located, and the second historical question and answer data may include, for example, a second preset number of other questions and answers before the target question, and so on; the second historical question and answer data and the second historical question and answer data may refer to the same data or different data. The historical strategy is a strategy that characterizes whether to perform a certain action on other questions in other questions and answers in the second historical question and answer data.

[0044] In an embodiment of the present invention, the input of the action strategy model may be a state, that is, the second historical question and answer data, the historical strategy (encoded as tokens) and the target question may be input into the action strategy model to obtain the action strategy output by the action strategy model. In an embodiment of the present invention, there is no specific limitation on the manner of inputting the second historical question and answer data, the historical strategy and the target question into the action strategy model to obtain the action strategy output by the action strategy model.

[0045] It should be noted that the second historical question and answer data and the first historical question and answer data can be the same. For example, the RAG pipeline of the intelligent question and answer agent uses the two most recent other questions and answers as the first historical question and answer data, and the action strategy model also uses the two most recent other questions and answers as the second historical question and answer data.

[0046] In an embodiment of the present invention, by inputting the second historical question and answer data, the historical strategy and the target question into the action strategy model to obtain the action strategy output by the action strategy model, the accuracy of the target answer can be improved.

[0047] Another optional technical solution is that the intelligent question-answering agent includes a question-answering model; according to the target question and the intelligent question-answering agent, a target answer to the target question is determined, including: according to the target question and the question-answering model, a target answer to the target question is determined.

[0048] In an embodiment of the present invention, the intelligent question-answering agent may include a question-answering model, and may determine a target answer to the target question based on the target question and the question-answering model. For example, the target question may be input into the question-answering model, and the target answer to the target question may be determined based on the output result of the question-answering model. The above scheme may achieve accurate determination of the target answer through the question-answering model.

[0049] Figure 2It is a flowchart of another question-answering method based on an intelligent question-answering agent provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the question-answering method based on the intelligent question-answering agent also includes: when the action strategy is to execute a certain action, determining the relevant questions and answers of the target question, and determining the target answer to the target question based on the relevant questions and answers, the target question and the intelligent question-answering agent.

[0050] The explanations of the terms that are the same as or corresponding to the above embodiments are not repeated here.

[0051] See also Figure 2 The method of this embodiment may specifically include the following steps:

[0052] S210: Acquire a trained action strategy model and a target question raised by a target object, and input the target question into the action strategy model to obtain an action strategy output by the action strategy model.

[0053] S220: When the action strategy is not to perform a determination action, determine a target answer to the target question based on the target question and the intelligent question-answering agent, wherein the determination action is an action to determine questions and answers related to the target question.

[0054] S230: When the action strategy is to execute a determined action, determine the relevant questions and answers of the target question, and determine the target answer to the target question based on the relevant questions and answers, the target question, and the intelligent question-answering agent.

[0055] In an embodiment of the present invention, the relevant questions and answers and the target question can be processed by prompt words, and the obtained third prompt word processing result is input into the trained question and answer model in the intelligent question and answer agent, and the target answer to the target question is determined according to the second output result of the question and answer model. In an embodiment of the present invention, the method of determining the target answer to the target question according to the relevant questions and answers, the target question and the intelligent question and answer agent is not specifically limited.

[0056] In an embodiment of the present invention, artificial feedback of the target answer can also be obtained. The artificial feedback is the feedback of the target object on whether the target answer is correct or satisfactory (which can be expressed by "thumbs-up" or "thumbs-down"). According to the artificial feedback, the action strategy model is trained. Specifically, according to the artificial feedback, the method of training the action strategy model can be the same as or different from the method of training the original strategy model. For example, the reward value of the target question can be determined according to the artificial feedback, and the action strategy model can be trained according to the reward value of the target question. Thereby, the artificial feedback result can be used in the reward calculation of the reinforcement learning model to form a closed loop of the system, thereby achieving the effect of continuously optimizing the question-answering process through artificial feedback.

[0057] In an embodiment of the present invention, when the action strategy is to execute a determined action, the first historical question and answer data can be obtained, and the target answer to the target question can be determined based on the first historical question and answer data, the related questions and answers, the target question, and the intelligent question and answer agent. The technical solution of an embodiment of the present invention, when the action strategy is to execute a determined action, determines the related questions and answers of the target question, and determines the target answer to the target question based on the related questions and answers, the target question, and the intelligent question and answer agent. When the related questions and answers need to be determined, the target answer can still be determined based on the determined related questions and answers, thereby ensuring the accuracy of the target answer.

[0058] An optional technical solution for determining relevant questions and answers of a target question includes: determining a target question vector of the target question, and determining a relevant question vector from at least one preset alternative question vector based on the target question vector; and using the alternative questions and answers corresponding to the relevant question vector as the relevant questions and answers of the target question.

[0059] Among them, the target question vector is a vector obtained by vector transformation of the target question. The candidate question vector is a vector obtained by vector transformation of the candidate questions. The candidate questions are questions in the candidate questions and answers. The candidate questions and answers are questions and answers of the candidate related questions and answers. The related question vector is a vector obtained by vector transformation of the related questions in the related questions and answers.

[0060] In the embodiment of the present invention, the target question can be vectorized by using a text embedding model such as the bge-small-zh-v1.5 model to obtain a target question vector. In the embodiment of the present invention, the method for determining the target question vector of the target question is not specifically limited.

[0061] In an embodiment of the present invention, at least one initial question vector may be recalled and sorted according to the target question vector, and a relevant question vector may be determined from at least one preset candidate question vector according to the operation result. Exemplarily, for each candidate question vector in at least one candidate question vector in the vector database, the vector distance between the target question vector and the candidate question vector may be determined, and at least one candidate question vector related to the target question may be determined from at least one candidate question vector according to the vector distances respectively corresponding to the at least one candidate question vector; and then, a sorting model such as a reranking model (BGE-Reranker) may be used to semantically rerank at least one candidate question vector, and the semantic reranking may be sorted according to the relevance to the target question, the completeness of the candidate questions and answers corresponding to the candidate question vectors, the commonness of the candidate questions and answers corresponding to the candidate question vectors, and so on; and the sorting result may be optimized, and a preset number of relevant question vectors may be determined from at least one candidate question vector after optimization and reranking, and the relevant question vector may be a preset number of candidate question vectors (Top-K) ranked before the at least one candidate question vector after optimization and reranking, and the preset number of questions may be, for example, 3. In the embodiment of the present invention, there is no specific limitation on the manner of determining the relevant question vector from at least one preset candidate question vector according to the target question vector.

[0062] In the embodiment of the present invention, the related questions corresponding to the related question vectors may be determined, and the candidate questions and answers to which the related questions belong may be determined from the FAQ question and answer database as the related questions and answers of the target question. In the embodiment of the present invention, the manner of using the candidate questions and answers corresponding to the related question vectors as the related questions and answers of the target question is not specifically limited.

[0063] In an embodiment of the present invention, relevant question vectors can be determined from at least one preset alternative question vector, and the alternative questions and answers corresponding to the relevant question vectors can be used as relevant questions and answers of the target question. The relevant questions and answers can be determined accurately and quickly, thereby further reducing computing resource consumption and response time while ensuring the accuracy of the target answer.

[0064] Based on the above scheme, another optional technical scheme, a question-and-answer method based on an intelligent question-and-answer agent, also includes: obtaining at least one alternative question and answer; for each alternative question and answer in the at least one alternative question and answer, performing vector conversion on the alternative question in the alternative question and answer to obtain an alternative question vector of the alternative question and answer, so as to obtain at least one alternative question vector.

[0065] In an embodiment of the present invention, multiple hospital diagnosis and treatment service questions and answers can be collected and organized as at least one alternative question and answer, and a FAQ question and answer library can be constructed based on the at least one alternative question and answer; for each alternative question and answer in at least one alternative question and answer in the FAQ question and answer library, the question text in the alternative question and answer is vectorized and converted into a semantic vector through a text embedding model to obtain an alternative question vector of the alternative question and answer, and the obtained at least one alternative question vector is stored in a vector database, which can be, for example, a FAISS vector database, so as to combine the generation energy of the intelligent question and answer agent with a large amount of local knowledge accumulated in medical services to intelligently answer diagnosis and treatment service-related questions raised by the target object. In an embodiment of the present invention, there is no specific limitation on the method of obtaining at least one alternative question and answer, and the method of vectorizing the alternative questions in the alternative question and answer to obtain the alternative question vector of the alternative question and answer.

[0066] In an embodiment of the present invention, at least one alternative question and answer may include multiple different types of alternative questions and answers. The type of at least one alternative question and answer may, for example, include at least one of medical insurance, appointment, triage, clinical department business, disease, drug, and indicator interpretation. Specifically, the medical insurance category is a category of common questions about medical insurance reimbursement, reimbursement ratio, and how to use the medical insurance card; the appointment category is a category of common questions about how to make an appointment online, how to cancel an appointment, and the difference between outpatient and examination appointments; the triage category is a category of common questions about how to choose a department, which department to go to for different symptoms, and the medical treatment process in an emergency; the clinical department business category is a category of common questions about the business scope of each department, the professional direction of the department doctors, etc.; the disease category is a category of common questions about the symptoms of a certain disease, treatment methods, whether hospitalization is required, etc.; the drug category is a category of common questions about the therapeutic effect of a certain drug, common side effects, interactions with other drugs, etc.; the indicator interpretation category is a category of common questions about what an abnormal laboratory indicator means and the need for further examination.

[0067] In an embodiment of the present invention, before the alternative questions in the alternative questions and answers are vectorized, the content text of at least one alternative question and answer in the FAQ question and answer library may be segmented and data cleaned to divide each alternative question and answer into a paragraph, and some meaningless content in at least one alternative question and answer may be filtered out by data cleaning, and at least one alternative question and answer may be updated based on the obtained segmentation and cleaning results, wherein the content text includes a question text and an answer text.

[0068] In an embodiment of the present invention, by obtaining at least one alternative question and answer, and for each alternative question and answer in the at least one alternative question and answer, performing vector conversion on the alternative questions in the alternative question and answer, an alternative question vector of the alternative question and answer is obtained to obtain at least one alternative question vector, thereby obtaining at least one alternative question vector for determining related questions and answers and the alternative questions and answers corresponding to the at least one alternative question vector, which helps to accurately and quickly determine related questions and answers.

[0069] Another optional technical solution is that the intelligent question and answer agent at least includes a calling tool module; based on relevant questions and answers, the target question and the intelligent question and answer agent, the target answer to the target question is determined, including: when the relevant questions and answers are empty, by calling the tool module, calling the transfer customer service tool to transfer the target question to the manual customer service, so that the manual customer service can determine the target answer to the target question.

[0070] The calling tool module is a module for calling the transfer customer service tool. The transfer customer service tool is a tool for transferring to manual customer service.

[0071] In an embodiment of the present invention, in order to avoid the problem that the intelligent question and answer agent generates an erroneous target answer due to the illusion of empty related questions and answers when the action strategy is to execute a certain action, the intelligent question and answer agent may at least include a calling tool module, so that when the relevant questions and answers are empty, the tool module is called to call the transfer customer service tool, and the target question is sent to the transferred manual customer service, so that the manual customer service can determine the target answer based on the target question.

[0072] In an embodiment of the present invention, by calling a tool module to transfer the call to manual customer service when the relevant questions and answers are empty, so that the manual customer service can answer the target question, it can be avoided that the intelligent question and answer agent generates wrong target answers when the action strategy is to execute a certain action, thereby improving the accuracy of the target question.

[0073] Figure 3 It is a flow chart of another question-and-answer method based on an intelligent question-and-answer agent provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the question-and-answer method based on an intelligent question-and-answer agent also includes: It also includes: obtaining first historical question-and-answer data; determining the target answer to the target question according to the target question and the intelligent question-and-answer agent, including: determining the target answer to the target question according to the first historical question-and-answer data, the target question and the intelligent question-and-answer agent; determining the target answer to the target question according to related questions and answers, the target question and the intelligent question-and-answer agent, including: determining the target answer to the target question according to the first historical question-and-answer data, related questions and answers, the target question and the intelligent question-and-answer agent.

[0074] The explanations of the terms that are the same as or corresponding to the above embodiments are not repeated here.

[0075] See also Figure 3 The method of this embodiment may specifically include the following steps:

[0076] S310: Obtain a trained action strategy model and a target question raised by a target object, and input the target question into the action strategy model to obtain an action strategy output by the action strategy model.

[0077] S320: Obtain first historical question and answer data.

[0078] Among them, the first historical question and answer data are other questions and answers that have been asked before the target question; the first historical question and answer data may, for example, include other questions and answers before the target question in the conversation where the target question is located, and the first historical question and answer data may, for example, include a first preset number of other questions and answers before the target question in the conversation where the target question is located, and the first historical question and answer data may, for example, include a second preset number of other questions and answers before the target question, and so on.

[0079] It is understandable that the target question may not be an isolated question, but may be a question in a group of sessions. In the session where the target question is located, there may be other questions that have been asked before the target question and other answers that have been answered to other questions. Therefore, each other question and answer in the session where the target question is located can be used as the first historical question and answer data, and the other questions and answers include a set of corresponding other answers and other questions and answers, so as to determine the target answer subsequently through the first historical question and answer data.

[0080] S330. When the action strategy is not to perform a determination action, determine a target answer to the target question based on the first historical question and answer data, the target question, and the intelligent question and answer agent, wherein the determination action is an action to determine the relevant questions and answers of the target question.

[0081] In an embodiment of the present invention, the target answer to the target question can be determined based on the first historical question and answer data, the target question, and the intelligent question and answer agent. For example, the first historical question and answer data and the target question can be processed by prompt words, etc. and then input into the question and answer model, and the target answer to the target question can be determined based on the third output result of the question and answer model. In an embodiment of the present invention, the method of determining the target answer to the target question based on the first historical question and answer data, the target question, and the intelligent question and answer agent is not specifically limited.

[0082] S340: When the action strategy is to execute a determined action, determine a target answer to the target question based on the first historical question and answer data, related questions and answers, the target question, and the intelligent question and answer agent.

[0083] In an embodiment of the present invention, the target answer to the target question can be determined based on the first historical question and answer data, the related questions and answers, the target question, and the intelligent question and answer agent. For example, the first historical question and answer data, the related questions and answers, and the target question can be processed by prompt words, etc. and then input into the question and answer model, and the target answer to the target question can be determined based on the fourth output result of the question and answer model. In an embodiment of the present invention, the method of determining the target answer to the target question based on the first historical question and answer data, the related questions and answers, the target question, and the intelligent question and answer agent is not specifically limited.

[0084] In an embodiment of the present invention, by also using the first historical question and answer data as a determining factor for determining the target answer, the accuracy of the target answer can be further improved. An optional technical solution, determining the target answer to the target question based on the first historical question and answer data, the target question, and the intelligent question and answer agent, includes: when the first historical question and answer data includes historical related data related to the target question, determining the target answer to the target question based on the first historical question and answer data, the target question, and the intelligent question and answer agent; determining the target answer to the target question based on the relevant questions and answers, the target question, and the intelligent question and answer agent, includes: when the first historical question and answer data includes historical related data related to the target question, or the relevant questions and answers are not empty, determining the target answer to the target question based on the first historical question and answer data, the relevant questions and answers, the target question, and the intelligent question and answer agent.

[0085] Among them, the historical relevant data is the historical question and answer data related to the target question.

[0086] In an embodiment of the present invention, a target answer to a target question may also be determined based on the first historical question and answer data, the target question, and the intelligent question and answer agent, including: when the first historical question and answer data includes historical relevant data related to the target question, the target answer to the target question is determined based on the first historical question and answer data, the target question, and the intelligent question and answer agent; and / or, the target answer to the target question is determined based on relevant questions and answers, the target question, and the intelligent question and answer agent, including: when the first historical question and answer data includes historical relevant data related to the target question, or the relevant questions and answers are not empty, the target answer to the target question is determined based on the first historical question and answer data, the relevant questions and answers, the target question, and the intelligent question and answer agent.

[0087] In an embodiment of the present invention, by determining the target answer when the first historical question and answer data includes historical relevant data related to the target question, and when the first historical question and answer data includes historical relevant data related to the target question or the relevant question and answer is not empty, it is possible to avoid the intelligent question and answer agent from generating incorrect target answers, thereby improving the accuracy of the target question.

[0088] Another optional technical solution, the intelligent question and answer agent at least includes a calling tool module; determining the target answer to the target question based on the first historical question and answer data, the target question and the intelligent question and answer agent, including: when the first historical question and answer data does not include historical related data related to the target question, calling the tool module to call the customer service transfer tool to transfer the target question and the first historical question and answer data to manual customer service, so that the manual customer service determines the target answer; determining the target answer to the target question based on relevant questions and answers, the target question and the intelligent question and answer agent, including: when the first historical question and answer data does not include historical related data related to the target question, and the relevant questions and answers are empty, calling the customer service transfer tool by calling the tool module to transfer the target question to manual customer service, so that the manual customer service determines the target answer.

[0089] In an embodiment of the present invention, the intelligent question and answer agent may at least include a calling tool module, so that when the first historical question and answer data does not include historical related data related to the target question, the intelligent question and answer agent may call the transfer customer service tool by calling the tool module, and send the target question and the first historical question and answer data to the transferred manual customer service, so that the manual customer service determines the target answer based on the target question and the first historical question and answer data; and, when the first historical question and answer data does not include historical related data related to the target question and the related question and answer is empty, the intelligent question and answer agent may call the transfer customer service tool by calling the tool module, and send the target question to the transferred manual customer service, so that the manual customer service determines the target answer based on the target question.

[0090] In an embodiment of the present invention, the target answer to the target question can also be determined based on the first historical question and answer data, the target question and the intelligent question and answer agent, including: when the first historical question and answer data does not include historical related data related to the target question, by calling the tool module, calling the customer service transfer tool to transfer the target question and the first historical question and answer data to manual customer service, so that the manual customer service determines the target answer; and / or, the target answer to the target question is determined based on relevant questions and answers, the target question and the intelligent question and answer agent, including: when the first historical question and answer data does not include historical related data related to the target question, and the relevant questions and answers are empty, by calling the tool module, calling the customer service transfer tool to transfer the target question to manual customer service, so that the manual customer service determines the target answer.

[0091] In an embodiment of the present invention, by calling a tool module to transfer the call to manual customer service when the first historical question and answer data does not include historical relevant data related to the target question, and when the first historical question and answer data does not include historical relevant data related to the target question and the relevant question and answer is empty, so that the manual customer service can answer the target question, it is possible to avoid the intelligent question and answer agent from generating incorrect target answers, thereby improving the accuracy of the target question.

[0092] In order to better understand the technical solution of the above embodiment of the present invention, an optional example is provided here. Figure 4 , when the action strategy is to perform a certain action, determining the target answer includes FAQ retrieval, answer generation, and question and answer library construction. Specifically, the question and answer library construction includes constructing a FAQ question and answer library of at least one alternative question and answer, and for each alternative question and answer in at least one alternative question and answer in the FAQ question and answer library, the question text in the alternative question and answer is vectorized and converted into a semantic vector through a semantic representation model to obtain an alternative question vector of the alternative question and answer, and the obtained at least one alternative question vector is stored in the vector library; FAQ retrieval includes vectorizing the question text in the target question through a semantic representation model and converting it into a semantic vector to obtain a target question vector, recalling at least one candidate question vector from at least one alternative question vector in the vector library, sorting at least one candidate question vector to determine the Top-K related question vectors from at least one candidate question vector, and taking the alternative questions and answers corresponding to the Top-K related question vectors as the Top-K related questions and answers ( Figure 4 Top-K questions and answers in the question answering process); answer generation includes, when the first historical question and answer data does not include historical related data and the related question and answer is empty, it means that the intelligent question and answer agent (Agent) cannot answer the question, calling the customer service transfer tool to transfer the target question to the manual customer service, and when the first historical question and answer data includes historical related data or the related question and answer is not empty, determining the target answer according to the first historical question and answer data, the related question and answer, the target question and the intelligent question and answer agent, and updating the first historical question and answer data according to the target question and the target answer, so as to facilitate subsequent question and answer through the first historical question and answer data.

[0093] Figure 5 It is a flow chart of another question-answering method based on an intelligent question-answering agent provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the action strategy model is pre-trained through the following steps: obtaining at least one sample question of the sample object; for each sample question in at least one sample question, determining the reward value of the sample question according to the sample question, the original strategy model and the intelligent question-answering agent; training the original strategy model according to the reward value corresponding to at least one sample question to obtain the action strategy model. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.

[0094] See also Figure 5 The method of this embodiment may specifically include the following steps:

[0095] S410: Obtain at least one sample question of a sample object.

[0096] The sample object is an object that raises sample questions; at least one sample question may belong to the same sample object or to different sample objects. The sample question is a question that needs to be answered based on the requirements raised by the sample object.

[0097] In the embodiment of the present invention, the method for obtaining at least one sample question of the sample object is not specifically limited.

[0098] S420. For each sample question in at least one sample question, determine a reward value of the sample question according to the sample question, the original strategy model, and the intelligent question-answering agent.

[0099] The original policy model is an untrained model of the policy used to determine whether to execute a certain action. The reward value is a signal used to guide the learning behavior of the intelligent question-answering agent, which is used to indicate how helpful the action executed by the sample policy output by the original policy model is in completing the task.

[0100] In an embodiment of the present invention, the original policy model can be a linear layer with exponential normalization (Softmax) activation added to the embedding of the special tag ([CLS] token) of the last layer of a pre-trained model such as a bidirectional encoding representation Transformer (Bidirectional Encoder Representations from Transformers, BERT) pre-trained model, mapping the state representation to a two-dimensional action space, adding two new tokens to the vocabulary of the pre-trained model, respectively representing the policy action execution determination action and non-execution of the determination action, where the execution of the determination action can be, for example, [Execute], and the non-execution of the determination action can be, for example, [Not Execute], and their embeddings are randomly initialized to obtain the original policy model.

[0101] In the embodiment of the present invention, for example, a sample answer to a sample question may be determined based on the sample question, the original policy model, and the intelligent question-answering agent, and a reward value for the sample question may be determined based on the sample answer. In the embodiment of the present invention, the method for determining the reward value for a sample question based on the sample question, the original policy model, and the intelligent question-answering agent is not specifically limited.

[0102] S430: Train the original strategy model according to the reward value corresponding to at least one sample question to obtain an action strategy model.

[0103] Exemplarily, considering that in reinforcement learning, the action strategy model is a model used to select actions, that is, a model for selecting whether to perform a certain action, and its goal is to maximize the cumulative reward (or return), and this goal is usually achieved by optimizing the loss function. Therefore, the quantized reward value can be set to r, and the at least one sample problem can be each sample problem in at least one sample session. For each sample session, the formula G t =r t+1 +γr t+2 +γ 2 r t+3 +…, determine the cumulative rewards and corresponding to each sample problem in the sample session, where t is the temporal order of the sample problem in the sample session, γ is the discount factor, indicating the attention paid to future rewards. In the embodiment of the present invention, more attention can be paid to immediate rewards, so γ can be set to a smaller value (0.1); the loss function of the original policy model can be the sum of the policy gradient loss and the entropy regularization loss, that is, it can be obtained through the loss function l t = -logπ θ (a t |s t )G t -λH(π θ (a t |s t )) and the cumulative rewards and corresponding to at least one sample problem, respectively, train the original policy model to obtain the action policy model, so that by combining the policy gradient loss and the entropy regularization loss, the action policy model can take into account the maximization of the return in the learning process, while maintaining the exploratory nature of the strategy and avoiding falling into the local optimal solution, where π θ (a t |s t ) is the output of the original strategy model, indicating that in state s t Next select action a t The probability of H(π θ (a t |s t ))=-∑ a π θ (a|s t )logπ θ (a|s t ) is the entropy of the strategy. Entropy is used here to measure the randomness of the strategy, that is, the uniformity of selecting different actions. A higher entropy indicates that the distribution of the strategy selection actions is more uniform and has greater exploratory power. λ is the weight of the entropy loss, which adjusts the degree of influence of entropy regularization on the overall loss. In the embodiment of the present invention, the original strategy model is trained according to the reward values ​​corresponding to at least one sample problem to obtain the action strategy model without specific limitation.

[0104] S440: Acquire the trained action strategy model and the target question raised by the target object, and input the target question into the action strategy model to obtain the action strategy output by the action strategy model.

[0105] S450: When the action strategy is not to perform a determination action, determine a target answer to the target question based on the target question and the intelligent question-answering agent, wherein the determination action is an action to determine questions and answers related to the target question.

[0106] The technical solution of the embodiment of the present invention determines the reward value corresponding to at least one sample problem, and trains the original strategy model according to the reward value corresponding to at least one sample problem to obtain an action strategy model, so that the accuracy of determining the action strategy of the trained action strategy model can be higher.

[0107] An optional technical solution determines the reward value of a sample question based on the sample question, the original strategy model, and an intelligent question-answering agent, including: inputting the sample question into the original strategy model, obtaining the sample strategy output by the original strategy model, and determining the predicted answer based on the sample question, the sample strategy, and the intelligent question-answering agent; obtaining answer feedback of the predicted answer, and determining the reward value of the sample question based on the answer feedback and / or the sample strategy.

[0108] The sample strategy is a strategy that indicates whether to perform a certain action on a sample question. The predicted answer is the answer predicted by the intelligent question-answering agent to the sample question. The answer feedback is feedback that indicates whether the predicted answer is correct.

[0109] In an embodiment of the present invention, third historical question and answer data of the sample question can also be obtained, and the sample question and the third historical question and answer data are input into the original policy model to obtain a sample policy output by the original policy model, wherein the third historical question and answer data are other sample questions and answers that have been asked before the sample question; the third historical question and answer data can, for example, include other sample questions and answers before the sample question in the session where the sample question is located; the third historical question and answer data can, for example, include a third preset number of other sample questions and answers before the sample question in the session where the sample question is located; the third historical question and answer data can, for example, include a third preset number of other sample questions and answers before the sample question, and so on.

[0110] In the embodiment of the present invention, there is no specific limitation on the method of inputting the sample problem into the original policy model to obtain the sample policy output by the original policy model.

[0111] In an embodiment of the present invention, when the sample strategy is not to perform a determined action, a predicted answer can be determined based on the sample question and the intelligent question-answering agent; when the sample strategy is to perform a determined action, the sample-related questions and answers of the sample question are determined, and the predicted answer is determined based on the sample-related questions and answers, the sample question and the intelligent question-answering agent. In an embodiment of the present invention, there is no specific limitation on the manner of determining the predicted answer based on the sample question, the sample strategy and the intelligent question-answering agent. It should be noted that the determined action mentioned in the process of training the action strategy model is the action of determining the sample-related questions and answers of the sample question.

[0112] In an embodiment of the present invention, a sample answer corresponding to a sample question may be obtained, and the answer feedback of the predicted answer may be determined manually or automatically based on at least one of the sample strategy, sample question, sample answer, predicted answer, and the third historical question and answer data. For example, the sample answer and the predicted answer may be compared to determine the answer feedback of the predicted answer; for another example, based on the sample question, the predicted answer, and the third historical question and answer data, the predicted answer may be given a "like" or "dislike" answer feedback. In an embodiment of the present invention, the method of obtaining the answer feedback of the predicted answer is not specifically limited.

[0113] In the embodiment of the present invention, the target feedback may be determined according to the answer feedback and / or the sample strategy, and the reward value of the sample question may be determined according to the target feedback. In the embodiment of the present invention, the method of determining the reward value of the sample question according to the answer feedback and / or the sample strategy is not specifically limited.

[0114] Exemplarily, it is possible to determine whether the action selected by the sample strategy is correct based on at least one of the sample strategy, sample question, sample answer, predicted answer, and third historical question and answer data. When the sample strategy selects an incorrect action, a target feedback of "step on" is provided. When the sample strategy selects a correct action, the answer feedback is used as the target feedback, and the preset value corresponding to the target feedback is used as the reward value for the sample question.

[0115] Optionally, when the sample strategy is not to perform a certain action and the answer feedback characterizes that the predicted answer is correct, the reward value is a first value; when the sample strategy is not to perform a certain action and the answer feedback characterizes that the predicted answer is wrong, the reward value is a second value; when the sample strategy is to perform a certain action, the reward value is a third value; wherein the first value is greater than the third value, and the third value is greater than the second value.

[0116] In an embodiment of the present invention, the answer feedback and / or sample strategy can be converted into a digital reward value (r). When the sample strategy is not to perform a certain action and the answer feedback represents that the predicted answer is correct, a first value which is a larger positive reward can be given, because this indicates that for the data input into the intelligent question-answering agent, there is no sample-related question and answer, and good output can still be generated; when the sample strategy is to perform a certain action, a second value which is a smaller positive reward is given; when the sample strategy is not to perform a certain action and the answer feedback represents that the predicted answer is wrong, a third value which is a negative reward is given. It should be noted that in order to minimize the cost of evaluation, only the case where the sample strategy is not to perform a determined action can be evaluated. That is, when the sample strategy is to perform a determined action, there is no need to determine whether the answer feedback represents that the predicted answer is correct, and a second value of a smaller positive reward is directly given. This is because the expected result of the complete question-answering process with the execution of the determined action is correct. Here, we hope to understand when not executing the determined action should be supported. If an incorrect result is generated after the execution of the determined action, it means that the internal question-answering process (intelligent question-answering agent or prompt word) needs to be improved, and the original strategy model outside the question-answering process cannot correct such errors. If the original strategy model chooses to execute the determined action, a smaller positive reward is directly given, and the answer feedback is no longer evaluated. As a result, the original strategy model must choose not to execute the determined action when it is fully confident that the correct answer can be obtained by not executing the determined action. Otherwise, it should choose not to execute the determined action, thereby further improving the accuracy of the determined action strategy of the trained action strategy model.

[0117] Exemplarily, referring to Table 1 below, when the sample strategy is not to perform a certain action and the answer feedback representation predicts that the answer is correct, the reward value is 2; when the sample strategy is not to perform a certain action and the answer feedback representation predicts that the answer is wrong, the reward value is -1; when the sample strategy is to perform a certain action, the reward value is 0.1.

[0118] Table 1 Reward value quantification

[0119]

[0120]

[0121] In an embodiment of the present invention, a sample strategy is determined, and a predicted answer is determined based on the sample question, the sample strategy, and the intelligent question-answering agent, and then answer feedback of the predicted answer is obtained, and a reward value for the sample question is determined based on the answer feedback and / or the sample strategy. The obtained reward value can be made more suitable for training the original strategy model, thereby further improving the accuracy of determining the action strategy of the trained action strategy model.

[0122] In order to better understand the technical solution of the above embodiment of the present invention, an optional example is provided here. Figure 6 , obtain the target question, the second historical question and answer data, the historical strategy and other states, and input the second historical question and answer data, the historical strategy and the target question into the action strategy model to obtain the action strategy output by the action strategy model; when the action strategy is not to perform a certain action, determine the target answer to the target question according to the target question and the intelligent question and answer agent; when the action strategy is to perform a certain action, retrieve related questions and answers, and determine the target answer to the target question according to the second historical question and answer data, the related questions and answers, the target question and the intelligent question and answer agent; obtain manual feedback on the target answer, which is represented by "like" or "dislike", and train the action strategy model based on the manual feedback. The above technical scheme can improve the service quality, that is, through the question-answering method based on the intelligent question-answering agent of the embodiment of the present invention, the hospital can more accurately and efficiently answer various inquiries of patients, reduce unnecessary context retrieval, ensure the response speed, and improve the patient's satisfaction and experience, thereby improving the overall service quality of the hospital; it can also reduce operating costs, that is, the question-answering method based on the intelligent question-answering agent of the embodiment of the present invention significantly reduces the computing resource consumption of the question-answering, and reduces the server load and operating costs by intelligently selecting whether to perform context retrieval. In addition, reducing dependence on manual evaluation also reduces labor costs, allowing hospitals to obtain higher benefits with lower investment; it can also improve patient trust and satisfaction, that is, through the question-answering method based on the intelligent question-answering agent of the embodiment of the present invention, faster and more accurate answers can be given, and patients can obtain the required medical information in the first time, reducing waiting time and information asymmetry, which not only improves patients' trust and satisfaction with the hospital, but also enhances patients' stickiness and improves the hospital's reputation.

[0123] Figure 7 This is a structural block diagram of a question-answering device based on an intelligent question-answering agent provided in an embodiment of the present invention. The device is used to execute the question-answering method based on an intelligent question-answering agent provided in any of the above embodiments. The device and the question-answering method based on an intelligent question-answering agent in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the question-answering device based on an intelligent question-answering agent, please refer to the embodiment of the question-answering method based on an intelligent question-answering agent. Figure 7 , the device may specifically include: an action strategy obtaining module 510 and a first target answer determining module 520.

[0124] Among them, the action strategy acquisition module 510 is used to obtain the trained action strategy model and the target question raised by the target object, and input the target question into the action strategy model to obtain the action strategy output by the action strategy model; the first target answer determination module 520 is used to determine the target answer to the target question according to the target question and the intelligent question and answer agent when the action strategy is not to perform a determined action, wherein the determined action is an action to determine the relevant questions and answers of the target question.

[0125] Optionally, the device also includes: a second target answer determination module, which is used to determine the relevant questions and answers of the target question when the action strategy is to execute a determined action, and determine the target answer to the target question based on the relevant questions and answers, the target question and the intelligent question and answer agent.

[0126] Optionally, based on the above-mentioned device, the second target answer determination module includes: a related question vector determination submodule, used to determine the target question vector of the target question, and determine the related question vector from at least one preset alternative question vector based on the target question vector; and a related question and answer as a submodule, used to use the alternative questions and answers corresponding to the related question vector as the related questions and answers of the target question.

[0127] Optionally, based on the above-mentioned device, the device also includes: an alternative question and answer acquisition module, used to obtain at least one alternative question and answer; an alternative question vector acquisition module, used to perform vector conversion on the alternative questions in the alternative questions and answers for each alternative question and answer in at least one alternative question and answer, and obtain an alternative question vector of the alternative question and answer, so as to obtain at least one alternative question vector.

[0128] Optionally, based on the above device, the intelligent question-answering agent at least includes a calling tool module;

[0129] The second target answer determination module includes: a transfer customer service tool calling submodule, which is used to call the transfer customer service tool by calling the tool module when the relevant questions and answers are empty, so as to transfer the target question to the manual customer service, so that the manual customer service can determine the target answer to the target question. Optionally, based on the above device, the device also includes: a first historical question and answer data acquisition module, which is used to acquire the first historical question and answer data; the first target answer determination module 520 includes: a first target answer determination submodule, which is used to determine the target answer to the target question based on the first historical question and answer data, the target question and the intelligent question and answer agent; the second target answer determination module includes: a second target answer determination submodule, which is used to determine the target answer to the target question based on the first historical question and answer data, the relevant questions and answers, the target question and the intelligent question and answer agent.

[0130] Optionally, based on the above-mentioned device, the first target answer determination submodule includes: a first target answer determination unit, which is used to determine the target answer to the target question based on the first historical question and answer data, the target question and the intelligent question and answer agent when the first historical question and answer data includes historical relevant data related to the target question; the second target answer determination module includes: a second target answer determination unit, which is used to determine the target answer to the target question based on the first historical question and answer data, the relevant questions and answers, the target question and the intelligent question and answer agent when the first historical question and answer data includes historical relevant data related to the target question, or the relevant questions and answers are not empty.

[0131] Optionally, based on the above-mentioned device, the intelligent question and answer agent at least includes a calling tool module; a first target answer determination sub-module, including: a first transfer customer service tool calling unit, which is used to call the transfer customer service tool by calling the tool module when the first historical question and answer data does not include historical related data related to the target question, so as to transfer the target question and the first historical question and answer data to manual customer service, so that the manual customer service can determine the target answer; a second target answer determination sub-module, including: a second transfer customer service tool calling unit, which is used to call the transfer customer service tool by calling the tool module when the first historical question and answer data does not include historical related data related to the target question and the relevant question and answer is empty, so as to transfer the target question to manual customer service, so that the manual customer service can determine the target answer.

[0132] Optionally, the device also includes: a historical strategy acquisition module, used to obtain the second historical question and answer data and the historical strategy corresponding to the second historical question and answer data; an action strategy acquisition module 510, including: an action strategy acquisition sub-module, used to input the second historical question and answer data, the historical strategy and the target question into the action strategy model to obtain the action strategy output by the action strategy model.

[0133] Optionally, the device also includes the following modules for pre-training to obtain the action strategy model: a sample question acquisition module, used to obtain at least one sample question of the sample object; a reward value determination module, used to determine the reward value of the sample question for each sample question in at least one sample question according to the sample question, the original strategy model and the intelligent question-answering agent; an action strategy model acquisition module, used to train the original strategy model according to the reward value corresponding to at least one sample question to obtain the action strategy model.

[0134] Optionally, based on the above-mentioned device, the reward value determination module includes: a predicted answer determination sub-module, which is used to input the sample question into the original strategy model, obtain the sample strategy output by the original strategy model, and determine the predicted answer based on the sample question, the sample strategy and the intelligent question and answer agent; a reward value determination sub-module, which is used to obtain answer feedback of the predicted answer, and determine the reward value of the sample question based on the answer feedback and / or the sample strategy.

[0135] Optionally, based on the above-mentioned device, when the sample strategy is not to perform a certain action and the answer feedback characterizes that the predicted answer is correct, the reward value is a first value; when the sample strategy is not to perform a certain action and the answer feedback characterizes that the predicted answer is wrong, the reward value is a second value; when the sample strategy is to perform a certain action, the reward value is a third value; wherein the first value is greater than the third value, and the third value is greater than the second value.

[0136] Optionally, the intelligent question-answering agent includes a question-answering model; the first target answer determination module 520 includes: a third target answer determination submodule, used to determine the target answer to the target question based on the target question and the question-answering model.

[0137] The question-answering device based on the intelligent question-answering agent provided by the embodiment of the present invention obtains the trained action strategy model and the target question raised by the target object through the action strategy obtaining module, and inputs the target question into the action strategy model to obtain the action strategy output by the action strategy model, so as to determine whether to execute the action of determining the relevant question and answer of the target question through the action strategy; through the first target answer determination module, when the action strategy is not to execute the determined action, the target answer of the target question is determined according to the target question and the intelligent question-answering agent, wherein the determined action is the action of determining the relevant question and answer of the target question, so as to achieve the answer to the target question. The above-mentioned device determines the action strategy, and when the action strategy is not to execute the determined action, does not execute the action of determining the relevant question and answer of the target question, and directly determines the target answer of the target question according to the target question and the intelligent question-answering agent, so as to improve the accuracy of the target answer without determining the relevant question and answer each time the target question is answered, thereby reducing the consumption of computing resources and the response time while ensuring the accuracy of the target answer.

[0138] The question-answering device based on the intelligent question-answering agent provided in the embodiment of the present invention can execute the question-answering method based on the intelligent question-answering agent provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0139] It is worth noting that in the above-mentioned embodiment of the question-answering device based on the intelligent question-answering agent, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0140] Figure 8 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0141] like Figure 8 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0142] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0143] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a question-answering method based on an intelligent question-answering agent.

[0144] In some embodiments, the question-answering method based on the intelligent question-answering agent may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the question-answering method based on the intelligent question-answering agent described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the question-answering method based on the intelligent question-answering agent by any other appropriate means (e.g., by means of firmware).

[0145] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0147] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0148] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0149] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0150] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0151] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0152] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A question-answering method based on an intelligent question-answering agent, characterized in that: include: Acquire a trained action strategy model and a target question raised by a target object, and input the target question into the action strategy model to obtain an action strategy output by the action strategy model; When the action strategy is not to perform a determination action, a target answer to the target question is determined according to the target question and the intelligent question-answering agent, wherein the determination action is an action to determine relevant questions and answers of the target question.

2. The method according to claim 1, characterized in that Also includes: When the action strategy is to execute the determined action, relevant questions and answers of the target question are determined, and a target answer to the target question is determined based on the relevant questions and answers, the target question and the intelligent question-and-answer agent.

3. The method according to claim 2, characterized in that The determination of the relevant questions and answers of the target question includes: Determining a target problem vector of the target problem, and determining a related problem vector from at least one preset candidate problem vector according to the target problem vector; The alternative questions and answers corresponding to the relevant question vectors are used as the relevant questions and answers of the target question.

4. The method according to claim 3, characterized in that Also includes: Get at least one alternative question and answer; For each alternative question and answer in the at least one alternative question and answer, vector transformation is performed on the alternative questions in the alternative question and answer to obtain an alternative question vector of the alternative question and answer, so as to obtain the at least one alternative question vector.

5. According to the method of claim 2, the intelligent question-answering agent at least comprises a calling tool module; Determining a target answer to the target question according to the relevant questions and answers, the target question, and the intelligent question-answering agent includes: When the related question and answer is empty, the calling tool module is used to call a customer service transfer tool to transfer the target question to a manual customer service, so that the manual customer service can determine a target answer to the target question.

6. The method according to claim 2, characterized in that Also includes: Get the first historical question and answer data; Determining a target answer to the target question based on the target question and the intelligent question-answering agent includes: Determining a target answer to the target question according to the first historical question and answer data, the target question, and the intelligent question and answer agent; Determining a target answer to the target question according to the relevant questions and answers, the target question, and the intelligent question-answering agent includes: A target answer to the target question is determined based on the first historical question and answer data, the related questions and answers, the target question, and the intelligent question and answer agent.

7. The method according to claim 6, wherein determining a target answer to the target question based on the first historical question-and-answer data, the target question, and the intelligent question-and-answer agent comprises: In a case where the first historical question and answer data includes historical related data related to the target question, determining a target answer to the target question according to the first historical question and answer data, the target question, and the intelligent question and answer agent; Determining a target answer to the target question according to the relevant questions and answers, the target question, and the intelligent question-answering agent includes: When the first historical question and answer data includes historical related data related to the target question, or the related question and answer is not empty, a target answer to the target question is determined based on the first historical question and answer data, the related question and answer, the target question and the intelligent question and answer agent.

8. According to the method of claim 6, the intelligent question-answering agent at least comprises a calling tool module; The step of determining a target answer to the target question according to the first historical question-and-answer data, the target question, and the intelligent question-and-answer agent includes: In the case where the first historical question and answer data does not include historical related data related to the target question, calling a customer service transfer tool through the calling tool module to transfer the target question and the first historical question and answer data to a manual customer service, so that the manual customer service determines the target answer; Determining a target answer to the target question according to the relevant questions and answers, the target question, and the intelligent question-answering agent includes: When the first historical question and answer data does not include historical related data related to the target question and the related question and answer is empty, the calling tool module is used to call the customer service transfer tool to transfer the target question to the manual customer service so that the manual customer service can determine the target answer.

9. The method according to claim 1, characterized in that: Also includes: Acquire second historical question and answer data and a historical strategy corresponding to the second historical question and answer data; The step of inputting the target problem into the action strategy model to obtain the action strategy output by the action strategy model includes: The second historical question-and-answer data, the historical strategy, and the target question are input into the action strategy model to obtain the action strategy output by the action strategy model.

10. The method according to claim 1, characterized in that The action strategy model is pre-trained through the following steps: Obtain at least one sample question of a sample object; For each sample question of the at least one sample question, determine a reward value of the sample question according to the sample question, the original strategy model and the intelligent question-answering agent; The original strategy model is trained according to the reward value corresponding to the at least one sample question to obtain the action strategy model.

11. The method according to claim 10, characterized in that Determining the reward value of the sample question according to the sample question, the original strategy model and the intelligent question-answering agent includes: Inputting the sample question into the original strategy model to obtain a sample strategy output by the original strategy model, and determining a predicted answer based on the sample question, the sample strategy and the intelligent question-answering agent; Obtain answer feedback of the predicted answer, and determine a reward value for the sample question based on the answer feedback and / or the sample strategy.

12. The method according to claim 11, characterized in that When the sample strategy is not to perform the determined action and the answer feedback indicates that the predicted answer is correct, the reward value is a first value; When the sample strategy is not to perform the determined action and the answer feedback indicates that the predicted answer is wrong, the reward value is a second value; In the case where the sample strategy is to execute the determined action, the reward value is a third value; The first value is greater than the third value, and the third value is greater than the second value.

13. The method according to claim 1, characterized in that The intelligent question-answering agent includes a question-answering model; Determining a target answer to the target question based on the target question and the intelligent question-answering agent includes: A target answer to the target question is determined based on the target question and the question-answering model.

14. A question-answering device based on an intelligent question-answering agent, characterized in that: include: An action strategy acquisition module is used to obtain a trained action strategy model and a target question raised by a target object, and input the target question into the action strategy model to obtain an action strategy output by the action strategy model; The first target answer determination module is used to determine the target answer to the target question based on the target question and the intelligent question-answering agent when the action strategy is not to perform a determination action, wherein the determination action is an action to determine the relevant questions and answers of the target question.

15. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the question-answering method based on an intelligent question-answering agent as described in any one of claims 1-13.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the question-answering method based on an intelligent question-answering agent as described in any one of claims 1 to 9 when executed.

17. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the question-answering method based on an intelligent question-answering agent according to any one of claims 1 to 13.

Images