Knowledge question and answer method and device, computer equipment and storage medium

Abstract

The embodiment of the invention provides a knowledge question answering method and device, computer equipment, a storage medium and a computer program product, and relates to the technical field of artificial intelligence. The method comprises the following steps: receiving a to-be-retrieved question sent by a retrieval terminal; obtaining a plurality of question and answer categories contained in a pre-constructed question and answer pair library; obtaining a target matching question corresponding to the to-be-retrieved question based on the to-be-retrieved question and the various question and answer categories by utilizing a large language model; and obtaining matching answer information corresponding to the to-be-retrieved question according to the target matching question. According to the method, the response accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a knowledge question answering method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] In the field of knowledge question answering, human-computer dialogue needs to be achieved, and the knowledge question answering system needs to be able to directly answer questions asked by users in natural language. The main indicator for evaluating the quality of automatic question answering is the correctness of the answer. Automatic question answering systems involve multiple different technologies, such as natural language processing, information retrieval, knowledge base and other technologies.

[0003] Among the current related technologies, some technologies use "retrieval recall + model refinement" to achieve it; other technologies use retrieval enhancement technology, both of which have the problem of low answer accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a knowledge question and answer method, device, computer equipment, storage medium and computer program product to address the above technical issues.

[0005] In a first aspect, the present application provides a knowledge question answering method. The method comprises:

[0006] Receiving the questions to be searched sent by the search terminal;

[0007] Get a variety of question and answer categories included in the pre-built question and answer library;

[0008] Using a large language model, based on the question to be retrieved and the multiple question and answer categories, obtaining a target matching question corresponding to the question to be retrieved;

[0009] According to the target matching question, matching answer information corresponding to the question to be retrieved is obtained.

[0010] In one of the embodiments, the use of a large language model to obtain a target matching question corresponding to the question to be retrieved based on the question to be retrieved and multiple question and answer categories includes: using the large language model to obtain at least one candidate question and answer category corresponding to the question to be retrieved based on the question to be retrieved and multiple question and answer categories; using a large language model to obtain the target matching question corresponding to the question to be retrieved based on the question to be retrieved and at least one candidate question and answer category.

[0011] In one embodiment, the using of a large language model to obtain the target matching question corresponding to the question to be retrieved based on the question to be retrieved and at least one of the candidate question and answer categories includes:

[0012] In the question-answer pair library, multiple candidate questions corresponding to at least one of the candidate question-answer categories are obtained; and using a large language model, based on the question to be retrieved and the multiple candidate questions, the target matching question corresponding to the question to be retrieved is obtained.

[0013] In one of the embodiments, the use of a large language model to obtain the target matching question corresponding to the question to be retrieved based on the question to be retrieved and multiple candidate questions includes: obtaining the relevance of each candidate question to the question to be retrieved; determining a question evaluation score for each candidate question based on the relevance; and determining the candidate question with the highest question evaluation score as the target matching question corresponding to the question to be retrieved.

[0014] In one of the embodiments, the use of the large language model to obtain at least one candidate question and answer category corresponding to the question to be retrieved based on the question to be retrieved and the multiple question and answer categories includes: inputting the question to be retrieved and the category name information of each of the multiple question and answer categories into the large language model to obtain the category screening result of the large language model; and obtaining at least one candidate question and answer category corresponding to the question to be retrieved based on the category screening result.

[0015] In one of the embodiments, obtaining at least one of the candidate question and answer categories corresponding to the question to be retrieved based on the category filtering result includes: if the category filtering result indicates that the large language model has filtered out the candidate question and answer categories, outputting at least one of the candidate question and answer categories; if the category filtering result indicates that the large language model has not filtered out the candidate question and answer categories, obtaining category association information of each of the multiple question and answer categories; and obtaining at least one of the candidate question and answer categories corresponding to the question to be retrieved based on the category association information.

[0016] In a second aspect, the present application provides a knowledge question answering device. The device comprises:

[0017] A receiving module, used for receiving the questions to be searched sent by the search terminal;

[0018] An acquisition module, used to acquire multiple question and answer categories contained in a pre-built question and answer pair library;

[0019] A screening module, used for obtaining a target matching question corresponding to the question to be retrieved based on the question to be retrieved and the multiple question and answer categories by using a large language model;

[0020] The answer output module is used to obtain the matching answer information corresponding to the question to be retrieved according to the target matching question.

[0021] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0022] Receiving the questions to be searched sent by the search terminal;

[0023] Get a variety of question and answer categories included in the pre-built question and answer library;

[0024] Using a large language model, based on the question to be retrieved and the multiple question and answer categories, obtaining a target matching question corresponding to the question to be retrieved;

[0025] According to the target matching question, matching answer information corresponding to the question to be retrieved is obtained.

[0026] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0027] Receiving the questions to be searched sent by the search terminal;

[0028] Get a variety of question and answer categories included in the pre-built question and answer library;

[0029] Using a large language model, based on the question to be retrieved and the multiple question and answer categories, obtaining a target matching question corresponding to the question to be retrieved;

[0030] According to the target matching question, matching answer information corresponding to the question to be retrieved is obtained.

[0031] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0032] Receiving the questions to be searched sent by the search terminal;

[0033] Get a variety of question and answer categories included in the pre-built question and answer library;

[0034] Using a large language model, based on the question to be retrieved and the multiple question and answer categories, obtaining a target matching question corresponding to the question to be retrieved;

[0035] According to the target matching question, obtain the matching answer information corresponding to the question to be retrieved.

[0036] In the above-mentioned knowledge question-answering method, apparatus, computer equipment, storage medium and computer program product, the questions to be retrieved sent by the retrieval terminal are received; multiple question-answering categories contained in a pre-built question-answering library are obtained; a large language model is used to obtain a target matching question corresponding to the question to be retrieved based on the question to be retrieved and the multiple question-answering categories; and matching answer information corresponding to the question to be retrieved is obtained according to the target matching question. In the method provided in the embodiment of the present application, a large model technical route is adopted, and when using the large model, the large model is not directly generated to generate answers, but the large model is selected from the pre-built question-answering library. The question that matches the user's intention is output to the user, and the answer corresponding to the question is output to the user, thereby improving the accuracy of the response. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0038] FIG1 is a schematic diagram of a flow chart of a knowledge question answering method provided in an embodiment of the present application;

[0039] FIG2 is a flow chart of another knowledge question answering method provided in an embodiment of the present application;

[0040] FIG3 is a schematic diagram of a process for obtaining a target matching problem provided by an embodiment of the present application;

[0041] FIG4 is a schematic diagram of another process for obtaining a target matching problem provided in an embodiment of the present application;

[0042] FIG5 is a schematic diagram of a process for screening at least one candidate question and answer category provided in an embodiment of the present application;

[0043] FIG6 is a schematic diagram of a process of cyclically calling a large language model according to an embodiment of the present application;

[0044] FIG7 is a structural block diagram of a knowledge question-answering device provided in an embodiment of the present application;

[0045] FIG8 is a diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0048] In an exemplary embodiment, as shown in FIG. 1 and FIG. 2 , a knowledge question answering method is provided. This embodiment is illustrated by applying the method to a server. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0049] Step 102: receiving the question to be searched sent by the search terminal.

[0050] Among them, the knowledge question and answer method provided in the embodiment of the present application can be applied to a knowledge question and answer system, and the knowledge question and answer system may include a retrieval terminal and a system server. The retrieval terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The user may register and log in to the knowledge question and answer system on the retrieval terminal, and the retrieval terminal displays an information query page in response to a login operation triggered by the user; the user may trigger an answer query request for the question to be retrieved on the information query page. The answer query request may include the question text information of the question to be retrieved, for example, the question to be retrieved may be how to apply for an account. The question text information may be in the form of keywords, natural language (eg, how to open an account) and / or entity-relationship structure.

[0051] Step 104: Acquire multiple question and answer categories contained in a pre-built question and answer pair library.

[0052] In this step, as shown in FIG. 2 , a question-answer pair library can be pre-built, and the question-answer pair library can include multiple sets of question-answer pairs, i.e., question-answer pairs. The multiple sets of question-answer pairs can be classified into categories to obtain multiple question-answer categories, such as new policy, information query, online payment, and claims. The construction of the question-answer pair library can include: the first step is to mine the initial question-answer pairs of the target business based on the historical business data of the target business. (1) The associated information of the target business can be obtained in the associated business system of the knowledge question-answer system, such as insurance terms, claims cases, business process documents, product descriptions, and other materials in the insurance industry. Extract the question-answer information of the target business from the associated information and convert it into the form of question-answer pairs; for example, (the question is "How much compensation can the insured get if he dies due to accidental injury?", and the answer is "The company pays the death benefit according to the insurance amount agreed in the contract.") (2) In the daily operation of the target business, the question-answer pairs related to the target business can be obtained; for example, construct question-answer pairs (the question is "How long is the cooling-off period of this insurance product?", and the answer is "The cooling-off period of this insurance product is 15 days.") (3) Obtain historical question-answer data in the knowledge question-answer system, and mine question-answer pairs related to the target business based on the historical question-answer data. The second step is to pre-process the questions in the initial question-answer pairs of the target business, standardize the question expressions, and classify and cluster the questions. Specifically, the collected questions are uniformly standardized and converted into a unified format to ensure that the language of the questions is clear, concise, and easy to understand, and avoid using vague, ambiguous, or overly professional words (unless specific field terms cannot be avoided). Classify and cluster the questions according to factors such as the subject, intention, and business field of the question. You can refer to the existing business classification system. For example, questions in insurance business can be divided into insurance, claims and inquiry categories. Questions with a similarity greater than the preset similarity threshold are classified into the same category. The third step is to pre-process the answers in the initial question-answer pair of the target business, for example, write and optimize the answers. Specifically, write accurate, complete and authoritative answers for each question. The answers should be based on reliable knowledge sources, such as internal company regulations, industry standards, laws and regulations, etc. For questions involving specific business processes or operations, the answers should describe the steps and precautions in detail. For example, for the question "How to apply for a claim?", the answer should include the specific process of the claim application (such as reporting, submitting materials, review process, etc.) and key information such as time nodes and material requirements that need to be paid attention to; convert the expression of the answer into a highly readable format. If the answer is more content, it can be organized in points and sections to improve the readability of the answer. For example, for complex claims process answers, detailed descriptions can be given step by step, such as "1. Report: After the accident occurs, call the customer service phone [specific number] in time to report the accident, and explain the basic situation such as the time, place and cause of the accident.2. Prepare materials: Prepare corresponding claim materials according to the type of accident, such as medical expense invoices, diagnosis certificates, original insurance policies, etc. (Detailed list of materials can be found on the official website or by consulting customer service). 3. Submit application: Submit the prepared materials to the designated claim service outlet or submit through the online claim platform. 4. Review process: The materials you submit will be reviewed, and the review time is generally [specific duration]. During this period, we may contact you to verify relevant circumstances. 5. Payment processing: After the review is passed, compensation will be paid in accordance with the contract agreement, and the compensation amount will be paid to your designated account within [specific time]. ". The fourth step is (1) selecting a suitable storage method. According to the architecture and requirements of the knowledge question-answering system, a suitable storage method is selected to store question-answer pairs. Databases (e.g., relational databases MySQL and non-relational databases MongoDB, etc.) can be used for storage, and questions and answers can be managed as different fields for easy query and retrieval. File storage (e.g., JSON format files, XML files, etc.) can also be used. (2) In order to improve the retrieval efficiency of question-answer pairs, a corresponding indexing mechanism is established, such as establishing a full-text index for the question field, so that relevant questions can be quickly located and matched when the question-answering system is running. At the same time, an effective question-answer pair management mechanism is established, including regular updates (e.g., timely updating of answers according to business changes, legal and regulatory adjustments, etc.), adding new question-answer pairs (continuously expanding the question-answer pair library as business develops and user needs change), deleting or modifying invalid or inaccurate question-answer pairs, etc., to ensure the quality and timeliness of the question-answer pair library. For example, when the terms of an insurance product change, the answers to the relevant question-answer pairs are updated in a timely manner; when a question is found to be no longer applicable or incorrectly stated, it is deleted or modified in a timely manner.

[0053] Step 106, using the large language model, based on the question to be retrieved and multiple question and answer categories, obtain the target matching question corresponding to the question to be retrieved.

[0054] Among them, first, multiple question and answer categories contained in the question and answer pair library can be obtained, and the large language model can be used to screen out at least one candidate question and answer category that matches the question to be retrieved from the multiple question and answer categories based on the question to be retrieved and the multiple question and answer categories. Specifically, the first step is to build an input data structure, and combine the category list composed of the extracted multiple question and answer categories and the question to be retrieved into an input format suitable for the large language model. For example, a data object containing a "category set" field and a "question to be retrieved" field is constructed, for example, using JSON format to represent: {"category set": ["new single insurance category", "claim problem category", "information query category", "online payment category"], "question to be retrieved": "I want to query my insurance policy information". The second step is to design the prompt words of the large language model, and carefully design the prompt words to guide the large model to perform the category screening task. Taking the insurance business as an example, the prompt words can contain the following key information: Role setting: clarify the role of the large language model, for example, "You are a professional insurance business consultant". Task description: Detailed description of the model's need to filter out the categories that best match the customer's questions from a given set of categories, for example, "Please select 1-3 categories from the provided set of categories that best match the customer's questions in terms of semantics, intent, and business scope." Input instructions: Tell the model the composition of the input data, for example, "The category set is [category list], and the question to be retrieved is [question text to be retrieved]." Output requirements: Specify the format and content requirements of the model output, for example, "Please output the filtered category names in a list format, with a number between 1 and 3." An example prompt is as follows: "You are a professional insurance business consultant. Now you are given a customer question and a category set. Your task is to select 1-3 categories from the category set that best match the customer question in terms of semantics, intent, and business scope. The category set is: [New policy category, claim problem category, information query category, online payment category], and the customer question is: [I want to query my insurance policy information]. Please output the filtered category names in a list, the number of which is between 1 and 3." The third step is large language model processing. After receiving the constructed input data and prompt words, the large language model begins the following processing: Semantic understanding and analysis: Deep semantic analysis of customer questions to identify key semantic elements, topic keywords, and intent expressions in the questions. For example, for the question "I want to query my insurance policy information", the model will extract key semantic content such as "query" and "insurance policy information". Category matching evaluation: For each category in the category set, the model performs matching evaluation based on the category name, semantic information related to the category in the pre-trained knowledge, and understanding of the customer's question. For the category names, their relevance to the key semantics of the customer questions is directly compared.For example, "information query" is highly relevant to "query insurance policy information" because the name directly reflects the query-related business; while "new policy insurance" is less relevant to the question because the question mainly revolves around existing policy information query rather than new policy insurance business. Using pre-trained knowledge, the model understands the specific meaning and scope of each category in the insurance business field, and further determines the degree of match with the customer's question. For example, the model knows that "information query" usually covers query operations for various insurance-related information, including policy information, claim progress, premium details, etc., thereby determining its high relevance to the customer's question. Screening and decision-making: Based on the matching evaluation results, the model screens out the category that best matches the customer's question from the category set. If the model determines that only one category has a very high degree of match with the customer's question, such as "information query" closely matches the customer's question in terms of semantics and business scope, and other categories have low relevance, then the model may only return this one category. If there are multiple categories with high relevance, such as "information query category" and "claim problem category" (assuming that the question to be retrieved may contain some information query intentions related to claims), but the relevance of "information query category" is slightly higher than that of "claim problem category", the model may select "information query category" as the main category according to a pre-set strategy (such as giving priority to the category with the highest relevance, or selecting a more common or more basic category when the relevance is similar), and may also return "claim problem category" to provide a more comprehensive possible related category. The number of candidate question and answer categories finally returned is between 1-3. Next, the question and answer questions contained in each of the at least one candidate question and answer category in the question and answer library can be obtained to obtain multiple candidate questions. Then, the large language model can be used to obtain the target matching question corresponding to the question to be retrieved based on the question to be retrieved and the multiple candidate questions. Specifically, the first step is to organize the candidate questions and the questions to be searched into an input structure that the big model can understand. For example, create a data object containing the "candidate question list" field and the "question to be searched" field, such as using JSON format to represent: {"candidate question list": ["What materials need to be submitted for claims?", "What is the process of claims?", "How long does it take to review claims?"], "question to be searched": "After my insurance claim application is submitted, how long will it take to receive the compensation?"}. The second step is to design the prompt words of the big language model and carefully write the prompt words to guide the big model to perform the screening task. Taking the insurance business as an example, the prompt words should include the following key parts: Role setting: clarify the role of the big model in this task, such as "You are a professional insurance business answer expert."Task description: clearly explain that the model needs to select one question from the candidate question list that best matches the question to be retrieved in terms of semantics, intent, and business logic. For example, "Please select a question from the provided candidate question list that best matches the question to be retrieved, and the best match is required in terms of semantic understanding, question intent, and insurance business relevance." Input instructions: inform the model of the composition of the input data, such as "the candidate question list is [candidate question set], and the question to be retrieved is [question text to be retrieved]." Output requirements: specify the format and content requirements of the model output, for example, "please directly output the best matching question that has been screened out." The following is an example prompt: "You are a professional insurance business answering expert. Now you are given a set of candidate questions and a question to be searched. Your task is to select a question from the candidate question list that best matches the question to be searched, and it is required to achieve the best match in terms of semantic understanding, question intent, and insurance business relevance. The candidate question list is: [What materials are required to submit for claims? What is the claim process? How long does it take to review claims?], and the question to be searched is: [How long will it take to receive the compensation after my insurance claim application is submitted?]. Please directly output the best matching question that has been screened out." The third step is large language model processing. After receiving the constructed input data and prompt words, the large language model performs the following operations: Semantic understanding and analysis: Perform deep semantic analysis on the question to be searched, and identify the key semantic elements, topic keywords, intent expressions, and logical relationships. For the question "How long will it take to receive the compensation after my insurance claim application is submitted?", the model will extract key semantic information such as "insurance claim application submission", "receiving compensation", and "time", and understand that customers are mainly concerned about information related to the claim time. Candidate question matching evaluation: For each question in the candidate question list, the model performs the following matching evaluation operations: Lexical level matching: Compare the proportion and importance of the same words in the candidate question and the question to be searched. For example, "How long does it take to review the claim?" has many common words with the question to be searched, such as "claim" and "time", which increases the possibility of matching. Semantic similarity calculation: Use semantic understanding technology to calculate the semantic similarity between the candidate question and the question to be searched. The model considers the semantic relationship of words, the similarity of sentence structure, and the consistency of semantic logic. For example, "How long does it take to review the claim?" and the question to be searched are both about time-related issues in the claim process in terms of semantic logic, and the similarity is high; while "What materials need to be submitted for claim?" is also related to claims, but the semantic focus is on the submission of materials, which is quite different from the semantic logic of the question to be searched. Intent matching judgment: Analyze whether the intent of the candidate question is consistent with the intent of the question to be searched. The model judges the intent behind each candidate question based on its understanding of the insurance business and pre-trained knowledge.For example, the intent of the question to be retrieved is obviously to inquire about the claim settlement time, and the intent of "How long does it take to review the claim?" is consistent with it, while the intent of "What is the claim settlement process?" is mainly to understand the process, which is not completely consistent with the intent of the question to be retrieved. Screening decision: Based on the matching evaluation results, the model calculates a comprehensive matching score for each candidate question, which takes into account factors such as vocabulary matching, semantic similarity, and intent matching. The model then selects the question with the highest score from the list of candidate questions as the question that best matches the customer's intent. For example, after calculation, "How long does it take to review the claim?" has the highest comprehensive matching score, and the model determines it as the best question finally screened out, that is, the target matching question corresponding to the question to be retrieved.

[0055] Step 108, according to the target matching question, obtain the matching answer information corresponding to the question to be retrieved.

[0056] Among them, after obtaining the target matching question corresponding to the question to be retrieved, the answer information corresponding to the target matching question can be obtained in the question and answer library, and the answer information can be used as the matching answer information corresponding to the question to be retrieved. Then, the matching answer information corresponding to the question to be retrieved can be output and displayed through the retrieval terminal.

[0057] In the method of this embodiment, the question to be retrieved sent by the retrieval terminal is received; multiple question and answer categories contained in the pre-built question and answer library are obtained; the target matching question corresponding to the question to be retrieved is obtained based on the question to be retrieved and the multiple question and answer categories using the large language model; and the matching answer information corresponding to the question to be retrieved is obtained according to the target matching question. In the method provided in the embodiment of the present application, the large model technology route is adopted, and when using the large model, the large model is not directly generated to generate the answer, but the large model is selected from the pre-built question and answer library. The question that matches the user's intention is output to the user, which improves the accuracy of the response.

[0058] In an exemplary embodiment, as shown in FIG3 , step 106 may include steps 302 to 304:

[0059] Step 302: using a large language model, based on the question to be retrieved and multiple question and answer categories, obtain at least one candidate question and answer category corresponding to the question to be retrieved.

[0060] Step 304: using the large language model, based on the question to be retrieved and at least one candidate question and answer category, obtain a target matching question corresponding to the question to be retrieved.

[0061] In an exemplary embodiment, as shown in FIG. 4 , step 304 may include steps 402 to 404:

[0062] Step 402: Obtain multiple candidate questions corresponding to at least one candidate question and answer category in the question and answer pair library.

[0063] Step 404 , using the large language model, based on the question to be retrieved and multiple candidate questions, obtain a target matching question corresponding to the question to be retrieved.

[0064] In an exemplary embodiment, step 404 may include:

[0065] Obtain the relevance of each candidate question to the question to be retrieved; determine the question evaluation score of each candidate question based on the relevance; and determine the candidate question with the highest question evaluation score as the target matching question corresponding to the question to be retrieved.

[0066] Specifically, the first step is to organize the candidate questions and questions to be searched into an input structure that the large model can understand. For example, create a data object containing a "candidate question list" field and a "question to be searched" field, such as using JSON format to represent it as follows: {"Candidate question list"":["What materials are required to submit a claim?", "What is the claims process?", "How long does it take to review a claim?"],"Question to be searched": "After I submit my insurance claim application, how long will it take for me to receive the compensation? ”}. The second step is to design the prompt words for the large language model and carefully write the prompt words to guide the large model to perform the screening task. Taking the insurance business as an example, the prompt words should include the following key parts: Role setting: clarify the role of the large model in this task, such as "You are a professional insurance business answering expert". Task description: clearly explain that the model needs to select one question from the candidate question list that best matches the question to be retrieved in terms of semantics, intent and business logic, such as "Please select a question from the provided candidate question list that best matches the question to be retrieved, and require the best match in terms of semantic understanding, question intent and insurance business relevance". Input instructions: inform the model of the composition of the input data, such as "The candidate question list is [candidate question set], and the question to be retrieved is [question text to be retrieved]". Output requirements: specify the format and content requirements of the model output, such as "Please directly output the best matching question screened out". The following is an example prompt word: "You are a professional insurance business answering expert. Now you are given a set of candidate questions and a question to be retrieved. Your task is to select a question from the candidate question list that best matches the question to be retrieved, and require the best match in terms of semantic understanding, question intent and insurance business relevance. The candidate question list is: [What materials need to be submitted for claims? What is the process of claims? How long does it take to review claims?], and the question to be searched is: [How long will it take to receive the compensation after my insurance claim application is submitted?] Please directly output the best matching question that has been screened out. "The third step is large language model processing. After receiving the constructed input data and prompt words, the large language model performs the following operations: Semantic understanding and analysis: Perform deep semantic analysis on the questions to be searched, and identify the key semantic elements, topic keywords, intention expressions, and logical relationships. For the question "How long will it take to receive the compensation after my insurance claim application is submitted?", the model will extract key semantic information such as "insurance claim application submission", "receiving compensation", and "time", and understand that customers are mainly concerned about information related to the claim time. Candidate question matching evaluation: For each question in the candidate question list, the model performs the following matching evaluation operations: Lexical level matching: Compare the proportion and importance of the same words in the candidate question and the question to be searched. For example, "How long will it take to review the claim? " has many common words with the question to be searched, such as "claim" and "time", which increases the possibility of matching.Semantic similarity calculation: Semantic understanding technology is used to calculate the semantic similarity between candidate questions and questions to be retrieved. The model considers the semantic relationship of words, the similarity of sentence structure, and the consistency of semantic logic. For example, "How long does it take to review the claim?" and the question to be retrieved are both about time-related issues in the claim process in terms of semantic logic, and have a high degree of similarity; while "What materials are required to submit for claim?" is also related to claim, but the semantic focus is on the submission of materials, which is quite different from the semantic logic of the question to be retrieved. Intent matching judgment: Analyze whether the intent of the candidate question is consistent with the intent of the question to be retrieved. The model judges the intent behind each candidate question based on its understanding of the insurance business and pre-trained knowledge. For example, the intent of the question to be retrieved is obviously to ask about the claim time, and the intent of "How long does it take to review the claim?" is consistent with it, while the intent of "What is the process of claim?" is mainly to understand the process, which is not completely consistent with the intent of the question to be retrieved. Screening decision: Based on the matching evaluation results, the model calculates a comprehensive matching score for each candidate question, that is, the question evaluation score, which comprehensively considers factors such as vocabulary matching, semantic similarity, and intent matching. The model then selects the question with the highest score from the candidate question list as the question that best matches the customer's intent. For example, after calculation, "How long does it take to review a claim?" has the highest comprehensive matching score, and the model determines it as the best question to be finally screened, that is, the target matching question corresponding to the question to be retrieved.

[0067] In the method of this embodiment, a large language model is introduced. When using the large language model, the large language model is not directly used to generate answers. Instead, the large model selects questions that match the user's intentions from a pre-built question-answer pair library and outputs the answers corresponding to the questions to the user, thereby improving the accuracy of the responses.

[0068] In an exemplary embodiment, as shown in FIG5 , step 302 may include steps 502 to 504:

[0069] Step 502: input the question to be retrieved and the category name information of each question and answer category in the multiple question and answer categories into the large language model to obtain the category screening result of the large language model.

[0070] Step 504: Obtain at least one candidate question and answer category corresponding to the question to be retrieved based on the category screening result.

[0071] In an exemplary embodiment, step 504 may include:

[0072] If the category filtering result indicates that the large language model has filtered out candidate question and answer categories, then output at least one candidate question and answer category; if the category filtering result indicates that the large language model has not filtered out candidate question and answer categories, then obtain category association information for each question and answer category in the multiple question and answer categories; based on the category association information, obtain at least one candidate question and answer category corresponding to the question to be retrieved.

[0073] As shown in FIG6 , it is a flowchart of looping and calling a large language model. The first step is to construct an input data structure, and combine the category list composed of multiple question and answer categories extracted and the questions to be retrieved into an input format suitable for the large language model. For example, a data object containing a "category set" field and a "question to be retrieved" field is constructed. For example, it is represented in JSON format as follows: {"category set": ["new policy insurance category", "claim problem category", "information query category", "online payment category"], "question to be retrieved": "I want to query my insurance policy information". The second step is to design the prompt words of the large language model, and carefully design the prompt words to guide the large model to perform category screening tasks. The third step is to input the question to be retrieved, the category list and the prompt word into the large language model, and obtain the category screening result of the large language model. The category screening result can be used to determine whether the large language model can accurately screen the category. If the category screening result indicates that the large language model has screened out the candidate question and answer category, then at least one candidate question and answer category is output; if the category screening result indicates that the large language model has not screened out the candidate question and answer category, then the category association information of each question and answer category in the multiple question and answer categories is obtained. Specifically, the question to be retrieved, the category list, the prompt word and the application programming interface (API) type and parameters can be input into the large language model, and the large language model is called again. At this time, the large language model has the ability to analyze and select APIs based on the input information to obtain more information that helps to screen the best category. Based on its understanding of the question to be retrieved and the category set, the large language model selects the API that it believes can obtain the most valuable information from the provided APIs. For example, if the large language model determines that the current question to be retrieved requires a deeper understanding of the specific meaning of a category for accurate screening, it may select the category meaning description API; if typical questions are needed to assist in judging the category, the category representative question API may be selected; if it is difficult to distinguish between several categories, the meaning distinction API of different categories may be selected. Further, the knowledge question answering system can obtain parameters based on the large language model selection and category name, etc. Specifically, the category meaning description API parameter acquisition: when the large language model selects the category meaning description API, the system accurately extracts the detailed meaning description content of the category from the category meaning description database according to the category name specified by the large language model, and passes it to the API as a parameter. For example, if the large language model selects the "claims problem" category, the system will obtain and pass the detailed meaning description of "claims problem" to the API. Category representative question API parameter acquisition: if the large language model selects the category representative question API, the system finds the category representative question list from the category representative question data source according to the category name specified by it, and passes it to the API as a parameter.For example, if the large language model specifies the category of "claims problem", the system will obtain a list of representative questions in the category of "claims problem" and provide it to the API. Obtaining API parameters for meaning differences of different categories: When the large language model selects the API for meaning differences of different categories, the system queries and obtains the detailed explanation of the difference in meaning between the two categories from the data structure storing the explanation of the difference in categories based on the two category names to be distinguished given by the large language model, and then passes it to the API as a parameter. For example, if the large language model chooses to distinguish "new policy insurance" and "claims problem", the system will obtain and pass the detailed explanation of the difference between the two categories to the API. Then, the knowledge question and answer system accurately calls the corresponding API based on the API selected by the large language model and the corresponding parameters obtained. After receiving the parameters, the API retrieves and returns the required information from its data source. For example, the meaning explanation API of the category returns the detailed meaning text of the specified category, the representative question API of the category returns the list of representative questions of the category, and the meaning difference API of different categories returns the explanation text of the difference in meaning between the two categories. The returned information will serve as an important input in the subsequent cycle calling the large language model to filter categories, to help the large language model filter categories more accurately. The cycle will be repeated up to 3 times. If the filter is still not possible after 3 times, the task fails.

[0074] In the method of this embodiment, when the large language model cannot accurately screen at least one candidate question and answer category corresponding to the question to be retrieved, category association information of each question and answer category in multiple question and answer categories can be obtained; based on the category association information, the large language model is called cyclically to perform multiple screenings, thereby improving the screening accuracy of the candidate question and answer categories.

[0075] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0076] Based on the same inventive concept, the embodiment of the present application also provides a knowledge question answering device for implementing the knowledge question answering method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more knowledge question answering device embodiments provided below can refer to the limitations of the knowledge question answering method above, and will not be repeated here.

[0077] In one embodiment, as shown in FIG. 7 , a knowledge question and answer device is provided, including: a receiving module 702, an acquisition module 704, a screening module 706 and an answer output module 708, wherein:

[0078] The receiving module 702 is used to receive the to-be-searched question sent by the search terminal;

[0079] An acquisition module 704 is used to acquire multiple question and answer categories contained in a pre-built question and answer pair library;

[0080] A screening module 706 is used to obtain a target matching question corresponding to the question to be retrieved based on the question to be retrieved and the multiple question and answer categories using a large language model;

[0081] The answer output module 708 is used to obtain matching answer information corresponding to the question to be searched according to the target matching question.

[0082] In one embodiment, the screening module 706 is further used to: utilize the large language model, based on the question to be retrieved and the multiple question and answer categories, obtain at least one candidate question and answer category corresponding to the question to be retrieved; utilize the large language model, based on the question to be retrieved and the at least one candidate question and answer category, obtain the target matching question corresponding to the question to be retrieved.

[0083] In one embodiment, the screening module 706 is further used to: obtain a plurality of candidate questions corresponding to at least one of the candidate question and answer categories in the question and answer pair library; and obtain the target matching question corresponding to the question to be retrieved based on the question to be retrieved and the plurality of candidate questions using a large language model.

[0084] In one embodiment, the screening module 706 is further used to: obtain the relevance between each of the candidate questions and the question to be retrieved; determine the question evaluation score of each of the candidate questions based on the relevance; and determine the candidate question with the highest question evaluation score as the target matching question corresponding to the question to be retrieved.

[0085] In one embodiment, the screening module 706 is further used to: input the category name information of the question to be retrieved and each of the multiple question and answer categories into the large language model to obtain the category screening result of the large language model; and obtain at least one of the candidate question and answer categories corresponding to the question to be retrieved based on the category screening result.

[0086] In one embodiment, the screening module 706 is further used to: if the category screening result indicates that the large language model has screened out the candidate question and answer categories, then output at least one of the candidate question and answer categories; if the category screening result indicates that the large language model has not screened out the candidate question and answer categories, then obtain category association information of each of the multiple question and answer categories; and based on the category association information, obtain at least one of the candidate question and answer categories corresponding to the question to be retrieved.

[0087] Each module in the above-mentioned knowledge question answering device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0088] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in FIG8. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store knowledge question and answer related data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a knowledge question and answer method is implemented.

[0089] Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0090] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0091] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0092] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0093] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0094] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0095] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A knowledge question answering method, characterized in that: The method comprises: Receiving the questions to be searched sent by the search terminal; Get a variety of question and answer categories included in the pre-built question and answer library; Using a large language model, based on the question to be retrieved and the multiple question and answer categories, obtaining a target matching question corresponding to the question to be retrieved; According to the target matching question, matching answer information corresponding to the question to be retrieved is obtained.

2. The method according to claim 1, characterized in that The using of the large language model to obtain a target matching question corresponding to the question to be retrieved based on the question to be retrieved and the multiple question and answer categories includes: Using the large language model, based on the question to be retrieved and the multiple question and answer categories, obtaining at least one candidate question and answer category corresponding to the question to be retrieved; The target matching question corresponding to the question to be retrieved is obtained by utilizing a large language model based on the question to be retrieved and at least one of the candidate question and answer categories.

3. The method according to claim 2, characterized in that The using of the large language model to obtain the target matching question corresponding to the question to be retrieved based on the question to be retrieved and at least one of the candidate question and answer categories includes: In the question-answer pair library, a plurality of candidate questions corresponding to at least one of the candidate question-answer categories are obtained; The target matching question corresponding to the question to be retrieved is obtained by utilizing a large language model based on the question to be retrieved and the plurality of candidate questions.

4. The method according to claim 3, characterized in that The using of the large language model to obtain the target matching question corresponding to the question to be retrieved based on the question to be retrieved and the plurality of candidate questions includes: Obtaining the relevance of each of the candidate questions to the question to be searched; Determining a question evaluation score for each of the candidate questions according to the relevance; The candidate question with the highest question evaluation score is determined as the target matching question corresponding to the question to be retrieved.

5. The method according to claim 2, characterized in that: The using the large language model to obtain at least one candidate question and answer category corresponding to the question to be retrieved based on the question to be retrieved and the multiple question and answer categories includes: Inputting the question to be retrieved and the category name information of each of the multiple question and answer categories into the large language model, and obtaining the category screening result of the large language model; At least one of the candidate question and answer categories corresponding to the question to be retrieved is obtained according to the category screening result.

6. The method according to claim 5, characterized in that The step of filtering the result according to the category to obtain at least one candidate question and answer category corresponding to the question to be retrieved includes: If the category screening result indicates that the large language model has screened out the candidate question and answer categories, outputting at least one of the candidate question and answer categories; If the category screening result indicates that the large language model does not screen out the candidate question and answer category, then obtaining category association information of each of the multiple question and answer categories; At least one of the candidate question and answer categories corresponding to the question to be retrieved is obtained according to the category association information.

7. A knowledge question-answering device, characterized in that: The device comprises: A receiving module, used for receiving the questions to be searched sent by the search terminal; An acquisition module, used to acquire multiple question and answer categories contained in a pre-built question and answer pair library; A screening module, used for obtaining a target matching question corresponding to the question to be retrieved based on the question to be retrieved and the multiple question and answer categories by using a large language model; The answer output module is used to obtain matching answer information corresponding to the question to be retrieved according to the target matching question.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images