Large model-based intelligent question answering method and system and storage medium

Abstract

The present invention relates to the technical field of artificial intelligence, and in particular to a large model-based intelligent question answering method and system and a storage medium. In the present invention, a user question is subjected to complexity evaluation by using a complexity evaluation function, and on the basis of a complexity evaluation result, different processing policies are used by means of dynamic planning and calling of an external tool to process the user question, thereby avoiding the phenomena that processing of simple questions is excessively complex and processing of complex questions is insufficient. By means of the mechanism of evaluating and iteratively optimizing the quality of a final answer, the phenomenon that the output final answer is unstable or has poor quality is avoided, and during knowledge extraction, the actual satisfaction of a user is fully considered, thereby improving the user experience.

Description

Intelligent question answering method and system based on large model and storage medium

[0001] This application claims priority to a Chinese patent application No.

[0002] 202411102673.8, entitled "Intelligent question answering method and system based on large model and storage medium", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to the field of artificial intelligence, in particular to an intelligent question answering method and system based on a large model and a storage medium. BACKGROUND

[0004] With the rapid development of artificial intelligence technology, large language models (LLM) have made significant breakthroughs in natural language processing, and these models have shown strong text understanding and generation capabilities, bringing new opportunities for the development of intelligent question answering systems. However, existing question answering systems based on large models generally use a unified method to process all types of questions, without adopting different processing strategies according to the complexity of the questions, which may lead to over-complexity in processing simple questions and insufficient processing of complex questions. SUMMARY

[0005] To solve the technical problems in the background art, the present application provides an intelligent question answering method and system based on a large model and a storage medium, which can adopt different processing strategies according to the complexity of the questions, thereby helping to solve the problem of over-complexity in processing simple questions and insufficient processing of complex questions.

[0006] To achieve the above technical solutions, in a first aspect, the present application provides an intelligent question answering method based on a large model, comprising the following steps:

[0007] Step one: pre-set a set of callable external tools, and describe the function of each external tool in the set of external tools;

[0008] Step two: classify the pre-set set of external tools according to the function of each external tool as described, to obtain a set of external tool functions;

[0009] Step three: use a complexity evaluation function to evaluate the complexity of the user's question and generate a complexity evaluation result, wherein the complexity evaluation result includes simple questions and professional questions;

[0010] Step four: adopt different processing strategies according to the generated complexity evaluation result to generate a final answer.

[0011] Further, the step four comprises:

[0012] If the complexity evaluation result is a simple question, a final answer is directly generated;

[0013] If the complexity evaluation result is a professional question, the following steps are performed:

[0014] A) retrieving knowledge entries related to the user question from a knowledge database;

[0015] B) planning answer generation steps based on the function set and the retrieved knowledge entries;

[0016] C) generating an external tool call plan for the planned answer generation steps, wherein the external tool call plan comprises the answer generation steps and external tools required to be called for each answer generation step;

[0017] D) evaluating the generated external tool call plan, if the plan evaluation result is reasonable, performing external tool calling according to the external tool call plan to generate a final answer, if the plan evaluation result is unreasonable, returning to step B).

[0018] Further, the performing external tool calling according to the external tool call plan to generate a final answer comprises:

[0019] According to the external tool call plan, external tools are sequentially called for each answer generation step;

[0020] The execution results of the external tools called for each answer generation step are evaluated, and an effect evaluation result is generated, wherein the effect evaluation result comprises an expected effect and an unexpected effect;

[0021] According to the evaluation result, a final answer is determined.

[0022] Further, the determining a final answer according to the evaluation result comprises:

[0023] If the expected effect is achieved, the output results of all answer generation steps are integrated as the final answer;

[0024] If there is an unexpected effect, the output results of all answer steps before the unexpected effect are integrated as the final answer.

[0025] Further, the step further comprises: performing quality evaluation on the generated final answer.

[0026] On the other hand, the present application provides an intelligent question and answer system based on a large model, comprising:

[0027] An interactive module for inputting a user question and displaying a final answer to the inputted user question;

[0028] A knowledge database for storing a plurality of knowledge entries, wherein a knowledge entry comprises a user question, answer generation steps for the user question, and external tools required to be invoked for each answer generation step;

[0029] An external tool set setting module for pre-setting an external tool set comprising a plurality of external tools;

[0030] A function classification module for classifying the plurality of external tools in the external tool set by functions to obtain a function set;

[0031] A complexity evaluation module for evaluating the complexity of the inputted user question and generating a complexity evaluation result;

[0032] A question and answer module for generating a final answer based on the complexity evaluation result by using different processing strategies.

[0033] Further, the question and answer module comprises:

[0034] A retrieval unit for retrieving a knowledge entry related to the user question from the knowledge database;

[0035] An answer generation step planning unit for planning answer generation steps;

[0036] An external tool invocation plan generation unit for generating an external tool invocation plan according to the planned answer generation steps;

[0037] A plan evaluation unit for evaluating the generated external invocation plan and generating a plan evaluation result.

[0038] In yet another aspect, the present application provides a computer-readable storage medium comprising a stored program, wherein the program, when executed, controls a device in which the computer-readable storage medium is located to perform the above-mentioned intelligent question and answer method based on a large model.

[0039] The present application has the following advantages:

[0040] (1) The present application evaluates the complexity of a user question by using a complexity evaluation function, and according to the complexity evaluation result, adopts different processing strategies for processing the user question by dynamic programming and invocation of external tools, thereby helping to avoid the phenomenon that simple problems are processed too complicatedly and complex problems are not processed sufficiently.

[0041] (2) By evaluating and iteratively optimizing the quality of the final answer, it helps to avoid the phenomenon that the output final answer is unstable or of poor quality.

[0042] (3) In the knowledge extraction process, the actual satisfaction of the user is fully considered, which helps to improve the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings, which form a part of the specification, are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification. The embodiments of the application, together with its description, serve to explain the application.

[0044] FIG. 1 is a flowchart of an intelligent question and answer method based on a large model according to the present application.

[0045] FIG. 2 is a flowchart of generating an answer when the evaluation result is a professional question according to the present application.

[0046] FIG. 3 is a flowchart of analyzing the dialogue history according to the present application. DETAILED DESCRIPTION

[0047] The application will be further described below with reference to the drawings and embodiments.

[0048] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the application. Unless otherwise defined, each technical and scientific term used in the present embodiment has the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0049] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and furthermore, it should be understood that when the terms "comprise" and / or "include" are used in the specification, there is a feature, step, operation, device, component and / or combination thereof.

[0050] In the present application, the terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom", etc. indicate the orientation or positional relationship shown in the drawings, which is only a relationship word determined for the convenience of describing the structural relationship of the components or elements of the present application, and cannot be understood as a limitation of the present application.

[0051] In the present application, the terms such as "fixedly connected", "connected", "connected" should be understood broadly, which means that it can be fixedly connected, integrally connected or detachably connected; it can be directly connected or indirectly connected through an intermediate medium. For relevant researchers or technicians in the art, the specific meaning of the above terms in the present application can be determined according to the specific circumstances, and cannot be understood as a limitation of the present application.

[0052] Example 1:

[0053] As shown in Figure 1, this embodiment provides an intelligent question-answering method based on a large model, including the following steps:

[0054] S1: Predefine the set of external tools that can be called, and describe the functionality of each external tool in the set.

[0055] For ease of explanation and understanding, the predefined set of callable external tools is represented as T = {t1, t2, ..., t}. n}, where t i Let represent the i-th tool in the external tool set T, where 1 ≤ i ≤ n.

[0056] The function and invocation method of each external tool in the external toolset are described. The external toolset includes at least one external tool.

[0057] S2: Classify the predefined set of external tools according to the function of each external tool to obtain the function set of external tools.

[0058] Clustering algorithms are used to classify each external tool in the external toolkit according to its functional similarity, resulting in a function set, which is represented as F = {f1, f2, ..., f...}. m}, where f j Let f represent the j-th function in the function set, where 1 ≤ j ≤ m and m ≤ n. For example, tools t1 and t3 belong to function f1 in the function set.

[0059] S3: Use a complexity evaluation function to evaluate the complexity of the user's problem and generate complexity evaluation results, which include simple problems and complex problems.

[0060] For ease of understanding and explanation, we denote the user problem as user problem Q. Using a complexity evaluation function, we analyze the problem based on factors such as problem length, keyword complexity, syntactic structure complexity, and domain specialization to obtain a complexity evaluation result for the user problem:

[0061] Where C(Q) is the problem complexity evaluation function, which is a complexity evaluation function based on a large model. By sending a specific prompt to the large model, the large model will analyze the problem according to the guidance given in the prompt and return a score of 0 or 1, where 0 represents a simple problem and 1 represents a complex problem requiring specialized knowledge. The specific implementation is as follows:

[0062] Example of prompt:

[0063] You are a professional question complexity assessment expert. You need to judge the complexity of the user's question and give a score of 0 or 1, where 0 represents a simple question and 1 represents a complex question that requires professional knowledge.

[0064] When evaluating, consider the following factors:

[0065] 1. The length and structure of the question;

[0066] 2. Whether it contains professional terms or concepts;

[0067] 3. The range of knowledge required to answer the question;

[0068] 4. Whether it requires multi-step reasoning or calculation.

[0069] Please analyze the user's question and give your score (0 or 1): Question: [User Question Q].

[0070] S4: Based on the generated complexity assessment result, different processing strategies are adopted to generate the final answer.

[0071] Specifically, the following steps are included:

[0072] S41: If the complexity assessment result is a simple question, directly use the large model to generate the final answer.

[0073] Specifically, if C(Q) = 0, i.e., the complexity assessment result is a simple question, directly use the pre-trained large model to generate the final answer, represented as A, i.e., A = M(Q), where M represents the large model function.

[0074] Where the large model adopts the Transformer architecture and is pre-trained and fine-tuned on a large amount of historical text data to obtain the pre-trained large model.

[0075] S42: If the assessment result is a professional question, perform the following steps as shown in Figure 2:

[0076] Q1: Retrieve knowledge items related to the user's question from the knowledge database.

[0077] Where the knowledge items include user question vectors, answer generation steps for user question vectors, and external tools to be called for user questions.

[0078] Convert the user question Q into a user question vector (user question vector represented as V Q ), calculate the similarity between V Q and the user question vector of each knowledge item in the knowledge database, and select the one with the highest similarity. QThe most relevant preset number (e.g., K) of knowledge items.

[0079] Q2: Based on the set of functionalities of external tools and the retrieved knowledge items, plan the answer generation steps.

[0080] Based on the set of functionalities F and the retrieved K knowledge items, use a large model to generate a sequence of steps required to answer the user's question, denoted as S = {s1, s2,..., sK}, where sk represents the kth step, and sk ∈ {s1, s2,..., sK}. L} where sk represents the kth step, and sk ∈ {s1, s2,..., sK}. k} where sk represents the kth step, and sk ∈ {s1, s2,..., sK}. k} where sk represents the kth step, and sk ∈ {s1, s2,..., sK}. L} where sk represents the kth step, and sk ∈ {s1, s2,..., sK}.

[0081] prompt example:

[0082] You are a senior AI assistant responsible for developing a step-by-step plan to solve complex problems based on the given question and relevant knowledge items. You need to develop a detailed step-by-step plan to answer the question, ensuring that your plan is logical, comprehensive, and executable.

[0083] Question: [Insert user question here]

[0084] Retrieved knowledge items related to the user's question:

[0085] 1. [Insert the first relevant knowledge item retrieved here];

[0086] 2. [Insert the second relevant knowledge item retrieved here];

[0087] 3. [Insert the third relevant knowledge item retrieved here];

[0088] ...

[0089] Available external tool functionalities: [Insert a description of the available external tool functionalities here]

[0090] Based on the above information, develop a detailed step-by-step plan to answer the user's question. Your plan should include the following:

[0091] 1. Step number;

[0092] 2. Step description: Clearly describe the task that needs to be completed in this step;

[0093] 3. Required functionality category: Choose the most suitable one from the above functionality categories;

[0094] 4. Expected output: Describe the result that should be obtained after completing this step;

[0095] 5. Relationship to next step: Explain how the output of this step is used in the next step.

[0096] Please ensure your step plan:

[0097] - makes full use of the relevant knowledge retrieved;

[0098] - is logically ordered and each step is necessary;

[0099] - considers possible exceptions and alternatives;

[0100] - is of a reasonable number of steps, usually not less than 3 and not more than 10.

[0101] Now, please start developing your step plan:

[0102] Step plan:

[0103] 1. Step number: 1

[0104] Step description:

[0105] Required function category:

[0106] Expected output:

[0107] Relationship to next step:

[0108] 2. Step number: 2

[0109] Step description:

[0110] Required function category:

[0111] Expected output:

[0112] Relationship to next step:

[0113] 3. Step number: 3

[0114] Step description:

[0115] Required function category:

[0116] Expected output:

[0117] Relationship to next step:

[0118] ...

[0119] Finally, please briefly summarize how this step plan effectively addresses the original problem and the main challenges that may be faced.

[0120] Q3: For the planned answer generation steps, generate an external tool call plan, which includes the planned answer generation steps and the external tools that need to be called for each answer generation step, and arranges each answer generation step and its required external tools accordingly.

[0121] Specifically, for each answer generation step s k , where s k ∈{s1,s2,...,s L Using an attention-based tool selection model, the matching score between each answer generation step and each external tool in the external toolkit is calculated. The external tool with the highest matching score is selected as the tool to be used in that step. This process is repeated step by step to form an external tool invocation plan, which is represented as P = {(s1,t1),(s2,t2),…,(s...}. k ,t i )}.

[0122] It should be noted that P = {(s1,t1),(s2,t2),…,(s...} k ,t i The above is merely an example for the purpose of illustration and understanding.

[0123] Q4: Evaluate the generated external tool invocation plan. If the plan evaluation result is reasonable, execute the external tool invocation according to the plan and generate the final answer. If the plan evaluation result is unreasonable, return to step Q2.

[0124] The specific steps are as follows:

[0125] Q41: Use a rule-based and large-model-based tool invocation plan evaluation function to assess the rationality of the external tool invocation plan and generate the following plan evaluation results:

[0126] Here, E(P) is the tool call plan evaluation function, and the evaluation criteria for E(P) include the logicality of the steps, the accuracy of the tool selection, and the completeness of the plan.

[0127] For example, the following is a Prompt example that evaluates the reasonableness of external tool invocation plans for a rule-based and large-model-based tool invocation plan evaluation function:

[0128] You are a professional tool usage plan evaluation expert. You need to evaluate whether a given tool usage plan is reasonable and give a score of 0 or 1, where 0 means the plan is unreasonable and 1 means the plan is reasonable.

[0129] When evaluating, please consider the following factors:

[0130] 1. The logical order of steps is correct

[0131] 2. The tools selected for each step are appropriate

[0132] 3. The plan is complete and addresses the user's problem

[0133] 4. There are no redundant or unnecessary steps

[0134] Please analyze the following tool invocation plan and give your score (0 or 1):

[0135] Original question: [User question Q]

[0136] External tool invocation plan:

[0137] [Insert detailed content of external tool invocation plan P here]

[0138] Score:

[0139] Reason:

[0140] Q41-1: If E(P) = 1, i.e., the plan evaluation result is reasonable, then according to the external tool invocation plan, invoke external tools for the answer generation steps to generate the answer.

[0141] Specifically includes the following steps:

[0142] A1: According to the external tool invocation plan, invoke external tools for the answer generation steps in turn.

[0143] According to the correspondence between each answer generation step in the external tool invocation plan and the external tool, invoke external tools for the answer generation steps in the order specified in the external tool invocation plan.

[0144] Specifically, for example, according to the external tool invocation plan P = {(s1, t1), (s2, t2), …, (s k , t i )}, the execution order is: first invoke external tool t1 for answer generation step s1, then invoke external tool t2 for answer generation step s2, and so on.

[0145] A2: Effectively evaluate the execution results of the invoked external tools.

[0146] The execution results of the invoked external tools ti for answer generation steps si are represented as ri = ti(si). k k k i k ​​​​), and using the effect evaluation function to judge the execution result r k whether the expected effect is achieved, which is specifically represented as follows:

[0147] wherein, W(r k ) is an effect evaluation function, which is evaluated based on predefined effect indicators and machine learning models.

[0148] The execution result is, for example, a weather query tool, and the execution result is the queried weather information.

[0149] A2-1: If W(r k ) is all 1, i.e. the execution result of the external tool called by each answer step in the external tool calling plan reaches the expected effect, the output results of all answer generation steps are integrated to generate the final answer, which is denoted as A_final.

[0150] A2-2: If there is W(r k ) = 0, i.e. when each external tool corresponding to each answer generation step is called and executed according to the external tool calling plan, the execution result does not reach the expected effect, the calling and execution are stopped, and the output results of all answer generation steps before this answer generation step are integrated as the final answer A_final.

[0151] For example, the external tool calling plan P = {(s1, t1), (s2, t2), …, (s k , t i )}, first answer generation step s1, call external tool t1, and evaluate the execution result of external tool t1 in answer generation step s1, if the expected effect is reached, continue to call external tool t2 for answer generation step s2, and evaluate the execution result of external tool t2 in answer generation step s2, if the expected effect is reached, continue to call external tool t3 for answer generation step s3, and evaluate the execution result of external tool t3 in answer generation step s3, if the expected effect is not reached, stop calling and executing, integrate the output results of steps s1, s2 and s3 as the final answer.

[0152] wherein, the Prompt example of effect evaluation is:

[0153] You are a professional tool execution effect evaluation expert. You need to evaluate whether the given tool execution result reaches the expected effect, and give a score of 0 or 1, where 0 means not reaching the expected effect, and 1 means reaching the expected effect.

[0154] When evaluating, please consider the following factors:

[0155] 1. Did the execution result answer the goal of the current step?

[0156] 2. How accurate and complete was the result?

[0157] 3. Did the result provide the necessary information for the next step?

[0158] Please analyze the tool execution result and give your score (0 or 1):

[0159] Original question: [Insert user question Q here]

[0160] Current step: [Insert description of the current step s k here]

[0161] Tool used: [Insert name and functional description of the tool t i used here]

[0162] Execution result: [Insert tool execution result r k here]

[0163] Score:

[0164] Reason:

[0165] Q41-2: If E(P) = 0, i.e., the plan evaluation result is unreasonable, return to step Q2 to re-plan the steps.

[0166] Q5: Quality assessment of the generated final answer.

[0167] Use the quality assessment function to evaluate the quality of the final answer to obtain the following quality assessment of the result:

[0168] Where Q(A_final) is the quality assessment function.

[0169] If Q(A_final) = 0, the quality of the generated final answer does not meet the requirements, return to step Q1 for iterative optimization until a satisfactory answer is obtained or the maximum number of iterations is reached, where the maximum number of iterations is pre-set.

[0170] Q6: Optimization and maintenance.

[0171] Q61: Regularly clean up and update the knowledge database, delete outdated or low-quality knowledge entries.

[0172] Q62: Continuously monitor system performance, including answer accuracy, user satisfaction, and response time, etc.

[0173] Q63: Continuously optimize the algorithms and models of each module based on user feedback and system logs.

[0174] Q64: Regularly update the external toolset and adjust the tool classification and selection strategy accordingly.

[0175] Embodiment 2:

[0176] The difference between this embodiment and embodiment 1 is that, as shown in FIG. 3, the method further comprises analyzing the dialogue history and extracting knowledge items from the dialogue history for storage into the knowledge database,

[0177] wherein the dialogue history is the dialogue question-answer data generated by the user using the process of generating answers according to questions, which is recorded into the dialogue history database.

[0178] The specific steps are as follows:

[0179] D1: Analyze the dialogue history.

[0180] For ease of understanding and description, the dialogue history H is represented as a series of role-content pairs: H = {(role1, content1), (role2, content2),..., (role i , content i )}, where role i ∈{user, agent}, content i represents the corresponding dialogue content.

[0181] When Role is user, the dialogue content is the question and requirement raised by the user; when role is agent, the dialogue content is the tool invocation plan and the final answer generated by the agent.

[0182] D2: Segment the dialogue history.

[0183] Using algorithms based on topic modeling and dialogue behavior analysis, the dialogue history H is segmented into multiple relatively independent paragraphs G = {g1, g2,..., g j}, where g j represents multiple question-answers about a topic.

[0184] Specifically, the dialogue history may contain a long sequence of user question 1, agent answer 1, user question 2, agent answer 2, user question 3, agent answer 3, etc., which may contain multiple different topics, for example, user question 1 and 2 and their corresponding answers are about topic 1, and later user question 3 and the corresponding answer are about topic 2. The segmentation here is to identify the topics in the dialogue and segment the dialogue history according to the topics. This process can be performed using the TADAM model.

[0185] D3: Satisfaction evaluation.

[0186] For each passage g j , use the satisfaction evaluation function S(g j ) based on rules and sentiment analysis to determine whether the user is satisfied and generate a satisfaction evaluation result:

[0187] Where the evaluation criteria include: user's explicit expression of satisfaction, user's emotional tendency, coherence and completeness of the dialogue.

[0188] Satisfaction evaluation Prompt example:

[0189] You are a professional user satisfaction evaluation expert. You need to evaluate whether the user is satisfied with the answers given by the intelligent agent in the given dialogue passage and give a score of 0 or 1, where 0 means the user is not satisfied and 1 means the user is satisfied.

[0190] When evaluating, please consider the following factors:

[0191] 1. User's explicit feedback (such as expressing thanks or affirmation);

[0192] 2. User's emotional tendency (judged by tone and words);

[0193] 3. Coherence and completeness of the dialogue;

[0194] 4. Whether the user asks follow-up questions or requests clarification;

[0195] Please analyze the following dialogue passage and give your score (0 or 1):

[0196] Dialogue passage:

[0197] [Insert the complete content of the dialogue passage gj here, including the user and the intelligent agent's dialogue]

[0198] Score:

[0199] Reason:

[0200] D4: Knowledge extraction and storage.

[0201] For passages that satisfy S(g j ) = 1, the system performs the following steps:

[0202] a) Use named entity recognition and key information extraction techniques to extract user question Q j .

[0203] b) Use sequence labeling and dependency syntax analysis to extract correct answer steps S j .

[0204] c) Extract the set of tools called T through keyword matching and context analysisj .

[0205] d) generating a structured knowledge item k j = (Q j , S j , T j ).

[0206] e) storing k j in a knowledge database and updating the relevant index.

[0207] The analysis of historical dialogues in this embodiment improves the accuracy of question and answer, forming a virtuous cycle.

[0208] Embodiment 3:

[0209] The embodiment provides an intelligent question and answer system based on a large model, comprising:

[0210] A human-computer exchange module for inputting a user question and displaying a final answer to the user question.

[0211] A knowledge database for storing a plurality of knowledge items, wherein the knowledge items include a user question, a question answering generation step, and an external tool required to be called for each question answering generation step.

[0212] An external tool set setting module for pre-setting an external tool set comprising a plurality of external tools.

[0213] A function classification module for classifying the plurality of external tools in the external tool set by function to obtain a function set.

[0214] A complexity evaluation module for evaluating the complexity of the input user question and generating a complexity evaluation result.

[0215] A question and answer module for generating a final answer based on the complexity evaluation result using different processing strategies.

[0216] The question and answer module comprises:

[0217] A retrieval unit for retrieving a knowledge item related to the user question from the knowledge database.

[0218] A question answering step planning unit for planning a question answering step.

[0219] An external tool calling plan generation unit for generating an external tool calling plan according to the planned question answering step.

[0220] A plan evaluation unit for evaluating the generated external calling plan and generating a plan evaluation result.

[0221] A system optimization module for optimizing and maintaining the system.

[0222] Embodiment 4:

[0223] The embodiment provides a computer readable storage medium, the computer readable storage medium comprising a stored program, wherein the program controls a device where the computer readable storage medium is located to perform the method of the method for intelligent question and answer based on a large model in the embodiment 1 when the program is running.

[0224] In the specification, identical or similar parts between the embodiments can be mutually referred to. In particular, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0225] In several embodiments provided in the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, system or unit, and can be electrical, mechanical or other forms.

[0226] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0227] In addition, it should be noted that the flowchart in the drawing shows the method of the embodiment of the present disclosure, and the operations or steps corresponding to the different blocks in the description of the flowchart or block diagram can also occur in an order different from the disclosure in the description. Sometimes, there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed in parallel, and sometimes they can be executed in reverse order, which can depend on the functions involved. Each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0228] The above merely provides the preferred embodiments of the present application, and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the principles and technical scope of the present application shall fall into the scope of the present application.

Claims

1. A large model-based intelligent question and answer method, characterized in that, The method comprises the following steps: Step 1: pre-setting an external tool set that can be called, and describing the function of each external tool in the external tool set; Step 2: classifying the pre-set external tool set according to the function of each external tool described to obtain a function set of external tools; Step 3: using a complexity evaluation function to evaluate the complexity of the user question, and generating a complexity evaluation result, wherein the complexity evaluation result comprises simple questions and professional questions; Step 4: generating a final answer according to the generated complexity evaluation result using different processing strategies.

2. The large model-based intelligent question answering method according to claim 1, characterized in that, The step 4 comprises: If the complexity evaluation result is a simple question, a final answer is directly generated; If the complexity evaluation result is a professional question, the following steps are performed: A) retrieving a knowledge item related to the user question from a knowledge database; B) planning an answer generation step based on the function set and the retrieved knowledge item; C) generating an external tool calling plan for the planned answer generation step, wherein the external tool calling plan comprises the answer generation step and the external tool that needs to be called for each answer generation step; D) evaluating the generated external tool calling plan, if the plan evaluation result is reasonable, executing external tool calling according to the external tool calling plan to generate a final answer, if the plan evaluation result is unreasonable, returning to step B).

3. The large model-based intelligent question answering method of claim 2, wherein, The execution of external tool calling according to the external tool calling plan to generate a final answer comprises: According to the external tool calling plan, external tools are called for each answer generation step in turn; The execution result of the external tool called for each answer generation step is evaluated, and an effect evaluation result is generated, wherein the effect evaluation result comprises achieving an expected effect and not achieving an expected effect; The final answer is determined according to the evaluation result.

4. The large model-based intelligent question answering method of claim 3, wherein, The determination of the final answer according to the evaluation result comprises: If the expected effect is achieved, the output results of all answer generation steps are integrated as the final answer; If there is an unexpected effect, the output results of all answer steps before the unexpected effect are integrated as the final answer.

5. The large model-based intelligent question answering method according to claim 1, characterized in that, The step further comprises: quality evaluation of the generated final answer.

6. A large model-based intelligent question and answer system, characterized in that, Comprise: A man-machine exchange module for inputting a user question and displaying a final answer of the input user question; A knowledge database for storing a plurality of knowledge items, wherein the knowledge items comprise a user question, an answer generation step of the user question, and an external tool required to be called for each answer generation step; An external tool set setting module for pre-setting an external tool set comprising a plurality of external tools; A function classification module for classifying the plurality of external tools in the external tool set to obtain a function set; A complexity evaluation module for evaluating the complexity of the input user question and generating a complexity evaluation result; A question and answer module for generating a final answer based on the complexity evaluation result using different processing strategies.

7. The large model-based intelligent question answering system according to claim 6, characterized in that, The question and answer module comprises: A retrieval unit for retrieving a knowledge item related to the user question from the knowledge database; An answer generation step planning unit for planning an answer generation step; An external tool calling plan generation unit for generating an external tool calling plan for the planned answer generation step, wherein the external tool calling plan comprises the answer generation step and the external tool that needs to be called for each answer generation step. The external tool calling plan generation unit is configured to generate an external tool calling plan according to the planned answering generation step; The plan evaluation unit is configured to evaluate the generated external calling plan and generate a plan evaluation result.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored program, wherein the computer readable storage medium controls the device where the computer readable storage medium is located to execute the large model-based intelligent question answering method in any one of claims 1 to 5 when the program is running.

Images