Intelligent question answering system and method

Abstract

The invention provides an intelligent question answering system and method. The intelligent question and answer method comprises the following steps. And receiving question information. A plurality of knowledge associated with the question information is collected. Based on the multiple pieces of knowledge, executing the large language model to generate a preliminary analysis result according to the question information; the preliminary analysis result comprises a plurality of steps required for answering the information. And recalling a plurality of candidate APIs associated with the plurality of steps according to the problem information and the preliminary analysis result. And selecting the selected API corresponding to each step from the plurality of candidate APIs to generate a problem solving plan. And based on a plurality of knowledge, executing the problem solving plan by using the large language model and dynamically adjusting the problem solving plan according to the execution result, and finally outputting the solution information based on the final execution result by using the large language model so as to improve the accuracy of the answer.

Description

Technical Field

[0001] The present invention relates to a question-answering system, and in particular to an intelligent question-answering system and method based on a large language model (LLM). Background Art

[0002] Generally speaking, question-answering systems employ large language models to automatically provide answers to questions. During the problem-solving process, the system analyzes the question based on its adapted domain knowledge and directly executes the corresponding application programming interface (API) based on the analysis to obtain the answer. However, current question-answering systems rely on a single domain knowledge structure, and the APIs they execute have low recall and precision, which in turn reduces the accuracy of the answers. Summary of the Invention

[0003] The embodiment of the present invention provides an intelligent question-answering method, which can improve the accuracy of answers.

[0004] The intelligent question-answering method of an embodiment of the present invention includes the following steps. Receive question information input by a user. Collect multiple knowledge related to the question information. Based on the multiple knowledge, execute a large language model to generate a preliminary analysis result based on the question information. The preliminary analysis result includes multiple steps required to answer the information. Based on the question information and the preliminary analysis result, recall multiple candidate APIs associated with the multiple steps. Select a selected API corresponding to each step from the multiple candidate APIs to generate a problem-solving plan. Based on the multiple knowledge, execute the problem-solving plan and the large language model to output the answer information.

[0005] An embodiment of the present invention also provides another intelligent question-answering system. The intelligent question-answering system includes a storage device and a processor. The storage device stores a large language model. The processor is coupled to the storage device. The processor is configured to execute the intelligent question-answering method described above.

[0006] Based on the above, the intelligent question-answering system and method of the embodiments of the present invention can utilize multiple knowledge sources by collecting multiple pieces of knowledge, thereby applying them to various professional fields. During the problem-solving process, by analyzing each step and selecting the selected API corresponding to each step, each step can be executed using the correct selected API. In this way, the intelligent question-answering system can enhance problem-solving capabilities and improve the recall and precision of executing the selected API, thereby improving the accuracy of answers.

[0007] In order to make the above features and advantages of the present invention more clearly understood, embodiments are given below with reference to the accompanying drawings for detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram of an intelligent question-answering system according to an embodiment of the present invention;

[0009] Figure 2 is a flow chart of an intelligent question-answering method according to another embodiment of the present invention;

[0010] Figure 3A is a flow chart of an intelligent question-answering method according to another embodiment of the present invention;

[0011] Figure 3B According to the present invention Figure 3A A flow chart of the intelligent question-answering method illustrated in the embodiment;

[0012] Figures 4A to 4E According to the present invention Figure 3B A schematic diagram of the operation of the intelligent question-answering method illustrated in the embodiment;

[0013] 5A to 5D is a flow chart of an intelligent question-answering method according to another embodiment of the present invention;

[0014] Figures 6A to 6B FIG. 4 is a flow chart of an intelligent question-answering method according to another embodiment of the present invention.

[0015] Explanation of Figure Numbers

[0016] 100: Intelligent question answering system;

[0017] 110: processor;

[0018] 120: storage device;

[0019] 121: Large language model;

[0020] 200: electronic device;

[0021] 411-412, 421, 431-432, 441: steps;

[0022] D1: Historical conversation records;

[0023] D2: contextual information;

[0024] DA: answer information;

[0025] DB: database;

[0026] DB1: synonym domain knowledge;

[0027] DB2: domain knowledge;

[0028] DB3: API database;

[0029] DQ: Question information;

[0030] DQ1: Complete service request;

[0031] DQ2: second request information;

[0032] DQ3: third request information;

[0033] R1: preliminary analysis results;

[0034] R2: intermediate execution result;

[0035] R3: Candidate API list;

[0036] R4: call result;

[0037] RF: Problem-solving plan;

[0038] RF1: First version of the problem-solving plan;

[0039] S210~260, S310~S340, S311~S343, S511~S554, S611~S652: steps. DETAILED DESCRIPTION

[0040] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0041] Figure 1 FIG is a block diagram of an intelligent question answering system according to an embodiment of the present invention. Figure 1 The intelligent question-answering system 100 can implement intelligent question-answering functions based on a large language model (LLM) 121. The intelligent question-answering system 100 can be applied in various professional fields and enhance problem-solving capabilities to output accurate answers (i.e., answer information DA).

[0042] exist Figure 1In an embodiment, the intelligent question-answering system 100 includes a processor 110 and a storage device 120. The processor 110 is coupled to the storage device 120, the electronic device 200, and the database DB. The intelligent question-answering system 100 can be set in the cloud. The intelligent question-answering system 100 can be, for example, a Software as a Service (SaaS) server to execute a corresponding Software as a Service application through an Application Programming Interface (API). The user can operate the electronic device 200 to call the intelligent question-answering system 100 through the API, and then reply to the user's questions through the intelligent question-answering system 100 based on the large language model 121. The electronic device 200 can be, for example, a mobile phone, a tablet computer, a laptop computer, a desktop computer and other devices.

[0043] In this embodiment, the storage device 120 stores the large language model 121. The storage device 120 may be, for example, dynamic random access memory (DRAM), flash memory, non-volatile random access memory (NVRAM), or a combination of these memories.

[0044] In this embodiment, the processor 110 accesses the storage device 120 and executes data in the storage device 120 and data (e.g., question information DQ) from the electronic device 200. The processor 110 may be, for example, a signal converter, a field programmable gate array (FPGA), a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessor, a digital signal processor (DSP), a programmable controller, an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or other similar devices or combinations thereof. It may load and execute computer program-related firmware or software to implement various computing functions.

[0045] Figure 2 FIG is a flow chart of an intelligent question-answering method according to another embodiment of the present invention. Figure 1 as well as Figure 2, the intelligent question answering system 100 performs steps S210 to S260 to solve the question based on the large language model 121. The order of these steps S210 to S260 is only for illustration.

[0046] In this embodiment, the user operates the electronic device 200 to input question information DQ through the electronic device 200. The question information DQ may be, for example, text information, such as "What is the total revenue this year?".

[0047] In step S210 , the processor 110 receives question information DQ input by a user.

[0048] In step S220, processor 110 collects multiple pieces of knowledge related to question information DQ. Specifically, processor 110 acquires expert domain knowledge related to question information DQ. Processor 110 also acquires knowledge related to question information DQ from the large language model 121 based on a pre-training dataset as internal knowledge, and acquires knowledge related to question information DQ found through a web search as external knowledge. Processor 110 collects these multiple pieces of knowledge and stores them in database DB.

[0049] In step S230, the processor 110 executes the large language model 121 based on multiple pieces of knowledge to generate a preliminary analysis result R1 based on the question information DQ. This preliminary analysis result R1 includes the multiple steps required to generate the answer information DA. The answer information DA is the answer to the question information DQ. In other words, the large language model 121 solves the question information DQ based on multiple pieces of knowledge. The large language model 121 analyzes the problem-solving process into multiple steps. In this way, the processor 110 sequentially solves the problem according to these multiple steps to ultimately generate the answer information DA.

[0050] In step S240, processor 110 retrieves multiple candidate APIs associated with multiple steps based on problem information DQ and preliminary analysis result R1. Specifically, based on problem information DQ and preliminary analysis result R1, processor 110 retrieves at least one API that may be applicable to each step as a candidate API. In this manner, processor 110 obtains multiple candidate APIs.

[0051] In step S250 , the processor 110 selects a selected API corresponding to each step from the plurality of candidate APIs to generate a problem-solving plan RF. Thus, the problem-solving plan RF includes the sorted plurality of steps and the API used by each step (ie, the selected API).

[0052] In step S260, processor 110 executes the problem-solving plan RF and large language model 121 based on the multiple pieces of knowledge stored in database DB to output answer information DA. Answer information DA may be text. For example, answer information DA may include, "According to the analysis results, the total revenue for 2024 is 29,235,990 yuan. The following is a summary of the analysis steps: 1. **Data Collection**: Sales data for 2024 to date was obtained, with a total sales volume of 63,978. 2. **Current Situation**: This year's sales volume is also 63,978. ... In summary, the total revenue for 2024 is 29,235,990 yuan."

[0053] It is worth mentioning that the intelligent question-answering system 110 can utilize multi-party knowledge by collecting multiple pieces of knowledge related to the question information DQ through the processor 110. In addition, during the problem-solving process, the processor 110 analyzes multiple steps and recalls multiple corresponding candidate APIs accordingly. The intelligent question-answering system 110 can analyze each of the multiple candidate APIs recalled for each step one by one to select the correct selected API and then construct a problem-solving plan RF. In this way, based on the large language model 121, the intelligent question-answering system 110 can enhance its problem-solving capabilities and improve the recall rate and accuracy of executing the selected API, thereby improving the accuracy of the answer information DA.

[0054] Figure 3A FIG. 4 is a flow chart of an intelligent question-answering method according to another embodiment of the present invention. Figure 3B According to the present invention Figure 3A Flowchart of the intelligent question answering method shown in the embodiment. Figure 3A as well as Figure 3B , the intelligent question answering system uses a processor (e.g., Figure 1 The processor 110 shown executes steps S310-S340 and S311-S343 to solve the problem based on the large language model.

[0055] In step S310, the processor receives the problem information DQ and analyzes the problem information DQ to generate a preliminary analysis result (eg, Figure 1 The preliminary analysis result R1 is shown. The preliminary analysis result includes multiple steps required to answer the information DA. The question information DQ may be, for example, "How are the sales in October?"

[0056] Specifically, the processor analyzes historical conversations preceding the question DQ and extracts contextual information, including synonyms and key entities, from these conversations to obtain information relevant to the problem. This information includes multiple rounds of conversations between the user and the intelligent question-answering system, boundary determination information, exception handling information, and permission determination information.

[0057] Next, in step S311, the processor retrieves multi-party knowledge based on the problem-solving information. The processor executes the large language model to integrate and confirm the details of the problem-solving process based on the problem-solving information. The processor represents these details as multiple steps to form a preliminary analysis result.

[0058] For reference Figures 4A to 4E , Figures 4A to 4E According to the present invention Figure 3B The operation diagram of the intelligent question answering method shown in the embodiment. Figure 4A In one embodiment, the preliminary analysis result R1 includes multiple steps arranged according to the problem-solving process. For example, the preliminary analysis result R1 includes a first step 411 to describe the current status of the problem information DQ. The preliminary analysis result R1 also includes a second step 412 to describe the historical status of the problem information DQ.

[0059] In step S320, the processor 110 analyzes the tools required for the problem information DQ according to the problem information DQ and the preliminary analysis result R1. The tools include at least one candidate API associated with each step.

[0060] Specifically, in step S321 , the processor 110 searches and recalls multiple candidate APIs associated with each step in parallel according to the problem information DQ and the preliminary analysis result R1 and the problem-solving idea.

[0061] like Figure 4B As shown in the preliminary analysis results R1, multiple candidate APIs may be used in step 421. These candidate APIs include "S45," "S20," and "S127." The "S45" API is used to query the expected number of shipments. "S20" is used to calculate the sales volume during the period. "S127" is used to calculate the revenue trend during the period.

[0062] In step S330, the processor formulates a problem-solving plan RF based on the preliminary analysis result R1 in step S310 and the multiple candidate APIs in step S320. The problem-solving plan RF includes multiple sorted steps and the selected API used in each step.

[0063] Specifically, in step S331, the processor determines the parameters and applicability of each candidate API corresponding to each step in the preliminary analysis result R1 according to the problem-solving strategy, and generates a problem-solving plan RF based on the determination result.

[0064] like Figure 4CAs shown, the solution plan RF includes multiple steps arranged according to the solution strategy, and also includes the selected API used in each step. For example, the solution plan RF includes a first step 431. First step 431 instructs calling the "S20" API based on the parameter "OOO" to calculate the sales quantity for the period. The solution plan RF also includes a second step 432, which follows first step 431. Second step 432 instructs calling the "S20" API based on the different parameters "XXX," "YYY," and "ZZZ" to calculate the sales quantity for the period.

[0065] In step S340 , the processor executes the problem-solving plan RF and the large language model 121 based on the collected knowledge to output answer information DA.

[0066] Specifically, in step S341, the processor sequentially executes the selected API for each step in the problem-solving plan RF to generate an intermediate execution result. In step S342, the processor dynamically adjusts the workflow based on the current intermediate execution result. In this manner, the processor executes the selected API for the current step to generate a call result for that step. The processor uses this call result as the intermediate execution result and continuously updates it until all steps are completed.

[0067] like Figure 4D As shown, the intermediate execution results include multiple steps ordered by problem-solving strategy, and also include the call results of each step using the selected API. For example, the solution plan RF includes step 1 441. Step 441 indicates the result of calling the "S20" API, which calculates the sales quantity of 1544 based on the parameter "OOO".

[0068] In step S343, after all steps are completed, the processor executes the large language model to summarize the call results of all steps to generate answer information DA. Answer information DA is as follows: Figure 4E The processor transmits the answer information DA back to the electronic device operated by the user to complete the answer to the question information DQ.

[0069] 5A to 5D FIG is a flow chart of an intelligent question-answering method according to another embodiment of the present invention. 5A to 5D , the intelligent question answering system uses a processor (e.g., Figure 1 The processor 110 shown executes steps S511 to S554 to illustrate the operating details of the intelligent question-answering method.

[0070] In this embodiment, the intelligent question-answering system provides functions of collecting information and outputting results.

[0071] Specifically, the intelligent question-answering system receives question information DQ through a processor and outputs preliminary analysis results R1, intermediate execution results R2, and execution plan RF during the problem-solving process through the processor.

[0072] In step S511 , the processor accesses the historical conversation record D1 according to the question information DQ and performs multiple rounds of processing on the context of the question information DQ to analyze the complete service request DQ1 .

[0073] Specifically, the question information DQ may be, for example, "How are the sales in October?". After obtaining the context of the question information DQ, the processor may analyze the complete service request DQ1 to be, for example, "How are the sales of Apple in October?".

[0074] In this embodiment, the intelligent question-answering system provides functions of expanding and integrating information.

[0075] Specifically, the processor executes a Retrieval Augmented Generation (RAG) module, which may be stored in a storage device of the intelligent question answering system.

[0076] In step S521, the RAG module searches the synonym domain knowledge DB1 according to the complete service request DQ1 to obtain reference information associated with the complete service request DQ1. The RAG module also integrates the complete service request DQ1 and the reference information into second request information DQ2.

[0077] Specifically, the question information DQ1 may be, for example, "How are Apple's sales in October?", and the reference information after the RAG module searches for synonyms may be integrated into the second request information DQ2, for example, "Apple's sales status in October".

[0078] In step S522, the processor accesses the historical conversation log D1 and context information D2, and integrates the historical conversation log D1, context information D2, and the second request information DQ2 to generate third request information DQ3. Specifically, third request information DQ3 incorporates the complete service request DQ1, reference information associated with the complete service request DQ1, the historical conversation log D1, and context information D2. Context information D2 may include, for example, information related to the person, event, time, place, and object associated with the question information DQ.

[0079] In this embodiment, the intelligent question-answering system provides a function for analyzing question information.

[0080] Specifically, in step S531, based on the third request information DQ3 or the intermediate execution result R2 (triggered by step S611), the processor executes the RAG module to retrieve the domain knowledge DB2 associated with the question information DQ. The processor also executes the large language model to retrieve internal knowledge associated with the question information DQ. The processor also executes the network tool to retrieve external knowledge associated with the question information DQ. Furthermore, the processor fuses the retrieved domain knowledge, internal knowledge, and external knowledge to form a plurality of knowledge.

[0081] It should be noted that the processor accesses all potentially useful knowledge in the problem-solving process. This includes expert knowledge such as exception handling and authority determination, internal knowledge used to train large language models, and external knowledge from the network. In this way, the intelligent question-answering system can leverage knowledge from multiple sources for flexible application in various professional fields.

[0082] In step S532, the processor executes the large language model to identify key entities associated with the question information DQ based on the plurality of knowledge in step S531. Specifically, the processor accesses the third request information DQ3 and the plurality of knowledge to identify key entities of the third request information DQ3.

[0083] In step S533, the processor sets multiple preference scores for the domain knowledge, internal knowledge, and external knowledge in step S531. The processor then executes the large language model to aggregate the multiple knowledge items based on the set preference scores. Furthermore, the processor also executes the large language model to embed the key entities in step S532 into each step based on the multiple knowledge items. In this way, the processor generates a preliminary analysis result R1 comprising multiple steps.

[0084] In this embodiment, multiple preference scores can be determined by the user to indicate the dependency weights of different knowledge. For example, the multiple preference scores include 10 points for domain knowledge, 5 points for internal knowledge, and 4 points for external knowledge.

[0085] In this embodiment, when there is a knowledge conflict, the processor selects one of domain knowledge, internal knowledge, and external knowledge based on the total score of the set multiple preference scores. In this way, the processor can select the knowledge corresponding to the knowledge with the highest total score as the correct knowledge benchmark.

[0086] Alternatively, when there is a conflict in knowledge and the total scores are the same, the processor selects one of the domain knowledge, the internal knowledge, and the external knowledge as the correct knowledge reference according to a selection instruction of the user.

[0087] It should be noted that by embedding key entities such as people, things, time, place, and objects involved in the question information DQ into each step, the intelligent question answering system can ensure that the large language model fully analyzes the details of the question information DQ during the problem-solving process.

[0088] For example, for the DQ question "What was the sales situation of apples in October?", the large language model uses "sales" as the analysis direction for the task objective. The large language model embeds "apples" (the object) and "October" (the month) into multiple subsequent steps. These steps include historical data, returns, year-on-year comparisons, month-on-month comparisons, and budget operations.

[0089] In this embodiment, the intelligent question answering system provides the tools required to analyze question information.

[0090] Specifically, in step S541, the processor expands the description of each step into multiple context descriptions based on the question information DQ and the preliminary analysis result R1. Specifically, the processor expands the description of each step into m context descriptions in parallel (i.e., simultaneously) based on the third request information DQ3 including the question information DQ and the preliminary analysis result R1, where m is a positive integer greater than 1.

[0091] In step S542, the processor executes the RAG module to retrieve multiple candidate APIs associated with each step in the multiple context descriptions from the API database DB3. Specifically, the RAG module concurrently (i.e., simultaneously) retrieves the API function descriptions and API usage examples associated with each step from the API database DB3 in multiple steps. In this way, the RAG module simultaneously retrieves one or more APIs that may be used for each step as multiple candidate APIs.

[0092] It should be noted that because the intelligent question-answering system operates not only based on the question information DQ itself but also on the context of the question information DQ, it can access complete information based on the question information DQ, thereby ensuring the integrity of the solution. In addition, by searching API function descriptions and API usage examples, the intelligent question-answering system can expand the search scope in both directions and thus find all possible candidate APIs.

[0093] It should be noted that since the intelligent question-answering system selects the candidate API corresponding to a single step, the intelligent question-answering system can reduce the complexity of the problem-solving process.

[0094] In step S543, the processor integrates the multiple candidate APIs in each step to generate a candidate API list R3. The candidate API list R3 includes multiple candidate APIs sorted by problem-solving ideas.

[0095] In this embodiment, the intelligent question-answering system provides a function for formulating a problem-solving plan RF.

[0096] Specifically, in step S551, the processor determines the parameters of each candidate API corresponding to each step based on the question information DQ, the preliminary analysis result R1, and the candidate API list R3 to generate a first determination result. Specifically, the processor determines (e.g., sets) the parameters of each candidate API in parallel (i.e., simultaneously) and independently in multiple steps based on the third request information DQ3 including the question information DQ, the preliminary analysis result R1, and the candidate API list R3.

[0097] In step S552, the processor determines whether each candidate API corresponding to each step is applicable, based on the first determination result in step S551, to generate a second determination result. Specifically, based on the first determination result (including the set API parameters), the processor determines whether each candidate API is applicable in parallel (i.e., simultaneously) and independently using a yes-or-no question format in multiple steps.

[0098] It should be noted that because the intelligent question-answering system considers each candidate API and its parameters for each step individually, it is able to convert multiple-choice questions into true-or-false questions. The intelligent question-answering system considers the applicability of each candidate API on a step-by-step basis. In this way, the intelligent question-answering system can reduce the complexity of the problem-solving process.

[0099] In step S553, the processor selects the selected API corresponding to each step based on the second judgment result in step S552 to generate the preliminary solution plan RF1. Specifically, based on the second judgment result (including whether the API is applicable), the processor selects the API that can be used in each step as the selected API. The processor aggregates the multiple selected APIs corresponding to all steps and converts the aggregated results into JSON format to generate the preliminary solution plan RF1.

[0100] In step S554, the processor determines the correctness of the initial solution plan RF1 based on the problem information DQ to generate a third determination result. Based on the third determination result, the processor modifies the parameters of the selected APIs to generate the solution plan RF. Specifically, the processor verifies the correctness of the initial solution plan RF1 based on the third request information DQ3 that includes the problem information DQ. Based on the determination result, the processor modifies and integrates the selected APIs and their parameters to generate the solution plan RF.

[0101] Figures 6A to 6B FIG is a flow chart of an intelligent question-answering method according to another embodiment of the present invention. Figures 6A to 6B , the intelligent question answering system uses a processor (e.g., Figure 1The processor 110 shown in FIG. 110 executes steps S611 to S652 to illustrate the operation details of the intelligent question answering method. Steps S611 to S652 may be, for example, a sequence of steps. 5A to 5D steps to execute the problem-solving plan RF and generate answer information DA accordingly.

[0102] During the execution of the problem-solving plan RF, the intelligent question-answering system provides the functions of collecting information and outputting results. The intelligent question-answering system also provides the function of managing the execution progress.

[0103] Specifically, in step S621, the processor executes the large language model to confirm the execution progress of the problem-solving plan RF.

[0104] In step S622, the processor determines whether the execution progress in step S621 is progressing normally. If the determination result is no, it indicates that an abnormality has occurred in the current step. The processor proceeds to step S623.

[0105] In step S623, the processor determines whether the number of exceptions that occurred in the current step exceeds a threshold. The threshold can be, for example, a user-defined value that indicates the conditions under which an attempt to resolve the exception is permitted. If the determination is negative, the number of exceptions that occurred in the current step is considered normal. The processor then proceeds to step S611 to attempt to resolve the exception that occurred in the current step.

[0106] In step S611, the processor regenerates the problem-solving plan RF based on the current execution progress and the intermediate execution result R2 of the execution progress. Figure 5B Step S531 shown, re-execute 5A to 5D The method shown.

[0107] On the other hand, when the judgment result in step S623 is yes, it means that the number of abnormalities in the current step is too many. At this time, the processor continues step S626 to ignore the abnormality that the current step occurs, and continues to execute the problem-solving plan RF.

[0108] In step S626, the processor records the abnormality of the execution result of the current step in the intermediate execution result R2 and adjusts the solution plan RF to ignore the current step. This allows the intelligent question-answering system to dynamically adjust the solution plan RF based on the intermediate execution result R2 while executing the solution plan RF. The processor continues executing the solution plan RF to proceed to step S624.

[0109] In step S624, the processor determines whether the execution progress is complete. If the result is yes, the problem-solving plan RF has been completed. In other words, the multiple selected APIs in all steps have been executed. The processor then proceeds to step S625.

[0110] In step S625 , the processor executes the large language model to summarize the execution progress and generates a final execution result as answer information DA.

[0111] On the other hand, when the determination result in step S624 is negative, indicating that the problem-solving plan RF has not been completed, the processor proceeds to step S631.

[0112] During the execution of the problem-solving plan RF, the intelligent question-answering system provides the functions of constructing API execution instructions, executing selected APIs, and updating execution progress.

[0113] Specifically, in step S631, when the execution progress of the solution plan RF has not yet been completed, the processor executes the large language model. Based on the execution progress, the large language model constructs API execution instructions R3 for the selected API corresponding to the next step. In other words, based on the current execution status of the solution plan RF, the large language model constructs, verifies, and modifies the instructions and input parameters for the next step to generate the API execution instructions R3 for that step. The API execution instructions R3 include the ordering number of the step within the solution plan RF. The API execution instructions R3 also include its own parameters and knowledge sources.

[0114] In step S641, the processor determines whether the API execution instruction R3 requires authorization based on multiple knowledge DBs. If the determination result is no, the processor proceeds to step S642.

[0115] In step S642 , the processor directly executes the API execution instruction R3 to generate a call result R4 .

[0116] In step S651, the processor executes the large language model to update the execution result in the problem-solving plan RF according to the call result R4. Based on the updated problem-solving plan RF, the processor returns to step S621. At the same time, the processor continues to step S652.

[0117] In step S652 , the processor executes the large language model to output an execution result as an intermediate execution result R2 .

[0118] On the other hand, if the result of the determination in step S641 is yes, it indicates that the execution of the API execution instruction R3 requires additional authorization. In this case, the processor sends an authorization request message to the electronic device, prompting the user through the user interface of the electronic device. The processor then proceeds to step S612.

[0119] In step S612, the processor receives the authorization information provided by the user, and determines whether to continue executing the API execution instruction R3 to proceed to step S613 based on the authorization information.

[0120] In step S613, the processor determines whether the authorization information is passed. If the determination result is yes, the processor proceeds to step S642 to execute the API execution instruction R3 according to the authorization information.

[0121] On the other hand, if the result of the determination in step S613 is negative, the processor sends a confirmation message to the electronic device, prompting the user via the user interface of the electronic device. The confirmation message includes information indicating that the solution plan RF has been paused, that the solution plan RF has been terminated, or that the user has manually guided the solution plan RF. Based on the user's response to the confirmation message, the processor correspondingly pauses, terminates, or executes the solution plan RF as instructed by the user.

[0122] In summary, the intelligent question-answering system and method of the embodiments of the present invention can be applied in various professional fields by collecting multiple pieces of knowledge related to question information. In the problem-solving process, the intelligent question-answering system can ensure the integrity of the problem-solving ideas and can expand the search scope of the API. In addition, the intelligent question-answering system parses each step and converts the multiple-choice questions into true or false questions to select the selected API corresponding to each step. Therefore, the intelligent question-answering system can reduce the complexity of the problem-solving process and can also select the correct selected API to implement the steps. In this way, the intelligent question-answering system can enhance the problem-solving ability and improve the recall rate and accuracy of executing the selected API. When the intelligent question-answering system executes the problem-solving plan with a large language model, it will also dynamically adjust the problem-solving plan according to the intermediate execution results, and finally output the answer information based on the final execution results with the large language model, thereby improving the accuracy of the answer.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent question-answering method, characterized in that: include: Receive question information entered by the user; Collecting a plurality of knowledge related to the problem information; executing a large language model based on the plurality of knowledge to generate a preliminary analysis result according to the question information, wherein the preliminary analysis result includes a plurality of steps required to answer the information; Recalling a plurality of candidate application programming interfaces (APIs) associated with the plurality of steps according to the problem information and the preliminary analysis result; Selecting a selected API corresponding to each step from the plurality of candidate APIs to generate a problem-solving plan; as well as Based on the plurality of pieces of knowledge, the problem-solving plan and the large language model are executed to output the answer information.

2. The intelligent question-answering method according to claim 1, wherein: The step of recalling the multiple candidate APIs associated with the multiple steps according to the problem information and the preliminary analysis result includes: Expand the description of each step into multiple scenario descriptions based on the problem information and the preliminary analysis results; executing a Retrieval Augmented Generation (RAG) module to retrieve the plurality of candidate APIs associated with each of the steps in the plurality of situation descriptions; and The plurality of candidate APIs of each of the steps are integrated to generate a candidate API list.

3. The intelligent question-answering method according to claim 1, wherein: The preliminary analysis result includes the multiple steps sorted according to problem-solving ideas, and the candidate API list includes the multiple candidate APIs sorted according to the problem-solving ideas.

4. The intelligent question-answering method according to claim 1, wherein: The RAG module simultaneously retrieves the plurality of candidate APIs for the plurality of steps.

5. The intelligent question-answering method according to claim 1, wherein: The step of selecting a selected API corresponding to each step from the plurality of candidate APIs to generate the problem-solving plan includes: According to the problem information, the preliminary analysis result, and the candidate API list, determining the parameters of each candidate API corresponding to each step to generate a first determination result; According to the first judgment result, determining whether each candidate API corresponding to each step is applicable one by one to generate a second judgment result; and According to the second judgment result, the selected API corresponding to each step is selected to generate a preliminary version of the problem-solving plan.

6. The intelligent question-answering method according to claim 5, characterized in that: The step of selecting a selected API corresponding to each of the steps from the plurality of candidate APIs to generate the problem-solving plan further includes: Judging the correctness of the initial version of the problem-solving plan based on the problem information to generate a third judgment result; and According to the third judgment result, the parameters of the multiple selected APIs are modified to generate the problem-solving plan.

7. The intelligent question-answering method according to claim 1, wherein: Based on the plurality of pieces of knowledge, the step of executing the problem-solving plan and the large language model to output the answer information includes: executing the large language model to construct, when the execution progress of the problem-solving plan has not yet been completed, an API execution instruction of the selected API corresponding to the next step according to the execution progress; Based on the multiple pieces of knowledge, determine whether the API execution instruction requires authorization, so as to directly execute the API execution instruction to generate a call result, or determine whether to continue executing the API execution instruction based on the authorization information provided by the user.

8. The intelligent question-answering method according to claim 7, characterized in that: Based on the plurality of pieces of knowledge, executing the problem-solving plan and the large language model to output the answer information further includes: Determine whether the number of exceptions that occur in the current step is greater than a threshold, so as to regenerate a problem-solving plan based on the execution progress and the intermediate execution results of the execution progress, or ignore the exceptions that occur in the current step and continue to execute the problem-solving plan.

9. The intelligent question-answering method according to claim 8, characterized in that: Based on the plurality of pieces of knowledge, executing the problem-solving plan and the large language model to output the answer information further includes: executing the large language model to update the execution result in the problem-solving plan according to the call result; and The large language model is executed to output the execution result as the intermediate execution result.

10. The intelligent question-answering method according to claim 1, wherein: The step of collecting the plurality of knowledge associated with the problem information includes: executing the RAG module to retrieve domain knowledge associated with the problem information; executing the large language model to retrieve internal knowledge associated with the question information; executing a network tool to retrieve external knowledge related to the problem information; and The domain knowledge, the internal knowledge, and the external knowledge are integrated to form the plurality of knowledge.

11. The intelligent question-answering method according to claim 10, wherein: The step of collecting the plurality of knowledge associated with the problem information further includes: Setting a plurality of preference scores for the domain knowledge, the internal knowledge, and the external knowledge respectively; When there is a knowledge conflict, one of the domain knowledge, the internal knowledge, and the external knowledge is selected according to the total score of the plurality of preference scores, or according to the user's selection instruction.

12. The intelligent question-answering method according to claim 11, wherein: Also includes: executing the large language model based on the plurality of knowledge to identify key entities associated with the question information; as well as The large language model is executed according to the plurality of knowledge to embed the key entities in each of the steps.

13. An intelligent question-answering system, characterized in that: include: A storage device for storing a large language model; as well as A processor, coupled to the storage device, is configured to execute the intelligent question-answering method according to claim 1.

Images