Multi-stage question answering method, system and device

Abstract

The embodiments of this specification provide a multi-stage question-answering method, system, and device, wherein the multi-stage question-answering method includes: splitting the target sub-problem corresponding to the current question-answering stage in the multi-stage question; when determining that the large language model has passed the cyclic splitting detection based on the target sub-problem and the sub-problem list, using the large language model, processing the target sub-problem according to the historical answer data corresponding to the target sub-problem to obtain target answer data, and the historical answer data is constructed based on the answer data corresponding to each question-answering stage before the current question-answering stage; splitting the next sub-problem corresponding to the next question-answering stage in the multi-stage question, and using the next sub-problem as the target sub-problem, repeatedly performing cyclic splitting detection on the large language model until the answer data corresponding to each question-answering stage is obtained, and generating the target answer corresponding to the multi-stage question based on the answer data corresponding to each question-answering stage.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and more particularly to a multi-stage question-answering method, system, and device. Background Art

[0002] In question-answering scenarios, existing retrieval-augmented generation (RAG) techniques typically employ a "recall-generate" framework, directly generating answers by retrieving documents. However, when faced with complex questions requiring multi-step reasoning, they lack the ability to construct deep logical chains, resulting in incomplete recall information or biased answers. While frameworks based on a "reason-action" cycle mitigate these issues by dynamically adjusting retrieval strategies, they still suffer from significant drawbacks. For example, the model is prone to self-looping into ineffective reasoning, generating contradictory content after multiple iterations. This often wastes computing resources and prevents users from tracing the path to the answer, severely impacting the credibility of the answer. Therefore, a more effective multi-stage question answering method is urgently needed to address these issues. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a multi-stage question-answering method. One or more embodiments of this specification also relate to a multi-stage question-answering system, a multi-stage question-answering apparatus, a computing device, a computer-readable storage medium, and a computer program product to address technical deficiencies in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a multi-stage question answering method is provided, comprising:

[0005] Split the target sub-problems corresponding to the current question-answering stage in the multi-stage problem;

[0006] When it is determined based on the target sub-question and the sub-question list that the large language model passes the loop splitting detection, using the large language model, processing the target sub-question according to the historical answer data corresponding to the target sub-question to obtain target answer data, wherein the historical answer data is constructed based on answer data corresponding to each question and answer stage before the current question and answer stage;

[0007] Splitting the multi-stage question into a next sub-problem corresponding to the next question-answering stage, and using the next sub-problem as the target sub-problem, and executing the step of, upon determining based on the target sub-problem and the sub-problem list that the large language model passes the loop splitting detection, processing the target sub-problem using the large language model according to the historical answer data corresponding to the target sub-problem to obtain target answer data;

[0008] Until the answer data corresponding to each question-answering stage is obtained, the target answer corresponding to the multi-stage question is generated based on the answer data corresponding to each question-answering stage.

[0009] According to a second aspect of an embodiment of this specification, a multi-stage question answering system is provided, comprising:

[0010] The client is used to submit multi-level questions to the server;

[0011] The server is used to split the target sub-problem corresponding to the current question-answering stage in the multi-stage question; when it is determined based on the target sub-problem and the sub-problem list that the large language model passes the loop splitting detection, use the large language model to process the target sub-problem according to the historical answer data corresponding to the target sub-problem, and obtain target answer data, wherein the historical answer data is constructed based on the answer data corresponding to each question-answering stage before the current question-answering stage; split the next sub-problem corresponding to the next question-answering stage in the multi-stage question, and use the next sub-problem as the target sub-problem, and execute the step of processing the target sub-problem according to the historical answer data corresponding to the target sub-problem according to the target sub-problem and the sub-problem list, and obtaining target answer data; until the answer data corresponding to each question-answering stage is obtained, generate the target answer corresponding to the multi-stage question based on the answer data corresponding to each question-answering stage, and send the target answer to the client.

[0012] According to a third aspect of the embodiments of this specification, a multi-stage question-answering device is provided, comprising:

[0013] The splitting module is configured to split the multi-stage question into the target sub-questions corresponding to the current question-answering stage;

[0014] a processing module configured to, upon determining based on the target sub-question and the sub-question list that the large language model passes the loop splitting test, process the target sub-question using the large language model according to historical answer data corresponding to the target sub-question to obtain target answer data, wherein the historical answer data is constructed based on answer data corresponding to each question-and-answer stage before the current question-and-answer stage;

[0015] an execution module configured to split the multi-stage question into a next sub-problem corresponding to the next question-answering stage, and use the next sub-problem as the target sub-problem, and execute the step of, upon determining based on the target sub-problem and the sub-problem list that the large language model passes the loop splitting test, processing the target sub-problem using the large language model according to the historical answer data corresponding to the target sub-problem to obtain target answer data;

[0016] The generation module is configured to generate the target answer corresponding to the multi-stage question based on the answer data corresponding to each question and answer stage until the answer data corresponding to each question and answer stage is obtained.

[0017] According to a fourth aspect of the embodiments of this specification, a computing device is provided, including:

[0018] memory and processor;

[0019] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned multi-stage question-answering method are implemented.

[0020] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned multi-stage question-answering method.

[0021] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program or instructions, which, when executed by a processor, implement the steps of the above-mentioned multi-stage question-answering method.

[0022] An embodiment of the present specification provides a multi-stage question-answering method, which splits the target sub-problem corresponding to the current question-answering stage in the multi-stage question; when it is determined based on the target sub-problem and the sub-problem list that the large language model has passed the cyclic splitting detection, the target sub-problem is processed using the large language model according to the historical answer data corresponding to the target sub-problem to obtain the target answer data, wherein the historical answer data is constructed based on the answer data corresponding to each question-answering stage before the current question-answering stage; the next sub-problem corresponding to the next question-answering stage is split in the multi-stage question, and the next sub-problem is used as the target sub-problem, and the step of processing the target sub-problem according to the historical answer data corresponding to the target sub-problem is performed based on the target sub-problem and the sub-problem list. When it is determined based on the target sub-problem and the sub-problem list that the large language model has passed the cyclic splitting detection, the target sub-problem is processed using the large language model according to the historical answer data corresponding to the target sub-problem to obtain the target answer data, and the multi-stage question is gradually answered by splitting the complex multi-stage question. Until the answer data corresponding to each question-answering stage is obtained, the target answer corresponding to the multi-stage question is generated based on the answer data corresponding to each question-answering stage.

[0023] The sub-question list can be used to determine whether the large language model is trapped in a loop splitting operation on multi-level questions, preventing the large language model from performing multiple, ineffective splits on multi-level questions. This saves the computing resources used by the large language model when processing multi-level questions and improves the performance of the large language model when handling complex questions. The target answer for the multi-level question is generated based on the answer data corresponding to each question-and-answer stage. This integrates the phased answer data obtained from at least two question-and-answer stages, preventing incorrect answers generated in the intermediate processing stages from affecting the accuracy of the target answer, thereby improving the user's question-and-answer experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a schematic diagram of a multi-stage question-answering method provided by an embodiment of this specification;

[0025] Figure 2 This is a flow chart of a multi-stage question-answering method provided by one embodiment of this specification;

[0026] Figure 3 This is a flowchart of a processing process of a multi-stage question-answering method provided by an embodiment of this specification;

[0027] Figure 4 This is a page diagram of a multi-stage question-answering method provided by an embodiment of this specification;

[0028] Figure 5 This is a question processing flow chart of a multi-stage question-answering method provided by an embodiment of this specification;

[0029] Figure 6 This is a schematic diagram of the structure of a multi-stage question answering system provided by an embodiment of this specification;

[0030] Figure 7 This is a schematic diagram of the structure of a multi-stage question-answering device provided by an embodiment of this specification;

[0031] Figure 8 This is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0032] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0033] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0034] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0035] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0036] First, the terms involved in one or more embodiments of this specification are explained.

[0037] Retrieval-Augmented Generation (RAG): A technical framework that combines information retrieval (IR) and generative models (such as large language models) to address the information shortage problem in open-domain knowledge-based tasks. Its core idea is to provide context for generative models by retrieving relevant documents from external knowledge bases, thereby improving the accuracy, timeliness, and relevance of generated content.

[0038] In the question-answering scenario, agentic-RAG is mainly based on the form of single-step task decomposition + execution. However, this form has a high probability of loop splitting and loop searching problems when splitting and searching. When executing the sub-questions obtained by splitting one by one, it will also cause error accumulation. Therefore, an embodiment of this specification provides a multi-stage question answering method, Figure 1A schematic diagram of a multi-stage question-answering method provided according to an embodiment of the present specification is shown, wherein the target sub-problem corresponding to the current question-answering stage is split out from the multi-stage question; when the large language model is determined to have passed the cyclic splitting test based on the target sub-problem and the sub-problem list, the target sub-problem is processed using the large language model according to the historical answer data corresponding to the target sub-problem to obtain the target answer data, wherein the historical answer data is constructed based on the answer data corresponding to each question-answering stage before the current question-answering stage; the next sub-problem corresponding to the next question-answering stage is split out from the multi-stage question, and the next sub-problem is used as the target sub-problem; when the large language model is determined to have passed the cyclic splitting test based on the target sub-problem and the sub-problem list, the target sub-problem is processed using the large language model according to the historical answer data corresponding to the target sub-problem to obtain the target answer data, and so on, by cyclically splitting and answering complex multi-stage questions, the multi-stage questions are gradually answered. Until the answer data corresponding to each question-answering stage is obtained, the target answer corresponding to the multi-stage question is generated based on the answer data corresponding to each question-answering stage.

[0039] The sub-question list can be used to determine whether the large language model is trapped in a loop splitting operation on multi-level questions, preventing the large language model from performing multiple, ineffective splits on multi-level questions. This saves the computing resources used by the large language model when processing multi-level questions and improves the performance of the large language model when handling complex questions. The target answer for the multi-level question is generated based on the answer data corresponding to each question-and-answer stage. This integrates the phased answer data obtained from at least two question-and-answer stages, preventing incorrect answers generated in the intermediate processing stages from affecting the accuracy of the target answer, thereby improving the user's question-and-answer experience.

[0040] In this specification, a multi-stage question-answering method is provided. This specification also involves a multi-stage question-answering system, a multi-stage question-answering device, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0041] See also Figure 2 , Figure 2 A flowchart of a multi-stage question-answering method provided according to an embodiment of this specification is shown, which specifically includes the following steps.

[0042] Step 202: Split the multi-stage question into the target sub-question corresponding to the current question-answering stage.

[0043] Specifically, a multi-order question is a complex question containing at least two sub-questions. Multi-order questions include problem derivation logic, and the answer to a multi-order question requires at least two inferences to determine the answer. A multi-order question corresponds to a thought chain, which contains at least two sequentially arranged questions to be answered. A multi-order question cannot be answered through a single question search; the final answer requires at least two consecutive question searches. The current question-answering stage can be any stage in which the multi-order question is processed in stages. The target sub-question can be a sub-question obtained by splitting the multi-order question in the current question-answering stage.

[0044] Based on this, when answering multi-stage questions, the current question-answering stage is determined, and the multi-stage questions are split based on the current question-answering stage to obtain the target sub-questions corresponding to the current question-answering stage.

[0045] In practical applications, processing multi-level problems requires at least two stages: problem splitting and problem answering. For multi-level problems, the initial split is performed to obtain the first sub-problem. This sub-problem is then answered to obtain the answer data. Based on the answer data, the next stage of splitting is determined based on the initial sub-problem. If necessary, the sub-problem splitting is performed. This process continues in this order until the multi-level problem is solved and the target answer is obtained.

[0046] Furthermore, considering the logic between the question-answering stages in a multi-stage question, when splitting the multi-stage question, it is also necessary to split it according to the question splitting prompt words in the multi-stage question. The specific implementation is as follows:

[0047] Determine at least two stages of question words contained in the multi-stage question, and determine the question splitting prompt words corresponding to the current question-answering stage among the at least two stages of question words; use the large language model to split the multi-stage question according to the question splitting prompt words to obtain the target sub-problem.

[0048] Specifically, the phase question words can be phrases or phrases in a multi-stage question. Phase question words are generally nouns corresponding to the objects contained in the multi-stage question. Question splitting prompt words refer to the phase question words corresponding to the current question-answering stage in at least two phase prompt words.

[0049] Based on this, we analyze the words and parts of speech in the multi-stage question and, based on the parts of speech, identify at least two phases of the question. From these at least two phases, we determine the question-splitting cue words corresponding to the current question-answering phase. Using a large language model, we split the multi-stage question according to the question-splitting cue words to obtain the target sub-questions.

[0050] For example, consider a multi-stage question: What city is the school that A's daughter attends? The question words are daughter, school, and city. If the question splitting prompt word corresponding to the current question-answering stage is "school," then by splitting this multi-stage question based on the question splitting prompt word, we can obtain the target sub-question: Which school does A's daughter attend?

[0051] To summarize, for the current question-answering stage, we use a large language model to split multi-stage questions according to question splitting prompt words, obtain target sub-questions, ensure that multi-stage questions are answered according to the logic of the questions, and improve the accuracy of answering multi-stage questions.

[0052] Step 204: When it is determined that the large language model passes the loop splitting detection based on the target sub-problem and the sub-problem list, the target sub-problem is processed using the large language model according to the historical answer data corresponding to the target sub-problem to obtain target answer data, wherein the historical answer data is constructed based on the answer data corresponding to each question and answer stage before the current question and answer stage.

[0053] Specifically, after splitting the target sub-problem corresponding to the current question-answering stage in the multi-level question, the target sub-problem can be processed using the large language model according to the historical answer data corresponding to the target sub-problem based on the target sub-problem and the sub-problem list to obtain the target answer data. The historical answer data is constructed based on the answer data corresponding to each question-answering stage before the current question-answering stage, and the sub-problem list is used to assist in determining whether the multi-level question is cyclically split, that is, to determine whether the multi-level question has undergone two or more rounds of splitting operations with the same splitting mechanism. The large language model is a deep learning model with large-scale model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than ten trillion model parameters, and is used to process question-answering tasks in a deep thinking manner. The target answer data includes the knowledge points recalled when searching for the target sub-problem, the key knowledge points hit by the target sub-problem among the recalled knowledge points, and the target sub-answers to the target sub-problem determined based on the key knowledge points. The historical answer data corresponding to the target sub-question refers to the sub-question answer data obtained by answering the sub-question corresponding to the previous question-and-answer stage of the current question-and-answer stage. The historical answer data includes the sub-question answer data of the sub-question corresponding to each previous question-and-answer stage. Each sub-question answer data includes the knowledge points recalled when searching for the sub-question, the key knowledge points that the sub-question hits among the recalled knowledge points, and the sub-question answer obtained by answering the sub-question. The target answer data includes the knowledge points recalled when searching for the target sub-question, the key knowledge points that the target sub-question hits among the recalled knowledge points, and the target sub-answer obtained by answering the target sub-question.

[0054] Based on this, after splitting the target sub-problem corresponding to the current question-and-answer stage in the multi-order question, in order to prevent the large language model from falling into an infinite loop of splitting the multi-order question, the large language model can be subjected to loop splitting detection based on the target sub-problem and the sub-problem list to detect whether the large language model is looping through multiple multi-order question splits. Using the large language model, the target sub-problem is processed according to the historical answer data corresponding to the target sub-problem to obtain the target answer data, wherein the historical answer data is constructed based on the answer data corresponding to each question-and-answer stage before the current question-and-answer stage. Since the sub-problem list is used to store the sub-problems obtained by splitting the multi-order question in each question-and-answer stage, after splitting the target sub-problem, the target sub-problem can be stored in the sub-problem list. Before storing the target subproblem in the subproblem list, determine whether there is a subproblem identical to the target subproblem in the subproblem list. If so, it means that the splitting operation of splitting the multi-order problem in the current question-answering stage is a cyclic splitting of the multi-order problem, indicating that the previous subproblem obtained by splitting the multi-order problem in the previous question-answering stage of the current question-answering stage is the same as the target subproblem, and the splitting operation of splitting the multi-order problem in the current question-answering stage is a repeated splitting operation. At this time, it is determined that the large language model has not passed the cyclic splitting test, and the problem splitting in the current question-answering stage is invalid, and further processing of the multi-order problem splitting and sub-problem answering in the next question-answering stage is required; if it does not exist, it means that the splitting operation of splitting the multi-order problem in the current question-answering stage is a normal subproblem splitting and answering step. At this time, it is determined that the large language model has passed the cyclic splitting test, and the problem splitting in the current question-answering stage is valid. The large language model can be used to process the target problem, that is, the target subproblem is processed according to the historical answer data corresponding to the target subproblem using the large language model to obtain the target answer data.

[0055] Furthermore, when using the large language model to process the target sub-question, it is necessary to combine historical answer data and the previous prompt word of the previous question-answering stage corresponding to the current question-answering stage to answer the target sub-question. The specific implementation is as follows:

[0056] Utilizing the large language model, the target sub-problem is processed according to the historical answer data and the previous prompt word corresponding to the target sub-problem to obtain the target answer data; wherein, the previous prompt word is determined based on the previous sub-problem corresponding to the previous question-and-answer stage of the current question-and-answer stage, and is used by the large language model to determine that the processing of the previous sub-problem is completed.

[0057] Specifically, the previous prompt word refers to the prompt word generated based on the previous sub-question after processing the previous sub-question of the previous question-answering stage corresponding to the current question-answering stage, which is used to prompt the large language model to process the target sub-question.

[0058] Based on this, after using the large language model to process the previous sub-problem and obtain the previous answer data, the sub-problem answer of the previous stage is completed. After splitting the current sub-problem from the multi-order question in the current question-answering stage, the current sub-problem needs to be stored in the sub-problem list. Before storing the current sub-problem in the sub-problem list, it is determined whether there is a sub-problem identical to the current sub-problem in the sub-problem list. If there is a sub-problem identical to the current sub-problem in the sub-problem list, it means that the large language model has not passed the loop splitting test and the current sub-problem is the same as the previous sub-problem. At this time, it is necessary to generate a previous prompt word for the previous sub-problem to prompt the large language model that the current sub-problem has been split and it is necessary to continue processing other sub-problems, that is, the target sub-problem. At this time, it is necessary to use the large language model to process the target sub-problem according to the historical answer data and the previous prompt word corresponding to the target sub-problem to obtain the target answer data. The previous prompt word is determined based on the previous sub-problem corresponding to the previous question-answering stage of the current question-answering stage and is used by the large language model to determine that the processing of the previous sub-problem is complete.

[0059] Continuing with the above example, the sub-question generated during the current question-and-answer phase is the target sub-question: Which school does Person A's daughter attend? During the previous question-and-answer phase, the previous sub-question obtained was: Who is Person A's daughter? Since the sub-question list does not contain the same sub-question as the previous one, the previous sub-question can be processed directly. If the current phase splits the multi-level question into the sub-question "Who is Person A's daughter?", this indicates that the large language model is stuck in an infinite loop of splitting the multi-level question. Based on the previous sub-question, it is necessary to generate the previous prompt: "For the current {user_previous sub-question}, {task_previous sub-question} has been executed and {task_results} has been obtained. Please continue the task from other aspects." Therefore, the large language model can be used to process the target sub-question based on the previous prompt and historical answer data. The historical answer data indicates: Person A's daughters include daughter 1, daughter 2, and a younger daughter 3. By processing the target sub-question, we can obtain the stage knowledge data, hitting knowledge point 2: Daughter 1 attends School A, Daughter 2 attends School B, and Daughter 3 attends School C, as well as the stage answer data for the target sub-question: Daughter 1 attends School A, Daughter 2 attends School B. The stage answer data and stage knowledge data for the target sub-question are used as the target answer data.

[0060] In summary, the target sub-question is processed according to the historical answer data corresponding to the target sub-question and the previous prompt word to obtain the target answer data, thereby ensuring that the target answer data obtained by processing the target sub-question has high accuracy.

[0061] Furthermore, considering that the target sub-problem may contain at least two parallel branch sub-problems, in order to ensure the comprehensiveness of the processing of the target sub-problem, it is necessary to process at least two branch sub-problems and determine the target answer data corresponding to the target sub-problem based on the processing results of each branch sub-problem. The specific implementation is as follows:

[0062] In the case where the target sub-problem contains at least two branch sub-problems, the historical answer data corresponding to the target sub-problem is determined; the at least two branch sub-problems are processed sequentially according to the historical answer data using the large language model, or the at least two branch sub-problems are processed in parallel according to the historical answer data using the large language model; the branch answer data corresponding to each branch sub-problem is determined based on the processing result, and the branch answer data corresponding to each branch sub-problem is used as the target answer data.

[0063] Specifically, the at least two branch sub-problems are branch sub-problems in a parallel relationship with each other in the target sub-problem, and are branch sub-problems obtained by splitting the target sub-problem.

[0064] Based on this, when the target sub-problem contains at least two branch sub-problems, the large language model is used to process the at least two branch sub-problems in parallel according to the historical answer data to obtain the branch sub-answer corresponding to each branch sub-problem. The target answer data for the target sub-problem is generated based on the at least two branch sub-answers, that is, the branch sub-answers corresponding to the at least two branch sub-problems are integrated to obtain the target answer data. In addition, when the target sub-problem contains at least two branch sub-problems, the large language model can also be used to process the at least two branch sub-problems in sequence according to the historical answer data to obtain the branch sub-answer corresponding to each branch sub-problem. The target answer data for the target sub-problem is generated based on the at least two branch sub-answers, that is, the branch sub-answers corresponding to the at least two branch sub-problems are integrated to obtain the target answer data.

[0065] Continuing with the previous example, let's consider the target sub-question: "Which school does Person A's daughter attend?" Since the previous sub-question was processed, we obtained historical answer data. This historical answer data includes: Key Point 1: Person A's daughters include Daughter 1, Daughter 2, and a younger daughter, Daughter 3. When processing the target sub-question, it now contains three branch sub-questions: Branch Sub-question 1: "Which school does Daughter 1 attend?"; Branch Sub-question 2: "Which school does Daughter 2 attend?"; Branch Sub-question 3: "Which school does Daughter 3 attend?" These three branch sub-questions can be processed in parallel or sequentially, obtaining the corresponding answer for each branch sub-question: "Daughter 1 attends School A, Daughter 2 attends School B, and Daughter 3 attends School C." In this case, the answer "Daughter 1 attends School A, Daughter 2 attends School B, and Daughter 3 attends School C" is used as the target answer data.

[0066] To sum up, when the target sub-problem contains at least two branch sub-problems, the at least two branch sub-problems are processed sequentially or in parallel to obtain the target answer data, thereby achieving a comprehensive answer to the target sub-problem and avoiding the problem of inaccurate target answers due to missing answers to branch sub-problems.

[0067] Step 206: Split the next sub-problem corresponding to the next question-answering stage from the multi-stage problem, and use the next sub-problem as the target sub-problem, and execute the step of using the large language model to process the target sub-problem according to the historical answer data corresponding to the target sub-problem to obtain target answer data when determining that the large language model passes the loop splitting detection based on the target sub-problem and the sub-problem list.

[0068] Specifically, in the above case where it is determined that the large language model has passed the loop splitting detection based on the target sub-problem and the sub-problem list, the large language model is used to process the target sub-problem according to the historical answer data corresponding to the target sub-problem to obtain the target answer data. After the historical answer data is constructed based on the answer data corresponding to each question and answer stage before the current question and answer stage, the next sub-problem corresponding to the next question and answer stage can be split out in the multi-order problem, and the next sub-problem is used as the target sub-problem, and step 204 is continued. That is, in the case where it is determined that the large language model has passed the loop splitting detection based on the target sub-problem and the sub-problem list, the large language model is used to process the target sub-problem according to the historical answer data corresponding to the target sub-problem to obtain the target answer data, and the processing for the next sub-problem is completed. The obtained target answer data is the next answer data for the next sub-problem.

[0069] Based on this, in the case where the large language model is determined to have passed the loop split detection based on the target sub-problem and the sub-problem list, the target sub-problem is processed using the large language model according to the historical answer data corresponding to the target sub-problem to obtain the target answer data. After the historical answer data is constructed based on the answer data corresponding to each question and answer stage before the current question and answer stage, the next sub-problem corresponding to the next question and answer stage is split out in the multi-stage question, and the next sub-problem is used as the target sub-problem. Furthermore, in the case where the large language model is determined to have passed the loop split detection based on the target sub-problem and the sub-problem list, the target sub-problem is processed using the large language model according to the historical answer data corresponding to the target sub-problem to obtain the target answer data, thereby realizing the processing of the next sub-problem of the next question and answer stage. When splitting the next sub-problem of the next question and answer stage, it can be realized based on the stage knowledge data in the target answer data, rather than based on the stage answer data in the target answer data.

[0070] Continuing with the above example, in the next Q&A phase of the current Q&A phase, when splitting the multi-stage question "What cities are the schools of Person A's daughters located in?", this can be achieved based on the stage-by-stage knowledge data in the target answer data, hitting knowledge point 2: Daughter 1 attends School A, Daughter 2 attends School B, and Daughter 3 attends School C. This is achieved instead of based on the stage-by-stage answer data for the target sub-question: Daughter 1 attends School A, Daughter 2 attends School B. By splitting the multi-stage question into sub-questions for the next Q&A phase based on the stage-by-stage knowledge data in the target answer data, the next sub-question is obtained: Which cities are Schools A, B, and C located in, respectively? Combined with the processing of the target sub-question, the next sub-question in the next Q&A phase will contain three branch questions: Which city is School A located in, which city is School B located in, and which city is School C located in. Process the next sub-problem in the same way as the target sub-problem, obtaining the answer data for the next sub-problem, i.e., School A is in City A, School B is in City B, and School C is in City C, and matching knowledge point 3: School A is located in City A, School B is located in City B, and School C is located in City C. Use the answer data for the next sub-problem and matching knowledge point 3 as the next answer data.

[0071] Step 208: until the answer data corresponding to each question-answering stage is obtained, generate the target answer corresponding to the multi-stage question based on the answer data corresponding to each question-answering stage.

[0072] Specifically, in the above-mentioned multi-stage problem, the next sub-problem corresponding to the next question-answering stage is split out, and the next sub-problem is used as the target sub-problem. When the large language model is determined based on the target sub-problem and the sub-problem list through cyclic split detection, the target sub-problem is processed according to the historical answer data corresponding to the target sub-problem using the large language model. After obtaining the target answer data, the target answer corresponding to the multi-stage problem can be generated based on the answer data corresponding to each question-answering stage. The target answer can include the answer data corresponding to each question-answering stage, as well as the inference answer determined based on the answer data corresponding to each question-answering stage. The target answer can also include the splitting and problem analysis ideas for splitting the multi-stage problem into sub-problems at different stages.

[0073] Based on this, in the above-mentioned step of splitting the next sub-problem corresponding to the next question-and-answer stage in the multi-order problem, and taking the next sub-problem as the target sub-problem, executing the step of determining that the large language model passes the loop splitting detection based on the target sub-problem and the sub-problem list, processing the target sub-problem according to the historical answer data corresponding to the target sub-problem, and obtaining the target answer data, the target answer corresponding to the multi-order problem can be generated based on the answer data corresponding to each question-and-answer stage when the answer data corresponding to each question-and-answer stage is obtained. The target answer includes the reasoning stage for reasoning the multi-order problem and the knowledge content searched in each reasoning stage, as well as the answer to the multi-order problem obtained by integrating the knowledge content searched in each reasoning stage.

[0074] In practice, the answer data for each sub-question at each question-and-answer stage will not affect the processing of subsequent sub-questions. The answer data in the target answer data obtained from processing the target sub-question will not be used to assist in processing the next sub-question of the target sub-question. Instead, it will be used to assist in splitting and processing the next sub-question of the target sub-question based on the knowledge points (stage knowledge data) found when searching for the target sub-question in the target answer data. This can prevent errors in the answer data of the target sub-question from being carried over to the next sub-question of the target sub-question, thereby preventing error accumulation.

[0075] Furthermore, considering that when generating the target answer of a multi-stage question directly based on the answer data of the sub-questions obtained in each question-answering stage, there is a problem of incomplete answer data of the sub-questions, which will lead to low accuracy of the target answer generated in the end, it is necessary to determine the supplementary answer data used to supplement the stage answer data in the stage knowledge data, and generate an accurate target answer based on the supplementary answer data and the stage answer data corresponding to each question-answering stage. The specific implementation is as follows:

[0076] Determine the stage answer data and stage knowledge data in the answer data corresponding to each question and answer stage; determine the supplementary answer data in the stage knowledge data corresponding to each question and answer stage, and generate the target answer corresponding to the multi-stage question based on the supplementary answer data and the stage answer data corresponding to each question and answer stage.

[0077] Specifically, stage answer data refers to the answer data for sub-questions during the question-and-answer phase. This data may contain missing answers, meaning it's incomplete and unable to answer the sub-questions. Stage knowledge data refers to the key knowledge points found during the search for sub-questions during the question-and-answer phase. Answer extraction based on analysis of these key knowledge points can determine the precise answer to the sub-question.

[0078] Based on this, stage answer data representing the predicted answer and stage knowledge data representing the knowledge points hit when searching for sub-questions are determined from the answer data corresponding to each question-and-answer stage. Supplementary answer data is determined from the stage knowledge data corresponding to each question-and-answer stage to supplement the stage answer data. Based on the supplementary answer data and the stage answer data corresponding to each question-and-answer stage, the target answer for the multi-stage question is generated, obtaining a complete and accurate target answer.

[0079] Continuing with the above example, when processing the target sub-question in the current question-and-answer phase to obtain the target answer data, the target answer data includes both the stage answer data and the stage knowledge data. The stage answer data is the answer data for the target sub-question: Daughter 1 attends School A, and Daughter 2 attends School B. The stage knowledge data is the hit knowledge point 2: Daughter 1 attends School A, Daughter 2 attends School B, and there is also a daughter 3 who attends School C. Comparing the answer data with the hit knowledge point shows that the answer data is incomplete, so it is necessary to identify "There is also a daughter 3 who attends School C" in the stage knowledge data as supplementary answer data. When generating the target answer to a multi-order question, you can integrate the answer data of the previous sub-question, the hit knowledge point 1, the answer data of the target sub-question, the hit knowledge point 2, and the answer data of the next sub-question, the hit knowledge point 3 to obtain the target answer to the multi-order question, that is, A's daughters include daughter 1, daughter 2 and youngest daughter 3. Daughter 1 attends School A, daughter 2 attends School B, and there is another daughter 3 who attends School C. School A is in City A, School B is in City B, and School C is in City C.

[0080] To sum up, based on the supplementary answer data and the stage answer data corresponding to each question-and-answer stage, the target answers corresponding to the multi-stage questions are generated to obtain complete and accurate target answers, avoiding the problem of inaccurate target answers due to incomplete stage answer data.

[0081] Furthermore, in the process of answering multi-level questions, the multi-level questions are split into sub-questions and answered at least twice. Since there is a staged answering order between the question-answering stages corresponding to the multi-level questions, in order to facilitate the viewing of the target answer, it is necessary to generate the target answer according to the question order. The specific implementation is as follows:

[0082] Generate answer data based on the stage knowledge data in the answer data corresponding to each question and answer stage, and determine the question order corresponding to the sub-questions of each question and answer stage based on the multi-stage question; according to the question order, generate the target answer corresponding to the multi-stage question based on the answer data and the stage knowledge data corresponding to each question and answer stage.

[0083] Specifically, it refers to the answer to a multi-stage question obtained by analyzing and summarizing the stage knowledge data corresponding to each question-and-answer stage. Question order refers to the order in which the multi-stage question is broken down into its sub-problems. In this embodiment, the question order is: previous sub-problem, target sub-problem, and next sub-problem. The knowledge data at each stage serves as explanation of the answers to the stage sub-problems and is combined with the answer data to generate the target answer. The stage knowledge data is used to assist users in understanding the target answer.

[0084] Based on this, answer data is generated based on the stage knowledge data in the answer data corresponding to each question-and-answer stage. That is, the answer to each question-and-answer stage is obtained by analyzing the stage knowledge data. The question order corresponding to the sub-questions of each question-and-answer stage is determined based on the multi-stage question, and the order in which the multi-stage question is split into sub-questions is used as the question order. According to the question order, the target answer corresponding to the multi-stage question is generated based on the answer data and the stage knowledge data corresponding to each question-and-answer stage. The stage knowledge data for each question-and-answer stage is arranged according to the question order, and the answer data is added to the end of the answer to obtain the target answer to the multi-stage question.

[0085] Continuing with the previous example, when generating the target answer to a multi-stage question, the hit knowledge points and sub-questions are sorted according to the question-answering stage to obtain the previous sub-question, hit knowledge point 1, target sub-question, hit knowledge point 2, next sub-question, and hit knowledge point 3. The answer data is further added: Person A's daughters include daughter 1, daughter 2, and the youngest daughter 3. Daughter 1 attends School A, daughter 2 attends School B, and daughter 3 attends School C. School A is in City A, School B is in City B, and School C is in City C. The answer data serves as a summary and induction for each sub-question and each hit knowledge point.

[0086] To sum up, according to the order of questions, the target answers corresponding to multi-order questions are generated based on the answer data and the stage knowledge data corresponding to each question-and-answer stage, so that the answer data is arranged at the end of the target answer, and the stage knowledge data is arranged in the order of questions, so that the target answer is traceable.

[0087] Furthermore, after obtaining the target data, the answer data of each stage of the knowledge data in the target data can be arranged and displayed on the question page. The specific implementation is as follows:

[0088] In the question page corresponding to the multi-stage question, the stage knowledge data corresponding to each question-and-answer stage is displayed in sequence, and the answer data is displayed.

[0089] Based on this, the knowledge data and answer data for each stage are displayed sequentially on the question page corresponding to the multi-stage question, that is, the knowledge data and answer data for each stage are typeset and displayed according to the order in which the data is determined. Alternatively, the knowledge data for each stage can be displayed on the question page after it is determined, that is, when the knowledge data for the previous stage is determined, the knowledge data for the previous stage is displayed on the question page, and then when the knowledge data for the stage corresponding to the target sub-question is determined, the knowledge data for the stage is displayed on the question page, and when the answer data is determined, the answer data is displayed on the question page. This allows the knowledge data for the stage and the answer data to be displayed instantly, reducing the time users have to wait for the answer data.

[0090] Continuing with the above example, after determining the stage knowledge data of the previous subproblem, the stage knowledge data of the target subproblem, the stage knowledge data of the next subproblem, and the answer data, arrange the stage knowledge data of the previous subproblem before the stage knowledge data of the target subproblem, arrange the stage knowledge data of the target subproblem before the stage knowledge data of the next subproblem, and arrange the stage knowledge data of the next subproblem before the answer data, and display them in sequence on the question page.

[0091] To summarize, in the question page corresponding to the multi-stage question, the stage knowledge data of the previous sub-question, the stage knowledge data of the target sub-question, the stage knowledge data and the answer data of the next sub-question are displayed in sequence, thereby improving the visualization of the target answer and enhancing the user's visual experience.

[0092] An embodiment of the present specification provides a multi-stage question-answering method, which splits the target sub-problem corresponding to the current question-answering stage in the multi-stage question; when it is determined based on the target sub-problem and the sub-problem list that the large language model has passed the cyclic splitting detection, the target sub-problem is processed using the large language model according to the historical answer data corresponding to the target sub-problem to obtain the target answer data, wherein the historical answer data is constructed based on the answer data corresponding to each question-answering stage before the current question-answering stage; the next sub-problem corresponding to the next question-answering stage is split in the multi-stage question, and the next sub-problem is used as the target sub-problem, and the step of processing the target sub-problem according to the historical answer data corresponding to the target sub-problem is performed based on the target sub-problem and the sub-problem list. When it is determined based on the target sub-problem and the sub-problem list that the large language model has passed the cyclic splitting detection, the target sub-problem is processed using the large language model according to the historical answer data corresponding to the target sub-problem to obtain the target answer data, and the multi-stage question is gradually answered by splitting the complex multi-stage question. Until the answer data corresponding to each question-answering stage is obtained, the target answer corresponding to the multi-stage question is generated based on the answer data corresponding to each question-answering stage.

[0093] The sub-question list can be used to determine whether the large language model is trapped in a loop splitting operation on multi-level questions, preventing the large language model from performing multiple, ineffective splits on multi-level questions. This saves the computing resources used by the large language model when processing multi-level questions and improves the performance of the large language model when handling complex questions. The target answer for the multi-level question is generated based on the answer data corresponding to each question-and-answer stage. This integrates the phased answer data obtained from at least two question-and-answer stages, preventing incorrect answers generated in the intermediate processing stages from affecting the accuracy of the target answer, thereby improving the user's question-and-answer experience.

[0094] The following combined Figure 3 Taking the application of the multi-stage question answering method provided in this specification in complex question answering as an example, the multi-stage question answering method is further explained. Figure 3 A flowchart of the processing process of a multi-stage question-answering method provided by an embodiment of this specification is shown, which specifically includes the following steps.

[0095] Step 302: Split the multi-stage question into the target sub-question corresponding to the current question-answering stage.

[0096] Multi-stage questions are complex questions, that is, questions that require at least two question-answering steps to answer. In complex question-answering scenarios, such as Figure 4 As shown, you can enter a complex question on the question page: "Dates of establishment of all wholly owned subsidiaries of Company A."

[0097] Step 304: Store the target sub-question into a sub-question list, and perform loop split detection on the large language model based on the sub-question list.

[0098] The complex question "What are the establishment dates of all wholly-owned subsidiaries of Company A?" is split according to the question word "wholly-owned subsidiaries" contained in the question to obtain the target sub-question: "1. What are all wholly-owned subsidiaries of Company A?"

[0099] Step 306: When the large language model passes the loop splitting detection, the target sub-question is processed using the large language model to obtain target answer data.

[0100] Answer the target sub-question "1. What are all the wholly-owned subsidiaries of Company A?" and obtain target answer data. The target answer data includes the hit key knowledge points and target answer data. The hit key knowledge points are: Hit key knowledge point 1: Company A's wholly-owned subsidiaries are Company A-1 and Company A-2, and there is another company, Company A-3. The target answer data is: Company A's wholly-owned subsidiaries are Company A-1 and Company A-2.

[0101] Step 308: Split the multi-stage question into the next sub-question of the next question-answering stage corresponding to the current question-answering stage.

[0102] Step 310: Store the next sub-question into a sub-question list, and perform loop split detection on the large language model based on the sub-question list.

[0103] Step 312: When the large language model passes the loop split detection, the large language model is used to process the next sub-question according to the target answer data and target prompt words associated with the target sub-question to obtain the next answer data.

[0104] When the large language model processes a multi-level question, after processing the target sub-problem, it may repeatedly split the multi-level question into sub-questions. In the next question-answering phase, when the next sub-question is split, it can determine whether the next sub-question is the same as the target sub-problem stored in the sub-question list. If so, this indicates that the current question splitting step is an infinite loop, resulting in an infinite loop splitting issue, and also indicates that the large language model has failed the loop splitting test. At this point, a target prompt word should be constructed based on the model prompt information and the question processing information obtained from processing the target sub-problem. The target prompt word could be "For the current {user_query}, {task_query} has been executed and {task_results} have been obtained. Please continue the task from other aspects." If not, the current splitting phase of the multi-level question is normal, indicating that the large language model has passed the loop splitting test and can continue processing the next sub-question: "2. What are the establishment dates of the wholly-owned subsidiaries?"

[0105] When the large language model processes the next sub-question, "2. What are the establishment dates of the wholly-owned subsidiaries?", it can refer to the target sub-question's processing method to solve the problem while preventing infinite loops. The next sub-question, "2. What are the establishment dates of the wholly-owned subsidiaries?", is processed to obtain the next answer data. The next answer data contains the hit key knowledge point and the next answer data. The hit key knowledge point is: Hit key knowledge point 2: Company A-1 was established on June 7, 2020, Company A-2 was established on June 7, 2020, and Company A-3 was established on June 7, 2020. The next answer data is that both Company A-1 and Company A-2 were established on June 7, 2020.

[0106] Step 314: Generate a target answer corresponding to the multi-stage question based on the target answer data and the next answer data, and display the target answer on the question page corresponding to the multi-stage question.

[0107] Analyze key knowledge point 1 in the target answer data and key knowledge point 2 in the next answer data to determine the target answer for the multi-level question. The target answer includes the following: Key knowledge point 1: Company A's wholly-owned subsidiaries are Company A-1 and Company A-2, and another company, Company A-3; Key knowledge point 2: Company A-1 was established on June 7, 2020, Company A-2 was established on June 7, 2020, and Company A-3 was established on June 7, 2020; and the answer obtained by analyzing and summarizing the key knowledge: Company A's wholly-owned subsidiaries, Company A-1, Company A-2, and Company A-3, were all established on June 7, 2020.

[0108] like Figure 4 The question page provides a question search box where you can enter the question you want to ask. When displaying the target answer to the user, it shows the thinking process when solving complex questions, the knowledge points hit, and the summarized answer.

[0109] In summary, when using a large language model to answer complex questions, a judgment module is used to address the infinite loop problem that may occur during the large language model's question-answering process. Specifically, the judgment module determines whether the large language model is stuck in a loop splitting a multi-stage question based on a list of sub-questions. When answering at least two sub-questions, each sub-question is searched in parallel, analyzing and integrating the corresponding stage answer data for each sub-question, thus avoiding the error accumulation caused by traditional serial question-answering. The think-while-output mode only displays the thought process when answering multi-stage questions, achieving a think-while-execute effect. This reduces the waiting time for users to receive answers to their questions and improves the user experience.

[0110] The following combined Figure 5Taking the application of the multi-stage question answering method provided in this specification in complex question answering as an example, the multi-stage question answering method is further explained. Figure 5 A question processing flow chart of a multi-stage question-answering method provided by an embodiment of this specification is shown, which specifically includes the following steps.

[0111] like Figure 5 The following is a flowchart of the multi-level question answering method. The query is the multi-level question that requires processing using a large language model. A multi-level question might be: "Analyze and summarize the main trends in global climate change over the past decade and its impact on agriculture." First, determine whether the multi-level question has been completed, that is, whether it has been fully solved. If it has, the result is directly output, and the question answering process ends. If it has not, proceed to the next step to determine whether the task has been split. Since this multi-level question is new and has not yet been split, it needs to be executed immediately to process the multi-level question. The multi-level question is split into sub-questions, which can be displayed on the front-end page during the multi-level question splitting process. The first stage of the multi-level question splitting results in the following sub-questions: Question 1: Analyze and summarize the main trends in global climate change over the past decade. Question 1 is the first target sub-question to be processed using RAG. The possible hit of knowledge point 1: Data on global temperature changes and changes in global rainfall patterns over the past decade is the chunk with the answer. At this point, Knowledge Point 1 and the answer to Question 1 are treated as historical answers, waiting for subsequent questions to be processed before the answer data is concatenated. Question 1 also needs to be stored in a sub-question list to facilitate the multi-stage question splitting and judgment of the large language model.

[0112] After processing question 1, a large language model may repeatedly split the multi-level question, i.e., fall into a loop of splitting the multi-level question. In this case, it is necessary to first determine whether the question-answering task for the multi-level question has been completed. Since the multi-level question can be further split into sub-questions, such as question 2: The impact of global climate change on agriculture in the past decade, and sub-question 2 has not yet been addressed, the task is determined to be incomplete. Further, based on the sub-question list, it is determined whether the question-answering task for question 2 has been split. If it has, a prompt is generated, indicating that the task corresponding to the multi-level question has been split and no further splitting is required, and the sub-questions after question 1 can be processed. If it has not been split, question 2 can be used as the next sub-question for further processing. Question 2 is processed using RAG, and Knowledge Point 2 is identified: "Over the past decade, the global average temperature has risen by 0.1°C / year, leading to an increase in extreme weather events, which has negatively impacted agricultural production. For every 1°C increase in temperature, wheat yields decrease by an average of 6%. Over the past decade, global wheat production has decreased by approximately 10% due to rising temperatures." At this point, the task is again determined to be complete. Since all subquestions of the multi-level question have been answered, the task is complete, and the answering process for the multi-level question is complete. The task is then checked to see if the hit chunk is empty. If so, the multi-level question is directly answered again based on the RAG, and the results are displayed on the front-end page. If the hit chunk is not empty, meaning that chunks 1-chunkN (knowledge points 1 and 2) are hit, the multi-level question is answered based on the recalled content of each subquestion and the hit knowledge points 1 and 2. The generated results include the thought process used to solve the complex question, the hit knowledge points, and the summarized answer. The displayed answer might be: "The hit chunk for each subquestion obtained by breaking down the complex question, and the answer summarized based on the hit chunks: Over the past decade, the global average temperature has risen by 0.1°C / year, leading to an increase in extreme weather events, which has negatively impacted agricultural production. Specifically, for every 1°C increase in temperature, wheat yields decrease by an average of 6%, and global total wheat production decreases by approximately 10%." This is the final answer presented to the user.

[0113] In summary, during the complex problem-solving process, a judgment module is proposed to determine whether the multi-level problem to be answered has already been split. This prevents the large language model from falling into the loop of multi-level problem splitting logic. During the multi-level problem splitting process, the processed sub-problems are stored in a sub-problem list. If the large language model finds that the next time it splits the multi-level problem, it obtains a sub-problem that is identical to the sub-problem stored in the sub-problem list, a supplementary prompt is immediately added to the task splitting module of the large language model, informing the splitting module that the problem does not need to be split and that the multi-level problem can be solved from other tasks. By adding prompt constraints, the large language model can effectively prevent the error of infinite loop splitting of multi-level problems. The knowledge points hit during the search for each sub-problem are stored in a cache. When generating the answer, the accuracy of the answer can be guaranteed only by answering the question based on the knowledge points accumulated in the cache. The cached knowledge points do not include all the recalled knowledge points, but only the knowledge points hit during the search for each sub-problem. This significantly reduces the amount of cached data and the processing pressure on the model. At the same time, the question-and-answer process for multi-level questions adopts a think-and-output model, that is, the user's front-end page only displays the thinking process, and the task execution process and the answer to each sub-question are only completed in the background, achieving the effect of think-and-execute, which can reduce the waiting time for users of multi-level questions and improve the user experience.

[0114] Corresponding to the above method embodiment, this specification also provides a multi-stage question answering system embodiment, Figure 6 FIG1 shows a schematic diagram of the structure of a multi-stage question answering system provided by an embodiment of this specification. Figure 6 As shown, the multi-level question answering system 600 includes a client 610 and a server 620; the client 610 is used to submit a multi-level question to the server 620; the server 620 is used to split the target sub-problem corresponding to the current question-answering stage in the multi-level question; when it is determined based on the target sub-problem and the sub-problem list that the large language model passes the loop splitting detection, the target sub-problem is processed using the large language model according to the historical answer data corresponding to the target sub-problem to obtain target answer data, wherein the historical answer data is constructed based on the answer data corresponding to each question-answering stage before the current question-answering stage; the next sub-problem corresponding to the next question-answering stage is split from the multi-level question, and the next sub-problem is used as the target sub-problem, and the step of determining that the large language model passes the loop splitting detection is executed when the target sub-problem is successfully stored in the sub-problem list; until the answer data corresponding to each question-answering stage is obtained, the target answer corresponding to the multi-level question is generated based on the answer data corresponding to each question-answering stage, and the target answer is sent to the client 610.

[0115] In practical applications, in complex problem scenarios, complex problems are usually questions that require multiple stages of question-and-answering to obtain answers. Users can enter multi-stage questions in the question-and-answer interface and submit the multi-stage questions to the server through the question-and-answer interface. The server splits the multi-stage question into target sub-problems corresponding to the current question-and-answer stage; when it is determined based on the target sub-problem and the sub-problem list that the large language model has passed the cyclic split detection, the large language model is used to process the target sub-problem according to the historical answer data corresponding to the target sub-problem to obtain target answer data, wherein the historical answer data is constructed based on the answer data corresponding to each question-and-answer stage before the current question-and-answer stage; the next sub-problem corresponding to the next question-and-answer stage is split from the multi-stage question, and the next sub-problem is used as the target sub-problem. When it is determined based on the target sub-problem and the sub-problem list that the large language model has passed the cyclic split detection, the large language model is used to process the target sub-problem according to the historical answer data corresponding to the target sub-problem to obtain the target answer data. By splitting the complex multi-stage question, the multi-stage question can be gradually answered. Until the answer data corresponding to each question-and-answer stage is obtained, the target answers corresponding to the multi-stage questions are generated based on the answer data corresponding to each question-and-answer stage, and the target answers are sent to the client. The sub-question list can be used to determine whether the large language model is trapped in a cyclic splitting operation of the multi-stage question, avoiding the large language model from performing multiple invalid splits on the multi-stage question, thereby saving the computing resources used by the large language model when processing multi-stage questions and improving the performance of the large language model when processing complex questions. Based on the answer data corresponding to each question-and-answer stage, the target answers corresponding to the multi-stage questions are generated, and the phased answer data obtained from at least two question-and-answer stages are integrated to avoid the incorrect answers generated in the intermediate processing stage affecting the accuracy of the target answer, thereby improving the user's question-and-answer experience.

[0116] The above is a schematic scheme of a multi-stage question-answering system of this embodiment. It should be noted that the technical scheme of the multi-stage question-answering system and the technical scheme of the multi-stage question-answering method described above are based on the same concept. For details not described in detail in the technical scheme of the multi-stage question-answering system, please refer to the description of the technical scheme of the multi-stage question-answering method described above.

[0117] Corresponding to the above method embodiment, this specification also provides a multi-stage question-answering device embodiment, Figure 7 FIG1 shows a schematic diagram of the structure of a multi-stage question-answering device provided by an embodiment of this specification. Figure 7 As shown, the device includes:

[0118] A splitting module 702 is configured to split the multi-stage question into target sub-questions corresponding to the current question-answering stage;

[0119] Processing module 704 is configured to, when it is determined based on the target sub-question and the sub-question list that the large language model passes the loop splitting test, process the target sub-question using the large language model according to the historical answer data corresponding to the target sub-question to obtain target answer data, wherein the historical answer data is constructed based on answer data corresponding to each question and answer stage before the current question and answer stage;

[0120] An execution module 706 is configured to split the multi-stage question into a next sub-problem corresponding to the next question-answering stage, and use the next sub-problem as the target sub-problem. When it is determined based on the target sub-problem and the sub-problem list that the large language model passes the loop splitting test, the execution module 706 processes the target sub-problem using the large language model according to the historical answer data corresponding to the target sub-problem to obtain target answer data.

[0121] The generation module 708 is configured to generate the target answer corresponding to the multi-stage question based on the answer data corresponding to each question and answer stage until the answer data corresponding to each question and answer stage is obtained.

[0122] In an optional embodiment, the processing module 704 is further configured to:

[0123] Using the large language model, the target sub-question is processed according to the historical answer data and the previous prompt word corresponding to the target sub-question to obtain the target answer data;

[0124] The previous prompt word is determined based on the previous sub-question corresponding to the previous question-answering stage of the current question-answering stage, and is used by the large language model to determine whether the processing of the previous sub-question is completed.

[0125] In an optional embodiment, the processing module 704 is further configured to:

[0126] Determining the historical answer data corresponding to the target sub-question;

[0127] Using the large language model to process the at least two branch sub-questions sequentially according to the historical answer data, or using the large language model to process the at least two branch sub-questions in parallel according to the historical answer data;

[0128] The branch answer data corresponding to each branch sub-question is determined according to the processing result, and the branch answer data corresponding to each branch sub-question is used as the target answer data.

[0129] In an optional embodiment, the generating module 708 is further configured to:

[0130] Determine the stage answer data and stage knowledge data in the answer data corresponding to each question and answer stage;

[0131] Supplementary answer data is determined in the stage knowledge data corresponding to each question-and-answer stage, and the target answer corresponding to the multi-stage question is generated based on the supplementary answer data and the stage answer data corresponding to each question-and-answer stage.

[0132] In an optional embodiment, the generating module 708 is further configured to:

[0133] Generate answer data based on the stage knowledge data in the answer data corresponding to each question-answering stage, and determine the question order corresponding to the sub-questions of each question-answering stage based on the multi-stage question;

[0134] According to the question sequence, the target answers corresponding to the multi-stage questions are generated based on the answer data and the stage knowledge data corresponding to each question-and-answer stage.

[0135] In an optional embodiment, the generating module 708 is further configured to:

[0136] In the question page corresponding to the multi-stage question, the stage knowledge data corresponding to each question-and-answer stage is displayed in sequence, and the answer data is displayed.

[0137] In an optional embodiment, the splitting module 702 is further configured to:

[0138] Determining at least two phase question words included in the multi-stage question, and determining the question splitting prompt words corresponding to the current question-answering phase from the at least two phase question words;

[0139] The large language model is used to split the multi-stage question according to the question splitting prompt words to obtain the target sub-questions.

[0140] The multi-stage question-answering device provided by one embodiment of the present specification splits the target sub-problem corresponding to the current question-answering stage in the multi-stage question; when it is determined based on the target sub-problem and the sub-problem list that the large language model has passed the cyclic splitting detection, the target sub-problem is processed according to the historical answer data corresponding to the target sub-problem using the large language model to obtain the target answer data, wherein the historical answer data is constructed based on the answer data corresponding to each question-answering stage before the current question-answering stage; the next sub-problem corresponding to the next question-answering stage is split in the multi-stage question, and the next sub-problem is used as the target sub-problem, and the step of processing the target sub-problem according to the historical answer data corresponding to the target sub-problem is performed based on the target sub-problem and the sub-problem list. When it is determined based on the target sub-problem and the sub-problem list that the large language model has passed the cyclic splitting detection, the target sub-problem is processed according to the historical answer data corresponding to the target sub-problem using the large language model to obtain the target answer data, and the complex multi-stage question is split to achieve a step-by-step solution to the multi-stage question. Until the answer data corresponding to each question-answering stage is obtained, the target answer corresponding to the multi-stage question is generated based on the answer data corresponding to each question-answering stage. The sub-question list can be used to determine whether the large language model is trapped in a loop splitting operation on multi-level questions, preventing the large language model from performing multiple, ineffective splits on multi-level questions. This saves the computing resources used by the large language model when processing multi-level questions and improves the performance of the large language model when handling complex questions. The target answer for the multi-level question is generated based on the answer data corresponding to each question-and-answer stage. This integrates the phased answer data obtained from at least two question-and-answer stages, preventing incorrect answers generated in the intermediate processing stages from affecting the accuracy of the target answer, thereby improving the user's question-and-answer experience.

[0141] The above is a schematic diagram of a multi-stage question-answering device according to this embodiment. It should be noted that the technical solution of the multi-stage question-answering device and the technical solution of the multi-stage question-answering method described above are based on the same concept. For details not described in detail in the technical solution of the multi-stage question-answering device, please refer to the description of the technical solution of the multi-stage question-answering method described above.

[0142] Figure 8 8 shows a block diagram of a computing device 800 according to one embodiment of the present disclosure. Components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0143] Computing device 800 also includes an access device 840 that enables computing device 800 to communicate via one or more networks 860. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. Access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, or a near field communication (NFC) interface.

[0144] In one embodiment of the present specification, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 8 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0145] Computing device 800 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or personal computer (PC). Computing device 800 can also be a mobile or stationary server.

[0146] The processor 820 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the multi-stage question-answering method.

[0147] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the aforementioned multi-stage question-answering method are based on the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the aforementioned multi-stage question-answering method.

[0148] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned multi-stage question-answering method.

[0149] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of this storage medium and the technical scheme of the multi-stage question-answering method described above are based on the same concept. For details not described in detail in the technical scheme of the storage medium, please refer to the description of the technical scheme of the multi-stage question-answering method described above.

[0150] An embodiment of the present specification further provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps of the above-mentioned multi-stage question-answering method.

[0151] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of this computer program product and the technical scheme of the aforementioned multi-stage question-answering method are based on the same concept. For details not described in detail in the technical scheme of the computer program product, please refer to the description of the technical scheme of the aforementioned multi-stage question-answering method.

[0152] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0153] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content of the computer-readable medium may be appropriately increased or decreased based on the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0154] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0155] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0156] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of the embodiments described herein. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification.

Claims

1. A multi-stage question answering method, characterized in that: include: Split the target sub-problems corresponding to the current question-answering stage in the multi-stage problem; When it is determined that the large language model passes the loop splitting test based on the target sub-problem and the sub-problem list, the target sub-problem is processed using the large language model according to the historical answer data corresponding to the target sub-problem to obtain target answer data, wherein the historical answer data is constructed based on answer data corresponding to each question and answer stage before the current question and answer stage, and the loop splitting test is used to detect whether there is a sub-problem in the sub-problem list that is the same as the target sub-problem; Splitting the multi-stage question into a next sub-problem corresponding to the next question-answering stage, and using the next sub-problem as the target sub-problem, and executing the step of, upon determining based on the target sub-problem and the sub-problem list that the large language model passes the loop splitting detection, processing the target sub-problem using the large language model according to the historical answer data corresponding to the target sub-problem to obtain target answer data; Until the answer data corresponding to each question-answering stage is obtained, the target answer corresponding to the multi-stage question is generated based on the answer data corresponding to each question-answering stage.

2. The multi-stage question answering method according to claim 1, characterized in that: The step of processing the target sub-question using the large language model according to the historical answer data corresponding to the target sub-question to obtain target answer data includes: Using the large language model, the target sub-question is processed according to the historical answer data and the previous prompt word corresponding to the target sub-question to obtain the target answer data; The previous prompt word is determined based on the previous sub-question corresponding to the previous question-answering stage of the current question-answering stage, and is used by the large language model to determine whether the processing of the previous sub-question is completed.

3. The multi-stage question answering method according to claim 1, characterized in that: In a case where the target sub-problem includes at least two branch sub-problems, the using of the large language model to process the target sub-problem according to the historical answer data corresponding to the target sub-problem to obtain target answer data includes: Determining the historical answer data corresponding to the target sub-question; Using the large language model to process the at least two branch sub-questions sequentially according to the historical answer data, or using the large language model to process the at least two branch sub-questions in parallel according to the historical answer data; The branch answer data corresponding to each branch sub-question is determined according to the processing result, and the branch answer data corresponding to each branch sub-question is used as the target answer data.

4. The multi-stage question answering method according to claim 1, characterized in that: Generating target answers corresponding to the multi-stage questions based on the answer data corresponding to each question-answering stage includes: Determine the stage answer data and stage knowledge data in the answer data corresponding to each question and answer stage; Supplementary answer data is determined in the stage knowledge data corresponding to each question-and-answer stage, and the target answer corresponding to the multi-stage question is generated based on the supplementary answer data and the stage answer data corresponding to each question-and-answer stage.

5. The multi-stage question answering method according to claim 4, characterized in that: After determining the stage answer data and stage knowledge data in the answer data corresponding to each question-answering stage, the method further includes: Generate answer data based on the stage knowledge data in the answer data corresponding to each question-answering stage, and determine the question order corresponding to the sub-questions of each question-answering stage based on the multi-stage question; According to the question sequence, the target answers corresponding to the multi-stage questions are generated based on the answer data and the stage knowledge data corresponding to each question-and-answer stage.

6. The multi-stage question answering method according to claim 5, characterized in that: After generating the target answers corresponding to the multi-stage questions according to the question sequence based on the answer data, and the stage answer data and stage knowledge data corresponding to each question-and-answer stage, the method further includes: In the question page corresponding to the multi-stage question, the stage knowledge data corresponding to each question-and-answer stage is displayed in sequence, and the answer data is displayed.

7. The multi-stage question answering method according to claim 1, characterized in that: The target sub-problems corresponding to the current question-answering stage are split out from the multi-stage questions, including: Determining at least two phase question words included in the multi-stage question, and determining question splitting prompt words corresponding to the current question-answering stage from among the at least two phase question words, wherein the at least two phase question words are phrases in the multi-stage question, and the question splitting prompt words are phase question words corresponding to the current question-answering stage from among the at least two phase prompt words; The large language model is used to split the multi-stage question according to the question splitting prompt words to obtain the target sub-questions.

8. A multi-stage question answering system, characterized in that: Including client and server; The client is used to submit multi-level questions to the server; The server is configured to split the target subproblem corresponding to the current question-answering stage from the multi-stage question; upon determining that the large language model passes the loop splitting test based on the target subproblem and the subproblem list, use the large language model to process the target subproblem according to the historical answer data corresponding to the target subproblem to obtain target answer data, wherein the historical answer data is constructed based on the answer data corresponding to each question-answering stage before the current question-answering stage, and the loop splitting test is used to detect whether there is a subproblem identical to the target subproblem in the subproblem list; split the next subproblem corresponding to the next question-answering stage from the multi-stage question, and use the next subproblem as the target subproblem, and execute the step of, upon determining that the large language model passes the loop splitting test based on the target subproblem and the subproblem list, using the large language model to process the target subproblem according to the historical answer data corresponding to the target subproblem to obtain target answer data; until the answer data corresponding to each question-answering stage is obtained, generate the target answer corresponding to the multi-stage question based on the answer data corresponding to each question-answering stage, and send the target answer to the client.

9. A multi-stage question-answering device, characterized in that: include: The splitting module is configured to split the multi-stage question into the target sub-questions corresponding to the current question-answering stage; a processing module configured to, upon determining based on the target sub-question and the sub-question list that the large language model passes the loop splitting test, process the target sub-question using the large language model according to historical answer data corresponding to the target sub-question to obtain target answer data, wherein the historical answer data is constructed based on answer data corresponding to each question-and-answer stage before the current question-and-answer stage, and the loop splitting test is used to detect whether there is a sub-question identical to the target sub-question in the sub-question list; an execution module configured to split the multi-stage question into a next sub-problem corresponding to the next question-answering stage, and use the next sub-problem as the target sub-problem, and execute the step of, upon determining based on the target sub-problem and the sub-problem list that the large language model passes the loop splitting test, processing the target sub-problem using the large language model according to the historical answer data corresponding to the target sub-problem to obtain target answer data; The generation module is configured to generate the target answer corresponding to the multi-stage question based on the answer data corresponding to each question and answer stage until the answer data corresponding to each question and answer stage is obtained.

10. A computing device, characterized in that include: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the multi-stage question-answering method described in any one of claims 1 to 7 are implemented.

11. A computer-readable storage medium, characterized in that It stores computer-executable instructions, which, when executed by a processor, implement the steps of the multi-stage question-answering method described in any one of claims 1 to 7.

12. A computer program product, characterized in that The method comprises a computer program or instructions, which, when executed by a processor, implements the steps of the multi-stage question answering method according to any one of claims 1 to 7.

Images