Question and answer method and device, nonvolatile storage medium and electronic equipment

Abstract

The invention discloses a question and answer method and device, a nonvolatile storage medium and electronic equipment. The method comprises the steps that a question input by a target object is received, and semantic information in the question is determined; the semantic information is input into the first model, a sub-step set output by the first model is obtained, and the sub-step set comprises all sub-steps needed for question answering; inputting the question and the sub-step set into a second model, and obtaining sub-step answers which are output by the second model and are in one-to-one correspondence with sub-steps in the sub-step set; and verifying the sub-step answers of all the sub-steps in the sub-step set, determining a final answer corresponding to the question according to the sub-step answers of all the sub-steps under the condition that the verification result of the sub-step answers is successful, and outputting the final answer. According to the method and the device, the technical problem of inaccurate answer caused by direct generation of a final answer during question answering of a large language model is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a question-answering method, device, non-volatile storage medium, and electronic device. Background Art

[0002] In recent years, with the advancement of deep learning technology, especially the development of large-scale pre-trained language models (such as BERT, GPT series, etc.), the application of natural language processing (NLP) has made significant breakthroughs. These models have mastered rich language knowledge by learning from large amounts of text data, and can perform well in multiple tasks such as machine translation, text summarization, question-answering systems, etc.

[0003] However, although these models have achieved performance close to or even above human level on many tasks, they still fail to perform well on some problems that require deep reasoning and multi-step logical deduction. For example, when solving problems that require multiple consecutive steps to reach a conclusion, existing large language models often accumulate errors or their reasoning paths deviate from the correct direction.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a question-answering method, apparatus, non-volatile storage medium, and electronic device to at least solve the technical problem of inaccurate answers caused by a large language model directly generating a final answer during question-answering.

[0006] According to one aspect of an embodiment of the present application, a question-answering method is provided, including: receiving a question input by a target object, and determining semantic information in the question; inputting the semantic information into a first model, and obtaining a sub-step set output by the first model, wherein the sub-step set includes each sub-step required to answer the question; inputting the question and the sub-step set into a second model, and obtaining sub-step answers output by the second model that correspond one-to-one to each sub-step in the sub-step set; verifying the sub-step answers of all sub-steps in the sub-step set, and when the verification result of the sub-step answer is verification passed, determining a final answer corresponding to the question based on the sub-step answers of all sub-steps, and outputting the final answer.

[0007] Optionally, determining the semantic information in the question includes: extracting key information in the question, wherein the key information includes keywords and key phrases; determining key information identifiers corresponding to the key information; and converting the key information and the key information identifiers into preset structured data as the semantic information of the question.

[0008] Optionally, after obtaining the set of sub-steps output by the first model, the method further includes: inputting the sub-steps and step verification prompt words into the first model for step verification, wherein the step verification prompt words are used to indicate the first model verification sub-steps, and the step verification includes at least one of the following: verifying the logic of the sub-steps, verifying the redundant status of the sub-steps, and verifying similar sub-steps of the sub-steps; obtaining the sub-step verification results output by the first model; and optimizing the sub-steps based on the sub-step verification results.

[0009] Optionally, optimizing the sub-steps based on the sub-step verification results includes: when the sub-step verification result is that the logic of the sub-step is incorrect, correcting the sub-step based on the first model, and checking the corrected sub-step based on the first model to obtain the sub-step verification result of the corrected sub-step; when the sub-step verification result is that the sub-step is redundant, deleting the sub-step; when the sub-step verification result is that there are similar sub-steps in the sub-step, merging the sub-step and the corresponding similar sub-step.

[0010] Optionally, after verifying the sub-step answer, the method further includes: if the verification result of the sub-step answer is a verification failure, re-obtaining the sub-step answer according to the second model; and verifying the re-obtained sub-step answer again according to the second model.

[0011] Optionally, a final answer corresponding to the question is determined based on the sub-step answers of all sub-steps, including: inputting the answers of each sub-step into a third model to obtain an answer summary output by the third model; and determining the answer summary and each sub-step and the answers of each sub-step as the final answer.

[0012] Optionally, before the answer summary and each sub-step and each sub-step answer are determined as the final answer, the method also includes: checking whether there is a logical contradiction between the answers to each sub-step based on a third model; when it is found that there is a contradiction between the answers to the sub-steps, correcting the sub-step answers based on the third model; and checking whether there is a logical contradiction between the corrected answers to the sub-steps based on the third model.

[0013] According to another aspect of an embodiment of the present application, a question-and-answer device is also provided, including: a determination module for receiving a question input by a target object and determining semantic information in the question; a first acquisition module for inputting the semantic information into a first model and acquiring a sub-step set output by the first model, wherein the sub-step set includes each sub-step required to answer the question; a second acquisition module for inputting the question and the sub-step set into a second model and acquiring sub-step answers output by the second model that correspond one-to-one to each sub-step in the sub-step set; an output module for verifying the sub-step answers of all sub-steps in the sub-step set, and when the verification result of the sub-step answer is verification passed, determining a final answer corresponding to the question based on the sub-step answers of all sub-steps and outputting the final answer.

[0014] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, in which a program is stored, wherein when the program is running, a device where the non-volatile storage medium is located is controlled to execute a question-and-answer method.

[0015] According to another aspect of an embodiment of the present application, there is further provided an electronic device, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the question-and-answer method is executed when the program is run.

[0016] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a computer program, and the computer program implements the question-and-answer method when executed by a processor.

[0017] In an embodiment of the present application, a question input by a target object is received to determine the semantic information in the question; the semantic information is input into a first model to obtain a sub-step set output by the first model, wherein the sub-step set includes each sub-step required to answer the question; the question and the sub-step set are input into a second model to obtain a sub-step answer output by the second model that corresponds to each sub-step in the sub-step set; the sub-step answers of all sub-steps in the sub-step set are verified, and when the verification result of the sub-step answer is passed, the final answer corresponding to the question is determined based on the sub-step answers of all sub-steps, and the final answer is output. By receiving and parsing user questions, a structured CoT (chain of thought) step is generated, the CoT is verified and optimized using a large model, each sub-step is executed and an intermediate answer is generated, the intermediate answer is verified and self-corrected, the answer is integrated and the consistency is checked, and finally a final answer containing a detailed CoT description is output, thereby achieving the purpose of improving the accuracy and transparency of the large language model in processing complex logical problems, thereby achieving the technical effect of improving the interpretability, robustness and flexibility of the large prediction model in processing problems, and thus solving the technical problem of inaccurate answers caused by the large language model directly generating the final answer when asking and answering questions. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a structural schematic diagram of a computer terminal provided according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of the architecture of each module according to an embodiment of the present application;

[0021] Figure 3is a schematic diagram of an interaction process of each module according to an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of an interaction process of each module according to an embodiment of the present application;

[0023] Figure 5 It is a flowchart of a question-and-answer method provided according to an embodiment of the present application;

[0024] Figure 6 4 is a schematic diagram of the structure of a question-answering device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0028] CoT (Chain of Thought): is an artificial intelligence technology that enhances the model's reasoning ability by simulating the human thought process when solving problems. When dealing with complex problems, CoT helps the model better understand the problem and find a solution by gradually showing the thinking process.

[0029] LLM (Large Language Model): refers to language models with a large number of parameters and large data sets, such as GPT-3. Such models can understand and generate natural language and perform various language-related tasks, such as text generation, translation, summarization, etc. LLM uses deep learning technology to capture the complex patterns and nuances of language and provide more accurate and humanized language processing capabilities.

[0030] Prompt: It is the input text used to guide the AI ​​model to understand and generate a specific type of answer. Prompts usually contain questions or instructions, as well as possible contextual information, to help the model understand the task requirements and generate appropriate output. In dialogue systems and text generation tasks, well-designed prompts are crucial to improving model performance and output quality.

[0031] At present, most large language models (LLMs) usually directly generate the final answer when dealing with complex logical problems. Although this method is simple and fast, it lacks a clear display of the problem-solving process, leading to several major problems:

[0032] 1. Poor interpretability: Directly generated answers make it difficult for people to understand how the model reached its conclusions, especially in the case of errors, it is difficult to track down the root cause of the error.

[0033] 2. Lack of robustness: For problems that require multi-step reasoning, direct generation methods are susceptible to initial assumption bias. Once an intermediate step goes wrong, all subsequent steps may be affected.

[0034] 3. Limited flexibility: Direct generation strategies may not adapt well when encountering new or unseen problem types because they do not explicitly learn problem-solving strategies.

[0035] In order to solve the above problems, relevant solutions are provided in the embodiments of the present application, which are described in detail below.

[0036] According to an embodiment of the present application, a method embodiment of a question-and-answer method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing the question-answering method. Figure 1As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0038] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0039] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the question-and-answer method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned question-and-answer method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0040] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0041] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0042] Optionally, Figure 2 The architecture diagram of each module is shown. Figure 3 The interaction process between each module is shown, such as Figure 2 and Figure 3 As shown, the overall architecture includes the following main modules:

[0043] 1. Question receiving and parsing module: responsible for receiving questions submitted by users, performing semantic analysis on them, and extracting key information of the questions.

[0044] 2. CoT generation module: Based on the key information of the problem, generate the chain of thought (CoT) required to solve the problem, that is, decompose the complex problem into a series of sub-steps.

[0045] 3. Sub-step processing module: responsible for inputting each sub-step in CoT together with the original question into the large language model and performing the reasoning process of each sub-step.

[0046] 4. Result integration and output module: Integrate the intermediate answers of all sub-steps into the final answer and return it to the user, while providing a detailed CoT description.

[0047] Optionally, the question receiving and parsing module is responsible for receiving questions submitted by users, performing semantic parsing on them, and extracting key information of the questions. Specific functions include:

[0048] 1. Question receiving interface: Users submit questions through the API or user interface.

[0049] 2. Semantic parser: performs preliminary semantic analysis on the question and extracts keywords and key phrases for subsequent CoT generation.

[0050] 3. Problem structuring: Convert the problem into a structured form to facilitate processing in subsequent modules.

[0051] Optionally, the CoT generation module generates the chain of thought (CoT) required to solve the problem based on the key information of the problem, that is, decomposing the complex problem into a series of sub-steps. Specific functions include:

[0052] 1. CoT generation algorithm: Generate structured CoT information based on the question.

[0053] 2. CoT Verifier: Use LLM to verify the generated CoT again through the prompt technique to ensure that the generated CoT is logically coherent and covers the key aspects of the problem.

[0054] 3. CoT Optimizer: Use LLM to optimize CoT based on the verification results to ensure it is concise and effective.

[0055] Optionally, the sub-step processing module is responsible for inputting each sub-step in the CoT together with the original question into the large language model and performing the reasoning process of each sub-step. Specific functions include:

[0056] 1. Sub-step executor: Input each sub-step together with the original question to the large language model and perform the inference process.

[0057] 2. Intermediate answer generator: Generates the intermediate answer for each sub-step.

[0058] 3. Intermediate answer verifier: Verify the intermediate answer of each sub-step to ensure its correctness.

[0059] 4. Self-correction mechanism: When an error is found, it traces back to where the error occurred and tries different solutions.

[0060] The result integration and output module integrates the intermediate answers of all sub-steps into the final answer and returns it to the user, while providing a detailed CoT description. Specific functions include:

[0061] 1. Answer Integrator: Integrate the intermediate answers of all sub-steps into a complete final answer.

[0062] 2. Answer consistency checker: Check the consistency and logic of the final answer.

[0063] 3. Result outputter: Returns the final answer to the user in a clear format with a detailed CoT description.

[0064] The above system architecture ensures that the whole process from question receiving to final answer output is efficient, accurate and transparent. The modules work together to ensure the systematization and modularization of the processing process, and improve the stability and reliability of the overall system.

[0065] Optionally, Figure 4 The interaction process between each module is shown, such as Figure 4 As shown:

[0066] First, the question receiving and parsing module receives the user's question input, performs semantic parsing on the question, and outputs the structured information to the CoT generation module;

[0067] The CoT generation module is responsible for content generation and verification of CoT content. If the verification is unreasonable, it will be optimized through optimization prompt. For example, if the CoT content is unreasonable, the optimization prompt can be "correct the unreasonable part of CoT". The optimized CoT content is verified again. If it is reasonable, it will be transmitted to the sub-step processing module through CoT optimization. CoT optimization includes: if the sub-step verification result is that the sub-step is redundant, the sub-step will be deleted; if the sub-step verification result is that there are similar sub-steps in the sub-step, the sub-step and the corresponding similar sub-step will be merged. After the above steps, CoT breaks down the problem into sub-steps.

[0068] The sub-step processing module executes each of the disassembled sub-steps according to the CoT, verifies the intermediate answers, performs self-correction in the case of errors, and outputs the intermediate answers to the result integration and output module in the case of correctness;

[0069] The result integration and output module integrates the answers and performs consistency checks. If they are consistent, the final results are output. If they are inconsistent, the contradictions are analyzed and the intermediate results are corrected.

[0070] In the above operating environment, the embodiment of the present application provides a question-answering method, such as Figure 5 As shown, the method comprises the following steps:

[0071] Step S502: receiving a question input by the target object and determining semantic information in the question.

[0072] In the technical solution provided in step S502, determining the semantic information in the question includes: extracting key information in the question, wherein the key information includes keywords and key phrases; determining the key information identifier corresponding to the key information; and converting the key information and the key information identifier into preset structured data as the semantic information of the question.

[0073] Optionally, receiving the question input by the target object includes receiving a question submitted by the user (i.e., the target object) through an API or a user interface, and the question may be any form of natural language expression. The receiving end receives the question text input by the user and stores it in a data structure within the system for subsequent processing. For example, a user may enter a question through a web form: "Calculate the average annual growth rate of Company A over the past five years." After receiving the question, the system saves it in a memory or a database for the next step of processing.

[0074] Optionally, extracting key information from the question includes performing a preliminary semantic analysis of the question text using natural language processing technology. Extract keywords and key phrases in the question, such as "Company A", "past five years", and "average annual growth rate". Analyze the intent and structure of the question and identify the core elements of the question. For example, the system will perform part-of-speech tagging, syntactic analysis, and semantic role tagging (i.e., key information identification corresponding to key information) on the question to understand the key components of the question. Through these analyses, the system can better understand the specific requirements of the question and prepare for subsequent CoT generation.

[0075] Optionally, convert the problem into a structured form to facilitate processing by subsequent modules. During the structuring process, ensure that the key information of the problem is fully retained.

[0076] For example, convert the question into a structured object containing the necessary information, such as JSON format:

[0077] {

[0078] "company":"Company A",

[0079] "time_period":"Last five years",

[0080] "metric":"Average annual growth rate"

[0081] }

[0082] Among them, "company" and others are key information identifiers, and "Company A" and others are keywords or key phrases.

[0083] Once structured, the system can more easily identify and address the various parts of the problem.

[0084] Step S504: input the semantic information into the first model, and obtain a sub-step set output by the first model, wherein the sub-step set includes each sub-step required to answer the question.

[0085] Optionally, the first model may be any general large language model (LLM with a general model parameter of more than 10B) with CoT generation capability, and the first model, the second model and the third model may be the same general large language model, or different models may be used.

[0086] Optionally, inputting semantic information into the first model, and obtaining a set of sub-steps output by the first model includes using a large language model with CoT generation capability (generally a LLM with model parameters above 10B), generating structured CoT steps through prompt fine-tuning, that is, breaking down complex problems into multiple simple sub-problems (a set of sub-steps).

[0087] The CoT generation algorithm needs to consider the complexity of the problem and the logical level that needs to be solved.

[0088] For example, for the above problem, the system can generate the following CoT (a set of sub-steps):

[0089] 1. Find the financial data of Company A for the past five years;

[0090] 2. Calculate the annual growth rate;

[0091] 3. Calculate the average growth rate.

[0092] The CoT generation algorithm ensures that each sub-step is logically coherent and easy to process.

[0093] In the technical solution provided in step S504, after obtaining the set of sub-steps output by the first model, the method also includes: inputting the sub-steps and step verification prompt words into the first model for step verification, wherein the step verification prompt words are used to indicate the first model verification sub-steps, and the step verification includes at least one of the following: verifying the logic of the sub-steps, verifying the redundant status of the sub-steps, and verifying similar sub-steps of the sub-steps; obtaining the sub-step verification results output by the first model; and optimizing the sub-steps based on the sub-step verification results.

[0094] Optionally, the sub-steps are input into the first model, and the generated CoT is verified again using the LLM (first model) through the prompt technique (step verification prompt) to ensure that the generated CoT is logically coherent and covers the key aspects of the problem. During the verification process, check whether the logical relationship of each sub-step is correct. For example, the system uses a prompt to ask the LLM: "Please verify whether the above CoT is reasonable?" The LLM will verify the rationality of the CoT based on the structured representation of the problem and feedback the verification results.

[0095] In the technical solution provided in step S504, optimizing the sub-steps based on the sub-step verification results includes: when the sub-step verification result is that the logic of the sub-step is incorrect, correcting the sub-step according to the first model, and checking the corrected sub-step according to the first model to obtain the sub-step verification result of the corrected sub-step; when the sub-step verification result is that the sub-step is redundant, deleting the sub-step; when the sub-step verification result is that there are similar sub-steps in the sub-step, merging the sub-step and the corresponding similar sub-step.

[0096] Optionally, use LLM (first model) to optimize CoT based on the verified results to ensure that it is concise and effective:

[0097] During the optimization process, if the logic of a sub-step is incorrect, the sub-step is corrected according to the first model; unnecessary sub-steps are deleted and similar sub-steps are merged to improve efficiency.

[0098] For example, if there are redundant steps in the CoT, the system will input the corresponding sub-steps into the LLM and prompt the LLM to optimize: "Please optimize the above CoT to make it more concise."

[0099] LLM optimizes CoT based on the verification results and returns the optimized version.

[0100] Step S506: input the question and the sub-step set into the second model, and obtain the sub-step answers output by the second model that correspond one-to-one to each sub-step in the sub-step set.

[0101] Optionally, each sub-step in the CoT (set of sub-steps) is input to the large language model (second model) together with the original question. The reasoning process of each sub-step is performed to generate an intermediate answer.

[0102] For example, for the first step "Find the financial data of Company A for the past five years", the system will input the question and sub-steps together into the LLM:

[0103] Question: Calculate the average annual growth rate of Company A over the past five years.

[0104] Sub-step: Find the financial data of Company A for the past five years.

[0105] LLM generates corresponding intermediate answers based on the sub-steps.

[0106] Optionally, obtaining sub-step answers output by the second model that correspond one-to-one to each sub-step in the sub-step set includes: using the large language model to generate an intermediate answer for each sub-step, ensuring that the answer to each sub-step is independent and correct.

[0107] For example, for the first step, the LLM might return:

[0108] Company A's financial data for the past five years are: ...

[0109] The intermediate answers for each sub-step need to be independent and reasonable.

[0110] Step S508, verifying the sub-step answers of all sub-steps in the sub-step set, and when the verification result of the sub-step answers is passed, determining the final answer corresponding to the question based on the sub-step answers of all sub-steps, and outputting the final answer.

[0111] In the technical solution provided in step S508, after verifying the sub-step answer, the method also includes: if the verification result of the sub-step answer is a verification failure, re-obtaining the sub-step answer according to the second model; and re-verifying the re-obtained sub-step answer according to the second model.

[0112] Optionally, LLM (second model) is used in combination with prompt technique to verify the answers of the sub-steps to ensure their correctness. During the verification process, special attention is paid to problems with simple logic, because LLM performs better on such problems.

[0113] For example, the system can use a prompt to ask the LLM:

[0114] Please verify that the above answers are correct.

[0115] LLM verifies the answer based on the context of the substeps and feeds back the verification result.

[0116] Optionally, when the verification result of the answer to the sub-step is a verification failure, re-obtaining the answer to the sub-step according to the second model includes: when an error is found, verifying the output result using the second model. Tracing back to the location where the error occurred and trying different solutions. After correcting the error, re-executing the affected sub-steps and verifying again.

[0117] For example, if LLM finds a problem with the answer to a step, the system will prompt:

[0118] The above answer is incorrect, please regenerate.

[0119] The second model tries a different solution and re-executes the affected substeps.

[0120] In the technical solution provided in step S508, the final answer corresponding to the question is determined based on the sub-step answers of all sub-steps, including: inputting the answers of each sub-step into the third model to obtain the answer summary output by the third model; and determining the answer summary and each sub-step and the answers of each sub-step as the final answer.

[0121] Optionally, determining the final answer corresponding to the question based on the sub-step answers of all sub-steps includes inputting the answers of each sub-step into the third model, and inputting a prompt word to require the third model to integrate the intermediate answers of all sub-steps into a complete final answer. During the integration process, the logical consistency of the answer is ensured.

[0122] For example, the prompt is: "Integrate the answers to all sub-steps into:".

[0123] Model output: Company A's average annual growth rate over the past five years is X%.

[0124] Make sure your final answer is logically coherent and complete.

[0125] In the technical solution provided in step S508, before the answer summary and each sub-step and the answer to each sub-step are determined as the final answer, the method also includes: checking whether there is a logical contradiction between the answers to each sub-step based on the third model; when it is checked that there is a contradiction between the answers to the sub-steps, correcting the answers to the sub-steps based on the third model; and checking whether there is a logical contradiction between the corrected answers to the sub-steps based on the third model.

[0126] Optionally, use LLM combined with prompt techniques to perform logical judgment on the intermediate answers generated by each sub-step. For answers with logical contradictions, analyze the contradictions and feed them back to the self-correction mechanism to ensure the consistency and logic of the final answer.

[0127] For example, the system can use a prompt to ask the LLM:

[0128] Please verify that the above answers are consistent.

[0129] LLM will check the consistency of answers and provide feedback on the results.

[0130] Optionally, if there is no logical contradiction between the answers to the various sub-steps, the final answer is returned to the user in a clear format. A detailed CoT description is also provided, including each sub-step and its corresponding reasoning process, to ensure that the user can understand how the model arrives at the final answer.

[0131] For example, the system returns the final answer and detailed CoT to the user:

[0132] Final Answer: Company A's average annual growth rate over the past five years has been X%.

[0133] CoT description: 1. Find the financial data of Company A for the past five years; 2. Calculate the growth rate for each year; 3. Calculate the average growth rate.

[0134] The present application embodiment also provides a question-answering method embodiment, in which a user (target object) submits a complex question about a company's financial statements through a user interface: "Calculate the average annual growth rate of Company A over the past five years." The specific implementation steps include:

[0135] 1. Question receiving and analysis module:

[0136] The user submits a question through the user interface: "Calculate the average annual growth rate of Company A over the past five years.".

[0137] The system receives the question and performs semantic parsing, extracting keywords and key phrases such as "Company A", "past five years", and "average annual growth rate".

[0138] Structure the question into an object containing the necessary information, such as:

[0139] {

[0140] "company":"Company A",

[0141] "time_period":"Last five years",

[0142] "metric":"Average annual growth rate"

[0143] }

[0144] 2.CoT generation module:

[0145] Steps to generate structured CoT using a large language model with CoT generation capability:

[0146] 1. Find the financial data of Company A for the past five years;

[0147] 2. Calculate the annual growth rate;

[0148] 3. Calculate the average growth rate.

[0149] Use LLM to verify the generated CoT using the prompt technique to ensure its logical coherence.

[0150] Use LLM to optimize CoT and ensure it is concise and effective.

[0151] 3. Sub-step processing module:

[0152] Each sub-step in CoT is fed into the large language model along with the original question.

[0153] Process each sub-step:

[0154] Step 1: Find the financial data of Company A for the past five years.

[0155] Middle answer: Company A’s financial data for the past five years are: revenue of 1 million in 2019, revenue of 1.2 million in 2020, revenue of 1.4 million in 2021, revenue of 1.6 million in 2022, and revenue of 1.8 million in 2023.

[0156] Verify the intermediate answers and confirm that they are correct.

[0157] Step 2: Calculate the annual growth rate.

[0158] Middle answer: The growth rate in 2020 is 20%, the growth rate in 2021 is 16.67%, the growth rate in 2022 is 14.29%, and the growth rate in 2023 is 12.5%.

[0159] Verify the intermediate answers and confirm that they are correct.

[0160] Step 3: Calculate the average growth rate.

[0161] Middle answer: The average annual growth rate over the past five years is 16%.

[0162] Verify the intermediate answers and confirm that they are correct.

[0163] 4. Result integration and output module:

[0164] Integrate the intermediate answers from all sub-steps into the final answer: "Company A's average annual growth rate over the past five years was 16%.".

[0165] Use LLM to check the consistency and logic of your final answer.

[0166] The final answer is returned to the user in a clear format with a detailed CoT description.

[0167] result:

[0168] The user received a detailed answer, including each step of the CoT and its corresponding reasoning process:

[0169] Final Answer: Company A's average annual growth rate over the past five years was 16%.

[0170] CoT Description:

[0171] 1. Find the financial data of Company A for the past five years;

[0172] 2. Calculate the annual growth rate;

[0173] 3. Calculate the average growth rate.

[0174] The present application embodiment also provides a question-answering method embodiment, in which a user (target object) submits a complex question about a contract clause through an API: "Interpret the breach of contract clause in this contract." The specific implementation steps include:

[0175] 1. Question receiving and analysis module:

[0176] A user submits a question via the API: "Interpret the breach of contract clause in this contract.".

[0177] The system receives the question and performs semantic analysis to extract keywords and key phrases, such as "contract" and "breach of contract clause".

[0178] Structure the question into an object containing the necessary information, such as:

[0179] {

[0180] "document_type":"Contract",

[0181] "clause_type":"Breach of Contract Clause"

[0182] }

[0183] 2.CoT generation module:

[0184] Steps to generate structured CoT using a large language model with CoT generation capability:

[0185] 1. Identify the breach of contract clauses in the contract;

[0186] 2. Explain the specific content of the breach of contract clause;

[0187] 3. Summarize the key points of the breach of contract clause.

[0188] Use LLM to verify the generated CoT using the prompt technique to ensure its logical coherence.

[0189] Use LLM to optimize CoT and ensure it is concise and effective.

[0190] 3. Sub-step processing module:

[0191] Each sub-step in CoT is fed into the large language model along with the original question.

[0192] Process each sub-step:

[0193] Step 1: Identify the breach of contract clause in the contract.

[0194] Intermediate answer: Article 5 of the contract is a breach of contract clause.

[0195] Verify the intermediate answers and confirm that they are correct.

[0196] Step 2: Explain the specific content of the breach of contract clause.

[0197] Middle answer: Clause 5 provides that if one party fails to perform its contractual obligations, the other party has the right to terminate the contract and claim damages.

[0198] Verify the intermediate answers and confirm that they are correct.

[0199] Step 3: Summarize the key points of the breach of contract clause.

[0200] Middle answer: The key point of a breach of contract clause is that failure to perform an obligation will result in termination of the contract and liability for damages.

[0201] Verify the intermediate answers and confirm that they are correct.

[0202] 4. Result integration and output module:

[0203] Integrate the intermediate answers to all sub-steps into the final answer: "A breach of contract clause in a contract provides that if one party fails to perform its contractual obligations, the other party has the right to terminate the contract and claim damages.".

[0204] Use LLM to check the consistency and logic of your final answer.

[0205] The final answer is returned to the user in a clear format with a detailed CoT description.

[0206] result:

[0207] The user received a detailed answer, including each step of the CoT and its corresponding reasoning process:

[0208] Final Answer: A breach of contract clause in a contract provides that if one party fails to perform its obligations under the contract, the other party has the right to terminate the contract and claim damages.

[0209] CoT Description:

[0210] 1. Identify the breach of contract clauses in the contract;

[0211] 2. Explain the specific content of the breach of contract clause;

[0212] 3. Summarize the key points of the breach of contract clause.

[0213] Through the above steps, the ability of the large language model in dealing with complex logical problems can be improved, and the model can build a clear chain of thought (CoT), that is, a series of intermediate steps to solve the problem before generating an answer. Such a method can not only enhance the accuracy of the model in solving problems, but also improve the transparency of its decision-making process, which is convenient for subsequent manual review or correction. The present invention significantly improves the performance of the large language model in solving complex logical problems by integrating CoT (Chain of Thought) technology and self-correction mechanism. Compared with the prior art, the present invention has significant advantages and effects. Specifically, the present invention improves the accuracy of the model generation results, solves the problems of poor interpretability, insufficient robustness and limited flexibility of traditional methods in dealing with complex problems, enhances the model's ability to handle complex logical problems, optimizes user experience, and provides new directions and methods for the development and application of intelligent dialogue systems.

[0214] First of all, the method embodiment of the present application significantly improves the accuracy of the results generated by the large language model by introducing the CoT technology. The traditional processing method often directly generates the final answer, while ignoring the importance of the intermediate reasoning process. The CoT technology breaks down complex problems into a series of sub-steps and solves them one by one to ensure the correctness of each step of reasoning. This method reduces the deviation of the final result caused by errors in the intermediate steps, thereby improving the accuracy of the overall generated results.

[0215] Secondly, the method embodiment of the present application solves the problem of poor interpretability of the prior art when dealing with complex problems. CoT technology not only improves the transparency of the reasoning process, but also allows users to clearly see how the model draws conclusions step by step. This is especially important for application scenarios that require high transparency and auditability, such as legal consulting, medical diagnosis and other fields. Through the CoT mechanism, users can more easily understand and trust the output results of the model.

[0216] Third, the method embodiments of the present application enhance the model's ability to handle complex logical problems. CoT technology allows the model to dynamically build solutions when faced with complex problems, rather than relying on fixed problem-solving patterns. This approach increases the model's adaptability to unknown problem types, allowing it to respond flexibly when faced with new problems. In addition, the self-correction mechanism further enhances the robustness of the model. When the model finds an error in a certain step, it can backtrack and try different solutions, thereby ensuring the stability of the entire reasoning chain.

[0217] Fourth, the method embodiment of the present application optimizes the user experience. By providing clear solution steps, users can better understand how the model reaches conclusions, thereby enhancing trust in the model output. In addition, step-by-step answers are also easier to accept and understand for non-professional users. For example, in the field of education, teachers can use this method to explain complex concepts to students, making them more intuitive and easy to understand.

[0218] Finally, the method embodiments of the present application provide new directions and methods for the development and application of intelligent dialogue systems. The CoT mechanism can not only solve the current problem, but also be used as part of the feedback mechanism to evaluate the performance of the model on different tasks and further optimize the model accordingly. By recording and analyzing each CoT process, researchers can gain important insights into the model's behavior and improve the model's design and training methods.

[0219] Specifically, the method embodiment of the present application has the following advantages:

[0220] 1. Enhanced explainability: Using the CoT generation algorithm, a large language model with CoT generation capability (LLM with general model parameters above 10B) is used to generate structured CoT steps through prompt fine-tuning. By breaking down complex problems into multiple sub-steps and clearly showing the reasoning process of each step, the generation of the final answer becomes transparent and easy to understand. In real-world applications, users often need to know how the model makes decisions, especially in situations where auditing or compliance is required. For example, in legal consultation, lawyers and clients need to clearly understand the logical basis behind each reasoning step. The present invention provides this transparency by generating a clear chain of thinking, allowing users to easily track the model's thinking process, thereby enhancing confidence in the model's output. Compared with the prior art, CoT technology significantly improves the model's ability to handle complex logical problems. By breaking down complex problems into multiple sub-steps and solving them one by one, the correctness of each step of reasoning is ensured, thereby improving the accuracy of the final result. In addition, CoT technology makes the reasoning process more transparent and enhances the interpretability of the results.

[0221] 2. Improve robustness: By solving complex problems in steps, the risk of overall failure due to errors in a single step is reduced. Through the verification of the intermediate answers of the sub-steps, LLM combined with the prompt technique is used to verify the answers of the sub-steps to ensure their correctness. Even if a certain step deviates, it can be compensated by subsequent verification and correction. When an error is found, the model can trace back to the location where the error occurred and try different solutions, thereby improving the reliability of the overall solution. When dealing with problems that require multi-step reasoning, the traditional way of directly generating answers is susceptible to errors in the intermediate steps, which in turn leads to inaccurate final results. The CoT method allows the model to check and correct at each step, thereby ensuring the robustness of the entire reasoning chain. For example, in medical diagnosis, incorrect initial assumptions may lead to completely wrong treatment recommendations. The CoT method can help avoid this situation. The LLM combined with the prompt technique is used to logically judge the intermediate answers generated by each sub-step. For answers with logical contradictions, the contradictions are analyzed and feedback is provided for self-correction. Compared with the existing technology, the self-correction mechanism significantly improves the robustness of the model. When an error occurs in a certain step, the model can backtrack and correct it in time, avoiding the final result deviation caused by the accumulation of errors. This method improves the reliability of the overall solution. Using LLM combined with the prompt technique to logically judge the intermediate answers generated by each sub-step to ensure the consistency and logic of the final answer, the result consistency check significantly improves the logical consistency and accuracy of the final answer. By verifying the intermediate answers of each sub-step, the coherence of the entire reasoning chain is ensured, avoiding logical contradictions and errors.

[0222] 3. Increase flexibility: The CoT method allows the model to dynamically build solution strategies when facing new problems, rather than relying on fixed pattern matching. This can enhance the model's adaptability to unknown problem types to a certain extent. In a constantly changing application environment, the model needs to have a certain generalization ability to deal with problems that have never been seen. Through the CoT mechanism, the model can flexibly build solutions based on the characteristics of the problem, rather than rigidly following pre-set rules. For example, in scientific research, problems are often changeable, and the model needs to be able to flexibly respond to different research needs. In addition, the method embodiment of the present application deletes unnecessary sub-steps and merges similar sub-steps through the optimization algorithm, thereby improving efficiency. Compared with the prior art, the CoT verification and optimization mechanism significantly improves the logical coherence and simplicity of CoT. Through verification and optimization, the effectiveness and simplicity of CoT are ensured, and the efficiency of the model in handling problems is improved.

[0223] 4. Improve user experience: By providing clear solution steps, users can better understand how the model reaches its conclusions, thereby enhancing their trust in the model output. In addition, step-by-step answers are also easier for non-professional users to accept and understand. In daily life, users with non-technical backgrounds also need the help of AI tools to solve problems. By simplifying complex problems into small, easy-to-understand steps, the present invention improves user acceptance and usage experience. For example, in the field of education, teachers can use this method to explain complex concepts to students, making them more intuitive and easy to understand.

[0224] 5. Improve the accuracy of the results generated by large language models: Through the CoT mechanism, the model can check and verify the correctness of each step of reasoning step by step before generating the final answer. This can significantly reduce the deviation of the final result caused by errors in the intermediate steps, thereby improving the accuracy of the overall generated results. In many critical applications, the accuracy of the results is crucial. The CoT mechanism ensures the correctness of each step of reasoning through step-by-step verification, and ultimately improves the overall quality of the answers generated by the model. For example, in financial analysis, small errors can lead to huge financial losses, and the CoT method can effectively prevent such situations from happening.

[0225] 6. Promote model optimization: The CoT mechanism can not only be used to solve current problems, but also as part of a feedback mechanism to evaluate the performance of the model on different tasks and further optimize the model accordingly. By recording and analyzing each CoT process, researchers can gain important insights into the model's behavior and improve the model's design and training methods. For example, in the development of self-driving cars, CoT can help engineers identify and fix potential defects in the model and improve the safety of the system.

[0226] The embodiment of the present application provides a question-answering device, Figure 6 is a schematic diagram of the structure of the device, such as Figure 6 As shown, the device includes: a determination module 60, which is used to receive a question input by a target object and determine the semantic information in the question; a first acquisition module 62, which is used to input the semantic information into the first model and obtain a sub-step set output by the first model, wherein the sub-step set includes each sub-step required to answer the question; a second acquisition module 64, which is used to input the question and the sub-step set into the second model and obtain sub-step answers output by the second model that correspond to each sub-step in the sub-step set; an output module 66, which is used to verify the sub-step answers of all sub-steps in the sub-step set, and when the verification result of the sub-step answer is verification passed, determine the final answer corresponding to the question based on the sub-step answers of all sub-steps, and output the final answer.

[0227] Optionally, the determination module 60 determines the semantic information in the question including: extracting key information in the question, wherein the key information includes keywords and key phrases; determining the key information identifier corresponding to the key information; and converting the key information and the key information identifier into preset structured data as the semantic information of the question.

[0228] Optionally, after the second acquisition module 64 acquires the set of sub-steps output by the first model, it also includes: inputting the sub-steps and step verification prompt words into the first model for step verification, wherein the step verification prompt words are used to indicate the first model verification sub-steps, and the step verification includes at least one of the following: verifying the logic of the sub-steps, verifying the redundant status of the sub-steps, and verifying similar sub-steps of the sub-steps; acquiring the sub-step verification results output by the first model; and optimizing the sub-steps based on the sub-step verification results.

[0229] Optionally, the second acquisition module 64 optimizes the sub-step according to the sub-step verification result, including: when the sub-step verification result is that the logic of the sub-step is incorrect, correcting the sub-step according to the first model, and checking the corrected sub-step according to the first model, and obtaining the sub-step verification result of the corrected sub-step; when the sub-step verification result is that the sub-step is redundant, deleting the sub-step; when the sub-step verification result is that there are similar sub-steps in the sub-step, merging the sub-step and the corresponding similar sub-step.

[0230] Optionally, after the output module 66 verifies the sub-step answer, it also includes: if the verification result of the sub-step answer is a verification failure, re-obtaining the sub-step answer according to the second model; and re-verifying the re-obtained sub-step answer according to the second model.

[0231] Optionally, the output module 66 determines the final answer corresponding to the question based on the sub-step answers of all sub-steps, including: inputting the answers of each sub-step into the third model to obtain the answer summary output by the third model; and determining the answer summary and each sub-step and the answers of each sub-step as the final answer.

[0232] Optionally, before the output module 66 summarizes the answer and determines each sub-step and the answer to each sub-step as the final answer, it also includes: checking whether there is a logical contradiction between the answers to each sub-step based on the third model; when it is found that there is a contradiction between the answers to the sub-steps, correcting the answers to the sub-steps based on the third model; and checking whether there is a logical contradiction between the corrected answers to the sub-steps based on the third model.

[0233] It should be noted that the various modules in the above-mentioned question-and-answer device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0234] An embodiment of the present application provides a non-volatile storage medium, in which a program is stored, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the following question-answering method: receiving a question input by a target object, and determining semantic information in the question; inputting the semantic information into a first model, and obtaining a sub-step set output by the first model, wherein the sub-step set includes each sub-step required to answer the question; inputting the question and the sub-step set into a second model, and obtaining sub-step answers output by the second model that correspond to each sub-step in the sub-step set; verifying the sub-step answers of all sub-steps in the sub-step set, and when the verification result of the sub-step answer is verification passed, determining a final answer corresponding to the question based on the sub-step answers of all sub-steps, and outputting the final answer.

[0235] An embodiment of the present application provides an electronic device, including: a memory and a processor, the processor being used to run a program stored in the memory, wherein the program executes the following question-answering method when it is run: receiving a question input by a target object, and determining semantic information in the question; inputting the semantic information into a first model, and obtaining a sub-step set output by the first model, wherein the sub-step set includes each sub-step required to answer the question; inputting the question and the sub-step set into a second model, and obtaining sub-step answers output by the second model that correspond to each sub-step in the sub-step set; verifying the sub-step answers of all sub-steps in the sub-step set, and when the verification result of the sub-step answer is verification passed, determining a final answer corresponding to the question based on the sub-step answers of all sub-steps, and outputting the final answer.

[0236] An embodiment of the present application provides a computer program product, including a computer program, which implements the following question-answering method when executed by a processor: receiving a question input by a target object, and determining semantic information in the question; inputting the semantic information into a first model, and obtaining a sub-step set output by the first model, wherein the sub-step set includes each sub-step required to answer the question; inputting the question and the sub-step set into a second model, and obtaining sub-step answers output by the second model that correspond one-to-one to each sub-step in the sub-step set; verifying the sub-step answers of all sub-steps in the sub-step set, and when the verification result of the sub-step answer is verification passed, determining a final answer corresponding to the question based on the sub-step answers of all sub-steps, and outputting the final answer.

[0237] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0238] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0239] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0240] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0241] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0242] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A question-answering method, characterized in that: include: Receiving a question input by a target object, and determining semantic information in the question; Inputting the semantic information into a first model to obtain a set of sub-steps output by the first model, wherein the set of sub-steps includes each sub-step required to answer the question; Inputting the question and the sub-step set into a second model, and obtaining sub-step answers output by the second model corresponding to each of the sub-steps in the sub-step set; The sub-step answers of all sub-steps in the sub-step set are verified. When the verification result of the sub-step answers is passed, a final answer corresponding to the question is determined based on the sub-step answers of all sub-steps, and the final answer is output.

2. The question-answering method according to claim 1, characterized in that: Determining the semantic information in the question includes: Extracting key information from the question, wherein the key information includes keywords and key phrases; Determining a key information identifier corresponding to the key information; The key information and key information identifier are converted into preset structured data as semantic information of the question.

3. The question-answering method according to claim 1, characterized in that: After obtaining the set of sub-steps output by the first model, the method further includes: Inputting the sub-step and the step verification prompt into the first model for step verification, wherein the step verification prompt is used to instruct the first model to verify the sub-step, and the step verification includes at least one of the following: verifying the logic of the sub-step, verifying the redundant state of the sub-step, and verifying similar sub-steps of the sub-step; Obtaining a sub-step verification result output by the first model; The sub-steps are optimized according to the verification results of the sub-steps.

4. The question-answering method according to claim 3, characterized in that: Optimizing the sub-steps according to the verification results of the sub-steps includes: When the sub-step verification result shows that the logic of the sub-step is incorrect, the sub-step is corrected according to the first model, and the corrected sub-step is verified according to the first model to obtain a sub-step verification result of the corrected sub-step; If the verification result of the sub-step is that the sub-step is in a redundant state, deleting the sub-step; When the sub-step verification result is that the sub-step has a similar sub-step, the sub-step and the corresponding similar sub-step are merged.

5. The question-answering method according to claim 1, characterized in that: After verifying the answers to the sub-steps, the method further includes: When the verification result of the answer to the sub-step is a verification failure, re-obtaining the answer to the sub-step according to the second model; The re-acquired sub-step answers are verified again according to the second model.

6. The question-answering method according to claim 1, characterized in that: Determining a final answer corresponding to the question based on the sub-step answers of all the sub-steps includes: Input the answers of each sub-step into the third model, and obtain the answer summary output by the third model; The answer summary and each of the sub-steps and each of the sub-step answers are determined as the final answer.

7. The question-answering method according to claim 6, characterized in that: Before determining the answer summary and each of the sub-steps and each of the sub-step answers as the final answer, the method further comprises: Checking whether there is a logical contradiction between the answers to the sub-steps according to the third model; In the event that a contradiction is detected between the answers to the sub-steps, correcting the answers to the sub-steps according to the third model; Check whether there is any logical contradiction between the corrected answers to the sub-steps based on the third model.

8. A question-answering device, characterized in that: include: A determination module, used to receive a question input by a target object and determine semantic information in the question; A first acquisition module, used for inputting the semantic information into a first model, and acquiring a sub-step set output by the first model, wherein the sub-step set includes each sub-step required to answer the question; A second acquisition module, used for inputting the question and the sub-step set into a second model, and acquiring sub-step answers output by the second model corresponding to each of the sub-steps in the sub-step set; The output module is used to verify the sub-step answers of all sub-steps in the sub-step set. When the verification result of the sub-step answers is passed, the final answer corresponding to the question is determined based on the sub-step answers of all sub-steps, and the final answer is output.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the question-answering method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the question-answering method according to any one of claims 1 to 7 is executed when the program is run.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the question-answering method according to any one of claims 1 to 7 are implemented.

Images