Processing method and device for optimizing model reasoning evidence

Abstract

The embodiment of the invention relates to a processing method and device for optimizing model reasoning evidence. The method comprises the steps that a plurality of prompt instruction templates are set for a target large language model; and when a question and answer task is processed, the template guide model is utilized to evaluate and optimize the original evidence, generate and optimize the deduced evidence, and predict and evaluate the inference chain and test the evidence completeness. The method can improve the reasoning evidence quality.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and apparatus for processing evidence for optimizing model reasoning. Background Technology

[0002] With the widespread application of Large Language Models (LLMs) in reasoning and question-answering scenarios, the quality of the processing of model inference evidence directly determines the accuracy and reliability of the answer. Research and analysis reveal several shortcomings in current technologies for optimizing model inference evidence: 1) Inconsistent quality of raw evidence: Current technologies often lack a systematic evaluation mechanism for the reliability, validity, and relevance of raw evidence, leading to a large influx of low-quality, irrelevant, or even erroneous evidence into the evidence set, thus reducing the credibility of the model output; 2) Lack of monitoring of derived evidence: Current technologies do not evaluate the support of the derived evidence's basis and its relevance to the question when performing inferences based on raw evidence, easily introducing noisy evidence and further reducing the credibility of the model output; 3) Lack of verification of the completeness of the inference chain's evidence: After generating the inference chain, current technologies do not verify whether each inference step can obtain effective support from the evidence set, easily leading to missing evidence in the inference chain and further reducing the credibility of the model output. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by providing a method, apparatus, electronic device, and computer-readable storage medium for processing evidence to optimize model inference. This invention provides several prompt instruction templates for optimizing model inference evidence for a target large language model (i.e., the first model): query generation template, original evidence evaluation optimization template, evidence derivation template, derivation evidence evaluation optimization template, original-derivation evidence integration template, inference chain prediction template, inference chain evaluation template, evidence completeness verification template, and target question answering template. Upon receiving the target question (i.e., the first question), the first model, driven by a query generation template, firstly generates a corresponding query instruction set for the first question. Based on this query instruction set, it performs information retrieval to obtain the original evidence set. Then, the first model, driven by an evaluation and optimization template, assesses the reliability, validity, and relevance of the original evidence set to the first question, and optimizes the original evidence set based on the evaluation results. Next, the first model, driven by an evidence derivation template, derives evidence from the original evidence set with the goal of answering the first question, obtaining a derived evidence set. Based on the evaluation and optimization template, the first model, driven by an evidence derivation template, assesses the support of the derivation basis and its relevance to the first question, and optimizes the derived evidence set based on the evaluation results. Finally, the first model, driven by an integration template, integrates the original and derived evidence sets. Evidence integration yields a first evidence set. Then, based on the inference chain prediction template, a first model predicts the reasoning path to answer the first question using the first evidence set, generating a first inference chain. A reasoning chain evaluation template then evaluates the logical consistency and step completeness of the first inference chain. If the evaluation fails, the first model continues to predict and evaluate the inference chain using both the inference chain prediction and evaluation templates until the evaluation passes. Next, an evidence completeness check template checks whether each reasoning step of the first inference chain can obtain valid evidence from the first evidence set. After passing the completeness check, a target question answer template guides the first model to perform multi-step deduction and answer to the first question using the first evidence set as reasoning evidence and the first inference chain as the reasoning path, outputting the corresponding first answer to the current user. This invention improves the quality of reasoning evidence and enhances the quality of model generation by setting multiple prompt instruction templates to guide the large language model in evaluating and optimizing the original evidence, optimizing the generation of deduced evidence, and predicting, evaluating, and checking the completeness of the inference chain.

[0004] To achieve the above objectives, a first aspect of the present invention provides a method for processing evidence to optimize model reasoning, the method comprising: The target large language model is used as the first model; and multiple prompt instruction templates are set for the first model; the multiple prompt instruction templates include query generation template, original evidence evaluation and optimization template, evidence derivation template, derivation evidence evaluation and optimization template, original-derivation evidence integration template, inference chain prediction template, inference chain evaluation template, evidence completeness test template, and target question answering template; The target question, received from user input, is the first question. Based on the query generation template, the first model is driven to generate a corresponding query instruction set for the first question; and information is queried according to the query instruction set to obtain the original evidence set; and based on the original evidence evaluation and optimization template, the first model is driven to evaluate the evidence reliability, evidence validity, and evidence relevance to the first question of the original evidence set, and optimize the original evidence set according to the evaluation results; Based on the evidence deduction template, the first model is driven to deduce evidence from the original evidence set with the goal of answering the first question, thereby obtaining a deduced evidence set; and based on the deduced evidence evaluation and optimization template, the first model is driven to evaluate the support of the deduction basis of the deduced evidence set and its evidence relevance to the first question, and optimize the deduced evidence set according to the evaluation results; Based on the original-derived evidence integration template, the first model is driven to integrate the original evidence set and the derived evidence set to obtain the first evidence set; The first model is driven by the inference chain prediction template to predict the reasoning path to answer the first question based on the first evidence set, thereby obtaining a first inference chain; and the first model is driven by the inference chain evaluation template to evaluate the logical consistency and step completeness of the first inference chain; and if the evaluation fails, the first model continues to predict and evaluate the inference chain based on the inference chain prediction template and the inference chain evaluation template until the evaluation passes. Based on the evidence completeness test template, the first model is driven to test whether each reasoning step of the first reasoning chain can obtain valid evidence from the first evidence set. After the completeness test is passed, the first model is driven by the target question answer template to perform multi-step deduction and answer to the first question using the first evidence set as reasoning evidence and the first reasoning chain as reasoning path, and outputs the corresponding first answer. The first answer will be sent back to the current user.

[0005] Preferably, the target large language model possesses text entity recognition capability, text diversity understanding capability, text generation capability, multi-step reasoning capability, thought chain reasoning capability, context understanding and memory capability, prompting engineering processing capability, and domain knowledge application capability; the text diversity understanding capability is the ability to understand the semantics, context, intent, emotion, and knowledge domain of the text in multiple dimensions. The query generation template, the original evidence evaluation and optimization template, the evidence derivation template, the derivation evidence evaluation and optimization template, the original-derived evidence integration template, the reasoning chain prediction template, the reasoning chain evaluation template, the evidence completeness verification template, and the target question answering template are each a formatted natural language text template; The query instruction set includes multiple query instructions; each query instruction includes query content and query object; the query content is query keywords; and the query object is a publicly available query engine, dataset, or knowledge base. The original evidence set includes multiple first original pieces of evidence; the first original pieces of evidence include the original evidence text and the original evidence source; the original evidence source is the retrieval source of the current evidence; The deduced evidence set includes multiple first deduced evidences; the first deduced evidences include deduced evidence texts and a set of deduced basis; the set of deduced basis consists of one or more of the original evidence texts; The first set of evidence includes multiple first evidence texts; The first inference chain is a node chain, consisting of multiple single-step inference nodes connected sequentially; the node attributes of each single-step inference node consist of a corresponding inference evidence set, inference operation type, and inference conclusion set; the inference evidence set is the set of reference evidence for the current single-step inference, consisting of one or more first evidence texts and / or one or more first conclusion texts; the inference operation type is the inference logic used in the current single-step inference, including causal inference, deductive inference, inductive inference, analogical inference, abductive inference, statistical inference, and Bayesian inference; the inference conclusion set is the output result of the current single-step inference, consisting of one or more first conclusion texts.

[0006] Preferably, the configurable parameters of the query generation template include question parameters; The query generation template is used to take the configured question parameters as the corresponding current question; and prompts the first model to perform query range reasoning on the factual evidence required to answer the current question, and generates and outputs the corresponding query instruction set based on the query range reasoning result.

[0007] Preferably, the configurable parameters of the original evidence evaluation optimization template include question parameters and evidence set parameters; the evidence set parameters include multiple evidence parameters; the evidence parameters include evidence text parameters and evidence source parameters. The original evidence evaluation optimization template is used to take the configured problem parameters and evidence set parameters as the corresponding current problem and current evidence set; and to explain the evaluation requirements for the three evaluation dimensions of evidence reliability, evidence validity, and evidence-problem relevance. It prompts the first model to score the reliability of each evidence source parameter in the current evidence set according to the evidence reliability evaluation requirements, and to mark evidence parameters with scores lower than a preset first reliability score threshold as unqualified parameters; to score the content validity of each evidence text parameter in the current evidence set according to the evidence validity evaluation requirements, and to mark evidence parameters with scores lower than a preset first validity score threshold as unqualified parameters; and to score the content relevance of each evidence text parameter in the current evidence set to the current problem according to the evidence-problem relevance evaluation requirements, and to mark evidence parameters with scores lower than a preset first relevance score threshold as unqualified parameters. The template then deletes the unqualified evidence parameters from the current evidence set and outputs the current evidence set after deleting the unqualified parameters as the final optimization result.

[0008] Preferably, the configurable parameters of the evidence derivation template include question parameters and original evidence set parameters; the original evidence set parameters include multiple original evidence text parameters. The evidence derivation template is used to take the configured question parameters and the original evidence set parameters as the corresponding current question and current evidence set; and prompts the first model to deduce the direct related evidence of the current question based on the current evidence set and generate and output the corresponding derivation evidence set based on the derivation result.

[0009] Preferably, the configurable parameters of the derivation evidence evaluation optimization template include question parameters and evidence set parameters; the evidence set parameters include multiple evidence parameters; the evidence parameters include evidence text parameters and derivation basis parameters; the derivation basis parameters consist of one or more basis text parameters. The derivation evidence evaluation optimization template is used to take the configured problem parameters and evidence set parameters as the corresponding current problem and current evidence set; and to explain the evaluation requirements of the two evaluation dimensions of derivation basis support and evidence-problem relevance; and to prompt the first model to score the derivation evidence support of each derivation basis parameter of the current evidence set to the corresponding evidence text parameter according to the derivation basis support evaluation requirements, and to mark the evidence parameters with scores lower than a preset first support threshold as unqualified parameters; and to score the content relevance of each evidence text parameter of the current evidence set to the current problem according to the evidence-problem relevance evaluation requirements, and to mark the evidence parameters with scores lower than a preset second relevance score threshold as unqualified parameters; and to delete the evidence parameters marked as unqualified parameters in the current evidence set, and to output the current evidence set after the unqualified parameters have been deleted as the final optimization result.

[0010] Preferably, the configurable parameters of the original-derived evidence integration template include original evidence set parameters and derived evidence set parameters; the original evidence set parameters include multiple original evidence text parameters; the derived evidence set parameters include multiple derived evidence text parameters. The original-derived evidence integration template is used to take the configured original evidence set parameters and the derived evidence set parameters as the corresponding current original evidence set and current derived evidence set; and provides an explanation of the duplicate text determination rule with text semantic similarity as the determination condition and a preset semantic similarity threshold as the determination threshold; and prompts the first model to take each original evidence text parameter of the current original evidence set and each derived evidence text parameter of the current derived evidence set as a corresponding first evidence text, and form an initial evidence set with the obtained first evidence texts, and cluster the duplicate texts in the initial evidence set according to the duplicate text determination rule to obtain multiple cluster subsets, and calculate the sum of similarity between each first evidence text in each cluster subset and other texts in the current subset, and extract the first evidence text with the largest sum of similarity in each cluster subset to form the final first evidence set and output it.

[0011] Preferably, the configurable parameters of the inference chain prediction template include question parameters and evidence set parameters; the evidence set parameters include multiple evidence text parameters. The inference chain prediction template is used to take the configured question parameters and evidence set parameters as the corresponding current question and current evidence set; and prompts the first model to construct a complete inference path to answer the current question under preset high temperature and low temperature conditions based on the current evidence set, on the premise of ensuring that the logic of the steps before and after the inference chain is consistent and there are no missing steps in the whole chain, thereby generating a structured high temperature inference chain and low temperature inference chain. The similarity score of the current pair of high and low temperature inference chains is calculated, and it is judged whether the similarity score exceeds the preset similarity score threshold. If it does not exceed the similarity score threshold, a new pair of high and low temperature inference chains is generated until the latest score exceeds the similarity score threshold. If it exceeds the similarity score threshold, the current low temperature inference chain is output as the corresponding first inference chain. Among them, temperature condition is a type of hyperparameter used to control the model generation strategy, specifically a positive real number; the value of the high temperature condition is greater than 1, and the value of the low temperature condition is less than 1; the high temperature inference chain generated under the high temperature condition has greater randomness than the low temperature inference chain, and the low temperature inference chain generated under the low temperature condition has greater logical rationality than the high temperature inference chain.

[0012] Preferably, the configurable parameters of the reasoning chain evaluation template include reasoning chain parameters; the reasoning chain parameters are a node chain, composed of multiple reasoning node parameters connected sequentially; the node attributes of each reasoning node parameter consist of a corresponding reasoning evidence set parameter, a reasoning operation type parameter, and a reasoning conclusion set parameter; the reasoning evidence set parameter consists of one or more evidence text parameters and / or conclusion text parameters; the reasoning operation type parameter includes causal reasoning, deductive reasoning, inductive reasoning, analogical reasoning, abductive reasoning, statistical reasoning, and Bayesian reasoning; the reasoning conclusion set parameter consists of one or more conclusion text parameters. The inference chain evaluation template is used to take the configured inference chain parameters as the corresponding current inference chain; and explains the evaluation requirements for two evaluation dimensions: logical consistency of preceding and following nodes and step integrity of the entire chain nodes; it prompts the first model to score the logical consistency of each inference node parameter of the inference chain parameters according to the logical consistency evaluation requirements of preceding and following nodes, and to mark the inference node parameter with a score lower than a preset logical consistency score threshold as an unqualified node; and to score the step integrity of the inference chain parameters according to the step integrity evaluation requirements of the entire chain nodes, and to identify whether the score is lower than a preset step integrity score threshold. If it is, the corresponding first evaluation result is set as failure; otherwise, the corresponding first evaluation result is set as success. The total number of unqualified nodes is counted to obtain the corresponding first total number, and the first total number and the first evaluation result are identified. If the first total number is greater than zero or the first evaluation result is failure, the corresponding inference chain evaluation result is set as failed and output; if the first total number is zero and the first evaluation result is success, the corresponding inference chain evaluation result is set as passed and output.

[0013] Preferably, the configurable parameters of the evidence completeness test template include inference chain parameters and evidence set parameters; the inference chain parameter is a node chain, which is composed of multiple inference node parameters connected sequentially; the node attributes of each inference node parameter consist of the corresponding inference evidence set parameter, inference operation type parameter, and inference conclusion set parameter; the inference evidence set parameter consists of one or more evidence text parameters and / or conclusion text parameters; the inference operation type parameter includes causal inference, deductive inference, inductive inference, analogical inference, abductive inference, statistical inference, and Bayesian inference; the evidence set parameter includes multiple evidence text parameters. The evidence completeness verification template is used to take the configured inference chain parameters and evidence set parameters as the corresponding current inference chain and current evidence set; and prompts the first model to count the total number of texts of each inference evidence set parameter of the current inference chain as the corresponding first total number of texts, and count the total number of texts from the current evidence set in each inference evidence set parameter as the corresponding second total number of texts, and calculate the effective evidence ratio of the current node = (second total number of texts / first total number of texts) × 100% based on a set of first and second total numbers of texts corresponding to each inference node, and record the inference node parameters whose effective evidence ratio is lower than the preset effective evidence ratio threshold as unqualified nodes, and count the total number of unqualified nodes to obtain the corresponding second total number, and identify the second total number. If the second total number is greater than zero, the corresponding completeness verification result is set as failed and output; if the second total number is equal to zero, the corresponding completeness verification result is set as passed and output.

[0014] Preferably, the configurable parameters of the target question answering template include question parameters, reasoning chain parameters, and evidence set parameters; the reasoning chain parameter is a node chain, composed of multiple reasoning node parameters connected sequentially; the node attributes of each reasoning node parameter consist of the corresponding reasoning evidence set parameter, reasoning operation type parameter, and reasoning conclusion set parameter; the reasoning evidence set parameter consists of one or more evidence text parameters and / or conclusion text parameters; the reasoning operation type parameter includes causal reasoning, deductive reasoning, inductive reasoning, analogical reasoning, abductive reasoning, statistical reasoning, and Bayesian reasoning; the evidence set parameter includes multiple evidence text parameters. The target question answer template is used to take the configured question parameters, inference chain parameters, and evidence set parameters as the corresponding current question, current inference chain, and current evidence set; and prompts the first model to use the current evidence set as inference evidence and the current inference chain as inference path to guide multi-step deduction of the current question, and to comprehensively summarize all deduction outputs, and use the summarized content as the reference context of the current question to perform text generation processing on the answer of the current question, and output the generated text as the corresponding first answer.

[0015] Preferably, the step of driving the first model to generate a corresponding query instruction set for the first question based on the query generation template specifically includes: Based on the first question, the question parameters of the query generation template are set to obtain the corresponding current template text; and the current template text is input as the corresponding current instruction text into the first model for processing to obtain the corresponding query instruction set.

[0016] Preferably, the step of obtaining the original evidence set by querying information according to the query instruction set specifically includes: Each query instruction in the query instruction set is taken as the current instruction; the query content and query object of the current instruction are taken as the current query content and the current query object, respectively; and information is queried on the current query object based on the current query content to obtain the corresponding current query text set, which consists of one or more query texts; each query text in the current query text set is taken as a corresponding original evidence text; and the original evidence source corresponding to each query text in the current query text set is set as the current query object; and each original evidence text and its corresponding original evidence source are combined to form a corresponding first original evidence; and all the obtained first original evidences are combined to form the corresponding original evidence set.

[0017] Preferably, the step of driving the first model based on the original evidence evaluation and optimization template to evaluate the reliability, validity, and relevance of the original evidence set to the first question, and optimizing the original evidence set based on the evaluation results, specifically includes: Based on the first question and the original evidence set, the question parameters and the evidence set parameters of the original evidence evaluation optimization template are set to obtain the corresponding current template text; and the current template text is input as the corresponding current instruction text into the first model for processing to obtain the optimized original evidence set.

[0018] Preferably, the step of driving the first model based on the evidence deduction template to derive the deduced evidence set from the original evidence set with the goal of answering the first question specifically includes: Based on the first question and the original evidence set, the question parameters and the original evidence set parameters of the evidence derivation template are set to obtain the corresponding current template text; and the current template text is input as the corresponding current instruction text into the first model for processing to obtain the corresponding derivation evidence set.

[0019] Preferably, the step of driving the first model based on the derivation evidence evaluation and optimization template to evaluate the support of the derivation basis of the derivation evidence set and its relevance to the first question, and optimizing the derivation evidence set according to the evaluation results, specifically includes: Based on the first question and the derived evidence set, the question parameters and evidence set parameters of the optimized template for the derivation evidence evaluation are set to obtain the corresponding current template text; and the current template text is input as the corresponding current instruction text into the first model for processing to obtain the optimized derivation evidence set.

[0020] Preferably, the step of driving the first model to integrate the original evidence set and the derived evidence set based on the original-derived evidence integration template to obtain the first evidence set specifically includes: Based on the original evidence set and the derived evidence set, the parameters of the original evidence set and the parameters of the derived evidence set of the original-derived evidence integration template are set to obtain the corresponding current template text; and the current template text is input as the corresponding current instruction text into the first model for processing to obtain the corresponding first evidence set.

[0021] Preferably, the step of driving the first model based on the inference chain prediction template to predict the inference path for answering the first question based on the first evidence set to obtain the first inference chain specifically includes: Based on the first question and the first evidence set, the question parameters and the evidence set parameters of the inference chain prediction template are set to obtain the corresponding current template text; and the current template text is input as the corresponding current instruction text into the first model for processing to obtain the corresponding first inference chain.

[0022] Preferably, the step of driving the first model to evaluate the logical consistency and step completeness of the first inference chain based on the inference chain evaluation template specifically includes: Based on the first inference chain, the inference chain parameters of the inference chain prediction template are set to obtain the corresponding current template text; and the current template text is input as the corresponding current instruction text into the first model for processing to obtain the corresponding inference chain evaluation result.

[0023] Preferably, the step of driving the first model based on the evidence completeness test template to test whether each reasoning step of the first inference chain can obtain valid evidence from the first evidence set specifically includes: Based on the first inference chain and the first evidence set, the inference chain parameters and the evidence set parameters of the evidence completeness test template are set to obtain the corresponding current template text; and the current template text is input as the corresponding current instruction text into the first model for processing to obtain the corresponding completeness test result.

[0024] Preferably, the step of driving the first model based on the target question answer template to perform multi-step deduction and answering of the first question using the first evidence set as reasoning evidence and the first reasoning chain as reasoning path, and outputting the corresponding first answer, specifically includes: Based on the first question, the first inference chain, and the first evidence set, the question parameters, inference chain parameters, and evidence set parameters of the target question answer template are set to obtain the corresponding current template text; and the current template text is input as the corresponding current instruction text into the first model for processing to obtain the corresponding first answer.

[0025] A second aspect of the present invention provides an apparatus for implementing the method for processing model reasoning evidence as described in the first aspect above. The apparatus includes: an instruction template setting module, a question receiving module, an original evidence processing module, a deduced evidence processing module, an evidence integration module, a reasoning chain processing module, an evidence completeness verification module, a question deduction module, and an answer feedback module. The instruction template setting module is used to take the target large language model as the first model; and to set multiple prompt instruction templates for the first model; the multiple prompt instruction templates include query generation template, original evidence evaluation and optimization template, evidence derivation template, derivation evidence evaluation and optimization template, original-derivation evidence integration template, inference chain prediction template, inference chain evaluation template, evidence completeness test template, and target question answering template; The question receiving module is used to receive the target question input by the user as the first question; The original evidence processing module drives the first model to generate a corresponding set of query instructions for the first question based on the query generation template; and performs information query based on the query instruction set to obtain the original evidence set; and drives the first model to evaluate the evidence reliability, evidence validity and evidence relevance to the first question of the original evidence set based on the original evidence evaluation and optimization template, and optimizes the original evidence set based on the evaluation results; The derivation evidence processing module drives the first model based on the evidence derivation template to derive the original evidence set with the goal of answering the first question, and drives the first model based on the derivation evidence evaluation and optimization template to evaluate the support of the derivation basis of the derivation evidence set and the evidence relevance to the first question, and optimizes the derivation evidence set according to the evaluation results. The evidence integration module drives the first model to integrate the original evidence set and the derivation evidence set based on the original-derived evidence integration template to obtain a first evidence set. The inference chain processing module drives the first model to predict the reasoning path to answer the first question based on the first evidence set, using the inference chain prediction template to obtain the first inference chain; and drives the first model to evaluate the logical consistency and step completeness of the first inference chain based on the inference chain evaluation template; and continues to drive the first model to predict and evaluate the inference chain based on the inference chain prediction template and the inference chain evaluation template until the evaluation passes. The evidence completeness verification module drives the first model based on the evidence completeness verification template to verify whether each reasoning step of the first inference chain can obtain valid evidence from the first evidence set. The problem derivation module is used to drive the first model to perform multi-step derivation and solution of the first problem based on the target problem solution template after the completeness test is passed, using the first evidence set as reasoning evidence and the first reasoning chain as reasoning path, and output the corresponding first answer. The answer feedback module is used to provide feedback of the first answer to the current user.

[0026] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a transceiver; The processor is used to couple with the memory, read and execute instructions in the memory to implement the steps of the method described in the first aspect above; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0027] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the instructions described in the first aspect.

[0028] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for processing evidence to optimize model inference. As described above, this invention provides multiple prompt instruction templates for optimizing model inference evidence for a target large language model (i.e., the first model): query generation template, original evidence evaluation optimization template, evidence derivation template, derivation evidence evaluation optimization template, original-derivation evidence integration template, inference chain prediction template, inference chain evaluation template, evidence completeness verification template, and target question answering template. Upon receiving the target question (i.e., the first question), the first model, driven by a query generation template, firstly generates a corresponding query instruction set for the first question. Based on this query instruction set, it performs information retrieval to obtain the original evidence set. Then, the first model, driven by an evaluation and optimization template, assesses the reliability, validity, and relevance of the original evidence set to the first question, and optimizes the original evidence set based on the evaluation results. Next, the first model, driven by an evidence derivation template, derives evidence from the original evidence set with the goal of answering the first question, obtaining a derived evidence set. Based on the evaluation and optimization template, the first model, driven by an evidence derivation template, assesses the support of the derivation basis and its relevance to the first question, and optimizes the derived evidence set based on the evaluation results. Finally, the first model, driven by an integration template, integrates the original and derived evidence sets. Evidence integration yields a first evidence set. Then, based on the inference chain prediction template, a first model predicts the reasoning path to answer the first question using the first evidence set, generating a first inference chain. A reasoning chain evaluation template then evaluates the logical consistency and step completeness of the first inference chain. If the evaluation fails, the first model continues to predict and evaluate the inference chain using both the inference chain prediction and evaluation templates until the evaluation passes. Next, an evidence completeness check template checks whether each reasoning step of the first inference chain can obtain valid evidence from the first evidence set. After passing the completeness check, a target question answer template drives the first model to perform a multi-step deduction to answer the first question using the first evidence set as reasoning evidence and the first inference chain as the reasoning path, outputting the corresponding first answer to the current user. This embodiment of the invention, through a customized dedicated model instruction template, optimizes the evaluation of original evidence, optimizes the generation of deduced evidence, and predicts, evaluates, and checks the completeness of the inference chain, thereby improving the quality of reasoning evidence and the quality of model generation. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of a method for processing evidence to optimize model reasoning, provided in Embodiment 1 of the present invention. Figure 2 This is a module structure diagram of a processing device for optimizing model reasoning evidence provided in Embodiment 2 of the present invention; Figure 3This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0031] Embodiment 1 of the present invention provides a method for processing evidence for optimizing model reasoning, such as... Figure 1 The schematic diagram shows a method for processing evidence to optimize model inference provided in Embodiment 1 of the present invention. The method mainly includes the following steps: Step 1: Use the target large language model as the first model; and set multiple prompt instruction templates for the first model.

[0032] Here, the target large language model of this invention should have the following text task processing capabilities: text entity recognition capability, text diversity understanding capability, text generation capability, multi-step reasoning capability, thought chain reasoning capability, context understanding and memory capability, prompting engineering processing capability, and domain knowledge application capability; text diversity understanding capability refers to the multi-dimensional understanding of the semantics, context, intent, emotion, and knowledge domain of the text.

[0033] The multiple prompt instruction templates in this embodiment of the invention include a query generation template, an original evidence evaluation and optimization template, an evidence deduction template, a deduced evidence evaluation and optimization template, an original-derived evidence integration template, a reasoning chain prediction template, a reasoning chain evaluation template, an evidence completeness verification template, and a target question answering template. Each of these nine types of templates is a formatted natural language text template.

[0034] 1) Query to generate template: The configurable parameters of the query generation template in this embodiment of the invention include question parameters.

[0035] The query generation template is used to take the configured question parameters as the corresponding current question; and prompts the first model to perform query range reasoning on the factual evidence required to answer the current question, and generates and outputs the corresponding query instruction set based on the query range reasoning results.

[0036] 2) Optimized template for original evidence evaluation: The configurable parameters of the original evidence evaluation optimization template in this embodiment of the invention include problem parameters and evidence set parameters; the evidence set parameters include multiple evidence parameters; the evidence parameters include evidence text parameters and evidence source parameters.

[0037] This original evidence evaluation optimization template is used to take the configured problem parameters and evidence set parameters as the corresponding current problem and current evidence set; it explains the evaluation requirements for the three evaluation dimensions of evidence reliability, evidence validity, and evidence-problem relevance; it prompts the first model to score the reliability of each evidence source parameter in the current evidence set according to the evidence reliability evaluation requirements, and marks the evidence parameters with scores lower than the preset first reliability score threshold as unqualified parameters; it scores the content validity of each evidence text parameter in the current evidence set according to the evidence validity evaluation requirements, and marks the evidence parameters with scores lower than the preset first validity score threshold as unqualified parameters; it scores the content relevance of each evidence text parameter in the current evidence set to the current problem according to the evidence-problem relevance evaluation requirements, and marks the evidence parameters with scores lower than the preset first relevance score threshold as unqualified parameters; it deletes the evidence parameters marked as unqualified parameters in the current evidence set, and outputs the current evidence set after deleting unqualified parameters as the final optimization result.

[0038] Here, the first reliability score threshold, the first effectiveness score threshold, and the first relevance score threshold in this embodiment of the invention are three preset threshold parameters.

[0039] 3) Evidence deduction template: The configurable parameters of the evidence derivation template in this embodiment of the invention include question parameters and original evidence set parameters; the original evidence set parameters include multiple original evidence text parameters.

[0040] This evidence derivation template is used to take the configured question parameters and original evidence set parameters as the corresponding current question and current evidence set; and prompts the first model to deduce the direct related evidence of the current question based on the current evidence set, and generate and output the corresponding derivation evidence set based on the derivation results.

[0041] 4) Derive an optimized template for evidence evaluation: The configurable parameters of the derivation evidence evaluation optimization template in this embodiment of the invention include problem parameters and evidence set parameters; the evidence set parameters include multiple evidence parameters; the evidence parameters include evidence text parameters and derivation basis parameters; the derivation basis parameters consist of one or more basis text parameters.

[0042] This derivation evidence evaluation optimization template is used to take the configured problem parameters and evidence set parameters as the corresponding current problem and current evidence set; it explains the evaluation requirements for the two evaluation dimensions of derivation basis support and evidence-problem relevance; it prompts the first model to score the derivation evidence support of each derivation basis parameter of the current evidence set to its corresponding evidence text parameter according to the derivation basis support evaluation requirements, and marks the evidence parameters with scores lower than the preset first support threshold as unqualified parameters; it also scores the relevance of each evidence text parameter of the current evidence set to the content of the current problem according to the evidence-problem relevance evaluation requirements, and marks the evidence parameters with scores lower than the preset second relevance score threshold as unqualified parameters; it deletes the evidence parameters marked as unqualified parameters in the current evidence set, and outputs the current evidence set after the unqualified parameters have been deleted as the final optimization result.

[0043] Here, the first support threshold and the second relevance score threshold in this embodiment of the invention are two pre-set threshold parameters.

[0044] 5) Template for integrating original and deduced evidence: The configurable parameters of the original-derived evidence integration template in this embodiment of the invention include original evidence set parameters and derived evidence set parameters; the original evidence set parameters include multiple original evidence text parameters; the derived evidence set parameters include multiple derived evidence text parameters.

[0045] This original-derived evidence integration template is used to take the configured original evidence set parameters and derived evidence set parameters as the corresponding current original evidence set and current derived evidence set. It provides a description of the duplicate text judgment rule with text semantic similarity as the judgment condition and a preset semantic similarity threshold as the judgment threshold. It prompts the first model to take each original evidence text parameter of the current original evidence set and each derived evidence text parameter of the current derived evidence set as a corresponding first evidence text, and form an initial evidence set with the obtained first evidence texts. According to the duplicate text judgment rule, the duplicate texts in the initial evidence set are clustered to obtain multiple cluster subsets. The sum of similarity between each first evidence text in each cluster subset and other texts in the current subset is calculated. The first evidence text with the largest sum of similarity in each cluster subset is extracted to form the final first evidence set and output.

[0046] Here, the semantic similarity threshold in this embodiment of the invention is a pre-set threshold parameter.

[0047] 6) Inference chain prediction template: The configurable parameters of the inference chain prediction template in this embodiment of the invention include question parameters and evidence set parameters; the evidence set parameters include multiple evidence text parameters.

[0048] The inference chain prediction template is used to take the configured question parameters and evidence set parameters as the corresponding current question and current evidence set. It prompts the first model to construct a complete inference path to answer the current question under preset high temperature and low temperature conditions, based on the current evidence set, while ensuring that the logic of the steps before and after the inference chain is consistent and there are no missing steps in the whole chain. This generates a structured high temperature inference chain and a low temperature inference chain. The similarity score of the current pair of high and low temperature inference chains is calculated, and it is judged whether the similarity score exceeds the preset similarity score threshold. If it does not exceed the similarity score threshold, a new pair of high and low temperature inference chains is generated until the latest score exceeds the similarity score threshold. If it exceeds the similarity score threshold, the current low temperature inference chain is output as the corresponding first inference chain.

[0049] It should be noted that the temperature condition is a hyperparameter used to control the generation strategy of the first model, specifically a positive real number. In this embodiment of the invention, the value of the high-temperature condition is greater than 1, and the value of the low-temperature condition is less than 1. In this embodiment of the invention, the high-temperature inference chain generated by the first model under high-temperature conditions has greater randomness than the low-temperature inference chain, and the low-temperature inference chain generated under low-temperature conditions has greater logical rationality than the high-temperature inference chain.

[0050] Here, the similarity score threshold in this embodiment of the invention is a pre-set threshold parameter.

[0051] 7) Inference Chain Evaluation Template: The configurable parameters of the inference chain evaluation template in this embodiment of the invention include inference chain parameters; the inference chain parameter is a node chain, which is composed of multiple inference node parameters connected sequentially; the node attributes of each inference node parameter consist of the corresponding inference evidence set parameter, inference operation type parameter, and inference conclusion set parameter; the inference evidence set parameter consists of one or more evidence text parameters and / or conclusion text parameters; the inference operation type parameter includes causal inference, deductive inference, inductive inference, analogical inference, abductive inference, statistical inference, and Bayesian inference; the inference conclusion set parameter consists of one or more conclusion text parameters.

[0052] This inference chain evaluation template is used to take the configured inference chain parameters as the corresponding current inference chain; it explains the evaluation requirements for two evaluation dimensions: logical consistency of preceding and following nodes, and step integrity of all chain nodes; it prompts the first model to score the logical consistency of each inference node parameter according to the logical consistency evaluation requirements of preceding and following nodes, and marks the inference node parameter with a score lower than the preset logical consistency score threshold as an unqualified node; it scores the step integrity of the inference chain parameter according to the step integrity evaluation requirements of all chain nodes, and identifies whether the score is lower than the preset step integrity score threshold. If it is, the corresponding first evaluation result is set to failure; otherwise, the corresponding first evaluation result is set to success. It also counts the total number of unqualified nodes to obtain the corresponding first total, and identifies the first total and the first evaluation result. If the first total is greater than zero or the first evaluation result is failure, the corresponding inference chain evaluation result is set to pass and output; if the first total is zero and the first evaluation result is success, the corresponding inference chain evaluation result is set to pass and output.

[0053] Here, the logical consistency scoring threshold and the step integrity scoring threshold in this embodiment of the invention are two pre-set threshold parameters.

[0054] 8) Template for Evidence Completeness Test: The configurable parameters of the evidence completeness test template in this embodiment of the invention include inference chain parameters and evidence set parameters. The inference chain parameter is a node chain, which is composed of multiple inference node parameters connected sequentially. The node attributes of each inference node parameter consist of the corresponding inference evidence set parameter, inference operation type parameter, and inference conclusion set parameter. The inference evidence set parameter consists of one or more evidence text parameters and / or conclusion text parameters. The inference operation type parameter includes causal inference, deductive inference, inductive inference, analogical inference, abductive inference, statistical inference, and Bayesian inference. The evidence set parameter includes multiple evidence text parameters.

[0055] This evidence completeness verification template uses the configured inference chain parameters and evidence set parameters as the corresponding current inference chain and current evidence set. It prompts the first model to count the total number of texts in each inference evidence set parameter of the current inference chain as the corresponding first total text, and to count the total number of texts from the current evidence set in each inference evidence set parameter as the corresponding second total text. Based on the first and second total texts corresponding to each inference node, it calculates the effective evidence ratio of the current node = (second total text / first total text) × 100%. Inference node parameters with an effective evidence ratio lower than the preset effective evidence ratio threshold are recorded as unqualified nodes. The total number of unqualified nodes is counted to obtain the corresponding second total. The second total is identified. If the second total is greater than zero, the corresponding completeness verification result is set as failed and output. If the second total is equal to zero, the corresponding completeness verification result is set as passed and output.

[0056] Here, the effective evidence ratio threshold in this embodiment of the invention is a pre-set threshold parameter.

[0057] 9) Target Question Answering Template: The configurable parameters of the target question answering template in this embodiment of the invention include question parameters, reasoning chain parameters, and evidence set parameters. The reasoning chain parameter is a node chain, which is composed of multiple reasoning node parameters connected sequentially. The node attributes of each reasoning node parameter consist of the corresponding reasoning evidence set parameter, reasoning operation type parameter, and reasoning conclusion set parameter. The reasoning evidence set parameter consists of one or more evidence text parameters and / or conclusion text parameters. The reasoning operation type parameter includes causal reasoning, deductive reasoning, inductive reasoning, analogical reasoning, abductive reasoning, statistical reasoning, and Bayesian reasoning. The evidence set parameter includes multiple evidence text parameters.

[0058] This target question answering template is used to take the configured question parameters, inference chain parameters, and evidence set parameters as the corresponding current question, current inference chain, and current evidence set; and prompts the first model to use the current evidence set as inference evidence and the current inference chain as inference path to guide multi-step deduction of the current question, and to comprehensively summarize all deduction outputs, and use the summarized content as the reference context of the current question to perform text generation processing of the answer to the current question, and output the generated text as the corresponding first answer.

[0059] Step 2: Receive the target question input by the user as the first question.

[0060] Step 3: Based on the query generation template, the first model generates a corresponding query instruction set for the first question; and based on the query instruction set, information is queried to obtain the original evidence set; and based on the original evidence evaluation and optimization template, the first model evaluates the reliability, validity, and relevance of the original evidence set to the first question, and optimizes the original evidence set based on the evaluation results. Specifically, this includes: Step 31, generating a corresponding set of query instructions for the first question based on the query generation template; Specifically, this includes: setting the question parameters of the query template based on the first question to obtain the corresponding current template text; and inputting the current template text as the corresponding current instruction text into the first model for processing to obtain the corresponding query instruction set; Here, the query instruction set in this embodiment of the invention includes multiple query instructions; the query instructions include query content and query object; the query content is query keywords; the query object is a public query engine, dataset, or knowledge base; Step 32, and obtain the original evidence set by performing information query according to the query instruction set; Specifically, this includes: taking each query instruction in the query instruction set as the current instruction; taking the query content and query object of the current instruction as the current query content and the current query object; and performing information query on the current query object based on the current query content to obtain the corresponding current query text set, which consists of one or more query texts; taking each query text in the current query text set as a corresponding original evidence text; taking the original evidence source corresponding to each query text in the current query text set as the current query object; and taking each original evidence text and its corresponding original evidence source to form a corresponding first original evidence; and taking all the obtained first original evidence to form the corresponding original evidence set. Here, the original evidence set in this embodiment of the invention includes multiple first original pieces of evidence; the first original pieces of evidence include the original evidence text and the original evidence source; the original evidence source is the retrieval source of the current evidence; Step 33, and based on the original evidence evaluation optimization template, drive the first model to evaluate the evidence reliability, evidence validity and evidence relevance to the first question of the original evidence set, and optimize the original evidence set according to the evaluation results; Specifically, this includes: setting the question parameters and evidence set parameters of the original evidence evaluation optimization template based on the first question and the original evidence set to obtain the corresponding current template text; and inputting the current template text as the corresponding current instruction text into the first model for processing to obtain the optimized original evidence set.

[0061] Step 4: Based on the evidence deduction template, drive the first model to deduce evidence from the original evidence set with the goal of answering the first question, and obtain the deduced evidence set; and based on the deduced evidence evaluation and optimization template, drive the first model to evaluate the support of the deduction basis of the deduced evidence set and the evidence relevance to the first question, and optimize the deduced evidence set according to the evaluation results; Specifically, it includes: Step 41, based on the evidence deduction template, the first model is driven to perform evidence deduction on the original evidence set with the goal of answering the first question, and the deduced evidence set is obtained; Specifically, this includes: setting the question parameters and original evidence set parameters of the evidence derivation template based on the first question and the original evidence set to obtain the corresponding current template text; and inputting the current template text as the corresponding current instruction text into the first model for processing to obtain the corresponding derivation evidence set; Here, the deduced evidence set in this embodiment of the invention includes multiple first deduced evidences; the first deduced evidences include deduced evidence texts and a deduced basis set; the deduced basis set consists of one or more original evidence texts; Step 42, and based on the derivation evidence evaluation optimization template, drive the first model to evaluate the support of the derivation basis of the derivation evidence set and the evidence relevance to the first question, and optimize the derivation evidence set according to the evaluation results; Specifically, this includes: setting the problem parameters and evidence set parameters of the derivation evidence evaluation optimization template based on the first problem and the derivation evidence set to obtain the corresponding current template text; and inputting the current template text as the corresponding current instruction text into the first model for processing to obtain the optimized derivation evidence set.

[0062] Step 5: Based on the original-derived evidence integration template, the first model is driven to integrate the original evidence set and the derived evidence set to obtain the first evidence set; Specifically, this includes: setting the parameters of the original evidence set and the parameters of the derived evidence set for the original-derived evidence integration template based on the original evidence set and the derived evidence set to obtain the corresponding current template text; and inputting the current template text as the corresponding current instruction text into the first model for processing to obtain the corresponding first evidence set.

[0063] Here, the first evidence set in this embodiment of the invention includes multiple first evidence texts; Step 6: Based on the reasoning chain prediction template, the first model is driven to predict the reasoning path for answering the first question according to the first evidence set to obtain the first reasoning chain; and based on the reasoning chain evaluation template, the first model is driven to evaluate the logical consistency and step completeness of the first reasoning chain; and if the evaluation fails, the first model is driven to predict and evaluate the reasoning chain again based on the reasoning chain prediction template and the reasoning chain evaluation template until the evaluation passes. Specifically, it includes: Step 61, using the inference chain prediction template to drive the first model to predict the inference path to answer the first question based on the first evidence set to obtain the first inference chain; Specifically, this includes: setting the question parameters and evidence set parameters of the inference chain prediction template based on the first question and the first evidence set to obtain the corresponding current template text; and inputting the current template text as the corresponding current instruction text into the first model for processing to obtain the corresponding first inference chain; Here, the first reasoning chain in this embodiment of the invention is a node chain, which is formed by sequentially connecting multiple single-step reasoning nodes; the node attributes of each single-step reasoning node consist of the corresponding reasoning evidence set, reasoning operation type, and reasoning conclusion set; the reasoning evidence set is the set of reference evidence for the current single-step reasoning, which consists of one or more first evidence texts and / or one or more first conclusion texts; the reasoning operation type is the reasoning logic used in the current single-step reasoning, including causal reasoning, deductive reasoning, inductive reasoning, analogical reasoning, abductive reasoning, statistical reasoning, and Bayesian reasoning; the reasoning conclusion set is the output result of the current single-step reasoning, which consists of one or more first conclusion texts; Step 62, and based on the inference chain evaluation template, drive the first model to evaluate the logical consistency and step integrity of the first inference chain; Specifically, this includes: setting the inference chain parameters of the inference chain prediction template based on the first inference chain to obtain the corresponding current template text; and inputting the current template text as the corresponding current instruction text into the first model for processing to obtain the corresponding inference chain evaluation result; Step 63, and if the evaluation fails, continue to drive the first model to perform inference chain prediction and evaluation based on the inference chain prediction template and the inference chain evaluation template until the evaluation passes.

[0064] Step 7: Based on the evidence completeness test template, drive the first model to test whether each reasoning step of the first reasoning chain can obtain valid evidence from the first evidence set; Specifically, this includes: setting the inference chain parameters and evidence set parameters of the evidence completeness test template based on the first inference chain and the first evidence set to obtain the corresponding current template text; and inputting the current template text as the corresponding current instruction text into the first model for processing to obtain the corresponding completeness test result.

[0065] It should be noted that if the completeness check result is passed, the process continues to step 8; if it fails, this embodiment of the invention will further locate which specific inference nodes in the first inference chain have insufficient evidence completeness, and will sequentially designate each inference node with insufficient evidence completeness as the current defect node. A corresponding current node problem will be formed by the first question and the inference conclusion set of the current defect node. Based on the current node problem, the question parameters of the query generation template will be set to obtain the corresponding current template text. The current template text will then be used as the corresponding current instruction text and input into the first model for processing to obtain the corresponding current query instruction set. Information will be queried based on the current query instruction set to obtain a new original evidence set, denoted as the current node evidence set. The original evidence will then be evaluated and optimized. The template-driven first model evaluates the reliability, validity, and relevance of the evidence set of the current node to the current node problem, and optimizes the current node evidence set based on the evaluation results. Then, a defective node evidence set is formed by combining the node evidence sets of all defective nodes. The evidence text of the defective node evidence set is then merged into the current first evidence set. After the evidence is merged, the template-driven first model is again tested based on the evidence completeness test to see if each reasoning step of the first reasoning chain can obtain valid evidence from the first evidence set. If the test passes, the subsequent step 8 continues; if it fails, the above process of defective node location, node problem assembly, node evidence set query, node evidence optimization evaluation, evidence merging, and evidence completeness test retest is repeated until the test passes.

[0066] Step 8: After the completeness test is passed, the first model is driven by the target question answer template to perform multi-step deduction and answer to the first question using the first evidence set as reasoning evidence and the first reasoning chain as reasoning path, and outputs the corresponding first answer. Specifically, this includes: setting the question parameters, inference chain parameters, and evidence set parameters of the target question answer template based on the first question, the first inference chain, and the first evidence set to obtain the corresponding current template text; and inputting the current template text as the corresponding current instruction text into the first model for processing to obtain the corresponding first answer.

[0067] Step 9: Provide feedback of the first answer to the current user.

[0068] Figure 2 This is a module structure diagram of a processing device for optimizing model reasoning evidence provided in Embodiment 2 of the present invention. This device can be a terminal device or server implementing the aforementioned method embodiments, or it can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiments. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 2As shown, the device includes: an instruction template setting module 201, a question receiving module 202, an original evidence processing module 203, a deduced evidence processing module 204, an evidence integration module 205, a reasoning chain processing module 206, an evidence completeness verification module 207, a question deduction module 208, and an answer feedback module 209.

[0069] The instruction template setting module 201 is used to take the target large language model as the first model; and to set multiple prompt instruction templates for the first model; the multiple prompt instruction templates include query generation template, original evidence evaluation and optimization template, evidence derivation template, derivation evidence evaluation and optimization template, original-derivation evidence integration template, inference chain prediction template, inference chain evaluation template, evidence completeness test template, and target question answering template.

[0070] The question receiving module 202 is used to receive the target question input by the user as the first question.

[0071] The original evidence processing module 203 generates a corresponding set of query instructions for the first question based on the query generation template-driven first model; and obtains the original evidence set by performing information queries based on the query instruction set; and evaluates the evidence reliability, evidence validity and evidence relevance to the first question of the original evidence set based on the original evidence evaluation and optimization template-driven first model, and optimizes the original evidence set based on the evaluation results.

[0072] The derivation evidence processing module 204 drives the first model based on the evidence derivation template to derive the original evidence set with the goal of answering the first question, and obtains the derivation evidence set. Based on the derivation evidence evaluation and optimization template, the first model evaluates the support of the derivation basis of the derivation evidence set and the evidence relevance to the first question, and optimizes the derivation evidence set according to the evaluation results.

[0073] The evidence integration module 205 drives the first model to integrate the original evidence set and the deduced evidence set based on the original-derived evidence integration template to obtain the first evidence set.

[0074] The reasoning chain processing module 206 drives the first model based on the reasoning chain prediction template to predict the reasoning path to answer the first question according to the first evidence set and obtain the first reasoning chain; and drives the first model based on the reasoning chain evaluation template to evaluate the logical consistency and step completeness of the first reasoning chain; and if the evaluation fails, it continues to drive the first model based on the reasoning chain prediction template and the reasoning chain evaluation template to predict and evaluate the reasoning chain until the evaluation passes.

[0075] The evidence completeness verification module 207 drives the first model based on the evidence completeness verification template to verify whether each reasoning step of the first reasoning chain can obtain valid evidence from the first evidence set.

[0076] The problem derivation module 208 is used to drive the first model to perform multi-step derivation and solution of the first problem based on the target problem solution template after the completeness test is passed, using the first evidence set as reasoning evidence and the first reasoning chain as reasoning path, and output the corresponding first answer.

[0077] The answer feedback module 209 is used to provide feedback on the first answer to the current user.

[0078] The present invention provides a processing device for optimizing model reasoning evidence, which can execute the method steps in the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0079] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, the instruction template setting module can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0080] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SOC).

[0081] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the foregoing method embodiments are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0082] Figure 3 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be a terminal device or server implementing the methods of the aforementioned embodiments, or it can be a terminal device or server connected to the aforementioned terminal device or server implementing the methods of the aforementioned embodiments. Figure 3 As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0083] exist Figure 3The system bus 305 mentioned can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include Non-Volatile Memory, such as at least one disk storage device.

[0084] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0085] It should be noted that the embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to perform the methods and processes provided in the above embodiments.

[0086] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for processing evidence to optimize model inference. As described above, this invention provides multiple prompt instruction templates for optimizing model inference evidence for a target large language model (i.e., the first model): query generation template, original evidence evaluation optimization template, evidence derivation template, derivation evidence evaluation optimization template, original-derivation evidence integration template, inference chain prediction template, inference chain evaluation template, evidence completeness verification template, and target question answering template. Upon receiving the target question (i.e., the first question), the first model, driven by a query generation template, firstly generates a corresponding query instruction set for the first question. Based on this query instruction set, it performs information retrieval to obtain the original evidence set. Then, the first model, driven by an evaluation and optimization template, assesses the reliability, validity, and relevance of the original evidence set to the first question, and optimizes the original evidence set based on the evaluation results. Next, the first model, driven by an evidence derivation template, derives evidence from the original evidence set with the goal of answering the first question, obtaining a derived evidence set. Based on the evaluation and optimization template, the first model, driven by an evidence derivation template, assesses the support of the derivation basis and its relevance to the first question, and optimizes the derived evidence set based on the evaluation results. Finally, the first model, driven by an integration template, integrates the original and derived evidence sets. Evidence integration yields a first evidence set. Then, based on the inference chain prediction template, a first model predicts the reasoning path to answer the first question using the first evidence set, generating a first inference chain. A reasoning chain evaluation template then evaluates the logical consistency and step completeness of the first inference chain. If the evaluation fails, the first model continues to predict and evaluate the inference chain using both the inference chain prediction and evaluation templates until the evaluation passes. Next, an evidence completeness check template checks whether each reasoning step of the first inference chain can obtain valid evidence from the first evidence set. After passing the completeness check, a target question answer template drives the first model to perform a multi-step deduction to answer the first question using the first evidence set as reasoning evidence and the first inference chain as the reasoning path, outputting the corresponding first answer to the current user. This embodiment of the invention, through a customized dedicated model instruction template, optimizes the evaluation of original evidence, optimizes the generation of deduced evidence, and predicts, evaluates, and checks the completeness of the inference chain, thereby improving the quality of reasoning evidence and the quality of model generation.

[0087] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0088] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing evidence to optimize model reasoning, characterized in that, The method includes: The target large language model is used as the first model; and multiple prompt instruction templates are set for the first model; the multiple prompt instruction templates include query generation template, original evidence evaluation and optimization template, evidence derivation template, derivation evidence evaluation and optimization template, original-derivation evidence integration template, inference chain prediction template, inference chain evaluation template, evidence completeness test template, and target question answering template; The target question, received from user input, is the first question. Based on the query generation template, the first model is driven to generate a corresponding query instruction set for the first question; and information is queried according to the query instruction set to obtain the original evidence set; and based on the original evidence evaluation and optimization template, the first model is driven to evaluate the evidence reliability, evidence validity, and evidence relevance to the first question of the original evidence set, and optimize the original evidence set according to the evaluation results; Based on the evidence deduction template, the first model is driven to deduce evidence from the original evidence set with the goal of answering the first question, thereby obtaining a deduced evidence set; and based on the deduced evidence evaluation and optimization template, the first model is driven to evaluate the support of the deduction basis of the deduced evidence set and its evidence relevance to the first question, and optimize the deduced evidence set according to the evaluation results; Based on the original-derived evidence integration template, the first model is driven to integrate the original evidence set and the derived evidence set to obtain the first evidence set. The first model is driven by the inference chain prediction template to predict the reasoning path to answer the first question based on the first evidence set, thereby obtaining a first inference chain; and the first model is driven by the inference chain evaluation template to evaluate the logical consistency and step completeness of the first inference chain; and if the evaluation fails, the first model continues to predict and evaluate the inference chain based on the inference chain prediction template and the inference chain evaluation template until the evaluation passes. Based on the evidence completeness test template, the first model is driven to test whether each reasoning step of the first reasoning chain can obtain valid evidence from the first evidence set. After the completeness test is passed, the first model is driven by the target question answer template to perform multi-step deduction and answer to the first question using the first evidence set as reasoning evidence and the first reasoning chain as reasoning path, and outputs the corresponding first answer. The first answer will be sent back to the current user.

2. The method for processing evidence for optimizing model reasoning according to claim 1, characterized in that, The target large language model has the ability to recognize text entities, understand text diversity, generate text, reason in multiple steps, reason along thought chains, understand and remember context, process prompts, and apply domain knowledge. The aforementioned text diversity comprehension ability refers to the ability to understand texts in multiple dimensions, including semantics, context, intent, emotion, and knowledge domain. The query generation template, the original evidence evaluation and optimization template, the evidence derivation template, the derivation evidence evaluation and optimization template, the original-derived evidence integration template, the reasoning chain prediction template, the reasoning chain evaluation template, the evidence completeness verification template, and the target question answering template are each a formatted natural language text template; The query instruction set includes multiple query instructions; each query instruction includes query content and query object; the query content is query keywords; and the query object is a publicly available query engine, dataset, or knowledge base. The original evidence set includes multiple first original pieces of evidence; the first original pieces of evidence include the original evidence text and the original evidence source; the original evidence source is the retrieval source of the current evidence; The deduced evidence set includes multiple first deduced evidences; the first deduced evidences include deduced evidence texts and a set of deduced basis; the set of deduced basis consists of one or more of the original evidence texts; The first set of evidence includes multiple first evidence texts; The first inference chain is a node chain, consisting of multiple single-step inference nodes connected sequentially; the node attributes of each single-step inference node consist of a corresponding inference evidence set, inference operation type, and inference conclusion set; the inference evidence set is the set of reference evidence for the current single-step inference, consisting of one or more first evidence texts and / or one or more first conclusion texts; the inference operation type is the inference logic used in the current single-step inference, including causal inference, deductive inference, inductive inference, analogical inference, abductive inference, statistical inference, and Bayesian inference; the inference conclusion set is the output result of the current single-step inference, consisting of one or more first conclusion texts.

3. The method for processing evidence to optimize model reasoning according to claim 2, characterized in that, The configurable parameters for the query generation template include question parameters; The query generation template is used to take the configured question parameters as the corresponding current question; and prompts the first model to perform query range reasoning on the factual evidence required to answer the current question, and generates and outputs the corresponding query instruction set based on the query range reasoning result.

4. The method for processing evidence for optimizing model reasoning according to claim 2, characterized in that, The configurable parameters of the original evidence evaluation optimization template include question parameters and evidence set parameters; the evidence set parameters include multiple evidence parameters; the evidence parameters include evidence text parameters and evidence source parameters. The original evidence evaluation optimization template is used to take the configured problem parameters and evidence set parameters as the corresponding current problem and current evidence set; and to explain the evaluation requirements for the three evaluation dimensions of evidence reliability, evidence validity, and evidence-problem relevance. It prompts the first model to score the reliability of each evidence source parameter in the current evidence set according to the evidence reliability evaluation requirements, and to mark evidence parameters with scores lower than a preset first reliability score threshold as unqualified parameters; to score the content validity of each evidence text parameter in the current evidence set according to the evidence validity evaluation requirements, and to mark evidence parameters with scores lower than a preset first validity score threshold as unqualified parameters; and to score the content relevance of each evidence text parameter in the current evidence set to the current problem according to the evidence-problem relevance evaluation requirements, and to mark evidence parameters with scores lower than a preset first relevance score threshold as unqualified parameters. The template then deletes the unqualified evidence parameters from the current evidence set and outputs the current evidence set after deleting the unqualified parameters as the final optimization result.

5. The method for processing evidence for optimizing model reasoning according to claim 2, characterized in that, The configurable parameters of the evidence derivation template include question parameters and original evidence set parameters; the original evidence set parameters include multiple original evidence text parameters. The evidence derivation template is used to take the configured question parameters and the original evidence set parameters as the corresponding current question and current evidence set; and prompts the first model to deduce the direct related evidence of the current question based on the current evidence set and generate and output the corresponding derivation evidence set based on the derivation result.

6. The method for processing evidence for optimizing model reasoning according to claim 2, characterized in that, The configurable parameters of the derivation evidence evaluation optimization template include question parameters and evidence set parameters; the evidence set parameters include multiple evidence parameters; the evidence parameters include evidence text parameters and derivation basis parameters; the derivation basis parameters consist of one or more basis text parameters. The derivation evidence evaluation optimization template is used to take the configured problem parameters and evidence set parameters as the corresponding current problem and current evidence set; and to explain the evaluation requirements of the two evaluation dimensions of derivation basis support and evidence-problem relevance; and to prompt the first model to score the derivation evidence support of each derivation basis parameter of the current evidence set to the corresponding evidence text parameter according to the derivation basis support evaluation requirements, and to mark the evidence parameters with scores lower than a preset first support threshold as unqualified parameters; and to score the content relevance of each evidence text parameter of the current evidence set to the current problem according to the evidence-problem relevance evaluation requirements, and to mark the evidence parameters with scores lower than a preset second relevance score threshold as unqualified parameters; and to delete the evidence parameters marked as unqualified parameters in the current evidence set, and to output the current evidence set after the unqualified parameters have been deleted as the final optimization result.

7. The method for processing evidence for optimizing model reasoning according to claim 2, characterized in that, The configurable parameters of the original-derived evidence integration template include original evidence set parameters and derived evidence set parameters; the original evidence set parameters include multiple original evidence text parameters; the derived evidence set parameters include multiple derived evidence text parameters. The original-derived evidence integration template is used to take the configured original evidence set parameters and the derived evidence set parameters as the corresponding current original evidence set and current derived evidence set; and provides an explanation of the duplicate text determination rules with text semantic similarity as the determination condition and a preset semantic similarity threshold as the determination threshold. The first model is prompted to treat each original evidence text parameter of the current original evidence set and each derivation evidence text parameter of the current derivation evidence set as a corresponding first evidence text, and to form an initial evidence set by the obtained first evidence texts. The duplicate texts in the initial evidence set are clustered according to the duplicate text determination rule to obtain multiple cluster subsets. The sum of similarity between each first evidence text in each cluster subset and other texts in the current subset is calculated. The first evidence text with the largest sum of similarity in each cluster subset is extracted to form the final first evidence set and output.

8. The method for processing evidence for optimizing model reasoning according to claim 2, characterized in that, The configurable parameters of the inference chain prediction template include question parameters and evidence set parameters; the evidence set parameters include multiple evidence text parameters. The inference chain prediction template is used to take the configured question parameters and evidence set parameters as the corresponding current question and current evidence set; and prompts the first model to construct a complete inference path to answer the current question under preset high temperature and low temperature conditions based on the current evidence set, on the premise of ensuring that the logic of the steps before and after the inference chain is consistent and there are no missing steps in the whole chain, thereby generating a structured high temperature inference chain and low temperature inference chain. The similarity score of the current pair of high and low temperature inference chains is calculated, and it is judged whether the similarity score exceeds the preset similarity score threshold. If it does not exceed the similarity score threshold, a new pair of high and low temperature inference chains is generated until the latest score exceeds the similarity score threshold. If it exceeds the similarity score threshold, the current low temperature inference chain is output as the corresponding first inference chain. Among them, temperature condition is a type of hyperparameter used to control the model generation strategy, specifically a positive real number; the value of the high temperature condition is greater than 1, and the value of the low temperature condition is less than 1; the high temperature inference chain generated under the high temperature condition has greater randomness than the low temperature inference chain, and the low temperature inference chain generated under the low temperature condition has greater logical rationality than the high temperature inference chain.

9. The method for processing evidence for optimizing model reasoning according to claim 2, characterized in that, The configurable parameters of the inference chain evaluation template include inference chain parameters; the inference chain parameters are a node chain, composed of multiple inference node parameters connected sequentially; the node attributes of each inference node parameter consist of a corresponding inference evidence set parameter, inference operation type parameter, and inference conclusion set parameter; the inference evidence set parameter consists of one or more evidence text parameters and / or conclusion text parameters; the inference operation type parameter includes causal inference, deductive inference, inductive inference, analogical inference, abductive inference, statistical inference, and Bayesian inference; the inference conclusion set parameter consists of one or more conclusion text parameters. The inference chain evaluation template is used to take the configured inference chain parameters as the corresponding current inference chain; and explains the evaluation requirements for two evaluation dimensions: logical consistency of preceding and following nodes and step integrity of the entire chain nodes; it prompts the first model to score the logical consistency of each inference node parameter of the inference chain parameters according to the logical consistency evaluation requirements of preceding and following nodes, and to mark the inference node parameter with a score lower than a preset logical consistency score threshold as an unqualified node; and to score the step integrity of the inference chain parameters according to the step integrity evaluation requirements of the entire chain nodes, and to identify whether the score is lower than a preset step integrity score threshold. If it is, the corresponding first evaluation result is set as failure; otherwise, the corresponding first evaluation result is set as success. The total number of unqualified nodes is counted to obtain the corresponding first total number, and the first total number and the first evaluation result are identified. If the first total number is greater than zero or the first evaluation result is failure, the corresponding inference chain evaluation result is set as failed and output; if the first total number is zero and the first evaluation result is success, the corresponding inference chain evaluation result is set as passed and output.

10. The method for processing evidence for optimizing model reasoning according to claim 2, characterized in that, The configurable parameters of the evidence completeness verification template include inference chain parameters and evidence set parameters. The inference chain parameter is a node chain, which is composed of multiple inference node parameters connected sequentially. The node attributes of each inference node parameter consist of the corresponding inference evidence set parameter, inference operation type parameter, and inference conclusion set parameter. The inference evidence set parameter consists of one or more evidence text parameters and / or conclusion text parameters. The inference operation type parameter includes causal inference, deductive inference, inductive inference, analogical inference, abductive inference, statistical inference, and Bayesian inference. The evidence set parameter includes multiple evidence text parameters. The evidence completeness verification template is used to take the configured inference chain parameters and evidence set parameters as the corresponding current inference chain and current evidence set; and prompts the first model to count the total number of texts of each inference evidence set parameter of the current inference chain as the corresponding first total number of texts, and count the total number of texts from the current evidence set in each inference evidence set parameter as the corresponding second total number of texts, and calculate the effective evidence ratio of the current node = (second total number of texts / first total number of texts) × 100% based on a set of first and second total numbers of texts corresponding to each inference node, and record the inference node parameters whose effective evidence ratio is lower than the preset effective evidence ratio threshold as unqualified nodes, and count the total number of unqualified nodes to obtain the corresponding second total number, and identify the second total number. If the second total number is greater than zero, the corresponding completeness verification result is set as failed and output; if the second total number is equal to zero, the corresponding completeness verification result is set as passed and output.

11. The method for processing evidence for optimizing model reasoning according to claim 2, characterized in that, The configurable parameters of the target question answering template include question parameters, reasoning chain parameters, and evidence set parameters. The reasoning chain parameter is a node chain, composed of multiple reasoning node parameters connected sequentially. The node attributes of each reasoning node parameter consist of the corresponding reasoning evidence set parameter, reasoning operation type parameter, and reasoning conclusion set parameter. The reasoning evidence set parameter consists of one or more evidence text parameters and / or conclusion text parameters. The reasoning operation type parameter includes causal reasoning, deductive reasoning, inductive reasoning, analogical reasoning, abductive reasoning, statistical reasoning, and Bayesian reasoning. The evidence set parameter includes multiple evidence text parameters. The target question answer template is used to take the configured question parameters, inference chain parameters, and evidence set parameters as the corresponding current question, current inference chain, and current evidence set; and prompts the first model to use the current evidence set as inference evidence and the current inference chain as inference path to guide multi-step deduction of the current question, and to comprehensively summarize all deduction outputs, and use the summarized content as the reference context of the current question to perform text generation processing on the answer of the current question, and output the generated text as the corresponding first answer.

12. An apparatus for performing the processing method for optimizing model reasoning evidence according to any one of claims 1-11, characterized in that, The device includes: an instruction template setting module, a question receiving module, an original evidence processing module, a deduced evidence processing module, an evidence integration module, a reasoning chain processing module, an evidence completeness verification module, a question deduction module, and an answer feedback module; The instruction template setting module is used to take the target large language model as the first model; and to set multiple prompt instruction templates for the first model; the multiple prompt instruction templates include query generation template, original evidence evaluation and optimization template, evidence derivation template, derivation evidence evaluation and optimization template, original-derivation evidence integration template, inference chain prediction template, inference chain evaluation template, evidence completeness test template, and target question answering template; The question receiving module is used to receive the target question input by the user as the first question; The original evidence processing module drives the first model to generate a corresponding set of query instructions for the first question based on the query generation template; and performs information query based on the query instruction set to obtain the original evidence set; and drives the first model to evaluate the evidence reliability, evidence validity and evidence relevance to the first question of the original evidence set based on the original evidence evaluation and optimization template, and optimizes the original evidence set based on the evaluation results; The derivation evidence processing module drives the first model based on the evidence derivation template to derive the original evidence set with the goal of answering the first question, and drives the first model based on the derivation evidence evaluation and optimization template to evaluate the support of the derivation basis of the derivation evidence set and the evidence relevance to the first question, and optimizes the derivation evidence set according to the evaluation results. The evidence integration module drives the first model to integrate the original evidence set and the derived evidence set based on the original-derived evidence integration template to obtain a first evidence set. The inference chain processing module drives the first model to predict the reasoning path to answer the first question based on the first evidence set, using the inference chain prediction template to obtain the first inference chain; and drives the first model to evaluate the logical consistency and step completeness of the first inference chain based on the inference chain evaluation template; and continues to drive the first model to predict and evaluate the inference chain based on the inference chain prediction template and the inference chain evaluation template until the evaluation passes. The evidence completeness verification module drives the first model based on the evidence completeness verification template to verify whether each reasoning step of the first inference chain can obtain valid evidence from the first evidence set. The problem derivation module is used to drive the first model to perform multi-step derivation and solution of the first problem based on the target problem solution template after the completeness test is passed, using the first evidence set as reasoning evidence and the first reasoning chain as reasoning path, and output the corresponding first answer. The answer feedback module is used to provide feedback on the first answer to the current user.

13. An electronic device, characterized in that, include: Memory, processor, and transceiver; The processor is configured to be coupled to the memory, read and execute instructions in the memory to implement the method according to any one of claims 1-11; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1-11.

Images