A question and answer task processing model training method and device, equipment and medium

Abstract

The application discloses a question and answer task processing model training method and device, equipment and medium, relates to the technical field of natural language processing, and comprises the following steps: extracting a plurality of key structural elements from a training sample question, and guiding a current question and answer task processing model to generate a corresponding initial reasoning path by using each key structural element; taking each initial reasoning path as a root node, determining a branch node based on the information entropy of each word position in the initial reasoning path, and then generating a branch reasoning path at the branch node to obtain an extended reasoning tree; calculating the node value of each node in the extended reasoning tree, and further calculating the global advantage, the local advantage and the path residual; training the current question and answer task processing model by using the reward value determined based on the global advantage, the local advantage and the path residual, and finally obtaining a trained target question and answer task processing model through multiple rounds of iteration, so that the target question and answer task processing model is used to process the obtained to-be-processed question and answer task.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method, apparatus, device, and medium for training a question-answering task processing model. Background Technology

[0002] Current reinforcement learning (RL) training for Large Language Models (LLMs) primarily employs an independent multi-chain sampling strategy, generating multiple independent response trajectories and providing a binary reward signal based on the correctness of the final result. This method has the following limitations: First, independent sampling struggles to effectively explore complex inference spaces, leading to insufficient sample diversity; second, sparse binary reward signals cannot provide fine-grained supervision of the inference process, affecting the model's learning efficiency; furthermore, the training process is prone to high variance and instability. While traditional tree search methods such as Monte Carlo tree search perform well in inference tasks, their direct application to LLM training faces challenges such as low computational efficiency and poor compatibility with existing inference engines. Furthermore, they typically require additional training of the reward model, which can easily lead to distribution shift and reward hacking problems.

[0003] Existing solutions also have shortcomings in process supervision: manual annotation of inference steps is costly, limiting the scale of supervision signals; supervision mechanisms trained on specific tasks have limited generalization ability and are difficult to transfer to new domains; online evaluation methods significantly increase computational overhead and reduce training efficiency. Current mainstream methods all struggle to achieve an ideal balance between exploration efficiency, computational cost, and supervision quality, thus limiting the performance improvement of LLM in complex inference tasks.

[0004] In summary, optimizing the training effect and inference ability of the model is a problem that needs to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for training a question-answering task processing model, which can optimize the training effect and reasoning ability of the model. The specific solution is as follows:

[0006] Firstly, this application discloses a method for training a question-answering task processing model, including:

[0007] Several key structural elements are extracted from the training sample questions, and the first prompt obtained by combining each key structural element with the training sample questions is input into the current question answering task processing model to generate the corresponding initial reasoning path using the current question answering task processing model.

[0008] Each initial inference path is used as the root node to construct the corresponding initial inference tree. Based on the information entropy of each word position in the initial inference path, the first branch node is determined in the initial inference tree. Then, branch inference paths are generated at the first branch node to obtain the extended inference tree.

[0009] Calculate the node value of each node in the extended inference tree, and determine the global advantage, local advantage and path residual of the target node in turn based on the node value of each target node, the node value of the corresponding root node, the node value of the parent node and the average node value of all ancestor nodes in the inference path.

[0010] The current question-answering task processing model is trained using the reward value determined based on the global advantage, the local advantage, and the path residual to obtain the updated question-answering task processing model;

[0011] The updated question-answering task processing model is used as the current question-answering task processing model. Then, the process jumps back to the step of extracting several key structural elements from the training sample questions until the preset iteration stopping condition is met, and the target question-answering task processing model is obtained. The target question-answering task processing model is then used to process the obtained question-answering tasks.

[0012] Optionally, determining the first branch node in the initial inference tree based on the information entropy of each word position in the initial inference path includes:

[0013] The first information entropy of each word position in the initial inference path is calculated using a preset enhanced information entropy expression; wherein, the enhanced information entropy expression is constructed based on the original information entropy expression and the first variance of the candidate word distribution, and the candidate word distribution is a discrete probability distribution of several candidate word values ​​for each word position;

[0014] Several target word positions are selected in descending order of the first information entropy, and the first branch node is determined in the initial inference tree based on the target word positions.

[0015] Optionally, determining the first branch node in the initial inference tree based on the target lexical position includes:

[0016] Based on the preset perturbation model, a number of perturbation question-answering task processing models are derived from the current question-answering task processing model, and the second information entropy of each word position in the initial inference path is calculated using the enhanced information entropy expression under each perturbation question-answering task processing model.

[0017] Calculate the second variance of each of the second information entropies, and determine a first preset number of first branch nodes in the initial inference tree based on the second variance and the target word position.

[0018] Optionally, generating branch inference paths at the first fork node to obtain an expanded inference tree includes:

[0019] The location of the first branch node and the training sample question are combined to obtain a second prompt, and the second prompt is input into the current question-answering task processing model to generate a second preset number of branch reasoning paths to obtain the current reasoning tree.

[0020] In the current inference tree, a second branch node is determined in the current inference tree based on the information entropy of each word position in each branch inference path, and a branch inference path is generated at the second branch node to obtain a new current inference tree. Then, the step of determining the second branch node in the current inference tree based on the information entropy of each word position in each branch inference path is repeated based on a preset number of iterations to obtain an extended inference tree; wherein, each branch inference path is a child node of the corresponding branch node.

[0021] Optionally, the node value is the probability of successful answer reasoning for all child nodes corresponding to any given node;

[0022] Accordingly, the process of determining the global advantage, local advantage, and path residual of each target node based on its node value, the node value of the corresponding root node, the node value of its parent node, and the average node value of all ancestor nodes in its inference path includes:

[0023] The node value of each target node is subtracted from the node value of the corresponding root node to obtain the global advantage, the node value of each target node is subtracted from the node value of the corresponding parent node to obtain the local advantage, and the node value of each target node is subtracted from the average node value of all ancestor nodes in the inference path to obtain the path residual.

[0024] Optionally, training the current question-answering task processing model using the reward value determined based on the global advantage, the local advantage, and the path residual to obtain an updated question-answering task processing model includes:

[0025] The initial reward value for each target node is determined based on the global advantage, the local advantage, the path residual, and the total number of child nodes corresponding to the target node.

[0026] The gradient norm of each target node is calculated using the current question-answering task processing model, and the initial reward value is enhanced using the gradient norm to obtain the target reward value.

[0027] The current question-answering task processing model is trained based on the target reward value and stability weight corresponding to each target node to obtain the updated question-answering task processing model.

[0028] Optionally, the question-answering task processing model training method further includes:

[0029] Obtain the historical reward values ​​of the target node in several historical training rounds, and calculate the reward value variance based on each of the historical reward values;

[0030] The stability weight is determined based on the variance of the reward value and a preset scaling factor.

[0031] Secondly, this application discloses a training device for a question-answering task processing model, comprising:

[0032] The reasoning path generation module is used to extract several key structural elements from the training sample questions, and input the first prompt obtained by combining each key structural element with the training sample questions into the current question answering task processing model, so as to generate the corresponding initial reasoning path using the current question answering task processing model.

[0033] The inference tree generation module is used to construct a corresponding initial inference tree by taking each initial inference path as the root node, and to determine the first branch node in the initial inference tree based on the information entropy of each word position in the initial inference path, and then to generate branch inference paths at the first branch node to obtain an extended inference tree.

[0034] The node value processing module is used to calculate the node value of each node in the extended inference tree, and to determine the global advantage, local advantage and path residual of the target node in sequence based on the node value of each target node, the node value of the corresponding root node, the node value of the parent node and the average node value of all ancestor nodes in the inference path.

[0035] The training module is used to train the current question-answering task processing model using the reward value determined based on the global advantage, the local advantage, and the path residual to obtain an updated question-answering task processing model.

[0036] The iteration module is used to take the updated question-answering task processing model as the current question-answering task processing model, and then jump back to the step of extracting several key structural elements from the training sample questions until the preset iteration stop condition is met, so as to obtain the target question-answering task processing model, and use the target question-answering task processing model to process the obtained question-answering tasks.

[0037] Thirdly, this application discloses an electronic device, including:

[0038] Memory, used to store computer programs;

[0039] A processor is used to execute the computer program to implement the steps of the aforementioned disclosed question-answering task processing model training method.

[0040] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed question-answering task processing model training method.

[0041] As can be seen, this application extracts several key structural elements from the training sample questions, and inputs the first prompt obtained by combining each key structural element with the training sample questions into the current question-answering task processing model to generate a corresponding initial inference path using the current question-answering task processing model; each initial inference path is used as a root node to construct a corresponding initial inference tree, and a first branch node is determined in the initial inference tree based on the information entropy of each word position in the initial inference path, and then a branch inference path is generated at the first branch node to obtain an expanded inference tree; the node value of each node in the expanded inference tree is calculated, and the node value of each target node is compared with the node value of the corresponding root node. The global advantage, local advantage, and path residual of the target node are determined sequentially by the value of the target node, the node value of the parent node, and the average node value of all ancestor nodes in the inference path. The current question-answering task processing model is trained using the reward value determined based on the global advantage, the local advantage, and the path residual to obtain an updated question-answering task processing model. The updated question-answering task processing model is used as the current question-answering task processing model, and then the process jumps back to the step of extracting several key structural elements from the training sample questions until a preset iteration stopping condition is met, thus obtaining the target question-answering task processing model, which is then used to process the obtained question-answering tasks.

[0042] Beneficial Effects: This application extracts key structural elements from training samples and guides the current question-answering task processing model to generate multiple targeted initial inference paths based on these elements. This allows the model to understand and process the input question from different perspectives, rather than providing only a fixed answer. This avoids the limitations of traditional independent sampling from a single perspective and ensures that the inference direction covers the core structure of the training sample question, thus improving the coverage and efficiency of subsequent tree search. Furthermore, this application uses each initial inference path as the root node to construct the corresponding initial inference tree, and determines the branching nodes and generates branch inference paths based on the information entropy of the word position. The purpose is to automatically focus on inference steps with high uncertainty, allowing the inference tree to expand in the direction most needed for exploration and enriching the structure of the search tree, which is beneficial for selecting better paths for supervised training. After obtaining the expanded inference tree, the global advantage, local advantage, and path residual derived from the node value are used to measure the quality improvement contribution of a node relative to its root node, parent node, and the entire inference path at different granularities. Finally, the reward value determined based on global advantage, local advantage, and path residual is used to train the current question-answering task processing model to obtain an updated question-answering task processing model. Finally, through iterative training, a target question-answering task processing model is obtained, which is then used to process the acquired question-answering tasks. In other words, the reward value in this application is directly calculated based on the structural features of the inference tree, without relying on additional manual annotation or external reward models, thus avoiding reward hacking and distribution shift problems, while reducing manual costs and computational overhead. The scheme in this application outperforms traditional methods in terms of inference coverage completeness, learning efficiency, training stability, and generalization ability, effectively optimizing the model's training effect and inference capability. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0044] Figure 1 This application discloses a flowchart of a question-answering task processing model training method.

[0045] Figure 2 This is a schematic diagram of a training device for a question-answering task processing model disclosed in this application.

[0046] Figure 3 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0048] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] See Figure 1 As shown in the embodiment of this application, a method for training a question-answering task processing model is disclosed, the method comprising:

[0050] Step S11: Extract several key structural elements from the training sample questions, and input the first prompt obtained by combining each key structural element with the training sample questions into the current question-answering task processing model, so as to generate the corresponding initial reasoning path using the current question-answering task processing model.

[0051] In this embodiment, several key structural elements are first extracted from the training sample questions. These key structural elements include definitions, functions, operators, variables, or computational objectives in the question. By performing syntactic and semantic analysis on the input training sample questions, representative keywords or fragments are identified as guiding points for the subsequent model to generate the initial inference path. Specifically, the current question-answering task processing model is built based on a Large Language Model (LLM).

[0052] For example, consider the problem: "Let f(x) = 3x" 2 +2x+1;Compute f(5)-f(2)”, we can extract the following key structural elements: “f(x)=3x 2 "+2x+1" is the function definition, "f(5)" and "f(2)" are function calls, and "Compute" is the action trigger word. These key structural elements will be used to better guide the current question-answering task processing model to generate targeted initial inference paths, so that the model can understand and process the input question from different perspectives, rather than just giving a fixed answer. This avoids the limitations of traditional independent sampling from a single perspective and ensures that the inference direction covers the core structure of the training sample question, which helps to improve the coverage and efficiency of subsequent tree search.

[0053] Specifically, it iterates through each extracted key structural element, such as "f(x) = 3x". 2+2x+1, "Compute", "f(5)", "f(2)", etc. For each key structural element, it is used as a guiding signal to be input into the current question answering task processing model, and combined with the original training sample question to form the first prompt, so that the model generates different reasoning starting content. For example, with "f(x)=..." as the clue, the model may generate "Define f(x)...", while with "f(5)" as the clue, the model may continue to reason out "Evaluate f(5)=86".

[0054] This guidance method helps the model reason from multiple semantic perspectives, ensuring that the generated paths cover different components of the problem. Ultimately, multiple initial reasoning paths will be obtained, each starting with a different structure, laying the foundation for subsequent tree-like expansion.

[0055] Step S12: Use each of the initial inference paths as the root node to construct the corresponding initial inference tree, and determine the first branch node in the initial inference tree based on the information entropy of each word position in the initial inference path. Then, generate branch inference paths at the first branch node to obtain the extended inference tree.

[0056] In this embodiment, after generating multiple structure-guided initial inference paths, a root node of a tree needs to be created for each initial inference path. This will serve as the basis for subsequent tree search expansion. Each initial inference path is treated as an independent starting point of thought, thereby initializing a tree structure. Each tree has only one root node, containing the content of the initial inference path, and multiple such initial inference trees are constructed in parallel.

[0057] For example, when dealing with the problem “Compute f(5)-f(2)”, the following initial reasoning path is obtained through structural analysis and guided generation:

[0058] Define f(x) = 3x 2 +2x+1;

[0059] “Evaluate f(5)=86;”

[0060] “Evaluate f(2)=21;”

[0061] Initialize them as the root nodes of the three trees respectively:

[0062] Tree 1 is rooted at "Define f(x)=...";

[0063] Tree 2 is rooted at “Evaluate f(5)=...”;

[0064] Tree 3 takes "Evaluate f(2)=..." as its root;

[0065] These initial reasoning trees form parallel starting points in the reasoning process, which helps improve the coverage and efficiency of subsequent tree search.

[0066] Further, after constructing the initial reasoning tree, a first branching node is determined in the initial reasoning tree based on the information entropy of each token position (i.e., Token) in the initial reasoning path, so that branch reasoning paths are generated at the first branching node to obtain an extended reasoning tree. The purpose is to automatically focus on reasoning steps with high uncertainty, make the reasoning tree expand in the direction that most needs exploration, enrich the structure of the search tree, and facilitate the subsequent selection of better paths for supervised training. Herein, Token refers to the smallest discrete unit when a language model generates text, which usually corresponds to words, symbols, etc.

[0067] It can be understood that, in this embodiment, it is necessary to calculate information entropy for each token position in the initial reasoning path, so as to select a branching node in the initial reasoning tree according to the information entropy. In a specific embodiment, said determining the first branching node in the initial reasoning tree based on the information entropy of each token position in the initial reasoning path comprises: calculating a first information entropy of each token position in the initial reasoning path by using a preset augmented information entropy expression; wherein the augmented information entropy expression is constructed based on an original information entropy expression and a first variance of candidate token distribution, and the candidate token distribution is a discrete probability distribution of a plurality of candidate token values at each token position; screening a plurality of target token positions in descending order of the first information entropy, and determining a first branching node in the initial reasoning tree based on the target token positions.

[0068] It should be noted that before formally expanding the tree structure, it is first necessary to identify which positions in the reasoning path have high uncertainty to be used as potential branching nodes. To this end, this embodiment designs an augmented information entropy calculation method, specifically, based on the original information entropy expression, the distribution of the top K candidate tokens (i.e., candidate token distribution) and the distribution variance are introduced as a regular term, and the augmented information entropy expression is as follows:

[0069] ;

[0070] wherein, represents the -th token position in the current reasoning path; represents the k-th candidate token, which comes from the top-K prediction results of the output distribution at the position; x is the original training sample question; y<t represents the context path before the current token, that is, the model generates the existing text sequence referenced at the time; represents a model the conditional probability of the k-th candidate token based on input x and context y<t. represents the variance of the probability values of the top-K candidate tokens, which is used to measure whether the distribution is "flat" or "fuzzy". is a smoothing factor, is a weight coefficient used to control the contribution of the variance regularization term.

[0071] For example, in "Evaluate f(5)=___", if the model is uncertain about the numerical value that should be filled in after "=", the probabilities of multiple candidate tokens such as "86", "75", and "None" are close. The Top-3 candidate Tokens and their probabilities are: "86" corresponds to 0.4, "None" corresponds to 0.35, and "75" corresponds to 0.25, that is, the distribution of the first 3 candidate tokens is [0.4, 0.35, 0.25], and then the corresponding variance is calculated based on the candidate token distribution. In this example, since the probabilities of multiple candidate tokens are close, it indicates that the uncertainty of the position after "=" is high, so this position will have a higher value and thus becomes a preferentially considered branching node.

[0072] In this embodiment, a plurality of target token positions are screened out in descending order of first information entropy, and a first branching node is determined in the initial inference tree based on the target token positions.

[0073] Further, determining the first branching node in the initial inference tree based on the target token positions includes: deriving a certain number of perturbed question answering task processing models from the current question answering task processing model based on a preset perturbation model, and calculating a second information entropy of each token position in the initial inference path by using the enhanced information entropy expression under each of the perturbed question answering task processing models; calculating a second variance of each of the second information entropies, and determining a first preset number of first branching nodes in the initial inference tree based on the second variance and the target token positions.

[0074] It can be understood that although information entropy can measure the prediction uncertainty of a model at a certain position, in actual inference, some high information entropy positions are not necessarily worthy of branching, for example, common stop words or positions that have no substantial influence on the following text. Therefore, in order to further improve the accuracy of branching node selection, the present application also introduces a "multi-view consistency filtering mechanism". The core idea of this mechanism is: if a position is considered "uncertain" under multiple model views, it is more likely to be a real decision branching point.

[0075] In a specific implementation, the current question-answering task processing model is based on a preset perturbation model. Several perturbation question-answering task processing models are derived. , ... It should be noted that the preset perturbation model can be constructed by rewriting dropout, temperature perturbation, or prompt. Then, under different versions of the perturbation question answering task processing model, each token position in the same path is processed. Calculate its second information entropy Furthermore, the entropy value at that position is statistically analyzed under multiple perspectives. Further, for the same token position, the second variance of its corresponding second information entropy is calculated, thereby determining a first preset number (Top-N) of first branch nodes in the initial inference tree based on the second variance and the target word position.

[0076] The calculation process for the second variance is as follows:

[0077] ;

[0078] in, Indicates the first inference path Each token location; This is the perspective of the processing model for the j-th perturbation question-answering task; For the model right The information entropy value is the prediction uncertainty at that location; Var[] represents the variance of the information entropy value across multiple perspectives. This indicates the consistency of the uncertainty at this location across different model perspectives.

[0079] If a token position appears "uncertain" from all model perspectives—meaning it has a high information entropy and a large second variance—it indicates that this position represents a generally divergent judgment from the model's perspective and is worth selecting as a branching point in the tree. Finally, those with high information entropy... High value and It also holds a high token position, serving as the true first fork node.

[0080] For example, in the path "Evaluate f(5)=___", the position after "=" is... , , Obvious candidate fluctuations occur from all perspectives, such as candidates "None", "86", and "f(2)", with large differences in probability distribution. In this case, the entropy variance at this position is high, and the position is selected into the branch point set; on the contrary, if a position has high entropy but is consistent across multiple perspectives, for example, the same token frequently appears, it may be an invalid high-entropy point and no branching is performed.

[0081] Further, the step of generating branch inference paths at the first branching node to obtain an extended inference tree comprises: combining the position of the first branching node with the training sample question to obtain a second prompt, inputting the second prompt into the current question-answering task processing model, so that the current question-answering task processing model generates a second preset number of branch inference paths to obtain a current inference tree; in the current inference tree, determining a second branching node in the current inference tree based on the information entropy of each token position in each branch inference path, generating a branch inference path at the second branching node to obtain a new current inference tree, and then repeating the step of determining the second branching node in the current inference tree based on the information entropy of each token position in each branch inference path according to a preset number of iteration rounds to obtain the extended inference tree; wherein each branch inference path is a child node of the corresponding branching node.

[0082] That is, after determining branching nodes with high uncertainty and multi-perspective consistency, the next step is to perform path expansion for these positions to generate new branch inference paths, thereby enriching the structure of the search tree. In the present application, the position of the first branching node is combined with the training sample question to obtain the second prompt, and the second prompt is input into the current question-answering task processing model, so that the current question-answering task processing model generates the second preset number of branch inference paths to obtain the current inference tree. Specifically, for each tree path, at the branching node position t selected for the tree path, the prefix y<t of the tree path is intercepted as a new context input, which is combined with the original training sample question to obtain the second prompt, and then the second prompt is processed by the current question-answering task processing model with the context as a condition, a plurality of possible continuation paths are sampled. Generally, T different branch inference paths are generated for each branching node to simulate multiple inference trends at the position, which is expressed by the following formula:

[0083] ;

[0084] wherein x is the original input training sample question; y<t is the part before the branched position in the current path; represents T new paths generated starting from this position; is the current question-answering task processing model, which is used for sampling output.

[0085] These newly generated branch inference paths will become child nodes of the branch node and be attached to the tree structure of the original path, forming new branches. As more branch nodes are added, the branches of the entire tree will become increasingly rich, which is beneficial for selecting better paths for supervised training later.

[0086] For example, in the path "Evaluate f(5)=___", the model is uncertain about the content after "=", so it uses "Evaluate f(5)=" as a prefix, starting from... The following three candidate paths were sampled:

[0087] “Evaluate f(5) = 75.”

[0088] “Evaluate f(5) = f(2) + 10.”

[0089] “Evaluate f(5) = None.”

[0090] These three paths will be added to the tree structure as the three child nodes of the "=" branch node.

[0091] It should also be noted that in order to build a well-structured and diverse reasoning tree, a single branching is usually not enough. Therefore, this embodiment also introduces an iterative mechanism, which allows the depth and breadth of the tree to be gradually expanded in multiple rounds of tree construction until the preset structural complexity is met.

[0092] Specifically, in each iteration, the information entropy of all unexpanded leaf nodes in the current paths is re-evaluated to filter new branch nodes and further generate branch inference paths. In this way, the tree structure expands layer by layer in multiple directions, simulating the evolution of the model on different policy branches. The operation flow for each iteration is as follows:

[0093] 1. Traverse all leaf nodes in the current tree;

[0094] 2. Perform high-entropy position identification for each leaf node;

[0095] 3. Perform multi-view consistency filtering on these high-entropy points;

[0096] 4. Perform path expansion at the selected branching points;

[0097] 5. Add the newly generated path as a child node of this branch node.

[0098] Assuming M is the number of initial inference trees; N is the number of high-entropy branch nodes selected in each path; T is the number of branch inference paths expanded by each branch node; and L is the total number of tree construction rounds, i.e. the number of iteration rounds; then in each round, each leaf node will expand into N×T new nodes, and the tree structure will grow exponentially.

[0099] Use the following formula to estimate the total number of nodes:

[0100] Total number of nodes = M·(1+(N·T)) 1 +(N·T) 2 +···+(N·T) L );

[0101] Assumptions: M=3, there are 3 initial paths; N=2: each path selects 2 branching points; T=3: each branching point generates 3 new paths; L=2: the tree construction depth is 2 rounds;

[0102] The total number of nodes is calculated to be 3 * (1 + 6). 1 +6 2 ) = 3·(1+6+36) = 3·43 = 129.

[0103] Step S13: Calculate the node value of each node in the extended inference tree, and determine the global advantage, local advantage and path residual of the target node in sequence based on the node value of each target node, the node value of the corresponding root node, the node value of the parent node and the average node value of all ancestor nodes in the inference path.

[0104] In this embodiment, after the extended inference tree is constructed, it is not only necessary to judge the quality of the path based on the final result, but also to deeply analyze the actual contribution of each intermediate step to the success of the final inference. Therefore, this embodiment needs to assign a reasonable and stable set of process-level reward signals to each node in the tree (i.e., each step in the model generation process) as the basis for subsequent reinforcement learning training.

[0105] Specifically, after constructing the extended inference tree, the node value of each node in the extended inference tree is further calculated. Based on the node value of each target node, the node value of its corresponding root node, the node value of its parent node, and the average node value of all ancestor nodes in its inference path, the global advantage, local advantage, and path residual of the target node are determined sequentially. The global advantage, local advantage, and path residual derived from the node value are used to measure the quality improvement contribution of a node relative to its root node, parent node, and the entire inference path at different granularities.

[0106] It should be noted that the node value is the probability of successful answer reasoning for all child nodes corresponding to any given node; correspondingly, the process of determining the global advantage, local advantage, and path residual of the target node based on the node value of each target node, the node value of the corresponding root node, the node value of the parent node, and the average node value of all ancestor nodes in the reasoning path, in sequence, includes: subtracting the node value of each target node from the node value of the corresponding root node to obtain the global advantage, subtracting the node value of each target node from the node value of the corresponding parent node to obtain the local advantage, and subtracting the node value of each target node from the average node value of all ancestor nodes in the reasoning path to obtain the path residual.

[0107] Understandably, the first step is to "score" each intermediate node in the expanded reasoning tree, which means estimating its overall effectiveness. The core idea is that the value of an intermediate node is equal to the proportion of correct inferences in the sub-paths it leads to. In other words, if most of the leaf nodes under a certain node can lead to the correct answer, then that node is a good intermediate decision and deserves positive incentives.

[0108] The node value is defined as follows:

[0109] ;

[0110] Among them, s n This is the current intermediate node; It is the set of all leaf nodes that can be reached from this node; The value is 1 if the leaf node l yields the correct conclusion, and 0 otherwise. This represents the average reasoning success rate of the node throughout the entire tree structure.

[0111] Understandably, the earlier a node makes the correct directional decision, the more correct subsequent paths it will have, and the higher its value. Conversely, if most of a node's subsequent paths fail, it indicates that it was an inefficient or even incorrect intermediate choice and should be given negative feedback. For example, if the path "f(5)=..." branches into 6 paths, and 4 of them eventually yield the correct answer, such as calculating f(5)=86, then the node value of this node is: V=4 / 6=0.666.

[0112] After estimating the value of each node, a multi-layered reward signal is constructed to more accurately reflect the role of each step in the entire inference path. Three advantage metrics are designed: global advantage, local advantage, and path residual, which measure the quality improvement contribution of a node relative to its parent node, root node, and the entire path at different granularities.

[0113] Global advantage is used to evaluate a node. Whether the root node of the entire tree is "better" can be expressed as follows:

[0114] ;

[0115] in, For the overall advantage, The value of the root node.

[0116] like A value greater than 0 indicates that the node's path is better than the initial strategy, and a positive incentive is given; if... A negative value indicates that the overall path is poor, and a suppression signal is given.

[0117] Local advantage measures the value increase between a node and its direct parent node, and its expression is:

[0118] ;

[0119] in, For the overall advantage, The value of the parent node.

[0120] This indicator is used to track the effect of "local decision improvement," that is, whether a step has a positive effect on the basis of the previous step.

[0121] Path residuals measure the deviation of a node's value from the average value of its ancestor nodes along the entire path, capturing abrupt changes or jumps in the overall path trend. Its expression is:

[0122] ;

[0123] in, For path residuals, For nodes The set of all ancestor nodes; This indicates whether the node broke the value stability in the path, jumping to a new region of superiority or inferiority. This residual helps to discover whether a step suddenly made a critical change, or whether a decision caused a "reversal" of the overall direction of the path.

[0124] Suppose that in a certain path, the value of the root node is V(root) = 0.3; and a certain intermediate node... The value of the parent node is =0.5, its own value is =0.7; its ancestor nodes (including parent nodes, grandparent nodes, etc.) have an average value of 0.45.

[0125] Therefore, the global advantage (GA) is 0.7 - 0.3 = 0.4; the local advantage (LA) is 0.7 - 0.5 = 0.2; and the path residual is: =0.7-0.45=0.25; these values ​​will be used as components of the subsequent reward signal fusion.

[0126] Step S14: Train the current question-answering task processing model using the reward value determined based on the global advantage, the local advantage, and the path residual to obtain the updated question-answering task processing model.

[0127] In this embodiment, after estimating the value of each node and calculating the three types of advantages, these factors are organically integrated to construct an initial reward value for each intermediate node that can be used for reinforcement learning training within the entire tree structure. This reward value not only reflects the independent contribution of a certain step of reasoning but also takes into account its comprehensive impact on the global path and local evolution.

[0128] In a specific implementation, training the current question-answering task processing model using the reward value determined based on the global advantage, the local advantage, and the path residual to obtain an updated question-answering task processing model includes: determining an initial reward value for each target node based on the global advantage, the local advantage, the path residual, and the total number of child nodes corresponding to the target node; calculating the gradient norm of each target node using the current question-answering task processing model to enhance the initial reward value using the gradient norm to obtain a target reward value; and training the current question-answering task processing model based on the target reward value and stability weights corresponding to each target node to obtain an updated question-answering task processing model.

[0129] The initial reward value is in the following form:

[0130] ;

[0131] in, This is the initial reward value; The number of leaf nodes under this node is used as a regularization term to control the impact of subtree size. For overall advantage; For local advantages; For path residuals; For path gain; and These are adjustable weighting coefficients that control the contribution ratio of different items.

[0132] For example, assuming global advantage GA = 0.4, local advantage LA = 0.2, and path residual... =0.25, number of leaf nodes |L|=9, let =0.5, ignoring the path gain term ( If =0), then the final reward is:

[0133] R=19·(0.4+0.2+0.5·0.25)=13·(0.4+0.2+0.125)=13·0.725≈0.241.

[0134] This reward will be used for the current question-answering task processing model. This guides the model to learn better generation paths.

[0135] Furthermore, in reinforcement learning training, in addition to focusing on the stability of samples, it is also necessary to consider the "update contribution" that a sample actually generates during training. Some nodes may have high rewards, but their effect on updating model parameters is small; while other nodes may have moderate rewards, but they trigger large gradient changes during training, significantly affecting policy adjustment. To capture this "update capability," this embodiment designs a gradient-dominated reward enhancement mechanism, that is, introducing an enhancement term related to the gradient norm of the sample into the final reward calculation, in order to identify and enhance those samples that have a greater influence on policy learning.

[0136] Specifically, the gradient norm of each target node needs to be calculated using the current question-answering task processing model. This gradient norm is then used to enhance the initial reward value to obtain the target reward value. The expression for the target reward value is as follows:

[0137] ;

[0138] in, The enhanced target reward value is used to participate in the gradient training of the model; This is the initial reward value; This is the current question-answering task processing model; For the sample The gradient of the logarithmic probability with respect to the parameters; The square norm of the gradient; The coefficient that controls the contribution of gradient enhancement.

[0139] Suppose that the original rewards of two nodes s1 and s2 are both 0.4, but their gradient norms are respectively:

[0140] =0.1;

[0141] =0.5;

[0142] Then, with β=1.0, we have: =0.4 + 0.01 = 0.41; =0.4+0.25=0.65; This indicates that s2 has a greater driving effect on training and therefore will obtain a higher learning priority.

[0143] Next, the current question-answering task processing model is trained based on the target reward value and stability weight corresponding to each target node to obtain the updated question-answering task processing model.

[0144] The method further includes: obtaining the historical reward values ​​of the target node in several historical training rounds, and calculating the reward value variance based on each historical reward value; and determining the stability weight based on the reward value variance and a preset scaling factor.

[0145] It should be noted that this embodiment introduces a sample temporal stability index to dynamically evaluate the degree of reward fluctuation of a node across multiple training cycles. If the reward value of a sample varies greatly in different training rounds, it indicates that its training signal is unstable, and its contribution to gradient updates should be reduced. The stability weights of the nodes are calculated as follows:

[0146] ;

[0147] in, For nodes The stability weights have values ​​ranging from (0, 1]. This represents the reward for this node in the t-th round of training. The variance of reward values ​​across rounds; This is the scaling factor, which controls the sensitivity to volatility.

[0148] This stability weight will be used as an important coefficient in the objective function of the training model, automatically attenuating the training impact of highly volatile samples and protecting the training process from being dominated by unstable samples.

[0149] For example, suppose a node's reward over the past 5 rounds is [0.8, 0.3, 0.9, 0.4, 0.85], which clearly shows a large fluctuation in reward; its variance is approximately 0.07; if α=10, then: =exp(-10·0.07)≈exp(-0.7)≈0.496; This indicates that the stability of this sample is poor, and its gradient will be multiplied by a coefficient less than 0.5, weakening its update effect.

[0150] After completing the sample stability weights and gradient-dominated enhancements, the policy gradient method in reinforcement learning can be used to update the language model policy based on these reward values, thereby improving its performance in complex reasoning tasks.

[0151] The objective function used is the maximized expectation with sample weighting, as follows:

[0152] ;

[0153] in, These are the parameters of the current question-answering task processing model; For sample stability weights; The target reward value after gradient enhancement; For the model to sample path The logarithmic probability of generation; D is the training sample set collected by the search tree; The goal is to minimize this loss function, that is, to maximize the expected weighted log probability.

[0154] Step S15: Use the updated question-answering task processing model as the current question-answering task processing model, and then jump back to the step of extracting several key structural elements from the training sample questions until the preset iteration stopping condition is met, and obtain the target question-answering task processing model, so as to process the obtained question-answering task using the target question-answering task processing model.

[0155] In this embodiment, the model is continuously optimized through multi-round iterative training, which gradually improves the inference quality. This allows the model to receive real-time feedback on the inference path generated in each round, making the inference behavior in the next round more accurate and the branch structure more in line with the inference logic. Finally, a well-trained target question-answering task processing model is obtained, which is then used to process the acquired question-answering tasks.

[0156] As can be seen, this application extracts key structural elements from training samples and guides the current question-answering task processing model to generate multiple targeted initial inference paths based on these key structural elements. This allows the model to understand and process the input question from different perspectives, rather than just providing a fixed answer. This avoids the limitations of traditional independent sampling from a single perspective and ensures that the inference direction covers the core structure of the training sample question, which helps improve the coverage and efficiency of subsequent tree search. Furthermore, this application uses each initial inference path as the root node to construct the corresponding initial inference tree, and determines the branching nodes and generates branch inference paths based on the information entropy of the word position. The purpose is to automatically focus on the inference steps with high uncertainty, expand the inference tree in the direction that needs to be explored most, and enrich the structure of the search tree, which is beneficial for selecting a better path for supervised training. After obtaining the expanded inference tree, the global advantage, local advantage, and path residual derived from the node value are used to measure the quality improvement contribution of a node relative to its root node, parent node, and the entire inference path at different granularities. Finally, the reward value determined based on global advantage, local advantage, and path residual is used to train the current question-answering task processing model to obtain an updated question-answering task processing model. Finally, through iterative training, a target question-answering task processing model is obtained, which is then used to process the acquired question-answering tasks. In other words, the reward value in this application is directly calculated based on the structural features of the inference tree, without relying on additional manual annotation or external reward models, thus avoiding reward hacking and distribution shift problems, while reducing manual costs and computational overhead. The scheme in this application outperforms traditional methods in terms of inference coverage completeness, learning efficiency, training stability, and generalization ability, effectively optimizing the model's training effect and inference capability.

[0157] See Figure 2 As shown in the figure, this application discloses a question-answering task processing model training device, which includes:

[0158] The reasoning path generation module 11 is used to extract several key structural elements from the training sample questions, and input the first prompt obtained by combining each key structural element with the training sample questions into the current question answering task processing model, so as to generate the corresponding initial reasoning path using the current question answering task processing model.

[0159] The reasoning tree generation module 12 is used to construct a corresponding initial reasoning tree by taking each initial reasoning path as the root node, and to determine a first branch node in the initial reasoning tree based on the information entropy of each word position in the initial reasoning path, and then to generate a branch reasoning path at the first branch node to obtain an extended reasoning tree.

[0160] The node value processing module 13 is used to calculate the node value of each node in the extended inference tree, and to determine the global advantage, local advantage and path residual of the target node in sequence based on the node value of each target node, the node value of the corresponding root node, the node value of the parent node and the average node value of all ancestor nodes in the inference path.

[0161] Training module 14 is used to train the current question answering task processing model using the reward value determined based on the global advantage, the local advantage and the path residual to obtain the updated question answering task processing model;

[0162] The iteration module 15 is used to take the updated question-answering task processing model as the current question-answering task processing model, and then jump back to the step of extracting several key structural elements from the training sample questions until the preset iteration stop condition is met, so as to obtain the target question-answering task processing model, and use the target question-answering task processing model to process the obtained question-answering task.

[0163] As can be seen, this application extracts key structural elements from training samples and guides the current question-answering task processing model to generate multiple targeted initial inference paths based on these key structural elements. This allows the model to understand and process the input question from different perspectives, rather than just providing a fixed answer. This avoids the limitations of traditional independent sampling from a single perspective and ensures that the inference direction covers the core structure of the training sample question, which helps improve the coverage and efficiency of subsequent tree search. Furthermore, this application uses each initial inference path as the root node to construct the corresponding initial inference tree, and determines the branching nodes and generates branch inference paths based on the information entropy of the word position. The purpose is to automatically focus on the inference steps with high uncertainty, expand the inference tree in the direction that needs to be explored most, and enrich the structure of the search tree, which is beneficial for selecting a better path for supervised training. After obtaining the expanded inference tree, the global advantage, local advantage, and path residual derived from the node value are used to measure the quality improvement contribution of a node relative to its root node, parent node, and the entire inference path at different granularities. Finally, the reward value determined based on global advantage, local advantage, and path residual is used to train the current question-answering task processing model to obtain an updated question-answering task processing model. Finally, through iterative training, a target question-answering task processing model is obtained, which is then used to process the acquired question-answering tasks. In other words, the reward value in this application is directly calculated based on the structural features of the inference tree, without relying on additional manual annotation or external reward models, thus avoiding reward hacking and distribution shift problems, while reducing manual costs and computational overhead. The scheme in this application outperforms traditional methods in terms of inference coverage completeness, learning efficiency, training stability, and generalization ability, effectively optimizing the model's training effect and inference capability.

[0164] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the question-answering task processing model training method performed by the electronic device disclosed in any of the foregoing embodiments.

[0165] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0166] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0167] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0168] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system 221 can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the question-answering task processing model training method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0169] Furthermore, embodiments of this application also disclose a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, it implements the question-answering task processing model training method steps disclosed in any of the foregoing embodiments.

[0170] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0171] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0172] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, compact disc read-only memory (CD-ROM), or any other form of storage medium known in the art.

[0173] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0174] The above provides a detailed description of the question-answering task processing model training method, apparatus, device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for training a question-answering task processing model, characterized in that, include: Several key structural elements are extracted from the training sample questions, and the first prompt obtained by combining each key structural element with the training sample questions is input into the current question answering task processing model to generate the corresponding initial reasoning path using the current question answering task processing model. Each initial inference path is used as the root node to construct the corresponding initial inference tree. Based on the information entropy of each word position in the initial inference path, the first branch node is determined in the initial inference tree. Then, branch inference paths are generated at the first branch node to obtain the extended inference tree. Calculate the node value of each node in the extended inference tree, and determine the global advantage, local advantage, and path residual of the target node in turn based on the node value of each target node, the node value of the corresponding root node, the node value of the parent node, and the average node value of all ancestor nodes in the inference path; wherein, the path residual is the difference between the node value of each target node and the average node value of all ancestor nodes in the inference path. The current question-answering task processing model is trained using the reward value determined based on the global advantage, the local advantage, and the path residual to obtain the updated question-answering task processing model; The updated question-answering task processing model is used as the current question-answering task processing model. Then, the process jumps back to the step of extracting several key structural elements from the training sample questions until the preset iteration stopping condition is met, and the target question-answering task processing model is obtained. The target question-answering task processing model is then used to process the obtained question-answering tasks. Wherein, determining the first branch node in the initial inference tree based on the information entropy of each word position in the initial inference path includes: The first information entropy of each word position in the initial inference path is calculated using a preset enhanced information entropy expression; wherein, the enhanced information entropy expression is constructed based on the original information entropy expression and the first variance of the candidate word distribution, and the candidate word distribution is a discrete probability distribution of several candidate word values ​​for each word position; Several target word positions are selected in descending order of the first information entropy, and the first branch node is determined in the initial inference tree based on the target word positions.

2. The question-answering task processing model training method according to claim 1, characterized in that, The step of determining the first branch node in the initial inference tree based on the target word position includes: Based on the preset perturbation model, a number of perturbation question-answering task processing models are derived from the current question-answering task processing model, and the second information entropy of each word position in the initial inference path is calculated using the enhanced information entropy expression under each perturbation question-answering task processing model. Calculate the second variance of each of the second information entropies, and determine a first preset number of first branch nodes in the initial inference tree based on the second variance and the target word position.

3. The question-answering task processing model training method according to claim 1, characterized in that, The step of generating branch inference paths at the first fork node to obtain an expanded inference tree includes: The location of the first branch node and the training sample question are combined to obtain a second prompt, and the second prompt is input into the current question-answering task processing model to generate a second preset number of branch reasoning paths to obtain the current reasoning tree. In the current inference tree, a second branch node is determined in the current inference tree based on the information entropy of each word position in each branch inference path, and a branch inference path is generated at the second branch node to obtain a new current inference tree. Then, the step of determining the second branch node in the current inference tree based on the information entropy of each word position in each branch inference path is repeated based on a preset number of iterations to obtain an extended inference tree; wherein, each branch inference path is a child node of the corresponding branch node.

4. The question-answering task processing model training method according to claim 1, characterized in that, The node value is the probability of successful answer reasoning for all child nodes corresponding to any given node. Accordingly, the process of determining the global advantage, local advantage, and path residual of each target node based on its node value, the node value of the corresponding root node, the node value of its parent node, and the average node value of all ancestor nodes in its inference path includes: The node value of each target node is subtracted from the node value of the corresponding root node to obtain the global advantage, the node value of each target node is subtracted from the node value of the corresponding parent node to obtain the local advantage, and the node value of each target node is subtracted from the average node value of all ancestor nodes in the inference path to obtain the path residual.

5. The question-answering task processing model training method according to any one of claims 1 to 4, characterized in that, The step of training the current question-answering task processing model using a reward value determined based on the global advantage, the local advantage, and the path residual to obtain an updated question-answering task processing model includes: The initial reward value for each target node is determined based on the global advantage, the local advantage, the path residual, and the total number of child nodes corresponding to the target node. The gradient norm of each target node is calculated using the current question-answering task processing model, and the initial reward value is enhanced using the gradient norm to obtain the target reward value. The current question-answering task processing model is trained based on the target reward value and stability weight corresponding to each target node to obtain the updated question-answering task processing model.

6. The question-answering task processing model training method according to claim 5, characterized in that, Also includes: Obtain the historical reward values ​​of the target node in several historical training rounds, and calculate the reward value variance based on each of the historical reward values; The stability weight is determined based on the variance of the reward value and a preset scaling factor.

7. A training device for a question-answering task processing model, characterized in that, include: The reasoning path generation module is used to extract several key structural elements from the training sample questions, and input the first prompt obtained by combining each key structural element with the training sample questions into the current question answering task processing model, so as to generate the corresponding initial reasoning path using the current question answering task processing model. The inference tree generation module is used to construct a corresponding initial inference tree by taking each initial inference path as the root node, and to determine the first branch node in the initial inference tree based on the information entropy of each word position in the initial inference path, and then to generate branch inference paths at the first branch node to obtain an extended inference tree. The node value processing module is used to calculate the node value of each node in the extended inference tree, and to determine the global advantage, local advantage and path residual of the target node in sequence based on the node value of each target node, the node value of the corresponding root node, the node value of the parent node and the average node value of all ancestor nodes in the inference path; wherein, the path residual is the difference between the node value of each target node and the average node value of all ancestor nodes in the inference path. The training module is used to train the current question-answering task processing model using the reward value determined based on the global advantage, the local advantage, and the path residual to obtain an updated question-answering task processing model. The iteration module is used to take the updated question-answering task processing model as the current question-answering task processing model, and then jump back to the step of extracting several key structural elements from the training sample questions until the preset iteration stop condition is met, so as to obtain the target question-answering task processing model, and use the target question-answering task processing model to process the obtained question-answering task. Specifically, the reasoning tree generation module is used to calculate the first information entropy of each word position in the initial reasoning path using a preset enhanced information entropy expression; wherein the enhanced information entropy expression is constructed based on the original information entropy expression and the first variance of the candidate word distribution, and the candidate word distribution is a discrete probability distribution of several candidate word values ​​for each word position; several target word positions are selected according to the first information entropy in descending order, and the first branching node is determined in the initial reasoning tree based on the target word positions.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the question-answering task processing model training method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the question-answering task processing model training method as described in any one of claims 1 to 6.

Images