Problem-solving path determination method, device, equipment, medium and program product

Abstract

The application provides a problem solving path determination method, device, equipment, medium and program product to solve the problem of poor problem solving effect in related technologies. The method comprises: analyzing a target mathematical problem obtained based on a pre-trained difficulty recognition model to determine the difficulty level of the target mathematical problem; and determining the problem solving path of the target mathematical problem based on the target mathematical problem and the difficulty level of the target mathematical problem by using a beam search algorithm. The application can improve the problem solving effect.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, medium, and program product for determining a problem-solving path. Background Technology

[0002] In related technologies, Large Language Models (LLM) and Monte Carlo Tree Search (MCTS) are commonly used to solve mathematical problems. These techniques primarily learn and predict solution steps for mathematical problems by using language models pre-trained on large datasets. These technologies leverage pattern recognition and sequence prediction capabilities to handle a wide range of mathematical problems, from simple arithmetic to complex proofs. However, the MCTS framework suffers from issues when solving mathematical problems: generating complex and lengthy steps for simple problems, or generating too few steps for complex problems, resulting in poor problem-solving performance. Summary of the Invention

[0003] This application provides a method, apparatus, device, medium, and program product for determining a problem-solving path, in order to solve the problem of poor problem-solving performance in related technologies.

[0004] To solve the above-mentioned technical problems, this application is implemented as follows:

[0005] In a first aspect, embodiments of this application provide a method for determining a problem-solving path, including:

[0006] The target mathematical problem is analyzed based on a pre-trained difficulty recognition model to determine the difficulty level of the target mathematical problem.

[0007] Based on the target mathematical problem and its difficulty level, a beam search algorithm is used to determine the solution path for the target mathematical problem.

[0008] Optionally, determining the solution path for the target mathematical problem using a beam search algorithm based on the target mathematical problem and its difficulty level includes:

[0009] An initial search bundle is generated based on the target mathematical problem using a bundle search algorithm;

[0010] Based on the pre-trained policy model and the initial search bundle, multiple candidate solution paths are generated;

[0011] Based on the difficulty level of the target mathematical problem, a penalty function is determined, which is used to determine the penalty value corresponding to the multiple candidate solution paths;

[0012] The multiple candidate solution paths are evaluated based on a pre-trained value model and the penalty function to determine the solution path for the target mathematical problem. The value model is used to score the multiple candidate solution paths, which include the solution path for the target mathematical problem.

[0013] Optionally, before generating multiple candidate solution paths based on the pre-trained policy model and the initial search bundle, the method further includes:

[0014] Using the Monte Carlo tree search algorithm, a solution path is generated for the sample problem, resulting in multiple nodes. These nodes are used to indicate the solution steps for the sample problem.

[0015] The contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​that correspond one-to-one with the multiple nodes. The contribution values ​​are used to characterize the influence of the nodes on generating the correct answer to the sample question.

[0016] The initial value model and the initial strategy model are trained based on the training data to obtain the pre-trained strategy model and the pre-trained value model. The training data includes the problem-solving data and contribution values ​​corresponding to the multiple nodes.

[0017] Optionally, the step of evaluating the contribution of the plurality of nodes to obtain a plurality of contribution values ​​corresponding one-to-one with the plurality of nodes includes:

[0018] The sample problem is analyzed based on the pre-trained difficulty recognition model to determine the difficulty level of the sample problem;

[0019] Based on the difficulty level of the sample problem and the preset function, the contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​corresponding to the multiple nodes.

[0020] Optionally, the Monte Carlo tree search algorithm is used to generate a solution path for the sample problem, resulting in multiple nodes, including:

[0021] Monte Carlo tree search is performed with the sample problem as the root node to obtain multiple nodes. Starting from the root node, the Polynomial Upper Bound Confidence Tree (PUCT) algorithm is used to select nodes, and node expansion is performed based on depth search and breadth search until the stopping condition is met and the termination node is reached. Backtracking is then performed to the root node to obtain the solution path for the sample problem.

[0022] Optionally, before analyzing the acquired target mathematical problem based on the pre-trained difficulty recognition model to determine the difficulty level of the target mathematical problem, the method further includes:

[0023] Obtain a training sample set, which includes multiple mathematical problems and their corresponding standard answers;

[0024] The multiple mathematical problems are analyzed using a preset model to obtain multiple analytical answers that correspond one-to-one with the multiple mathematical problems;

[0025] Based on the multiple analytical answers and multiple target answers, multiple difficulty levels are determined that correspond one-to-one with the multiple mathematical problems;

[0026] The initial difficulty recognition model is trained based on the multiple mathematical problems and their corresponding difficulty levels to obtain the pre-trained difficulty recognition model.

[0027] Secondly, embodiments of this application provide a problem-solving path determination device, comprising:

[0028] The first determining module is used to analyze the acquired target mathematical problem based on a pre-trained difficulty recognition model and determine the difficulty level of the target mathematical problem.

[0029] The second determining module is used to determine the solution path of the target mathematical problem based on the target mathematical problem and the difficulty level of the target mathematical problem, using a beam search algorithm.

[0030] Optionally, the second determining module includes:

[0031] The first generation unit is used to generate an initial search bundle based on the target mathematical problem using a bundle search algorithm.

[0032] The second generation unit is used to generate multiple candidate problem-solving paths based on the pre-trained policy model and the initial search bundle;

[0033] The first determining unit is used to determine a penalty function based on the difficulty level of the target mathematical problem, wherein the penalty function is used to determine the penalty value corresponding to the plurality of candidate solution paths;

[0034] The second determining unit is used to evaluate the plurality of candidate solution paths based on the pre-trained value model and the penalty function, and determine the solution path of the target mathematical problem. The value model is used to score the plurality of candidate solution paths, and the plurality of candidate solution paths include the solution path of the target mathematical problem.

[0035] Optionally, the second determining module further includes:

[0036] The third generation unit is used to generate a solution path for the sample problem using the Monte Carlo tree search algorithm, and obtain multiple nodes, which are used to indicate the solution steps of the sample problem.

[0037] An evaluation unit is used to evaluate the contribution of the plurality of nodes and obtain a plurality of contribution values ​​corresponding one-to-one with the plurality of nodes. The contribution values ​​are used to characterize the influence of the node on generating the correct answer to the sample question.

[0038] The training unit is used to train the initial value model and the initial strategy model based on the training data to obtain the pre-trained strategy model and the pre-trained value model. The training data includes the problem-solving data and contribution values ​​corresponding to the multiple nodes.

[0039] Optionally, the evaluation unit is specifically used for:

[0040] The sample problem is analyzed based on the pre-trained difficulty recognition model to determine the difficulty level of the sample problem;

[0041] Based on the difficulty level of the sample problem and the preset function, the contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​corresponding to the multiple nodes.

[0042] Optionally, the third generation unit is specifically used for:

[0043] Monte Carlo tree search is performed with the sample problem as the root node to obtain multiple nodes. Starting from the root node, the Polynomial Upper Bound Confidence Tree (PUCT) algorithm is used to select nodes, and node expansion is performed based on depth search and breadth search until the stopping condition is met and the termination node is reached. Backtracking is then performed to the root node to obtain the solution path for the sample problem.

[0044] Optionally, the device further includes:

[0045] The acquisition module is used to acquire a training sample set, which includes multiple mathematical problems and their corresponding standard answers.

[0046] The parsing module is used to analyze the multiple mathematical problems using a preset model to obtain multiple analytical answers that correspond one-to-one with the multiple mathematical problems;

[0047] The third determining module is used to determine multiple difficulty levels corresponding one-to-one with the multiple analytical answers and multiple target answers;

[0048] The training module is used to train the initial difficulty recognition model based on the multiple mathematical problems and their corresponding difficulty levels, so as to obtain the pre-trained difficulty recognition model.

[0049] Thirdly, embodiments of this application provide an electronic device, including a transceiver and a processor, wherein the processor is used for:

[0050] The target mathematical problem is analyzed based on a pre-trained difficulty recognition model to determine the difficulty level of the target mathematical problem.

[0051] Based on the target mathematical problem and its difficulty level, a beam search algorithm is used to determine the solution path for the target mathematical problem.

[0052] Optionally, the processor is specifically used for:

[0053] An initial search bundle is generated based on the target mathematical problem using a bundle search algorithm;

[0054] Based on the difficulty level of the target mathematical problem, a penalty function is determined, which is used to determine the penalty value corresponding to the multiple candidate solution paths;

[0055] Based on the pre-trained policy model and the initial search bundle, multiple candidate solution paths are generated;

[0056] The multiple candidate solution paths are evaluated based on a pre-trained value model and the penalty function to determine the solution path for the target mathematical problem. The value model is used to score the multiple candidate solution paths, which include the solution path for the target mathematical problem.

[0057] Optionally, the processor is further configured to:

[0058] Using the Monte Carlo tree search algorithm, a solution path is generated for the sample problem, resulting in multiple nodes. These nodes are used to indicate the solution steps for the sample problem.

[0059] The contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​that correspond one-to-one with the multiple nodes. The contribution values ​​are used to characterize the influence of the nodes on generating the correct answer to the sample question.

[0060] The initial value model and the initial strategy model are trained based on the training data to obtain the pre-trained strategy model and the pre-trained value model. The training data includes the problem-solving data and contribution values ​​corresponding to the multiple nodes.

[0061] Optionally, the processor is specifically used for:

[0062] The sample problem is analyzed based on the pre-trained difficulty recognition model to determine the difficulty level of the sample problem;

[0063] Based on the difficulty level of the sample problem and the preset function, the contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​corresponding to the multiple nodes.

[0064] Optionally, the processor is specifically used for:

[0065] Monte Carlo tree search is performed with the sample problem as the root node to obtain multiple nodes. Starting from the root node, the Polynomial Upper Bound Confidence Tree (PUCT) algorithm is used to select nodes, and node expansion is performed based on depth search and breadth search until the stopping condition is met and the termination node is reached. Backtracking is then performed to the root node to obtain the solution path for the sample problem.

[0066] Optionally, the processor is further configured to:

[0067] Obtain a training sample set, which includes multiple mathematical problems and their corresponding standard answers;

[0068] The multiple mathematical problems are analyzed using a preset model to obtain multiple analytical answers that correspond one-to-one with the multiple mathematical problems;

[0069] Based on the multiple analytical answers and multiple target answers, multiple difficulty levels are determined that correspond one-to-one with the multiple mathematical problems;

[0070] The initial difficulty recognition model is trained based on the multiple mathematical problems and their corresponding difficulty levels to obtain the pre-trained difficulty recognition model.

[0071] Fourthly, embodiments of this application provide an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the steps of the problem-solving path determination method as described in the first aspect above.

[0072] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the problem-solving path determination method described in the first aspect above.

[0073] Sixthly, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the problem-solving path determination method as described in the first aspect above.

[0074] In this embodiment, a pre-trained difficulty recognition model is used to analyze the acquired target mathematical problem and determine its difficulty level. Based on the target mathematical problem and its difficulty level, a beam search algorithm is used to determine the solution path. Thus, by using the beam search algorithm to determine the solution path based on the difficulty level of the target mathematical problem, the beam search algorithm is integrated with the difficulty level classification of mathematical problems. This allows the difficulty level of the target mathematical problem to be considered during the determination of the solution path, enabling adaptive derivation of optimal solution steps based on the difficulty of the mathematical problem. This improves the problem-solving effect and provides users with better guidance and reference. Attached Figure Description

[0075] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 This is a flowchart of a problem-solving path determination method provided in an embodiment of this application;

[0077] Figure 2 This is a schematic diagram of a node width search provided in an embodiment of this application;

[0078] Figure 3 This is a schematic diagram of the structure of an initial difficulty recognition model provided in an embodiment of this application;

[0079] Figure 4 This is a schematic diagram of an adaptive generation system for mathematical problems of varying difficulty provided in an embodiment of this application;

[0080] Figure 5 This is a schematic diagram of an adaptive reasoning process for mathematical problems of varying difficulty provided in an embodiment of this application;

[0081] Figure 6 This is a schematic diagram of the structure of a problem-solving path determination device provided in an embodiment of this application;

[0082] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0083] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0084] For ease of understanding, the following describes some aspects of the embodiments of this application:

[0085] Large Language Models (LLMs) have demonstrated increasingly strong capabilities in solving mathematical problems, making progress in everything from simple arithmetic to complex proofs. Their problem-solving strategies primarily rely on powerful pattern recognition and sequence prediction abilities, learning mathematical symbols, formulas, logical reasoning, and the connections and expressions of natural language through pre-training on massive amounts of data. However, relying solely on pre-trained knowledge and pattern matching, LLMs still face challenges when dealing with complex mathematical reasoning, problems requiring multi-step derivation, or creative thinking, and are prone to logical errors or illogical answers. To improve LLMs' ability to solve mathematical problems, researchers have explored various methods, among which the combination of Chain-of-Thought (CoT) hints and Monte Carlo Tree Search (MCTS) has shown significant advantages.

[0086] However, a significant shortcoming of using the MCTS framework to solve mathematical problems is that large models cannot provide optimal solution steps based on the difficulty of the problem. For example, simple problems may involve complex reasoning, which is not user-friendly; or difficult problems may have too few solution steps, which are not easy to use as a reference.

[0087] Relevant literature discloses a mathematical problem-solving scheme based on the MCTS framework. This scheme systematically elaborates on the routine of solving mathematical problems using Monte Carlo Tree Search and provides key modules such as training data construction, Supervised Fine Tuning (SFT) training, and model inference. The experimental results in the paper also demonstrate the effectiveness of this scheme. However, the drawbacks of this scheme include: it only considers the accuracy of the generated answer and cannot control the length of the generated inference steps; it cannot generate personalized inference steps for mathematical problems of different difficulties.

[0088] Relevant literature has investigated the impact of the number of inference steps on mathematical problems, conducting a series of experiments to explore the effect of different inference steps on the performance of gsm8k. The final conclusion is that the benefit of increasing inference steps depends on the nature of the task: simple tasks require fewer inference steps, while longer inference processes bring significant advantages when dealing with more complex tasks. However, the drawbacks of this approach include: it only considers simple COT (Content-Oriented Problems), which cannot guarantee the accuracy of the generated results; and it only provides general conclusions, stating that simple problems require shorter inference steps and complex problems require longer inference steps, thus failing to achieve adaptability.

[0089] In this application embodiment, a method, apparatus, device, medium, and program product for determining a problem-solving path are proposed to solve the problem of poor problem-solving effect in related technologies.

[0090] See Figure 1 , Figure 1 This is a flowchart of a problem-solving path determination method provided in an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0091] Step 101: Analyze the acquired target mathematical problem based on the pre-trained difficulty recognition model to determine the difficulty level of the target mathematical problem.

[0092] In this step, the pre-trained difficulty recognition model can be a machine learning or deep learning model trained on a large amount of data. It is used to output the difficulty level or a specific numerical score after a mathematical problem is input. The target mathematical problem can be a specific mathematical problem provided by the user, which can be a textual description, a formula expression, or the content of the problem in an image. The difficulty level can be the classification result of the pre-trained difficulty recognition model after analyzing the target mathematical problem, used in subsequent steps to select appropriate problem-solving strategies or algorithm parameters.

[0093] Step 102: Based on the target mathematical problem and its difficulty level, determine the solution path for the target mathematical problem using a beam search algorithm.

[0094] In this step, the beam search algorithm described above can be used to explore the solution steps for the target mathematical problem. Beam search is a graph search algorithm used to find the optimal path in a graph or tree. It is a heuristic search algorithm and can be seen as an optimization of breadth-first search. Beam search reduces the complexity of the search space and improves search efficiency by retaining the k most likely candidate paths at each step; k is called the beam size. The solution path described above can be a specific sequence of steps or a logical flow for solving the target mathematical problem.

[0095] In the embodiments of the present application, the above steps determine the problem-solving path of the target mathematical problem using a beam search algorithm based on the difficulty level of the target mathematical problem, which integrates the beam search algorithm with the difficulty classification of mathematical problems. This enables the difficulty level of the target mathematical problem to be considered in the process of determining the problem-solving path, so that relatively optimal problem-solving steps can be adaptively derived according to the difficulty of the mathematical problem, thereby improving the problem-solving effect and providing users with better guidance and reference.

[0096] Optionally, the determining the problem-solving path of the target mathematical problem using a beam search algorithm based on the target mathematical problem and the difficulty level of the target mathematical problem includes:

[0097] generating an initial search beam based on the target mathematical problem using the beam search algorithm;

[0098] determining a penalty function based on the difficulty level of the target mathematical problem, wherein the penalty function is configured to determine penalty values corresponding to the plurality of candidate problem-solving paths;

[0099] generating a plurality of candidate problem-solving paths based on a pre-trained policy model and the initial search beam;

[0100] evaluating the plurality of candidate problem-solving paths based on a pre-trained value model and the penalty function to determine the problem-solving path of the target mathematical problem, wherein the value model is configured to score the plurality of candidate problem-solving paths, and the plurality of candidate problem-solving paths comprises the problem-solving path of the target mathematical problem.

[0101] It can be understood that the above penalty function is a rule for imposing additional cost on partial selections in the problem-solving path based on the difficulty level and the length of the problem-solving path.

[0102] Specifically, the steps of determining the problem-solving path of the target mathematical problem using the beam search algorithm are as follows, wherein the parameters include a beam search parameter B1 (representing the number of beams), B2 (representing the number of nodes), an initial mathematical problem q, and a policy model π θ , a value model V φ , and a maximum search step count T:

[0103] 1. Calculate the difficulty d of the initial mathematical problem q, and initialize the penalty function α(d,t);

[0104] 2. Generate the initial search beam, where C=[s0]*B1, t=0;

[0105] 3. For t<T, perform the following operations in a loop:

[0106] 3.1 For each node s in C t , generate B2 candidate steps through the policy model

[0107] 3.2 Traverse from b=1 to B2, and execute:

[0108] 3.2.1s t+1 =Cat[s t ,a b [Change the current state s] t Following candidate step a b To splice;

[0109] 3.2.2 Using the value function V φ (s t+1 Assess the current state s t+1 The value of this needs to be considered, and a penalty function α(d,t) needs to be added.

[0110] 3.2.3 Update the B1 paths with the highest contribution as evaluated in the previous step to C;

[0111] Ultimately, the path that contributes the most to the solution in C is returned, which is also the solution path to the target mathematical problem.

[0112] In this implementation, multiple candidate solution paths are evaluated using a pre-trained value model and penalty function to determine the solution path for the target mathematical problem. This approach considers the difficulty level of the target mathematical problem and the path length of the reasoning steps when determining the solution path, thereby further improving the problem-solving effect.

[0113] Optionally, before generating multiple candidate solution paths based on the pre-trained policy model and the initial search bundle, the method further includes:

[0114] Using the Monte Carlo tree search algorithm, a solution path is generated for the sample problem, resulting in multiple nodes. These nodes are used to indicate the solution steps for the sample problem.

[0115] The contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​that correspond one-to-one with the multiple nodes. The contribution values ​​are used to characterize the influence of the nodes on generating the correct answer to the sample question.

[0116] The initial value model and the initial strategy model are trained based on the training data to obtain the pre-trained strategy model and the pre-trained value model. The training data includes the problem-solving data and contribution values ​​corresponding to the multiple nodes.

[0117] It should be noted that Monte Carlo Tree Search (MCTS) is a tree-search-based algorithm that estimates the value of each action by simulating a large number of random games and selects the action with the best chance of winning. It has achieved significant success in the game domain, especially in complex games such as Go, and has also been applied to other fields, such as planning, scheduling, and optimization problems.

[0118] The core idea of ​​MCTS is to represent the search space as a tree, where each node represents a game state and each edge represents a possible action. MCTS continuously expands this tree by simulating the game and updating the statistics of each node, such as the number of visits and average score. Ultimately, MCTS selects the action corresponding to the node with the most visits or the highest average score.

[0119] The main steps of MCTS can be summarized into the following four stages, commonly referred to as "Selection-Expansion-Simulation-Backpropagation" (EESB):

[0120] Selection: Starting from the root node, MCTS selects a child node based on a strategy, such as the Upper Confidence Bound 1 (UCB1) algorithm. The UCB1 algorithm balances exploration and exploitation, favoring nodes with fewer visits or higher average scores. This selection process continues until a leaf node, i.e., a node that has not yet been expanded, is reached.

[0121] Expansion: If the selected leaf node is not in a terminating state (e.g., game over), MCTS will select one or more feasible actions and add the states corresponding to these actions to the tree as new child nodes.

[0122] Simulation: Starting with a newly expanded node, MCTS plays a simulated game until a termination state is reached. The simulation process typically employs a randomized strategy, such as randomly selecting actions. The simulation result, such as winning or losing the game, is used to evaluate the node's value.

[0123] Backpropagation: The value of the simulation results is propagated back to the parent nodes of the search tree, updating the statistics of each node, such as the number of visits and the average score.

[0124] These four steps will iterate continuously until the preset time limit or computing resource limit is reached. Finally, MCTS will select the action corresponding to the child node with the most visits or the highest average score under the root node.

[0125] Specifically, the sample problem mentioned above can be one or more mathematical problems. By using Monte Carlo random sampling, reasoning paths for different mathematical problems are constructed. A large number of reasoning paths are used to generate a Monte Carlo search tree, which in turn generates training data. Specifically, it is divided into four stages: selection, expansion, evaluation, and backtracking.

[0126] The initial policy model described above is responsible for generating solution steps for the mathematical problem, while the initial value model is responsible for scoring each generated step, i.e., evaluating the contribution of each generated node, mainly considering its impact on the correctness of the final answer. The training data can be divided into positive and negative samples. Training the initial value model and initial policy model based on the training data can be understood as supervised fine-tuning the large model. Specifically, the initial policy model can be trained on positive samples, while the initial value model is trained on both positive and negative samples.

[0127] For example, the training loss of a model can include two parts: one is the cross-entropy loss of the policy model, and the other is the mean squared error loss (MSE loss) of predictions at each solution step, as shown in the following formula:

[0128]

[0129] Where, π θ It is a policy model that needs to be trained, and it is only trained on positive samples. V φ (s t (x) is a value model that needs to be trained, and it is trained on each positive and negative sample. + Let x represent a positive sample. - Let represent a positive sample, T(x) represent the number of steps to generate the answer x, β represent the loss balancing hyperparameter used to balance the loss of the policy model and the loss of the value model, and q represent the mathematical problem.

[0130] In this implementation, different inference paths are constructed based on the Monte Carlo tree search algorithm to generate a Monte Carlo search tree, which in turn generates training data. The training data is then used to train the policy model and the value model, thereby improving the generalization ability of the policy model and the value model.

[0131] Optionally, the step of evaluating the contribution of the plurality of nodes to obtain a plurality of contribution values ​​corresponding one-to-one with the plurality of nodes includes:

[0132] The sample problem is analyzed based on the pre-trained difficulty recognition model to determine the difficulty level of the sample problem;

[0133] Based on the difficulty level of the sample problem and the preset function, the contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​corresponding to the multiple nodes.

[0134] Specifically, the aforementioned preset function can be a formula that associates the difficulty level of the sample problem with the inference path length and the contribution of the calculation node. The specific settings can be made based on experience and experiments.

[0135] It is understandable that when evaluating the contribution of the multiple nodes, leaf nodes (middle nodes) and final nodes can be distinguished. For example, the specific formula is as follows:

[0136]

[0137] Wherein, V(s) t ) is the value model's prediction for the current node, where i represents the value of the current state s. t In the i-th simulation, r is the exact contribution value at the endpoint, where r = 1 if the answer is correct and r = 0 if the answer is incorrect. alpha is a hyperparameter relating difficulty (d) and reasoning path length (t), which can be set according to the reasoning path of mathematical problems of different difficulties. For example, if the problem difficulty is <= 3, the most suitable reasoning steps are 2; if the difficulty is between 4 and 6, the most suitable reasoning steps are 3, and so on. An example formula is shown below:

[0138]

[0139] After the nodes are extended to the final endpoint, the final path of each inference step, i.e., edge(s,a), is calculated, and the contribution value of each node on this path is updated according to the following formula:

[0140]

[0141] V(s t+1 )=Q(s t ,a t )

[0142] Wherein, N(s) t a) represents the current node s t The number of visits, Q(st,a) represents the current node s. t The contribution value, j represents the j-th simulation, there are i simulations in total, and a represents the contribution value from the current node s. t The next step to take is the next answer generated by the large model based on the current node.

[0143] In this implementation, based on the difficulty level of the sample problem and a preset function, the contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​corresponding to each of the multiple nodes. This ensures that the difficulty level is taken into account when determining the contribution of a node, that is, providing shorter reasoning steps for simple problems and longer reasoning steps for complex problems, thereby further improving the problem-solving effect.

[0144] Optionally, the Monte Carlo tree search algorithm is used to generate a solution path for the sample problem, resulting in multiple nodes, including:

[0145] Monte Carlo tree search is performed with the sample problem as the root node to obtain multiple nodes. Starting from the root node, the Polynomial Upper Bound Confidence Tree (PUCT) algorithm is used to select nodes, and node expansion is performed based on depth search and breadth search until the stopping condition is met and the termination node is reached. Backtracking is then performed to the root node to obtain the solution path for the sample problem.

[0146] Specifically, Monte Carlo Tree Search consists of four stages: node selection, node expansion, node evaluation, and node backtracking. Node selection starts from the initial state (i.e., the sample problem) and gradually generates solution steps. Each time a node is selected, the Polynomial Upper Confidence Bounds for Trees (PUCT) algorithm is followed. This algorithm has two factors: one considers the node's contribution value Q, and the other considers the node's traversal degree. If the number of traversals is low, the node may be more likely to be selected. The PUCT algorithm is mainly used to balance the solution efficiency with the diversity of reasoning paths. The specific formula is as follows:

[0147]

[0148] Where, τ k Let q(s) represent the set of nodes. t a) represents the contribution value of the current node, c puct It is a hyperparameter, N parent (a) represents node s t The number of visits to the parent node, N(s) t ,a) represents the number of times the current node has been accessed.

[0149] Specifically, the node expansion mentioned above refers to starting from the current node, the larger model generates the next inference step. The temperature coefficient can be set relatively high to increase the diversity of expanded nodes; the specific value can be set based on experience and experiments. The depth search mentioned above refers to expanding child nodes as deeply as possible along a branch during the node expansion process until a termination condition or simulation result feedback is reached. The width search mentioned above refers to generating more child nodes in multiple branches at the current level and then exploring downwards, such as... Figure 2 As shown.

[0150] In this implementation, node expansion is based on a combination of depth search and breadth search, which allows multiple paths to be explored simultaneously during the problem-solving path generation process and avoids local optima traps. This enables the correct solution to be converged quickly within a limited time. Furthermore, the computational cost is optimized through early termination and backtracking mechanisms, thereby improving the efficiency and accuracy of determining the problem-solving path.

[0151] Optionally, before analyzing the acquired target mathematical problem based on the pre-trained difficulty recognition model to determine the difficulty level of the target mathematical problem, the method further includes:

[0152] Obtain a training sample set, which includes multiple mathematical problems and their corresponding standard answers;

[0153] The multiple mathematical problems are analyzed using a preset model to obtain multiple analytical answers that correspond one-to-one with the multiple mathematical problems;

[0154] Based on the multiple analytical answers and multiple target answers, multiple difficulty levels are determined that correspond one-to-one with the multiple mathematical problems;

[0155] The initial difficulty recognition model is trained based on the multiple mathematical problems and their corresponding difficulty levels to obtain the pre-trained difficulty recognition model.

[0156] Specifically, the aforementioned training sample set can be raw data containing mathematical problems and their standard answers, used to train the model. This data can be obtained by collecting corpora of mathematical problems from elementary to graduate levels both domestically and internationally. For example, it could include approximately 5 million high-quality question-and-answer pairs. The aforementioned mathematical problems can be specific questions requiring classification of difficulty, and the aforementioned standard answers can be correct solutions provided by experts or authoritative sources.

[0157] The aforementioned preset model can be a mathematical problem-solving tool in related technologies. Based on the multiple analytical answers and multiple target answers, multiple difficulty levels are determined that correspond one-to-one with the multiple mathematical problems. This can be based on the accuracy of the generated answers as an objective score of the difficulty of the mathematical problems, or it can be based on the derivation steps and mathematical knowledge used in the problem-solving process to determine the difficulty level.

[0158] The above-mentioned training of the initial difficulty recognition model based on the multiple mathematical problems and their corresponding difficulty levels can be achieved by quantifying the difficulty levels into labels for supervised learning of the initial difficulty recognition model.

[0159] The initial difficulty recognition model described above can be obtained by modifying a large Transformer structure model, that is, by adding a fully connected linear layer at the end and outputting a normalized difficulty score through a Sigmoid function. For example, Figure 3 This is a schematic diagram of the structure of an initial difficulty recognition model provided in an embodiment of this application, such as... Figure 3 As shown.

[0160] In this implementation, a preset model is used to analyze the multiple mathematical problems. Based on the multiple analytical answers and multiple target answers, multiple difficulty levels corresponding to the multiple mathematical problems are determined. An initial difficulty recognition model is trained based on the multiple mathematical problems and their corresponding difficulty levels to obtain the pre-trained difficulty recognition model. This enables automated analysis and annotation of the training sample set, as well as supervised learning of the difficulty recognition model. This reduces the cost of manual annotation and classification and improves the accuracy and efficiency of determining the difficulty level of the target mathematical problems.

[0161] For example, the above method can be applied to an adaptive generation system for mathematical problems of varying difficulty. Figure 4 This is a schematic diagram of an adaptive generation system for mathematical problems of varying difficulty provided in an embodiment of this application, such as... Figure 4 As shown, the adaptive generation system for mathematical problems of different difficulties includes a mathematical problem difficulty rating module, an MCTS data construction module, an SFT training module, and an inference module. The mathematical problem difficulty rating module is used to rate the difficulty of mathematical problems, the MCTS data construction module is used to generate training data based on MCTS, the SFT training module is used to fine-tune the model based on the training data, and the inference module is used to generate solution paths for mathematical problems based on the fine-tuned model.

[0162] Figure 5 This is a schematic diagram of an adaptive reasoning process for mathematical problems of varying difficulty provided in an embodiment of this application, such as... Figure 5As shown, after completing model training, the model enters the final inference stage. This stage mainly targets mathematical problems of different difficulties and adaptively generates the most appropriate solution steps. The entire inference process is divided into two parts in sequence. The first part is to have the large model score the difficulty of the mathematical problem and obtain a difficulty score d. The second part is to use the trained policy model and value model to adaptively generate inference steps for the mathematical problem and obtain the final solution path. The second part proposes a beam search algorithm based on difficulty level. When selecting inference steps, it not only considers the score of the value model, but also penalizes the difficulty of the current mathematical problem and the path length of the inference steps.

[0163] See Figure 6 , Figure 6 This is a schematic diagram of a problem-solving path determination device provided in an embodiment of this application, as shown below. Figure 6 As shown, the problem-solving path determination device 600 includes:

[0164] The first determining module 601 is used to analyze the acquired target mathematical problem based on a pre-trained difficulty recognition model and determine the difficulty level of the target mathematical problem.

[0165] The second determining module 602 is used to determine the solution path of the target mathematical problem based on the target mathematical problem and the difficulty level of the target mathematical problem using a beam search algorithm.

[0166] Optionally, the second determining module 601 includes:

[0167] The first generation unit is used to generate an initial search bundle based on the target mathematical problem using a bundle search algorithm.

[0168] The second generation unit is used to generate multiple candidate problem-solving paths based on the pre-trained policy model and the initial search bundle;

[0169] The first determining unit is used to determine a penalty function based on the difficulty level of the target mathematical problem, wherein the penalty function is used to determine the penalty value corresponding to the plurality of candidate solution paths;

[0170] The second determining unit is used to evaluate the plurality of candidate solution paths based on the pre-trained value model and the penalty function, and determine the solution path of the target mathematical problem. The value model is used to score the plurality of candidate solution paths, and the plurality of candidate solution paths include the solution path of the target mathematical problem.

[0171] Optionally, the second determining module 602 further includes:

[0172] The third generation unit is used to generate a solution path for the sample problem using the Monte Carlo tree search algorithm, and obtain multiple nodes, which are used to indicate the solution steps of the sample problem.

[0173] An evaluation unit is used to evaluate the contribution of the plurality of nodes and obtain a plurality of contribution values ​​corresponding one-to-one with the plurality of nodes. The contribution values ​​are used to characterize the influence of the node on generating the correct answer to the sample question.

[0174] The training unit is used to train the initial value model and the initial strategy model based on the training data to obtain the pre-trained strategy model and the pre-trained value model. The training data includes the problem-solving data and contribution values ​​corresponding to the multiple nodes.

[0175] Optionally, the evaluation unit is specifically used for:

[0176] The sample problem is analyzed based on the pre-trained difficulty recognition model to determine the difficulty level of the sample problem;

[0177] Based on the difficulty level of the sample problem and the preset function, the contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​corresponding to the multiple nodes.

[0178] Optionally, the third generation unit is specifically used for:

[0179] Monte Carlo tree search is performed with the sample problem as the root node to obtain multiple nodes. Starting from the root node, the Polynomial Upper Bound Confidence Tree (PUCT) algorithm is used to select nodes, and node expansion is performed based on depth search and breadth search until the stopping condition is met and the termination node is reached. Backtracking is then performed to the root node to obtain the solution path for the sample problem.

[0180] Optionally, the device further includes:

[0181] The acquisition module is used to acquire a training sample set, which includes multiple mathematical problems and their corresponding standard answers.

[0182] The parsing module is used to analyze the multiple mathematical problems using a preset model to obtain multiple analytical answers that correspond one-to-one with the multiple mathematical problems;

[0183] The third determining module is used to determine multiple difficulty levels corresponding one-to-one with the multiple analytical answers and multiple target answers;

[0184] The training module is used to train the initial difficulty recognition model based on the multiple mathematical problems and their corresponding difficulty levels, so as to obtain the pre-trained difficulty recognition model.

[0185] It should be noted that the problem-solving path determination device provided in this application embodiment is a device capable of executing the above-described problem-solving path determination method. Therefore, all implementation methods in the above-described problem-solving path determination method embodiments are applicable to this device and can achieve the same or similar beneficial effects. To avoid repetition, this embodiment will not elaborate further.

[0186] For details, see Figure 7 As shown in the figure, this application embodiment also provides an electronic device, including a bus 701, a transceiver 702, an antenna 703, a bus interface 704, a processor 705, and a memory 706.

[0187] Processor 705 is used for:

[0188] The target mathematical problem is analyzed based on a pre-trained difficulty recognition model to determine the difficulty level of the target mathematical problem.

[0189] Based on the target mathematical problem and its difficulty level, a beam search algorithm is used to determine the solution path for the target mathematical problem.

[0190] exist Figure 7 In this document, a bus architecture (represented by bus 701) is used. Bus 701 can include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 705 and memory represented by memory 706. Bus 701 can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 704 provides an interface between bus 701 and transceiver 702. Transceiver 702 can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 705 is transmitted over a wireless medium via antenna 703, which further receives data and transmits data to processor 705.

[0191] Processor 705 manages bus 701 and general processing, and also provides various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. Memory 706 can be used to store data used by processor 705 during operation.

[0192] Optionally, the processor 705 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD).

[0193] Optionally, the processor 705 is specifically used for:

[0194] An initial search bundle is generated based on the target mathematical problem using a bundle search algorithm;

[0195] Based on the pre-trained policy model and the initial search bundle, multiple candidate solution paths are generated;

[0196] Based on the difficulty level of the target mathematical problem, a penalty function is determined, which is used to determine the penalty value corresponding to the multiple candidate solution paths;

[0197] The multiple candidate solution paths are evaluated based on a pre-trained value model and the penalty function to determine the solution path for the target mathematical problem. The value model is used to score the multiple candidate solution paths, which include the solution path for the target mathematical problem.

[0198] Optionally, the processor 705 is further configured to:

[0199] Using the Monte Carlo tree search algorithm, a solution path is generated for the sample problem, resulting in multiple nodes. These nodes are used to indicate the solution steps for the sample problem.

[0200] The contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​that correspond one-to-one with the multiple nodes. The contribution values ​​are used to characterize the influence of the nodes on generating the correct answer to the sample question.

[0201] The initial value model and the initial strategy model are trained based on the training data to obtain the pre-trained strategy model and the pre-trained value model. The training data includes the problem-solving data and contribution values ​​corresponding to the multiple nodes.

[0202] Optionally, the processor 705 is specifically used for:

[0203] The sample problem is analyzed based on the pre-trained difficulty recognition model to determine the difficulty level of the sample problem;

[0204] Based on the difficulty level of the sample problem and the preset function, the contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​corresponding to the multiple nodes.

[0205] Optionally, the processor 705 is specifically used for:

[0206] Monte Carlo tree search is performed with the sample problem as the root node to obtain multiple nodes. Starting from the root node, the Polynomial Upper Bound Confidence Tree (PUCT) algorithm is used to select nodes, and node expansion is performed based on depth search and breadth search until the stopping condition is met and the termination node is reached. Backtracking is then performed to the root node to obtain the solution path for the sample problem.

[0207] Optionally, the processor 705 is further configured to:

[0208] Obtain a training sample set, which includes multiple mathematical problems and their corresponding standard answers;

[0209] The multiple mathematical problems are analyzed using a preset model to obtain multiple analytical answers that correspond one-to-one with the multiple mathematical problems;

[0210] Based on the multiple analytical answers and multiple target answers, multiple difficulty levels are determined that correspond one-to-one with the multiple mathematical problems;

[0211] The initial difficulty recognition model is trained based on the multiple mathematical problems and their corresponding difficulty levels to obtain the pre-trained difficulty recognition model.

[0212] It should be noted that the electronic device provided in this application embodiment is a device capable of executing the above-described problem-solving path determination method. Therefore, all implementation methods in the above-described problem-solving path determination method embodiments are applicable to this electronic device and can achieve the same or similar beneficial effects. To avoid repetition, this embodiment will not elaborate further.

[0213] This invention also provides an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the various processes of the above-described problem-solving path determination method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0214] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described problem-solving path determination method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0215] This application also provides a computer program product, including computer instructions. When these computer instructions are executed by a processor, they implement the various processes of the above-described problem-solving path determination method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0216] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0217] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0218] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for determining a problem-solving path, characterized in that, include: The target mathematical problem is analyzed based on a pre-trained difficulty recognition model to determine the difficulty level of the target mathematical problem. Based on the target mathematical problem and its difficulty level, a beam search algorithm is used to determine the solution path for the target mathematical problem.

2. The method according to claim 1, characterized in that, The step of determining the solution path for the target mathematical problem using a beam search algorithm based on the target mathematical problem and its difficulty level includes: An initial search bundle is generated based on the target mathematical problem using a bundle search algorithm; Based on the pre-trained policy model and the initial search bundle, multiple candidate solution paths are generated; Based on the difficulty level of the target mathematical problem, a penalty function is determined, which is used to determine the penalty value corresponding to the multiple candidate solution paths; The multiple candidate solution paths are evaluated based on a pre-trained value model and the penalty function to determine the solution path for the target mathematical problem. The value model is used to score the multiple candidate solution paths, which include the solution path for the target mathematical problem.

3. The method according to claim 2, characterized in that, Before generating multiple candidate solution paths based on the pre-trained policy model and the initial search bundle, the method further includes: Using the Monte Carlo tree search algorithm, a solution path is generated for the sample problem, resulting in multiple nodes. These nodes are used to indicate the solution steps for the sample problem. The contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​that correspond one-to-one with the multiple nodes. The contribution values ​​are used to characterize the influence of the nodes on generating the correct answer to the sample question. The initial value model and the initial strategy model are trained based on the training data to obtain the pre-trained strategy model and the pre-trained value model. The training data includes the problem-solving data and contribution values ​​corresponding to the multiple nodes.

4. The method according to claim 3, characterized in that, The step of evaluating the contribution of the plurality of nodes to obtain a plurality of contribution values ​​corresponding one-to-one with the plurality of nodes includes: The sample problem is analyzed based on the pre-trained difficulty recognition model to determine the difficulty level of the sample problem; Based on the difficulty level of the sample problem and the preset function, the contribution of the multiple nodes is evaluated to obtain multiple contribution values ​​corresponding to the multiple nodes.

5. The method according to claim 3, characterized in that, The Monte Carlo tree search algorithm is used to generate a solution path for the sample problem, resulting in multiple nodes, including: Monte Carlo tree search is performed with the sample problem as the root node to obtain multiple nodes. Starting from the root node, the Polynomial Upper Bound Confidence Tree (PUCT) algorithm is used to select nodes, and node expansion is performed based on depth search and breadth search until the stopping condition is met and the termination node is reached. Backtracking is then performed to the root node to obtain the solution path for the sample problem.

6. The method according to claim 1, characterized in that, Before analyzing the acquired target mathematical problem based on the pre-trained difficulty recognition model to determine the difficulty level of the target mathematical problem, the method further includes: Obtain a training sample set, which includes multiple mathematical problems and their corresponding standard answers; The multiple mathematical problems are analyzed using a preset model to obtain multiple analytical answers that correspond one-to-one with the multiple mathematical problems; Based on the multiple analytical answers and multiple target answers, multiple difficulty levels are determined that correspond one-to-one with the multiple mathematical problems; The initial difficulty recognition model is trained based on the multiple mathematical problems and their corresponding difficulty levels to obtain the pre-trained difficulty recognition model.

7. A problem-solving path determination device, characterized in that, include: The first determining module is used to analyze the acquired target mathematical problem based on a pre-trained difficulty recognition model and determine the difficulty level of the target mathematical problem. The second determining module is used to determine the solution path of the target mathematical problem based on the target mathematical problem and the difficulty level of the target mathematical problem, using a beam search algorithm.

8. An electronic device, characterized in that, Includes a transceiver and a processor, the processor being used for: The target mathematical problem is analyzed based on a pre-trained difficulty recognition model to determine the difficulty level of the target mathematical problem. Based on the target mathematical problem and its difficulty level, a beam search algorithm is used to determine the solution path for the target mathematical problem.

9. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the problem-solving path determination method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the problem-solving path determination method as described in any one of claims 1 to 6.

11. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the problem-solving path determination method as described in any one of claims 1 to 6.

Images