Large model and knowledge graph dual-drive-based guide type inference system and method

Abstract

The invention discloses a large model and knowledge graph dual-drive-based guided reasoning system and method, belongs to the technical field of artificial intelligence reasoning, and solves the problem of how to improve the process reasoning ability of a large language model for engineering subject courses and the reliability of solving complex engineering problems. According to the method, metadata is extracted from teaching materials, structured problem representation is constructed, and a knowledge graph is constructed to form a complete knowledge system; the method comprises the following steps: decomposing a complex engineering problem into a structured solving plan with knowledge marks, and carrying out iterative loop based on a Monte Carlo tree search algorithm to generate a search tree comprising a plurality of high-quality candidate problem solving paths; then determining an optimal answer and a corresponding reasoning path in all simulated paths in a voting mode, performing confidence evaluation on each node of each path in the candidate path set, and further selecting a path of an optimal solution; and the process reasoning capability of the large language model on engineering subject courses and the reliability of solving complex engineering problems are effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence reasoning technology, and relates to a guided reasoning system and method based on a dual-drive approach of large model and knowledge graph. Background Technology

[0002] "New Engineering" refers to majors targeting emerging industries, such as artificial intelligence, intelligent manufacturing, robotics, and cloud computing, and also includes the upgrading and transformation of traditional engineering majors. Against the backdrop of AI technology driving educational reform, AI technology, especially Large Language Modeling (LLM), is widely used in smart education platforms to build intelligent question-answering and problem-solving systems, becoming a core driving force for the intelligent transformation of education. However, due to the highly logical, structured, and multi-constrained nature of engineering courses, current smart education platforms have significant shortcomings in teaching engineering subjects (such as Computer Organization), making it difficult to meet the problem-solving needs of New Engineering education. Specifically: (1) The platform lacks the inherent logic of new engineering knowledge: Due to the lack of in-depth modeling of the essence of engineering knowledge, it cannot accurately capture the inherent relationship between knowledge points such as hardware components and system architecture, digital logic and instruction sets, and it is difficult to guide students to carry out innovative thinking based on knowledge networks. (2) The platform lacks reasoning ability for new engineering: Faced with complex engineering problems that are interdisciplinary and unstructured, the platform lacks the ability to guide dynamic iterative reasoning and cannot generate a solution process that conforms to hardware constraints and cross-knowledge relationships. (3) The large language model has reliability defects, including "presuppositional knowledge defects", which are manifested in the randomness of calling basic knowledge points and the deviation of the reasoning starting point due to the lack of key premises; and "process reasoning defects", which are manifested in the easy breakage of the logical chain in multi-step reasoning and the inability to independently verify the rationality of the steps. More importantly, the existing educational platform has not built a collaborative mechanism for knowledge structuring and dynamic reasoning, which can neither rely on knowledge relationships to provide credible support for reasoning nor has it provided real-time logical verification of the reasoning process.

[0003] Existing technologies, such as the invention patent with publication number CN120125392A, disclose a smart education method and system based on student behavior sequence recommendations. This method models student behavior sequences, uses knowledge graph reasoning to capture the relationships between knowledge points, and generates personalized learning paths for students based on the student behavior sequence modeling results and the knowledge graph reasoning results. However, this "weak association + no verification" model is insufficient to meet the teaching needs of solving complex engineering problems, making it difficult for the question-and-answer results to meet the reasoning rigor and process interpretability requirements of engineering courses. Therefore, there is an urgent need to construct a novel question-and-answer support system that integrates the structured constraints of knowledge graphs with the dynamic reasoning capabilities of large models. Summary of the Invention

[0004] The technical problem to be solved by this invention is how to improve the process reasoning ability of large language models for engineering courses and the reliability of solving complex engineering problems.

[0005] The present invention solves the above-mentioned technical problems through the following technical solutions: A guided reasoning system driven by both large models and knowledge graphs includes: The metadata extraction module is used to extract metadata from the course's teaching materials, including core elements of the questions, prerequisite knowledge, and analysis of common mistakes. The knowledge graph construction module is used to convert extracted metadata into a structured knowledge graph. The adaptive planning generation module is used to receive the original complex engineering problem, perform in-depth analysis, and generate a structured solution plan with knowledge tags. The probabilistic search reasoning module uses the Monte Carlo tree search algorithm to perform iterative loops and outputs a search tree containing multiple complete solution paths and corresponding evaluation information. The consistency verification module verifies the answers to multiple complete problem-solving paths based on a self-consistency voting strategy, and selects the final candidate path set. The progressive evaluation module is used to perform node-by-node confidence evaluation on the paths selected from the final candidate path set, and select the best solution path based on the evaluation results. The high-quality data training module is used to collect path samples and filter high-quality training data, and optimize the model parameters of the large language model based on supervised fine-tuning and reinforcement learning.

[0006] Furthermore, the core elements of the question described in the metadata extraction module include known conditions, variables to be determined, and constraints. The aforementioned prerequisite knowledge specifically refers to the axioms, theorems, and other course knowledge points that are essential for solving the problem; The analysis of common pitfalls specifically refers to computational traps or conceptual misunderstandings present in the problem.

[0007] Furthermore, the adaptive planning generation module includes: The parsing unit is used to receive the original question input by the user. Based on a large language model, semantic parsing of the problem is performed to extract its core elements. The complexity of the problem is evaluated based on metadata, using the following logical representation:

[0008] in, For the original problem Complexity score, To determine the number of macro-level steps required to solve the problem, The number of dependencies between steps. The weighting factor is the type of operation involved in the problem. , and They are respectively , and The corresponding weighting coefficients; When complexity score When the preset threshold is exceeded, subsequent planning units are initiated; when the complexity score is... If the preset threshold is not exceeded, the probabilistic search reasoning module is executed directly; Planning unit, used to transform the original problem The problem is broken down and a series of macro-level solution steps in pseudocode form are generated, with each step defined as a subproblem. During the generation of each step, embedded tags are used to explicitly associate the step with one or more core theoretical nodes in the knowledge graph, forming knowledge tags and outputting the solution plan. This can be represented using the following logic:

[0009] in, For a complete solution plan, For the first Each sub-problem represents a macro-level problem-solving step; In order to be with the first A set of knowledge tags associated with each sub-problem To solve the planning The total number of subproblems contained therein .

[0010] Furthermore, the probabilistic search reasoning module includes the following: In the selection phase, each iteration starts from the root node of the search tree and iterates through the sibling nodes of each level until a leaf node that has not yet been fully expanded is reached. Node selection is performed based on the upper confidence bound formula, using the following logical representation:

[0011] in, For nodes The value of the upper confidence level, For nodes The average reward value, For nodes Number of visits, For nodes The parent node, This represents the number of times the parent node has been visited. For the exploration coefficient; During the expansion and dynamic pruning phase, when a leaf node is reached, the large language model generates multiple candidate decisions for solving the next sub-problem based on the method represented by the current node. Utilizing the constraints of the knowledge graph, all candidate decisions are dynamically pruned, filtering out and eliminating all logically invalid decisions. This dynamic pruning includes semantic similarity calculation and logical consistency verification. For each candidate decision that passes pruning, the system creates a new formal node for it and adds it to the search tree. In the simulation phase, a complete simulation path is formed. When a leaf node is reached, a new inference decision-making extension search tree is generated using the large language model. Simulated reasoning is then performed based on the structured knowledge of the knowledge graph, and the feasibility scoring function is calculated as follows:

[0012] in, This is the feasibility scoring function value. To simulate the path, For the first The weight of each reasoning step, For the first One reasoning step, For reasoning steps The matching degree with the knowledge point set KG in the knowledge graph is calculated using semantic embedding vector similarity. "time" indicates a complete match. This indicates a partial conflict. The time indicates a complete conflict. ; Feasibility rating As a fundamental assessment of the overall quality of the path, and combined with other external evaluation dimensions, a weighted calculation is performed to arrive at the final reward value for this simulation, expressed using the following logic:

[0013] in, This is the final reward value, which will be used in subsequent backpropagation phases; A positive constant bias term is used to ensure that when When the range of values ​​includes negative numbers, the base number inside the parentheses remains positive; The result self-validation score is used to measure the simulation path. To what extent does the final answer satisfy the inherent constraints of the original problem? The knowledge point coverage rate of the knowledge graph is used to measure the degree to which the reasoning process utilizes the knowledge graph. The efficiency of the reasoning steps can be considered in terms of the number of reasoning steps and computational complexity. , and They are respectively , and The corresponding reward value coefficient; The backpropagation phase is used to convert the final reward value calculated in the simulation phase. Along the path traversed in this simulation Propagating backward from the leaf node to the root node, passing through every node along the path. The stored statistics will be updated according to the following logic: First, update the node's access count. :

[0014] in, For nodes Updated visit count For nodes The number of visits before the update; secondly, the average reward value of the updated node. :

[0015] in, For nodes Updated average reward value, For nodes Average reward value before the update; After all iterations of the Monte Carlo tree search are completed, the final output is a search tree containing multiple complete solution paths and corresponding evaluation information.

[0016] Furthermore, the consistency verification module includes: The candidate answer extraction unit is used to identify all candidate answers from the root node to the leaf node from the search tree output by the probabilistic search reasoning module. A complete reasoning path constitutes the original path set. For each complete path Extract the final leaf node answer. Thus obtaining a containing A set of candidate answers , set of answers Passed to the self-consistency voting unit. ; Self-consistent voting units are used to evaluate the extracted set of candidate answers. Conduct voting statistics to determine the answer with the highest consensus, specifically including the following: Define an indicator function To determine whether two answers are consistent, the following logical expression can be used:

[0017] in, It is a set The first in 10 candidate answers It is a unique candidate answer; based on the above indicator function. Calculate each unique answer The frequency of occurrence is recorded as the number of votes. This can be represented using the following logic:

[0018] in, For unique answers In all The total number of times each candidate answer appears in the n-10 ... express The cumulative value of each candidate answer; The following logic is used to determine the optimal answer that receives the most votes among all candidate answers. :

[0019] in, This represents the answer with the highest degree of consensus. Indicates from all possible Find the one in The parameter with the largest function value ; Candidate path filtering unit, used to filter from the original path set Filter out all results that lead to the optimal answer. The reasoning path is used to generate the final candidate path set. This can be represented using the following logic:

[0020] in, For the final set of candidate paths, For the original path set The first in Path, For path The corresponding final answer.

[0021] Furthermore, the progressive evaluation module includes: Node confidence quantification scoring unit, used to traverse the candidate path set. For each path in the code, and for each node on the path... The node confidence is calculated through a dual verification mechanism that combines reward model evaluation and executable code verification. ; The optimal path selection unit is used to calculate the normalized total path confidence for each path. And select the one with the highest score; for the path All nodes This can be represented using the following logic:

[0022] for All paths :

[0023] in, This indicates the final selected solution path. For path Normalized path confidence, For path Length, This indicates finding the path corresponding to the normalized path confidence.

[0024] Furthermore, the confidence level Using the following logical representation: First, an independent reward model evaluates and scores the logical quality of the nodes. As shown in the following formula:

[0025] in, This indicates that the reward model applies to nodes. The rating, with a value range that can be normalized to... ; Represents a reward model. Represents a node The textual description of the reasoning steps represented. Represents a node The status information of the parent node; Secondly, for nodes containing computable steps The verification score is calculated by calling an executable Python code interpreter. To verify the correctness of its calculation, the following logical representation can be used:

[0026] in, This represents the verification score of the executable code. This indicates the process of executing the code interpreter. Represents the node The executable code snippet extracted from the description, This indicates that a node is considered complete when an executable code segment can be executed by the interpreter without errors. Validation score =1; This indicates when an executable code segment is interrupted during execution and throws an exception, the node... Validation score =0; Finally, the scores will be evaluated. With verification score Perform a weighted summation to obtain the node confidence score of that node. :

[0027] in, For nodes The final confidence score, and These are the weighting coefficients corresponding to the evaluation score and the verification score, respectively.

[0028] Furthermore, the high-quality data training module includes the following: The high-quality training data collection and classification phase collects all path samples processed by any preceding module. It combines the self-consistency and normalized total confidence of the path samples calculated in the consistency verification and progressive evaluation modules to evaluate all path samples. The classification includes: Expert-level samples include a set of paths that simultaneously satisfy high self-consistency and high process confidence; specifically, the final answer of the path. It must be equal to the optimal answer determined by the consistency verification module. Furthermore, the total confidence of the normalized path is higher than the high confidence threshold. , recorded as > ; A typical positive sample contains the following: High self-consistency, recorded as = ,but Paths at a medium level are denoted as < ≤ , Indicates the low confidence threshold; Low self-consistency, recorded as ,but Very high paths are denoted as > ; Negative samples, including the total confidence of the normalized path. The lower path is denoted as ≤ ; During the supervised fine-tuning phase, expert-level samples are used as training data for each optimal path. They are all converted into a "command-response" pair ,in, For the original question, Optimal path The text representing the complete, step-by-step reasoning process, with the objective function Training is performed to minimize the difference between the model's generated response and the expert-level sample, using the following logical representation:

[0029] in, To supervise the fine-tuning of the loss function, The parameters representing the large language model. express The first in the text Each word element, Indicates the first All lexical sequences preceding each lexical unit This represents the probability that the model will generate the next word given the preceding context. In the reinforcement learning phase based on course learning, the parameters of the large language model are further optimized based on reinforcement learning algorithms.

[0030] Furthermore, the reinforcement learning phase based on curriculum learning includes the following: First, define the reward function. For all path samples Assign a quantified training reward, the definition of which is directly related to the course level of the path, using the following logical representation:

[0031] in, Indicates when path sample When the sample is expert-level, the reward function Pick , The highest reward; Indicates when path sample When the sample is a normal positive sample, the reward function is... Pick , For smaller positive rewards; Indicates when path sample When the sample is negative, the reward function Pick , As punishment; Secondly, adopting a terminal reward distribution strategy will Transformed into each time step Instant rewards For a path consisting of T steps The following logic represents instant rewards. Allocation method:

[0032] in, Indicates the time steps in the path. , Representing a path The last time step; Then, reinforcement learning algorithms are used to process the parameters of the large language model. Optimization is performed, specifically by calculating the advantage function. To evaluate the merits of taking a certain action in a specific state, the advantage function is calculated based on the immediate reward as defined above. and state value function This can be represented using the following logic:

[0033] in, In time step The advantage function, As a discount factor, The value function is used to estimate the state. The cumulative rewards that may be obtained in the future For the next time step Corresponding state The cumulative rewards that may be obtained in the future; Based on the calculated advantage function The model parameters are updated by pruning the objective function. The objective function is represented by the following logic. :

[0034] in, Let PPO be the objective function to be optimized. This represents the expected value along the policy trajectory. The ratio of the new to the old strategies. , Indicates the parameters of the current model The defined new strategy This refers to the old strategy used to collect training data before the start of this optimization iteration; In time step The advantage function represents the advantage of taking a certain action compared to the average level in a given state. For the pruning function, limit the policy ratio to Within the interval, This is a hyperparameter for the clipping range.

[0035] This invention also provides a guided reasoning method based on a dual-drive approach of large models and knowledge graphs, comprising the following steps: S1 extracts metadata from the course's teaching materials, including core elements of the questions, prerequisite knowledge, and common mistakes. S2, convert the extracted metadata into a structured knowledge graph; S3 receives the original complex engineering problem and performs in-depth analysis to generate a structured solution plan with knowledge tags; S4, based on the Monte Carlo tree search algorithm, performs iterative loops to output a search tree containing multiple complete solution paths and corresponding evaluation information; S5 verifies the answers to multiple complete problem-solving paths based on a self-consistent voting strategy, and selects the final candidate path set; S6. Analyze the confidence of each node of the paths selected from the final candidate path set, and select the best solution path based on the evaluation results. S7 collects path samples and filters high-quality training data, and optimizes the model parameters of the large language model based on supervised fine-tuning and reinforcement learning.

[0036] The advantages of this invention are: This invention is applicable to complex problem-solving in emerging engineering disciplines and adaptive learning scenarios in smart education. First, it extracts core elements of the problem from teaching materials, extracts metadata, and constructs a structured problem representation. Second, it converts the metadata into a multi-level constrained graph structure, constructing a knowledge graph to form a complete knowledge system. Next, leveraging the characteristic of large-scale models that their planning ability surpasses their reasoning ability when dealing with complex problems, it decomposes complex engineering problems into structured solution plans with knowledge labels. It then iteratively iterates using a Monte Carlo tree search algorithm to generate a search tree containing multiple high-quality candidate solution paths. Finally, it uses a voting method to determine the optimal answer and its corresponding reasoning path from all simulated paths, forming a candidate path set. The confidence of each node in the candidate path set is evaluated to select the path corresponding to the best solution. Finally, high-quality training data is obtained by hierarchically filtering the path samples generated during the process. The model parameters of the large language model are optimized through a training paradigm that first supervises fine-tuning by imitating experts and then explores curriculum-based learning. The updated large language model is then redeployed to continuously enhance its knowledge depth and reasoning reliability.

[0037] This invention realizes the bidirectional driving and deep integration of the dynamic reasoning ability of large models and the static knowledge system of knowledge graphs, effectively solving the key technical problems in intelligent teaching of new engineering disciplines. It overcomes the problems of insufficient prior experience, process reasoning defects, and lack of verification mechanisms in the teaching of new engineering disciplines such as computer organization principles of existing large models, and effectively improves the process reasoning ability of large language models for engineering courses and the reliability of solving complex engineering problems. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the structure of the guided reasoning system based on a large model and a knowledge graph, according to Embodiment 1 of the present invention. Figure 2 This is a schematic diagram of the execution logic for guided reasoning in the course of computer organization principles according to Embodiment 1 of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments: Example 1 This invention takes the engineering course of Computer Organization and Architecture as an example, and explains the two core technologies involved as follows: Large language models, trained on massive amounts of data, can generate high-quality text content and possess powerful natural language understanding capabilities. This invention employs the Qwen-7B large language model. Qwen-7B, a next-generation open-source model developed by Alibaba DAMO Academy, is based on a Transformer architecture with 7 billion parameters. Through RoPE positional encoding and dynamic NTK-aware interpolation technology, it significantly improves long text processing capabilities, demonstrating outstanding performance, particularly in engineering disciplines such as computer organization. This model has undergone targeted fine-tuning with a specialized corpus of computer architecture. The output pseudocode framework not only guides problem-solving but, more importantly, enables dynamic verification through the guided reasoning mechanism provided by this invention.

[0041] Knowledge graphs, a technology capable of analyzing multi-labeled big data reasoning knowledge, are based on a triple representation of <head entity, relation, tail entity>. Through a graph structure composed of nodes and edges, they comprehensively depict a multi-layered knowledge system from the digital logic layer to the system architecture layer. Within the teaching framework of Computer Organization Principles, core concepts of each chapter serve as independent nodes, with their subordinate knowledge points forming a network of child nodes. A hierarchical knowledge topology is constructed through semantic relationships such as "containment" and "premise." Specifically, each knowledge point node is associated with its hardware implementation characteristics as attribute nodes, forming an enhanced knowledge representation structure that supports quantitative analysis. In the field of computer engineering education, the structured knowledge representation characteristics of knowledge graph technology are naturally compatible with the hierarchical features of computer architecture, providing a reliable knowledge infrastructure for intelligent teaching.

[0042] To provide a better understanding of the technical solution and effects achieved by the present invention, a detailed description is provided below, accompanied by preferred embodiments and accompanying drawings: like Figures 1-2 Specifically, a guided reasoning system based on a large model and a knowledge graph is disclosed, including: a metadata extraction module, a knowledge graph construction module, an adaptive planning generation module, a probabilistic search reasoning module, a consistency verification module, a progressive evaluation module, and a high-quality data training module.

[0043] The metadata extraction module is used to extract metadata from the course's teaching materials, including core elements of the questions, prerequisite knowledge, and common mistakes.

[0044] In this embodiment, the teaching materials include textbooks, exercise sets, and lab manuals. Taking the Computer Organization Principles course as an example, the course content is intelligently processed through a metadata extraction module to extract high-level information of the course as metadata from the teaching materials. Specifically, the metadata extraction module includes the following: First, multi-level text analysis technology is used to capture key knowledge points and the relationships between them in the teaching materials as core elements of the questions. These core elements specifically include known conditions, variables to be solved, and constraints. In this embodiment, the core elements can be floating-point arithmetic, two's complement representation, etc. Specifically, the multi-level text analysis technology utilizes Named Entity Recognition (NER) and Relation Extraction (RE) techniques to determine the relationships between these entities, thereby transforming unstructured text into structured core elements.

[0045] Secondly, the system identifies the implicit prerequisite knowledge and potential pitfalls within the test questions. Then, it utilizes the summarizing capabilities of a large language model to generalize and summarize the identified content, forming the prior knowledge of the questions as prerequisite knowledge and a common error analysis. In this embodiment, the prerequisite knowledge specifically refers to axioms, theorems, and other course knowledge points essential for solving the problem; the common error analysis specifically identifies potential computational pitfalls or conceptual misunderstandings within the problem. For example, for test questions involving the conversion of two's complement to original code, the system summarizes key information such as the steps involved in this conversion based on the large language model, providing a basis for subsequent processing of the test questions based on prior knowledge.

[0046] Furthermore, during the process of summarizing and generalizing using a large language model, inaccurate or redundant information may arise, requiring subsequent manual review. To reduce the difficulty of manual review and improve the accuracy of summarization, this embodiment incorporates Python programming for auxiliary processing. The test questions are preprocessed to remove obvious interfering information, and then the processed content is input into the large language model for summarization.

[0047] In this embodiment, the metadata extraction module identifies the structured features in the questions, transforming the unstructured original questions into structured question representations rich in semantic information. It then extracts implicit prerequisite knowledge and analyzes common mistakes to form prior knowledge. Finally, it constructs a complete metadata system that includes question semantics, calculation rules, and typical error patterns. This provides accurate and high-quality input metadata for all subsequent modules and provides high-quality prior knowledge support for various test-related operations based on the above metadata, thereby improving the performance and effectiveness of the entire system in test question processing.

[0048] The knowledge graph construction module is used to convert the extracted metadata into a structured knowledge graph, specifically: Based on Natural Language Processing (NLP) technology and the semantic understanding capabilities of Large Language Models (LLM), this method identifies entities and relationships between entities in metadata. The identified entities and relationships are then converted into triples for storage, constructing a high-quality metadata dataset covering the entire knowledge system of the course. This transforms the metadata from unstructured text into a structured knowledge graph. The triples are represented as <head entity, relation, tail entity> or <entity, attribute, attribute value>. In this embodiment, the extracted metadata is converted into a graph structure with multi-level constraints, modeled as nodes, edges, and attributes. Nodes serve as the basic units in the knowledge graph, representing head and tail entities. Edges represent relationships between nodes, while attributes provide more detailed information about nodes and edges, describing their characteristics or states.

[0049] Furthermore, this embodiment employs a dual mechanism of rule verification and large-scale model error correction to filter redundant information and complete missing relationships in the knowledge graph, ensuring the accuracy and completeness of the knowledge graph and providing a structured knowledge base for subsequent reasoning. Specifically, rule verification involves checking the generated triples based on preset knowledge graph construction rules. In this embodiment, the preset knowledge graph construction rules can include entity naming conventions, relationship rationality rules, etc. Large-scale model error correction utilizes the large-scale model's understanding of knowledge points to correct potentially erroneous or unreasonable triples. This mechanism filters out redundant information, such as repeated descriptions of the same entity relationships, and completes missing relationships, such as actual but unrecognized associations between two entities, providing the system with an accurate and complete structured knowledge base.

[0050] The adaptive planning generation module is used to receive the original complex engineering problem and perform in-depth analysis to generate a structured solution plan with knowledge tags.

[0051] In this embodiment, since large language models often possess superior planning capabilities compared to their simple reasoning capabilities when dealing with complex problems, the adaptive planning generation module leverages the large language model to generate a solution plan that does not involve specific data calculations. This solution plan decomposes the complex problem into a series of ordered and easier-to-handle sub-problems, thereby reducing the difficulty of solving the complex problem. Specifically, the adaptive planning generation module includes the following: The parsing unit is used to receive the original question input by the user. The problem is semantically parsed based on a Large Language Model (LLM) to extract its core elements, including known conditions, variables to be solved, and constraints. Then, the complexity of the problem is evaluated based on the metadata, and a problem complexity score is generated. Calculated using the following formula:

[0052] in, For the original problem Complexity score, To determine the number of macro-level steps required to solve the problem, The number of dependencies between steps. The weighting factor is the type of operation involved in the problem. , and They are respectively , and The corresponding weighting coefficients.

[0053] In this embodiment, when complexity scoring When the preset threshold is exceeded, subsequent planning units are initiated; when the complexity score is... If the preset threshold is not exceeded, the probabilistic search reasoning module is executed directly.

[0054] Planning units, as the generators of the solution program ( The core of ) is used to address the original problem. It can be broken down into an ordered sequence of subproblems.

[0055] In this embodiment, the Large Language Model (LLM), guided by the knowledge graph, transforms the original question... The problem is broken down and a series of macro-level solution steps in pseudocode form are generated, with each step defined as a subproblem. During the generation of each step, embedded tags are used to explicitly associate that step with one or more core theoretical nodes in the knowledge graph, forming knowledge tags. The final planning unit outputs the solution plan. It is a structured sequence, represented using the following logic:

[0056] in, For a complete solution plan, For the first A sub-problem, that is, a macro-level problem-solving step; In order to be with the first A set of knowledge tags associated with each sub-problem To solve the planning The total number of subproblems contained therein .

[0057] The adaptive planning generation module described in this embodiment, by parsing the problem's metadata and evaluating its complexity, can adaptively decompose complex problems into knowledge-tagged, structured solution plans (SPs). These solution plans are presented as a series of ordered hierarchical subproblems, and the knowledge points associated with each step in the plan are verified through knowledge path consistency checks to ensure that their logical order conforms to the sequential dependencies defined in the knowledge graph, thus guaranteeing logical rigor. The adaptive planning generation module not only effectively reduces the difficulty of solving complex problems but also provides a clear and reliable guiding framework for the subsequent probabilistic search and reasoning module, thereby significantly solving the problems of logical discrepancies and lack of constraints inherent in traditional large-scale models for solving complex engineering problems.

[0058] The probabilistic search reasoning module iterates through the Monte Carlo tree search algorithm and outputs a search tree containing multiple complete solution paths and corresponding evaluation information.

[0059] This embodiment applies the Monte Carlo Tree Search algorithm (hereinafter referred to as the MCTS algorithm) to the solution framework of a hierarchical subproblem. Each level in the search tree constructed in this embodiment corresponds to a macroscopic step (i.e., a subproblem) in the solution planning. Each layer contains multiple sibling nodes, representing various approaches to solving the same subproblem. This embodiment utilizes the MCTS algorithm for efficient exploration and evaluation of these different approaches. Its ultimate goal is to generate a search tree containing multiple high-quality candidate solution paths for the subsequent consistency verification module to make the optimal decision. Specifically, the probabilistic search inference module includes the following: In the selection phase, each iteration starts from the root node of the search tree and iterates through the sibling nodes of each level until a leaf node that has not yet been fully expanded is reached. To balance the preference for known efficient solutions and novel solutions, this embodiment uses the upper confidence bound formula (hereinafter referred to as the UCB formula) for node selection, using the following logical representation:

[0060] in, For nodes The value of the upper confidence level, For nodes The average reward value reflects the expected return that can be obtained by starting the simulation from this node; For nodes The number of visits to the node indicates the number of visits to the node. The total number of times the item was accessed during the entire search process; For nodes The parent node, The number of visits to the parent node reflects the depth of the search path exploration; The exploration coefficient is an adjustable hyperparameter used to control the weight of exploration.

[0061] At each level of the search tree, the system calculates the UCB value for all its sibling nodes (i.e., the multiple candidate solutions to the current subproblem). Then, the system greedily selects the node with the highest UCB value at the current level and enters that node, using it as the starting point for the next level's decision. This process is repeated level by level, starting from the root node, until a leaf node with no children or with unexpanded children is finally selected. In this way, the UCB value acts as a navigation signal for each decision step, guiding the entire selection process to form a path towards the frontier of exploration.

[0062] In the expansion and dynamic pruning phase, when the selection phase reaches a leaf node, the Large Language Model (LLM) generates multiple candidate decisions for solving the next sub-problem based on the method represented by the current node. At this point, the expansion and dynamic pruning phase begins and performs the following operations: First, using the constraints of the knowledge graph, all candidate decisions are dynamically pruned, filtering and eliminating all logically invalid decisions. This dynamic pruning includes semantic similarity calculation and logical consistency verification. Semantic similarity calculation involves converting the text content of the candidate decision and the descriptions of related nodes in the knowledge graph into word embedding vectors, and calculating the cosine similarity between them to determine their semantic relevance. Logical consistency verification utilizes the knowledge graph to check whether the candidate decision violates any known domain axioms or pre-existing dependencies.

[0063] Then, for each candidate decision that has been pruned, the system creates a new formal node for it and adds it to the search tree, thereby constructing a search space that is structurally sound and logically effective.

[0064] The simulation phase is used to form a complete simulation path. In this embodiment, a complete and lightweight simulation inference is performed by quickly going through the solution steps of all sub-problems. The core task of this phase is to quickly predict the quality of the final result that may be achieved starting from the current new node, so as to provide an overall reward signal for backpropagation.

[0065] When the search reaches a leaf node, a new reasoning decision extension search tree is generated using the large language model, and simulated reasoning is performed based on the structured knowledge of the knowledge graph. The feasibility scoring function is calculated as follows:

[0066] in, This is the feasibility scoring function value. To simulate the path, For the first The weight of each reasoning step is dynamically determined based on the importance, difficulty level, and dependencies of the knowledge points. For the first One reasoning step, For reasoning steps The matching degree with the knowledge point set KG in the knowledge graph is calculated using semantic embedding vector similarity. "time" indicates a complete match. This indicates a partial conflict. The time indicates a complete conflict. .

[0067] Furthermore, in obtaining a feasibility score Subsequently, the system uses this as a fundamental assessment of the overall path quality, combining it with other external assessment dimensions to calculate the final reward value for this simulation using the following logic:

[0068] in, This is the final reward value, which will be used in subsequent backpropagation phases; A positive constant bias term is used to ensure that when When the range of values ​​includes negative numbers, the base number inside the parentheses remains positive; The result self-validation score is used to measure the simulation path. To what extent does the final answer satisfy the inherent constraints of the original problem? The knowledge point coverage rate of the knowledge graph is used to measure the degree to which the reasoning process utilizes the knowledge graph. The efficiency of the reasoning steps can be considered in terms of the number of reasoning steps and computational complexity. , and They are respectively , and The corresponding reward value coefficient.

[0069] The backpropagation phase is used to convert the final reward value calculated in the simulation phase. Along the path traversed in this simulation Propagating backward from the leaf node to the root node, passing through every node along the path. The stored statistics will be updated according to the following logic: First, update the node's access count. :

[0070] in, For nodes Updated visit count For nodes The number of visits before the update; secondly, the average reward value of the updated node. :

[0071] in, For nodes Updated average reward value, For nodes The average reward value before the update. These updated statistics will directly affect the calculation of the subsequent UCB formula, thus intelligently guiding subsequent search choices.

[0072] In this embodiment, after all iterations of the Monte Carlo tree search are completed, a fully structured and explored search tree is finally output. This search tree contains multiple complete solution paths from the root node to the leaf node and the final answer corresponding to each leaf node. This search tree will be passed as a whole to the subsequent consistency verification module.

[0073] The consistency verification module verifies the answers to multiple complete problem-solving paths based on a self-consistency voting strategy, and selects the final candidate path set. .

[0074] In this embodiment, multiple candidate inference paths are first received from upstream, and the focus is on verifying the final results obtained from these paths. Specifically, a "majority voting" strategy is used to count the votes on the final answers obtained from all paths to determine the optimal answer that has the highest consensus and frequency of occurrence among multiple independent inferences. Subsequently, all inference paths that can lead to this optimal answer are filtered out to form a high-confidence candidate path set, which serves as the final output of the consistency verification module and is handed over to the subsequent progressive evaluation module for deeper path quality evaluation and optimal selection. This verification method effectively filters out isolated or erroneous answers generated during the inference process, providing a reliable target for determining the final solution. Specifically, the consistency verification module includes the following: The candidate answer extraction unit is used to identify all candidate answers from the root node to the leaf node from the search tree output by the probabilistic search reasoning module. A complete reasoning path constitutes the original path set. For each complete path Extract the final leaf node answer. Thus obtaining a containing A set of candidate answers , set of answers Passed to the self-consistency voting unit. .

[0075] The self-consistent voting unit, as the core of the consistency verification module, is used to evaluate the extracted candidate answer set. Conduct voting statistics to determine the answer with the highest consensus, specifically including the following: Define an indicator function To determine whether two answers are consistent, the following logical expression can be used:

[0076] in, It is a set The first in 10 candidate answers It is a unique candidate answer. Indicates when When it is equivalent to a, The output value is 1; Indicates when When not equivalent to a, The output value is 0; in this embodiment, the equivalence judgment can be adapted according to the question type. For example, for numerical answers, it can be exact equality, and for textual answers, it can be semantic equivalence.

[0077] Based on the above indicator function Calculate each unique answer The frequency of occurrence is recorded as the number of votes. This can be represented using the following logic:

[0078] in, For unique answers In all The total number of times each candidate answer appears in the n-10 ... express The cumulative value of each candidate answer.

[0079] The following logic is used to determine the optimal answer that receives the most votes among all candidate answers. :

[0080] in, This represents the answer with the highest degree of consensus. Indicates from all possible Find the one in The parameter with the largest function value .

[0081] Candidate path filtering unit, used to filter from the original path set Filter out all results that lead to the optimal answer. The inference path is used to generate a final candidate path set containing multiple paths. This can be represented using the following logic:

[0082] in, For the final set of candidate paths, For the original path set The first in Path, For path The corresponding final answer. In this embodiment, the set of candidate paths containing the highest consensus answer provides a highly reliable input for the final decision.

[0083] The progressive evaluation module is used to evaluate the final candidate path set. The selected paths undergo node-by-node confidence evaluation, and a unique optimal solution path is selected based on the evaluation results. Specifically, the progressive evaluation module includes the following: Node confidence quantification scoring unit, used to traverse the candidate path set. For each path in the code, and for each node on the path... The node confidence is calculated through a dual verification mechanism that combines reward model evaluation and executable code verification. This includes the following: First, an independent reward model evaluates the logical quality of the nodes. This reward model can be any specially fine-tuned language model distinct from the main inference language model. The evaluation score... As shown in the following formula:

[0084] in, This indicates that the reward model applies to nodes. The rating, with a value range that can be normalized to... ; Represents a reward model. Represents a node The textual description of the reasoning steps represented. Represents a node The state information of the parent node is used to provide context for evaluating the score.

[0085] Secondly, for nodes that contain computable steps... The system calculates the verification score by calling an executable Python code interpreter. To verify the correctness of its calculation, the following logical representation can be used:

[0086] in, This represents the verification score of the executable code. This indicates the process of executing the code interpreter. Represents the node The executable code snippet extracted from the description, This indicates that a node is considered complete when an executable code segment can be executed by the interpreter without errors. Validation score =1; This indicates when an executable code segment is interrupted during execution and throws an exception, the node... Validation score The value is 0. In this embodiment, for nodes that do not contain computable steps, It can be set to a neutral default value.

[0087] Finally, the two scores are weighted and summed to obtain the node confidence score. :

[0088] in, For nodes The final confidence score, and These are the weighting coefficients corresponding to the evaluation score and the verification score, respectively.

[0089] The optimal path selection unit is used to calculate the confidence scores of all nodes on all candidate paths. Next, the normalized total path confidence is used to calculate the total confidence of each path. And select the one with the highest score; for the path All nodes This can be represented using the following logic:

[0090] for All paths :

[0091] in, This indicates the final selected solution path. For path Normalized path confidence, For path Length, This indicates finding the path corresponding to the normalized path confidence.

[0092] In this embodiment, the progressive evaluation module performs a refined evaluation of the candidate path set, and the final output is not only a correct answer verified by most inferences, but also a complete reasoning process that supports the answer, has undergone node-by-node quality evaluation, and is logically the most robust.

[0093] The high-quality data training module is used to collect path samples and filter high-quality training data, and optimize the model parameters of the large language model based on supervised fine-tuning and reinforcement learning.

[0094] In this embodiment, the main task of the high-quality data training module is to collect and label the decision data and solution samples generated by any preceding module during the reasoning process, and then fine-tune the parameters of the system's built-in Large Language Model (LLM) using a training paradigm. This allows the system to learn from successful problem-solving experiences and draw lessons from failed attempts, thereby continuously improving its autonomous reasoning and problem-solving capabilities. Specifically, the high-quality data training module includes the following: The high-quality training data collection and classification phase involves collecting all path samples processed by any preceding module. Based on the course learning approach, and combining the self-consistency and normalized total confidence of the path samples calculated in the consistency verification and progressive evaluation modules, all path samples are classified. The classification is performed. In this embodiment, a high confidence threshold is preset. and a medium confidence threshold To quantify and classify paths, including: Expert-level samples include a set of paths that simultaneously satisfy high self-consistency and high process confidence; specifically, the final answer of the path. It must be equal to the optimal answer determined by the consistency verification module. Furthermore, the total confidence of the normalized path is higher than the high confidence threshold. , recorded as > .

[0095] Ordinary positive samples include the following: (1) high self-consistency, denoted as = ,but Paths at a medium level are denoted as < ≤ , (2) Low self-consistency is denoted as . ,but Very high paths are denoted as > Among them, the high confidence threshold With medium confidence threshold The experiment was set up according to actual needs. These ordinary positive samples represent valuable, but not optimal, reasoning attempts.

[0096] Negative samples, including the total confidence of the normalized path. The lower path is denoted as ≤ Self-consistent results are not considered. This sample is designed to penalize all paths with obvious flaws in their reasoning process, even if they happen to arrive at a consensus answer.

[0097] The supervised fine-tuning phase uses expert-level samples as training data to ensure that the model first imitates and masters the highest quality problem-solving paradigms that combine process reliability and result consensus. Each optimal path... They are all converted into a "command-response" pair ,in, For the original question, Optimal path The text representing the complete, step-by-step reasoning process. Based on the objective function. Training is performed to minimize the difference between the model's generated response and the expert-level sample, using the following logical representation:

[0098] in, To supervise the fine-tuning of the loss function, The parameters representing the large language model. express The first in the text Each word element, Indicates the first All lexical sequences preceding each lexical unit This represents the probability that the model will generate the next word given the preceding context. By minimizing the loss function, the model can quickly imitate and master correct, high-quality problem-solving paradigms, laying a solid foundation for subsequent exploratory learning.

[0099] The reinforcement learning phase based on curriculum learning is used to initiate the reinforcement learning phase after the LLM has been pre-trained by supervised fine-tuning (SFT). Based on the reinforcement learning algorithm (PPO), the parameters of the large language model are further optimized to improve the model's exploration and decision-making capabilities.

[0100] First, a reward function needs to be defined. For all path samples Assign a quantified training reward, the definition of which is directly related to the course level of the path, using the following logical representation:

[0101] in, Indicates when path sample When the sample is expert-level, the reward function Pick , For the highest reward (e.g.) (for +1.0). Indicates when path sample When the sample is a normal positive sample, the reward function is... Pick , For smaller positive rewards (e.g.) (for +0.2). Indicates when path sample When the sample is negative, the reward function Pick , For punishment (e.g.) (The value is -1.0). This embodiment defines a reward function. The reward allocation method affirms the rigorous reasoning process without excessively encouraging deviations from mainstream consensus, thus achieving a balance between the accuracy and creativity of the model.

[0102] Secondly, adopting a terminal reward distribution strategy will Transformed into each time step Instant rewards For a path consisting of T steps The following logic represents instant rewards. Allocation method:

[0103] in, Indicates the time steps in the path. , Representing a path The last time step.

[0104] Then, the parameters of the Large Language Model (LLM) are processed using a reinforcement learning algorithm (PPO). Optimization is performed. The PPO algorithm calculates the advantage function. To evaluate the merits of taking a certain action in a specific state, the advantage function is calculated based on the immediate reward as defined above. and state value function This can be represented using the following logic:

[0105] in, In time step The advantage function, As a discount factor, The value function is used to estimate the state. The cumulative rewards that may be obtained in the future For the next time step Corresponding state The cumulative rewards that may be obtained in the future.

[0106] Based on the calculated advantage function The PPO algorithm updates model parameters using a pruning objective function to ensure training stability. The objective function can be represented by the following logic. :

[0107] in, Let PPO be the objective function to be optimized. This represents the expected value along the policy trajectory. The ratio of the new to the old strategies. , Indicates the parameters of the current model The defined new strategy This refers to the old strategy used to collect training data before the start of this optimization iteration; In time step The advantage function represents the advantage of taking a certain action compared to the average level in a given state. For the pruning function, limit the policy ratio to Within the specified range, to ensure training stability. This is a hyperparameter for the clipping range.

[0108] In this embodiment, by first supervising and fine-tuning the imitation expert and then exploring the curriculum based on reinforcement learning, the system can optimize the model parameters of the large language model more stably and efficiently. The updated large language model will be deployed back to the probabilistic search and reasoning module, enabling the model to continuously adapt to new teaching content and methods, reduce reliance on human intervention, achieve autonomous performance improvement, and gradually enhance the knowledge depth and reasoning reliability of the large language model, thereby demonstrating stronger performance in the next problem-solving task and forming a complete adaptive learning closed loop of "generation-verification-evaluation-optimization".

[0109] This invention discloses a guided reasoning system based on a dual-drive approach of large-scale models and knowledge graphs. As a novel technical architecture integrating dynamic programming from large-scale models and structured verification from knowledge graphs, its fundamental purpose is to establish a collaborative mechanism between dynamic programming and rigorous verification, thereby significantly improving the reliability of solving complex problems and providing interpretable reasoning support for adaptive learning systems. Through the organic combination of pseudocode-level planning generated by large language models and structured verification from knowledge graphs, it achieves intelligent solving of complex problems in emerging engineering disciplines, providing key technical support for intelligent education.

[0110] This invention also provides a guided reasoning method based on a dual-drive approach of large models and knowledge graphs, comprising the following steps: S1 extracts metadata from the course's teaching materials, including core elements of the questions, prerequisite knowledge, and common mistakes.

[0111] In step S1, the teaching materials include textbooks, exercise sets, and lab manuals, etc. Taking the Computer Organization Principles course as an example, the course content is intelligently processed to extract high-level information from the teaching materials as metadata. Specifically, S1 includes the following: First, multi-level text analysis technology is used to capture key knowledge points and the relationships between them in the teaching materials as core elements of the questions. These core elements specifically include known conditions, variables to be solved, and constraints. In this embodiment, the core elements can be floating-point arithmetic, two's complement representation, etc. Specifically, the multi-level text analysis technology utilizes Named Entity Recognition (NER) and Relation Extraction (RE) techniques to determine the relationships between these entities, thereby transforming unstructured text into structured core elements.

[0112] Secondly, the system identifies the implicit prerequisite knowledge and potential pitfalls within the test questions. Then, it utilizes the summarizing capabilities of a large language model to generalize and summarize the identified content, forming the prior knowledge of the questions as prerequisite knowledge and a common error analysis. In this embodiment, the prerequisite knowledge specifically refers to axioms, theorems, and other course knowledge points essential for solving the problem; the common error analysis specifically identifies potential computational pitfalls or conceptual misunderstandings within the problem. For example, for test questions involving the conversion of two's complement to original code, the system summarizes key information such as the steps involved in this conversion based on the large language model, providing a basis for subsequent processing of the test questions based on prior knowledge.

[0113] Furthermore, during the process of summarizing and generalizing using a large language model, inaccurate or redundant information may arise, requiring subsequent manual review. To reduce the difficulty of manual review and improve the accuracy of summarization, this embodiment incorporates Python programming for auxiliary processing. The test questions are preprocessed to remove obvious interfering information, and then the processed content is input into the large language model for summarization.

[0114] S2 converts the extracted metadata into a structured knowledge graph.

[0115] In step S2, based on Natural Language Processing (NLP) technology and the semantic understanding capabilities of Large Language Model (LLM), entities and relationships between entities in the metadata are identified; the identified entities and relationships are converted into triples for storage, and a high-quality metadata dataset covering the entire knowledge system of the course is constructed, transforming the metadata from unstructured text into a structured knowledge graph; the triples are represented as <head entity, relation, tail entity> or <entity, attribute, attribute value>. In this embodiment, the extracted metadata is converted into a graph structure with multi-level constraints, which are modeled as nodes, edges, and attributes, respectively. Nodes serve as the basic units in the knowledge graph to represent head entities and tail entities, edges are used to represent the relationships between nodes, and attributes provide more detailed information about nodes and edges to describe the characteristics or states of nodes and edges.

[0116] Furthermore, this embodiment employs a dual mechanism of rule verification and large-scale model error correction to filter redundant information and complete missing relationships in the knowledge graph, ensuring the accuracy and completeness of the knowledge graph and providing a structured knowledge base for subsequent reasoning. Specifically, rule verification involves checking the generated triples based on preset knowledge graph construction rules. In this embodiment, the preset knowledge graph construction rules can include entity naming conventions, relationship rationality rules, etc. Large-scale model error correction utilizes the large-scale model's understanding of knowledge points to correct potentially erroneous or unreasonable triples. This mechanism filters out redundant information, such as repeated descriptions of the same entity relationships, and completes missing relationships, such as actual but unrecognized associations between two entities, providing the system with an accurate and complete structured knowledge base.

[0117] S3 receives the original complex engineering problem and performs in-depth analysis to generate a structured solution plan with knowledge tags.

[0118] In this embodiment, since the planning ability of a large language model is often superior to its simple reasoning ability when dealing with complex problems, S3 uses the large language model to generate a solution plan that does not involve specific data calculation processes. This solution plan decomposes the complex problem into a series of ordered and easier-to-handle sub-problems, thereby reducing the difficulty of solving the complex problem. Specifically, S3 includes the following: S31, used to receive the original question input by the user. The problem is semantically parsed based on a Large Language Model (LLM) to extract its core elements, including known conditions, variables to be solved, and constraints. Then, the complexity of the problem is evaluated based on the metadata, and a problem complexity score is generated. Calculated using the following formula:

[0119] in, For the original problem Complexity score, To determine the number of macro-level steps required to solve the problem, The number of dependencies between steps. The weighting factor is the type of operation involved in the problem. , and They are respectively , and The corresponding weighting coefficients.

[0120] In this embodiment, when complexity scoring When the preset threshold is exceeded, subsequent planning units are initiated; when the complexity score is... If the preset threshold is not exceeded, execute S4 directly.

[0121] S32, as the generating solution program ( The core of ) is used to address the original problem. It can be broken down into an ordered sequence of subproblems.

[0122] In this embodiment, the Large Language Model (LLM), guided by the knowledge graph, transforms the original question... The problem is broken down and a series of macro-level solution steps in pseudocode form are generated, with each step defined as a subproblem. During the generation of each step, embedded tags are used to explicitly associate that step with one or more core theoretical nodes in the knowledge graph, forming knowledge tags. The final planning unit outputs the solution plan. It is a structured sequence, represented using the following logic:

[0123] in, For a complete solution plan, For the first A sub-problem, that is, a macro-level problem-solving step; In order to be with the first A set of knowledge tags associated with each sub-problem To solve the planning The total number of subproblems contained therein .

[0124] The S3 method described in this embodiment, by parsing the problem's metadata and evaluating its complexity, adaptively decomposes complex problems into knowledge-tagged, structured solution plans (SPs). These solution plans are presented as a series of ordered hierarchical subproblems, and the knowledge points associated with each step in the plan are verified through knowledge path consistency checks to ensure that their logical order conforms to the sequential dependencies defined in the knowledge graph, thus guaranteeing logical rigor. S3 not only effectively reduces the difficulty of solving complex problems but also provides a clear and reliable guiding framework for the subsequent S4 method, thereby significantly solving the problems of logical discrepancies and lack of constraints inherent in traditional large-scale models for solving complex engineering problems.

[0125] S4 iterates through the Monte Carlo tree search algorithm to output a search tree containing multiple complete solution paths and corresponding evaluation information.

[0126] In step S4, this embodiment applies the Monte Carlo Tree Search algorithm (hereinafter referred to as the MCTS algorithm) to the solution framework of a hierarchical subproblem. Each level in the search tree constructed in this embodiment corresponds to a macroscopic step (i.e., a subproblem) in the solution planning. Each layer contains multiple sibling nodes, representing various methods for solving the same subproblem. This embodiment utilizes the MCTS algorithm to efficiently explore and evaluate these different methods. Its ultimate goal is to generate a search tree containing multiple high-quality candidate solution paths for optimal decision-making in subsequent step S5. Specifically, step S4 includes the following: S41, in each iteration, starting from the root node of the search tree, the sibling nodes of each level are traversed step by step until a leaf node that has not yet been fully expanded is reached. To balance the preference for known efficient solutions and novel solutions, this embodiment uses the upper confidence bound formula (hereinafter referred to as the UCB formula) for node selection, using the following logical representation:

[0127] in, For nodes The value of the upper confidence level, For nodes The average reward value reflects the expected return that can be obtained by starting the simulation from this node; For nodes The number of visits to the node indicates the number of visits to the node. The total number of times the item was accessed during the entire search process; For nodes The parent node, The number of visits to the parent node reflects the depth of the search path exploration; The exploration coefficient is an adjustable hyperparameter used to control the weight of exploration.

[0128] At each level of the search tree, the system calculates the UCB value for all its sibling nodes (i.e., the multiple candidate solutions to the current subproblem). Then, the system greedily selects the node with the highest UCB value at the current level and enters that node, using it as the starting point for the next level's decision. This process is repeated level by level, starting from the root node, until a leaf node with no children or with unexpanded children is finally selected. In this way, the UCB value acts as a navigation signal for each decision step, guiding the entire selection process to form a path towards the frontier of exploration.

[0129] In step S42, when step S41 reaches a leaf node, the Large Language Model (LLM) generates multiple candidate decisions for solving the next sub-problem based on the method represented by the current node. At this point, step S42 is initiated and performs the following operations: First, using the constraints of the knowledge graph, all candidate decisions are dynamically pruned, filtering and eliminating all logically invalid decisions. This dynamic pruning includes semantic similarity calculation and logical consistency verification. Semantic similarity calculation involves converting the text content of the candidate decision and the descriptions of related nodes in the knowledge graph into word embedding vectors, and calculating the cosine similarity between them to determine their semantic relevance. Logical consistency verification utilizes the knowledge graph to check whether the candidate decision violates any known domain axioms or pre-existing dependencies.

[0130] Then, for each candidate decision that has been pruned, the system creates a new formal node for it and adds it to the search tree, thereby constructing a search space that is structurally sound and logically effective.

[0131] S43 is used to form a complete simulation path. In this embodiment, by quickly completing the solution steps of all sub-problems, a complete and lightweight simulation reasoning is performed. The core task of this stage is to quickly predict the quality of the final result that may be achieved starting from the current new node, so as to provide an overall reward signal for backpropagation.

[0132] When the search reaches a leaf node, a new reasoning decision extension search tree is generated using the large language model, and simulated reasoning is performed based on the structured knowledge of the knowledge graph. The feasibility scoring function is calculated as follows:

[0133] in, This is the feasibility scoring function value. To simulate the path, For the first The weight of each reasoning step is dynamically determined based on the importance, difficulty level, and dependencies of the knowledge points. For the first One reasoning step, For reasoning steps The matching degree with the knowledge point set KG in the knowledge graph is calculated using semantic embedding vector similarity. "time" indicates a complete match. This indicates a partial conflict. The time indicates a complete conflict. .

[0134] Furthermore, in obtaining a feasibility score Subsequently, the system uses this as a fundamental assessment of the overall path quality, combining it with other external assessment dimensions to calculate the final reward value for this simulation using the following logic:

[0135] in, This is the final reward value, which will be used in subsequent S44. A positive constant bias term is used to ensure that when When the range of values ​​includes negative numbers, the base number inside the parentheses remains positive; The result self-validation score is used to measure the simulation path. To what extent does the final answer satisfy the inherent constraints of the original problem? The knowledge point coverage rate of the knowledge graph is used to measure the degree to which the reasoning process utilizes the knowledge graph. The efficiency of the reasoning steps can be considered in terms of the number of reasoning steps and computational complexity. , and They are respectively , and The corresponding reward value coefficient.

[0136] S44 is used to calculate the final reward value in S43. Along the path traversed in this simulation Propagating backward from the leaf node to the root node, passing through every node along the path. The stored statistics will be updated according to the following logic: First, update the node's access count. :

[0137] in, For nodes Updated visit count For nodes The number of visits before the update; secondly, the average reward value of the updated node. :

[0138] in, For nodes Updated average reward value, For nodes The average reward value before the update. These updated statistics will directly affect the calculation of the subsequent UCB formula, thus intelligently guiding subsequent search choices.

[0139] In this embodiment, after all iterations of the Monte Carlo tree search are completed, a fully structured and explored search tree is finally output. This search tree contains multiple complete solution paths from the root node to the leaf node and the final answer corresponding to each leaf node. This search tree will be passed as a whole to the subsequent S5.

[0140] S5, based on a self-consistent voting strategy, verifies the answers to multiple complete problem-solving paths and selects the final candidate path set. S5 includes the following: S51 is used to identify all nodes from the root node to the leaf node from the search tree output by S4. A complete reasoning path constitutes the original path set. For each complete path Extract the final leaf node answer. Thus obtaining a containing A set of candidate answers , set of answers Passed to S52, .

[0141] S52, as the core of S5, is used to process the extracted candidate answer set. Conduct voting statistics to determine the answer with the highest consensus, specifically including the following: Define an indicator function To determine whether two answers are consistent, the following logical expression can be used:

[0142] in, It is a set The first in 10 candidate answers It is a unique candidate answer. Indicates when When it is equivalent to a, The output value is 1; Indicates when When not equivalent to a, The output value is 0; in this embodiment, the equivalence judgment can be adapted according to the question type. For example, for numerical answers, it can be exact equality, and for textual answers, it can be semantic equivalence.

[0143] Based on the above indicator function Calculate each unique answer The frequency of occurrence is recorded as the number of votes. This can be represented using the following logic:

[0144] in, For unique answers In all The total number of times each candidate answer appears in the n-10 ... express The cumulative value of each candidate answer.

[0145] The following logic is used to determine the optimal answer that receives the most votes among all candidate answers. :

[0146] in, This represents the answer with the highest degree of consensus. Indicates from all possible Find the one in The parameter with the largest function value .

[0147] S53, used from the original path set Filter out all results that lead to the optimal answer. The inference path is used to generate a final candidate path set containing multiple paths. This can be represented using the following logic:

[0148] in, For the final set of candidate paths, For the original path set The first in Path, For path The corresponding final answer. In this embodiment, the set of candidate paths containing the highest consensus answer provides a highly reliable input for the final decision.

[0149] S6, based on the final candidate path set The selected paths undergo node-by-node confidence evaluation, and a unique optimal solution path is selected based on the evaluation results. Specifically, S6 includes the following: S61 is used to traverse the candidate path set. For each path in the code, and for each node on the path... The node confidence is calculated through a dual verification mechanism that combines reward model evaluation and executable code verification. This includes the following: First, an independent reward model evaluates the logical quality of the nodes. This reward model can be any specially fine-tuned language model distinct from the main inference language model. The evaluation score... As shown in the following formula:

[0150] in, This indicates that the reward model applies to nodes. The rating, with a value range that can be normalized to... ; Represents a reward model. Represents a node The textual description of the reasoning steps represented. Represents a node The state information of the parent node is used to provide context for evaluating the score.

[0151] Secondly, for nodes that contain computable steps... The system calculates the verification score by calling an executable Python code interpreter. To verify the correctness of its calculation, the following logical representation can be used:

[0152] in, This represents the verification score of the executable code. This indicates the process of executing the code interpreter. Represents the node The executable code snippet extracted from the description, This indicates that a node is considered complete when an executable code segment can be executed by the interpreter without errors. Validation score =1; This indicates when an executable code segment is interrupted during execution and throws an exception, the node... Validation score The value is 0. In this embodiment, for nodes that do not contain computable steps, It can be set to a neutral default value.

[0153] Finally, the two scores are weighted and summed to obtain the node confidence score. :

[0154] in, For nodes The final confidence score, and These are the weighting coefficients corresponding to the evaluation score and the verification score, respectively.

[0155] S62 is used to calculate the confidence level of all nodes on all candidate paths. Next, the normalized total path confidence is used to calculate the total confidence of each path. And select the one with the highest score; for the path All nodes This can be represented using the following logic:

[0156] for All paths :

[0157] in, This indicates the final selected solution path. For path Normalized path confidence, For path Length, This indicates finding the path corresponding to the normalized path confidence.

[0158] S7 involves collecting path samples and filtering high-quality training data to optimize the model parameters of the large language model based on supervised fine-tuning and reinforcement learning. Specifically, S7 includes the following: S71 is used to collect all path samples processed by any of the preceding steps; based on the course learning ideas, it combines the self-consistency of path samples and the normalized total confidence of the paths to process all path samples. The classification is performed. In this embodiment, a high confidence threshold is preset. and a medium confidence threshold To quantify and classify paths, including: Expert-level samples include a set of paths that simultaneously satisfy high self-consistency and high process confidence; specifically, the final answer of the path. It must be equal to the optimal answer determined by S5. Furthermore, the total confidence of the normalized path is higher than the high confidence threshold. , recorded as > .

[0159] Ordinary positive samples include the following: (1) high self-consistency, denoted as = ,but Paths at a medium level are denoted as < ≤ , (2) Low self-consistency is denoted as . ,but Very high paths are denoted as > Among them, the high confidence threshold With medium confidence threshold The experiment was set up according to actual needs. These ordinary positive samples represent valuable, but not optimal, reasoning attempts.

[0160] Negative samples, including the total confidence of the normalized path. The lower path is denoted as ≤ Self-consistent results are not considered. This sample is designed to penalize all paths with obvious flaws in their reasoning process, even if they happen to arrive at a consensus answer.

[0161] S72 is used to employ expert-level samples as training data to ensure that the model first imitates and masters the highest quality problem-solving paradigms that combine process reliability and result consensus. Each optimal path... They are all converted into a "command-response" pair ,in, For the original question, Optimal path The text representing the complete, step-by-step reasoning process. Based on the objective function. Training is performed to minimize the difference between the model's generated response and the expert-level sample, using the following logical representation:

[0162] in, To supervise the fine-tuning of the loss function, The parameters representing the large language model. express The first in the text Each word element, Indicates the first All lexical sequences preceding each lexical unit This represents the probability that the model will generate the next word given the preceding context. By minimizing the loss function, the model can quickly imitate and master correct, high-quality problem-solving paradigms, laying a solid foundation for subsequent exploratory learning.

[0163] S73 is used to initiate the reinforcement learning phase after the LLM has been pre-trained by supervised fine-tuning (SFT). Based on the reinforcement learning algorithm (PPO), it further optimizes the parameters of the large language model to improve the model's exploration and decision-making capabilities.

[0164] First, a reward function needs to be defined. For all path samples Assign a quantified training reward, the definition of which is directly related to the course level of the path, using the following logical representation:

[0165] in, Indicates when path sample When the sample is expert-level, the reward function Pick , For the highest reward (e.g.) (for +1.0). Indicates when path sample When the sample is a normal positive sample, the reward function is... Pick , For smaller positive rewards (e.g.) (for +0.2). Indicates when path sample When the sample is negative, the reward function Pick , For punishment (e.g.) (The value is -1.0). This embodiment defines a reward function. The reward allocation method affirms the rigorous reasoning process without excessively encouraging deviations from mainstream consensus, thus achieving a balance between the accuracy and creativity of the model.

[0166] Secondly, adopting a terminal reward distribution strategy will Transformed into each time step Instant rewards For a path consisting of T steps The following logic represents instant rewards. Allocation method:

[0167] in, Indicates the time steps in the path. , Representing a path The last time step.

[0168] Then, the parameters of the Large Language Model (LLM) are processed using a reinforcement learning algorithm (PPO). Optimization is performed. The PPO algorithm calculates the advantage function. To evaluate the merits of taking a certain action in a specific state, the advantage function is calculated based on the immediate reward as defined above. and state value function This can be represented using the following logic:

[0169] in, In time step The advantage function, As a discount factor, The value function is used to estimate the state. The cumulative rewards that may be obtained in the future For the next time step Corresponding state The cumulative rewards that may be obtained in the future.

[0170] Based on the calculated advantage function The PPO algorithm updates model parameters using a pruning objective function to ensure training stability. The objective function can be represented by the following logic. :

[0171] in, Let PPO be the objective function to be optimized. This represents the expected value along the policy trajectory. The ratio of the new to the old strategies. , Indicates the parameters of the current model The defined new strategy This refers to the old strategy used to collect training data before the start of this optimization iteration; In time step The advantage function represents the advantage of taking a certain action compared to the average level in a given state. For the pruning function, limit the policy ratio to Within the specified range, to ensure training stability. This is a hyperparameter for the clipping range.

[0172] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A guided reasoning system based on a dual-drive approach of large models and knowledge graphs, characterized in that: include: The metadata extraction module is used to extract metadata from the course's teaching materials, including core elements of the questions, prerequisite knowledge, and analysis of common mistakes. The knowledge graph construction module is used to convert extracted metadata into a structured knowledge graph. The adaptive planning generation module is used to receive the original complex engineering problem, perform in-depth analysis, and generate a structured solution plan with knowledge tags. The probabilistic search reasoning module uses the Monte Carlo tree search algorithm to perform iterative loops and outputs a search tree containing multiple complete solution paths and corresponding evaluation information. The consistency verification module verifies the answers to multiple complete problem-solving paths based on a self-consistency voting strategy, and selects the final candidate path set. The progressive evaluation module is used to perform node-by-node confidence evaluation on the paths selected from the final candidate path set, and select the best solution path based on the evaluation results. The high-quality data training module is used to collect path samples and filter high-quality training data, and optimize the model parameters of the large language model based on supervised fine-tuning and reinforcement learning.

2. The guided reasoning system based on a large model and knowledge graph dual-drive as described in claim 1, characterized in that, The core elements of the question described in the metadata extraction module include known conditions, variables to be determined, and constraints. The aforementioned prerequisite knowledge specifically refers to the axioms, theorems, and other course knowledge points that are essential for solving the problem; The analysis of common pitfalls specifically refers to computational traps or conceptual misunderstandings present in the problem.

3. The guided reasoning system based on a large model and knowledge graph dual-drive as described in claim 1, characterized in that, The adaptive planning generation module includes: The parsing unit is used to receive the original question input by the user. Based on a large language model, semantic parsing of the problem is performed to extract its core elements. The complexity of the problem is evaluated based on metadata, using the following logical representation: in, For the original problem Complexity score, To determine the number of macro-level steps required to solve the problem, The number of dependencies between steps. The weighting factor is the type of operation involved in the problem. , and They are respectively , and The corresponding weighting coefficients; When complexity score When the preset threshold is exceeded, subsequent planning units are initiated; when the complexity score is... If the preset threshold is not exceeded, the probabilistic search reasoning module is executed directly; Planning unit, used to process the original problem The problem is broken down and a series of macro-level solution steps in pseudocode form are generated, with each step defined as a subproblem. During the generation of each step, embedded tags are used to explicitly associate the step with one or more core theoretical nodes in the knowledge graph, forming knowledge tags and outputting the solution plan. This can be represented using the following logic: in, For a complete solution plan, For the first Each sub-problem represents a macro-level problem-solving step; In order to be with the first A set of knowledge tags associated with each sub-problem To solve the planning The total number of subproblems contained therein. .

4. The guided reasoning system based on a large model and knowledge graph dual-drive as described in claim 3, characterized in that, The probabilistic search reasoning module includes the following: In the selection phase, each iteration starts from the root node of the search tree and iterates through the sibling nodes of each level until a leaf node that has not yet been fully expanded is reached. Node selection is performed based on the upper confidence bound formula, using the following logical representation: in, For nodes The value of the upper confidence level, For nodes The average reward value, For nodes Number of visits, For nodes The parent node, This represents the number of times the parent node has been visited. For the exploration coefficient; During the expansion and dynamic pruning phase, when a leaf node is reached, the large language model generates multiple candidate decisions for solving the next sub-problem based on the method represented by the current node. Using the constraints of the knowledge graph, all candidate decisions are dynamically pruned, filtering out and eliminating all logically invalid decisions. This dynamic pruning includes semantic similarity calculation and logical consistency verification. For each candidate decision that passes pruning, the system creates a new formal node for it and adds it to the search tree. In the simulation phase, a complete simulation path is formed. When a leaf node is reached, a new inference decision-making extension search tree is generated using the large language model. Simulated reasoning is then performed based on the structured knowledge of the knowledge graph, and the feasibility scoring function is calculated as follows: in, This is the feasibility scoring function value. To simulate the path, For the first The weight of each reasoning step, For the first One reasoning step, For reasoning steps The matching degree with the knowledge point set KG in the knowledge graph is calculated using semantic embedding vector similarity. "time" indicates a complete match. This indicates a partial conflict. The time indicates a complete conflict. ; Feasibility rating As a fundamental assessment of the overall quality of the path, the final reward value for this simulation is calculated by combining it with other external evaluation dimensions and using the following logical representation: in, This is the final reward value, which will be used in subsequent backpropagation phases; A positive constant bias term is used to ensure that when When the range of values ​​includes negative numbers, the base number inside the parentheses remains positive; The result self-validation score is used to measure the simulation path. To what extent does the final answer satisfy the inherent constraints of the original problem? The knowledge point coverage rate of the knowledge graph is used to measure the degree to which the reasoning process utilizes the knowledge graph. The efficiency of the reasoning steps can be considered in terms of the number of reasoning steps and computational complexity. , and They are respectively , and The corresponding reward value coefficient; The backpropagation phase is used to convert the final reward value calculated in the simulation phase. Along the path traversed in this simulation Propagating backward from the leaf node to the root node, passing through every node along the path. The stored statistics will be updated according to the following logic: First, update the node's access count. : in, For nodes Updated visit count For nodes The number of visits before the update; secondly, the average reward value of the updated node. : in, For nodes Updated average reward value, For nodes Average reward value before the update; After all iterations of the Monte Carlo tree search are completed, the final output is a search tree containing multiple complete solution paths and corresponding evaluation information.

5. The guided reasoning system based on a large model and knowledge graph dual-drive as described in claim 4, characterized in that, The consistency verification module includes: The candidate answer extraction unit is used to identify all candidate answers from the root node to the leaf node from the search tree output by the probabilistic search reasoning module. A complete reasoning path constitutes the original path set. For each complete path Extract the final leaf node answer. Thus obtaining a containing A set of candidate answers , set of answers Passed to the self-consistency voting unit. ; Self-consistent voting units are used to evaluate the extracted set of candidate answers. Conduct voting statistics to determine the answer with the highest consensus, specifically including the following: Define an indicator function To determine whether two answers are consistent, the following logical expression can be used: in, It is a set The first in 10 candidate answers It is a unique candidate answer; based on the above indicator function. Calculate each unique answer The frequency of occurrence is recorded as the number of votes. This can be represented using the following logic: in, For unique answers In all The total number of times each candidate answer appears in the n-10 ... express The cumulative value of each candidate answer; The following logic is used to determine the optimal answer that receives the most votes among all candidate answers. : in, This represents the answer with the highest degree of consensus. Indicates from all possible Find the one in The parameter with the largest function value ; Candidate path filtering unit, used to filter from the original path set Filter out all results that lead to the optimal answer. The reasoning path is used to generate the final candidate path set. This can be represented using the following logic: in, For the final set of candidate paths, For the original path set The first in Path, For path The corresponding final answer.

6. The guided reasoning system based on a large model and knowledge graph dual-drive as described in claim 5, characterized in that, The progressive evaluation module includes: Node confidence quantification scoring unit, used to traverse the candidate path set. For each path in the code, and for each node on the path... The node confidence is calculated through a dual verification mechanism that combines reward model evaluation and executable code verification. ; The optimal path selection unit is used to calculate the normalized total path confidence for each path. And select the one with the highest score; for the path all nodes This can be represented using the following logic: for All paths : in, This indicates the final selected solution path. For path Normalized path confidence, For path Length, This indicates finding the path corresponding to the normalized path confidence.

7. The guided reasoning system based on a large model and knowledge graph dual-drive as described in claim 6, characterized in that, The confidence level Using the following logical representation: First, an independent reward model evaluates and scores the logical quality of the nodes. As shown in the following formula: in, This indicates that the reward model applies to nodes. The rating, with a value range that can be normalized to... ; Represents a reward model. Represents a node The textual description of the reasoning steps represented. Represents a node The status information of the parent node; Secondly, for nodes containing computable steps The verification score is calculated by calling an executable Python code interpreter. To verify the correctness of its calculation, the following logical representation can be used: in, This represents the verification score of the executable code. This indicates the process of executing the code interpreter. Represents the node The executable code snippet extracted from the description, This indicates that a node is considered complete when an executable code segment can be executed by the interpreter without errors. Validation score =1; This indicates when an executable code snippet is interrupted during execution and throws an exception, the node... Validation score =0; Finally, the scores will be evaluated. With verification score Perform a weighted summation to obtain the node confidence score of that node. : in, For nodes The final confidence score, and These are the weighting coefficients corresponding to the evaluation score and the verification score, respectively.

8. The guided reasoning system based on a large model and knowledge graph dual-drive as described in claim 6, characterized in that, The high-quality data training module includes the following: The high-quality training data collection and classification phase collects all path samples processed by any preceding module. It combines the self-consistency and normalized total confidence of the path samples calculated in the consistency verification and progressive evaluation modules to evaluate all path samples. The classification includes: Expert-level samples include a set of paths that simultaneously satisfy high self-consistency and high process confidence; specifically, the final answer of the path. It must be equal to the optimal answer determined by the consistency verification module. Furthermore, the total confidence of the normalized path is higher than the high confidence threshold. , recorded as > ; A typical positive sample contains the following: High self-consistency, recorded as = ,but Paths at a medium level are denoted as < ≤ , Indicates the low confidence threshold; Low self-consistency, recorded as ,but Very high paths are denoted as > ; Negative samples, including the total confidence of the normalized path. The lower path is denoted as ≤ ; During the supervised fine-tuning phase, expert-level samples are used as training data for each optimal path. They are all converted into a "command-response" pair ,in, For the original question, Optimal path The text representing the complete, step-by-step reasoning process, with the objective function Training is performed to minimize the difference between the model's generated response and the expert-level sample, using the following logical representation: in, To supervise the fine-tuning of the loss function, The parameters representing the large language model. express The first in the text Each word element, Indicates the first All lexical sequences preceding each lexical unit, This represents the probability that the model will generate the next word given the preceding context. In the reinforcement learning phase based on course learning, the parameters of the large language model are further optimized based on reinforcement learning algorithms.

9. The guided reasoning system based on a large model and knowledge graph dual-drive as described in claim 8, characterized in that, The course-based reinforcement learning phase includes the following: First, define the reward function. For all path samples Assign a quantified training reward, the definition of which is directly related to the course level of the path, using the following logical representation: in, Indicates when path sample When the sample is expert-level, the reward function Pick , The highest reward; Indicates when path sample When the sample is a normal positive sample, the reward function is... Pick , For smaller positive rewards; Indicates when path sample When the sample is negative, the reward function Pick , As punishment; Secondly, adopting a terminal reward distribution strategy will Transformed into each time step Instant rewards For a path consisting of T steps The following logic represents instant rewards. Allocation method: in, Indicates the time steps in the path. , Representing a path The last time step; Then, reinforcement learning algorithms are used to process the parameters of the large language model. Optimization is performed, specifically by calculating the advantage function. To evaluate the merits of taking a certain action in a specific state, the advantage function is calculated based on the immediate reward defined above. and state value function This can be represented using the following logic: in, In time step The advantage function, As a discount factor, The value function is used to estimate the state. The cumulative rewards that may be obtained in the future. For the next time step Corresponding state The cumulative rewards that may be obtained in the future; Based on the calculated advantage function The model parameters are updated by pruning the objective function. The objective function is represented by the following logic. : in, Let PPO be the objective function to be optimized. This represents the expected value along the policy trajectory. The ratio of the new to the old strategies. , Indicates the current model parameters The defined new strategy This refers to the old strategy used to collect training data before the start of this optimization iteration; In time step The advantage function represents the advantage of taking a certain action compared to the average level in a given state. For the pruning function, limit the policy ratio to Within the interval, This is a hyperparameter for the clipping range.

10. A guided reasoning method based on a dual-drive approach of large models and knowledge graphs, characterized in that: Includes the following steps: S1 extracts metadata from the course's teaching materials, including core elements of the questions, prerequisite knowledge, and common mistakes. S2, convert the extracted metadata into a structured knowledge graph; S3 receives the original complex engineering problem and performs in-depth analysis to generate a structured solution plan with knowledge tags; S4, based on the Monte Carlo tree search algorithm, performs iterative loops to output a search tree containing multiple complete solution paths and corresponding evaluation information; S5 verifies the answers to multiple complete problem-solving paths based on a self-consistent voting strategy, and selects the final candidate path set; S6. Analyze the confidence of each node of the paths selected from the final candidate path set, and select the best solution path based on the evaluation results. S7 collects path samples and filters high-quality training data, and optimizes the model parameters of the large language model based on supervised fine-tuning and reinforcement learning.

Images