Intention-consistent program repair and trajectory-driven feedback generation system for programming education

Abstract

This invention discloses an intent-consistent program repair and trajectory-driven feedback generation system for programming education, including a data acquisition and preprocessing module, an adversarial test case generation module, a Monte Carlo tree search program repair module, and a hierarchical feedback generation module. This invention can reliably improve the credibility and consistency of repair results in a real teaching environment, reduce the interference of pseudo-correct submissions on teaching evaluation, and reduce the burden of repetitive Q&A for teachers through trajectory-driven process feedback, thereby improving the overall teaching quality and intelligence level of online programming teaching systems.

Description

Technical Field

[0001] This invention relates to the field of program repair and trajectory-driven feedback, specifically for programming education. Figure 1 To the program repair and trajectory-driven feedback generation system. Background Technology

[0002] With the widespread adoption of online education and online judge (OJ) platforms, automated evaluation, automated program repair (APR), and automated feedback generation for large-scale programming assignments have gradually become important means to improve teaching efficiency and learning outcomes. In current engineering practices, OJ platforms typically use pre-built test cases as the core for black-box verification of student-submitted programs. Beyond the "pass / fail" decision, they further introduce rule-based, static analysis, or large language model (LLM)-based program repair and suggestion generation modules to quickly provide runnable repair versions or textual suggestions when student code contains errors.

[0003] The inventors have found that existing technologies have at least the following drawbacks: First, patch verification heavily relies on a limited number of test cases. The incompleteness of test verification leads to the widespread phenomenon of "accidental correctness / pseudo-correctness," meaning that faulty programs or overfitting patches may pass existing tests but violate the semantic constraints of the problem. This distorts the repair evaluation and selection mechanism, which is centered on the test pass rate, making it difficult to serve as a reliable "judge" and closed-loop driving signal in educational scenarios, thereby weakening the reliability and relevance of subsequent feedback generation. Second, in educational scenarios, the same problem often has multiple correct solutions and multiple "student original intentions." However, existing LLM-based APRs mostly focus on "correct results / passing tests," which easily leads to over-repair (rewriting functions, replacing algorithms, changing control structures, etc.). Furthermore, commonly used similarity metrics mostly remain at the level of edit distance or text / surface structure similarity, making it difficult to characterize and constrain the algorithm's intent. Figure 1 This lack of consistency ultimately makes repair in teaching more like "answer replacement" than "correction of thinking." Third, the generation and repair processes of prompts are often disconnected: the prompt module often provides a post-hoc summary description based on Diff after the repair is completed, lacking a traceable deductive chain and a progressive guidance structure. It is difficult to form a process-oriented teaching scaffold of "locating the error - explaining the cause - prompting the direction," thus limiting students' understanding, transfer, and improvement of their independent debugging abilities. Summary of the Invention

[0004] The purpose of this invention is to provide a solution for programming education. Figure 1 The program repair and trajectory-driven feedback generation system includes a data acquisition and preprocessing module, an adversarial test case generation module, a Monte Carlo tree search program repair module, and a hierarchical feedback generation module.

[0005] The data acquisition and preprocessing module acquires the question description, input and output specifications, original test cases, standard solutions / historical correct codes, and user-submitted code to be repaired from the online evaluation platform. It then processes the code to be repaired to generate runnable code to be repaired. Finally, it performs intent skeleton analysis on the code to be repaired, extracts features related to the problem-solving approach through intent similarity algorithm, and calculates the similarity between the code and the standard solutions / historical correct codes.

[0006] The adversarial test case generation module generates enhanced test sets based on standard solutions or historical correct code.

[0007] The Monte Carlo tree search program repair module uses the enhanced test set as the patch verification judge and generates a runnable repair patch for the code to be repaired using Monte Carlo tree search.

[0008] The hierarchical feedback generation module backtracks the optimal path from the root node to the target leaf node in the Monte Carlo tree search, uses this path as the repair trajectory, extracts the semantic differences before and after each step of the code change on the repair trajectory, maps the differences to teaching intentions, and generates progressive hierarchical prompts.

[0009] Furthermore, the steps of the data acquisition and preprocessing module in processing the code to be repaired include:

[0010] An abstract syntax tree is used to standardize and structure the code to be repaired, and to record basic semantic information.

[0011] Run the code to be repaired. If the code to be repaired cannot be run, call the large language model to perform runnable repair on the code to be repaired and generate runnable code to be repaired. If the code to be repaired can be run, then the code to be repaired is used as runnable code to be repaired.

[0012] Furthermore, when calling the large language model to perform runnable repairs on the code to be fixed, the constraint is to maintain the original intent and minimize the changes.

[0013] Furthermore, intent similarity is jointly represented by the structural similarity between the control flow graph and the data flow graph;

[0014] In this control flow graph, nodes represent executable control units;

[0015] Edges in a control flow graph represent control transfer relationships, including sequential jumps, conditional branches, loop back edges, break / continue / return jumps, and exception transfers.

[0016] Nodes in a control flow graph represent executable control units, i.e., statement-level nodes or basic block nodes (a sequence of sequentially executed, unbranched statements). Edges in a control flow graph represent control transfer relationships, including sequential jumps, conditional branches (true / false), loop back edges, break / continue / return jumps, and optional exception transfers.

[0017] The steps for representing intent similarity include: first, performing an AST structured representation of the code; then, constructing a CFG+DFG to extract structural features related to the problem-solving approach; and finally, calculating the similarity.

[0018] The approach to solving the problem is as follows: using the standard solution / historical correct code as a reference, the relevant structural features extracted from the reference implementation through CFG+DFG are regarded as the structural representation of this approach.

[0019] There are two comparison objects: the code to be fixed vs. the standard solution / correct code submitted in the past (used to find reference correct code with the same solution approach), and the candidate patch vs. the code to be fixed (used to ensure that the solution approach is not changed during the fix).

[0020] Structural features (quantitative criteria related to problem-solving approaches) include: the control structure skeleton (if / for / while / nested levels), key logical nodes predicates / boundaries (comparison operators, index / slice boundaries, state update statements), and key variable def-use dependency subgraphs.

[0021] Furthermore, the steps for generating the enhanced test set include:

[0022] Step 1) Using the standard solution or historical correct code as a benchmark, use the mutant agent to perform constrained semantic nearest neighbor transformation on the key logical nodes of the standard solution / historical correct code submission, and generate a set of mutants that are semantically similar to the problem but functionally incorrect, in order to simulate the performance of pseudo-correct implementation or overfit patch under different error modes.

[0023] Constraints of semantic nearest neighbor transformation include: still being able to run after mutation, maintaining I / O specifications, not allowing overall reconstruction or algorithm change, and the mutant producing an observable difference from the standard solution on at least one input;

[0024] The system must remain operational and maintain I / O specifications after mutation; mutations must be minor local changes (due to editing budget constraints); complete refactoring / algorithm replacement is not allowed; and the mutant must produce an observable difference (i.e., incorrect) from the standard solution on at least one input.

[0025] Key logic node types include conditional branches, loop boundaries, comparisons, indexes / slices, and state updates;

[0026] Step 2) Combining the semantic constraints of the question, the boundaries of the input domain, and the differences between variants, use the test-generating agent to generate candidate input test cases as test input data;

[0027] Step 3) Execute each candidate input on the standard solution / historical correct code, the code to be repaired, and the mutant set (run these programs separately for the same input data x), collect output differences, anomaly information, and execution trajectory, and complete the validity verification and discrimination ability calculation of the candidate test cases; discrimination ability refers to the proportion of differences between the mutant and the standard solution in output differences, anomaly information, and execution trajectory;

[0028] Step 4) Calculate the distinguishable loss of variants. ,Right now:

[0029] (1)

[0030] in, This indicates that the variants can be distinguished from the loss. This indicates that student errors expose losses; Indicates semantic coverage loss; Indicates a loss of diversity; Indicates a loss of effectiveness; , , , , These are the weighting coefficients;

[0031] Step 5) Based on the discriminative ability and consistency verification results, deduplication, minimization and optimization of candidate tests are performed to form an enhanced test set.

[0032] Deduplication: Deduplicatize candidate inputs based on their effect signatures—that is, if the output-anomaly-trajectory results on the standard solution / student / variant are the same or highly similar, they are considered duplicates, and simpler candidate inputs are retained.

[0033] Minimization: Perform incremental shrinking on a single input (reducing the size, narrowing the numerical range, and removing redundant elements), and re-perform the validation after each shrinkage; accept the shrinkage as long as the ability to distinguish and the validity are still maintained.

[0034] Optimal approach: Select a subset within the budget that maximizes overall discriminative power / coverage / diversity / effectiveness. These objectives align with the loss terms for variant discriminability, error exposure, semantic coverage, diversity, and effectiveness mentioned in the text.

[0035] Furthermore, key logic nodes include conditional branches, loop boundaries, comparison operations, indexes / slices, and state update statements.

[0036] Furthermore, the candidate input test cases cover boundary values, extreme scales, special structures, and branches that are likely to be triggered; special structures include empty sets, repeated elements, single elements, and increasing / decreasing sequences.

[0037] Furthermore, the steps for generating a runnable patch for the code to be fixed using Monte Carlo tree search include:

[0038] Step 1) Construct a search tree with the executable code to be fixed as the root node and the candidate code to be fixed (the candidate patch version in the search tree) as the leaf nodes;

[0039] An enhanced test suite is a standard set of inputs and outputs used to validate executed code.

[0040] Step 2) Search for a repair path through four iterative stages: Monte Carlo tree search selection, expansion, simulation, and backpropagation;

[0041] The selection phase refers to selecting nodes to be expanded from the explored nodes based on the tree strategy.

[0042] The selected nodes to be expanded are shown below:

[0043] (2)

[0044] in It is the average value of the composite reward function, which belongs to the utilization part; This belongs to the exploration section;

[0045] The expansion phase refers to generating multiple agreement graphs, modifying candidate repair codes, and expanding them into child nodes;

[0046] Candidate repair patches are generated using a large model (LLM), but constraints must be met. ( This is the original code that needs to be fixed. It involves setting a similarity threshold and limiting modifications to areas near suspected errors or key logic nodes.

[0047] The process is as follows: LLM generates k candidates → runnability filtering → intent threshold filtering → proceed to extension.

[0048] The candidate repair code here is generated by applying the candidate patches generated by LLM that meet the constraints to the code to be repaired.

[0049] The simulation phase refers to performing patch verification and intent evaluation on child nodes to obtain node value;

[0050] Patch verification function As shown below:

[0051] (3)

[0052] In the formula, It is an enhanced test set; The function represents the pass / fail status of a test case;

[0053] Intent evaluation function As shown below:

[0054] (4)

[0055] In the formula, As a proportion; , To control the structural similarity between the flow graph and the data flow graph;

[0056] The backpropagation phase refers to transmitting the evaluation results back to update the statistics of the path nodes.

[0057] Furthermore, the generated fix patch is the candidate fix patch with the highest overall value in terms of composite rewards;

[0058] Among them, compound rewards As shown below:

[0059] (5)

[0060] In the formula, It is the cost function for editing or modifying code; These are the weighting coefficients.

[0061] Furthermore, the progressive hierarchical hints include three levels of hints. The first level of hints is used to point out the possible location of the error or the range of related statements. The second level of hints is used to explain the cause of the error or the violation of the problem's constraints. The third level is used to provide directions for modification, key boundary conditions, or guidance on the thought process.

[0062] The technical effects of this invention are undeniable, and the beneficial effects of this invention compared to the prior art are as follows:

[0063] In terms of reliability verification, this invention no longer relies solely on existing test points in the question bank to determine the pass / failability of student programs or candidate patches. Instead, it introduces a test case generation mechanism based on "mutation-adversarial" principles: by constructing pseudo-correct variants with similar semantics but logical deviations, and generating counterexample inputs that can effectively distinguish between correct and pseudo-correct implementations, an enhanced test set with stronger semantic coverage is formed. With this enhanced test set, programs and patches that might have previously "passed by chance" or "overfitted only to existing test points" can be reliably identified and eliminated. This significantly reduces the risk of overfitting at the patch verification level, improves the degree to which the repair results satisfy the semantic constraints of the question, and enhances the reliability and robustness of the method under scenarios of large-scale question banks and diverse submissions.

[0064] care Figure 1 Regarding program repair, existing automatic repair schemes based on large models generally prioritize "results" and are prone to "solution-changing repair" due to complete rewriting or replacement of algorithms / control structures. This invention models the repair process as a constrained local search process, employing Monte Carlo tree search to iteratively explore the candidate patch space. A composite reward function, combining test pass rate and intent similarity, is introduced into the node value evaluation. Intent similarity constrains the search direction by measuring the consistency of program structure and semantic features before and after repair, making the search more inclined to choose repair paths with "minor changes, maintained structure, and unchanged approach," thus avoiding rewriting-style repairs driven solely by tests. Therefore, this invention can improve the repair success rate while maintaining the continuity of students' original intent and problem-solving strategies, achieving "error correction without changing the solution," which better meets the requirements of consistency and interpretability in programming education scenarios.

[0065] Regarding the interpretability and usability of teaching feedback, this invention does not limit feedback generation to a post-hoc summary of the final patch. Instead, it utilizes the trajectory information naturally generated during the repair search process to construct a traceable "repair thought chain" from the gradual evolution path from the error code to the final patch. This invention extracts the error cause and corrective intent corresponding to each minor modification from the trajectory, generating hierarchical feedback consisting of "location—explanation—guidance," enabling prompts to clearly answer "why change this, what to change it to, and what problem this modification solves." Compared to feedback methods that only output the final answer or only provide fragmented differences, the trajectory-driven feedback of this invention has a stronger causal correspondence and teaching interpretability: it reduces the risk of directly revealing the answer and provides a gradual process scaffold when students get stuck, promoting the formation of transferable debugging and error correction abilities.

[0066] In summary, this invention can reliably improve the credibility and consistency of repair results in a real teaching environment, reduce the interference of false correct submissions on teaching evaluation, and reduce the burden of repetitive Q&A for teachers through trajectory-driven process feedback, thereby improving the overall teaching quality and intelligence level of the online programming teaching system. Attached Figure Description

[0067] Figure 1 This is the overall flowchart of the present invention;

[0068] Figure 2 This is a diagram of the overall algorithm architecture of the present invention;

[0069] Figure 3 This is a flowchart of the adversarial agent test case generation process of the present invention;

[0070] Figure 4This is a flowchart of the Monte Carlo tree search and repair and trajectory-driven feedback generation process of the present invention. Detailed Implementation

[0071] The present invention will be further described below with reference to embodiments, but it should not be construed that the scope of the present invention is limited to the following embodiments. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the above-described technical concept of the present invention should be included within the scope of protection of the present invention.

[0072] Example 1:

[0073] See Figures 1 to 4 The intention of programming education Figure 1 The program repair and trajectory-driven feedback generation system includes a data acquisition and preprocessing module, an adversarial test case generation module, a Monte Carlo tree search program repair module, and a hierarchical feedback generation module.

[0074] The data acquisition and preprocessing module acquires the question description, input and output specifications, original test cases, standard solutions / historical correct codes, and user-submitted code to be repaired from the online evaluation platform. It then processes the code to be repaired to generate runnable code to be repaired. Finally, it performs intent skeleton analysis on the code to be repaired, extracts features related to the problem-solving approach through intent similarity algorithm, and calculates the similarity between the code and the standard solutions / historical correct codes.

[0075] The adversarial test case generation module generates enhanced test sets based on standard solutions or historical correct code.

[0076] The Monte Carlo tree search program repair module uses the enhanced test set as the patch verification judge and generates a runnable repair patch for the code to be repaired using Monte Carlo tree search.

[0077] The hierarchical feedback generation module backtracks the optimal path from the root node to the target leaf node in the Monte Carlo tree search, uses this path as the repair trajectory, extracts the semantic differences before and after each step of the code change on the repair trajectory, maps the differences to teaching intentions, and generates progressive hierarchical prompts.

[0078] Example 2:

[0079] Intent for programming education Figure 1 The system for program repair and trajectory-driven feedback generation is technically the same as in Embodiment 1. Further, the data acquisition and preprocessing module processes the code to be repaired in the following steps:

[0080] An abstract syntax tree is used to standardize and structure the code to be repaired, and to record basic semantic information.

[0081] Run the code to be repaired. If the code to be repaired cannot be run, call the large language model to perform runnable repair on the code to be repaired and generate runnable code to be repaired. If the code to be repaired can be run, then the code to be repaired is used as runnable code to be repaired.

[0082] Example 3:

[0083] Intent for programming education Figure 1 The program repair and trajectory-driven feedback generation system has the same technical content as any one of embodiments 1-2. Furthermore, when calling the large language model to perform runnability repair on the code to be repaired, the constraint is to keep the original intention and minimize the changes.

[0084] Example 4:

[0085] Intent for programming education Figure 1 The program repair and trajectory-driven feedback generation system has the same technical content as any one of embodiments 1-3. Furthermore, the intent similarity is jointly characterized by the structural similarity of the control flow graph and the data flow graph.

[0086] In this control flow graph, nodes represent executable control units;

[0087] Edges in a control flow graph represent control transfer relationships, including sequential jumps, conditional branches, loop back edges, break / continue / return jumps, and exception transfers.

[0088] Nodes in a control flow graph represent executable control units, i.e., statement-level nodes or basic block nodes (a sequence of sequentially executed, unbranched statements). Edges in a control flow graph represent control transfer relationships, including sequential jumps, conditional branches (true / false), loop back edges, break / continue / return jumps, and optional exception transfers.

[0089] The steps for representing intent similarity include: first, performing an AST structured representation of the code; then, constructing a CFG+DFG to extract structural features related to the problem-solving approach; and finally, calculating the similarity.

[0090] The approach to solving the problem is as follows: using the standard solution / historical correct code as a reference, the relevant structural features extracted from the reference implementation through CFG+DFG are regarded as the structural representation of this approach.

[0091] There are two comparison objects: the code to be fixed vs. the standard solution / correct code submitted in the past (used to find reference correct code with the same solution approach), and the candidate patch vs. the code to be fixed (used to ensure that the solution approach is not changed during the fix).

[0092] Structural features (quantitative criteria related to problem-solving approaches) include: the control structure skeleton (if / for / while / nested levels), key logical nodes predicates / boundaries (comparison operators, index / slice boundaries, state update statements), and key variable def-use dependency subgraphs.

[0093] Example 5:

[0094] Intent for programming education Figure 1 The system for program repair and trajectory-driven feedback generation, with technical content identical to any one of embodiments 1-4, further includes the following steps for generating an enhanced test set:

[0095] Step 1) Using the standard solution or historical correct code as a benchmark, use the mutant agent to perform constrained semantic nearest neighbor transformation on the key logical nodes of the standard solution / historical correct code submission, and generate a set of mutants that are semantically similar to the problem but functionally incorrect, in order to simulate the performance of pseudo-correct implementation or overfit patch under different error modes.

[0096] Constraints of semantic nearest neighbor transformation include: still being able to run after mutation, maintaining I / O specifications, not allowing overall reconstruction or algorithm change, and the mutant producing an observable difference from the standard solution on at least one input;

[0097] The system must remain operational and maintain I / O specifications after mutation; mutations must be minor local changes (due to editing budget constraints); complete refactoring / algorithm replacement is not allowed; and the mutant must produce an observable difference (i.e., incorrect) from the standard solution on at least one input.

[0098] Key logic node types include conditional branches, loop boundaries, comparisons, indexes / slices, and state updates;

[0099] Step 2) Combining the semantic constraints of the question, the boundaries of the input domain, and the differences between variants, use the test-generating agent to generate candidate input test cases as test input data;

[0100] Step 3) Execute each candidate input on the standard solution / historical correct code, the code to be repaired, and the mutant set (run these programs separately for the same input data x), collect output differences, anomaly information, and execution trajectory, and complete the validity verification and discrimination ability calculation of the candidate test cases; discrimination ability refers to the proportion of differences between the mutant and the standard solution in output differences, anomaly information, and execution trajectory;

[0101] Step 4) Calculate the distinguishable loss of variants. ,Right now:

[0102] (1)

[0103] in, This indicates that the variants can be distinguished from the loss. This indicates that student errors expose losses; Indicates semantic coverage loss; Indicates a loss of diversity; Indicates a loss of effectiveness; , , , , These are the weighting coefficients;

[0104] Step 5) Based on the discriminative ability and consistency verification results, deduplication, minimization and optimization of candidate tests are performed to form an enhanced test set.

[0105] Deduplication: Deduplicatize candidate inputs based on their effect signatures—that is, if the output-anomaly-trajectory results on the standard solution / student / variant are the same or highly similar, they are considered duplicates, and simpler candidate inputs are retained.

[0106] Minimization: Perform incremental shrinking on a single input (reducing the size, narrowing the numerical range, and removing redundant elements), and re-perform the validation after each shrinkage; accept the shrinkage as long as the ability to distinguish and the validity are still maintained.

[0107] Optimal approach: Select a subset within the budget that maximizes overall discriminative power / coverage / diversity / effectiveness. These objectives align with the loss terms for variant discriminability, error exposure, semantic coverage, diversity, and effectiveness mentioned in the text.

[0108] Example 6:

[0109] Intent for programming education Figure 1 The program repair and trajectory-driven feedback generation system has the same technical content as any one of embodiments 1-5. Furthermore, the key logic nodes include conditional branches, loop boundaries, comparison operations, indexes / slices, and state update statements.

[0110] Example 7:

[0111] Intent for programming education Figure 1 The system for program repair and trajectory-driven feedback generation is technically the same as any one of embodiments 1-6. Furthermore, the candidate input test cases cover boundary values, extreme scales, special structures, and easily triggered branching; special structures include empty sets, repeating elements, single elements, and increasing / decreasing sequences.

[0112] Example 8:

[0113] Intent for programming education Figure 1 The system for program repair and trajectory-driven feedback generation, with technical content identical to any one of embodiments 1-7, further includes the following steps for generating a runnable repair patch for the code to be repaired using Monte Carlo tree search:

[0114] Step 1) Construct a search tree with the executable code to be fixed as the root node and the candidate code to be fixed (the candidate patch version in the search tree) as the leaf nodes;

[0115] An enhanced test suite is a standard set of inputs and outputs used to validate executed code.

[0116] Step 2) Search for a repair path through four iterative stages: Monte Carlo tree search selection, expansion, simulation, and backpropagation;

[0117] The selection phase refers to selecting nodes to be expanded from the explored nodes based on the tree strategy.

[0118] The selected nodes to be expanded are shown below:

[0119] (2)

[0120] in It is the average value of the composite reward function, which belongs to the utilization part; ; This belongs to the exploration section;

[0121] The expansion phase refers to generating multiple agreement graphs, modifying candidate repair codes, and expanding them into child nodes;

[0122] Candidate repair patches are generated using a large model (LLM), but constraints must be met. ( This is the original code that needs to be fixed. It involves setting a similarity threshold and limiting modifications to areas near suspected errors or key logic nodes.

[0123] The process is as follows: LLM generates k candidates → runnability filtering → intent threshold filtering → proceed to extension.

[0124] The candidate repair code here is generated by applying the candidate patches generated by LLM that meet the constraints to the code to be repaired.

[0125] The simulation phase refers to performing patch verification and intent evaluation on child nodes to obtain node value;

[0126] Patch verification function As shown below:

[0127] (3)

[0128] In the formula, It is an enhanced test set; The function represents the pass / fail status of a test case;

[0129] Intent evaluation function As shown below:

[0130] (4)

[0131] In the formula, As a proportion; , To control the structural similarity between the flow graph and the data flow graph;

[0132] The backpropagation phase refers to transmitting the evaluation results back to update the statistics of the path nodes.

[0133] Example 9:

[0134] Intent for programming education Figure 1 The program repair and trajectory-driven feedback generation system has the same technical content as any one of embodiments 1-8. Furthermore, the generated repair patch is the candidate repair patch with the highest comprehensive value of composite reward.

[0135] Among them, compound rewards As shown below:

[0136] (5)

[0137] In the formula, It is the cost function for editing or modifying code; These are the weighting coefficients.

[0138] Example 10:

[0139] Intent for programming education Figure 1 The system for program repair and trajectory-driven feedback generation is technically the same as any one of embodiments 1-9. Furthermore, the progressive hierarchical prompts include three levels of prompts: the first level of prompts is used to indicate the possible location of the error or the range of related statements; the second level of prompts is used to explain the cause of the error or the violation of the problem constraints; and the third level is used to provide the direction of modification, key boundary conditions, or guidance on the thought process.

[0140] Example 11:

[0141] Intent for programming education Figure 1 To the program repair and trajectory-driven feedback generation system,

[0142] This invention provides a method for programming education Figure 1 This invention relates to a program repair and trajectory-driven feedback generation system, aiming to address issues in programming education scenarios such as accidental correctness due to incomplete test cases, rewrite-based repairs focused solely on passing tests, deviations from students' original problem-solving intentions, and uninterpretable feedback. The invention constructs a complete end-to-end technical solution: as follows... Figure 1As shown, the overall method consists of data acquisition and preprocessing, adversarial test case generation, and intention... Figure 1 The program consists of four key stages: constrained Monte Carlo Tree Search (MCTS) program repair, hierarchical feedback generation, and program repair. The program repair stage adopts a composite reward mechanism that combines test pass rate and intent similarity to seek a balance between functional correctness and intent preservation, thereby achieving the educational goal of correcting errors without changing the solution.

[0143] like Figure 2 The diagram shown is the overall architecture of the algorithm of this invention. To achieve the above objectives, the method of this invention includes the following steps:

[0144] Step 201: Data Acquisition and Preprocessing.

[0145] The process involves acquiring the problem descriptions, input / output specifications, original test cases, standard solutions / historical correct submissions, and student-submitted code for repair from the online evaluation platform. This code is then standardized and structured, including at least an Abstract Syntax Tree (AST) representation, and basic semantic information is recorded to support subsequent fine-grained changes, difference metrics, and similarity assessments. Furthermore, the student-submitted code is executed, and the results are collected: if a syntax error is detected causing the code to be unrunnable, the larger model is invoked to perform runnability repairs under the constraint of "preserving the original intent and minimizing modifications," allowing the code to proceed to the subsequent logic repair process; if the code is runnable but contains runtime or logical errors, subsequent steps will perform targeted repairs and verification.

[0146] Step 202: Intent skeleton extraction and intent similarity baseline calculation.

[0147] The intent skeleton analysis is performed on the student's code to be repaired to extract structural features related to the problem-solving approach and calculate the structural similarity between the features and the original code. The intent similarity can be jointly represented by the structural similarity of the control flow graph (CFG) and data flow graph (DFG), which serves as an important component of the subsequent program repair search constraints and composite reward evaluation, and is used to suppress repair behaviors that deviate from the intent, such as algorithm replacement and control structure reconstruction.

[0148] Step 203: Construct a test case generation mechanism based on mutation-adversarial approaches.

[0149] To address the issues of accidental correctness and patch overfitting, a multi-agent adversarial generation process is constructed: a mutation agent mutates key logic nodes based on the standard solution or a high-confidence correct solution, generating a set of semantically similar but functionally incorrect variants; a test generation agent combines the semantics of the problem with the differences between the variants to generate input test cases that can distinguish between the correct implementation and the variants, and obtains an enhanced test set with stronger semantic coverage through multiple rounds of screening and consistency verification. This enhanced test set serves as a patch verification judge in subsequent search and evaluation, improving the degree to which the repair results satisfy the semantic constraints of the problem and reducing the risk of incorrect passes caused by only fitting the original test points.

[0150] Step 204: Meaning based on Monte Carlo tree search Figure 1 Search for program repair.

[0151] A search tree is constructed using the student's original error code as the root node and candidate repair codes as leaf nodes. A Monte Carlo tree search (MCTS) is employed, using a four-stage iterative search process involving selection, expansion, simulation, and backpropagation to find the repair path. Specifically:

[0152] 1) Selection phase: Select nodes to be expanded from the explored nodes according to the tree strategy;

[0153] 2) Expansion phase: Generate multiple minor modification candidates for the consensus graph and expand them into child nodes;

[0154] 3) Simulation / Evaluation Phase: Perform patch verification and intent evaluation on child nodes to obtain node value;

[0155] 4) Backpropagation phase: The evaluation results are sent back to update the statistics of the path nodes.

[0156] During the search process, candidate patch nodes are generated based on the principle of local minor modifications, and simulation evaluation is performed on the candidate nodes: on the one hand, the test pass rate is calculated using the enhanced test set obtained in step 203, and on the other hand, the intention similarity between the candidate patch and the original code is calculated; the two are merged into the node value and form a composite reward function, and then the statistical information of each node is continuously updated in the process of selection, expansion, simulation and backpropagation, so that the search gradually converges to the repair result that satisfies both functional correctness and maintains the student's original intention and problem-solving strategy as much as possible.

[0157] Step 205: Determine the optimal patch and output the intended meaning. Figure 1 To the repair results.

[0158] When candidate patches that meet a preset threshold are found, the patch with the highest overall value is output as the final fix result. The threshold may include requirements for passing the enhancement test set and a constraint that the intent similarity is not lower than a threshold. If multiple feasible patches exist, the patch with the smallest modification and highest intent similarity is selected first. Figure 1Patches with higher consistency are used to avoid "answer replacement" output in educational scenarios, thereby improving the teaching usability and transferability of the repair results.

[0159] Step 206: Generation of hierarchical feedback based on the repair trajectory.

[0160] Once the final patch is determined, the optimal path from the root node to the target leaf node is traced back through the MCTS search tree, and this path is used as the patch trajectory. The semantic differences before and after each step of the code change on the trajectory are extracted, and the differences are mapped to teaching intentions to generate progressive hierarchical prompts: the first layer indicates the possible error location or the range of related statements, the second layer explains the cause of the error or the violation of the problem constraints, and the third layer provides the direction of modification, key boundary conditions, or thought process guidance, thereby providing process-oriented teaching scaffolding while avoiding the direct disclosure of the complete answer.

[0161] like Figure 3 As shown, the test case generation mechanism based on adversarial agents can be further refined into the following sub-steps:

[0162] Step 301: Variant Construction. Based on the standard solution / high-confidence correct solution, the mutant agent performs constrained semantic nearest neighbor transformations on key logical nodes (such as conditional branches, loop boundaries, comparison operations, indexes / slices, state update statements, etc.) to generate a set of variants that are semantically similar to the problem statement but functionally incorrect. This set is used to simulate the performance of "pseudo-correct implementation / overfitting patch" under different error modes.

[0163] Step 302: Candidate Test Generation. The test generation agent combines the semantic constraints of the question, the boundaries of the input domain, and the differences between variants to generate candidate input test cases, prioritizing the coverage of boundary values, extreme scales, special structures (empty sets, repeated elements, single elements, increasing and decreasing sequences, etc.), and input forms that are prone to triggering branching.

[0164] Step 303: Execution and Adjudication. For each candidate input test case, execute it on the standard solution / high-confidence correct solution, the student program, and the mutant set, respectively. Collect the output differences, exception information, and execution trajectory to complete the validity verification of the candidate test cases (the input domain is valid, and the reference program can be executed normally) and the discrimination ability calculation (whether it can distinguish between the correct implementation and the mutant, and whether it can expose the error of the student program).

[0165] Step 304: Calculate adversarial information. Integrating indicators such as "ability to kill mutants," "ability to expose student errors," and "coverage gain and diversity" into adversarial optimization objectives, the general formula for the adversarial loss is given below:

[0166]

[0167] in, This indicates that the mutant can be distinguished, which is used to encourage the generation of tests that can distinguish between the standard solution and the mutant output (i.e., improve the mutant kill rate). This represents the loss from student error exposure, used to encourage the generation of counterexample inputs that can trigger differences between student programs and standard solutions, thereby reducing the chance of accidental correctness; This represents semantic coverage loss, used to encourage tests to cover more branches, boundaries, and problem constraints; This represents the diversity loss, used to suppress homogeneous use cases that are repeated with existing test sets; This represents a validity loss, used to penalize invalid tests that do not meet input constraints or cause errors in the reference program. , which is a weighting coefficient used to achieve a controllable balance between "discrimination capability, error exposure, coverage, diversity, and effectiveness".

[0168] Step 305: Screening and Enhancement Set Generation. Based on the adversarial information and consistency verification results, candidate tests are deduplicated, minimized, and optimized to form an enhanced test set.

[0169] like Figure 4 As shown, the present invention will... Figure 1 The program repair and trajectory-driven hierarchical feedback generation are integrated into a closed-loop design:

[0170] In terms of program repair, a search tree is constructed with the student's original error code as the root node, and the Monte Carlo tree search is iteratively executed in four stages: selection, expansion, simulation / evaluation, and backpropagation.

[0171] 1. During the selection phase, select nodes to be expanded from the explored nodes according to the tree strategy;

[0172] 2. During the expansion phase, candidate patches are generated and child nodes are formed based on minor local modifications to the consensus graph;

[0173] 3. In the simulation / evaluation phase, candidate patches are verified and their value is evaluated. The value is at least a combination of the test pass rate and intent similarity on the enhanced test set to form a composite reward to constrain the search direction and make the search tend to the repair path with small changes, unchanged structure and unchanged ideas.

[0174] 4. During the backpropagation phase, the evaluation results are sent back to update the statistics of the path nodes until the pass rate / intent threshold is met or the search budget is reached.

[0175] In terms of feedback generation, once the optimal patch is determined, the optimal path from the root node to the target leaf node is obtained by backtracking from the search tree, and this path is used as the repair trajectory. Furthermore, the semantic differences before and after each step of the code change on the trajectory, as well as the corresponding error causes and correction intentions, are extracted, and progressive hierarchical prompts are generated accordingly: including at least "error location layer, cause explanation layer, and modification direction guidance layer", so that the feedback can correspond one-to-one with the causal link of the repair process, thereby providing process teaching scaffolding while avoiding direct disclosure of the complete answer, and improving the interpretability and teaching usability of the prompts.

Claims

1. An intent-consistent program repair and trajectory-driven feedback generation system for programming education, characterized in that: It includes a data acquisition and preprocessing module, an adversarial test case generation module, a Monte Carlo tree search program repair module, and a hierarchical feedback generation module; The data acquisition and preprocessing module acquires the question description, input and output specifications, original test cases, standard solutions / historical correct codes, and user-submitted code to be repaired from the online evaluation platform. It then processes the code to be repaired to generate runnable code to be repaired. Finally, it performs intent skeleton analysis on the code to be repaired, extracts features related to the problem-solving approach through intent similarity algorithm, and calculates the similarity between the code and the standard solutions / historical correct codes. The adversarial test case generation module generates enhanced test sets based on standard solutions or historical correct code. The Monte Carlo tree search program repair module uses the enhanced test set as the patch verification judge and generates a runnable repair patch for the code to be repaired using Monte Carlo tree search. The hierarchical feedback generation module backtracks the optimal path from the root node to the target leaf node in the Monte Carlo tree search, uses this path as the repair trajectory, extracts the semantic differences before and after each step of the code change on the repair trajectory, maps the differences to teaching intentions, and generates progressive hierarchical prompts.

2. The intent-consistent program repair and trajectory-driven feedback generation system oriented to programming education of claim 1, wherein: The data acquisition and preprocessing module processes the code to be repaired in the following steps: An abstract syntax tree is used to standardize and structure the code to be repaired, and to record basic semantic information. Run the code to be repaired. If the code to be repaired cannot be run, call the large language model to perform runnable repair on the code to be repaired and generate runnable code to be repaired. If the code to be repaired can be run, then the code to be repaired is used as runnable code to be repaired.

3. The intent-consistent program repair and trajectory-driven feedback generation system for programming education according to claim 2, characterized in that: When calling the large language model to perform runnability fixes on the code to be repaired, the constraint is to maintain the original intent and minimize the changes.

4. The intent-consistent program repair and trajectory-driven feedback generation system for programming education according to claim 1, characterized in that: Intent similarity is jointly represented by the structural similarity between the control flow graph and the data flow graph; In this control flow graph, nodes represent executable control units; Edges in a control flow graph represent control transfer relationships, including sequential jumps, conditional branches, loop back edges, break / continue / return jumps, and exception transfers.

5. The intent-consistent program repair and trajectory-driven feedback generation system for programming education according to claim 1, characterized in that, The steps to generate an enhanced test set include: Step 1) Using the standard solution or historical correct code as a benchmark, use the mutant agent to perform constrained semantic nearest neighbor transformation on the key logical nodes of the standard solution / historical correct code submission, and generate a set of mutants that are semantically similar to the problem but functionally incorrect, in order to simulate the performance of pseudo-correct implementation or overfit patch under different error modes. Constraints of semantic nearest neighbor transformation include: still being able to run after mutation, maintaining I / O specifications, not allowing overall reconstruction or algorithm change, and the mutant producing an observable difference from the standard solution on at least one input; Key logic node types include conditional branches, loop boundaries, comparisons, indexes / slices, and state updates; Step 2) Combining the semantic constraints of the question, the boundaries of the input domain, and the differences between variants, use the test-generating agent to generate candidate input test cases as test input data; Step 3) Execute each candidate input on the standard solution / historical correct code, the code to be repaired, and the mutant set, collect the output differences, anomaly information, and execution trajectory, and complete the validity verification and discrimination ability calculation of the candidate test cases; discrimination ability refers to the proportion of differences between the mutant and the standard solution in output differences, anomaly information, and execution trajectory; Step 4) Calculate the distinguishable loss of variants. ,Right now: ；（1） in, This indicates that the variants can be distinguished from the loss. This indicates that student errors expose losses; Indicates semantic coverage loss; Indicates a loss of diversity; Indicates a loss of effectiveness; , , , , These are the weighting coefficients; Step 5) Based on the discriminative ability and consistency verification results, deduplication, minimization and optimization of candidate tests are performed to form an enhanced test set.

6. The intent-consistent program repair and trajectory-driven feedback generation system for programming education according to claim 5, characterized in that, Key logical nodes include conditional branches, loop boundaries, comparison operations, indexes / slices, and state update statements.

7. The intent-consistent program repair and trajectory-driven feedback generation system for programming education according to claim 5, characterized in that, Candidate input test cases cover boundary values, extreme scales, special structures, and branches that are likely to be triggered; Special structures include empty sets, repeating elements, single elements, and increasing / decreasing sequences.

8. The intent-consistent program repair and trajectory-driven feedback generation system for programming education according to claim 1, characterized in that, The steps for generating a runnable patch for the code to be fixed using Monte Carlo tree search include: Step 1) Construct a search tree with the executable code to be repaired as the root node and the candidate code to be repaired as the leaf nodes; Step 2) Search for a repair path through four iterative stages: Monte Carlo tree search selection, expansion, simulation, and backpropagation; The selection phase refers to selecting nodes to be expanded from the explored nodes based on the tree strategy. The selected nodes to be expanded are shown below: ；（2） in It is the average value of the composite reward function, which belongs to the utilization part; ; This belongs to the exploration section; The expansion phase refers to generating multiple agreement graphs, modifying candidate repair codes, and expanding them into child nodes; The simulation phase refers to performing patch verification and intent evaluation on child nodes to obtain node value; Patch verification function As shown below: ；（3） In the formula, It is an enhanced test set; The function represents the pass / fail status of a test case; Intent evaluation function As shown below: ；（4） In the formula, As a proportion; , To control the structural similarity between the flow graph and the data flow graph; The backpropagation phase refers to transmitting the evaluation results back to update the statistics of the path nodes.

9. The intent-consistent program repair and trajectory-driven feedback generation system for programming education according to claim 8, characterized in that, The generated patch is the candidate patch with the highest overall value in terms of composite rewards; Among them, compound rewards As shown below: ；（5） In the formula, It is the cost function for editing or modifying code; These are the weighting coefficients.

10. The intent-consistent program repair and trajectory-driven feedback generation system for programming education according to claim 1, characterized in that, The progressive layered hints consist of three layers. The first layer indicates the possible location of the error or the scope of the relevant statement. The second layer explains the cause of the error or the violation of the problem's constraints. The third layer provides directions for modification, key boundary conditions, or guidance on the thought process.

Images