Code attribute graph and large model-based code defect automatic repair method

Abstract

The invention discloses an automatic code defect repairing method based on a code attribute graph and a large model, which comprises the following steps: collecting a code sample, analyzing a source code, generating the code attribute graph fusing an abstract syntax tree, a control flow graph and a data flow graph, and then carrying out dynamic pruning according to node importance calculated based on node degrees, PageRank and data propagation dependency, so as to obtain a large model; key nodes are reserved to improve the graph learning efficiency; extracting features of the optimized graph through GAT, introducing multi-scale coding, and generating a graph embedding vector; in combination with node importance, high-value Tokens are screened, semantic embedding of structure perception is carried out, and more accurate text representation is obtained; generating a comprehensive feature by fusing the image and the text vector, and then inserting a mask at a position with weak dependence or high risk; performing fine tuning optimization on the mask position; and inputting the comprehensive features and the mask source codes into a pre-training language model to generate patches, performing multi-task evaluation and reordering, and selecting an optimal patch to complete repair.

Description

Technical Field

[0001] The present invention relates to the field of software defect repair, and in particular to a method for automatically repairing code defects based on a code property graph and a large model. Background Art

[0002] Currently, software systems, especially large and complex ones, often require continuous maintenance and updates. These systems, in a rapidly iterative development environment, are prone to introducing various vulnerabilities and logical flaws. Different types of software flaws often require different technical approaches for repair, such as static analysis, semantic analysis, and program reasoning. However, as software scale and logical complexity grow, code repair becomes time-consuming and labor-intensive. Manual repair is not only inefficient but also prone to introducing new errors. Therefore, how to effectively improve the quality and reliability of automated code repair has been a key research topic in software engineering.

[0003] In recent years, deep learning technologies, represented by pre-trained large language models, have achieved breakthroughs in code comprehension and generation tasks. These models can capture semantic features in code snippets by learning from massive amounts of code data, helping developers quickly identify and automatically fix certain code errors, effectively reducing the cost of manual maintenance. However, despite the outstanding performance of pre-trained models in code semantic understanding, existing technologies still have the following significant issues:

[0004] (1) Most current methods only input source code as a plain text sequence into the model, ignoring the multidimensional structural information contained in the code, such as abstract syntax trees, control flow graphs, and data dependency graphs. These structures are critical for understanding program semantics and determining control paths and variable scopes. The lack of modeling this structural information limits the model's expressiveness and accuracy in dealing with complex logical errors, cross-statement dependencies, and other issues.

[0005] (2) Existing methods often adopt static or random masking strategies and lack masking guidance mechanisms based on structural features (such as weak dependency edges and control path anomalies in the figure). At the same time, the multiple repair candidates generated are usually not screened and are not ranked and evaluated using indicators such as code structure similarity or execution feasibility. This can easily lead to patches that are semantically correct but logically incorrect, reducing the effectiveness of actual applications.

[0006] (3) When applied to code repair, current mainstream large models generally have input sequence length limitations (e.g., 512 tokens). When processing very long code snippets, the input must be truncated. This truncation process may lose important code context information, resulting in a decrease in the quality of the repair patches generated by the model, or even the introduction of new defects.

[0007] (4) Although large model pre-training provides good initialization capabilities, its performance in specific scenarios (such as specific code structures or defect patterns) is still insufficient. Current mainstream practices rely on large-scale retraining or fine-tuning, which is costly and has limited generalization capabilities. The lack of a lightweight fine-tuning mechanism that integrates graph structure information and can quickly adapt to specific structural logic scenarios with small samples limits its flexible deployment and portability in industrial environments.

[0008] In summary, while existing automatic code repair technologies have made progress in efficiency and automation, they still face numerous limitations in structural modeling, long patch generation optimization, text input processing, and model adaptability. Therefore, a new automatic code defect repair method is urgently needed that combines the structural information of the code with the semantic understanding advantages of large language models to overcome these technical bottlenecks and improve the accuracy, efficiency, and interpretability of code repair. Summary of the Invention

[0009] Technical Problem to be Solved: To address the shortcomings of existing technologies and the aforementioned problems, this paper proposes an automatic code defect repair method based on a code property graph and a large model. Compared to existing technologies, this paper offers significant technical advancements in repair granularity, structure-awareness capabilities, patch generation mechanisms, and overall process completeness.

[0010] A method for automatically repairing code defects based on code property graphs and large models. The main steps are as follows:

[0011] 1) Data collection and preprocessing: collecting code samples containing known vulnerabilities, normal code samples, and their corresponding labels. The resulting defective code dataset is processed and parsed to generate a code property graph containing abstract syntax trees, control flow graphs, and data flow graphs. Node importance is then calculated based on node degree, PageRank score, and data dependency propagation. The code property graph is dynamically pruned based on the importance score, retaining key information nodes to generate a sparsely optimized code property graph.

[0012] 2) A graph neural network based on an attention mechanism is used to extract node features from the optimized code attribute graph. A multi-scale encoding strategy is introduced to extract local subgraph features and global features, which are then combined to form a unified graph embedding vector.

[0013] 3) Use a pre-trained language model (such as CodeBERT) to embed code snippets into text. Combined with structural importance, a structure-aware truncation strategy is designed to filter high-value tokens and mitigate semantic loss caused by excessively long inputs. Then, the graph embedding and text embedding are concatenated to form a comprehensive feature representation, which serves as the input to the repair model.

[0014] 4) Based on the optimized code attribute graph, weak dependency edges between nodes and node risks are detected. When the dependency strength is less than the preset threshold or the risk score is greater than the risk threshold, a mask mark is inserted in the corresponding source code segment.

[0015] 5) Perform small-scale fine-tuning training on mask samples to make the large model better suited to the defect filling task;

[0016] 6) The fused features and mask codes are fed into a pre-trained language model (e.g., GraphCodeBERT) to generate multiple patch candidates using contextual semantics and structural awareness. Based on semantic similarity, graph structure similarity, executableness, and static analysis results, a multi-task scoring function is designed to re-rank the patch candidates and select the optimal repair solution.

[0017] 7) Apply the final patch to the target code and verify the repair effect through syntax verification and unit testing.

[0018] Beneficial effects:

[0019] The automatic code defect repair method based on code attribute graph and large model proposed in the present invention effectively overcomes the problems of low defect repair quality and poor interpretability in the existing technology caused by ignoring code structural features, limited input length and unclear repair decisions, by fusing the structural information and semantic information of the code. Specifically, the present invention improves the ability to capture complex logic and dependency relationships within the code by constructing a code attribute graph and combining it with a graph neural network for structural feature modeling; designs a structure-aware truncation and residual representation strategy, which significantly alleviates the problem of information loss when long codes are truncated; through a mask mechanism and fine-tuning training guided by weak dependency edges and risk mechanisms, the repair model's ability to accurately locate defect locations and carry out targeted repairs is enhanced; at the same time, through a multi-task patch evaluation mechanism, the semantics, structure and execution quality of the repair patch are ensured. Therefore, the present invention significantly improves the accuracy, efficiency and transparency of code repair tasks, and has strong technical advantages and practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of the overall process of a code defect automatic repair model provided by the present invention

[0021] Figure 2 A specific flow chart of a code defect automatic repair model provided by the present invention

[0022] Figure 3 Technical framework diagram of a code defect automatic repair model provided by the present invention

[0023] Figure 4 A technical framework for code structure-semantic fusion modeling provided by an embodiment of the present invention DETAILED DESCRIPTION

[0024] The present invention will now be further described with reference to the embodiments and accompanying drawings:

[0025] This paper proposes a method for automatically repairing code defects based on code attribute graph and large model. Figure 1 This is a schematic diagram of the overall process of a code defect automatic repair model provided by the present invention. Figure 2 This is a specific flow chart of a code defect automatic repair model provided by the present invention. Figure 3 This is a technical framework diagram of a code defect automatic repair model provided by the present invention. The technical solutions adopted to solve its technical problems include the following:

[0026] Step 1-1: Data preprocessing and samples

[0027] In one embodiment, data collection and preprocessing are first performed. The present invention uses Defects4J datasets including versions 1.2 and 2.0 to collect code samples containing known vulnerabilities, normal code samples and their corresponding labels.

[0028] Step 1-2: Build the code property graph

[0029] Converting code into a graph structure is a key step in building a multidimensional code graph. In one embodiment, the present invention converts code into a unified multidimensional code graph structure to model the code's syntax, control flow, and data dependencies. Nodes represent the basic logical units of a program, such as conditionals, loops, method calls, variables, and constants; edges represent control flow, data dependencies, call relationships, and define-use relationships.

[0030] Steps 1-3: Generate graph embeddings

[0031] In another embodiment, the PyTorchGeometric framework is used to process and pool the graph. In order to highlight the core information, a node importance scoring function is designed: α i =λ1×deg(v i )+λ2×PageRank v(i) +λ3×CtrlDepth v(i) Among them, α i Represents node v i The comprehensive importance score of deg(v i ) is the in- and out-degree of the node; PageRank v(i) Indicates the global importance score of the node in the entire graph; CtrlDepth v(i)It indicates the nesting depth of the statement block where the node is located, reflecting the importance of its control flow nesting; λ1, λ2 and λ3 are weighting coefficients, and satisfy λ1+λ2+λ3=1. The present invention sets them to equal weights (i.e., 0.33 / 0.33 / 0.34) by default. You can also appropriately amplify one of them according to the distribution characteristics of the defect type to better match the structural optimization goal.

[0032] In one embodiment, dynamic pruning is performed based on a set threshold σ, and only the importance score α is retained. i Nodes with size ≥σ, optimize the graph structure.

[0033] In another embodiment, the present invention uses GAT to encode the optimized graph structure. First, the input preparation is: the optimized code attribute graph G(V,E); the node set V: represents syntax nodes, variables, control nodes, etc.; the edge set E: represents control flow, data dependency, call, etc.; each node v i ∈V has the initial eigenvector h i ∈R F , such as type, context label, grammatical position, etc.

[0034] Secondly, the GAT encoding process is performed. First, the features of each node are linearly mapped and unified into an intermediate representation of F' dimension: h i '∈Wh i ,W∈R F'×F .

[0035] Then, for each pair of adjacent nodes i and j, the attention mechanism is used to learn the weight of the adjacent edge: ij =LeakyReLU(a T [h' i ||h' j ])where a∈R 2F' is a learnable weight vector, P is a vector concatenation operation, and LeakyReLU is an activation function that increases the stability of negative gradient propagation.

[0036] Next, the normalized attention coefficient is calculated by softmax: Here N(i) represents the set of neighbor nodes of node i.

[0037] Finally, aggregate the neighbor information and update the node vector. The node representation is updated as: Among them, σ is the ReLU activation function, and the output h i out It is a node representation that integrates structural relationships.

[0038] Step 2: Multi-scale graph encoding strategy

[0039] In a specific embodiment, in order to enhance the structural understanding, a multi-scale graph encoding strategy is further adopted. Specifically, H is obtained by weighted pooling. global : where α i For node v i The comprehensive importance score of

[0040] Then, the code attribute graph is divided into several subgraphs {G1, G2, ..., G k}, for each subgraph G k , use the shared GAT model to calculate their embeddings: H (i) local =Pool(GAT(G i )).

[0041] Finally, the global graph is embedded into H global With each local subgraph embedded {H (i) local} are spliced ​​and fused to form the final graph structure vector representation: where γ i is the subgraph importance weight (such as determined by subgraph size or average node score), and the final H graph It integrates global control dependencies and local semantic relationships to improve the model's ability to represent complex code structures.

[0042] Step 3-1: Generate text embeddings

[0043] In one embodiment, the present invention uses a pre-trained language model (such as CodeBERT) to extract the semantic embedding of code snippets. Assume that the code input consists of T tokens, and the model maps it to a text vector H text .

[0044] In another embodiment, considering the maximum input length limit of large language models (such as 512 tokens), a structure-aware truncation strategy is proposed, which comprehensively considers the Token attention score obtained in CodeBERT. i and the importance score α of the corresponding node i , calculate the comprehensive weight: composite_score i =γ1×attention_score i +γ2×α i Among them, attention_score i is the attention score of Token, α iis the importance score of the node corresponding to the Token, γ1 and γ2 are weighting coefficients, and they satisfy γ1+γ2=1. The weighting coefficients can be automatically adjusted according to task requirements, empirical rules or training set verification.

[0045] Then, retain the top K tokens with the highest scores to generate the main segment embedding Average pooling is used on the truncated token to obtain the residual representation r, which is defined as: Among them, t i The embedding vector of the truncated Token i. The final text embedding is:

[0046] Step 3-2: Feature Fusion

[0047] In one embodiment, in order to ensure the combination of graph structure information and text information so that the model can consider both information at the same time during processing, the present invention embeds the graph structure into the vector H graph and the embedding vector H of the code snippet text Splicing to form a comprehensive feature representation: H fusion =[H graph ||H text ], Figure 4 A technical framework diagram of overall code structure-semantic fusion modeling provided for one implementation of the present invention.

[0048] Step 4: Mask Design

[0049] In one embodiment, the present invention identifies potential defect areas based on the strength of structural dependencies between nodes in the code property graph and the logical complexity of the nodes themselves, and inserts [MASK] tags at corresponding source code locations to generate large model input samples for training.

[0050] Specifically, define the dependency strength ω between any two nodes ij for: Where: P ij Represents the slave node v i To node v j The number of shortest dependency paths in the code property graph; deg(v i ) represents node v i The number of outgoing or incoming edges; μ is an adjustable coefficient used to control the relative weight of the path and node degree. ij When it is less than the set threshold θ, the edge (i, j) is considered a weakly dependent edge.

[0051] At the same time, the present invention introduces the node risk scoring function r(v i), combined with the following factors for evaluation: low structural centrality index (such as PageRank score), high grammatical complexity of the node corresponding statement (such as AST subtree depth), key nodes of control flow (such as conditional judgment, exception handling statement). The risk scoring function form is: r(v i )=α·(1-PageRank(v i ))+βgASTDepth(v i )+ηgC(v i ∈ConrtrolNodes), where α, β, η are weight coefficients, C(·) is the indicator function, when r(v i )>τ(risk threshold), the system will node v i The [MASK] tag is inserted into the corresponding source code snippet to mask the potential defect location.

[0052] For example, the original code is as follows:

[0053]

[0054] This method can effectively focus on locations with isolated logic, weak dependencies, or complex expressions, thereby improving the effectiveness and accuracy of patch generation.

[0055] Step 5: Mask fine-tuning training

[0056] In one embodiment, for masked samples, a pre-trained language model (such as CodeBERT) is used for small-scale fine-tuning (Masked Fine-tuning), and only the [MASK] position prediction is optimized. The training objectives are: where y t is the true label, Predict probabilities for the model.

[0057] The fine-tuning sample size accounts for 1%-5% of the total data, and the learning rate is 1 / 10 of the original pre-training learning rate. The attribute graph eliminates the reliance of mask design solely on surface syntax and instead bases it on the full-graph structure, improving the accuracy and specificity of repair suggestions.

[0058] Step 6: Patch Generation and Evaluation

[0059] In one embodiment, for each input sample, the model may generate multiple restoration candidates.

[0060] The following process is adopted: In one embodiment, the present invention uses a pre-trained language model (such as GraphCodeBERT) as a patch generation module, and generates repair suggestions for the code snippet inserted with [MASK] in step 6 by combining graph structure embedding and text semantic embedding.

[0061] The specific process is as follows: First, the fusion feature vector H obtained in step 3-2 fusion As the [CLS] vector to initialize a special position or as an additional token to be inserted, it participates in the self-attention propagation process of the subsequent layer and is fed into the Transformer encoding structure of the large model together with the code fragment containing the [MASK] tag as input. In the multi-layer self-attention calculation, the model simultaneously models the semantic dependency and structural nesting logic of the code context, and then makes target predictions for the [MASK] position, generating a set of semantically reasonable and structurally feasible candidate repair patches {p1,p2,...,p k}.

[0062] In another embodiment, a multi-task scoring function is calculated for each candidate patch p. i , calculate its semantic similarity s with the original code i Compatibility with graph structure g i , exec_score i Patch executable detection score, static_score i For static analysis score, define the comprehensive scoring function: similarity_score(p i )=λ1·s i +λ2·g i +λ3·exec_score i +λ4·static_score i , where λ1,λ2,λ3,λ4∈[0,1], and satisfy λ1+λ2+λ3+λ4=1, which can be set based on experience or learned.

[0063] Step 7: Complete the repair

[0064] In one embodiment, the present invention ultimately selects the patch with the highest score as the repair result and applies it to the target code, ensuring the effectiveness of the repair through syntax verification and unit testing. Specifically, before applying the patch, the repaired code is first converted into an abstract syntax tree to check whether the syntax of the code conforms to the specifications of the target language and ensure that no new syntax errors are introduced. After applying the patch, all relevant unit tests are automatically run to ensure that the repair patch can solve the original problem without breaking existing functions or introducing new defects.

[0065] This paper integrates structure-aware graph modeling, weak dependency-guided masking, large-scale model fine-tuning, and a multi-task evaluation mechanism to construct a comprehensive, efficient, and interpretable automatic code repair method. Compared to existing technologies, it significantly improves the accuracy of logic defect identification and repair effectiveness. Unlike previous methods that only use code attribute graphs for detection and prediction, this paper is suitable for a variety of application scenarios, including automatic vulnerability repair and intelligent software maintenance.

[0066] It should be understood that the present embodiment is only used to illustrate the technical principles of the present invention, rather than to limit the scope of protection of the present invention. Those skilled in the art can easily realize other embodiments of the present invention after considering the description and the disclosure. These variations, uses or adaptive changes follow the general principles of the present invention and include common knowledge or customary technical means in the art that are not specifically described in this application. Therefore, the scope of protection of the present invention is not limited to the above-mentioned embodiment, but should be based on the scope of protection determined by the claims.

Claims

1. A method for automatically repairing code defects based on code attribute graph and large model, characterized in that: The method comprises the following steps: S1: Process and parse the defective code dataset to generate a code property graph containing abstract syntax trees, control flow graphs, and data flow graphs. Node importance is then calculated based on node degree, PageRank score, and data dependency propagation. Dynamic pruning of the code property graph is performed based on the importance score, retaining key information nodes to generate a sparsely optimized code property graph. S2: We use a graph neural network based on an attention mechanism to extract node features from the optimized code attribute graph. We also introduce a multi-scale encoding strategy to extract local subgraph features and global features, and then combine them to form a unified graph embedding vector. S3: Use a pre-trained language model to semantically embed the code text, combine it with the node comprehensive score to perform structure-aware truncation, screen high-value tokens, and generate a text embedding vector. The graph embedding vector and the text embedding vector are concatenated and normalized to form a comprehensive feature representation. S4: Based on the optimized code attribute graph, weak dependency edges between nodes and node risks are detected. When the dependency strength is less than a preset threshold or the risk score is greater than the risk threshold, mask markers are inserted into the corresponding source code segment to generate masked input. S5: For masked input samples, a pre-trained language model is used to perform small-scale mask training to optimize the [MASK] position prediction capability; S6: The source code with comprehensive feature representation and insertion mask is input into the pre-trained language model. Multiple candidate repair patches are generated through context understanding and logical reasoning. Based on semantic similarity, graph structure similarity, patch executable detection score and static analysis score, the repair candidates are evaluated and re-ranked in a multi-task manner, and the patch with the highest score is selected. S7: Apply the selected patch to the target code, perform syntax verification and unit testing, and complete the automatic repair.

2. The method for automatically repairing code defects according to claim 1, characterized in that: In the step S1, the defect code dataset is processed and parsed to obtain a code attribute graph, which includes multiple nodes and edges representing grammatical structure, control flow and data dependency. The importance of nodes in the code attribute graph is calculated by comprehensively considering the following three factors: first, the connectivity of the node in the graph structure, which reflects its local information transmission capability; second, the global influence of the node in the overall graph, which represents the PageRank score of its centrality in the program dependency path; third, the nesting level of the node in the control flow structure, which measures its control depth in the program logic; the specific calculation is based on the weighted summation of the above factors according to the preset weight coefficient, and the weight can be configured according to the task type or experience; a threshold is set according to the calculation result, and only key nodes with scores higher than the threshold are retained, and the remaining nodes are dynamically pruned to obtain a sparse graph structure with simplified structure for subsequent graph neural network encoding.

3. The method for automatically repairing code defects according to claim 1, characterized in that: The multi-scale encoding in step S2 specifically includes: local pooling of local subgraphs (divided by function level, statement block, and logic branch), global pooling of the entire code attribute graph, and splicing local features with global features to form a unified graph embedding vector.

4. The method for automatically repairing code defects according to claim 1, characterized in that: During the structure-aware truncation process of step S3, in order to avoid the loss of semantic information caused by the input limitation of the large model, the structure-aware truncation method comprehensively considers the attention score of each token in the pre-trained language model and the importance score of the graph node corresponding to the token, and fuses them according to a preset weight ratio. The fusion ratio can be set through task experience or automatic search of the training set, and accordingly retains the token sequence with the highest importance score.

5. The method for automatically repairing code defects according to claim 1, characterized in that: In the step S4, weak dependency edge detection is performed on the optimized code attribute graph. The weak dependency edge detection is mainly based on the graph attention weight between node pairs. When the dependency strength ω between nodes ij When it is lower than the threshold θ, it is marked as a weak dependency edge. Specifically, the dependency strength ω between any two nodes is defined as ij for: Among them, P ij Represents the slave node v i To node v j The number of shortest dependency paths in the code property graph, deg(v i ) represents node v i The number of outgoing or incoming edges, μ is an adjustable coefficient used to control the relative weight of the path and node degree. ij When the edge (i, j) is less than the set threshold θ, it is considered as a weakly dependent edge. At the same time, the method introduces a node risk score function r(v i ), specifically combined with the following factors for evaluation: low PageRank score of the structural centrality indicator, high grammatical complexity of the node corresponding statement (specifically the AST subtree depth), and key nodes of the control flow (specifically conditional judgment and exception handling statements), the risk scoring function r(v i ) is a weighted fusion of the above three factors. When r(v i )>τ(risk threshold), the system will node v i The [MASK] tag is inserted into the corresponding source code snippet to mask the potential defect location.

6. The method for automatically repairing code defects according to claim 1, characterized in that: The mask training in step S5 is specifically performed using a small-scale data set, with the fine-tuning sample size accounting for 1%-5% of the total data, and the learning rate being 1 / 10 of the original pre-training learning rate.

7. The method for automatically repairing code defects according to claim 1, characterized in that: In step S6, the semantic similarity, graph structure compatibility, executable score and static analysis score of each candidate repair patch are evaluated respectively, and the weighted sum of each indicator is calculated according to the preset weights to calculate the comprehensive score result. The repair candidates are subjected to multi-task comprehensive evaluation and re-ranking, and the patch with the highest score is selected. The weights can be set according to the task type, experience or obtained through training.

8. The method for automatically repairing code defects according to claim 1, characterized in that: Before the patch is applied in step S7, it must pass syntax verification based on abstract syntax tree parsing and automated unit testing to ensure that the patch does not introduce new syntax or logic defects.

Images