A code defect automatic repair method based on code attribute graph and large model

Abstract

The application discloses a code defect automatic repairing method based on code attribute graph and large model, comprising the following steps: collecting code samples, analyzing source code, generating code attribute graph fusing abstract syntax tree, control flow graph and data flow graph, then performing dynamic pruning according to node importance calculated based on node degree, PageRank and data propagation dependency degree, and retaining key nodes to improve graph learning efficiency; extracting features of the optimized graph through GAT, introducing multi-scale coding, and generating graph embedding vectors; combining node importance, screening high-value Token, performing structure-aware semantic embedding, and obtaining more accurate text representation; fusing graph and text vectors to generate comprehensive features, then inserting masks at weakly dependent or high-risk positions; fine-tuning and optimizing the mask positions; inputting the comprehensive features and the mask source code into a pre-trained language model to generate patches, performing multi-task evaluation and reordering, and selecting the optimal patch to complete the repairing.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of software defect repair, in particular to a code defect automatic repair method based on code attribute graph and large model. BACKGROUND

[0002] Currently, software systems, especially large and complex software systems, often need to be maintained and updated continuously. These systems are prone to introduce various vulnerabilities or logical defects in a fast iteration development environment. Different types of software defects often require different technical methods for repair, such as static analysis, semantic analysis, program reasoning, etc. However, as the size and logical complexity of software grow, code repair work often consumes time and effort, and manual repair not only is inefficient, but also is prone to introduce new errors. Therefore, how to efficiently improve the quality and reliability of automatic code repair has always been one of the important research focuses in the field of software engineering.

[0003] In recent years, deep learning technologies represented by pre-trained large language models have made certain breakthroughs in code understanding and generation tasks. Such models can capture semantic features in code snippets through learning from massive code data, assisting developers in quickly identifying and automatically repairing some code errors, and effectively reducing the cost of manual maintenance. However, although pre-trained models perform outstandingly in code semantic understanding, existing technologies still have the following significant problems:

[0004] (1) Most current methods only input source code as a pure text sequence into the model, ignoring the multi-dimensional structural information contained in the code, such as abstract syntax trees, control flow graphs, data dependency graphs, etc. These structures play a key role in understanding program semantics and judging control paths and variable scopes. The lack of modeling of such structural information limits the model's expressiveness and repair accuracy when dealing with complex logical errors and cross-statement dependencies.

[0005] (2) Existing methods often use static or random masking strategies, lacking a masking guidance mechanism based on structural features such as weak dependency edges in graphs and control path anomaly points. Meanwhile, the generated multiple repair candidates are usually not screened, and are not ranked and evaluated using indicators such as code structural similarity or execution feasibility, which can easily produce patches that are semantically correct but logically incorrect, reducing the actual application effect.

[0006] (3) Current mainstream large models generally have a limit on the length of input sequences (e.g., 512 Tokens) when applied to code repair. When dealing with ultra-long code snippets, the input must be truncated. The truncation process can lose important code context information, resulting in a decline in the quality of the repair patches generated by the model, and even introducing new defects.

[0007] (4)Although large model pre-training provides good initialization capability, there are still deficiencies in specific scenarios (such as specific code structure or defect mode). The current mainstream approach relies on large-scale retraining or fine-tuning, which is costly and has limited generalization ability. The lack of a lightweight fine-tuning mechanism that integrates graph structure information and quickly adapts to specific structural logic scenarios with small samples limits its flexible deployment and portability in industrial environments.

[0008] In summary, although the existing automatic code repair technology has made certain progress in efficiency and automation, there are still many limitations in structure modeling, long patch generation optimization, text input processing, and model adaptability. Therefore, it is urgent to propose a new code defect automatic repair method that integrates the structural information of code and the semantic understanding advantages of large language models, which can overcome the above technical bottlenecks and improve the accuracy, efficiency and explainability of code repair. SUMMARY

[0009] The technical problem to be solved: In view of the deficiencies of the prior art, in order to solve the above problems, the present application proposes a code defect automatic repair method based on code attribute graph and large model. Compared with the prior art, the present application has significant technical progress in repair granularity, structure perception ability, patch generation mechanism and overall process completeness.

[0010] A code defect automatic repair method based on code attribute graph and large model, the main steps are as follows:

[0011] 1) Data collection and preprocessing, collect code samples containing known vulnerabilities, normal code samples and their corresponding labels, process and analyze the obtained defect code dataset, generate code attribute graph containing abstract syntax tree, control flow graph and data flow graph information, then calculate node importance based on node degree, PageRank score and data dependence propagation degree, and perform dynamic pruning on the code attribute graph according to the importance score, retain key information nodes, and generate a sparse optimized code attribute graph;

[0012] 2) Node feature extraction is performed on the optimized code attribute graph using a graph neural network based on attention mechanism, and a multi-scale encoding strategy is introduced to extract local subgraph features and global features, and then the unified graph embedding vector is formed by splicing;

[0013] 3) Use a pre-trained language model (such as CodeBERT) to perform text embedding on the code snippet, and design a structure-aware truncation strategy combined with structure importance to filter high-value Tokens, and alleviate the semantic loss caused by input overlength, then splice the graph embedding and text embedding to form a comprehensive feature representation, which is used as the input of the repair model;

[0014] 4) Based on the optimized code attribute graph, the weak dependence edges between nodes and the risk of the nodes are detected, and when the dependence strength is less than the preset threshold or the risk score is greater than the risk threshold, mask marks are inserted in the corresponding source code segment.

[0015] 5) Small-scale fine-tuning training is performed on the mask samples, so that the large model is better adapted to the defect filling task;

[0016] 6) The fused features and the mask code are input into a pre-trained language model (such as GraphCodeBERT), and a plurality of patch candidates are generated by using the context semantic and structure perception ability. Based on semantic similarity, graph structure similarity, executability and static analysis results, a multi-task scoring function is designed to reorder the patch candidates and select the optimal repair scheme.

[0017] 7) The final patch is applied to the target code, and the repair effect is verified through syntax verification and unit test closed loop verification.

[0018] Beneficial effects:

[0019] The code defect automatic repair method based on the code attribute graph and the large model provided by the application effectively overcomes the problems of low defect repair quality and poor explainability caused by ignoring code structure features, limited input length and unclear repair decision in the prior art. Specifically, the application constructs a code attribute graph and combines a graph neural network to model the structure features, thereby improving the capture ability of the code internal complex logic and dependence relationship. A structure perception truncation and residual representation strategy is designed to significantly alleviate the information loss problem when long code is truncated. Through the mask mechanism and fine-tuning training based on weak dependence edges and risk mechanism guidance, the repair model is enhanced to accurately locate the defect position and has targeted repair ability. At the same time, through the multi-task patch evaluation mechanism, the semantic, structure and execution quality of the repair patch are ensured. Therefore, the application significantly improves the accuracy, efficiency and transparency of the code repair task, and has strong technical advantages and practical application value. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The application provides a general flow diagram of a code defect automatic repair model

[0021] Figure 2 The application provides a specific flow diagram of a code defect automatic repair model

[0022] Figure 3 The application provides a technical framework diagram of a code defect automatic repair model

[0023] Figure 4 The application provides a technical framework of code structure-semantic fusion modeling DETAILED DESCRIPTION

[0024] The present application will be further described in conjunction with the embodiments, drawings:

[0025] The present application proposes a code defect automatic repair method based on code attribute graph and large model, Figure 1 The present application provides a general flowchart of a code defect automatic repair model, Figure 2 The present application provides a specific flowchart of a code defect automatic repair model, Figure 3 The technical scheme adopted by the present application to solve its technical problems includes the following contents:

[0026] Step 1-1: data preprocessing and sample

[0027] In one embodiment, data collection and preprocessing are first performed. The present application selects Defects4J dataset including version 1.2 and 2.0, collects code samples containing known vulnerabilities, normal code samples and their corresponding labels.

[0028] Step 1-2: build code attribute graph

[0029] In the construction of multi-dimensional code graph structure, the key step is to convert code into graph structure. In one embodiment, the present application converts code into a unified multi-dimensional code graph structure to model the syntax structure, control flow and data dependency of code. The node represents the basic logic unit of the program, such as conditional judgment, loop, method call, variable and constant, etc.; the edge represents the control flow, data dependency, call relationship and definition-use relationship.

[0030] Step 1-3: generate graph embedding

[0031] In another embodiment, the graph is processed and pooled using the PyTorchGeometric framework. To highlight the core information, a node importance scoring function is designed: α i = λ1×deg(v i )+ λ2×PageRank v(i) + λ3×CtrlDepth v(i) Where, α i represents the comprehensive importance score of node v i ; deg(v i ) is the in-out degree of the node; PageRank v(i) represents the global importance score of the node in the whole graph; CtrlDepth v(i)The sentence block nesting depth where the node is located is represented, reflecting the importance of its control flow nesting; λ1, λ2 and λ3 are weighting coefficients, and satisfy λ1+λ2+λ3=1, and the application defaults to equal weight (i.e. 0.33 / 0.33 / 0.34), and one of them can be appropriately amplified according to the distribution characteristics of the defect type, so as to better match the structure optimization goal.

[0032] In one embodiment, according to the set threshold σ, dynamic pruning is performed, only nodes with importance score α i ≥σ are retained, and the graph structure is optimized.

[0033] In another embodiment, the application encodes the optimized graph structure using GAT. First, input preparation: the optimized code attribute graph G(V, E); node set V: representing syntax nodes, variables, control nodes, etc.; edge set E: representing control flow, data dependency, call, etc. relationship; each node v i ∈V has an initial feature vector h i ∈R F , such as type, context label, syntax position, etc.

[0034] Secondly, the GAT encoding process is carried out. First, the features of each node are linearly mapped once, and are uniformly represented as F' dimensional intermediate representation: h i '∈Wh i ,W∈R F'×F .

[0035] Then, for each pair of adjacent nodes i and j, the weight of the adjacent edge is learned using the attention mechanism: e ij =LeakyReLU(a T [h' i ||h' j ]) where a∈R 2F' is a learnable weight vector, P is a vector splicing operation, and LeakyReLU is an activation function to increase the stability of negative gradient propagation.

[0036] Next, the normalized attention coefficient is calculated by softmax: Here N(i) represents the neighbor node set of node i.

[0037] Finally, the neighbor information is aggregated and the node vector is updated. The node representation is updated as: where σ is the ReLU activation function, and the output h i out is the node representation fused with the structure relationship.

[0038] Step 2: Multi-scale graph encoding strategy

[0039] In one specific embodiment, to enhance structural understanding, a multi-scale graph encoding strategy is further adopted. Specifically, H is obtained through weighted pooling. global : Where α i For node v i The overall importance score;

[0040] Then, the code attribute graph is divided into several subgraphs {G1,G2,...,G} according to function level, statement block, and logical branch. k For each subgraph G k The embeddings of H are computed using a shared GAT model. (i) local =Pool(GAT(G i )).

[0041] Finally, embed the global graph into H global Embedded with each local subgraph {H (i) local The data is then spliced ​​and merged to form the final graph structure vector representation: Where γ i The final H is obtained by assigning importance weights to the subgraph (e.g., determined by subgraph size or average node score). graph It integrates global control dependencies and local semantic relationships to improve the model's ability to represent complex code structures.

[0042] Step 3-1: Generate text embeddings

[0043] In one embodiment, the present invention uses a pre-trained language model (such as CodeBERT) to extract semantic embeddings of code snippets. Assuming the code input consists of T tokens, the model maps them to text vectors H. text .

[0044] In another embodiment, considering the maximum input length limit (e.g., 512 tokens) of large language models, a structure-aware truncation strategy is proposed, taking into account the token attention score obtained in CodeBERT. i and the importance score α of the corresponding node i Calculate the composite score: composite_score i =γ1×attention_score i +γ2×α i Among them, attention_score i α is the attention score for the token. iis the importance score of the node corresponding to the Token, and γ1, γ2 are weighting coefficients, and satisfy γ1+γ2=1, wherein the weighting coefficients can be automatically adjusted according to task requirements, experience rules or training set verification.

[0045] Then, the top K Token scores are reserved to generate the main segment embedding The residual representation r is obtained by using average pooling on the truncated Token, and is defined as: Where t i is the embedding vector of the i-th truncated Token. The final text embedding is:

[0046] Step 3-2: Feature fusion

[0047] In an embodiment, in order to ensure the combination of graph structure information and text information, so that the model can consider both kinds of information at the same time when processing, the present application splices the graph structure embedding vector H graph with the embedding vector H text of the code snippet to form a comprehensive feature representation: H fusion =[H graph ||H text ], Figure 4 The overall code structure-semantics fusion modeling technical framework diagram provided by an embodiment of the present application.

[0048] Step 4: Mask design

[0049] In an embodiment, the present application identifies potential defect areas based on the structural dependency strength between nodes in the code attribute graph and the logical complexity of the nodes themselves, and inserts [MASK] markers at the corresponding source code locations to generate large model input samples for training.

[0050] Specifically, the dependency strength ω ij between any two nodes is defined as: Where: P ij represents the number of shortest dependency paths in the code attribute graph from node v i to node v j ; deg(v i ) represents the number of outgoing edges or incoming edges of node v i ; μ is an adjustable coefficient for controlling the relative weight of the path and the node degree. When ω ij is less than a set threshold θ, the edge (i,j) is considered as a weak dependency edge.

[0051] At the same time, the present application introduces a node risk score function r(v iThe assessment considers the following factors: low structural centrality (e.g., PageRank score), high syntactic complexity of statements corresponding to nodes (e.g., AST subtree depth), and critical control flow nodes (e.g., conditional statements, exception handling statements). The risk scoring function is in the form of: r(v i )=α·(1-PageRank(v i ))+βgASTDepth(v i )+ηgC(v i ∈ConrtrolNodes), where α,β,η are weight coefficients, C(·) is the index function, and when r(v i When the risk threshold is greater than τ, the system will move node v. i Insert the [MASK] tag into the corresponding source code snippet to cover up the location of potential defects.

[0052] For example, the original code is as follows:

[0053]

[0054] This method can effectively focus on logically isolated, weakly dependent, or complexly expressed locations, thereby improving the effectiveness and accuracy of patch generation.

[0055] Step 5: Mask fine-tuning training

[0056] In one embodiment, for masked samples, a pre-trained language model (such as CodeBERT) is used for small-scale fine-tuning, optimizing only the [MASK] position prediction. The training objective is: Where y t For real labels, Predict probabilities for the model.

[0057] Fine-tuning samples comprise 1%-5% of the total data, with a learning rate of 1 / 10 of the original pre-training learning rate. Attribute graphs allow mask design to move beyond relying solely on surface grammar and instead be based on full-graph structural information, improving the accuracy and relevance of repair suggestions.

[0058] Step 6: Patch Generation and Evaluation

[0059] In one embodiment, the model can generate multiple repair candidates for each input sample.

[0060] The following process is adopted: In one embodiment, the present invention uses a pre-trained language model (such as GraphCodeBERT) as a patch generation module to generate repair suggestions for the code snippet with [MASK] inserted in step 6, combining graph structure embedding and text semantic embedding.

[0061] The specific process is as follows: first, the fusion feature vector H obtained in step 3-2 is input into the model fusion As an initialization special position [CLS] vector or as an additional Token insertion, participating in the subsequent layer self-attention propagation process, together with the code fragment containing the [MASK] mark as input, into the Transformer encoding structure of the large model. In the multi-layer self-attention calculation, the model simultaneously models the semantic dependency relationship and structural nested logic of the code context, and then performs target prediction on the [MASK] position to generate a set of semantic reasonable and structurally feasible candidate repair patches {p1, p2,..., p k}.

[0062] In another embodiment, a multi-task scoring function is calculated for the candidate patch, specifically, the semantic similarity s i of each candidate patch p i with the original code, the graph structure compatibility g i , the exec_score i of the patch executability detection score, and the static_score i of the static analysis score are calculated, and a comprehensive scoring function is defined: similarity_score(p i )=λ1·s i +λ2·g i +λ3·exec_score i +λ4·static_score i , wherein λ1, λ2, λ3, λ4∈[0, 1], and λ1+λ2+λ3+λ4=1, which can be set according to experience or learned.

[0063] Step 7: Complete repair

[0064] In one embodiment, the present application finally selects the patch with the highest score as the repair result and applies it to the target code to ensure the effectiveness of the repair through syntax verification and unit testing. Specifically, before applying the patch, first convert the repaired code into an abstract syntax tree, check whether the code syntax conforms to the specifications of the target language, and ensure that no new syntax errors are introduced. After applying the patch, automatically run all related unit tests to ensure that the repair patch can solve the original problem and does not damage existing functions or introduce new defects.

[0065] The application fuses structure-aware graph modeling, weak dependency guided mask, large model fine-tuning and multi-task evaluation mechanism, and constructs a complete, efficient and interpretable code automatic repair method. Compared with the prior art, the logic defect recognition accuracy and repair effect are significantly improved. Unlike the previous method of using code attribute graph only for detection and prediction, the application is suitable for various application scenarios such as automatic vulnerability repair and intelligent software maintenance.

[0066] It should be understood that the embodiments are only used to illustrate the technical principles of the application, and not to limit the protection scope of the application. Other embodiments of the application can be easily realized by those skilled in the art after considering the specification and disclosure. These modifications, uses or adaptations follow the general principles of the application and include common knowledge or conventional technical means in the art which are not specifically described in the application. Therefore, the protection scope of the application is not limited to the above embodiments, but should be determined by the protection scope of the claims.

Claims

1. A method for automatic code defect repair based on code attribute graphs and large models, characterized in that, The method includes the following steps: S1: Process and parse the defective code dataset to generate a code attribute graph containing information such as abstract syntax tree, control flow graph and data flow graph. Then, calculate the importance of nodes based on node degree, PageRank score and data dependency propagation degree. Perform dynamic pruning on the code attribute graph according to the importance score, retain key information nodes, and generate a sparse optimized code attribute graph. S2: The optimized code attribute graph is processed by a graph neural network based on an attention mechanism to extract node features. A multi-scale coding strategy is introduced to extract local subgraph features and global features, which are then concatenated to form a unified graph embedding vector. S3: Semantically embed the code text using a pre-trained language model, perform structure-aware truncation by combining node comprehensive scores, filter high-value tokens, and generate text embedding vectors; concatenate and normalize the graph embedding vectors and text embedding vectors to form a comprehensive feature representation. S4: Based on the optimized code attribute graph, detect weak dependency edges between nodes and the risk of nodes. When the dependency strength is less than the preset threshold or the risk score is greater than the risk threshold, insert a mask mark in the corresponding source code segment to generate masked input. S5: For masked input samples, perform small-scale mask training using a pre-trained language model to optimize the [MASK] position prediction capability; S6: Input the source code of the comprehensive feature representation and the insertion mask into the pre-trained language model, generate multiple repair candidate patches through contextual understanding and logical reasoning, and perform multi-task comprehensive evaluation and re-ranking of the repair candidates based on semantic similarity, graph structure similarity, patch executability detection score and static analysis score, and select the patch with the highest score. S7: Apply the selected patch to the target code, perform syntax verification and unit testing, and complete the automatic repair.

2. The automatic code defect repair method according to claim 1, characterized in that, In step S1, the defective code dataset is processed and parsed to obtain a code attribute graph. This code attribute graph includes multiple nodes and edges representing syntax structure, control flow, and data dependencies. The importance of nodes in this code attribute graph is calculated by comprehensively considering the following three factors: first, the connectivity of the node in the graph structure, reflecting its local information transmission capability; second, the global influence of the node in the overall graph, representing its PageRank score of centrality in the program dependency path; and third, the nesting level of the node in the control flow structure, measuring its control depth in the program logic. The specific calculation is based on a weighted sum of the above factors according to preset weight coefficients, and the weights can be configured according to the task type or experience. A threshold is set according to the calculation results, and only key nodes with scores higher than the threshold are retained, while the remaining nodes are dynamically pruned to obtain a simplified sparse graph structure for subsequent graph neural network encoding.

3. The automatic code defect repair method according to claim 1, characterized in that, The multi-scale encoding in step S2 specifically includes: performing local pooling on the local subgraph and global pooling on the overall code attribute graph, and concatenating the local features and global features to form a unified graph embedding vector.

4. The automatic code defect repair method according to claim 1, characterized in that, In the structure-aware truncation process of step S3, in order to avoid the loss of semantic information caused by the large model input limitation, the structure-aware truncation method comprehensively considers the attention score of each token in the pre-trained language model and the importance score of the graph node corresponding to the token, and fuses them according to a preset weight ratio. The fusion ratio can be set by task experience or automatic search of the training set, and the token sequence with the higher importance score is retained accordingly.

5. The automatic code defect repair method according to claim 1, characterized in that, In step S4, weak dependency edge detection is performed on the optimized code attribute graph. Weak dependency edge detection is based on the graph attention weights between node pairs. When the dependency strength between nodes is... Below the threshold When an edge is weakly dependent, it is marked as such. Specifically, the strength of the dependency between any two nodes is defined. for: ,in, Indicates from node To the node The number of shortest dependency path edges in the code property graph Represents a node The number of outgoing or incoming edges. It is an adjustable coefficient used to control the relative weight of the path and node degree. Less than the set threshold At that time, on the side Edges are considered weakly dependent; meanwhile, the method introduces a node risk scoring function. The evaluation is specifically based on the following factors: PageRank score (structural centrality indicator), syntactic complexity of the statements corresponding to the nodes, critical control flow nodes, and risk scoring function. It is a weighted fusion of structural centrality index scores, syntactic complexity of statements corresponding to nodes, and key control flow nodes. hour, As a risk threshold, the system will [determine] the nodes Insert the [MASK] tag into the corresponding source code snippet to cover up the location of potential defects.

6. The automatic code defect repair method according to claim 1, characterized in that, In step S5, mask training is specifically performed using a small dataset, with the fine-tuning sample size accounting for 1%-5% of the total data, and the learning rate being 1 / 10 of the original pre-training learning rate.

7. The automatic code defect repair method according to claim 1, characterized in that, In step S6, for each repair candidate patch, its semantic similarity, graph structure compatibility, executability score and static analysis score are evaluated respectively. The indicators are weighted and summed according to the preset weights to calculate the comprehensive score result. The repair candidates are evaluated and re-ranked in a multi-task comprehensive manner, and the patch with the highest score is selected. The weights can be set according to the task type, experience or obtained through training.

8. The automatic code defect repair method according to claim 1, characterized in that, Before applying the patch in step S7, it is necessary to perform syntax verification and automated unit testing based on abstract syntax tree parsing to ensure that the patch does not introduce new syntax or logic defects.

Images