Program automatic repairing method based on disturbance and freezing pre-training model

Abstract

The invention relates to an automatic program repairing method based on a disturbance and freezing pre-training model. The method comprises the steps that firstly, a defect repair data set is constructed through an open source code warehouse, defect codes, code contexts and repair codes are extracted from historical submission containing defect repair, and the data set is cleaned; adding noise to the pre-training model through a matrix-based disturbance method; reducing the training parameter quantity through the encoder network of the frozen model; in addition, a plurality of check points in the model training process are stored through a check point integration strategy; finally, a candidate patch list is generated through reasoning of the multiple check points, after the patch list is reordered, a final candidate patch list is obtained, finally, a test suite is run on Top-N patches in the list, whether the patches can pass all test cases or not is checked, and correct patches are output. According to the method, the training cost is reduced, and the repair effect of the model can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of software debugging and artificial intelligence, and in particular relates to a program automatic repair method based on perturbation and freezing pre-training models. Background Art

[0002] Automatic program repair refers to the use of automated tools to generate repair patches for defective codes, reducing the cost of developers debugging codes and improving development efficiency.

[0003] Most existing studies use deep learning methods to train models to automatically generate patches, such as models based on neural machine translation and pre-trained models. However, the use of neural machine translation-based repair solutions is limited by the lack of training data sets and simple model architecture. The model cannot fully understand the semantics of the program and the repair effect is poor. For solutions using pre-trained models, due to the overfitting problem between the pre-trained model and the program repair task, the model can only achieve suboptimal repair effects. In addition, because the number of parameters of the pre-trained model is huge, a lot of time and computing resources are required during the training process. Summary of the invention

[0004] In order to solve the problems of poor repair effect and high training cost of current program repair solutions, the present invention proposes a program automatic repair method based on perturbation and freezing pre-training models, which includes the following four steps:

[0005] Step 1, construct a program defect repair dataset;

[0006] The step 1, constructing a program defect repair dataset, includes the following steps:

[0007] Step 1.1, in the code commit history of the open source code repository, query the commit information containing the historical records of defect fixes;

[0008] Step 1.2, extract defect code, code context, and correct patch from the defect repair history to build a dataset;

[0009] Step 1.3, clean the data set, filter out duplicate data and repair code missing data in the data set;

[0010] Step 2: Fine-tune the pre-trained model by perturbation and freezing methods;

[0011] The step 2, fine-tuning the pre-trained model, includes the following steps:

[0012] Step 2.1 perturb the pre-trained model parameters;

[0013] According to the variance of the parameter matrix, different noise intensities are added to different parameter matrices to help the model better adapt to the data of downstream tasks and alleviate the overfitting problem of the model in the pre-training stage;

[0014] Step 2.2, freeze the encoder of the model;

[0015] The reason for performing this step is that the code repair task is a generative task. Freezing only the encoder can retain the semantic information and repair capabilities learned by the model on the original dataset, while reducing the number of training parameters, speeding up training, and saving computing resources.

[0016] Step 3, generate patches through multiple checkpoints of the model;

[0017] The step 3, generating patches through multiple checkpoints of the model, comprises the following steps:

[0018] Step 3.1, during the model training phase, save the checkpoints in k (k=5) steps;

[0019] Step 3.2, in the model reasoning phase, for the defective code that needs to be repaired, the defective code and code context are used as the input of the checkpoint, and beam search is used on k checkpoints to generate n (n=100) candidate patches for the defective code at each checkpoint.

[0020] Step 4, patch sorting and verification;

[0021] The step 4, patch sorting and verification, includes the following steps:

[0022] Step 4.1, merge the k candidate patch lists, first filter out duplicate patches, then sort the patches according to their ranking and confidence scores to get the final candidate patch list;

[0023] Step 4.2: For the Top-N (N=100) patches in the candidate list, run the corresponding test suite to check whether they can pass all test cases.

[0024] The reason for performing this step is that the patches ranked higher in the list have a higher probability of being correct patches, and the burden of the verification stage can be reduced by verifying only the Top-N patches;

[0025] Finally, all patches that pass the test cases are output to complete the program repair.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] In the model training stage, the present invention adds noise to the parameters of the pre-trained model through a matrix-based perturbation method, alleviates the overfitting problem of the pre-trained model and the downstream tasks, improves the repair performance of the model, and reduces the amount of parameters that need to be trained during fine-tuning by freezing the encoder network, thereby reducing the time and computing resources required for model training. In addition, in the model training process, a checkpoint integration strategy is applied to save checkpoints of multiple models, and multiple checkpoints are used to generate and verify patches in the inference stage, so as to better capture the diversity in defect repair data.

[0028] In general, the method of the present invention can reduce the cost of model training and improve the success rate of repair. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the overall process of the method of the present invention. DETAILED DESCRIPTION

[0030] In order to more clearly demonstrate the purpose and technical solution of the present invention, the specific implementation of the present invention will be described in more detail below with reference to the accompanying drawings.

[0031] A method for automatically repairing a program based on perturbing and freezing a pre-trained model comprises the following steps:

[0032] Step 1, construct a program defect repair dataset;

[0033] This step collects real defects and repair results in open source repositories, cleans the data, and builds a large-scale high-quality dataset for fine-tuning the pre-trained model.

[0034] The step 1 is to construct a program defect repair dataset, such as Figure 1 As shown in step 1, the following steps are included:

[0035] Step 1.1, in the code commit history of the open source code repository, query the commit information containing the historical records of defect fixes;

[0036] The open source code repository refers to the open source projects on GitHub and GitLab;

[0037] Step 1.2, extract defect code, code context, and correct patch from the defect repair history to build a dataset;

[0038] The code context refers to the function where the defective code is located.

[0039] Step 1.3, clean the data set and filter out duplicate data and repair code missing data in the data set.

[0040] The duplicate data refers to the same defect repair data that has appeared in multiple warehouses;

[0041] The data of fixing code defects refers to that the developer has completed the repair task by directly deleting the defective code, and there is no corresponding repair code;

[0042] Step 2: Fine-tune the pre-trained model by perturbation and freezing methods;

[0043] This step fine-tunes the pre-trained model CodeT5. First, the parameters of the pre-trained model are disturbed to alleviate the overfitting problem and enhance the repair effect. Then, the encoder of the model is frozen to reduce the amount of training parameters and save time and computing resources.

[0044] In step 2, the pre-trained model is fine-tuned by the perturbation and freezing method, such as Figure 1 As shown in step 2, the following steps are included:

[0045] Step 2.1, perturb the pre-trained model parameters;

[0046] According to the variance of the parameter matrix, different noise intensities are added to different parameter matrices to help the model better adapt to the data of downstream tasks and alleviate the overfitting problem of the model in the pre-training stage;

[0047] For the parameter matrix of the pre-trained model [W1,W2,…,W N ], where N is the number of parameter matrix types, and the perturbed parameter matrix for:

[0048]

[0049] Where std represents the standard deviation, U(a,b) represents the uniformly distributed noise of (a,b), and λ is a hyperparameter that controls the noise intensity;

[0050] Step 2.2, freeze the encoder of the model;

[0051] The freezing means setting the requires_grad attribute of the encoder network to False during model training.

[0052] Step 3, generate patches through multiple checkpoints of the model;

[0053] In step 3, patches are generated through multiple checkpoints of the model, such as Figure 1 As shown in step 3, the following steps are included:

[0054] Step 3.1, during the model training phase, save the checkpoints in k (k=5) steps;

[0055] Step 3.2, in the model reasoning phase, for the defective code that needs to be repaired, the defective code and code context are used as the input of the checkpoint, and beam search is used on k checkpoints to generate n (n=100) candidate patches for the defective code at each checkpoint;

[0056] During inference, set the beam search with a beam size of 100 and a batch size of 8.

[0057] Step 4, patch sorting and verification;

[0058] Step 4, patch sorting and verification, as Figure 1 As shown in step 4, the following steps are included:

[0059] Step 4.1: For the k candidate patch lists, first remove duplicate patches, then sort the patches in the order of their ranking in the original list. For patches with the same ranking, sort them in reverse order according to their confidence scores to obtain the final candidate patch list.

[0060] The patch ranking refers to the order of the patches in each patch list generated by checkpoint inference after sorting the patches by confidence scores.

[0061] The confidence score refers to the probability value given by the model for the generated result;

[0062] Step 4.2: For the Top-n (N=100) patches in the candidate list, run the corresponding test suite to check whether they can pass all test cases.

[0063] Finally, all patches that pass the test cases are output to complete the program repair.

Claims

1. A method for automatic program repair based on perturbation and freezing of pre-trained models, characterized in that: The following steps are involved: Step 1, constructing a program defect repair dataset, wherein step 1 comprises the following steps: Step 1.1, in the code commit history of the open source code repository, query the commit information containing the historical records of defect fixes; Step 1.2, extract defect code, code context, and correct patch from the defect repair history to build a dataset; Step 1.3, clean the data set, filter out duplicate data and repair code missing data in the data set; Step 2, fine-tuning the pre-trained model by perturbation and freezing methods, wherein step 2 comprises the following steps: Step 2.1, perturb the parameters of the pre-trained model CodeT5; Step 2.2, freeze the encoder of the pre-trained model CodeT5; Step 3, generating patches through multiple checkpoints of the model, said step 3 comprises the following steps: Step 3.1, during the model training phase, save the checkpoints in k (k=5) steps; Step 3.2, in the model reasoning phase, for the defective code that needs to be repaired, the defective code and code context are used as the input of the checkpoint, and beam search is used on k checkpoints to generate n (n=100) candidate patches for the defective code at each checkpoint; Step 4, patch sorting and verification, said step 4 comprises the following steps: Step 4.1: For the k candidate patch lists, first remove duplicate patches, then sort the patches in the order of their ranking in the original list. For patches with the same ranking, sort them in reverse order according to their confidence scores to obtain the final candidate patch list. Step 4.2: For the Top-N (N=100) patches in the candidate list, run the corresponding test suite to check whether they can pass all test cases.

2. The method according to claim 1, characterized in that: The step 2, fine-tuning the pre-trained model by the perturbation and freezing method, further comprises: Perturb model parameters: add noise of different strengths to different parameter matrices according to the variance of the parameter matrix; Freeze the model network: During model training, set the requires_grad property of the encoder network to freeze the encoder network of the model.

Images