Automatic program repairing method based on template and code language model fusion

Abstract

The invention discloses an automatic program repairing method based on template and code language model fusion. The method comprises the following steps: firstly, obtaining perfect fault positioning information, and designing a universal mask template and a matched mask template; and selecting a matching mask template according to the feature information of the error line. And taking the context of the error line, the selected matching mask template and all the universal mask templates as the input of a code language model, and predicting the mask marks by using the code language model to generate patches. And reordering the candidate patches according to the conditional probability marked by the mask and the template coefficient, and performing patch verification through the test case. And through the designed general mask template and the matched mask template, the model is helped to carry out patch prediction more accurately. And in consideration of different capabilities of generating correct patches by different mask templates, a template coefficient is also provided to correct patch sorting, so that the time spent on patch verification is reduced. Experiments prove that the error repair performance of the method is ahead of that of the existing research.

Description

Technical Field

[0001] The present invention belongs to the technical field of software engineering, relates to automatic program repair and natural language processing, and specifically to an automatic program repair method based on the fusion of templates and code language models. Background Art

[0002] Software systems have penetrated into every aspect of daily life, from monitoring financial transactions to assisting healthcare tools, so software errors in the system can have a huge impact or cause economic losses. In order to correct these potential problems, developers usually need to invest a lot of manual work, and it is estimated that 35% to 50% of the time in the entire system development cycle is spent on debugging.

[0003] In order to improve debugging efficiency, automatic program repair (APR) technology has emerged to simplify and accelerate the error repair process by automatically generating patches. Among them, the generation and verification (G&V) technology is a popular method of APR. It first uses fault localization to narrow the search space by calculating the suspicious program locations that may cause errors. Then, a series of candidate patches are generated. After the patches are sorted, the candidate patches are compiled and verified in sequence, and finally a reasonable patch list that passes all tests is generated.

[0004] Traditional APR technology can be further divided into search-based, constraint-based, and template-based methods according to the patch generation method. Template-based patch generation methods use predefined repair patterns to convert erroneous code snippets into correct code snippets, and have been widely regarded as the most advanced technology. In the past two years, learning-based APR technology has become mainstream. It uses the latest advances in deep learning and the NMT model in the field of natural language processing (NLP). It regards the program repair problem as a neural machine translation (NMT) problem and converts the erroneous program into the repaired version by training the neural network.

[0005] Although learning-based methods can theoretically extract any potential repair pattern, in practice, they tend to generate small-scale repair code, including a line of code or a subtree of an abstract syntax tree. The reason is that during the training process, the model is more likely to learn the grammatical and semantic rules of short sequences, while for long sequence generation, the model may encounter problems such as gradient vanishing or gradient exploding, making it difficult to capture long-distance dependencies. In addition, in the patch sorting stage, most learning-based APR tools only use the beam search strategy. Although beam search has obvious advantages in repair accuracy and search efficiency, its limitation is that it only considers the logarithmic probability of the next token and lacks comprehensive consideration of the overall information of the code. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes an automatic program repair method based on the fusion of templates and code language models. The error line is located to generate a mask template, which is passed into the code language model together with the context. Patches are generated by predicting mask marks, which increases the number of correct patches generated by the model. According to the average value of the conditional probability of the mask mark and the template coefficient, the candidate patches are sorted and the candidate patches that cannot be compiled are filtered, which takes into account the overall information of the patch more comprehensively and reduces the repair time.

[0007] An automatic program repair method based on the fusion of template and code language model specifically includes the following steps:

[0008] Step 1: Obtain perfect fault location information, complete the initialization work for each bug in the data set, and extract the error line position of the bug.

[0009] Step 1-1: Check each bug project in the defect data and export it as a separate bug program, including the project name to which the bug belongs, the version number of the bug, and the corresponding position of the bug project.

[0010] Step 1-2: Based on the file comparison between the original faulty program and the repaired program, obtain the location of the faulty program that needs to be repaired and determine the location of the bug line.

[0011] Step 2: Design multiple mask templates, including a general mask template and a matching mask template. The general mask template does not consider the error row feature and directly replaces it. The matching mask template is related to the error row feature.

[0012] Step 2-1: determine the association between the error line feature and the matching mask template through a regular expression, select the matching mask template related to the error line feature, and input it into the code language model together with the context of the error line.

[0013] Step 2-2: Input all common mask templates and the context of the error line into the code language model.

[0014] Step 3: Use the code language model to predict the mask position and generate patches.

[0015] The code language model is used to iteratively predict each mask token in the mask template. In each iteration, the one with the highest score is retained according to the conditional probability. After all mask tokens complete the iteration, the candidate patch is obtained. In order to generate a replacement token for the mask token, CodeBERT looks at the tokens before and after the mask position. However, in the set repair template, the mask token is not always unique. In most cases, multiple mask tokens appear in blocks. Therefore, the values ​​of the mask tokens are iteratively predicted in order from front to back. In this way, each mask token will have at least an adjacent previous context to assist in the prediction.

[0016] Each time the code language model is queried, the first N most likely tags after the current tag are retained based on the probability score. N is the beam width, which is an adjustable parameter. The larger the value, the more tags are retained for each prediction. For each round of query, the Temp Joint score (M) of the generated tags is calculated, and the first N highest scoring fixes are retained each time:

[0017]

[0018] Where n is the total length of the mask mark, M = {m1,m2,…,m p-1 ,m p ,…,m n} are all mask marks. So far, p-1 marks have been generated, 1<=p<=n, let C * (M,m p ) is to generate m while retaining the first p-1 generated tags in sequence M p The conditional probability of .

[0019] The code language model is used to mask the original pre-training target of language modeling, and the next token is predicted based on the information predicted in the previous text. By iterating the above process, the top N patches with the highest probability corresponding to the mask template are obtained.

[0020] Step 4: Reorder the candidate patches and verify the patches through test cases.

[0021] Step 4-1: Starting from the complete patch, mask one mark m in the patch one by one i And query the code language model to obtain the conditional probability C(M,m i ), calculate the joint score (M):

[0022]

[0023] n represents the number of mask markers in a complete patch.

[0024] Multiply the joint score with the coefficient TFj of the jth mask template to obtain the final score of the patch Final score(M):

[0025] Finalscore(M)=Jointscore(M)*TFj

[0026] The template factor TF j The distribution of patches generated by the mask template in multiple known data sets in the candidate list is determined.

[0027] Step 4-2: Verify the candidate patch.

[0028] All candidate patches are sorted according to the final scores in step 4-1, and verified in order. Based on the verification results, the candidate patches are divided into uncompilable patches, timed patches, patches that failed the test, and patches that passed the test. Developers manually review the patches that passed the test and identify the patches that passed the review as correct patches.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. Based on the common repair patterns of general programs, matching mask templates and universal mask templates are proposed to limit the patch generation space to the predefined template structure, reduce the length that the model needs to predict, and improve the model's ability to generate long sequence patches.

[0031] 2. Considering that different mask templates have different probabilities of generating correct patches, templates that have feature matching with the error line are more likely to successfully repair the patch. Therefore, a template coefficient is proposed to help the model consider the overall information of the patch more comprehensively. By adjusting the weights according to the relative effectiveness of the templates, the impact of abnormally high-scoring patches on the final result can be reduced, thereby improving overall performance and reducing repair time.

[0032] 3. By introducing templates into the task of patch generation from the code language model, the model is provided with prior knowledge, which enables it to better understand the structure and context of the code. At the same time, the generation space of the model is reduced. By limiting the generation space to the predefined template structure, the possibilities that the model needs to consider are reduced, and the number of correct patches generated by the model is increased. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a framework diagram of an automatic program repair method based on the fusion of templates and code language models;

[0034] Figure 2 An input example of the code language model in the embodiment;

[0035] Figure 3 This is an example of the patch generation process in the embodiment;

[0036] Figure 4 This is an example of the patch sorting process in the embodiment. DETAILED DESCRIPTION

[0037] The present invention will be further explained below with reference to the accompanying drawings;

[0038] An automatic program repair method based on the fusion of template and code language model. The overall framework is as follows: Figure 1 As shown, the specific steps are as follows:

[0039] Step 1: Obtain perfect fault location information.

[0040] Obtain the error position by comparing the error file and the repair file and determine the error line.

[0041] Step 2: First, design mask templates, including 5 general mask templates MT1~MT5 and matching mask templates MT6~MT10. As shown in Table 1, the general mask template does not consider the error line information, including adding conditional checks, adding missing statements, directly deleting error lines, moving statement positions, and changing error lines:

[0042]

[0043]

[0044] Table 1

[0045] As shown in Table 2, the matching mask template includes changing common words, changing operators, changing conditional expressions, changing return expressions, and adjusting method names or parameters:

[0046]

[0047]

[0048] Table 2

[0049] For the error line with a determined location, the mask template is matched first. The relevant matching mask template and all common mask templates are selected through regular expressions. The mask template and the context of the error line are input into the code language model CodeBERT together.

[0050] Figure 2 An input example of an error program is provided. The example code comes from Closure-57 in the Defects4J project. The input example shows that the input is composed of three parts, including the context, the mask template, and the context. The error is a typical long sequence patch. The patch provided by the developer has nearly twenty tags. It would be very difficult to generate all of them by the model. However, by using the mask template, the search space of the model can be alleviated. For example, Figure 2 The first mask line in is generated by MT5.3, which removes the latter part and adds the mask mark. The model only needs to generate the '&&' logical operator, the second conditional judgment expression and '){'. Figure 2The second mask line in the example is generated by MT8.1, which removes the conditional expression and adds a mask tag. The model only needs to generate two conditional expressions and the '&&' logical operator in between. The third mask line is generated by MT8.3, which retains the original conditional expression and adds a logical operator and mask tag. The model only needs to generate one conditional expression. It can be seen that by using templates, the generation space can be limited to the predefined template structure, reducing the possibilities that the model needs to consider. At the same time, it can be noted that compared to Figure 2 The first universal mask template in , the second and third matching mask templates are more likely to generate correct patches, which also provides theoretical guidance for the subsequent patch sorting strategy in .

[0051] Step 3: Use the code language model to predict the mask position and generate a patch. Figure 3 As shown, from the mask template 'if(target!=null&& <mask> … <mask>)', and query CodeBERT to determine the N most likely replacement tokens before the mask token. In this example, N=2, which means that only the first two most likely results will be retained for each query. In the first round of queries, the two most likely tokens after '&&' are queried, and the results are 'target' and '! ', with conditional probabilities of 0.983 and 0.005 respectively. In the second round of queries, the two most likely tokens after 'target' and '! ' are queried respectively, and a total of four results are obtained. Finally, the temporary joint score is obtained by taking the average of the two conditional probabilities and retaining the two results with higher scores.

[0052] Step 4: Reorder the candidate patches and verify the patches through test cases. Figure 4 As shown, the error line 'if(target!=null){' generates a patch 'if(target==null){' through the conversion of the mask template MT8.1. Mask the predicted mask tags one by one, and query CodeBERT to get the corresponding conditional probabilities. The joint score is the average of the probabilities generated by each mask tag, and the final score needs to be multiplied by the template coefficient corresponding to the 8th mask template on the basis of the joint score. By re-querying the joint score obtained by CodeBERT and the template coefficient corresponding to each mask template, a more accurate patch ranking can be provided for priority sorting when only a subset of the generated patches can be verified.

[0053] The sorted candidate patches are verified by test cases in turn. The verified candidate patches will be divided into the following four categories: uncompilable patches, timed patches, patches that failed the test, and patches that passed the test. Among them, the patches that passed the test will be manually reviewed by the developers, and those that passed will be considered correct patches.

[0054] To prove the effectiveness of this method, the experimental results are compared with the commonly used methods in the prior art. The results are shown in Table 3:

[0055]

[0056] Table 3

[0057] Table 3 shows the comparison of the number of repair patches of our method with the most advanced learning and template-based APR tools (such as Tbar, CIRCLE, AlphaRepair, and TENURE) in Defects4J v1.2.0 under the perfect fault localization setting. The data in the table does not include deprecated errors. Our method can successfully generate correct repairs for 87 errors, which outperforms all previous baselines, including template-based and learning-based APR techniques. At the same time, our method has a leading performance in the Closure, Lang, Math, and Mockito projects of the Defects4Jv1.2.0 dataset, successfully repairing 24, 17, 27, and 6 errors, respectively.< / mask> < / mask>

Claims

1. An automatic program repair method based on the fusion of template and code language model. The method first obtains perfect fault location information, completes the initialization of each bug in the data set, extracts the error line position of the bug, uses a mask template to replace the error line, and inputs the error line and the context of the error line into the code language model to generate a patch sequence, sorts the candidate patches and verifies them through test cases, and is characterized by: Design multiple mask templates, including a general mask template and a matching mask template; Select a matching mask template related to the error line feature through a regular expression, and input the selected matching mask template and all common mask templates together with the context of the error line into the code language model; The method for sorting candidate patches is to mask one mark m in the patch in turn. i , get the conditional probability C(M,m i ), calculate the joint score (M): n represents the number of mask markers in the complete patch, M = {m1,m2,…,m n } represents the mask tag sequence; the joint score is combined with the coefficient TF of the mask template used j Multiply to get the final score of the patch: Final score(M)=Joint score(M)*TF j The patches are sorted from high to low according to their final scores (M); the template coefficient TF j The distribution of patches generated by the mask template in multiple known data sets in the candidate list is determined.

2. The automatic program repair method based on the fusion of template and code language model as claimed in claim 1, characterized in that: Check each bug item in the defect data and export it as a separate bug program, including the project name to which the bug belongs, the version number of the bug, and the corresponding position of the bug item; Based on the file comparison between the original faulty program and the repaired program, the location of the faulty program that needs to be repaired is obtained, and the location of the error line of the bug is determined.

3. The automatic program repair method based on the fusion of template and code language model as claimed in claim 1, characterized in that: The general mask template does not consider the error line information, including adding conditional checks, adding missing statements, directly deleting the error line, moving statement positions, and changing the error line.

4. The automatic program repair method based on the fusion of template and code language model as claimed in claim 1 or 3, characterized in that: The general mask template is shown in Table 1: Table 1.

5. The automatic program repair method based on the fusion of template and code language model as claimed in claim 1, characterized in that: The matching mask template includes changing common words, changing operators, changing conditional expressions, changing return expressions, and adjusting method names or parameters.

6. The automatic program repair method based on the fusion of template and code language model as claimed in claim 1 or 5, characterized in that: The matching mask template is shown in Table 2: Table 2.

7. The automatic program repair method based on the fusion of template and code language model as claimed in claim 1, characterized in that: The code language model is CodeBERT.

8. The automatic program repair method based on the fusion of template and code language model as claimed in claim 1 or 7, characterized in that: The code language model CodeBERT iteratively predicts the value of the mask token in order from front to back. Each time the code language model is queried, the top N most likely tokens after the current token are retained according to the probability score; In each round of query, the Temp Joint score (M) of the generated tags is calculated, and the top N repairs with the highest scores are retained each time. The top N patches with the highest probability corresponding to the mask template are obtained through iteration: Where n is the total length of the mask mark, M = {m1,m2,…,m p-1 ,m p ,…,m n } for all mask tags; So far, p-1 marks have been generated, 1<=p<=n, let C * (M,m p ) means to generate a tag m while retaining the first p-1 generated tags in sequence M p The conditional probability of .

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 3, 5, and 7.

Images