Dynamic code review opinion generation method based on information retrieval and deep learning
Through dynamic integration of information retrieval and deep learning methods, the semantic classifier is used to automatically generate code review opinions, solve the problem of inefficient code review efficiency, realize high-quality automatic review opinions generation, and improve software development efficiency.
Patent Information
- Application Number
- CN202311592854.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-27
- Publication Date
- 2025-03-11
AI Technical Summary
In the prior art, code review is labor-intensive, relying on manual inspection and written feedback, resulting in inefficiency and development delays, and the performance of deep learning methods is limited by data sets, and information retrieval methods cannot generate new annotations.
A dynamic code review opinion generation method based on information retrieval and deep learning is proposed. Through a semantic classifier, a high-quality review opinion is automatically generated. The specific steps include using CodeBert to convert the code into a vector, obtaining similar codes and review opinions through information retrieval, using the semantic similarity classifier to judge semantic similarity, and generating new review opinions through CodeT5.
It realizes automatic code review opinion generation, improves the efficiency of the code review process, and the quality of the generated review opinions is higher than the baseline method, which can more effectively support the software development process.
Smart Images
Figure CN119597609A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of software engineering, and in particular to the application in the field of automatic code review opinion generation. Background Art
[0002] Code review is a well-established quality assurance practice that is widely adopted in modern software development for both commercial and open source projects. During the code review process, reviewers manually examine the newly developed code and provide written feedback, known as code review comments. Developers modify the source code based on the feedback and integrate it into the code repository after the reviewer's approval. Some studies have found that the code review process helps improve the overall quality of software systems, including reducing software defects and improving software maintainability.
[0003] However, performing code reviews is one of the biggest challenges facing the industry because code reviews are labor-intensive and reviewers must manually review the code and provide written feedback. In addition, in order to ensure the quality of the review, reviewers also need to spend time understanding the relevant code and the purpose of the change. As software projects become more complex, the number of reviews has increased over time. For example, Microsoft has 3,000 reviews per month and Linux has 500 reviews per month. The extremely costly review process also requires code authors to spend more time waiting for review comments, resulting in reduced efficiency in the entire software development process. Therefore, for a given code change, a method that can automatically recommend code review comments will help save manual input in the code review process.
[0004] Based on the above facts, automatically generating code review comments will greatly save manpower in the code review process and speed up the development process. Currently, both deep learning methods and information retrieval methods have been studied in the field of code review. However, the performance of deep learning methods is limited by the data set, and there are also problems such as high training costs. For information retrieval methods, they can only retrieve the review comments existing in the corpus as output, and cannot generate corresponding new comments based on the given code information.
[0005] The present invention aims to dynamically fuse information retrieval methods and deep learning methods to automatically generate high-quality review comments. The existing fusion methods, such as the paper "Source code annotation generation based on neural networks and information retrieval" published in the journal Computer System Applications, only input the results of information retrieval into the deep learning model as static data and use it as part of information enhancement. This static fusion method is extremely dependent on specific tasks and data sets, has poor generalization ability and has technical problems that cannot be generalized.
[0006] How to dynamically integrate information retrieval methods and deep learning methods has become a problem that the present invention needs to solve. Summary of the invention
[0007] The present invention proposes a method for generating dynamic code review opinions based on information retrieval and deep learning. The method applies a semantic classifier and dynamically integrates information retrieval methods and deep learning methods to solve the limitations of current mainstream methods and can automatically generate higher quality review opinions.
[0008] The inventive idea of the present invention is as follows: the present invention includes three parts: the first part is information retrieval. In this part, CodeBert is used to convert the code into a vector, and the cosine distance between the vectors is calculated to retrieve 10 candidate codes. Then, the GPM algorithm is used to calculate the text similarity between the target code and the candidate code to retrieve the most similar code and the corresponding review opinions. The second part is the construction of a semantic similarity classifier. In this stage, the semantic classifier is trained using a corpus validation set. The target code and the retrieved similar code are input into the trained semantic classifier. The classifier will give a semantic label score to determine whether the two are semantically similar. If the semantics are similar, the review opinions corresponding to the similar codes are output. If they are not similar, the target code is input into the third part. The third part is the construction of a code review generator model. This part first needs to train the CodeT5 lightweight model, take the target code as input, and generate new review opinions as output. It is worth noting that unlike the previous information-enhanced hybrid method, the present invention uses a semantic similarity classifier to achieve dynamic generation of review opinions.
[0009] The present invention is implemented by the following operations: a method for generating dynamic code review opinions based on information retrieval and deep learning, which includes the following steps:
[0010] (1) 382,955 data pairs were collected from 4,901 GitHub projects and 6,388 Gerrit projects. In order to reduce the noise and improve the quality of the corpus, data cleaning operations were used for preprocessing. Finally, 167,799 data pairs were selected as the experimental corpus. Based on the code pre-training model CodeBert, all the codes in the collected projects were converted into corresponding code vector representations, and these vector representations were divided into training set and validation set according to 9:1. The specific preprocessing includes the following steps:
[0011] (1-1) First, when mining data, we excluded all comments posted by the code author because they do not represent the reviewer’s review of the code and cannot be used as review opinions.
[0012] (1-2) Use the lizard library to parse the review code to ensure that the review code is valid after the change. Delete the invalid review code data pairs that have not been changed.
[0013] (1-3)Delete data pairs with multiple different semantic review opinions because this will cause noise in model training and output.
[0014] (1-4)Finally, delete those data pairs with irrelevant review opinions, that is, the provided review opinions are useless, such as "I think it's okay", "This code looks good", etc.
[0015] (2)Based on the training set and the validation set, fine-tune the code pre-trained model CodeT5 to build the code review opinion generation model DG. Use AdamW to optimize all hyperparameters. The following are the specific hyperparameter settings:
[0016] The initial learning rate is set to 0.00005;
[0017] The training batch is set to 16;
[0018] The maximum lengths of the encoder and decoder are set to 512 and 64 respectively.
[0019] (3)Based on the validation set, fine-tune the code pre-trained model CodeT5 to build the semantic similarity classifier MC, which can be used to judge whether the target code and the retrieved similar code are semantically similar. The specific steps are as follows:
[0020] (3-1)For each code in the validation set SC, first obtain the code review opinion CRDL generated by the code review opinion generation model DG and the code review opinion CRIR of the most similar code selected by information retrieval respectively. Subsequently, using the BLEU metric, calculate the similarity between the code review opinion CRDL and the code review opinion CRIR and the reference review opinion of the code. If the similarity of CRDL exceeds the similarity of CRIR, the semantic similarity label value is set to 0, otherwise the semantic similarity label value is set to 1. Through the above steps, for each code in the validation set SC, a corresponding triple <CodeT, CodeS, semantic similarity label> can be generated.
[0021] (3-2)Divide the triples obtained based on the validation set SC in a ratio of 9:1, where the first 90% is used for training and denoted as SCT, and the last 10% is used for hyperparameter tuning and denoted as SCV. Then fine-tune the code pre-trained model CodeT5 to build the semantic similarity classifier MC. Use AdamW to optimize all hyperparameters. The following are the specific hyperparameter settings:
[0022] The initial training rate is 2e-5;
[0023] The learning rate decay factor is 0.99;
[0024] The batch size is set to 16;
[0025] The training step size is set to 2000.
[0026] (4) Use CodeBert to convert the target code CodeT into a vector representation. By calculating the cosine distance and the GPM algorithm, retrieve the code CodeS with the highest text similarity in the training set and the corresponding review opinion CodeR. The specific steps include the following:
[0027] (4-1) For the target code CodeT, use camelCase naming to decompose it and convert the code representation into a sequence representation.
[0028] (4-2) Select the code pre-training model CodeBert to convert the code sequence into a code vector representation.
[0029] (4-3) Use cosine distance to calculate the lexical similarity between the target code vector and all code vectors in the training set. Select several code snippets with high lexical similarity scores as candidates. Use the SequenceMatcher function implemented in the difflib Python library to calculate the cosine distance between code vectors. The specific calculation formula is as follows:
[0030]
[0031] Among them, M train and M test Represents the code vector. The smaller the cosine distance, the higher the vocabulary similarity.
[0032] (4-4) The GPM algorithm first calculates the length of the longest common substring between the target code and several candidate codes in turn, and regards it as the text similarity. Then it returns the candidate code CodeS with a high text similarity score and the corresponding review opinion CodeR. The specific calculation formula is as follows:
[0033]
[0034] Where N common Refers to the total number of matching characters in the common substring, N changed is the number of characters in the target code, N retrieved is the number of characters in the similar method. After the retrieval is completed, the target code CodeT and the retrieved similar code CodeS are input into the semantic similarity classifier MC.
[0035] (5) If the semantic similarity classifier MC predicts that the target code CodeT and the most similar code CodeS retrieved are semantically similar, the code review opinion CodeR corresponding to CodeS is directly used as the code review opinion of the target code CodeT. Otherwise, CodeT is input into the code review generation model DG, and the code review opinion automatically generated by it is used.
[0036] Compared with the prior art, the present invention has the following beneficial effects:
[0037] The present invention proposes a method for generating dynamic code review opinions based on information retrieval and deep learning. The method proposes an improved information retrieval method, and uses a semantic classifier to judge the semantic information for generating review opinions, and dynamically integrates the information retrieval method and the deep learning method. The present invention makes up for the shortcoming that the information retrieval method cannot generate new annotations, alleviates the problem of high training cost of the deep learning method, and proposes a dynamic hybrid method that is different from previous static hybrid methods such as information enhancement. Experimental results show that the present invention is superior to all baseline methods in automatic evaluation indicators, indicating that it can generate more effective review opinions, improve the efficiency of the code review process, and is competitive. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The accompanying drawings are used to better explain the present invention, including specific stages and steps, and like the examples, are only used as explanations and demonstrations of the invention and will not limit the present invention.
[0039] Figure 1 A framework diagram of a method for generating dynamic code review opinions based on information retrieval and deep learning provided by the present invention.
[0040] Figure 2 A flow chart of a method for generating dynamic code review opinions based on information retrieval and deep learning provided by the present invention. DETAILED DESCRIPTION
[0041] In order to make the code reviewer more aware of the specific implementation process of the present invention, the following will be described in detail with reference to the accompanying drawings. The description in this section is only used as an explanation and demonstration of the invention, and does not limit the scope of the invention.
[0042] Example 1
[0043] See also Figure 1 to Figure 2 , which is the specific process of this embodiment, Figure 1 This is the framework diagram of this embodiment, which includes three stages, specifically including the following contents:
[0044] (1) The corpus comes from 4,901 GitHub projects and 6,388 Gerrit projects. Because they are all data pairs from real projects, there is a lot of noise. In order to improve the quality of the corpus, preprocessing operations are performed on it. Finally, a total of 167,799 data pairs were collected as experimental objects. The specific processing steps are as follows:
[0045] (1-1) First, when mining data, we excluded all comments posted by the code author because they do not represent the reviewer’s review of the code and cannot be used as review opinions.
[0046] (1-2) Use the lizard library to parse the review code to ensure that the review code is valid after the change. Delete the invalid review code data pairs that have not been changed.
[0047] (1-3) Delete data pairs with multiple different semantic reviews, because these noises will reduce the effect of model training. Finally, delete data pairs with irrelevant reviews, that is, the reviews provided are useless, such as "I think it's OK", "This code looks good", etc.
[0048] (1-4) Based on the code pre-training model CodeBert, all the codes in the collected project are converted into corresponding code vector representations, and the corpus is divided into a training set (90%) and a validation set (10%). After the division is completed, the corpus is divided into 151,019 pairs of training sets and 16,780 pairs of validation sets. Table 1 shows the specific data statistics of the corpus.
[0049] Table 1 Statistical information of the experimental corpus
[0050]
[0051] (2) Based on the training set and validation set, the code review opinion generation model DG is constructed by fine-tuning the code pre-training model CodeT5. AdamW is used to optimize all hyperparameters. The specific hyperparameter value settings are as follows:
[0052] The initial learning rate is set to 0.00005;
[0053] The training batch is set to 16;
[0054] The maximum lengths of the encoder and decoder are set to 512 and 64, respectively.
[0055] (3) Based on the validation set, the semantic similarity classifier MC is constructed by fine-tuning the code pre-training model CodeT5. The classifier can be used to determine whether the target code and the retrieved similar code are semantically similar. The specific implementation process is as follows:
[0056] (3-1) For each piece of code in the validation set SC, first separately obtain the code review opinion CRDL generated by the code review opinion generation model DG and the code review opinion CRIR of the most similar code selected by information retrieval. Subsequently, with the help of the BLEU metric, calculate the similarity between the code review opinion CRDL and the code review opinion CRIR and the reference review opinion of the code. If the similarity of CRDL exceeds the similarity of CRIR, the semantic similarity label value is set to 0; otherwise, the semantic similarity label value is set to 1. Through the above steps, for each piece of code in the validation set SC, a corresponding triple <CodeT, CodeS, semantic similarity label> can be generated.
[0057] (3-2) Divide the triples obtained based on the validation set SC in a ratio of 9:1, where the first 90% is used for training and denoted as SCT, and the last 10% is used for parameter tuning and denoted as SCV. By fine-tuning the code pre-training model CodeT5, construct a semantic similarity classifier MC. Use AdamW to optimize all hyperparameters. Table 2 shows the specific hyperparameter value settings:
[0058] Table 2 Semantic Similarity Classifier MC Training Parameter Settings
[0059]
[0060] (4) Use CodeBert to convert the target code CodeT into a vector representation. Through calculating the cosine distance and the GPM algorithm, retrieve the code CodeS with the highest text similarity in the training set and the corresponding review opinion CodeR. The specific implementation process is as follows:
[0061] (4-1) For the target code, decompose it using the camel case naming method and convert the code representation into a sequence representation X a and X b .
[0062] (4-2) Select the code pre-training model CodeBert to convert the code sequence into a code vector representation.
[0063] (2-3) Use the cosine distance to calculate the lexical similarity between the target code vector and all code vectors in the training set. Select the 10 code segments with the largest lexical similarity as candidates. Use the SequenceMatcher function implemented in the difflib Python library to calculate the cosine distance between code vectors. The specific calculation formula is as follows:
[0064]
[0065] where M train and M testRepresents the code vector. The smaller the cosine distance, the higher the vocabulary similarity.
[0066] (2-4) The GPM algorithm first calculates the length of the longest common substring between the target code and the 10 candidate codes in turn, and regards it as the text similarity. It then returns the candidate code CodeS with the highest text similarity and the corresponding review opinion CodeR. The specific calculation formula is as follows:
[0067]
[0068] Where N common Refers to the total number of matching characters in the common substring, N changed is the number of characters in the target code, N retrieved is the number of characters in the similar method. After the retrieval is completed, the target code CodeT and the retrieved similar code CodeS are input into the semantic similarity classifier.
[0069] (5) If the semantic similarity classifier predicts that the target code CodeT and the most similar code CodeS retrieved are semantically similar, then the code review opinion CodeR corresponding to CodeS is directly used as the code review opinion of the target code CodeT. Otherwise, CodeT is input into the code review generation model DG, and the code review opinion automatically generated by it is used.
[0070] (6) To verify the effectiveness of this embodiment, we selected the information retrieval methods LSTM and CommmentFinder, the deep learning methods Tufano et al. and Codebert, and the fusion method Rencos as baseline methods. We used automatic evaluation indicators to evaluate all methods (BLEU-4, Perfect Prediction, and ROUGE-L). The experimental subjects used the benchmark corpus we processed. Finally, our experimental results are as follows:
[0071] Table 3 Comparison of results between the method of this embodiment and other methods
[0072]
[0073] In the table, the scores in bold indicate the best performance. As can be seen from the table, this embodiment achieved 28.4%, 722, and 37.8% on the three automatic evaluation index scores respectively. Compared with the five advanced baselines based on information retrieval, deep learning, and hybrid, the three indicators have been improved by at least 19.3%, 16.6%, and 10.2%. In the perfect prediction (PP) indicator, this embodiment can predict 313 more correct review opinions on average than the baseline method.
[0074] Experiments show that the dynamic fusion method proposed in this embodiment can generate review opinions more effectively than five advanced benchmark methods based on deep learning, information retrieval and hybrid methods. The effectiveness of the dynamic hybrid method based on semantic classifiers is confirmed, indicating that code semantic information is crucial for generation tasks, highlighting the competitiveness of this embodiment.
[0075] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for generating dynamic code review opinions based on information retrieval and deep learning, characterized in that: The following steps are involved: (1) Collect actual development projects from GitHub and Gerrit open source communities as a corpus, delete the comments of the code authors and the code reviews that have not been changed, and convert all the codes in the collected projects into corresponding code vector representations based on the code pre-training model CodeBert, and divide these vector representations into training sets and validation sets; (2) Based on the training set and validation set, we fine-tune the code pre-training model CodeT5, build the code review opinion generation model DG, and use AdamW to optimize all hyperparameters; (3) Based on the validation set, the semantic similarity classifier MC is constructed by fine-tuning the code pre-training model CodeT5. The classifier is used to determine whether the target code is semantically similar to the retrieved similar code. (4) Given a target code CodeT for which code review comments need to be generated, CodeBert is first used to generate a vector representation of the target code. Then, the top ten candidate codes that are most similar to the target code vector are selected from the training set. Then, the GPM algorithm is used to select the code CodeS and the corresponding code review comments CodeR with high similarity from these ten candidate codes based on text similarity with the help of the longest common subsequence matching. (5) If the semantic similarity classifier predicts that the target code CodeT and the most similar code CodeS retrieved are semantically similar, the code review opinion CodeR corresponding to CodeS is directly used as the code review opinion of the target code CodeT. Otherwise, CodeT is input into the code review generation model DG, and the code review opinion automatically generated by it is used.
2. The method for generating dynamic code review opinions based on information retrieval and deep learning proposed in claim 1 is characterized in that: In step (2), the code review opinion generation model DG is constructed. With the help of the training set and validation set obtained in step (1), the CodeT5 model is trained by fine-tuning to enable it to have the expertise of downstream tasks. All hyperparameters are optimized using AdamW to achieve the best results.
3. The method for generating dynamic code review opinions based on information retrieval and deep learning according to claim 1 is characterized in that: The step (3) of constructing a semantic similarity classifier MC comprises the following steps: (3-1) For each code in the validation set SC, first obtain the code review opinion CRDL generated by the code review opinion generation model DG and the code review opinion CRIR of the most similar code selected by information retrieval, and then use the BLEU indicator to calculate the similarity between the code review opinion CRDL and the code review opinion CRIR and the reference review opinion of the code. If the similarity of CRDL exceeds the similarity of CRIR, the semantic similarity label value is set to 0, otherwise the semantic similarity label value is set to 1; (3-2) For each code in the validation set SC, generate the corresponding triple <CodeT, CodeS, semantic similarity label>; divide the triples obtained based on the validation set SC according to a fixed ratio, where the first 90% of the data is used for training and denoted as SCT, and the last 10% of the data is used for parameter tuning and denoted as SCV. By fine-tuning the code pre-training model CodeT5, construct a semantic similarity classifier MC and use AdamW to optimize all hyperparameters.
4. The method for generating dynamic code review opinions based on information retrieval and deep learning according to claim 1 is characterized in that: The steps for retrieving the most similar code in step (4) are as follows: (4-1) For the target code, decompose it using camel case naming and convert the code representation into a sequence representation; (4-2) Select the code pre-training model CodeBert to convert the code sequence into a code vector representation; (4-3) Use cosine distance to calculate the lexical similarity between the target code vector and all code vectors in the training set. Select several code fragments with high lexical similarity scores as candidates, and use the SequenceMatcher function implemented in the difflib Python library to calculate the cosine distance between code vectors. The calculation formula is as follows: Among them, M train and M test Represents the code vector, the smaller the cosine distance, the higher the vocabulary similarity; (4-4) The GPM algorithm first calculates the length of the longest common substring between the target code and several candidate codes in sequence, and regards it as the text similarity. Subsequently, return the candidate code CodeS with a high text similarity score and the corresponding review opinion CodeR. The calculation formula is as follows: Where N common Refers to the total number of matching characters in the common substring, N changed is the number of characters in the target code, N retrieved is the number of characters in the similar method. After the retrieval is completed, the target code CodeT and the retrieved similar code CodeS are input into the semantic similarity classifier MC.