Java code review comment generation method and device based on code structured information and review knowledge

Abstract

The application relates to a Java code review comment generation method and device based on code structured information and review knowledge. The method comprises the following steps: obtaining original Java code review data, and performing vectorization representation on code text and code review comments; in an unsupervised training stage, introducing noise to word vectors by respectively disturbing vector representation and adding mask labels, establishing a discriminative graph model and a language model, and restoring disturbed input content; in a supervised training stage, generating code review comments by fusing the model in the unsupervised training stage, and assisting with a code change prediction task; and automatically outputting code review comments of input to-be-reviewed codes according to the model generated in the unsupervised training stage and the supervised training stage. The application can improve the efficiency of code review comment generation, combines unsupervised training and supervised training, has high decoupling, and can also improve the original performance of the model.

Description

Technical Field

[0001] This invention relates to a method and apparatus for generating Java code review comments based on structured code information and review knowledge, belonging to the field of computer technology applications. Background Technology

[0002] Code review has made great strides over the past few decades.

[0003] However, in practice, two factors typically hinder the realization of these advantages: 1) Writing code review comments is time-consuming and of inconsistent quality; 2) Understanding the content of the reviewed code is equally time-consuming, and the level of understanding varies from person to person, leading to biases in the results. Although some work has addressed these issues, engineers still need to dedicate a significant amount of time to writing code review comments in today's software development process.

[0004] Currently, code review methods are mainly divided into static analysis methods and machine learning model methods. Static analysis methods have the following main shortcomings: First, model performance is limited and unstable, depending on the accuracy of each static analysis tool. Furthermore, the model cannot review code outside the scope of the selected static analysis tools, and its performance relies heavily on the quality of feature engineering and the accuracy of manual analysis. Second, the automation process is insufficient, resulting in poor generalization. This is because the model input is the result of secondary processing, not the code itself being reviewed, and therefore cannot perform specific analysis of the code's content, thus limiting its application scenarios. As for existing machine learning model methods, since their design principles are geared towards all relevant text-based tasks, their performance and professionalism in automatically generating code review comments are insufficient. They also suffer from issues such as coarse-grained representation and destruction of code structure information, thus leaving room for further improvement.

[0005] In summary, although the technology for automatically generating code review comments has been initially demonstrated in academia, existing work has neglected the structured knowledge unique to code and other useful information in code review scenarios, and the performance of the models used is relatively limited, making them impractical for real-world scenarios. Summary of the Invention

[0006] This invention proposes a pre-trained model for generating Java code review comments based on structured code information and review knowledge. The method involves modeling the problem, combining structured Java code information (in the form of an abstract syntax tree) and review knowledge (in the form of code change information), validating the model's performance through experiments, and analyzing the effectiveness of the generated results.

[0007] The technical solution adopted by this invention to solve its technical problem is:

[0008] A method for generating Java code review comments based on structured code information and review knowledge, comprising the following steps:

[0009] Data preparation phase: Obtain raw Java code review data;

[0010] Word vector representation stage: The code text in the original Java code review data is parsed using an abstract syntax tree to obtain its structured representation. The code review comments in the original Java code review data are processed by word segmentation and other methods. Both are vectorized according to the corresponding model vocabulary, including code text vectorization, code structure vectorization, and code review comment text vectorization, generating corresponding word vectors.

[0011] Unsupervised training phase: Based on the language model, noise is introduced into the word vectors in two ways: scrambling the vector representation and adding mask labels. Then, a discriminative graph model and language model (belonging to the graph model and language model in the pre-trained model) are built. The scrambled input content is restored through unsupervised training.

[0012] Supervised training phase: By integrating the graph model and language model from the unsupervised training phase, supervised training is performed on the basis of these models to generate code review comments, supplemented by a code change prediction task; the code change prediction task is to predict whether the current code segment needs to be deleted, added, or modified.

[0013] Based on the models generated during the unsupervised and supervised training phases, the system automatically outputs code review comments on the input code to be reviewed.

[0014] Furthermore, methods for establishing word vector representations of code text include: establishing serialized representations of code text and structured representations of abstract syntax trees.

[0015] Furthermore, the vector representations of the code text and code review comments are obtained through the following steps:

[0016] A code abstract syntax tree is built, and a formal representation of the code text is formed through hierarchical extraction. Then, the SBT (structure-based traversal) algorithm is used to transform it into a serialized vector representation.

[0017] Create a mapping dictionary to map English annotations to corresponding numbers, thus obtaining a vector representation of the annotation data (code review comment text).

[0018] Furthermore, the code review comments include opinions retained after reviewing the code, including but not limited to: bug finding, code improvement, and team communication.

[0019] Furthermore, code text vectorization and code structure vectorization are achieved through the following steps:

[0020] 1) By using the formal representation features in the javalang toolkit, code text can be represented as a code abstract syntax tree;

[0021] 2) The Java code text and its module hierarchy are displayed in a "tree" format, and function-level nodes are extracted and separated to achieve code text vectorization;

[0022] 3) The SBT algorithm is used to transform structured information into serialized information represented by special symbols through depth search;

[0023] 4) Input the obtained serialized information into the T5 model to obtain a vectorized representation of the structured information of the code text, thereby realizing code structure vectorization.

[0024] Furthermore, the structured information includes: node information, node relationships, and hierarchical division.

[0025] Furthermore, the language model is a pre-trained model on a large dataset, which can be used for transfer learning, that is, fine-tuning on new tasks to achieve better performance.

[0026] Furthermore, the language model includes: the T5 model (Text-to-Text Transfer Transformer).

[0027] Furthermore, the T5 model includes the t5-base-java model open-sourced by the Huggingface community.

[0028] Furthermore, the method for automatically outputting code review comments from the input code to be reviewed includes: employing a "noise generation and denoising" mode using random shuffling and masking models during the unsupervised training phase, and a multi-task fine-tuning mode during the supervised training phase. Here, multi-task refers to the code change prediction task and the code review comment generation task; the multi-task fine-tuning mode refers to fine-tuning the parameters of the pre-trained model based on these two tasks.

[0029] Furthermore, the automated code review comment output from the input code to be reviewed is achieved through the following steps:

[0030] 1) Discriminative Training. Based on the T5 model, a discriminator is added after its output layer to reverse predict the positions of damaged words in order to recover the original text data. The discriminator predicts the information of damaged nodes based on the text word vectors output by the T5 model, takes the highest predicted probability as the prediction result, and compares it with the original result to obtain feedback. The training is conducted for 5000 steps, with 16 sets of data per step.

[0031] 2) Masking Model Training. Text words are randomly masked, and then the masked words are reverse-predicted to restore the content. The training runs for 5000 steps, with 16 sets of data per step. The masking model may mask a single word, multiple consecutive words, or none at all.

[0032] 3) Code change information prediction. For each line of code in the code to be reviewed, the model performs a four-class classification task in the output layer by capturing the sentence vector label represented by the previous layer model, including: added, deleted, modified, and unchanged. The training takes 3600 steps, with 16 sets of data per step.

[0033] 4) Code review comment generation. The model generates text using the text transformation method of the T5 model. It predicts the output words sequentially in the output layer until a stop character is output or the maximum prediction length is reached. The training takes 3600 steps, with 16 sets of data per step.

[0034] Furthermore, the destructive words include: random word shuffling, random word removal, random word copying, and random word rotation.

[0035] A Java code review comment generation device based on structured code information and review knowledge, comprising:

[0036] The data acquisition module is used to obtain raw Java code review data;

[0037] The vectorization representation module is used to vectorize the code text and code review comments in the raw Java code review data, including code text vectorization, code structure vectorization, and code review comment text vectorization, generating corresponding word vectors.

[0038] The unsupervised training module is used to introduce noise into word vectors during the unsupervised training phase by scrambling the vector representation and adding mask labels. Then, a discriminative graph model and language model are built to restore the scrambled input content.

[0039] The supervised training module is used to generate code review comments by fusing the graph model and language model from the unsupervised training phase during the supervised training phase, while also assisting with code change prediction tasks.

[0040] The review comment generation module is used to automatically generate code review comments from the input code to be reviewed, based on the models generated during the unsupervised and supervised training phases.

[0041] A computer device includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.

[0042] A computer-readable storage medium storing a computer program that, when executed by a computer, implements the various steps of the method of the present invention.

[0043] Compared with the prior art, the significant advantages of this invention are:

[0044] 1) This invention uses a pre-trained language model to automatically generate code review comments. This model has the ability to automatically express features, which improves the efficiency of code review comment generation and alleviates the problems of time-consuming, labor-intensive and unstable effectiveness in manual review.

[0045] 2) This invention proposes a language model based on structured information and censorship knowledge. By introducing censorship scenario knowledge and explicit structured semantic information, the text representation capability of the model is further improved.

[0046] 3) This invention establishes two phases: unsupervised training and supervised training. In the unsupervised training phase, a "noise generation-de-noise reduction" model is used to teach the model code text representation and review comment text representation. In the supervised training phase, the model's ability to generate code review comments is improved, including by using code change information prediction. Unsupervised training learns high-quality text representations, while supervised training improves the quality of the generated results. The combination of these two approaches provides high decoupling while also enhancing the model's original performance. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the steps involved in generating code review comments based on structured code information and review knowledge.

[0048] Figure 2 This is a flowchart of the overall process for building the code review dataset.

[0049] Figure 3 This is a comparison example between this method and existing methods.

[0050] Figure 4 This is an example of automated code review comments. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0052] The Java code review comment generation method based on code structure information and review knowledge of the present invention has the following process: Figure 1 As shown, it includes the following steps:

[0053] Step 1: Collect the code review text content from the code files to be reviewed in the project, and preprocess the text content to generate a code review dataset;

[0054] Step 2: Vectorize the text data in the code review dataset, including code text vectorization, code structure vectorization, and code review comment text vectorization;

[0055] Step 3: Utilize the structured information of the code and perform unsupervised training on the model (T5) to obtain text data representations and continuously improve the model to obtain high-quality representations;

[0056] Step 4: Utilize knowledge of code review scenarios and conduct supervised training on the model (T5) to automatically generate and predict code review comments, thereby improving the quality of code review comments;

[0057] Step 5: For the new code to be reviewed, execute step 2 to obtain the identifier vector corresponding to the code to be reviewed. Then, input it into the model built through steps 3 and 4 to obtain the automated code review comments.

[0058] In one embodiment of the present invention, a method for generating code review comments based on code structure information and review knowledge is provided, the specific steps of which are as follows:

[0059] Step 1: As Figure 2 As shown, the code review dataset is generated by collecting code text content from the project's code files to be reviewed and preprocessing the text content. The specific steps are as follows:

[0060] Step 11: Crawl historical review data of the project from the GitHub community platform;

[0061] Step 12: Code review activities typically occur during software iterations, so this data is stored in the project's pull history. Each commit undergoes at least one code review. Leveraging this characteristic, this data is crawled to obtain the raw code review data.

[0062] Step 13: Data cleaning stage, which mainly involves removing invalid and duplicate code review activity data, including non-Java code, code that is too short or review comments, and response review data, etc.

[0063] Step 14: Use data augmentation methods, such as synonym replacement, to expand the target review comment text corresponding to the same code text to 10 copies, thereby increasing the original dataset size by 10 times.

[0064] Step 2: Vectorize the text data, including code text vectorization, code structure vectorization, and code review comment text vectorization. The specific steps are as follows:

[0065] Step 21: Represent it as a code abstract syntax tree by using the formal representation features in Javalang;

[0066] Step 22: Extract and separate the function-level nodes to obtain the function representations in the code file, thereby achieving code text vectorization;

[0067] Step 23: Using the SBT algorithm, the structured information is transformed into serialized information represented by special symbols through a deep search, thereby realizing the vectorization of the code structure;

[0068] Step 24: Use NLTK tools for word segmentation and part-of-speech tagging, use Oxford Dictionary to recover abbreviations, and remove non-text data, including emojis, etc.

[0069] Step 25: Build a mapping dictionary to map English words to corresponding numbers, thereby obtaining a vector representation of the code review text data and realizing the vectorization of code review comment text.

[0070] Step 3: Utilize the structured information of the code and perform unsupervised training using a language model (T5) to obtain text data representations, and continuously improve the model to obtain high-quality representations. The specific steps are as follows:

[0071] Step 31: Based on the T5 model, add a discriminator after its output layer to reverse predict the position of the damaged words in order to recover the original text data. The discriminator predicts the damaged node information based on the text word vector representation output by the T5 model, and takes the maximum predicted probability as the prediction result.

[0072] Step 32: Randomly mask the text words using a mask, then perform reverse prediction on the masked words to restore the content. The masking model may mask a single word or multiple consecutive words, or it may not mask any words.

[0073] Step 33: Using the T5 model, the damaged text is restored, and the result is compared with the original text to measure the goodness of fit. This process is repeated for further training.

[0074]

[0075]

[0076] Where, p D This represents the predicted probability of the original text, which is achieved using the Softmax method, meaning that for a specific word position x... t Its predicted probability is based on the T5 model h. T5 The learned word vector representation e(x) is normalized to a probability value. Naturally, the learning objective here is to improve the model's prediction accuracy for each word to be predicted. And reduce the prediction failure rate Inverting the latter and summing it with the former yields the Loss function L of the discriminant model. u1 x represents the text to be predicted, n represents the total predicted text, and θ represents the total predicted text. D Indicates model parameters, Represents the mathematical expectation. Represents the original text. The vector unit is 1.

[0077] Step 4: Utilize knowledge of the code review scenario and conduct supervised training using a language model (T5) to automatically generate and predict code review comments, thereby improving the quality of code review comments. The specific steps are as follows:

[0078] Step 41: For each line of code in the code to be reviewed, predict the code change information. The model captures the sentence vector labels represented by the previous layer model in the output layer and performs a four-class classification task, including: added, deleted, modified, and unchanged.

[0079] Step 42: The model generates text using the text transformation format of the T5 model, i.e. code review comment generation. It predicts the output words in sequence at the output layer until a stop character is output or the maximum prediction length is reached.

[0080] Step 43: Using the T5 model, the target content is automatically generated and compared with the labeled data to measure the goodness of fit. This process is repeated for training.

[0081]

[0082]

[0083] in, This indicates a change in the prediction loss function due to code modification. p represents the loss function for generating code review comments. l ij This represents the probability that the current i-th line of modified code is predicted to have the j-th type of modification label (0 means unchanged, 1 means added, 2 means deleted, 3 means modified), y l ij This represents a binary prediction label (0 for a wrong prediction, 1 for a correct prediction). Therefore, L s1 The average fit of the predictions in L lines of code in the current code was statistically analyzed, where S represents the total review comment text. This represents the prediction probability of the i-th reviewed text. This indicates the i-th original text to be reviewed.

[0084] Step 5: For the new defect report, execute steps (2) and (3) to obtain the vector corresponding to the defect report, and then input it into the defect report model saved in step 4 to obtain the list of repairers most suitable for repairing the defect.

[0085] Figure 3 This is a comparison example between our method and existing methods. Here, "Review Code Changes" indicates code changes, "Reviewer's comments" indicates code review comments, "Our Model" indicates the method of this invention, and "LSTM, CopyNet, and CodeBERT" represent three existing automatic text generation methods.

[0086] Figure 4 This is an example of automated code review comments. The gray text box contains the content of the code review comments automatically generated using the method of this invention, while the others are the code functions to be reviewed.

[0087] To illustrate the performance advantages of this invention, comparative experiments were conducted using LSTM, CopyNet, and CodeBERT text automatic generation models as benchmarks. To ensure objectivity and standardization of the results, the benchmark models in this experiment directly adopted the original implementations of each method, using the same random seed. The code review data was divided into training, validation, and test sets in an 8:1:1 ratio. After repeated experiments and averaging the results, the word accuracy of the LSTM classification method was 12.80%, CopyNet's word accuracy was 13.74%, and CodeBERT's word accuracy was 21.52%. The code review comment generation method based on code structure information and review knowledge achieved the highest prediction accuracy of 26.11%, which is 21.33% higher than the best benchmark model, CodeBERT.

[0088] Another embodiment of the present invention provides a Java code review comment generation device based on code structure information and review knowledge, comprising:

[0089] The data acquisition module is used to obtain raw Java code review data;

[0090] The vectorization representation module is used to vectorize the code text and code review comments in the raw Java code review data, including code text vectorization, code structure vectorization, and code review comment text vectorization, generating corresponding word vectors.

[0091] The unsupervised training module is used to introduce noise into word vectors during the unsupervised training phase by scrambling the vector representation and adding mask labels. Then, a discriminative graph model and language model are built to restore the scrambled input content.

[0092] The supervised training module is used to generate code review comments by fusing the graph model and language model from the unsupervised training phase during the supervised training phase, while also assisting with code change prediction tasks.

[0093] The review comment generation module is used to automatically generate code review comments from the input code to be reviewed, based on the models generated during the unsupervised and supervised training phases.

[0094] For the specific implementation process of each module, please refer to the description of the method of the present invention above.

[0095] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.

[0096] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) storing a computer program that, when executed by a computer, implements the various steps of the method of the present invention.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating Java code review comments based on structured code information and review knowledge, characterized in that, include: Obtain raw Java code review data; The code text and code review comments in the original Java code review data are vectorized, including code text vectorization, code structure vectorization, and code review comment text vectorization, generating corresponding word vectors; In the unsupervised training phase, noise is introduced into the word vectors in two ways: by scrambling the vector representation and by adding mask labels. Then, a discriminative graph model and a language model are built to restore the scrambled input content. During the supervised training phase, code review comments are generated by integrating the graph model and language model from the unsupervised training phase, while code change prediction tasks are also performed. Based on the models generated during the unsupervised and supervised training phases, the system automatically outputs code review comments on the input code to be reviewed. The loss function used in the unsupervised training phase is as follows: in, This represents the predicted probability of the original text, which uses the Softmax method, meaning that for a specific word position... Its predicted probability is based on the word vector representation learned by the T5 model. Normalize it to a probability value; x represents the text to be predicted. n This represents the total prediction text. Indicates model parameters, Represents the mathematical expectation. Represents the original text. The vector unit is one; The automated code review and comment output for the input code to be reviewed includes: 1) Discriminative training: Based on the T5 model, a discriminator is added after its output layer to reverse predict the position of the damaged words in order to recover the original text data; the discriminator predicts the damaged node information based on the text word vector representation output by the T5 model, takes the maximum predicted probability as the prediction result, and compares it with the original result to obtain the feedback result. 2) Masking model training: Text words are randomly masked, and then the masked words are reverse-predicted to restore the content. The masking model masks one word or multiple consecutive words, or does not mask any words. 3) Code change information prediction: For each line of code in the code to be reviewed, the output layer captures the sentence vector label represented by the previous layer model and performs a four-class classification task, including: added, deleted, modified, and unchanged; 4) Code review comment generation: Text is generated using the text transformation of the T5 model. The output words are predicted sequentially in the output layer until a stop character is output or the maximum prediction length is reached.

2. The method as described in claim 1, characterized in that, Methods for vectorizing the code text and the code review comments include: A code abstract syntax tree is built, and a formal representation of the code text is formed through hierarchical extraction. Then, the SBT algorithm is used to transform it into a serialized vector representation. Create a mapping dictionary to map English annotations to corresponding numbers, thus obtaining a vector representation of code review comments.

3. The method as described in claim 1, characterized in that, The following steps are used to vectorize code text and code structure: By using the formal representation features in the javalang toolkit, code text can be represented as a code abstract syntax tree; The Java code text and its module hierarchy are displayed in a "tree" format. Function-level nodes are extracted and separated to achieve code text vectorization. The SBT algorithm is used to transform structured information into serialized information represented by special symbols through a deep search. The obtained serialized information is input into the T5 model to obtain a vectorized representation of the structured information of the code text, thereby realizing the vectorization of the code structure; the structured information includes: node information, node relationships, and hierarchical division.

4. The method as described in claim 1, characterized in that, The language model includes the T5 model; the T5 model includes the t5-base-java model open-sourced by the Huggingface community.

5. The method as described in claim 1, characterized in that, The automated code review comment output of the input code to be reviewed includes: adopting a "noise generation and denoising" mode of random shuffling and masking model in the unsupervised training stage, and adopting a multi-task fine-tuning mode in the supervised training stage, where multi-task refers to code change prediction task and code review comment generation task.

6. The method as described in claim 1, characterized in that, The destructive words include: random word shuffling, random word removal, random word copying, and random word rotation.

7. A Java code review comment generation device based on code structure information and review knowledge, employing the method described in any one of claims 1 to 6, characterized in that, include: The data acquisition module is used to obtain raw Java code review data; The vectorization representation module is used to vectorize the code text and code review comments in the raw Java code review data, including code text vectorization, code structure vectorization, and code review comment text vectorization, generating corresponding word vectors. The unsupervised training module is used to introduce noise into word vectors during the unsupervised training phase by scrambling the vector representation and adding mask labels. Then, a discriminative graph model and language model are built to restore the scrambled input content. The supervised training module is used to generate code review comments by fusing the graph model and language model from the unsupervised training phase during the supervised training phase, while also assisting with code change prediction tasks. The review comment generation module is used to automatically generate code review comments from the input code to be reviewed, based on the models generated during the unsupervised and supervised training phases.

8. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 6.

Images