Code comment analysis method, computer device, storage medium and product

Abstract

The application relates to a code comment analysis method, computer equipment, a storage medium and a product. The code comment analysis method comprises the following steps: obtaining a target code comment pair, the target code comment pair comprising target code and target comment information; inputting the target code comment pair into a code comment analysis model to obtain an output result; the output result comprising an analysis result for indicating consistency between the target code and the target comment information; wherein the code comment analysis model is obtained by fine-tuning training based on a large language model, the fine-tuning training comprising fine-tuning training based on a code comment consistency detection task and / or fine-tuning training based on a code comment consistency repair task. The consistency between the code and the comment information is judged by using the code comment analysis model which is fine-tuned in a targeted manner, so that the accuracy of the quality evaluation of the code comment pair can be realized, and the software development and maintenance efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a code annotation analysis method, computer device, storage medium, and product. Background Technology

[0002] Code comments play a vital role in software development, helping developers understand code logic and facilitating team collaboration. However, during code evolution, inconsistencies may arise between the code and comments, misleading code readers and potentially masking functional defects or errors, thereby impacting the efficiency and quality of software development and maintenance.

[0003] In related technologies, to address the inconsistency between code and comments, predefined rules or patterns are typically used to detect inconsistencies. However, due to limitations in rule coverage, implicit or semantic inconsistencies are often overlooked. While machine learning-based methods can detect inconsistencies that are difficult to express using rules, they require a large amount of labeled data to train the model, and the model's performance largely depends on the quality of the training data, resulting in poor performance in code comment analysis. Summary of the Invention

[0004] This application provides a code comment analysis method, computer device, storage medium, and product. By utilizing a code comment analysis model that has been specifically fine-tuned and trained, the consistency between code and comment information can be determined, enabling accurate assessment of code comment quality and improving software development and maintenance efficiency. The technical solution is as follows.

[0005] On the one hand, a code comment analysis method is provided, which includes:

[0006] Obtain target code comment pairs, wherein the target code comment pairs contain target code and target comment information;

[0007] The target code annotation pair is input into the code annotation analysis model to obtain the output result; the output result includes analysis results indicating the consistency between the target code and the target annotation information;

[0008] The code annotation analysis model is obtained by fine-tuning training based on a large language model. The fine-tuning training includes fine-tuning training based on a code annotation consistency detection task and / or fine-tuning training based on a code annotation consistency repair task.

[0009] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the above-described code annotation analysis method.

[0010] On the other hand, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer program, which is loaded and executed by a processor to implement the above-described code annotation analysis method.

[0011] On the other hand, a computer program product is provided, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to execute to implement the code comment analysis method provided in the various optional implementations described above.

[0012] The code comment analysis method provided in this application, when analyzing target code comment pairs, inputs the target code comment pairs into a code comment analysis model trained based on a large language model, obtains and outputs analysis results indicating the consistency between code and comments in the target code comment pairs; wherein, the code comment analysis model is obtained through fine-tuning training of code comment consistency detection tasks and / or repair tasks; by using the code comment analysis model trained with targeted fine-tuning to judge the consistency between code and comment information, the accuracy of the quality assessment of code comment pairs can be achieved, improving software development and maintenance efficiency; in addition, the code comment analysis model obtained by fine-tuning training based on a large language model can improve model training efficiency and model training effect.

[0013] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0015] Figure 1 A flowchart of a code annotation analysis method provided in an exemplary embodiment of this application is shown;

[0016] Figure 2 This diagram illustrates a framework diagram of model training and code annotation analysis provided in an exemplary embodiment of this application.

[0017] Figure 3 A flowchart of a code annotation analysis method provided by another exemplary embodiment of this application is shown;

[0018] Figure 4 This application shows a structural block diagram of a computer device according to an exemplary embodiment.

[0019] Figure 5 A structural block diagram of another computer device illustrated in an exemplary embodiment of this application is shown. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of methods consistent with some aspects of this application as detailed in the appended claims.

[0021] This application provides a code annotation analysis method. Based on a large language model, a code annotation analysis model capable of performing code annotation consistency detection tasks is obtained through fine-tuning and training of the large language model. Alternatively, a code annotation analysis model capable of performing code annotation consistency detection and repair tasks simultaneously can be obtained. By fine-tuning and training the large language model to construct the code annotation analysis model, the model training efficiency and training effect can be improved. Applying this code annotation analysis model to code annotation analysis can improve analysis efficiency while enhancing the performance of code annotation consistency detection and repair, thereby improving the overall code annotation analysis effect.

[0022] Figure 1 This application illustrates a flowchart of a code comment analysis method provided in an exemplary embodiment. This method can be executed by a code comment analysis device, which can be a computer device, such as a server or terminal. Figure 1 As shown, this code comment analysis method may include the following steps.

[0023] Step 110: Obtain the target code comment pair, which contains the target code and target comment information.

[0024] Among them, the target code comment pair is a data unit that combines the target code and its corresponding target comment information, serving as the basic input unit for subsequent analysis; the target code refers to the program code segment that needs to be analyzed, and the target comment information is a text description added to the target code to explain the function, purpose, input and output requirements, implementation logic, etc., to assist relevant personnel in understanding the code. The comment information is usually embedded in the code in a specific syntax format. Different programming languages ​​have different commenting methods, and this application does not impose any restrictions on this.

[0025] The target code comment pair can be uploaded during code modification or when verifying code already in the repository. For example, the target code comment pair can be code snippets and their corresponding comment information uploaded by relevant personnel during code review. Alternatively, the target code comment pair can be code and its comments automatically extracted by code parsing tools based on the syntax rules of the programming language, traversing the code repository. During code parsing, comment identification and extraction can be performed based on key comment elements. For example, key comment elements can be @return, @param, and @summary, etc. Different programming languages ​​may have different key comment elements. This application does not limit the input method or source of the target code comment pair.

[0026] Step 120: Input the target code comment pair into the code comment analysis model to obtain the output result; the output result contains the analysis result used to indicate the consistency between the target code and the target comment information.

[0027] The code annotation analysis model is obtained by fine-tuning the large language model. The fine-tuning training includes fine-tuning training based on the code annotation consistency detection task and / or fine-tuning training based on the code annotation consistency repair task.

[0028] Indicatively, the large language model can be a series of large language models such as GPT or LLaMA. By fine-tuning and training the large language model based on different tasks, a code annotation analysis model can be obtained. This code annotation analysis model is used to analyze the input code annotation pairs and determine the consistency between the code and the annotation information.

[0029] The consistency between code and comment information can be measured from multiple dimensions, such as functional consistency (whether the function described in the comment information matches the actual function implemented in the code) and logical consistency (whether the logic described in the comment is consistent with the execution logic). The analysis results can be presented in different forms. For example, the analysis results can be numerical values ​​representing the degree of consistency, such as values ​​between 0 and 1, with higher values ​​indicating stronger consistency. Alternatively, the analysis results can be a classification result, such as consistent, inconsistent, or partially consistent. Different classification results correspond to different numerical ranges of consistency. This application does not restrict the correspondence between the degree of consistency and the numerical range.

[0030] When the code comment analysis model is fine-tuned and trained on a large language model based on a code comment consistency detection task, it can be used to analyze the consistency between code and comment information in input code comment pairs. When the code comment analysis model is fine-tuned and trained on both code comment consistency detection and code comment consistency repair tasks, it can not only analyze the consistency between code and comment information in input code comment pairs, but also repair the code comment pairs when inconsistencies are detected. The label information of the sample code comment pairs corresponding to different training tasks is adapted to the training task.

[0031] When applying the code comment analysis model, after receiving the target code comment pair, the model uses the knowledge and capabilities it has learned during model training to perform consistency analysis and reasoning on the code and comment information in the target code comment pair, and obtain the analysis results.

[0032] In summary, the code comment analysis method provided in this application, when analyzing target code comment pairs, inputs the target code comment pairs into a code comment analysis model trained based on a large language model, and obtains and outputs analysis results indicating the consistency between code and comments in the target code comment pairs. This code comment analysis model is obtained through fine-tuning training of code comment consistency detection and / or repair tasks. Using a code comment analysis model trained with targeted fine-tuning to determine the consistency between code and comment information can achieve accurate evaluation of the quality of code comment pairs, improving software development and maintenance efficiency. Furthermore, fine-tuning the code comment analysis model based on a large language model can improve model training efficiency and effectiveness.

[0033] In this embodiment, the code annotation analysis method can be divided into a model training phase and a code annotation analysis phase. Figure 2 This application illustrates a framework diagram for model training and code annotation analysis provided in an exemplary embodiment, as shown below. Figure 2 As shown, in the code annotation analysis model training phase, the model training device 210 fine-tunes the large language model using a pre-set training sample set (including positive sample code annotation pairs and negative sample code annotation pairs) to obtain the code annotation analysis model. In the code annotation analysis phase, the code annotation analysis device 220 analyzes the input code annotation pairs based on this model to obtain the output results for the sample code annotation pairs. These output results include analysis results indicating the consistency between the code and annotation information.

[0034] The aforementioned model training device 210 and code annotation analysis device 220 can be computer devices, such as fixed computer devices like personal computers and servers, or mobile computer devices like tablet computers and e-book readers.

[0035] Optionally, the model training device 210 and the code annotation analysis device 220 can be the same device, or they can be different devices. Furthermore, when the model training device 210 and the code annotation analysis device 220 are different devices, they can be of the same type, such as both being servers; or they can be of different types, such as the code annotation analysis device 220 being a personal computer or terminal, while the model training device 210 can be a server, etc. This application embodiment does not limit the specific types of the model training device 210 and the code annotation analysis device 220.

[0036] Taking the example that the model training equipment and the model application equipment are the same equipment, collectively referred to as computer equipment, Figure 3 A flowchart of a code annotation analysis method provided in another exemplary embodiment of this application is shown. This method can be executed by a computer device, which can be implemented as a server or a terminal, such as... Figure 3 As shown, this code comment analysis method may include the following steps.

[0037] During the model training phase:

[0038] Step 301: Obtain code commit records. These records include historical code comment pairs for each original code at different change stages. Each historical code comment pair contains historical code and its corresponding historical comment information.

[0039] Code commit records are generated during software development when relevant personnel submit completed code modifications or new code snippets to the code management system. These records can include historical code comment pairs from each change phase, as well as the commit time. In the case of existing code, these commit records can originate from the code database of the code management system, meaning that code snippets from different change phases and their corresponding comment information are already stored in the code repository. In the case of code submission to the repository, these commit records are generated based on real-time recording of code commit operations.

[0040] Original code refers to the basic code initially written or introduced in a software development project. It serves as the foundation for subsequent modifications, optimizations, and extensions. In the code commit history, each piece of original code can correspond to multiple pairs of historical code comments. Each pair of historical code comments corresponds to a different change phase. A change phase can correspond to one code commit operation. This code commit operation can be initiated proactively by relevant personnel or automatically initiated by the system according to a target cycle. In a change phase, at least one of the code and its corresponding comment information may change, or neither the code nor its corresponding comment information may change.

[0041] In some embodiments, after obtaining the initial code commit record, the computer device can preprocess the data in the collected initial code commit record to improve data quality. The preprocessing process may include removing non-substantive changes such as noise and erroneous data, such as adjusting the comment format or correcting typos, to ensure the accuracy of subsequent consistency analysis and avoid consistency judgment errors caused by data quality issues.

[0042] Step 302: Semantic recognition is performed on the historical code comments of the target source code at different change stages to obtain the semantic recognition result of the target source code. The semantic recognition result is used to indicate the consistency between historical codes and the consistency of historical comment information at different change stages. The target source code is any source code in the code commit record.

[0043] An original code may correspond to multiple change stages, and each change stage has a corresponding historical code comment pair. For historical code comment pairs of different change stages, the computer device can perform semantic recognition and comparison on adjacent historical code comment pairs in chronological order. In the comparison, the semantic recognition results of the historical code in different historical code comment pairs are compared, as well as the semantic recognition results of the historical comment information in different historical code comment pairs, to determine whether code changes and comment information changes have occurred in different historical code comment pairs.

[0044] In some embodiments, a computer device can utilize natural language processing and code analysis techniques to understand and analyze the comment information and code in historical code comment pairs, mining their respective semantic information for comparison. The semantic recognition result can be represented as a numerical value indicating the degree of consistency, such as 80%, 50%, etc., or it can be represented as a consistency category, i.e., consistent or inconsistent. Different consistency categories can correspond to different consistency value ranges. For example, when the consistency value falls within the range of 80% to 100%, the semantic recognition result is determined to be consistent; when the consistency value falls within the range of 0% to 30%, the result is determined to be inconsistent. If the semantic recognition result is determined to be inconsistent, and does not fall into either of the two regions, the semantic recognition result is determined to be partially consistent. The above consistency value range is only illustrative and can be set differently based on different actual consistency comparison needs. This application does not impose any restrictions on this. Furthermore, in order to improve the accuracy of model training, when constructing the training dataset, the historical code annotation pairs corresponding to the original code with partially consistent semantic recognition results can be removed, or the historical code annotation pairs of the original code with partially consistent semantic recognition results can be manually judged and their consistency values ​​modified so that they fall into the numerical range corresponding to consistency or inconsistency.

[0045] Step 303: Based on the semantic recognition results of the target original code, perform consistent labeling on each historical code annotation pair of the target original code to obtain sample code annotation pairs in the training sample set.

[0046] The training sample set for fine-tuning the large language model includes positive sample code annotation pairs and negative sample code annotation pairs. The consistency label of the positive sample code annotation pair indicates that the sample code is consistent with the annotation information, while the consistency label of the negative sample code annotation pair indicates that the sample code is inconsistent with the annotation information.

[0047] In positive sample code comment pairs, the sample code and comment information are consistent in function and logic. For example, in a sample code comment pair, a piece of code implements the function of calculating the sum of two data, and its corresponding comment information is "#This function is used to calculate the sum of two input data", then the sample code comment pair is a positive sample code comment pair. In negative sample code comment pairs, the sample code and comment information may have problems such as inconsistent function description or logical interpretation errors. For example, in a sample code comment pair, a piece of code is used to implement file reading operation, and its corresponding comment information is "#This code is used to write to a file", then the sample code comment pair is a negative sample code comment pair.

[0048] In the case of fine-tuning training of a large language model based on the code annotation consistency repair task, negative sample code annotation pairs include repair suggestion information labels. The code annotation analysis model is also used to output repair suggestion information and / or analysis results of the target code annotation pair when the target code and target annotation information are inconsistent.

[0049] In other words, for cases where the sample code and comment information in a negative sample code comment pair are inconsistent, modification suggestion labels can be pre-defined. For example, for the negative sample code comment pair with the above-mentioned file reading code comment error, the corresponding repair suggestion label could be "modify the comment information to '#This code is used to read file content'". By fine-tuning the large language model with negative sample code comment pairs containing modification suggestion labels, the trained code comment analysis model learns the ability to repair code comment consistency. Thus, when the model is applied, it can automatically correct code comments, reduce manual intervention, and improve work efficiency.

[0050] Therefore, when constructing the training sample set, it is necessary to determine the labels of each historical code annotation pair based on the semantic recognition results of each original code. In some embodiments, the process of consistent labeling of each historical code annotation pair of the target original code based on the semantic recognition results of the target original code to obtain the sample code annotation pairs in the training sample set can be implemented as follows:

[0051] If the semantic recognition result indicates that the original code is consistent with the historical code under different change stages, and the historical comment information is consistent, then the consistency label of each historical code comment pair corresponding to the original code indicates the positive sample code comment pair; since the historical code has not changed in different change stages, and the historical comment information has not changed either, it means that the relevant personnel did not modify the code and comment information when submitting the code, and the historical code and historical comment information are consistent.

[0052] If the semantic recognition result indicates that the historical code of the original code is consistent across different change stages, but the historical comment information is inconsistent, then the consistency label of the historical code comment pair before the original code change is determined to be a negative sample code comment pair, and the historical comment information after the change is the repair suggestion information label of the historical code comment pair before the change. Since the historical code has not changed across different change stages, but the historical comment information has changed, it indicates that the relevant personnel made corresponding adjustments or corrections to the historical comment information when submitting the code. Therefore, it can be determined that the historical code before the change is inconsistent with the historical comment information, and its corresponding historical code comment pair is a negative sample code comment pair. The historical code after the change is consistent with the historical comment information, and its corresponding historical code comment pair is a positive sample code comment pair. At the same time, the historical comment information after the change, compared with the historical code comment pair before the change, can be used as the repair suggestion information label of the historical code comment pair before the change.

[0053] If the semantic recognition result indicates that the historical code of the original code is inconsistent under different change stages, and the historical annotation information is inconsistent, then the consistency label of the historical code annotation pair corresponding to the original code is determined to be a positive sample code annotation pair. Since the historical code has changed in different change stages, the historical annotation information has also changed, indicating that when the relevant personnel made code submissions, they adjusted the historical code and the corresponding historical annotation information to maintain consistency between the two. Therefore, historical code annotation pairs where the historical code and historical annotation information are inconsistent can be determined as positive sample code annotation pairs.

[0054] Step 304: In the case of fine-tuning training including fine-tuning training based on code annotation consistency detection task, the target sample code annotation pair and the first prompt word are input into the large language model to obtain the first prediction result output by the large language model; the first prediction result is used to indicate the consistency between the sample code and the annotation information in the target sample code annotation pair; the target sample code annotation pair is any one of the training samples; the first prompt word is used to instruct the large language model to determine the consistency between the code and the annotation information in the input code annotation.

[0055] Fine-tuning is a model training technique based on a pre-trained model. In this embodiment, based on a large language model, a relatively small-scale code comment consistency detection task and / or a code comment consistency repair task are used to further train and adjust some or all of the model's parameters. The fine-tuning steps include: forward propagation, calculating the loss function, backpropagation, and updating parameters.

[0056] Since large language models typically process and analyze data based on smaller units, in some embodiments, before the first prompt word and the sample code annotation pairs from the training sample set are input into the large language model, the computer device can use a word segmenter to segment the first prompt word and the sample code annotation pairs to obtain a data format suitable for the large language model to receive as input, and which the large language model can output in the same form after processing. Here, word segmentation refers to dividing the text into smaller word units, also called tokens, according to certain rules. Converting the sample code annotation pairs into word unit form helps the model better understand and process the data.

[0057] Taking the target sample code annotation pair as an example, during model training, the target code annotation pair after word segmentation and the first prompt word are input into the large language model. The first prompt word is used to instruct the large language model to reason according to the first thought chain, which is to judge whether there is an inconsistency in the code annotation. Based on the first thought chain, the large language model predicts the performance of the (n+1)th token according to the nth token, and obtains the logist tensor output by the large language model, which is the first prediction result, thus completing the forward propagation.

[0058] Step 305: Calculate the first loss function for the corresponding consistency label based on the first prediction result and the sample code annotation.

[0059] Schematic, the first loss function can be expressed as:

[0060]

[0061] Where ϵ is the label smoothing coefficient, used to reduce the impact of errors in manual labeling on the task, and P(I│"Ins,C,M") is the large language model's judgment on consistency I given the first prompt word Ins, sample code C and annotation information M.

[0062] Alternatively, the first loss function can also be expressed as:

[0063]

[0064] Step 306: Fine-tune the parameters of the large language model based on the function value of the first loss function until the training completion condition is met, and obtain the code annotation analysis model.

[0065] After calculating the value of the first loss function, the computer device can calculate the model gradient through backpropagation and update the model parameters. After multiple iterations and reaching the training completion condition, the code-annotated analysis model is obtained. The training completion condition may refer to the accuracy of the model output reaching an accuracy threshold, or the number of training iterations reaching a number threshold, etc. This application does not impose any restrictions on this.

[0066] Step 307: In the case of fine-tuning training including fine-tuning training based on code annotation consistency detection task and fine-tuning training based on code annotation consistency repair task, the target sample code annotation pair and the second prompt word are input into the large language model to obtain the second prediction result output by the large language model; the second prediction result is used to indicate that the sample code in the target sample code annotation pair is consistent with the annotation information, or the second prediction result is used to indicate that the sample code in the target sample code annotation pair is inconsistent with the annotation information, and the prediction repair suggestion information corresponding to the target sample code annotation pair is output; the target sample code annotation pair is any one in the training sample set.

[0067] Corresponding to step 304, during model training, the target code annotation pairs after word segmentation and the second prompt word are input into the large language model. The second prompt word instructs the large language model to reason according to the second thought chain. The second thought chain is to first determine whether there is any inconsistency in the code annotations, and then repair the inconsistencies. Based on the second thought chain, the large language model predicts the performance of the (n+1)th token according to the nth token, and obtains the logist tensor output by the large language model, which is the second prediction result, thereby completing the forward propagation.

[0068] Taking the target sample code comment pair as an example, when performing code comment inconsistency detection and repair, the large language model receives the target sample code comment pair and generates a judgment and repair of code comment consistency according to the sequence. The sequence generated by the large language model is as follows:

[0069]

[0070] Large language models will As input for calculation The probability distribution is determined, and the next token is selected by sampling. In the process of continuously generating the next token, consistency judgment and repair of the code comment pairs of the target sample are generated.

[0071] Step 308: Calculate the comprehensive loss function based on the second prediction result and the sample code annotation for the corresponding consistency label and repair suggestion information label.

[0072] Generally, when calculating model loss, each token has an equal weight. However, since the proportion of tokens used to judge code comment consistency is relatively low when training through code comment consistency detection and code comment consistency repair tasks, equal distribution would result in fewer training weights for the code comment consistency detection task. To ensure the overall performance of the detection and repair tasks, the computer can train the model using key reinforcement fine-tuning. The improvement direction of key reinforcement fine-tuning is to guide the optimization direction of the model by changing the calculation method of the loss function, thereby enabling the trained model to have better performance. That is, by strengthening the tokens corresponding to the code comment consistency detection task, the performance of the code comment consistency detection task is guaranteed, thus ensuring the overall performance of the detection and repair tasks.

[0073] The comprehensive loss function consists of a first loss function corresponding to the code comment consistency detection task and a second loss function corresponding to the code comment consistency repair task; the first loss function and the second loss function each have corresponding training weights.

[0074] Schematic, the comprehensive loss function can be expressed as:

[0075]

[0076] in, For the first loss function, For the second loss function, x i For the i-th generated token, the second loss function can represent the probability that the model inferences to obtain the repair suggestion information tag; This is a hyperparameter that can adjust the training weights for the code comment consistency detection task and the code comment consistency repair task.

[0077] Step 309: Fine-tune the parameters of the large language model based on the function value of the comprehensive loss function until the training completion condition is met, and obtain the code annotation analysis model.

[0078] After obtaining the value of the comprehensive loss function, the computer device uses backpropagation to calculate the model gradient and update the model parameters. After multiple iterations and reaching the training completion condition, the code-annotated analysis model is obtained.

[0079] Once trained, the code comment analysis model can be applied to real-world scenarios, such as performing code comment consistency checks and repairs in scenarios where code has already been incorporated into the repository to maintain the accuracy of code comment pairs in the repository; and performing code comment consistency checks and repairs in code submission scenarios to ensure the accuracy of submitted code.

[0080] As an illustration, in one possible scenario, this code comment analysis model can be applied to integrated development environments (IDEs) commonly used by developers, such as Visual Studio, Eclipse, and IntelliJ IDEA. This model can be installed as a plugin. When a developer modifies existing code (lines) during the coding process, the plugin detects this in real time and sends the code before and after the modification, along with necessary contextual information (such as the code segment of the current function, the declaration of the called function, etc.), to the code comment analysis model. The model then provides the developer with comment suggestions for the modified code (lines), which the developer can adopt with a single click. The plugin feeds back the developer's adoption status, their own comments, and modifications made to the comments provided by the larger model through a reward function in the background, continuously optimizing the behavior and output of the code comment analysis model.

[0081] In another possible scenario, a code comment analysis step can be added to the CI / CD (Continuous Integration / Continuous Delivery) build and testing process. This can be integrated with CI / CD tools (such as Jenkins, Travis CI, Circle CI, etc.) to automate code comment consistency checks and fixes. For example, if the code comment analysis model determines that the received code comments are inconsistent, it will issue an alert to remind relevant personnel to check and confirm. The confirmation results and content of the relevant personnel can be fed back to the large model in the background through a reward function, continuously optimizing the behavior and output of the code comment analysis model.

[0082] In another possible scenario, when updating the code annotation analysis model, during the preparation of the training sample set for the code annotation analysis model, the quality of code data can be detected and improved through the current code annotation analysis model. Combined with manual correction, this can improve the quality of sample code annotation pairs, thereby improving the model update effect.

[0083] In another possible scenario, this code comment analysis model can also be implemented outside of code hosting platforms, such as GitHub, GitLab, and Bitbucket, to provide code review and ensure code quality. This code comment analysis model can interface with the API of the code hosting platform to obtain information such as code commits and merge requests, and perform automated code comment consistency detection and repair.

[0084] It should be noted that the above application scenarios are only illustrative. Based on different actual needs, the code annotation analysis model and code annotation analysis method provided in this application can be applied to corresponding scenarios, and this application does not impose any restrictions on them.

[0085] During the code comment analysis phase:

[0086] Step 310: Obtain the target code comment pair, which contains the target code and target comment information.

[0087] Step 311: Input the target code comment pair into the code comment analysis model to obtain the output results; the output results include analysis results used to indicate the consistency between the target code and the target comment information.

[0088] In the case where the code comment pair analysis model is fine-tuned based on the code comment consistency detection task and the code comment consistency repair task, the output results also include repair suggestions for the target code comment pair when the analysis results indicate that the target code and the target comment information are inconsistent.

[0089] In some embodiments, the computer device may display the output results through a user interface to provide feedback to the user.

[0090] Furthermore, users can also provide feedback on the output results through the user interface. The user feedback information can be binary information, such as good or bad, or it can be score information, such as a score of 1-5. Correspondingly, the computer device receives the feedback information submitted by the user based on the user interface, which corresponds to the output results. In some embodiments, the computer device can store the user feedback information and count the types of feedback information received within the target time period.

[0091] If the number of negative feedback messages received within the target time period reaches the target condition, the code annotation analysis model is retrained.

[0092] In other words, users can judge the accuracy of the model output by analyzing the consistency of code comments and modification suggestions in the received output results, and submit feedback through the user interface. If there are a lot of negative feedback from users within a certain period of time, it is determined that the analysis effect of the current code comment analysis model needs to be improved, and the code comment analysis model is retrained, i.e., return to step 301.

[0093] In other embodiments, the computer device can reinforce the code annotation analysis model based on received user feedback through a feedback mechanism. Illustratively, this process can be implemented by converting user feedback into reward scores and optimizing the model using techniques such as Reinforcement Learning from Human Feedback (RLHF). For example, if user feedback indicates that the model's output is accurate and helpful, a higher reward score is assigned to that output; if user feedback indicates that the model's output is incorrect or irrelevant, a lower reward score is assigned. Data labeled with reward scores is collected as training samples and input into a reinforcement learning-based training process to reinforce the code annotation consistency analysis model. This allows the model to continuously adjust its parameters to maximize the reward score, enabling the reinforced model to output more in line with user expectations and further improving the overall performance of the code annotation analysis model.

[0094] In summary, the code comment analysis method provided in this application, when analyzing target code comment pairs, inputs the target code comment pairs into a code comment analysis model trained based on a large language model, obtains and outputs analysis results indicating the consistency between code and comments in the target code comment pairs; wherein, the code comment analysis model is obtained through fine-tuning training of code comment consistency detection tasks and / or repair tasks; using the code comment analysis model trained with targeted fine-tuning to judge the consistency between code and comment information can achieve accurate evaluation of the quality of code comment pairs, improving software development and maintenance efficiency; and obtaining the code comment analysis model through fine-tuning training based on a large language model can improve model training efficiency and model training effect.

[0095] Furthermore, the code annotation analysis model trained with negative sample code annotation pairs and positive sample code annotation pairs containing repair suggestion information labels can output repair suggestion information after determining inconsistencies in code annotations, thereby improving the maintenance effect or modification efficiency of code annotations and improving the accuracy of code annotations.

[0096] Figure 4This diagram illustrates a structural block diagram of a computer device 400 according to an exemplary embodiment of this application. This computer device can be implemented as a model training device or a code annotation analysis device as described above. The computer device 400 includes a Central Processing Unit (CPU) 401, a system memory 404 including Random Access Memory (RAM) 402 and Read-Only Memory (ROM) 403, and a system bus 405 connecting the system memory 404 and the CPU 401. The computer device 400 also includes a mass storage device 406 for storing an operating system 409, a client 410, and other program modules 411. The system memory 404 and the mass storage device 406 can be collectively referred to as memory.

[0097] According to various embodiments of this application, the computer device 400 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 400 can be connected to a network 408 via a network interface unit 407 connected to the system bus 405, or the network interface unit 407 can be used to connect to other types of networks or remote computer systems (not shown).

[0098] The memory also includes at least one instruction, at least one program, code set, or instruction set, which are stored in the memory. The central processing unit 401 executes the at least one instruction, at least one program, code set, or instruction set to implement all or part of the steps in the code annotation analysis method shown in the above embodiments.

[0099] Figure 5 A structural block diagram of another computer device 500 illustrated in an exemplary embodiment of this application is shown. This computer device 500 can be implemented as the model training device or code annotation analysis device described above. For example, the computer device can be an Android terminal device; typically, the computer device 500 includes a processor 501 and a memory 502. The memory 502 may include one or more computer-readable storage media for storing at least one instruction, which is executed by the processor 501 to implement all or part of the steps in the code annotation analysis method shown in the method embodiments of this application.

[0100] In some embodiments, the computer device 500 may optionally include a peripheral device interface 503 and at least one peripheral device. The processor 501, memory 502, and peripheral device interface 503 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 503 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 504, a display screen 505, a camera assembly 506, an audio circuit 507, and a power supply 508. In some embodiments, the computer device 500 further includes one or more sensors 509. These sensors 509 include, but are not limited to, an accelerometer 510, a gyroscope 511, a pressure sensor 512, an optical sensor 513, and a proximity sensor 514. Those skilled in the art will understand that... Figure 5 The structure shown does not constitute a limitation on the computer device 500, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0101] In one exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program that is loaded and executed by a processor to implement all or part of the steps in the code annotation analysis method described above. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, or optical data storage device, etc.

[0102] In one exemplary embodiment, a computer program product is also provided, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the above-described actions. Figure 1 or Figure 2 All or part of the steps of the embodiments shown in any of the embodiments.

[0103] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0104] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A code comment analysis method, characterized in that, The method includes: Obtain target code comment pairs, wherein the target code comment pairs contain target code and target comment information; The target code annotation pair is input into the code annotation analysis model to obtain the output result; the output result includes analysis results indicating the consistency between the target code and the target annotation information; The code annotation analysis model is obtained by fine-tuning a large language model. The fine-tuning training includes fine-tuning training based on the code annotation consistency detection task. The training sample set for fine-tuning the large language model includes positive sample code annotation pairs and negative sample code annotation pairs. The consistency label of the positive sample code annotation pair indicates that the sample code is consistent with the annotation information, and the consistency label of the negative sample code annotation pair indicates that the sample code is inconsistent with the annotation information. The fine-tuning training based on the code comment consistency detection task includes: The target sample code annotation pair and the first prompt word are input into the large language model to obtain the first prediction result output by the large language model; the first prediction result is used to indicate the consistency between the sample code and the annotation information in the target sample code annotation pair; the target sample code annotation pair is any one of the training samples; the first prompt word is used to instruct the large language model to determine the consistency between the code and the annotation information in the input code annotation; Based on the first prediction result and the sample code annotation, a first loss function is calculated for the corresponding consistency label; Based on the function value of the first loss function, the parameters of the large language model are fine-tuned until the training completion condition is met, thus obtaining the code annotation analysis model.

2. The method according to claim 1, characterized in that, The fine-tuning training also includes fine-tuning training based on the code comment consistency repair task.

3. The method according to claim 1 or 2, characterized in that, The negative sample code annotation pair includes repair suggestion information tags, and the code annotation analysis model is also used to output repair suggestion information of the target code annotation pair and / or the analysis results when the target code and the target annotation information are inconsistent.

4. The method according to claim 1, characterized in that, The method includes: Obtain code commit records, which include historical code comment pairs of each original code at different change stages. The historical code comment pairs contain historical code and the historical comment information corresponding to the historical code. Semantic recognition is performed on historical code comments of the target source code at different change stages to obtain the semantic recognition result of the target source code. The semantic recognition result is used to indicate the consistency between the historical code and the consistency of historical comment information at different change stages. The target source code is any source code in the code commit record. Based on the semantic recognition results of the target original code, consistency labeling is performed on each historical code annotation pair of the target original code to obtain the sample code annotation pairs in the training sample set.

5. The method according to claim 4, characterized in that, Based on the semantic recognition result of the target original code, consistent labeling is performed on each historical code annotation pair of the target original code to obtain sample code annotation pairs in the training sample set, including: If the semantic recognition result indicates that the original code is consistent with the historical code under different change stages, and the historical annotation information is consistent, then the consistency label of each historical code annotation pair corresponding to the original code is determined to be a positive sample code annotation pair; If the semantic recognition result indicates that the historical code of the original code is consistent under different change stages, but the historical annotation information is inconsistent, then the consistency label of the historical code annotation pair before the change of the original code is determined to be the negative sample code annotation pair, and the historical annotation information after the change is the repair suggestion information label of the historical code annotation pair before the change. If the semantic recognition result indicates that the historical code of the original code is inconsistent under different change stages, and the historical annotation information is inconsistent, then the consistency label of the historical code annotation pair corresponding to the original code is determined to be a positive sample code annotation pair.

6. The method according to claim 2, characterized in that, The method includes fine-tuning training based on a code comment consistency detection task and fine-tuning training based on a code comment consistency repair task, wherein the negative sample code comment pairs have corresponding repair suggestion information labels. The target sample code annotation pair and the second prompt word are input into the large language model to obtain the second prediction result output by the large language model; the second prediction result is used to indicate that the sample code in the target sample code annotation pair is consistent with the annotation information, or the second prediction result is used to indicate that the sample code in the target sample code annotation pair is inconsistent with the annotation information, and outputs prediction repair suggestion information corresponding to the target sample code annotation pair; the target sample code annotation pair is any one of the training sample sets; Based on the second prediction result and the sample code annotation, a comprehensive loss function is calculated for the corresponding consistency label and repair suggestion information label; The parameters of the large language model are fine-tuned based on the value of the comprehensive loss function until the training completion condition is met, thus obtaining the code annotation analysis model.

7. The method according to claim 6, characterized in that, The comprehensive loss function consists of a first loss function corresponding to the code comment consistency detection task and a second loss function corresponding to the code comment consistency repair task; the first loss function and the second loss function each have corresponding training weights.

8. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The output results are displayed through a user interface.

9. The method according to claim 8, characterized in that, The method further includes: Receive feedback information from the user based on the user interface, corresponding to the output result; The types of feedback information received within the target time period are statistically analyzed. If the number of negative feedback messages received within the target time period reaches the target condition, the code annotation analysis model is retrained.

10. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, which is loaded and executed by the processor to implement the code annotation analysis method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the code annotation analysis method as described in any one of claims 1 to 9.

12. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the code annotation analysis method as described in any one of claims 1 to 9.

Images