Code annotation generation method based on large language model and multi-modal comparative learning

Abstract

The invention discloses a code annotation generation method based on a large language model and multi-modal comparative learning, which comprises the following steps: firstly, designing a universal prompt template to help the large language model to generate similar code segments, and introducing the thought of gradually generating a thinking chain into the prompt template to effectively improve the accuracy of a generation result; meanwhile, the realization modes and styles of the generated similar code segments and source codes are variable, and the quality is higher; secondly, aligning source codes, similar codes and annotations thereof through multi-modal comparative learning, and enhancing the understanding and matching capability of the model on semantic relationships between the codes, the similar codes and the annotations by designing intra-modal comparative learning and inter-modal comparative learning, so as to better capture deep association of the similar codes, the annotations and the source codes; and finally, designing an evaluation method based on a large language model, and more comprehensively evaluating the quality of the generated annotations from four aspects of correlation, integrity, clarity and understandability by means of the understanding ability of the large language model.

Description

Technical Field

[0001] The present invention relates to the technical field of code comment generation, and in particular to a code comment generation method based on a large language model and multimodal contrastive learning. Background Art

[0002] With the continuous increase in software functionality and complexity, understanding and maintaining software development has become a major challenge for developers. Understanding source code, in particular, often consumes a significant amount of time and effort. Source code summaries, as a natural language expression, play an indispensable role in software development. However, manually writing source code summaries is not only time-consuming and labor-intensive, but also often becomes outdated with the continuous iteration of software, resulting in reduced accuracy and practicality. Therefore, being able to automatically generate concise and easy-to-understand source code summaries has become a key task in improving software development efficiency.

[0003] In the research field of code summary generation, existing technologies can be mainly divided into three categories: keyword-based methods, retrieval-based methods, and deep learning-based methods. Keyword-based methods generate summaries by extracting key information from code snippets. Although these methods can handle simple code snippets, they may face challenges when dealing with complex code structures and identifiers with irregular names. Therefore, retrieval-based methods have gradually become mainstream. Retrieval-enhanced methods use code clone detection technology to retrieve code snippets similar to the input code snippet from the code base, and generate summaries for the code snippets based on their corresponding annotation information. By training on a large-scale training set of code-annotation pairs, deep learning models can learn the semantic mapping relationship between code snippets and their corresponding natural language summaries, and then generate high-quality code summaries. Existing research on code comment generation based on retrieval enhancement has made important progress, but it still faces the following challenges:

[0004] (1) Existing general content-based retrieval methods may mistakenly retrieve and merge code snippets that are syntactically similar but semantically different or only partially semantically similar. This may adversely affect the performance of model generation by introducing mismatched noise data.

[0005] (2) Existing methods make use of the retrieved code snippets in a coarse-grained manner, usually by direct concatenation or semantic expansion, treating the retrieved similar code as an extension of the source code. This method cannot effectively learn the deep relationship between similar code snippets and the source code, and ignores the information of existing high-quality manually written code comments.

[0006] (3) Existing code comment evaluation metrics mainly rely on machine translation evaluation metrics such as BLEU, METEOR, and ROUGE. These evaluation metrics mainly evaluate text generation quality based on n-gram matching. Their core logic is to calculate the word recurrence rate between the generated text and the reference text. They cannot effectively identify texts with different wording but the same semantics, and thus cannot provide a comprehensive evaluation of code comment quality. Summary of the Invention

[0007] The present invention provides a code comment generation method based on a large language model and multimodal contrastive learning to solve the problems existing in the above-mentioned existing code comment generation method based on retrieval enhancement.

[0008] According to the first aspect, an embodiment provides a method for generating code comments based on a large language model and multimodal contrastive learning, the method comprising:

[0009] Constructing a first prompt template, and generating similar code of the source code using a large language model based on the first prompt template;

[0010] Inputting the source code and similar code into a pre-built and trained code annotation generation model to generate code annotations, wherein the code annotation model includes an encoder and a decoder and is based on a multimodal contrastive loss learning strategy including inter-modality contrastive learning and intra-modality contrastive learning to achieve inter-modality alignment and intra-modality alignment of the source code, similar code, and code annotation modalities;

[0011] A second prompt template is constructed, and based on the second prompt template, a large language model is used to perform quality assessment on the generated code comments.

[0012] Furthermore, a first prompt template is constructed, specifically including:

[0013] The first prompt template is constructed by combining the thought chain prompt method and the few-sample prompt method, including using the thought chain idea to preliminarily guide the large language model to understand the source code, and using the few-sample prompt method to list sample cases to construct the prompt template.

[0014] Furthermore, the code comment generation model specifically includes:

[0015] The encoder includes n stacked Transformer encoding layers. The encoder is used to first convert the input sequence including source code, similar code and code comments into a word embedding vector of a specific dimension through a Token embedding layer, and then add position information to the three language tags through a position embedding layer. After processing by the Token embedding layer and the position embedding layer, the source code, similar code and code comments are respectively converted into embedding vector representations.

[0016] Furthermore, the code comment generation model specifically includes:

[0017] The decoder uses an autoregressive mechanism when generating code comments to predict the next possible word based on the content generated in the previous text. Each layer of the decoder performs additional cross-attention calculations on the last hidden layer of the encoder and is connected through cross-attention. After obtaining the decoder output, the feature representation is transformed through a linear layer and a softmax layer to generate a predicted probability distribution for the next word. To improve the quality of the generated results, the decoder uses beam search to generate the next output.

[0018] Furthermore, based on a multimodal contrastive loss learning strategy including inter-modality contrastive learning and intra-modality contrastive learning, we achieve inter-modality and intra-modality alignment of source code, similar code, and code comment modalities, specifically including:

[0019] Inter-modality contrastive learning includes the comparison between similar codes and code comments, and the comparison between source code and code comments; intra-modality contrastive learning includes the comparison between source code and similar codes.

[0020] Furthermore, based on a multimodal contrastive loss learning strategy including inter-modality contrastive learning and intra-modality contrastive learning, we achieve inter-modality and intra-modality alignment of source code, similar code, and code comment modalities, specifically including:

[0021] The inter-modality and intra-modality contrastive learning loss functions are constructed as follows:

[0022]

[0023] Where i and j represent the labels of the codes in the same batch, B represents the training batch size of the model, τ is the temperature hyperparameter, and C i It's the code, S i , S j is a similar code, N i , N j is a natural language annotation; and stands for inter-modality contrast loss, and the optimization goal of the function is to maximize the semantic similarity between the annotation and its paired code snippet, and minimize the semantic similarity between the annotation and its unpaired code snippet; The objective of the representative intra-modal loss function is to learn better representations of the codes, where similar codes have similar representations, while different codes have different representations.

[0024] The overall multimodal contrastive learning loss function is constructed as:

[0025] L total =L cn +L cs+L ca .

[0026] Furthermore, the training method of the code comment generation model includes:

[0027] Using a cross entropy loss function And the contrast loss function L total The joint loss function trains the model parameters:

[0028] L=L S +α·L total

[0029] Where α represents the multimodal contrast loss hyperparameter;

[0030] The definition is as follows:

[0031]

[0032] Among them, i represents the position of different words in the sentence, x represents the word in the source annotation, and y represents the current predicted word. Represents the probability of the next word.

[0033] Furthermore, a second prompt template is constructed, and based on the second prompt template, a large language model is used to perform a quality assessment on the generated code comments, specifically including:

[0034] For each generated code comment, a large language model is used to construct a hint template to score each code comment. The scoring is based on four aspects: completeness, clarity, comprehensibility, and maintainability.

[0035] Furthermore, the large language model adopts the Spark 3.5 large language model.

[0036] According to a second aspect, an embodiment provides a code comment generation system based on a large language model and multimodal contrastive learning, the system comprising:

[0037] A knowledge enhancement module, configured to construct a first prompt template, and generate similar code of the source code based on the first prompt template using a large language model;

[0038] a code annotation generation module, configured to input the source code and similar code into a pre-built and trained code annotation generation model to generate code annotations, wherein the code annotation model includes an encoder and a decoder and is based on a multimodal contrastive loss learning strategy including inter-modality contrastive learning and intra-modality contrastive learning to achieve inter-modality alignment and intra-modality alignment of source code, similar code, and code annotation modalities;

[0039] An evaluation module is used to construct a second prompt template and perform quality evaluation on the generated code comments based on the second prompt template using a large language model.

[0040] The present invention provides a code comment generation method based on a large language model and multimodal contrastive learning, which has the following beneficial effects:

[0041] (1) A code knowledge enhancement module based on a large language model is proposed, aiming to generate diversified and high-quality code generation methods. By using prompt learning to assist the large language model in generating similar codes with diverse styles and high quality, the quality of code comment generation is improved.

[0042] (2) A multimodal contrastive learning framework was designed to align the multimodal information in code comments; intra-modal and inter-modal contrastive learning was designed to capture the association between code and code and between code and comments, enhancing the model's ability to understand and match the semantic relationship between code and comments.

[0043] (3) A large language model-based annotation evaluation metric is proposed to evaluate the quality of generated annotations from four aspects: relevance, completeness, clarity, and comprehensibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flowchart of a method for generating code comments based on a large language model and multimodal contrastive learning, provided in accordance with one embodiment of the present invention;

[0045] Figure 2 A network architecture diagram of a code comment generation method based on a large language model and multimodal contrastive learning provided by one embodiment of the present invention;

[0046] Figure 3 A schematic diagram of a first prompt template in a code comment generation method based on a large language model and multimodal contrastive learning provided by one embodiment of the present invention;

[0047] Figure 4 A schematic diagram of a second prompt template in a code comment generation method based on a large language model and multimodal contrastive learning provided by one embodiment of the present invention;

[0048] Figure 5 A schematic diagram of the logical structure of a code comment generation system based on a large language model and multimodal contrastive learning, provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0049] The present invention will be further described in detail below by means of specific embodiments in conjunction with the accompanying drawings. Similar elements in different embodiments are numbered with associated similar elements. In the following embodiments, many detailed descriptions are provided to enable the present invention to be better understood. However, those skilled in the art will readily appreciate that some of the features may be omitted under different circumstances, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the present invention are not shown or described in the specification. This is to avoid the core of the present invention being overwhelmed by excessive descriptions, and for those skilled in the art, it is not necessary to describe these related operations in detail. They can fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0050] In addition, the features, operations, or characteristics described in the specification may be combined in any appropriate manner to form various embodiments. Furthermore, the steps or actions in the method description may be reordered or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various sequences in the specification and drawings are provided solely for the purpose of clearly describing a particular embodiment and are not intended to be mandatory, unless otherwise specified.

[0051] like Figure 1 As shown, the first embodiment of the present invention provides a code comment generation method based on a large language model and multimodal contrastive learning. Figure 1 and Figure 2 Provide detailed explanation.

[0052] The code generation method of this embodiment mainly includes two parts: a knowledge enhancement part and a code comment generation part. In the knowledge enhancement part, given a section of target code, a general prompt template is first designed by combining the thought chain and the few-sample prompt method to assist the large language model in generating similar code with similar functions to the source target code but different implementation methods. In the code comment generation part, a multimodal comparative learning strategy is first designed to align source code, similar code, and multiple modalities of comments. In order to fully learn the relationship between source code, similar code, and comments, this embodiment designs two methods: inter-modal comparative learning and intra-modal comparative learning, which are used to enhance the semantic consistency within the same modality and the semantic alignment ability between different modalities. Subsequently, a classic encoder-decoder model architecture is designed to generate natural language comments for the code. In addition, in order to more accurately evaluate the quality of the comments generated by this method, a code comment evaluation method based on a large language model is introduced. For natural language comments, the quality of the generated comments is comprehensively evaluated from four aspects: completeness, clarity, comprehensibility, and maintainability.

[0053] like Figure 1As shown, in step S100, a first prompt template is constructed, and similar code of the source code is generated based on the first prompt template using a large language model.

[0054] The above steps specifically include:

[0055] In this embodiment, the first prompt template is constructed by combining the thought chain prompt method and the few-sample prompt method, including using the thought chain idea to preliminarily guide the large language model to understand the source code, and using the few-sample prompt method to list sample cases to construct the prompt template.

[0056] This section of this embodiment uses the prior knowledge of a large language model to enhance the external knowledge of the source code. In this embodiment, the large language model uses the Spark 3.5 large language model.

[0057] The quality of the prompt template can affect the performance of the large language model on downstream specific tasks. Therefore, a high-quality prompt template is very important. As the connection between the large language model and the downstream specific tasks, the prompt template can often determine the data quality of the model output. In order to fully improve the quality and diversity of similar codes generated by the large language model, this embodiment designs a prompt template that combines thought chain and few-sample prompts to guide the large model to generate similar codes with clear structure and strong context consistency. The prompt template is designed as follows: Figure 3 shown.

[0058] A few-sample prompt is a template that uses a small number of specific sample cases. Considering that a simple description of the task purpose may not allow the large language model to fully understand the task requirements, some specific input and output cases are added to help the large language model understand the task. A thought chain prompt is a template that draws on the human problem-solving process. When faced with difficult and complex problems, humans often break down large problems into small problems and think about them step by step to get the final correct answer. By allowing the large language model to divide the task goals into multiple small problems, it helps understand the task goals and ultimately obtain the correct output. The prompt template designed in this embodiment starts from the role and goal. In order to ensure that the large language model fully understands the functional meaning of the code, the thought chain idea is used to preliminarily guide the model to understand the source code, and the few-sample prompt method is used to list sample cases to construct the prompt template.

[0059] like Figure 1 As shown, in step S200, the source code and similar code are input into a pre-built and trained code annotation generation model to generate code annotations. The code annotation model includes an encoder and a decoder, and is based on a multimodal contrast loss learning strategy including inter-modality contrast learning and intra-modality contrast learning to achieve inter-modality alignment and intra-modality alignment of source code, similar code and code annotation modalities.

[0060] The above steps specifically include:

[0061] In this embodiment, the encoder includes n stacked Transformer encoding layers, which are used to first convert the input sequence including source code, similar code and code comments into a word embedding vector of a specific dimension through a Token embedding layer, and then add position information to the three language tags through a position embedding layer. After processing by the Token embedding layer and the position embedding layer, the source code, similar code and code comments are respectively converted into embedding vector representations.

[0062] Specifically, the encoder consists of n stacked Transformer encoding layers. The purpose of the encoder is to capture the semantic associations between source code, similar code, and comments in the input sequence, providing high-quality representations for multimodal contrastive learning tasks. Specifically, given source code, similar code, and comment information, the input sequence first passes through a Token Embedding layer to convert the three language tokens into word embedding vectors of a specific dimension. At the same time, to capture the order between tokens in the input sequence, a Position Embedding Layer is introduced to add position information to each token, enhancing the model's ability to perceive the order and relative position of tokens in the sequence. After processing by the Token Embedding Layer and the Position Embedding Layer, the input data is converted into a vector space of a specific dimension.

[0063] The goal of multimodal contrastive learning is to optimize the alignment of the model between modalities by constructing similarities and dissimilarities between source code, similar code, and annotations. Multimodal contrastive learning consists of two parts: inter-modal contrast and intra-modal contrast. In this embodiment, inter-modal contrastive learning includes the comparison of similar code and code annotations, and the comparison of source code and code annotations. The goal is to maximize the semantic similarity between the code modality and its paired annotation modality, and minimize the semantic similarity between the code modality and its unpaired annotation modality, so as to learn the connection between code and annotations and make full use of the potential of high-quality manually written annotations; intra-modal contrast includes the comparison of source code and similar code. The goal is to learn the connection between source code and similar code, so that similar codes have similar representations, and the representations of different code segments can be effectively distinguished, thereby enhancing the model's ability to understand the code.

[0064] The inter-modality and intra-modality contrastive learning loss functions are constructed as follows:

[0065]

[0066] Where i and j represent the labels of the codes in the same batch, B represents the training batch size of the model, τ is the temperature hyperparameter, and C i It's the code, S i , S j is a similar code, N i , N j is a natural language annotation; and stands for inter-modality contrastive loss, and the optimization objective of the function is to maximize the semantic similarity between the annotation and its paired code snippet, and minimize the semantic similarity between the annotation and its unpaired code snippet. The objective of the representative intra-modal loss function is to learn better representations of the codes, where similar codes have similar representations, while different codes have different representations.

[0067] The overall multimodal contrastive learning loss function is constructed as:

[0068] L total =L cn +L cs +L ca .

[0069] In this embodiment, the decoder module and the encoder module use the same model structure, both built on the Transformer architecture. Unlike the encoder, the decoder uses an autoregressive mechanism when generating annotations, predicting the next possible word based on the previously generated content. Specifically, at the decoder stage, to ensure that the model is unaware of the current input and word information at future times, the decoder module uses a masking mechanism to block the influence of the current word and future moments, while ensuring the autoregressive characteristics.

[0070] In addition, each decoder layer performs additional cross-attention calculations on the last hidden layer of the encoder and connects through cross-attention. Each decoder layer performs an attention operation on the final hidden state of the encoder output, thereby incorporating the contextual information captured by the encoder into the decoder generation process, making the generated comments closer to the logical and functional semantics of the source code.

[0071] After obtaining the decoder output, the feature representation is transformed through a linear layer and a softmax layer to generate the predicted probability distribution of the next word. In addition, to improve the quality of the generated results, the decoder uses beam search to generate the next output:

[0072] P(y i+1 |y1,…,y i )=softmax(h i W+b)

[0073] Among them, h i represents the hidden state of the model at time step i, W represents the weight matrix, b represents the bias vector, and y i Represents the i-th word of the generated comment.

[0074] In this embodiment, the cross entropy loss function is used And the contrast loss function Ltotal The joint loss function trains the model parameters:

[0075] L=L S +α·L total

[0076] Here, α represents the multimodal contrastive loss hyperparameter.

[0077] The definition is as follows:

[0078]

[0079] Among them, i represents the position of different words in the sentence, x represents the word in the source annotation, and y represents the current predicted word. Represents the probability of the next word.

[0080] like Figure 1 As shown, in step S300, a second prompt template is constructed, and the quality of the generated code comments is evaluated using a large language model based on the second prompt template.

[0081] The above steps specifically include:

[0082] In this embodiment, for each generated code comment, a hint template is constructed and a large language model is used to score each code comment. The scoring is based on four aspects: completeness, clarity, comprehensibility, and maintainability.

[0083] The evaluation indicators of existing code comment generation tasks usually use machine translation evaluation indicators such as BLEU, ROUGE, and METEOR. Although such indicators can calculate the correlation between the generated comments and the reference comments through word recurrence rate, they only focus on the matching at the vocabulary level through word recurrence rate, ignoring the situation where different words may express the same semantics. At the same time, it is very sensitive to word order. It is impossible to effectively evaluate the quality of generated comments by only using evaluation indicators such as BLEU. Based on the defects of the above-mentioned evaluation indicators, this embodiment proposes an annotation evaluation method based on a large language model. Using a large language model to understand the specific expression meaning of the generated comments and the original comments solves the problem that traditional indicators only focus on vocabulary-level matching and are sensitive to word order.

[0084] Specifically, for each generated comment, a prompt template is constructed and the large language model is required to score each comment. The scoring is based on four aspects: completeness, clarity, comprehensibility, and maintainability, with scores ranging from 1 to 5. The higher the score, the higher the quality of the comment. Completeness is intended to measure whether the generated comment can fully express the function of the function, clarity is intended to measure whether the generated comment is clear and concise, comprehensibility is intended to measure whether the generated comment is easy to understand, and maintainability is intended to measure the possibility of the generated comment continuing to be maintained and used in the future. The specific prompt template is as follows: Figure 4 shown.

[0085] To verify the performance of the method in this embodiment, we tested it on two mainstream high-level language datasets, JSCD and PCSD. The JSCD dataset was split into training, validation, and test sets with a ratio of 8:1:1. The PCSD dataset was split into training, validation, and test sets with a ratio of 6:2:2. The model input length was fixed, with the maximum length of code sequences set to 256 and the maximum length of comments set to 128.

[0086] In order to verify the effectiveness of the method model proposed in this example (KASG for short) in the code comment generation task, it is compared with multiple baseline models, including Transformer, CodeBERT, UniXcoder, UniXcoder (Self-Improved), etc. The model performance is evaluated using multiple indicators such as BLEU, ROUGE-L, and METEOR to fully demonstrate the performance of the KASG model. The experimental results are shown in Table 1. It can be seen that the KASG model performs significantly better than the baseline model on both the JSCD and PCSD datasets.

[0087] BLEU stands for Bilingual Evaluation Underscore. It focuses on measuring the similarity between the output and the reference, emphasizing sentence accuracy and exact matching. BLEU evaluates the accuracy of text by calculating the matching degree of N-grams:

[0088]

[0089] Among them, P n is the n-gram matching accuracy of subsequences of length n. n = 1 / N, where N is capped at 4, meaning only 4-gram accuracy is counted. BP is a brevity penalty factor that penalizes sentences that are too short to prevent the training results from favoring short sentences. It is defined as follows:

[0090]

[0091] Where c is the length of the candidate sentence and r is the reference length. Since high-order n-grams may not overlap, the NIST smoothed BLEU-4 is evaluated. Smoothed BLEU-4 is denoted as BLEU-4(s).

[0092] ROUGE-L is a recall-based similarity metric that primarily examines the adequacy and authenticity of reference translations. It uses the longest shared clause co-occurrence statistics (precision P) and recall F. Its calculation is almost identical to BLEU, but n-gram phrases are generated from reference translations.

[0093]

[0094] Among them, R represents the reference annotation, that is, the code annotation; C represents the candidate annotation, that is, the code annotation generated by the model. c Indicates the length of C, l r Represents the length of R. LCS(R,C) represents the longest common subsequence of R and C, β is a hyperparameter, β is set to 1.

[0095] METEOR uses WordNet to expand the synonym set, while taking into account the word part of speech (for example, like and likes should both be counted correctly); in terms of calculation method, it combines precision and recall, and uses the harmonic mean of the two as the evaluation criterion.

[0096] METEOR=(1-P en )F mean

[0097]

[0098] Where m is the number of 1-gram matches and ch is the number of chunks. The METEOR metric parameters are set as follows: α = 0.85, β = 0.2, and γ = 0.6. BLEU and METEOR are implemented using the NLTK library.

[0099] Table 1 Experimental results

[0100]

[0101]

[0102] In order to further explore the influence of hyperparameters on the model, this embodiment also analyzes the influence of different values ​​​​on the loss function parameters of the model. Let the hyperparameter α be {0.1, 0.3, 0.5, 0.7, 0.9} respectively, and the remaining parameter settings are consistent with the original model. According to the results, on the JSCD dataset, affected by the hyperparameter α, the performance of the model shows a trend of first decreasing, then increasing, and then decreasing. When the hyperparameter α is 0.7, the model performance is the best. On the PCSD dataset, the change pattern of the hyperparameter α and the model performance is the same as above, both showing a trend of first decreasing and then increasing. When the hyperparameter α is 0.1, the model performance is the best.

[0103] This embodiment proposes a code comment generation method based on a large language model and multimodal contrastive learning. First, similar code of the source code is generated by a large language model enhancement method. Specifically, a general prompt template is designed to help the large language model generate similar code segments. The prompt template introduces the idea of ​​gradual generation of thought chains, which effectively improves the accuracy of the generation results. At the same time, it also ensures that the generated similar code segments and the implementation methods of the source code are varied and of higher quality. Then, through multimodal contrastive learning, the source code, similar code and its comments are aligned. By designing intra-modal contrastive learning and inter-modal contrastive learning, the model's understanding and matching capabilities of the semantic relationship between code, similar code and comments are enhanced, thereby better capturing the deep-level associations between similar code, comments and source code. Finally, in response to the limitations of existing evaluation indicators, an evaluation method based on a large language model is designed. With the help of the understanding ability of the large language model, the quality of the generated comments is more comprehensively evaluated from four aspects: relevance, completeness, clarity and comprehensibility.

[0104] Corresponding to the above-disclosed method for generating code comments based on a large language model and multimodal contrastive learning, the embodiment of the present invention further discloses a system for generating code comments based on a large language model and multimodal contrastive learning, such as Figure 5 As shown, it specifically includes:

[0105] A knowledge enhancement module, configured to construct a first prompt template, and generate similar code of the source code based on the first prompt template using a large language model;

[0106] a code annotation generation module, configured to input the source code and similar code into a pre-built and trained code annotation generation model to generate code annotations, wherein the code annotation model includes an encoder and a decoder and is based on a multimodal contrastive loss learning strategy including inter-modality contrastive learning and intra-modality contrastive learning to achieve inter-modality alignment and intra-modality alignment of source code, similar code, and code annotation modalities;

[0107] An evaluation module is used to construct a second prompt template and perform quality evaluation on the generated code comments based on the second prompt template using a large language model.

[0108] It should be noted that for a detailed description of a code annotation generation system based on a large language model and multimodal contrastive learning provided in an embodiment of the present invention, reference can be made to the relevant description of a code annotation generation method based on a large language model and multimodal contrastive learning provided in an embodiment of the present invention, which will not be repeated here.

[0109] In addition, an embodiment of the present invention also provides an electronic device, comprising: a processor and a memory; the memory is used to store one or more program instructions; the processor is used to run one or more program instructions to execute the steps of a code comment generation method based on a large language model and multimodal contrastive learning as described in any of the above items.

[0110] It should be noted that for a detailed description of an electronic device provided in an embodiment of the present invention, reference can be made to the relevant description of a code annotation generation method based on a large language model and multimodal contrastive learning provided in an embodiment of the present application, which will not be repeated here.

[0111] In addition, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a code annotation generation method based on a large language model and multimodal contrastive learning as described in any of the above items are implemented.

[0112] It should be noted that for a detailed description of a computer-readable storage medium provided in an embodiment of the present invention, reference can be made to the relevant description of a code annotation generation method based on a large language model and multimodal contrastive learning provided in an embodiment of the present application, which will not be repeated here.

[0113] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer program. When all or part of the functions in the above embodiments are implemented by computer program, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above functions can be implemented. In addition, when all or part of the functions in the above embodiments are implemented by computer program, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and saved in the memory of the local device by downloading or copying, or the system of the local device is updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be implemented.

[0114] The above examples are used to illustrate the present invention, which are only used to help understand the present invention and are not intended to limit the present invention. Those skilled in the art can make several simple deductions, modifications or substitutions based on the concept of the present invention.

Claims

1. A code comment generation method based on a large language model and multimodal contrastive learning, characterized in that: The method comprises: Constructing a first prompt template, and generating similar code of the source code using a large language model based on the first prompt template; Inputting the source code and similar code into a pre-built and trained code annotation generation model to generate code annotations, wherein the code annotation model includes an encoder and a decoder and is based on a multimodal contrastive loss learning strategy including inter-modality contrastive learning and intra-modality contrastive learning to achieve inter-modality alignment and intra-modality alignment of the source code, similar code, and code annotation modalities; A second prompt template is constructed, and based on the second prompt template, a large language model is used to perform quality assessment on the generated code comments.

2. The method for generating code comments based on a large language model and multimodal contrastive learning according to claim 1, wherein: Construct the first prompt template, including: The first prompt template is constructed by combining the thought chain prompt method and the few-sample prompt method, including using the thought chain idea to preliminarily guide the large language model to understand the source code, and using the few-sample prompt method to list sample cases to construct the prompt template.

3. The method for generating code comments based on a large language model and multimodal contrastive learning according to claim 1, wherein: The code comment generation model specifically includes: The encoder includes n stacked Transformer encoding layers. The encoder is used to first convert the input sequence including source code, similar code and code comments into a word embedding vector of a specific dimension through a Token embedding layer, and then add position information to the three language tags through a position embedding layer. After processing by the Token embedding layer and the position embedding layer, the source code, similar code and code comments are respectively converted into embedded vector representations.

4. The method for generating code comments based on a large language model and multimodal contrastive learning according to claim 1, wherein: The code comment generation model specifically includes: The decoder uses an autoregressive mechanism when generating code comments to predict the next possible word based on the content generated in the previous text. Each layer of the decoder performs additional cross-attention calculations on the last hidden layer of the encoder and is connected through cross-attention. After obtaining the decoder output, the feature representation is transformed through a linear layer and a softmax layer to generate a predicted probability distribution for the next word. To improve the quality of the generated results, the decoder uses beam search to generate the next output.

5. The method for generating code comments based on a large language model and multimodal contrastive learning according to claim 1, wherein: Based on a multimodal contrastive loss learning strategy that includes inter-modality contrastive learning and intra-modality contrastive learning, it is used to achieve inter-modality and intra-modality alignment of source code, similar code, and code comment modalities. Specifically, it includes: Inter-modality contrastive learning includes the comparison between similar codes and code comments, and the comparison between source code and code comments; intra-modality contrastive learning includes the comparison between source code and similar codes.

6. The method for generating code comments based on a large language model and multimodal contrastive learning according to claim 5, wherein: Based on a multimodal contrastive loss learning strategy that includes inter-modality contrastive learning and intra-modality contrastive learning, it is used to achieve inter-modality and intra-modality alignment of source code, similar code, and code comment modalities. Specifically, it includes: The inter-modality and intra-modality contrastive learning loss functions are constructed as follows: Where i and j represent the labels of the codes in the same batch, B represents the training batch size of the model, τ is the temperature hyperparameter, and C i It's the code, S i , S j is a similar code, N i , N j is a natural language annotation; and stands for inter-modality contrast loss, and the optimization goal of the function is to maximize the semantic similarity between the annotation and its paired code snippet, and minimize the semantic similarity between the annotation and its unpaired code snippet; The objective of the representative intra-modal loss function is to learn better representations of the codes, where similar codes have similar representations, while different codes have different representations. The overall multimodal contrastive learning loss function is constructed as: L total =L cn +L cs +L sn 7. The method for generating code comments based on a large language model and multimodal contrastive learning according to claim 6, wherein: The training method of the code comment generation model includes: Using a cross entropy loss function And the contrast loss function L total The joint loss function trains the model parameters: L=L S +α·L total Where α represents the multimodal contrast loss hyperparameter; The definition is as follows: Among them, i represents the position of different words in the sentence, x represents the word in the source annotation, and y represents the current predicted word. Represents the probability of the next word.

8. The method for generating code comments based on a large language model and multimodal contrastive learning according to claim 1, wherein: Construct a second prompt template, and use a large language model to perform quality assessment on the generated code comments based on the second prompt template, specifically including: For each generated code comment, a large language model is used to construct a hint template to score each code comment. The scoring is based on four aspects: completeness, clarity, comprehensibility, and maintainability.

9. The method for generating code comments based on a large language model and multimodal contrastive learning according to claim 1, wherein: The large language model adopts the Spark 3.5 large language model.

10. A code comment generation system based on a large language model and multimodal contrastive learning, characterized in that: The system comprises: A knowledge enhancement module, configured to construct a first prompt template, and generate similar code of the source code based on the first prompt template using a large language model; a code annotation generation module, configured to input the source code and similar code into a pre-built and trained code annotation generation model to generate code annotations, wherein the code annotation model includes an encoder and a decoder and is based on a multimodal contrastive loss learning strategy including inter-modality contrastive learning and intra-modality contrastive learning to achieve inter-modality alignment and intra-modality alignment of source code, similar code, and code annotation modalities; An evaluation module is used to construct a second prompt template and perform quality evaluation on the generated code comments based on the second prompt template using a large language model.

Images