Method and system for automatic generation of source code comments based on word-level retrieval
By using a word-level retrieval method, high-quality code annotations are generated using an encoder-decoder network and a nearest neighbor database. This solves the problem of insufficient utilization of semantic information in existing technologies and improves the quality and interpretability of annotation generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2023-05-16
- Publication Date
- 2026-07-21
AI Technical Summary
Existing code annotation generation methods based on sentence-level retrieval fail to effectively utilize the semantic information of the code, resulting in low-quality generated annotations and the introduction of a large amount of noise.
We employ a word-level retrieval method, extracting semantic and structural representation vectors of code function text and abstract syntax trees through an encoder-decoder network. Combined with a nearest neighbor word database, we generate high-quality code annotations through autoregression and use multi-head self-attention and cross-attention modules to fuse word-level probability distributions.
It improved the quality of code annotation generation, reduced noise, enhanced the interpretability of the model, and increased the generation probability of low-frequency words.
Smart Images
Figure CN116627487B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of natural language processing text generation, and specifically to a method and system for automatically generating source code comments based on word-level retrieval. Background Technology
[0002] As large-scale software development becomes increasingly complex and its systems increasingly massive, the barrier to entry for software developers to quickly understand and maintain software is rising. In this context, code comments, as indispensable textual information in the large-scale software development and maintenance process, can greatly alleviate the development and maintenance burdens on software developers. Due to the complexity of code, generating high-quality code text comments is a challenging yet meaningful task. In conventional code comment generation methods, code is treated as a form of human language text, utilizing common natural language processing methods and an encoder-decoder structure to automate the generation of code comments. The encoder primarily understands the code, while the decoder decodes the understood code semantics to generate textual comments.
[0003] In conventional encoder-decoder-based generative frameworks, researchers design various semantic extraction methods based on the semantic understanding of the code itself. Furthermore, besides enhancing the model's semantic understanding to generate better code annotations, researchers have also considered how to utilize external information to assist the model in generating code annotations. The main process is as follows: First, text similarity metrics are used to retrieve which codes in the code repository are similar to the target code for which annotations are to be generated. The most similar codes and their paired annotations are extracted. An additional encoder encodes the similar codes and / or their paired annotation texts to obtain similar code representations. These similar code representations are then aggregated with the target code's representation. The aggregated representations are then fed into the decoder for decoding to generate the target code's annotations. In this sentence-level retrieval-based code annotation generation process, the model utilizes the entire retrieved code text and annotation sentences, directly integrating them into the decoder's target word generation process.
[0004] However, the above sentence-level retrieval method has at least the following drawbacks: (1) It uses traditional text similarity distance or edit distance to retrieve the most similar code, only considering the text similarity between the two code segments, without considering the semantic information of the code at all; (2) It directly uses an encoder to extract the semantics of the retrieved code text and its annotation sentences and concatenates them with the semantic feature vector of the target code, introducing a lot of irrelevant or even negative semantic representation information. Due to the existence of the above two limitations, the retrieved code may not be similar to the target code; in addition, the method of directly encoding and concatenating similar code and its annotation text will introduce a lot of irrelevant word representation information to a large extent, thereby bringing a lot of noise and greatly affecting the final annotation generation quality of the model.
[0005] Generating high-quality code comments is a crucial and highly productive task in software engineering. Recently, researchers have attempted to use external knowledge to assist in model generation with some success. However, how to utilize external knowledge in a fine-grained manner is key to the final quality of the generated model and requires further exploration. Summary of the Invention
[0006] To address the shortcomings of existing sentence-level retrieval methods, this invention provides a method and system for automatically generating source code comments based on word-level retrieval.
[0007] The objective of this invention is achieved through the following technical solution:
[0008] Firstly, this invention proposes an automated method for generating source code comments based on word-level retrieval, comprising:
[0009] (1) Obtain a training dataset consisting of code function text, code abstract syntax tree, and code comment text;
[0010] (2) Train the encoder-decoder network using the training dataset. The encoder in the encoder-decoder network is used to extract the semantic representation vector of the code function text and the structural representation vector of the code abstract syntax tree. The decoder in the encoder-decoder network is used to interact with the attention results, semantic representation vector and structural representation vector of the code annotation text and decode to generate annotation text.
[0011] (3) Using the trained encoder-decoder network, obtain the overall representation vector of each annotation word in the code annotation text and construct a nearest neighbor word database;
[0012] (4) For the code function text to be annotated and its abstract syntax tree, the trained encoder-decoder network is used to generate the model-based target word probability distribution and target word overall representation vector at the current time step in an autoregressive manner;
[0013] (5) Retrieve the K nearest neighbor words with the highest cosine similarity to the overall representation vector of the target word from the nearest neighbor word database, and generate the target word probability distribution based on the nearest neighbor words;
[0014] (6) The target word probability distribution based on the model and the target word probability distribution based on the nearest neighbor word are fused, and the target word with the highest probability in the fused probability distribution is taken as the annotation word generated at the current time step;
[0015] (7) Repeat steps (4)-(6) until the comment ends.
[0016] Furthermore, the decoder includes a masked multi-head self-attention module, a semantic-based multi-head cross-attention module, a structure-based multi-head cross-attention module, and a feedforward neural network;
[0017] First, a masked multi-head self-attention module is used to obtain the attention results of the input code comment text. Then, it interacts with a semantic-based multi-head cross-attention module and a structure-based multi-head cross-attention module respectively. Finally, a feedforward neural network generates a model-based target word probability distribution.
[0018] Furthermore, step (3) includes:
[0019] (3-1) Encode the code function text to obtain a semantic representation vector, and encode the code abstract syntax tree to obtain a structural representation vector;
[0020] (3-2) Obtain the word representation vector of each comment word in the code comment text, and obtain the attention score of each word in the code function text by taking the dot product of the word representation vector and the semantic representation vector of each comment word in the code comment text; obtain the attention score of each node in the code structure by taking the dot product of the word representation vector and the structural representation vector of each comment word in the code comment text.
[0021] (3-3) The attention score of each word in the code function text is weighted and summed with the semantic representation vector to obtain the overall semantic representation vector of the comment words in the code comment text for the code function text; the attention score of each node in the code structure is weighted and summed with the structure representation vector to obtain the overall structure representation vector of the comment words in the code comment text for the code abstract syntax tree.
[0022] (3-4) Concatenate the word representation vector of each annotation word with the overall semantic representation vector and the overall structural representation vector. After regularizing the concatenation result, obtain the overall representation vector of the annotation word. Put the overall representation vector of the annotation word and the corresponding annotation word as key-value pairs into the nearest neighbor word database.
[0023] Furthermore, the method for obtaining the word representation vector of each comment word in the code comment text is as follows:
[0024] The code comment text is used as input to the masked multi-head self-attention module in the decoder, and the output is the attention result of the comment text. The attention result of the comment text and the semantic representation vector are used as input to the semantic-based multi-head cross-attention module in the decoder, and the output is the semantic-based attention result. The semantic-based attention result and the structural representation vector are used as input to the structural-based multi-head cross-attention module in the decoder, and the output is the structural-based attention result. After regularization, the word representation result is obtained. Each vector in the word representation result corresponds to the word representation vector of each comment word in the code comment text.
[0025] Furthermore, in the semantic-based multi-head cross-attention module, the attention result of the annotated text is used as the query vector in the attention mechanism, and the semantic representation vector is used as the key vector and value vector in the attention mechanism; in the structure-based multi-head cross-attention module, the semantic-based attention result is used as the query vector in the attention mechanism, and the structure representation vector is used as the key vector and value vector in the attention mechanism.
[0026] Furthermore, in step (4), when obtaining the overall representation vector of the target word, the annotation words S generated before the current time step t are included. ＜t As input to the masked multi-head self-attention module in the decoder, the output annotation word S ＜t The attention results are then processed by a semantic-based multi-head cross-attention module and a structure-based multi-head cross-attention module to generate word representation results. The last vector in the word representation results is used as the current word representation vector. The current word representation vector is concatenated with the overall semantic representation vector and the overall structure representation vector. The concatenation result is then regularized to obtain the overall representation vector of the target word.
[0027] Furthermore, the target word probability distribution based on the model and the target word probability distribution based on the nearest neighbor words are weighted to obtain the fused probability distribution.
[0028] Secondly, this invention discloses an automated source code annotation generation system based on word-level retrieval, comprising:
[0029] The data acquisition module is used to acquire the training dataset consisting of code function text, code abstract syntax tree, and code comment text, as well as to acquire the code function text to be annotated and its abstract syntax tree;
[0030] The encoder-decoder network module is used to extract the semantic representation vector of the code function text and the structural representation vector of the code abstract syntax tree, and to interact with the attention results, semantic representation vector and structural representation vector of the code comment text and decode to generate comment text;
[0031] The encoder-decoder network training module is used to train the encoder-decoder network module using the training dataset.
[0032] The nearest neighbor word database construction module is used to obtain the overall representation vector of each annotation word in the code annotation text using the trained encoder-decoder network, and to construct the nearest neighbor word database.
[0033] The initial target word probability distribution calculation module is used to generate the model-based target word probability distribution and the overall target word representation vector at the current time step using a trained encoder-decoder network in an autoregressive manner for the code function text to be annotated and its abstract syntax tree.
[0034] The nearest neighbor word search module is used to retrieve the K nearest neighbor words with the highest cosine similarity to the overall representation vector of the target word from the nearest neighbor word database, and generate the target word probability distribution based on the nearest neighbor words;
[0035] The probability distribution fusion module is used to fuse the target word probability distribution based on the model and the target word probability distribution based on the nearest neighbor word, and take the target word with the highest probability in the fused probability distribution as the annotation word generated at the current time step.
[0036] The beneficial effects of this invention are:
[0037] This invention utilizes the representation vectors of the encoder of a natural language model, the encoder of a graph neural network, and the decoder of a natural language processing system to represent the semantic information of each target word, constructing a large-scale (word representation - target word) database. During each time step of the model decoder's target word generation process, the retrieved nearest neighbor words are integrated. This intelligently uses the retrieved nearest neighbor words to correct the original word distribution probability of the model, significantly improving the annotation generation quality of the original model and also increasing the generation probability of low-frequency words in code annotations. This invention is the first to propose a representation retrieval mechanism at the word level with fine granularity, resulting in a model with strong interpretability. Attached Figure Description
[0038] Figure 1 This is a block diagram of the overall structure of the source code annotation automatic generation method based on word-level retrieval of the present invention.
[0039] Figure 2 This is a schematic diagram of the nearest neighbor database of the method of the present invention. Detailed Implementation
[0040] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0041] This invention provides an automated source code annotation generation method based on word-level retrieval, comprising the following steps:
[0042] (1) Constructing a code-annotation dataset: Using code from common programming languages (Python, Java, C), we construct a code abstract syntax tree (AST) using the open-source tree-sitter tool. Each data sample includes: code function text, code abstract syntax tree, and code annotation text.
[0043] (2) Model building and training: For code function text, a code function text encoder Transformer_encoder is built to represent the positional relationship between different words in the code using the relative position representation method; for code abstract syntax tree, a code structure encoder GAT_encoder is built to represent the structural information of the code abstract syntax tree; for code comment text, a code comment text decoder Transformer_decoder is built to generate code comment text.
[0044] In this embodiment, the overall architecture based on the encoder and decoder is as follows: Figure 1 As shown, the code function text encoder consists of a graph self-attention network (GNN, such as GAT), an activation function (such as ReLU), and a residual structure, used to obtain the semantic representation vector of the code function text; the code structure encoder consists of a multi-head attention module and a feedforward neural network, used to obtain the structural representation vector of the code abstract syntax tree. The two encoders can also employ other network structures available in the art.
[0045] The decoder consists of a masked multi-head self-attention module, a semantic-based multi-head cross-attention module, a structure-based multi-head cross-attention module, and a feedforward neural network. First, the masked multi-head self-attention module acquires the attention result of the input code comment text. Then, the semantic-based and structure-based multi-head cross-attention modules respectively perform feature interaction. In terms of interaction order, the interaction occurs first under the attention mechanism of the semantic representation vector of the code function text, followed by the interaction under the attention mechanism of the structural representation vector of the code abstract syntax tree.
[0046] The two encoders and one decoder described above are trained and optimized simultaneously using teacher-lorcing and minimizing cross-entropy loss. Once training is complete, all model parameters are fixed and will not be optimized further in subsequent operations. The entire model training only needs to be performed once.
[0047] (3) Construction of the nearest neighbor word database: Using the trained model described above, a nearest neighbor word database is constructed using the training dataset. All samples from the training set are sequentially input into the model. Assume the code consists of p words, i.e., the code function text representation is t1, t2, ..., t... p In the code abstract syntax tree, there are q nodes in total, that is, the code abstract syntax tree is represented as n1. ， n2 ， …,n q In the code comment text, assume that the code comment text consists of m comment words, that is, the code function text is represented as z1, z2, ..., z m .
[0048] (3-1) For code function text, use a code function text encoder to encode it. Each word t in the code function text is encoded. i All will be encoded into semantic representation h i That is, the semantic representation vector of the code function text is represented as h1, h2, ..., h p .;
[0049] For the code abstract syntax tree, a code structure encoder is used for encoding. Each node n in the code abstract syntax tree... i All will be encoded as structural representation r i That is, the structural representation vector of the code abstract syntax tree is r1, r2, ..., r q .
[0050] The above encoding process can be represented as:
[0051] h1, h2, ..., h p =Transformer_encoder(t1,t2,…,t p )
[0052] r1, r2, ..., r q =GAT_encoder(n1,n2,…,n q )
[0053] (3-2) For code comment text, z1, z2, ..., z m As input to the masked multi-head self-attention module in the decoder, the output is the attention result Z of the annotated text; the attention result Z of the annotated text and the semantic representation vectors h1, h2, ..., h2 are then used as input. p As input to the semantic-based multi-head cross-attention module in the decoder, Z is used as the query vector in the attention mechanism, h1, h2, ..., h p As key and value vectors in the attention mechanism, the output is a semantic-based attention result. semantic-based attention results and structural representation vectors r1, r2, ..., r q As input to the structure-based multi-head cross-attention module in the decoder, where As the query vector in the attention mechanism, r1, r2, ..., r q As the key and value vectors in the attention mechanism, the output is a structure-based attention result. After further regularization, the word representation results are obtained as D = d1, d2, ... d m For the comment word 'z' in the code comment text i Its word representation vector is d i .
[0054] Each comment word in the code comment text i The word representation vector d i With semantic representation vectors h1, h2, ..., h p The attention scores α1, α2, ..., α3 of each word in the code function text are obtained using the dot product method. p ; Each comment word in the code comment text i The word representation vector d i With structural representation vectors r1, r2, ..., r q The attention scores β1, β2, ..., β of each node in the code structure are obtained using the dot product method. q Its formula is expressed as follows:
[0055] d i ×[h1, h2, ..., h p ] = [α1, α2, ..., α p ]
[0056] d i ×[r1, r2, ..., r q ] = [β1, β2, ..., β q ]
[0057] (3-3) The attention scores α1, α2, ..., α3 for each word in the code function text. p With semantic representation vectors h1, h2, ..., h p Perform a weighted summation to obtain the comment word z in the code comment text. i For the overall semantic representation vector H of the code function text i Similarly, the attention scores β1, β2, ..., β of each node in the code structure are... q With structural representation vectors r1, r2, ..., r q Perform a weighted summation to obtain the comment word z in the code comment text. iFor the overall structural representation vector R of the code abstract syntax tree i .
[0058] The above process can be represented as:
[0059] [α1, α2, ..., α] p ]×[h1, h2, ..., h p ] T =H i
[0060] [β1, β2, ..., β q ]×[r1, r2, ..., r q ] T =R i
[0061] (3-4) Comment word z i The word representation vector d i With the overall semantic representation vector H i Overall structural representation vector R i By splicing, we get T. i After regularization using L2_Normalize, the annotation term z is obtained. i The overall representation vector With annotation word z i Forming a binary pair As key-value pairs (key, value), they are added to the nearest neighbor database, such as Figure 2 As shown.
[0062] T i =Concat(d i H i R i )
[0063]
[0064] (3-5) After adding the annotation words of all training samples to the nearest neighbor database, the final nearest neighbor database is completed. The FAISS.METRIC_INNER_PRODUCT function from the FAISS library is used to build an index for the nearest neighbor database, facilitating subsequent fast retrieval.
[0065] (4) Model Autoregressive Generation Process: For the code function text to be annotated and its abstract syntax tree, the traditional autoregressive generation method is used for decoding. At each time step, the probability distribution of the target word in the vocabulary and the overall representation vector q of the target word are obtained. t Let this probability distribution be denoted as the model-based target word probability distribution P. m (S t |Code, S <t), representing the word S generated based on the code function text and code structure Code, as well as the previous time step. ＜t Generate the current target word S t The probability distribution.
[0066] The target word overall representation vector q t The generation process is the same as step (3) above, except that S is used. ＜t As input to the masked multi-head self-attention module in the decoder, the last vector in the word representation result is used as the word representation vector d. t The word representation vector d t With the overall semantic representation vector H t Overall structural representation vector R t The concatenation is performed, and the concatenation result is regularized to obtain the overall representation vector q of the target word. t .
[0067] (5) Nearest neighbor word retrieval process: The overall representation vector q of the target word t Using the cosine similarity index, a search is performed in the nearest neighbor database constructed in step (3) to obtain the Top_K = 8 nearest neighbor words [neigh1, neigh2, ..., neigh8], and the similarity distances between the nearest neighbor words and the overall representation vector of the target word [dist1, dist2, ..., dist8]. The probability distribution P of the target word based on the nearest neighbor words is calculated according to the similarity distance. r (S t |Code, S <t The calculation formula is as follows:
[0068]
[0069] g[dist, Temperature]=[dist1, dist2,…, dist8]*Temperature
[0070] Where g[dist, Temperature] represents the scaled similarity distance, Temperature represents the temperature adjustment coefficient, exp(.) represents the normalization exponential function, and exp(g[dist, Temperature]) readjusts the probability distribution of the nearest neighbor words retrieved based on the similarity distance. Indicates when V t =S t The value is 1 if it is true, otherwise it is 0; V t It represents an element in the nearest neighbor word [neigh1, neigh2, ..., neigh8].
[0071] (6) Fusion of probability distributions: The model-based target word probability distribution P from step (4) is fused. m (S t |Code, S <t The target word probability distribution P based on nearest neighbor words in step (5) r (S t |Code, S <t The target word probability distribution is obtained by fusing the data according to a certain weight coefficient λ. The calculation formula is as follows:
[0072] P(S t |Code, S <t )=(1-λ)*P n (S t |Code, S <t )+λ*P r (S t |Code, S <t )
[0073] Based on the final target word probability distribution, the word with the highest probability is taken as the output result of the current time step.
[0074] This embodiment also provides an automated source code comment generation system based on word-level retrieval, which is used to implement the above embodiments. The terms "module," "unit," etc., used below can refer to a combination of software and / or hardware that performs a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible.
[0075] This embodiment provides an automated source code annotation generation system based on word-level retrieval, including:
[0076] The data acquisition module is used to acquire the training dataset consisting of code function text, code abstract syntax tree, and code comment text, as well as to acquire the code function text to be annotated and its abstract syntax tree;
[0077] The encoder-decoder network module is used to extract the semantic representation vector of the code function text and the structural representation vector of the code abstract syntax tree, and to interact with the attention results, semantic representation vector and structural representation vector of the code comment text and decode to generate comment text;
[0078] The encoder-decoder network training module is used to train the encoder-decoder network module using the training dataset.
[0079] The nearest neighbor word database construction module is used to obtain the overall representation vector of each annotation word in the code annotation text using the trained encoder-decoder network, and to construct the nearest neighbor word database.
[0080] The initial target word probability distribution calculation module is used to generate the model-based target word probability distribution and the overall target word representation vector at the current time step using a trained encoder-decoder network in an autoregressive manner for the code function text to be annotated and its abstract syntax tree.
[0081] The nearest neighbor word search module is used to retrieve the K nearest neighbor words with the highest cosine similarity to the overall representation vector of the target word from the nearest neighbor word database, and generate the target word probability distribution based on the nearest neighbor words;
[0082] The probability distribution fusion module is used to fuse the target word probability distribution based on the model and the target word probability distribution based on the nearest neighbor word, and take the target word with the highest probability in the fused probability distribution as the annotation word generated at the current time step.
[0083] In one specific embodiment of the present invention, the calculation process of the nearest neighbor word database construction module includes:
[0084] Encoding the code function text yields a semantic representation vector, and encoding the code abstract syntax tree yields a structural representation vector;
[0085] Obtain the word representation vector of each comment word in the code comment text. Take the dot product of the word representation vector and the semantic representation vector of each comment word in the code comment text to obtain the attention score of each word in the code function text. Take the dot product of the word representation vector and the structural representation vector of each comment word in the code comment text to obtain the attention score of each node in the code structure.
[0086] The attention score of each word in the code function text is weighted and summed with the semantic representation vector to obtain the overall semantic representation vector of the comment words in the code comment text for the code function text; the attention score of each node in the code structure is weighted and summed with the structure representation vector to obtain the overall structure representation vector of the comment words in the code comment text for the code abstract syntax tree.
[0087] The word representation vector of each annotation word is concatenated with the overall semantic representation vector and the overall structural representation vector. The concatenation result is then regularized to obtain the overall representation vector of the annotation word. The overall representation vector of the annotation word and the corresponding annotation word are then placed into the nearest neighbor word database as key-value pairs.
[0088] For the system embodiments, since they basically correspond to the method embodiments, relevant details can be found in the descriptions of the method embodiments; the implementation methods of the remaining modules will not be repeated here. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0089] The system embodiments of the present invention can be applied to any device with data processing capabilities, such as a computer or other similar device. The system embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution.
[0090] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for automatically generating source code comments based on word-level retrieval, characterized in that, include: (1) Obtain the training dataset consisting of code function text, code abstract syntax tree, and code comment text; (2) Train the encoder-decoder network using the training dataset. The encoder in the encoder-decoder network is used to extract the semantic representation vector of the code function text and the structural representation vector of the code abstract syntax tree. The decoder in the encoder-decoder network is used to interact with the attention results, semantic representation vector and structural representation vector of the code annotation text and decode to generate annotation text. The decoder includes a masked multi-head self-attention module, a semantic-based multi-head cross-attention module, a structure-based multi-head cross-attention module, and a feedforward neural network. First, the masked multi-head self-attention module is used to obtain the attention result of the input code annotation text. Then, the semantic-based multi-head cross-attention module and the structure-based multi-head cross-attention module interact with each other. Finally, the feedforward neural network generates a model-based target word probability distribution. (3) Using the trained encoder-decoder network, obtain the overall representation vector of each annotation word in the code annotation text and construct a nearest neighbor word database; (4) For the code function text to be annotated and its abstract syntax tree, the trained encoder-decoder network is used to generate the model-based target word probability distribution and target word overall representation vector at the current time step in an autoregressive manner; (5) Retrieve the K nearest neighbor words with the highest cosine similarity to the overall representation vector of the target word from the nearest neighbor word database, and generate the target word probability distribution based on the nearest neighbor words. The calculation formula is as follows: ； ； in, This represents the probability distribution of the target word based on its nearest neighbors. This represents the code function text to be commented and its abstract syntax tree. This refers to the annotation words generated before the current time step t. The target word representing the current time step t, Indicates neighboring words One of the elements, This represents the Kth nearest neighbor word with the highest cosine similarity. This represents the scaled similarity distance, and Temperature represents the temperature adjustment factor. This represents the similarity distance between the K nearest neighbors with the highest cosine similarity to the overall representation vector of the target word. This represents the normalized exponential function; Indicates when The value is 1 if the condition is met, and 0 otherwise. (6) The target word probability distribution based on the model and the target word probability distribution based on the nearest neighbor are fused, and the target word with the highest probability in the fused probability distribution is taken as the annotation word generated at the current time step; (7) Repeat steps (4)-(6) until the comment ends.
2. The method for automatically generating source code comments based on word-level retrieval according to claim 1, characterized in that, Step (3) includes: (3-1) The semantic representation vector is obtained by encoding the code function text, and the structural representation vector is obtained by encoding the code abstract syntax tree; (3-2) Obtain the word representation vector of each comment word in the code comment text, and obtain the attention score of each word in the code function text by taking the dot product of the word representation vector of each comment word in the code comment text and the semantic representation vector; obtain the attention score of each node in the code structure by taking the dot product of the word representation vector of each comment word in the code comment text and the structural representation vector. (3-3) The attention score of each word in the code function text is weighted and summed with the semantic representation vector to obtain the overall semantic representation vector of the comment words in the code comment text for the code function text; the attention score of each node in the code structure is weighted and summed with the structure representation vector to obtain the overall structure representation vector of the comment words in the code comment text for the code abstract syntax tree. (3-4) Concatenate the word representation vector of each annotation word with the overall semantic representation vector and the overall structural representation vector. After regularizing the concatenation result, obtain the overall representation vector of the annotation word. Put the overall representation vector of the annotation word and the corresponding annotation word as key-value pairs into the nearest neighbor word database.
3. The method for automatically generating source code comments based on word-level retrieval according to claim 2, characterized in that, The specific method for obtaining the word representation vector of each comment word in the code comment text is as follows: The code comment text is used as input to the masked multi-head self-attention module in the decoder, and the output is the attention result of the comment text; the attention result of the comment text and the semantic representation vector are used as input to the semantic-based multi-head cross-attention module in the decoder, and the output is the semantic-based attention result. The semantic-based attention result and structural representation vector are used as input to the structure-based multi-head cross-attention module in the decoder, and the output is the structure-based attention result. After regularization, the word representation result is obtained. Each vector in the word representation result corresponds to the word representation vector of each annotation word in the code annotation text.
4. The method for automatically generating source code comments based on word-level retrieval according to claim 3, characterized in that, In the semantic-based multi-head cross-attention module, the attention result of the annotated text is used as the query vector in the attention mechanism, and the semantic representation vector is used as the key vector and value vector in the attention mechanism; in the structure-based multi-head cross-attention module, the semantic-based attention result is used as the query vector in the attention mechanism, and the structure representation vector is used as the key vector and value vector in the attention mechanism.
5. The method for automatically generating source code comments based on word-level retrieval according to claim 2, characterized in that, In step (4), when obtaining the overall representation vector of the target word, the annotation words generated before the current time step t are included. As input to the masked multi-head self-attention module in the decoder, the output is annotation words. Attention results; After interacting with the semantic-based multi-head cross-attention module and the structure-based multi-head cross-attention module respectively, word representation results are generated. The last vector in the word representation result is used as the current word representation vector. The current word representation vector is concatenated with the overall semantic representation vector and the overall structure representation vector. The concatenation result is regularized to obtain the overall representation vector of the target word.
6. The method for automatically generating source code comments based on word-level retrieval according to claim 1, characterized in that, The model-based target word probability distribution and the neighbor-based target word probability distribution are weighted to obtain the fused probability distribution.
7. A source code annotation automatic generation system based on word-level retrieval, characterized in that, include: The data acquisition module is used to acquire the training dataset consisting of code function text, code abstract syntax tree, and code comment text, as well as to acquire the code function text to be annotated and its abstract syntax tree; The encoder-decoder network module includes an encoder and a decoder. The encoder extracts the semantic representation vector of the code function text and the structural representation vector of the code abstract syntax tree. The decoder interacts with the attention results, semantic representation vectors, and structural representation vectors of the code annotation text and decodes them to generate annotation text. The decoder includes a masked multi-head self-attention module, a semantic-based multi-head cross-attention module, a structure-based multi-head cross-attention module, and a feedforward neural network. First, the masked multi-head self-attention module is used to obtain the attention results of the input code annotation text. Then, the semantic-based multi-head cross-attention module and the structure-based multi-head cross-attention module interact with each other. Finally, the feedforward neural network generates a model-based target word probability distribution. The encoder-decoder network training module is used to train the encoder-decoder network module using the training dataset. The nearest neighbor word database construction module is used to obtain the overall representation vector of each annotation word in the code annotation text using the trained encoder-decoder network, and to construct the nearest neighbor word database. The initial target word probability distribution calculation module is used to generate the model-based target word probability distribution and the overall target word representation vector at the current time step using a trained encoder-decoder network in an autoregressive manner for the code function text to be annotated and its abstract syntax tree. The nearest neighbor search module retrieves the K nearest neighbors with the highest cosine similarity to the overall representation vector of the target word from the nearest neighbor database, and generates a probability distribution of the target word based on the nearest neighbors. The calculation formula is as follows: ； ； in, This represents the probability distribution of the target word based on its nearest neighbors. This represents the code function text to be commented and its abstract syntax tree. This refers to the annotation words generated before the current time step t. The target word representing the current time step t, Indicates neighboring words One of the elements, This represents the Kth nearest neighbor word with the highest cosine similarity. This represents the scaled similarity distance, and Temperature represents the temperature adjustment factor. This represents the similarity distance between the K nearest neighbors with the highest cosine similarity to the overall representation vector of the target word. This represents the normalized exponential function; Indicates when The value is 1 if the condition is met, and 0 otherwise. The probability distribution fusion module is used to fuse the target word probability distribution based on the model and the target word probability distribution based on the nearest neighbor word, and take the target word with the highest probability in the fused probability distribution as the annotation word generated at the current time step.
8. The source code annotation automatic generation system based on word-level retrieval according to claim 7, characterized in that, The calculation process of the nearest neighbor database construction module includes: Encoding the code function text yields a semantic representation vector, and encoding the code abstract syntax tree yields a structural representation vector; Obtain the word representation vector of each comment word in the code comment text. Take the dot product of the word representation vector and the semantic representation vector of each comment word in the code comment text to obtain the attention score of each word in the code function text. Take the dot product of the word representation vector and the structural representation vector of each comment word in the code comment text to obtain the attention score of each node in the code structure. The attention score of each word in the code function text is weighted and summed with the semantic representation vector to obtain the overall semantic representation vector of the comment words in the code comment text for the code function text; the attention score of each node in the code structure is weighted and summed with the structure representation vector to obtain the overall structure representation vector of the comment words in the code comment text for the code abstract syntax tree. The word representation vector of each annotation word is concatenated with the overall semantic representation vector and the overall structural representation vector. The concatenation result is then regularized to obtain the overall representation vector of the annotation word. The overall representation vector of the annotation word and the corresponding annotation word are then placed into the nearest neighbor word database as key-value pairs.