Code annotation generation method and device based on fusion encoder

Abstract

The invention relates to the technical field of code annotation automatic generation, in particular to a code annotation generation method and device based on a fusion encoder. The method comprises the steps of obtaining a text sequence vector of a code, a structure sequence vector of the code and an adjacent matrix of the code; obtaining a multi-level sequence vector based on the text sequence vector; obtaining a multi-view structure vector based on the structure sequence vector and the adjacent matrix; splicing the multi-level sequence vector and the multi-view structure vector to obtain a spliced vector; and decoding the splicing vector to generate a code annotation. According to the method, a fusion model of a multi-level sequence encoder and a multi-view structure encoder is adopted, and the accuracy and readability of automatic generation of code annotations are remarkably improved through multi-modal feature fusion and a self-adaptive attention mechanism.

Description

Technical Field

[0001] This invention relates to the field of automatic code comment generation technology, and in particular to a code comment generation method and apparatus based on a fusion encoder. Background Technology

[0002] As software scales, software development projects are typically completed collaboratively by multiple developers. Before participating in a project, each developer needs to read the relevant documentation to understand the various components and modules, as well as the related code within those modules. However, code is an abstract and structured language, strictly adhering to syntactic and semantic rules, yet also influenced by the individual styles of developers. Differences in coding habits among developers can make it difficult to quickly understand each other's code, creating obstacles to collaborative software development. Code comments become the best way for developers to understand the code. By generating concise natural language comments, developers can quickly understand the code's functionality, usage, version information, and other relevant details, greatly improving development efficiency.

[0003] Early code comment generation primarily relied on template-based and information retrieval-based methods. However, due to the lack of flexibility and semantic accuracy in the generated comments, these methods were gradually replaced by deep learning-based approaches. Deep learning-based methods, through end-to-end learning, can capture more complex code-comment mapping relationships, significantly improving the quality of code comment generation. However, current deep learning-based methods either generate comments based on code text sequences or code structure sequences, both of which suffer from insufficient representation learning or incomplete code representation, creating a bottleneck in code comment generation quality. How to automatically generate high-quality code comments remains a technical problem to be solved.

[0004] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention

[0005] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention proposes a code annotation generation method and apparatus based on a fusion encoder, which can significantly improve the accuracy and readability of automatic code annotation generation.

[0006] The embodiments of the present invention adopt the following technical solutions: In a first aspect, the present invention provides a code annotation generation method based on a fusion encoder, specifically: obtaining the text sequence vector of the code, the structure sequence vector of the code, and the adjacency matrix of the code; Based on the text sequence vector, a multi-level sequence vector is obtained; Based on the structure sequence vector and the adjacency matrix, a multi-view structure vector is obtained; By concatenating the multi-level sequence vector and the multi-view structure vector, a concatenated vector is obtained; The concatenated vector is decoded to generate code comments.

[0007] Preferably, obtaining the text sequence vector of the code, the structure sequence vector of the code, and the adjacency matrix of the code includes: Obtain the code sequence and divide the code sequence into multiple subsequences; Each subsequence is represented as a fixed-size word embedding matrix, resulting in a text sequence vector comprising multiple word embedding matrices; Obtain the abstract syntax tree of the code, wherein the abstract syntax tree includes multiple child nodes; Each child node is represented as a word embedding vector of a fixed size, resulting in the structure sequence vector that includes multiple word embedding vectors; The adjacency matrix is ​​obtained based on the abstract syntax tree.

[0008] Preferably, the adjacency matrix takes values ​​of 0 and 1, and obtaining the adjacency matrix based on the abstract syntax tree includes: Parse the abstract syntax tree to obtain the edge relationships between each child node; If there are edges between child nodes, then set the corresponding position in the adjacency matrix to 1; If there are no edges between child nodes, the corresponding position in the adjacency matrix is ​​set to 0.

[0009] Preferably, based on the text sequence vector, a multi-level sequence vector is obtained, including: Based on the word embedding matrix in the text sequence vector, the corresponding position encoding matrix is ​​obtained; the word embedding matrix and the position encoding matrix are fused to obtain the position sequence vector; The semantic information of the internal lexical elements and the dependencies between the lexical elements in the position sequence vector are extracted to obtain the lexical semantic features; Extract the sentence semantic information from the lexical semantic features to obtain the sentence semantic features; Extract the temporal information relationship of the semantic features of the sentence to obtain temporal features; The temporal features are processed to obtain the multi-level sequence vector.

[0010] Preferably, based on the word embedding matrix in the text sequence vector, the corresponding position encoding matrix is ​​obtained, including: Determine the column index of the embedding element in each word embedding matrix; If the column index is even, the column index is normalized to obtain a position factor, and the sine function value corresponding to the position factor is calculated. The sine function value is used as the position encoding information corresponding to the embedded element. If the column index is odd, the column index is normalized to obtain a position factor, and the cosine function value corresponding to the position factor is calculated. The cosine function value is used as the position encoding information corresponding to the embedded element. The position encoding matrix is ​​obtained by integrating the position encoding information corresponding to all embedded elements.

[0011] Preferably, based on the structure sequence vector and the adjacency matrix, a multi-view structure vector is obtained, including: Global information is extracted from the structure sequence vector to obtain global features; Extract the structural information from the adjacency matrix to obtain structural features; Local information is extracted from the structural sequence vector to obtain local features; The multi-view structural vector is obtained by fusing the global features, the structural features, and the local features.

[0012] Preferably, the concatenated vector is decoded to generate code comments, including: Based on the starting identifier and the concatenation vector, the annotation probability distribution of the first time step and the hidden state information corresponding to the first time step are generated. The annotation with the highest probability value in the annotation probability distribution of the first time step is taken as the target annotation of the first time step; Based on the target annotation and the hidden state information corresponding to the first time step, the annotation probability distribution and the hidden state information corresponding to the second time step are generated. The annotation with the highest probability value in the annotation probability distribution of the second time step is taken as the target annotation of the second time step. This process continues until the generated target annotation becomes the end identifier. The target annotations generated in all time steps are concatenated in order to obtain the code annotations corresponding to the code sequence.

[0013] Preferably, concatenating the multi-level sequence vector and the multi-view structure vector to obtain a concatenated vector includes: Adjust the multi-level sequence vector and the multi-view structure vector respectively, so that the adjusted multi-level sequence vector and the adjusted multi-view structure vector have the same length in the first dimension; The adjusted multi-level sequence vector is concatenated along the second dimension to the adjusted multi-view structure vector to obtain the concatenated vector.

[0014] In a second aspect, the present invention provides a code annotation generation apparatus based on a fusion encoder, specifically comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor to perform the code annotation generation method based on a fusion encoder in the first aspect.

[0015] Thirdly, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors to perform the method provided by the method described in the first aspect.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention starts with two feature information of code, namely textual information and structural information, and uses a multi-level sequence encoder to encode the sequence information and a multi-view structural encoder to encode the structural information. The two encoders can achieve a complete feature representation of the code and generate a unified vector representation, which makes up for the shortcomings of single modality information and significantly improves the accuracy and readability of code annotation generation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0018] Figure 1 This is a flowchart illustrating a code annotation generation method based on a fusion encoder provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the method for obtaining the adjacency matrix provided in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the method for obtaining a position encoding matrix provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the abstract syntax tree provided in the embodiments of the present invention; Figure 5 This is a schematic diagram of a code annotation generation device based on a fusion encoder provided in an embodiment of the present invention; The reference numerals in the attached figures are as follows: 21: Processor; 22: Memory. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] Unless the context otherwise requires, throughout the specification and claims, the term "comprising" is interpreted as openly inclusive, meaning "including, but not limited to." In the description of the specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics mentioned may be included in any suitable manner in any one or more embodiments or examples; that is, although they may be incorporated into embodiments or examples using the above terms for reasons such as order and position, it does not limit them to be incorporated in combination by a single embodiment or example.

[0021] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, for example, the description may use the prefix "A" or "B" to describe the same type of nouns as two independent entities. In this case, the corresponding features defined with "A" and "B" are used only to distinguish between similar entities and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0022] In the description of this invention, the expression “A and / or B” (where A and B are used to formally represent specific features) will be used. The corresponding expression includes the following three combinations: only A, only B, and a combination of A and B.

[0023] As used in this invention, “about,” “approximately,” or “approximately” includes the stated value and the average value within an acceptable range of deviation from a particular value, wherein the acceptable range of deviation is determined by a person skilled in the art taking into account the measurement under discussion and the error associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0024] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0025] Example 1: like Figure 1 As shown, Embodiment 1 of the present invention provides a code annotation generation method based on a fusion encoder, which specifically includes the following steps: Step 101: Obtain the text sequence vector of the code, the structure sequence vector of the code, and the adjacency matrix of the code.

[0026] In one embodiment, the code sequence Code={ , ,..., The code sequence is divided into multiple subsequences. , ,..., Each subsequence is represented as a fixed-size word embedding matrix, with each word embedding matrix having a size of m×d1, resulting in the text sequence vector S={ which includes multiple word embedding matrices. ,..., }; Obtain the abstract syntax tree of the code, which includes multiple child nodes and the connection relationships between each child node; Represent each child node as a fixed-size word embedding vector, with each word embedding vector having a size of d2, to obtain the structure sequence vector N={ including multiple word embedding vectors. , ,..., Based on the structure sequence vector, the adjacency matrix A is obtained. Here, n represents the number of subsequences, l represents the number of child nodes, m represents the total number of lexical elements in the subsequences, d1 represents the dimension of word embeddings, and d2 represents the dimension of word embeddings. In this embodiment, the values ​​of d1 and d2 can both be 512. The values ​​of d1 and d2 can be adjusted according to actual conditions. The above examples do not limit this embodiment. Lexical elements include keywords (such as def), identifiers (such as add), and separators (such as ,) in the code.

[0027] Step 102: Based on the text sequence vector, obtain multi-level sequence vectors.

[0028] In one embodiment, a corresponding positional encoding matrix is ​​obtained based on the word embedding matrix in the text sequence vector; the word embedding matrix and the positional encoding matrix are then fused to obtain the positional sequence vector. Position sequence vector By injecting location information, the contextual understanding capability of the data is significantly enhanced, transforming the original sequence from disordered data into interpretable and analyzable intelligent data. Semantic information of internal lexical elements and dependencies between lexical elements are extracted from the location sequence vector to obtain lexical semantic features. Lexical and semantic features This represents the new semantic meaning expressed by each lexical element after being combined with a specific context; the sentence semantic information of the lexical semantic features is extracted to obtain the sentence semantic features. Sentence semantic features This means that after knowing the specific meaning of the lexical elements in the sequence, the complete semantics of the sequence is captured; the temporal information relationship of the semantic features of the sentences is extracted to obtain temporal features. Temporal characteristics This means that after knowing the complete semantics of a sequence, the temporal order of that sequence with other sequences is obtained; the temporal features are processed, and the specific meanings of lexical elements, the complete semantics of the sequence, and the temporal order of the sequence are analyzed to gain a more comprehensive understanding of the entire sequence, resulting in the multi-level sequence vector. .

[0029] Step 103: Based on the structure sequence vector and the adjacency matrix, obtain the multi-view structure vector.

[0030] In one embodiment, global information is extracted from the structural sequence vector N to obtain global feature G, which represents the new semantics expressed by a node after fusing the semantics of all nodes; structural information is extracted from the adjacency matrix A to obtain structural feature S, which represents the new semantics expressed by a node after fusing the semantics of nodes directly connected to it; local information is extracted from the structural sequence vector N to obtain local feature L, which represents the new semantics expressed by a node after fusing the semantics of nodes connected before and after it; by fusing the global feature, the structural feature, and the local feature, the entire sequence is understood more comprehensively, resulting in the multi-view structural vector. .

[0031] Step 104: Concatenate the multi-level sequence vector and the multi-view structure vector to obtain the concatenated vector.

[0032] In one embodiment, for multi-level sequence vectors and multi-view structure vector The concatenation is performed to obtain the concatenated vector Fusioncode. First, global average pooling is used to adjust the multi-level sequence vector and the multi-view structure vector, making the adjusted multi-level sequence vector and the adjusted multi-view structure vector have the same length in the first dimension. Then, the adjusted multi-level sequence vector is concatenated along the second dimension to the adjusted multi-view structure vector to obtain the concatenated vector. The concatenation method is Fusioncode=[ : ],in, Represents a multi-level sequence vector. ":" represents a multi-view structure vector; ":" represents a vector concatenation operation. For example, the size of the multi-level sequence vector [[1,2,3][4,2,3]] is (2,3); the size of the multi-view structure vector [[2,3][4,2]] is (2,2). Since the two vectors have the same length in the first dimension, they are concatenated directly along the second dimension to obtain the concatenated vector [[2,3,1,2,3][4,2,4,2,3]].

[0033] Step 105: Decode the concatenated vector to generate code comments.

[0034] In one embodiment, the decoder starts from the start identifier. <bos>Starting at time step t=0, based on the start identifier and the concatenation vector, the annotation probability distribution and the corresponding hidden state information for the first time step are generated. The annotation with the highest probability value in the annotation probability distribution of the first time step is taken as the target annotation for the first time step. Based on the target annotation and the corresponding hidden state information for the first time step, the annotation probability distribution and the corresponding hidden state information for the second time step are generated. The annotation with the highest probability value in the annotation probability distribution of the second time step is taken as the target annotation for the second time step. In subsequent time steps, the annotation probability distribution and the hidden state information for the current time step are obtained based on the target annotation and the hidden state information of the previous time step, and the annotation with the highest probability value in the annotation probability distribution of the current time step is taken as the target annotation for the current time step. This process continues until the generated target annotation becomes the end identifier. The target annotations generated by all time steps are concatenated in sequence to obtain the code annotations corresponding to the code sequence. The probability formula for calculating the annotation probability distribution generated at time step t is P( |Fusioncode)= , where P( |Fusioncode) is the output probability function. This represents the word with the highest probability of being generated at time step t, and n represents the length of the output result.

[0035] In this embodiment, starting from the two types of feature information of the code, namely textual information and structural information, a multi-level sequence encoder is used to encode the sequence information, and a multi-view structural encoder is used to encode the structural information. The two encoders can achieve a complete feature representation of the code and generate a unified vector representation, which makes up for the shortcomings of single modality information and significantly improves the accuracy and readability of code annotation generation.

[0036] To achieve optimal performance, the decoder needs to be trained.

[0037] In one embodiment, the training phase of the decoder specifically includes: representing the code target annotation Y as a code target annotation vector through vector embedding; using the concatenated vector Fusioncode and the code target annotation vector as input to the decoder; comparing the code annotation sequence generated by the decoder with the code target annotation vector; calculating the loss value using minimizing cross-entropy; and iteratively training using the code training dataset until the loss value gradually converges to a stable value, indicating that the decoder training is complete. The stable value is set according to the training objective, where the code target annotation Y = { <bos> } refers to the code sequence Code={ , ,..., Manually written target annotations <bos>This is the starting identifier for code comments. <eos>This is the end-of-comment identifier, used to determine the start and end of a code comment. The loss function used by the decoder is as follows:

[0038] in, This represents the logarithm of the probability of generating the corresponding word. This represents the learnable parameters in the model, where N represents the length of the input code target annotation Y. This represents all generated results before time step t. This indicates the result generated at time step t.

[0039] like Figure 2 As shown, the specific method for obtaining the adjacency matrix provided in this embodiment includes the following steps: Step 201: Parse the abstract syntax tree to obtain the relationship between the edges between each child node.

[0040] In one embodiment, an abstract syntax tree (API) is a structured representation of code. The API retains only the key syntactic structures of the code logic, ignoring specific syntactic details (such as parentheses, semicolons, and other irrelevant symbols). The API is a tree structure that connects each child node through parent-child relationships defined by syntactic rules. By parsing the API, the relationships between each child node can be explicitly obtained. Child nodes mainly include declaration nodes (such as function declarations), statement nodes (such as return statements), and identifier nodes (such as variable references).

[0041] Step 202: If there are edges between child nodes, set the corresponding position in the adjacency matrix to 1.

[0042] Step 203: If there is no edge between child nodes, set the corresponding position in the adjacency matrix to 0.

[0043] like Figure 3 As shown, the specific method for obtaining the position encoding matrix provided in this embodiment includes the following steps: Step 301: Determine the column index of the embedding element in each word embedding matrix.

[0044] In one embodiment, a subsequence includes multiple lexical elements. A subsequence is represented as a word embedding matrix. In this case, each row of the word embedding matrix corresponds to the vector representation of a lexical element. Each value in the row is called an embedding element. The number of columns in the word embedding matrix represents the number of lexical elements in a subsequence.

[0045] Step 302: If the column index is even, normalize the column index to obtain the position factor, calculate the sine function value corresponding to the position factor, and use the sine function value as the position encoding information corresponding to the embedded element.

[0046] In one embodiment, when the column index of the embedded element is even, the position encoding information corresponding to the embedded element is obtained using the following formula: Where j is the column index of the embedded element, The vector dimension representing the subsequence. In this embodiment, the positional encoding information corresponding to the embedded element is used. It can be 512, where, Adjustments can be made according to the actual situation, and the above examples do not limit this embodiment.

[0047] Step 303: If the column index is odd, normalize the column index to obtain a position factor, calculate the cosine function value corresponding to the position factor, and use the cosine function value as the position encoding information corresponding to the embedded element.

[0048] In one embodiment, when the column index of the embedded element is odd, the position encoding information corresponding to the embedded element is obtained using the following formula: Where j is the column index of the embedded element, The vector dimension representing the subsequence. In this embodiment, the positional encoding information corresponding to the embedded element is used. It can be 512, where, Adjustments can be made according to the actual situation, and the above examples do not limit this embodiment.

[0049] Step 304: Integrate the position encoding information corresponding to all embedded elements to obtain the position encoding matrix.

[0050] In one embodiment, the size of the position encoding matrix is ​​the same as that of the word embedding matrix.

[0051] Example 2: The embodiments of the present invention are based on Embodiment 1, and the solutions are described in conjunction with specific scenarios.

[0052] The solution is illustrated using a code snippet that adds two numbers as an example. Comments are automatically generated for this code snippet, as follows: def add(a, b): return a+b The code snippet above is represented as a code sequence, code={'def','add','(','a','b','(',':','return','a','+','b'}. The length of the code sequence code is 12, and it includes 12 lexical elements: 'def','add','(','a','b','(',':','return','a','+','b'. In this embodiment, the code snippet is divided line by line. If the code snippet has two lines, the code sequence is divided into two subsequences. , , where subsequence ={'def','add','(','a',','b')',':'}, subsequence It is an 8×512 matrix; subsequence ={'return','a','+','b'}, subsequence It is a 4×512 matrix. At this time, each lexical element in the code sequence is converted into a 512-dimensional vector, and finally the text sequence vector S={s1,s2} is obtained. The text sequence vector S is a list containing two matrices.

[0053] The above code snippet is parsed using Python's `ast` module, generating an abstract syntax tree (AST). The AST consists of eight child nodes, as follows: Figure 4 As shown, the eight child nodes are 'FunctionDef', 'arguments', 'arg:a', 'arg:b', 'Return', 'BinOp', 'Name:a', and 'Name:b'. Representing each of the eight child nodes as a fixed-size word embedding vector yields the structure sequence vector N={ , ,..., The structure sequence vector N is an 8×512 matrix, where each row of the matrix is ​​a fixed-size word embedding vector, corresponding to the feature representation of a node in the graph.

[0054] Generate an adjacency matrix A from the abstract syntax tree. Adjacency matrix A is an 8×8 matrix. The formula for obtaining adjacency matrix A is as follows:

[0055] Where A represents the adjacency matrix, and edge(i,j) represents the connection relationship between child nodes and the edges between child nodes.

[0056] The automatic generation of code comments is essentially the process of encoding code semantics into vector representations and then decoding them into natural language descriptions. However, for the task of automatically generating massive amounts of code comments, traditional sequence models, such as recurrent neural networks (RNNs), cannot fully capture long-distance dependencies. Therefore, a self-attention mechanism is needed. The self-attention mechanism can identify that key syntactic structures in the code (such as function definitions and conditional branches) are important, while redundant details in the code (such as formatting characters and repeated variables) are not important. This reduces the attention weight distribution on irrelevant features and improves the accuracy and readability of code comments.

[0057] Self-attention mechanisms themselves do not involve understanding the sequence order. Therefore, when processing text sequence vectors S, it is necessary to first add positional encoding to the text sequence vector S, integrating positional information into the text sequence vector S to obtain a positional sequence vector. ={ , The subsequence s1 has a length of 8, which generates 8 positional codes. Each positional code is a 512-dimensional vector. These are then added positionally to the 8 word embedding vectors of subsequence s1 to obtain the positional sequence vector. Similarly, positional encoding is added to subsequence s2 in the same way to obtain the positional sequence vector. .

[0058] In one embodiment, a self-attention mechanism is applied independently to each subsequence, with the position sequence vector... It is an 8×512 matrix, using three learnable weight matrices. , and , position sequence vector Each word embedding vector in the query vector is converted into a query vector Q, a key vector K, and a value vector V, with the conversion method Q=x K=x V=x Where x represents the position sequence vector Each word embedding vector and each weight matrix in the dataset has a size of 512×d_k, where d_k is typically 64.

[0059] Calculate the position sequence vector Self-attention SAN ( ), self-attention SAN ( ) is an 8×512 matrix that obtains self-attention SAN ( The formula is as follows: SAN ( )=softmax( V Where Q is the position sequence vector Through the above weight matrix The resulting query vector, K, is a position sequence vector. Through the above weight matrix The resulting key vector, V, is a position sequence vector. Through the above weight matrix The transformed value vector It is the dimension of the key vector K.

[0060] position sequence vector and self-attention SAN ( ) lexical semantic features are obtained by adding them position by position. =SAN( )+ Lexical and semantic features Represents a position sequence vector Each lexical element in the vector expresses a new semantic meaning when combined with a specific context. Similarly, for position sequence vectors... Perform the same operation to obtain The final lexical and semantic features obtained ={ , }

[0061] In one embodiment, an LSTM network is used to encode each semantic feature to obtain sentence semantic features. These sentence semantic features represent the complete semantics of the sequence after knowing the specific meaning of each lexical element within the sequence. The specific LSTM encoding process is as follows: ... The input is an LSTM, which processes each lexical element of the sequence step by step, obtaining the hidden state at the last time step. Hide state as a subsequence The representation of the hidden state. It has 512 dimensions. (Pair of subsequences) Perform the same operation to obtain the hidden state. , It also has 512 dimensions. The final sentence semantic features obtained are... It is a 2×512 matrix, where each row of the matrix represents a subsequence, and each subsequence is represented by a 512-dimensional vector.

[0062] In one embodiment, a cross-self-attention mechanism is applied between two sub-sequences. By dynamically calculating the semantic association weights between the sequences, bidirectional information interaction is achieved, i.e., capturing the dependency relationship between the two sub-sequences (two lines of code). For example, if the first line is a function definition and the second line is a return statement, the cross-self-attention mechanism captures the association between the function definition and the return statement. The specific operation of applying the cross-self-attention mechanism is as follows: [The text abruptly ends here, likely due to an incomplete sentence or a missing section.] As input, two new 512-dimensional vectors are obtained through self-attention computation, which are the time-series features. Obtaining temporal features The formula is as follows:

[0063] =SAN( )+

[0064] Here, the SAN function represents a self-attention network. This indicates that the i-th subsequence represents feature information. This represents the temporal information feature of the i-th subsequence.

[0065] In one embodiment, time-series features Inputting the data into an LSTM network yields multi-level sequence vectors. The LSTM network processing procedure is as follows: The LSTM network first processes... The corresponding output vector 1 is generated through internal calculation, and the internal memory state of the LSTM network is updated simultaneously before processing. At this point, the LSTM network will combine the current content... The LSTM network then performs a combined calculation with the internal memory state retained from the previous step to obtain output vector 2. Finally, the LSTM network outputs a new matrix consisting of two 512-dimensional vectors. The first vector is... The result after LSTM processing is the second vector. The result after LSTM processing and incorporation of preceding context information.

[0066] The aforementioned preprocessing of text sequences (such as word segmentation, embedding, and sequence modeling) can effectively capture surface semantic information. However, for data with inherent hierarchical dependencies, such as code and molecular structures, linear sequences alone are insufficient to fully represent their complex logic. Therefore, we further introduce structural sequences for analysis. By constructing structural sequence vectors and adjacency matrices, we explicitly model the topological relationships between elements, thereby enhancing the model's ability to understand grammatical rules or functional logic.

[0067] In one embodiment, a global self-attention network is used to extract global information from the structure sequence vector N to obtain global features G. This is first achieved through three learnable weight matrices. , and Each node vector in the structure sequence vector N is converted into a query vector Q, a key vector K, and a value vector V. The conversion method is the same as the word embedding vector conversion method described above, and will not be repeated here. The query vector Q, key vector K, and value vector V used subsequently are the same, and will not be repeated here.

[0068] By calculating the self-attention GAN(N) of the structure sequence vector N, the global feature G is obtained. The self-global feature G is an 8×512 matrix, where each row of the matrix represents a node. The new semantics of each node fuses the information of all other nodes. The formula for obtaining the self-attention GAN(N) is as follows: GAN(N) = softmax( V in, Let K be the dimension of the key vector.

[0069] In one embodiment, based on a global self-attention network, the adjacency matrix A is used to limit the attention range, and the attention SiSAN(N) is calculated to obtain the structural features S. The structural features S is an 8×512 matrix, where each row represents a node, and each node only incorporates information from its direct neighbors. Attention only occurs when nodes are connected; therefore, when calculating attention, only positions where the adjacency matrix A is 1 are considered. The formula for calculating the attention SiSAN(N) is as follows: SiSAN(N) = softmax( V, Where A is the adjacency matrix. Let K be the dimension of the key vector.

[0070] In one embodiment, a sliding window with a window size of 3 is used to calculate local attention. In this case, each node only focuses on its immediate and adjacent nodes, where "immediately adjacent" refers to sequential order. For example, for node i, only nodes i-1, i, and i+1 are considered. The attention LSAN(N) is calculated using these three nodes, yielding local features L. Local features L is an 8×512 matrix, where each row represents a node. Each node incorporates information from its immediate and adjacent nodes. The formula for calculating self-attention LSAN(N) is as follows: LSAN(N) = softmax( V in, Let K be the dimension of the key vector. Let i be a mask matrix with a size of 1×3. When initializing the mask matrix, set the position of node i and its adjacent nodes before and after it to 1, and set the other positions to 0.

[0071] In one embodiment, the three attention outputs are added bitwise to obtain X = G + S + L, where X is an 8×512 matrix. The three attention outputs are then fused using a feed-forward neural network (FFN) to obtain a multi-view structure vector. Multi-view structure vector It is an 8×512 matrix. The formula for FFN to process the three attention outputs is as follows: F(X) = ReLU(X) )

[0072] in, and The parameters represent the learning parameters, b1 and b2 represent the bias terms, and ReLU(X) represents the non-linear activation function, ReLU(X) = max(0,X). In this embodiment, It can be a matrix with a shape of 512×2048. It can be a 2048-dimensional vector; It can be a matrix with a shape of 2048×512. It can be a 512-dimensional vector. , b1 and b2 can be adjusted according to the actual situation, and the above examples do not limit this embodiment.

[0073] In one embodiment, due to multi-level sequence vectors It is a 2×512 matrix, a multi-view structure vector. It is an 8×512 matrix, a multi-level sequence vector. and multi-view structure vector Since the lengths are inconsistent in the first dimension, global average pooling needs to be used to process the multi-level sequence vectors first. and multi-view structure vector Adjust them separately, transforming the multi-level sequence vectors and multi-view structure vector This transforms the vectors into vectors with the same length in the first dimension. The specific process of global average pooling is as follows: for multi-level sequence vectors... The average of the two vectors is taken to obtain a vector of size 1×512; for multi-view structure vectors The average of the eight vectors is taken to obtain a vector of size 1×512. Then, according to the multi-view structure vector... Prior to this, multi-level sequence vectors The concatenation of these vectors in sequence yields the concatenated vector Fusioncode, which has 1024 dimensions.

[0074] After obtaining the concatenated vector Fusioncode, the decoder is used to decode the concatenated vector to obtain the code comments. The concatenated vector Fusioncode is used as the initial input to the decoder, and comments are generated autoregressively. Specifically, the input is a start identifier. <bos>The decoder outputs the annotation probability distribution and the corresponding hidden state information of the first time step, and selects the word with the highest probability in the annotation probability distribution (such as "This") as the output. It then uses "This" and the corresponding hidden state information of the first time step as the input for the next time step, continuing the generation process until an end identifier is generated. <eos>Table 1 shows the decoder inference process.

[0075] Table 1 Decoder Inference Process

[0076] Before using the decoder in the inference process, it needs to be trained to obtain a decoder that can generate high-quality code. During decoder training, a target annotation sequence needs to be initialized. The loss is calculated using the target annotation sequence and the code annotation sequence generated by the decoder. This process is repeated through backpropagation to modify the decoder's parameters and optimize its generation performance.

[0077] The initialization process for the target annotation sequence is as follows: Add special symbols: <bos>This function addstwo numbers <eos>Each word (including) <bos>and <eos>The seven words are mapped into a 512-dimensional vector through word embedding, forming the target annotation sequence Y∈R^{7×512}, where the target annotation sequence Y is a matrix of shape 7×512.

[0078] Example 3: Based on the code annotation generation method based on a fusion encoder provided in the foregoing embodiments, the present invention also provides an apparatus for generating code annotations based on a fusion encoder that can be used to implement the above method, such as... Figure 5 The diagram shown is a schematic representation of the device architecture according to an embodiment of the present invention. The device for generating code comments based on a fusion encoder in this embodiment includes one or more processors 21 and a memory 22. Wherein, Figure 5 Take a processor 21 as an example.

[0079] Processor 21 and memory 22 can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0080] The memory 22, as a non-volatile computer-readable storage medium for code annotation generation based on a fusion encoder, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the code annotation generation method based on a fusion encoder in the foregoing embodiments. The processor 21 executes various functional applications and data processing of the code annotation generation device based on a fusion encoder by running the non-volatile software programs, instructions, and modules stored in the memory 22, thereby implementing the code annotation generation method based on a fusion encoder in the foregoing embodiments.

[0081] Memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 22 may optionally include memory remotely located relative to processor 21, which can be connected to processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0082] The program instructions / modules are stored in memory 22. When executed by one or more processors 21, they execute the code annotation generation method based on the fusion encoder described in the foregoing embodiments, for example, executing the above-described... Figures 1-3 The steps shown.

[0083] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 5 One of the processors 21 can enable the one or more processors to execute the code annotation generation method based on the fusion encoder in the foregoing embodiments, for example, to execute the above-described... Figures 1-3 The steps shown.

[0084] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device and system are based on the same concept as the processing method embodiment of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0085] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0086] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.< / eos> < / bos> < / eos> < / bos> < / eos> < / bos> < / eos> < / bos> < / bos> < / bos>

Claims

1. A code annotation generation method based on a fusion encoder, characterized in that, include: Obtain the text sequence vector, the structure sequence vector, and the adjacency matrix of the code; Based on the text sequence vector, a multi-level sequence vector is obtained; Based on the structure sequence vector and the adjacency matrix, a multi-view structure vector is obtained; By concatenating the multi-level sequence vector and the multi-view structure vector, a concatenated vector is obtained; The concatenated vector is decoded to generate code comments.

2. The code annotation generation method based on a fusion encoder according to claim 1, characterized in that, The method further includes: Obtain the code sequence and divide the code sequence into multiple subsequences; Each subsequence is represented as a fixed-size word embedding matrix, resulting in a text sequence vector comprising multiple word embedding matrices; Obtain the abstract syntax tree of the code, wherein the abstract syntax tree includes multiple child nodes; Each child node is represented as a word embedding vector of a fixed size, resulting in the structure sequence vector that includes multiple word embedding vectors; The adjacency matrix is ​​obtained based on the abstract syntax tree.

3. The code annotation generation method based on a fusion encoder according to claim 2, characterized in that, The adjacency matrix takes values ​​of 0 and 1, and the method further includes: Parse the abstract syntax tree to obtain the edge relationships between each child node; If there are edges between child nodes, then set the corresponding position in the adjacency matrix to 1; If there are no edges between child nodes, the corresponding position in the adjacency matrix is ​​set to 0.

4. The code annotation generation method based on a fusion encoder according to claim 1, characterized in that, The method further includes: Based on the word embedding matrix in the text sequence vector, the corresponding position encoding matrix is ​​obtained; the word embedding matrix and the position encoding matrix are fused to obtain the position sequence vector; The semantic information of the lexical elements and the dependencies between the lexical elements in the position sequence vector are extracted to obtain the lexical semantic features; Extract the sentence semantic information from the lexical semantic features to obtain the sentence semantic features; Extract the temporal information relationship of the semantic features of the sentence to obtain temporal features; The temporal features are processed to obtain the multi-level sequence vector.

5. The code annotation generation method based on a fusion encoder according to claim 4, characterized in that, The method further includes: Determine the column index of the embedding element in each word embedding matrix; If the column index is even, the column index is normalized to obtain a position factor, and the sine function value corresponding to the position factor is calculated. The sine function value is used as the position encoding information corresponding to the embedded element. If the column index is odd, the column index is normalized to obtain a position factor, and the cosine function value corresponding to the position factor is calculated. The cosine function value is used as the position encoding information corresponding to the embedded element. The position encoding matrix is ​​obtained by integrating the position encoding information corresponding to all embedded elements.

6. The code annotation generation method based on a fusion encoder according to claim 1, characterized in that, The method further includes: Global information is extracted from the structure sequence vector to obtain global features; Extract the structural information from the adjacency matrix to obtain structural features; Local information is extracted from the structural sequence vector to obtain local features; The multi-view structural vector is obtained by fusing the global features, the structural features, and the local features.

7. The code annotation generation method based on a fusion encoder according to claim 1, characterized in that, The method further includes: Based on the starting identifier and the concatenation vector, the annotation probability distribution of the first time step and the hidden state information corresponding to the first time step are generated. The annotation with the highest probability value in the annotation probability distribution of the first time step is taken as the target annotation of the first time step; Based on the target annotation and the hidden state information corresponding to the first time step, the annotation probability distribution and the hidden state information corresponding to the second time step are generated. The annotation with the highest probability value in the annotation probability distribution of the second time step is taken as the target annotation of the second time step. This process continues until the generated target annotation becomes the end identifier. The target annotations generated in all time steps are concatenated in order to obtain the code annotations corresponding to the code sequence.

8. The code annotation generation method based on a fusion encoder according to claim 1, characterized in that, The method further includes: Adjust the multi-level sequence vector and the multi-view structure vector respectively, so that the adjusted multi-level sequence vector and the adjusted multi-view structure vector have the same length in the first dimension; The adjusted multi-level sequence vector is concatenated along the second dimension to the adjusted multi-view structure vector to obtain the concatenated vector.

9. A code annotation generation device based on a fusion encoder, characterized in that, The device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the code annotation generation method based on a fusion encoder as described in any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions, which are executed by one or more processors to perform the code annotation generation method based on a fusion encoder as described in any one of claims 1-8.

Images