Code abstract generation model based on control flow diagram and multi-modal attention mechanism

Abstract

The invention provides a code abstract generation model based on a control flow graph and a multi-modal attention mechanism, which comprises a code vocabulary sequence encoder for encoding a vocabulary sequence of a code by using a Transform architecture so as to capture vocabulary and syntactic information of the code; the control flow graph encoder is used for encoding a control flow graph (CFG) by adopting a graph attention network (GAT) and extracting control flow information of a code so as to capture a macroscopic execution path and logic characteristics of the code; and the joint decoder is used for fusing the output of the code vocabulary sequence encoder and the output of the control flow graph encoder, and generating a natural language abstract conforming to code semantics and execution logic by utilizing a multi-head attention mechanism. According to the method, the control flow graph CFG is introduced, the actual execution path of the code is captured, and the code vocabulary sequence is combined, so that the logic flow and the control relation of the code can be effectively extracted, and the generated abstract better conforms to the intention of the code.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic code summarization, and in particular to a code summary generation model based on a control flow graph and a multimodal attention mechanism. Background Art

[0002] In the process of software development and maintenance, Automatic Code Summarization technology can automatically generate concise natural language descriptions of code snippets, helping developers quickly understand the logic and functions of the code and significantly improve development efficiency. At present, traditional methods for generating code summaries are mostly based on Abstract Syntax Tree (AST) and Information Retrieval (IR) technology. These methods usually generate descriptions by decomposing the grammatical structure of the code and extracting key nodes and features. However, when parsing the code, AST generates a large number of fine-grained grammatical nodes, making the structure too complex and redundant, making it difficult to effectively capture the macro-semantic features of the code, especially in codes involving complex control flows, where AST's expressiveness is obviously insufficient.

[0003] In recent years, deep learning models have been widely used in automatic code summary generation. Typical deep learning code summary generation methods use the encoder-decoder architecture. This architecture initially achieved remarkable results in machine translation tasks and has been gradually applied to code summary generation. The encoder part converts the code into semantic embeddings, and the decoder translates the embeddings into natural language descriptions. However, due to problems such as encoding complexity, node redundancy, and insufficient interpretability of generated logic, deep learning models based on AST still face problems of low generation quality and low efficiency in practice. Summary of the invention

[0004] In view of the defects in the prior art, the purpose of the present invention is to provide a code summary generation model based on control flow graph and multimodal attention mechanism.

[0005] The code summary generation model based on control flow graph and multimodal attention mechanism provided by the present invention includes:

[0006] Code word sequence encoder: Encodes the code word sequence using the Transformer architecture to capture the code's lexical and syntactic information;

[0007] Control flow graph encoder: Use graph attention network GAT to encode the control flow graph CFG and extract the control flow information of the code to capture the macro execution path and logical characteristics of the code;

[0008] Joint Decoder: Fusion of the outputs of the code vocabulary sequence encoder and the control flow graph encoder, using a multi-head attention mechanism to generate a natural language summary that conforms to the code semantics and execution logic.

[0009] Preferably, the code vocabulary sequence encoder comprises:

[0010] Preprocess the code snippets to segment the code strings into independent vocabulary units, and further segment the words named in camel case and snake case;

[0011] The preprocessed code sequence X = {x1, x2, ..., x t} Input to the Word2Vec embedding layer, each vocabulary unit x i Converted to a fixed-dimensional embedding vector This generates the initial embedding matrix of the code word sequence Where F represents the dimension of the embedding vector, 1<<i<<t, t is the number of vocabulary units;

[0012] Add position encoding to each word embedding vector to ensure that the model captures the order of words. Position encoding uses sine and cosine functions to generate values ​​of different frequencies, which are added to the word embedding to generate an encoding with order information. The expression is:

[0013]

[0014] Among them, pos is the position of the word, i is the dimension index of the position encoding, and d is the embedding dimension;

[0015] The position-encoded word embedding vector is input into a multi-layer self-attention mechanism. The self-attention mechanism generates an attention weight matrix by calculating the relevance of each word with other words, so that the model dynamically focuses on the information in the input sequence.

[0016] After multi-layer Transformer encoding, the code word sequence encoder finally generates the encoded embedding vector sequence Used in the subsequent decoder generation process.

[0017] Preferably, the calculation process of the self-attention mechanism includes:

[0018] Embed each word into a vector Generate the corresponding query Query, key Key and value Value vector through linear transformation;

[0019] The attention weight between each pair of words is calculated by the Softmax function, expressed as:

[0020]

[0021] Among them, d k is the dimension of the Key vector, which is used for scaling to stabilize the attention weight;

[0022] The self-attention operation is calculated in parallel through multiple heads, each head focuses on different feature dimensions, and finally the outputs of all heads are concatenated and linearly transformed to enhance the model's ability to capture multi-level information.

[0023] Preferably, the output of the multi-head attention layer is passed through a feedforward neural network FFN, which includes two fully connected layers and an activation function. The feedforward network acts independently on each vocabulary embedding vector and introduces nonlinear features to enhance the representation capability. The expression of FFN is:

[0024] FFN(x)=max(0,xW1+b1)W2+b2

[0025] Among them, W1 and W2 are weights, and b1 and b2 are bias parameters.

[0026] Preferably, after each sublayer of the Transformer encoder, residual connections and layer normalization are used to stabilize the training process of the model. Through the residual connection, the input is directly added to the output, and then layer normalization is performed to reduce the risk of gradient vanishing and speed up convergence.

[0027] Preferably, the control flow graph encoder comprises:

[0028] Parse the code snippet into a control flow graph CFG, where each node of the CFG represents a code statement and the edge of the CFG represents the control flow path of the code;

[0029] The words in each sentence are segmented, and the segmented words are processed to generate a word vector sequence through Word2Vec, which is then input into the bidirectional long short-term memory network Bi-LSTM to capture the sequential dependency and contextual information within the sentence. The output of Bi-LSTM generates a fixed-length semantic embedding vector v for each node. ni , as the initial embedding of the code statement;

[0030] In CFG, natural sequence edges NS, True edges, and False edges are included to represent the natural execution order of statements and the control flow of branches. Marked nodes are added to CFG, and the marked nodes generate corresponding vocabulary embeddings to connect the corresponding path information in the control flow.

[0031] After generating the initial embedding of the nodes, the control flow graph encoder uses the graph attention network GAT to encode the CFG, aggregate the information of adjacent nodes layer by layer, and obtain the structural features and semantic information of the CFG;

[0032] After multiple layers of GAT encoding, the control flow graph encoder outputs the final embedding vector set for each node Each of these Represents the node embedding after multiple layers of semantic aggregation and structural aggregation.

[0033] Preferably, the control flow graph encoder uses the graph attention network GAT to encode the CFG, including:

[0034] Each node calculates the attention weight according to the characteristics of its neighboring nodes, and the expression is:

[0035]

[0036] Among them, α ij represents the attention weight of node i and its neighbor node j, W is the linear transformation matrix, a is the attention vector, is the set of neighbor nodes of node i, ∥ represents the vector concatenation operation; the LeakyReLU activation function is used to control the gradient flow and ensure the stability of weight calculation;

[0037] The GAT encoder uses a multi-head attention mechanism, which processes multiple sets of weight calculations in parallel. By focusing on different feature spaces, the model aggregates neighbor information in different semantic dimensions. The output of the multi-head attention mechanism concatenates the embeddings of multiple heads and integrates them through a linear layer. The calculation formula is as follows:

[0038]

[0039] Among them, K represents the number of attention heads, and W k are the weight and linear transformation matrix of the kth attention head respectively, σ is the activation function, and the concatenated result is used as the final embedding of the node.

[0040] Preferably, after each layer of GAT is calculated, residual connections and layer normalization are used on the generated node embeddings to prevent gradient vanishing and accelerate convergence; residual connections add the input and output of the node, and then layer normalization is applied to ensure the numerical stability of the embedding;

[0041] To further capture high-order structural information and semantic relationships in the control flow graph, the control flow graph encoder stacks multiple layers of GAT so that the node embeddings aggregate more distant neighbor information layer by layer.

[0042] Preferably, the joint decoder comprises:

[0043] When generating a summary, the joint decoder starts from a preset start tag and gradually generates words until the end tag is generated. Each generated word is first converted into a vector representation of fixed dimension through word embedding, and position encoding is added to retain the order information in the generation process. Sine and cosine functions are used to embed the order information of the words into the word embedding.

[0044] The generated word sequence is input into the multi-head self-attention layer to capture the contextual relationship of the generated words. During the generation process, the self-attention mechanism calculates the relevance of each word with other words in the sequence and generates an attention weight matrix to ensure that each generated word is consistent in the context. The expression is:

[0045]

[0046] Among them, Q, K, and V are query, key, and value matrices respectively;

[0047] The joint decoder uses a masking mechanism that ensures the consistency of the decoding process by masking the words after the current word in the attention weight matrix so that the model can only see the previous words in the generation path.

[0048] Preferably, the joint decoder aligns the outputs of the code vocabulary sequence encoder and the control flow graph encoder with the hidden state of the decoder respectively through two parallel multimodal attention layers, and enables the decoder to capture vocabulary and control flow information at the same time through the multimodal attention mechanism, specifically:

[0049] The word sequence attention layer receives the output V' of the code word sequence encoder X , the attention calculation is performed on the encoded word sequence through the query matrix Q and the key matrix K. Each hidden state in the decoder generates a query vector, which is dot-producted with the encoding vector of the word sequence encoder to generate attention weights, thereby capturing the key words that need to be paid attention to when generating summaries, ensuring that the decoder dynamically pays attention to the core word features in the code during the generation process;

[0050] The control flow graph attention layer receives the output V' of the control flow graph encoder N , and uses a similar attention mechanism to calculate the correlation between the decoder's hidden state and the control flow graph nodes. The control flow graph attention layer calculates the attention weight of each node to enable the decoder to identify the key control flow structure in the code and ensure that the generated summary conforms to the logical order of the code;

[0051] Both attention layers use a multi-head attention mechanism, which enables the decoder to focus on the vocabulary and control flow information of the code from different feature dimensions. The multi-head mechanism calculates in parallel through multiple attention heads. Finally, the outputs of all heads are spliced ​​together and linearly transformed to form a unified encoding input for decoding;

[0052] The outputs of the vocabulary sequence attention layer and the control flow graph attention layer are concatenated in the channel dimension to form a joint feature representation, which is converted into the probability distribution of the target vocabulary through the linear layer. The calculation process of the linear layer is:

[0053] P(y t )=softmax(W o [Attention]; b o )

[0054] Among them, W o and b o are the weights and biases of the linear transformation, [∵;] represents the concatenation operation, y t is the probability distribution of words generated at the tth time step;

[0055] The joint decoder usually adopts a teacher forcing strategy in the training phase, that is, the target vocabulary of each step is used as the input of the next step; in the inference phase, the decoder starts from the preset start token and generates the vocabulary of each time step through greedy search or beam search strategy until the end token is generated;

[0056] The goal of the joint decoder is to generate a summary that matches the code function. The loss function uses cross entropy loss to measure the difference between the generated vocabulary sequence and the target summary. The loss function is defined as:

[0057]

[0058] Where T represents the length of the target summary, P(y t ∣y <t ,V' X ,V' N ) is the conditional probability of each word generated by the model. By minimizing the loss function, the model learns a generation strategy that is closer to the target summary.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] (1) The present invention captures the actual execution path of the code by introducing the control flow graph (CFG). Combined with the code vocabulary sequence, it can effectively extract the logical flow and control relationship of the code, making the generated summary more consistent with the intention of the code. In practical applications, the control flow graph encoder of the CoSS model helps the model focus on the statement-level structure and control flow characteristics of the code, thereby improving the accuracy and practicality of the generated summary.

[0061] (2) The model of the present invention reduces the interference of redundant syntax nodes in the processing process by using the control flow graph CFG to replace the AST structure, which can significantly reduce the consumption of computing resources. At the same time, through the parallel design of dual encoders (code vocabulary sequence encoder and control flow graph encoder), the CoSS model can process and generate high-quality summaries in a shorter time, thereby better meeting the needs of rapid understanding and real-time iteration in actual development.

[0062] (3) The model of the present invention uses a control flow graph (CFG) to represent the control flow of the code, and aggregates the information of adjacent nodes layer by layer through the graph attention network (GAT) in the control flow graph encoder, thereby enhancing the model's ability to capture global semantic and structural information. This method enables the CoSS model to maintain semantic continuity during the processing of long code segments, and clearly convey the complete functionality of the code, providing developers with more accurate summary information.

[0063] (4) The control flow graph encoder of the model of the present invention directly encodes the control flow path of the code, which can accurately capture the logical flow of the code. In addition, the CoSS model introduces a multimodal attention mechanism in the joint decoder, so that when generating summaries, it can pay attention to both vocabulary features and control flow information, ensuring the clarity and logical consistency of the summary structure. Through this design, the generated summary has higher interpretability and can more intuitively help developers understand the logical path and intention of code execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0065] Figure 1 Schematic diagram of a code vocabulary sequence encoder and a control flow graph encoder;

[0066] Figure 2 is a schematic diagram of a joint decoder;

[0067] Figure 3 Generate schematic diagrams for node embeddings. DETAILED DESCRIPTION

[0068] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0069] Example

[0070] This paper proposes a code summary generation model CoSS (Code Summarization with Statement Semantics) based on control flow graph (CFG) and multimodal attention mechanism. The model adopts a dual encoder structure, in which one encoder encodes the vocabulary sequence of the code based on Transformer to capture the lexical features of the code; the other encoder encodes the control flow graph based on the graph attention network (GAT) to capture the control flow information of the code. Finally, the joint decoder generates high-quality code summaries through a multi-head attention mechanism. The following is the specific technical process and structure of the solution.

[0071] 1. Overall structure of the model

[0072] The overall structure of the CoSS model is as follows Figure 1 and Figure 2 The model consists of three parts: code vocabulary sequence encoder (Transformer-based Encoder), control flow graph encoder (GAT-based Encoder) and joint decoder:

[0073] Code Lexical Sequence Encoder: Encodes the code lexical sequence using the Transformer architecture to capture the lexical and syntactic information of the code.

[0074] Control Flow Graph Encoder: A Graph Attention Network (GAT) is used to encode the Control Flow Graph (CFG) to obtain the control flow information of the code, especially control structures such as branches and loops.

[0075] Joint Decoder: Fusion of the outputs of the two encoders, using a multi-head attention mechanism to generate a natural language summary that is consistent with the code semantics and execution logic.

[0076] 2. Model composition description

[0077] 2.1. Code Vocabulary Sequence Encoder

[0078] The core task of the code vocabulary sequence encoder is to extract the vocabulary and grammatical features of the code, laying the foundation for generating high-quality code summaries. The encoder uses a multi-layer structure based on Transformer to fully capture the semantics and sequential relationships in the code snippets. The detailed processing flow is as follows:

[0079] 2.1.1. Code preprocessing and vocabulary segmentation

[0080] At the input stage, the code snippet is first preprocessed to split the code string into independent vocabulary units, called "tokens". These vocabulary units include identifiers (such as variable names, function names), operators, keywords, and special symbols (such as brackets, quotation marks, etc.) in the code. In addition, words named in camelCase and snake_case are further split, for example, helloWorld or hello_world is split into two words, hello and world. The purpose of this processing is to ensure that the model can learn the semantic features of each word more carefully and reduce the interference of redundant information.

[0081] 2.1.2. Vocabulary Embedding Generation

[0082] The code sequence after word segmentation is X = {x1, x2, ..., x t} is input to the Word2Vec embedding layer. Each vocabulary unit x i Converted to a fixed-dimensional embedding vector This generates the initial embedding matrix of the code word sequence Where F represents the dimension of the embedding vector. This step enables the model to learn based on semantic relationships in the vector space by mapping each word into a vector.

[0083] 2.1.3. Positional Encoding

[0084] The Transformer model does not directly consider the order information of the input vocabulary, so position encoding is added to each vocabulary embedding vector to ensure that the model can capture the order relationship of the vocabulary. Position encoding uses sine and cosine functions to generate values ​​of different frequencies, which are added to the vocabulary embedding to generate encoding with order information. The formula is as follows:

[0085]

[0086] Among them, pos is the position of the word, i is the dimension index of the position encoding, and d is the embedding dimension. This method allows the model to perceive the relative position and order of words.

[0087] 2.1.4. Multi-Head Attention Layer

[0088] The word embedding vector sequence V after position encoding X Input to the multi-layer self-attention mechanism. The self-attention mechanism generates an attention weight matrix by calculating the relevance of each word with other words, allowing the model to dynamically focus on important information in the input sequence.

[0089] The calculation process of the self-attention mechanism includes the following steps:

[0090] Query, Key, and Value vectors: embedding vectors for each word Generate the corresponding Query (Q), Key (K) and Value (V) vectors through linear transformation.

[0091] Attention weight calculation: The attention weight between each pair of words is calculated by the Softmax function, and the formula is:

[0092]

[0093] Among them, d k is the dimension of the Key vector, which is used for scaling to make the attention weight stable.

[0094] Multi-head mechanism: The self-attention operation is calculated in parallel through multiple heads. Each head focuses on different feature dimensions. Finally, the outputs of all heads are concatenated and linearly transformed, which enhances the model's ability to capture multi-level information.

[0095] 2.1.5. Feed-Forward Network (FFN)

[0096] The output of the multi-head attention layer passes through a feedforward neural network (FFN), which consists of two fully connected layers and an activation function (usually ReLU). The feedforward network acts on each word embedding vector independently and introduces nonlinear features to enhance representation capabilities. The calculation formula of FFN is as follows:

[0097] FFN(x)=max(0,xW1+b1)W2+b2

[0098] Among them, W1, W2, b1, b2 are weight and bias parameters.

[0099] 2.1.6.Add&Norm layer

[0100] After each sublayer of the Transformer encoder (such as the self-attention layer and the feed-forward layer), residual connections and layer normalization are used to stabilize the model training process. Through residual connections, the input is directly added to the output, and then layer normalization is performed to reduce the risk of gradient vanishing and speed up convergence.

[0101] 2.1.7. Output Encoding

[0102] After multi-layer Transformer encoding, the code word sequence encoder finally generates the encoded embedding vector sequence Used in the subsequent decoder generation process. These encoding vectors contain the lexical and grammatical features of the code snippet, as well as the long-range dependencies between words, laying the foundation for generating a logical summary.

[0103] Through the above steps, the code vocabulary sequence encoder can effectively capture the lexical semantics, syntactic structure and long-range dependencies in the code, providing rich information support for the joint decoder to generate accurate code summaries.

[0104] 2.2. Control Flow Graph Encoder

[0105] The control flow graph encoder is designed to extract the control flow information of the code to capture the macro execution path and logical characteristics of the code. This patented method combines the control flow graph (CFG) with the graph attention network (GAT) to obtain the control flow embedding of the code by aggregating the graph structure layer by layer. The following is the detailed process:

[0106] 2.2.1. Control flow graph generation and node representation

[0107] 2.2.1.1. Control flow graph generation:

[0108] The code snippet is first parsed into a control flow graph (CFG). Each node of the CFG represents a code statement, and the edge represents the control flow path of the code (such as a conditional branch or loop). The structure of the CFG enables the model to intuitively capture the code execution path and logical flow, especially in the branch and loop structures, where the CFG provides clear node and path relationships. For example, a conditional statement generates two child nodes representing the True and False branches, respectively, while the back edge of the control flow in the loop structure represents a repeated execution path.

[0109] 2.2.1.2.Node embedding generation:

[0110] Each CFG node represents a code statement. In order to generate the initial semantic embedding vector of the node, the words in each statement are first segmented. The segmented words are generated into a word vector sequence through Word2Vec and input into a bidirectional long short-term memory network (Bi-LSTM) to capture the sequential dependency and context information within the sentence. The output of Bi-LSTM generates a fixed-length semantic embedding vector v for each node. ni , which represents the initial embedding of code statements, such as Figure 3 shown.

[0111] 2.2.1.3. Special node identification:

[0112] In CFG, there are three special types of edges: natural sequence edges (NS), True edges, and False edges, which are used to represent the natural execution order of statements and the control flow of branches. To handle these special edges, CFG adds marked nodes, such as the True node marked as <t>, False nodes are marked as <f>, and introduce an entry node <e>To mark the beginning of the code. These marker nodes generate corresponding word embeddings that are used to connect specific path information in the control flow.

[0113] 2.2.2. Graph Attention Network (GAT) Encoding

[0114] After generating the initial embedding of the node, the control flow graph encoder uses the Graph Attention Network (GAT) to encode the CFG, aggregate the information of adjacent nodes layer by layer, and obtain the structural features and semantic information of the CFG. The specific steps are as follows:

[0115] 2.2.2.1 Neighbor Node Aggregation

[0116] Each node calculates the attention weight based on the characteristics of its neighbor nodes. The attention weight reflects the importance of different neighbor nodes to the current node. For example, the True and False neighbor nodes of a branch statement have different weights because they represent different control flow paths. GAT uses the self-attention mechanism to calculate the weights of adjacent nodes, enabling the model to dynamically adjust nodes based on the contextual features of their neighbors. The calculation formula for attention weight is:

[0117]

[0118] Among them, α ij represents the attention weight of node i and its neighbor node j, W is the linear transformation matrix, a is the attention vector, is the set of neighbor nodes of node i, and ∥ represents a vector concatenation operation. The LeakyReLU activation function is used to control the gradient flow and ensure the stability of weight calculation.

[0119] 2.2.2.2. Multi-head attention mechanism

[0120] To improve the expressiveness and stability of the model, the GAT encoder uses a multi-head attention mechanism. Multi-head attention processes multiple sets of weight calculations in parallel, and each set of attention heads focuses on different feature spaces, enabling the model to aggregate neighbor information in different semantic dimensions. The output of the multi-head attention concatenates the embeddings of multiple heads and integrates them through a linear layer. The calculation formula is as follows:

[0121]

[0122] Among them, K represents the number of attention heads, and W k are the weight and linear transformation matrix of the kth attention head respectively, σ is the activation function (usually ELU), and the concatenated result is used as the final embedding of the node.

[0123] 2.2.2.3. Layer Normalization and Residual Connections

[0124] After each layer of GAT is calculated, residual connections and layer normalization are used on the generated node embeddings to prevent gradient vanishing and accelerate convergence. Specifically, the residual connection adds the input and output of the node, and then layer normalization is applied to ensure the numerical stability of the embedding.

[0125] 2.2.2.4. Multi-layer GAT stacking

[0126] In order to further capture the high-level structural information and semantic relationships in the control flow graph, the control flow graph encoder stacks multiple layers of GAT so that the embedding of nodes can aggregate more distant neighbor information layer by layer. The design of multi-layer GAT enables each node to not only obtain the semantic information of direct neighbors, but also gradually obtain the global context information on the path, thereby enhancing the model's ability to express complex control flow structures.

[0127] 2.2.2.5 Final Output of Control Flow Graph Encoder

[0128] After multiple layers of GAT encoding, the control flow graph encoder outputs the final embedding vector set for each node Each of these Represents the node embedding after multiple layers of semantic aggregation and structural aggregation. These embedding vectors contain the control flow information of the code, which makes it easier for the joint decoder to focus on the logical path and macro structure of the code when generating summaries.

[0129] Through the above steps, the control flow graph encoder can effectively express the control flow characteristics of the code and the logical relationship between statements, making the generated code summary more accurate and logically consistent, and improving the interpretability of the code.

[0130] 2.3 Joint Decoder

[0131] The joint decoder is a key module of this patent model, which is responsible for integrating the output information of the code vocabulary sequence encoder and the control flow graph encoder to generate a natural language summary that conforms to the code logic. The decoder dynamically pays attention to the vocabulary features and control flow characteristics of the code through a multimodal attention mechanism to ensure the accuracy and coherence of the generated summary. The following is the detailed process of the joint decoder and the technical details of each component.

[0132] 2.3.1. Input Preprocessing and Multi-Head Self-Attention Layer

[0133] 2.3.1.1. Vocabulary Embedding and Positional Encoding:

[0134] The joint decoder generates the summary from a special start marker <sos>Start and gradually generate vocabulary until the end tag is generated <eos>Each generated word is first converted into a fixed-dimensional vector representation by word embedding (Word2Vec), and position encoding is added to preserve the order information during the generation process. This position encoding method is similar to the position encoding in the encoder, using sine and cosine functions to embed the order information of the word into the word embedding.

[0135] 2.3.1.2. Multi-head self-attention layer:

[0136] The generated word sequence is input to the multi-head self-attention layer to capture the contextual relationship of the generated words. During the generation process, the self-attention mechanism calculates the correlation between each word and other words in the sequence to generate an attention weight matrix to ensure that each generated word is consistent in the context. This mechanism helps the model dynamically pay attention to previously generated words, making the context of the summary more coherent. The specific formula is:

[0137]

[0138] Among them, Q, K, and V are query, key, and value matrices respectively.

[0139] 2.3.1.3. Mask mechanism:

[0140] To avoid information leakage, that is, to ensure that the word generation at each position depends only on the current word and its previous words, the joint decoder uses a masking mechanism. The masking mechanism masks the words after the current word in the attention weight matrix so that the model can only see the previous words in the generation path, thereby ensuring the consistency of the decoding process.

[0141] 2.3.2. Encoder-Decoder Multimodal Attention Layer

[0142] The joint decoder aligns the outputs of the code vocabulary sequence encoder and the control flow graph encoder with the hidden states of the decoder respectively through two parallel multimodal attention layers. The design of the multimodal attention mechanism enables the decoder to capture both vocabulary and control flow information, enhancing the accuracy of the generated summary. Specifically:

[0143] 2.3.2.1. Vocabulary Sequence Attention Layer:

[0144] The word sequence attention layer receives the output V' of the code word sequence encoder X , the attention calculation is performed on the encoded word sequence through the query matrix Q and the key matrix K. Each hidden state in the decoder generates a query vector, which is dot-producted with the encoding vector of the word sequence encoder to generate attention weights, thereby capturing the key words that need to be paid attention to when generating summaries. This mechanism ensures that the decoder dynamically pays attention to the core word features in the code during the generation process.

[0145] 2.3.2.2. Control Flow Graph Attention Layer

[0146] The control flow graph attention layer receives the output V' of the control flow graph encoder N , and uses a similar attention mechanism to calculate the correlation between the decoder's hidden state and the control flow graph nodes. The control flow graph attention layer calculates the attention weight of each node, enabling the decoder to identify key control flow structures in the code (such as conditional statements, loops, etc.) and ensure that the generated summary conforms to the logical order of the code. For example, for a code snippet containing a branch structure, the decoder will calculate different weights for the True and False paths to more accurately describe the branch logic.

[0147] 2.3.2.3. Multi-head mechanism

[0148] Both attention layers use a multi-head attention mechanism, which enables the decoder to focus on the vocabulary and control flow information of the code from different feature dimensions. The multi-head mechanism provides rich semantic information for each decoding step through parallel calculation of multiple attention heads, enhancing the expressiveness of the generated summary. Finally, the outputs of all heads are concatenated and linearly transformed to form a unified encoding input for decoding.

[0149] 2.3.3 Joint decoding and generation process

[0150] 2.3.3.1. Output concatenation and linear layer

[0151] The outputs of the word sequence attention layer and the control flow graph attention layer are concatenated in the channel dimension to form a joint feature representation. This joint representation is converted into the probability distribution of the target word through the linear layer. The calculation process of the linear layer is:

[0152] P(y t )=softmax(W o [Attention]; b o )

[0153] Among them, W o and b o are the weights and biases of the linear transformation, [∵;] represents the concatenation operation, y t is the probability distribution of the vocabulary generated at the tth time step.

[0154] 2.3.3.2. Generation strategy

[0155] The joint decoder usually adopts a teacher forcing strategy during the training phase, that is, the target vocabulary of each step is used as the input of the next step to improve training efficiency and generation quality. In the inference phase, the decoder starts from the start tag 〈S05> and generates the vocabulary of each time step through a greedy search or beam search strategy until the end tag 〈EOS〉 is generated. Greedy search selects the vocabulary with the highest probability in each step, while beam search selects from multiple possible paths to ensure the best quality of the generated summary.

[0156] 2.3.3.3. Loss Function and Training Objective

[0157] The goal of the joint decoder is to generate a summary that matches the code function. The loss function uses cross entropy loss to measure the difference between the generated word sequence and the target summary. The loss function is defined as:

[0158]

[0159] Where T represents the length of the target summary, P(y t ∣y <t ,V' X ,V' N ) is the conditional probability of each word generated by the model. By minimizing the loss function, the model can learn a generation strategy that is closer to the target summary.

[0160] 2.3.3.4. Decoding output and summary generation

[0161] After the above steps, the joint decoder gradually generates a natural language code summary, ensuring that the summary can accurately and coherently describe the function and execution logic of the code snippet. Through the multimodal attention mechanism of the joint decoder, the generated summary not only contains the key words in the code, but also faithfully reflects the control flow path of the code, making the generated summary more logical and interpretable. This method makes the generated code summary highly practical in development scenarios, making it easier for developers to quickly understand the code logic.

[0162] The present invention also has the following alternatives:

[0163] 1. Alternatives based on Abstract Syntax Tree (AST)

[0164] Solution description: The abstract syntax tree (AST) can be used to replace the control flow graph (CFG) as the structural representation of the code. The grammatical structure of the code is captured through the nodes of the AST, and an encoder based on RNN or Transformer is used for serialization encoding to generate a code summary.

[0165] 2. Alternatives based on Code Property Graph (CPG)

[0166] Solution description: Code Property Graph (CPG) can combine the characteristics of AST, Data Flow Graph (DFG) and Control Flow Graph (CFG) to represent multiple information of code in a graph structure. CPG encoder can use graph convolutional network (GCN) or graph attention network (GAT) for encoding to capture the syntax, control flow and data flow information of the code.

[0167] 3. Alternatives based on Data Flow Graph (DFG)

[0168] Solution description: Use data flow graphs to represent data dependencies in the code, and use variables and data dependencies in the code snippet as nodes and edges to generate summaries.

[0169] 4. Alternatives based on pre-trained models (such as CodeBERT, GPT)

[0170] Solution description: Use pre-trained models such as CodeBERT or GPT to pre-train on a large number of code-text pairs, and then generate natural language summaries for specific code snippets. This method can capture the semantic features of the code through the Transformer structure.

[0171] Those skilled in the art know that, in addition to implementing the system, device and its various modules provided by the present invention in a purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and its various modules provided by the present invention can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing the method and structures within the hardware component.

[0172] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.< / eos> < / sos> < / e> < / f> < / t>

Claims

1. A code summary generation model based on control flow graph and multimodal attention mechanism, characterized in that: include: Code word sequence encoder: Encodes the code word sequence using the Transformer architecture to capture the code's lexical and syntactic information; Control flow graph encoder: Use graph attention network GAT to encode the control flow graph CFG and extract the control flow information of the code to capture the macro execution path and logical characteristics of the code; Joint Decoder: Fusion of the outputs of the code vocabulary sequence encoder and the control flow graph encoder, using a multi-head attention mechanism to generate a natural language summary that conforms to the code semantics and execution logic.

2. The code summary generation model based on control flow graph and multimodal attention mechanism according to claim 1, characterized in that: The code vocabulary sequence encoder comprises: Preprocess the code snippets to segment the code strings into independent vocabulary units, and further segment the words named in camel case and snake case; The pre - processed code sequence X = {x1, x2, …, x t} is input into the Word2Vec embedding layer, and each lexical unit x i is converted into an embedding vector of a fixed dimension to generate the initial embedding matrix of the code vocabulary sequence where F represents the dimension of the embedding vector, 1 << i << t, and t is the number of lexical units; Add position encoding to each word embedding vector to ensure that the model captures the order of words. Position encoding uses sine and cosine functions to generate values ​​of different frequencies, which are added to the word embedding to generate an encoding with order information. The expression is: Among them, pos is the position of the word, i is the dimension index of the position encoding, and d is the embedding dimension; The position-encoded word embedding vector is input into a multi-layer self-attention mechanism. The self-attention mechanism generates an attention weight matrix by calculating the relevance of each word with other words, so that the model dynamically focuses on the information in the input sequence. After multi-layer Transformer encoding, the code word sequence encoder finally generates the encoded embedding vector sequence Used in the subsequent decoder generation process.

3. The code summary generation model based on control flow graph and multimodal attention mechanism according to claim 2 is characterized in that: The calculation process of the self-attention mechanism includes: Embed each word into a vector Generate the corresponding query Query, key Key and value Value vector through linear transformation; The attention weight between each pair of words is calculated by the Softmax function, expressed as: Among them, d k is the dimension of the Key vector, which is used for scaling to stabilize the attention weight; The self-attention operation is calculated in parallel through multiple heads, each head focuses on different feature dimensions, and finally the outputs of all heads are concatenated and linearly transformed to enhance the model's ability to capture multi-level information.

4. The code summary generation model based on control flow graph and multimodal attention mechanism according to claim 3 is characterized in that: The output of the multi-head attention layer is passed through the feedforward neural network FFN. The feedforward neural network FFN consists of two fully connected layers and an activation function. The feedforward network acts independently on each word embedding vector and introduces nonlinear features to enhance the representation ability. The expression of FFN is: FFN(x)=max(0,xW1+b1)W2+b2 Among them, W1 and W2 are weights, and b1 and b2 are bias parameters.

5. The code summary generation model based on control flow graph and multimodal attention mechanism according to claim 4, characterized in that: After each sublayer of the Transformer encoder, residual connections and layer normalization are used to stabilize the training process of the model. Through the residual connection, the input is directly added to the output, and then layer normalization is performed to reduce the risk of gradient vanishing and speed up convergence.

6. The code summary generation model based on control flow graph and multimodal attention mechanism according to claim 5, characterized in that: The control flow graph encoder comprises: Parse the code snippet into a control flow graph CFG, where each node of the CFG represents a code statement and the edge of the CFG represents the control flow path of the code; The words in each sentence are segmented, and the segmented words are used to generate a word vector sequence through Word2Vec, which is then input into the bidirectional long short-term memory network Bi-LSTM to capture the sequential dependency and contextual information within the sentence. The output of Bi-LSTM generates a fixed-length semantic embedding vector for each node. As an initial embedding of a code statement; In CFG, natural sequence edges NS, True edges, and False edges are included to represent the natural execution order of statements and the control flow of branches. Marked nodes are added to CFG, and the marked nodes generate corresponding vocabulary embeddings to connect the corresponding path information in the control flow. After generating the initial embedding of the nodes, the control flow graph encoder uses the graph attention network GAT to encode the CFG, aggregate the information of adjacent nodes layer by layer, and obtain the structural features and semantic information of the CFG; After multiple layers of GAT encoding, the control flow graph encoder outputs the final embedding vector set for each node Each of these Represents the node embedding after multiple layers of semantic aggregation and structural aggregation.

7. The code summary generation model based on control flow graph and multimodal attention mechanism according to claim 6, characterized in that: The control flow graph encoder uses the graph attention network GAT to encode CFG including: Each node calculates the attention weight according to the characteristics of its neighboring nodes, and the expression is: Among them, α ij represents the attention weight of node i and its neighbor node j, W is the linear transformation matrix, a is the attention vector, is the set of neighbor nodes of node i, ∥ represents the vector concatenation operation; the LeakyReLU activation function is used to control the gradient flow and ensure the stability of weight calculation; The GAT encoder uses a multi-head attention mechanism, which processes multiple sets of weight calculations in parallel. By focusing on different feature spaces, the model aggregates neighbor information in different semantic dimensions. The output of the multi-head attention mechanism concatenates the embeddings of multiple heads and integrates them through a linear layer. The calculation formula is as follows: Among them, K represents the number of attention heads, and W k are the weight and linear transformation matrix of the kth attention head respectively, σ is the activation function, and the concatenated result is used as the final embedding of the node.

8. The code summary generation model based on control flow graph and multimodal attention mechanism according to claim 7, characterized in that: After each layer of GAT is calculated, residual connections and layer normalization are used on the generated node embeddings to prevent gradient vanishing and accelerate convergence; residual connections add the input and output of the node, and then layer normalization is applied to ensure the numerical stability of the embedding; To further capture high-order structural information and semantic relationships in the control flow graph, the control flow graph encoder stacks multiple layers of GAT so that the node embeddings aggregate more distant neighbor information layer by layer.

9. The code summary generation model based on control flow graph and multimodal attention mechanism according to claim 8, characterized in that: The joint decoder comprises: When generating a summary, the joint decoder starts from a preset start tag and gradually generates words until the end tag is generated. Each generated word is first converted into a vector representation of fixed dimension through word embedding, and position encoding is added to retain the order information in the generation process. Sine and cosine functions are used to embed the order information of the words into the word embedding. The generated word sequence is input into the multi-head self-attention layer to capture the contextual relationship of the generated words. During the generation process, the self-attention mechanism calculates the relevance of each word with other words in the sequence and generates an attention weight matrix to ensure that each generated word is consistent in the context. The expression is: Among them, Q, K, and V are query, key, and value matrices respectively; The joint decoder uses a masking mechanism that ensures the consistency of the decoding process by masking the words after the current word in the attention weight matrix so that the model can only see the previous words in the generation path.

10. The code summary generation model based on control flow graph and multimodal attention mechanism according to claim 9, characterized in that: The joint decoder aligns the outputs of the code vocabulary sequence encoder and the control flow graph encoder with the hidden state of the decoder through two parallel multimodal attention layers. The multimodal attention mechanism enables the decoder to capture both vocabulary and control flow information at the same time. Specifically: The word sequence attention layer receives the output V' of the code word sequence encoder X , the attention calculation is performed on the encoded word sequence through the query matrix Q and the key matrix K. Each hidden state in the decoder generates a query vector, which is dot-producted with the encoding vector of the word sequence encoder to generate attention weights, thereby capturing the key words that need to be paid attention to when generating summaries, ensuring that the decoder dynamically pays attention to the core word features in the code during the generation process; The control flow graph attention layer receives the output V' of the control flow graph encoder N , and uses a similar attention mechanism to calculate the correlation between the decoder's hidden state and the control flow graph nodes. The control flow graph attention layer calculates the attention weight of each node to enable the decoder to identify the key control flow structure in the code and ensure that the generated summary conforms to the logical order of the code; Both attention layers use a multi-head attention mechanism, which enables the decoder to focus on the vocabulary and control flow information of the code from different feature dimensions. The multi-head mechanism calculates in parallel through multiple attention heads. Finally, the outputs of all heads are spliced ​​together and linearly transformed to form a unified encoding input for decoding; The outputs of the vocabulary sequence attention layer and the control flow graph attention layer are concatenated in the channel dimension to form a joint feature representation, which is converted into the probability distribution of the target vocabulary through the linear layer. The calculation process of the linear layer is: P(y t )=softmax(W o [Attention];b o ) Among them, W o and b o are the weights and biases of the linear transformation, [∵;] represents the concatenation operation, y t is the probability distribution of words generated at the tth time step; The joint decoder usually adopts a teacher forcing strategy in the training phase, that is, the target vocabulary of each step is used as the input of the next step; in the inference phase, the decoder starts from the preset start token and generates the vocabulary of each time step through greedy search or beam search strategy until the end token is generated; The goal of the joint decoder is to generate a summary that matches the code function. The loss function uses cross entropy loss to measure the difference between the generated vocabulary sequence and the target summary. The loss function is defined as: Where T represents the length of the target summary, P(y t ∣y <t ,V' X ,V' N ) is the conditional probability of each word generated by the model. By minimizing the loss function, the model learns a generation strategy that is closer to the target summary.

Images