Code abstract generation method oriented to semantic structure feature joint enhancement

Abstract

The invention discloses a code abstract generation method oriented to semantic structure feature joint enhancement. According to the method, an encoder-decoder architecture is followed, and a multi-form combined semantic feature enhancement method is adopted to automatically generate a code abstract. Wherein the encoder learns code semantics at token and statement levels, and the decoder receives code embedding and generates a code abstract in combination with an attention mechanism. Unlike a single tree or graph coding process, the method uses graph representation to enhance the interval divided tree structure. The joint decoder aggregates the text sequence information and the structure sequence information containing the location node features, and predicts and generates a model understood code summary text. According to the method, the problems of repetition and redundancy of code texts with different lengths caused by disordered code text features are relieved. In a relatively complex code text data set, a word ambiguity phenomenon caused by insufficient knowledge information understanding is reduced, and a code abstract with relatively high quality is generated. Meanwhile, the method has good convergence performance.

Description

Technical Field

[0001] The invention relates to a code summary generation method oriented to the joint enhancement of semantic structure features, and belongs to the technical field of software engineering. Background Art

[0002] As an emerging technology in the field of artificial intelligence, code intelligence is committed to understanding, generating and optimizing code through machine learning, deep learning, natural language processing and knowledge mining. The core of code intelligence application is code representation and understanding. Converting source code into a form that can be processed by machines is the main process of code representation and understanding. Effective code representation and understanding can provide developers with a higher level of abstraction and a more convenient development method in practical application scenarios such as code generation, code summary, and code quality assessment. Among them, code summary is one of the core downstream tasks of code intelligence that is closely related to code representation and understanding. Unlike code comments, code summary is not a descriptive text added by humans, but a summary information that is independent of the original code and automatically generated by an algorithm. This technology of compressing source code into a short and clear summary can provide great application value for code maintenance, code search, automatic document generation and software engineering research. In this process, the information used by common code representation models includes code token sequence, abstract syntax tree (AST), data flow graph (DFG) and control flow graph (CFG).

[0003] At present, the use of feature representation methods to obtain semantic information or structural information corresponding to code snippets of different lengths is prone to feature disorder problems, making the text generated by the summary document redundant and the functional description inaccurate. At the same time, code snippets at different maintenance stages involve different prior knowledge backgrounds or different development languages. The currently generated automatic summaries have the problem of text comprehension ambiguity, which makes the character interpretability of the summary text poor, the context relevance low, and the understanding of proprietary knowledge terms inaccurate. Therefore, it is necessary to propose an improved code summarization method for specific scenarios, while paying attention to the structural features of the source code, retaining the semantic information of the code text to the greatest extent, thereby improving the performance of code representation and understanding, and ultimately improving the generation quality and efficiency of the code summary text. Summary of the invention

[0004] Purpose of the invention: In view of the problems and shortcomings in the prior art, the present invention proposes a code summary optimization method with joint enhancement of semantic structural features to solve the repetition and redundancy problems of code texts of different lengths caused by feature disorder. The specific process requires that the code snippet be represented as AST, and then the AST is preprocessed and finally feature fusion is performed. The preprocessing will divide the AST into many subtrees using the interval segmentation method, add corresponding structural feature weights to each subtree and converge them into the AST total tree. Among them, the structural feature weights are calculated by the graph convolutional neural network (GCN) of the input tree structure information. By adding feature weights, the defect of the tree structure focusing on the feature is compensated. The GCN input features do not need to be reconstructed again, but an adjacency matrix is ​​constructed based on the total tree features and subtree features processed in the AST. In the code feature representation stage, the advantages of obtaining weights by integrating the graph structure relationship features are improved to improve the utilization efficiency of the structural features of the tree representation process. In the encoding stage, the processed semantic features of the code snippet are further used, and the cross-attention mechanism is used to fuse the optimized structural features and semantic features to generate a higher quality code summary.

[0005] Technical solution: A code summary generation method for joint enhancement of semantic structural features. The present invention uses two types of information streams: code token sequence features and AST features represented by code text. In particular, a new optimization representation method is designed for AST features. In the AST representation stage, the AST of the code text is segmented into AST intervals using a method that aggregates local features and traces global structures. Then, the corresponding adjacency matrix is ​​constructed based on the AST features. The feature weights corresponding to the AST intervals are obtained through the graph structure to optimize the AST feature representation. Then, a specially customized joint decoder is used to accept the two types of features, semantic embedding and AST structural embedding. The code summary text is generated with the help of the feature-improved attention mechanism. The method comprises the following steps:

[0006] Step 1: Obtain the code text for which summary needs to be generated based on the public dataset of the programming website, and divide the public dataset into a training set and a validation set according to a set ratio. According to the vector sequence encoded by the known source code text, convert the known source code into a text vector sequence.

[0007] Step 2: Perform AST tree modeling on the code text for which summary is to be generated. Obtain the modeling representation of AST and corresponding nodes. Use a method of aggregating local features and tracing global structure to segment the AST interval and input it into the neural network model to obtain the initial structural feature vector embedding.

[0008] Step 3: Construct an adjacency matrix based on the subtree vector embedding and the total tree vector embedding, and obtain the corresponding graph embedding vector after processing by the graph convolutional neural network (GCN). Then input a convolution-based self-attention model to obtain the structural feature embedding vector that enhances the AST feature weight.

[0009] Step 4: The code text is represented by the vector obtained by the character encoding method, and the semantic feature embedding of the code text is obtained by using the Transformer-based multi-head attention neural network model. The calculation process of semantic feature embedding is independent of the calculation process of structural feature vector embedding.

[0010] Step 5: Input the code text embedding, structural feature embedding vector, and semantic feature embedding vector into a joint decoder using a double cross attention mechanism. The processing of the structural feature embedding vector and the semantic feature embedding vector in the decoder can be compared.

[0011] Step 6: Use the cross entropy function as the loss function, input the training set into the deep learning model, and train the deep learning model. When the loss function reaches the minimum value or the number of iterations is met, the pre-trained deep learning model is obtained. The stacked decoder is further combined with the replication mechanism to improve efficiency, and the gradient descent algorithm is used to update the parameters in the model and speed up the calculation to minimize the loss function and finally obtain the generated code summary.

[0012] In the step 1, according to the vector sequence corresponding to the summary of the known source code, the summary of the known source code is converted into a target vector sequence using One-Hot vector encoding.

[0013] In step 2, the modeled AST tree structure uses a depth-first traversal method to process the entire tree. The segmentation rule of the AST interval is to insert a placeholder node that acts as an encapsulation when encountering a composite structure that generates a new branch (such as a for loop structure or an if conditional judgment structure). With the encapsulation node as a marker, the composite structure is traversed internally to form a subtree. The external total tree adds the processed subtree semantics. After being processed by the recursive subtree method, the complete AST structure is divided into several local interval subtrees and a complete encapsulation semantic total tree that retains local structural information. The interval-divided AST subtree is vector-encoded, and the node vector representation in the AST subtree is input into the recursive neural network, and the forward propagation of the subtree encoding is performed to calculate and obtain the hidden state of the node as:

[0014]

[0015] Where W A and W B is a learnable weight matrix, Gh(v i ) are the hidden state, token embedding and node v respectively i For a special leaf node v i , equal

[0016] All nodes of the calculation are aggregated through pooling operation, and the maximum pooling is used as the aggregation standard to generate the embedding vector of the aggregated subtree. Then the global structure embedding is added to the embedding vector of the pooled aggregated subtree. The subtree structure relationship of different partition intervals in the AST total tree embedding is processed by a second recursive neural network to obtain the optimized tree structure embedding vector of each subtree interval in the global structure of the AST total tree.

[0017] In step 3, the subtree vector embedding and the total tree vector embedding are respectively input into a GCN module with a three-layer structure. The propagation method between layers of a GCN is:

[0018]

[0019] In the formula, I is the identity matrix, is the sequence feature of the current layer, σ is the nonlinear activation function, W (l) as a weight matrix.

[0020] The graph embedding vectors of the subtree and the total tree are then input into a convolution-based self-attention model. The model calculation result is an embedding vector containing the weights of the subtree and the total tree enhanced AST structure, expressed as:

[0021]

[0022] In the formula, Q T Represents the convolution result query of the total tree, K T Represents the convolution result key of the total tree, V T Represents the convolution result value of the subtree.

[0023] The enhanced AST structure embedding vector and the optimized tree structure embedding vector are subjected to a block-level self-attention and normalized to obtain the improved AST embedding vector as the structural feature embedding vector, expressed as:

[0024]

[0025] In step 4, the code text uses One-Hot to generate the initial vector representation of the characters: The corresponding token sequence is o=(o1, ..., o n ). The text is then fed into a Transformer with multi-head attention to get the structural feature vector embedding. Through the Transformer’s multi-head attention mechanism, the code token sequence o is processed into an output vector u = (u1, ..., u n). The calculation process of one of the Transformer heads is:

[0026]

[0027]

[0028] In the formula, W q , W k , W v is a trainable matrix for query, key, and value; d k is the dimension of query and key; and is the relative position of position n and position m.

[0029] In step 5, a mask is added to the code text embedding and self-attention is used as the output query Q. The structural feature embedding vector and the semantic feature embedding are respectively input into the corresponding cross attention model, and finally the decoding feature is obtained through the double cross attention model. The calculation process is:

[0030]

[0031]

[0032]

[0033] In the formula, and are the trainable projection matrices for queries, keys, and values, and i is the number of encoding layer features.

[0034] In step 6, the stacked decoder further combines the replication mechanism to improve efficiency, and uses the gradient descent algorithm to update the parameters in the model and speed up the calculation of the minimization loss function to finally obtain the generated code summary result and evaluate the quality. The loss function is expressed as:

[0035]

[0036] Where N is the total number of training data, n is the length of each target summary, is the jth word in the i-th sentence, is the probability of generating the jth word. Finally, the neural network model that uses both structural features and semantic features can generate high-quality code summary content based on the code text.

[0037] A computer device, characterized in that: the computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the code summary generation method for joint enhancement of semantic structure features as described in any one of claims 1 to 8 is implemented.

[0038] A computer-readable storage medium, characterized in that: the computer-readable storage medium stores a code summary generation method for semantic structure feature joint enhancement according to any one of claims 1 to 7.

[0039] Beneficial effects: Compared with the prior art, the code summary generation method for semantic structure feature joint enhancement provided by the present invention has the following advantages:

[0040] (1) A feature joint representation method is used to learn the semantic features and AST structural features of the code text, which alleviates the problems of short text repetition and long text redundancy in the automatic code summary generated during software development and maintenance, and improves the readability of the code summary text.

[0041] (2) Use graph neural networks to optimize the structural feature extraction process. Construct an adjacency matrix for the recursively divided AST structural features and assign corresponding feature weights, so that the constructed AST structural features can simultaneously consider the overall and local structural features of the total tree and subtrees, solving the problem of disordered structural feature representation.

[0042] (3) The structural features and semantic features of the input are processed by feature decoding fusion, which alleviates the problem of incomplete description of the functional information of the code summary text and ambiguous understanding of the text information caused by the single serialization structure of the code token.

[0043] (4) The present invention alleviates the problem of repeated redundancy of code texts of different lengths in code summaries and the ambiguity problem of poor character interpretability of code summaries. At the same time, code summaries are generated on a public Java dataset, and have high code summary text quality and good convergence performance. This method has good generalization ability in extracting summaries in actual code business scenarios, and has high code summary generation quality and good convergence performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0045] Figure 2 The source code of the embodiment of the present invention;

[0046] Figure 3 The process of obtaining text embedding vectors for the One-Hot method;

[0047] Figure 4 Get the embedding hidden state process for the abstract syntax tree structure;

[0048] Figure 5 It is a schematic diagram of an overview of the abstract syntax tree enhancement structure of an embodiment of the invention;

[0049] Figure 6 It is a graph of the result of enhancing the interval partitioning of the abstract syntax tree structure of the embodiment of the invention;

[0050] Figure 7 The overall schematic diagram of graph neural network enhanced structural feature extraction;

[0051] Figure 8 This is the fusion decoding flowchart of the Transformer decoder;

[0052] Fig. 9 A schematic diagram of prediction results generated according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The present invention is further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0054] like Figure 1 As shown, the code summary generation method for semantic structure feature joint enhancement includes the following steps:

[0055] Step 1: Based on the GitHub open source data, a Java public dataset with known code summaries is constructed, and the Java public dataset is divided into a training set and a validation set according to a set ratio. Figure 2 Take the public void retrieveWaypoints source code in the source code as an example to convert the internal source code into a vector sequence. Taking "public" as an example, the encoding result is a vector containing the word "public", and the length of the vector is equal to the size of the vocabulary. Use One-Hot vector encoding to convert the summary of the known source code into a target vector sequence. Figure 3 In an exemplary manner, the vector sequence corresponding to the summary of the known source code is encoded using a One-Hot encoding method to encode the target code summary, and each word contained in the corpus is encoded as a one-dimensional One-Hot vector to form a word pair, and then the words contained in the current target code summary are replaced with the One-Hot vector to form a source code text vector sequence.

[0056] Step 2: If Figure 4As shown, the AST tree modeling is performed on the public void retrieveWaypoints source code. The node embedding vector generated according to the code call relationship and the embedding matrix is ​​D=(D1, ..., D n ). First, use the JDT compiler to extract syntax and generate the abstract syntax tree AST of the code snippet. Figure 3 The One-Hot encoding method in the abstract syntax tree AST is used to represent the node method name in a one-dimensional vector. The length of the vector corresponding to each node is consistent with the number of nodes in the AST tree N. Construct an embedding matrix and save the AST serialization data in pkl format. The nodes in the abstract syntax tree AST are represented by features. There are N nodes. Assuming that each node has F features, an embedding matrix of size F×N can be formed; the embedding matrix is ​​multiplied by the node vector representation after One-Hot encoding to obtain the node initialization vector representation; that is, assuming that the feature matrix of a node is F×N and the One-Hot vector is N×1, the initialization vector representation of the node is F×1. After obtaining the node embedding vector generated by the embedding matrix, a method of aggregating local features and tracing the global structure is used to split the represented AST interval and input it into the neural network model to obtain the preliminary structural feature vector embedding, where the input dimension is [512,512]. Use a tree optimization strategy for feature enhancement, as follows:

[0057] The modeled AST tree structure uses a depth-first traversal method to process the entire tree T n .like Figure 5 As shown in the figure, the segmentation rule of the AST interval is to insert a placeholder node that acts as an encapsulation when encountering a composite structure that generates a new branch (such as a for loop structure or an if conditional judgment structure). With the encapsulation node as a marker, the composite structure is traversed internally to form a subtree. Figure 6 As shown, the external total tree adds the processed subtree semantics. After being processed by the recursive subtree method, the complete AST structure is divided into several local interval subtrees T subs and a complete encapsulation semantic tree T that retains local structural information total The public void retrieveWaypoints source code AST subtree of the interval segmentation is vector-encoded, and the node vector representation in the AST subtree is input into the recursive neural network. The subtree encoding is forward propagated, and the hidden state of the node is calculated and represented as:

[0058]

[0059] Where W A and W B is a learnable weight matrix, Gh(vi ) are the hidden state, token embedding and node v respectively i For a special leaf node v i , equal Taking the subtree segmentation process of public void retrieveWaypoints as an example, calculate the hidden states of all subtree nodes in the segmentation interval All nodes D are aggregated through pooling operation, the maximum pooling is used as the aggregation standard, and the embedding vector T of the aggregated subtree is generated. v The formula is as follows:

[0060]

[0061] Then the pooled aggregate subtree embedding vector T v Add global structure embedding. Perform a second recursive neural network process on the subtree structure relationship of different partition intervals in the AST total tree embedding to obtain the optimized tree structure embedding vector of the global structure of each subtree interval in the AST total tree. The formula is as follows:

[0062]

[0063] The retrieveWaypoints source code Total tree hidden state added to T v The features are aggregated again to obtain a syntax tree representation T with global features z

[0064] Step 3: If Figure 7 As shown, according to the retrieveWaypoints subtree vector embedding T v With the total tree vector embedding T z The adjacency matrix is ​​constructed and processed by a graph convolutional neural network (GCN) to obtain the corresponding graph embedding vector. Then a convolution-based self-attention model is input to obtain the structural feature embedding vector that enhances the AST feature weight, and the corresponding output vector dimension is [30, 128, 1, 1].

[0065] In step 2, the subtree vector is embedded into T v The total tree vector embedding is input into a three-layer GCN module. The propagation method between layers of one GCN is:

[0066]

[0067] In the formula, I is the identity matrix, is the sequence feature of the current layer, σ is the nonlinear activation function, W (l) As the weight matrix. The processed subtree vector embedding is expressed as The total tree vector embedding is represented as The graph embedding vectors of the subtree and the total tree are then input into a convolution-based self-attention model. The model calculation result is an embedding vector containing the weights of the subtree and the total tree enhanced AST structure, expressed as:

[0068]

[0069] In the formula, Q T Represents the convolution result query of the total tree, K T Represents the convolution result key of the total tree, V T The enhanced AST structure embedding vector and the optimized tree structure embedding vector are subjected to a block-level self-attention and normalized to obtain the improved AST embedding vector as the structural feature embedding vector F T , expressed as:

[0070]

[0071] Step 4: After obtaining the vector initial representation of the code text using the character encoding method, the Transformer-based multi-head attention neural network model is used to obtain the semantic feature embedding of the code text. The calculation process of semantic feature embedding is independent of the calculation process of structural feature vector embedding. In step 1, the token sequence c of the code text is converted into (c1, ..., c n ), and input it into the Transformer with multi-head attention to get the structural feature vector embedding of the text. Through the multi-head attention mechanism of the Transformer, the code token sequence c is processed into the output vector U = (u1, ..., u n ). The calculation process of one of the Transformer heads is:

[0072]

[0073]

[0074] In the formula, W q , W k , W v is a trainable matrix for query, key, and value; d k is the dimension of query and key; and is the relative position of position n and position m.

[0075] Step 5: Figure 8 As shown, the AST structural features are embedded in F T , the semantic feature embedding U is input into a joint decoder using a double cross attention mechanism, and the final structure vector embedding is G d , the final semantic vector is embedded as S u The processing of the structural feature embedding vector and the semantic feature embedding vector in the decoder can be compared. In step 5, a mask is added to the code text embedding and self-attention is used as the output query Q. The structural feature embedding vector and the semantic feature embedding are respectively input into the corresponding cross attention model, and finally the decoding feature is obtained through the double cross attention model. The calculation process is:

[0076]

[0077]

[0078]

[0079] In the formula, and is the trainable projection matrix for query, key and value, i is the number of encoding layer features. and Finally, the vectorized representation of the prediction result is obtained through back propagation

[0080] Step 6: Use the cross entropy function as the loss function, input the training set into the deep learning model, train the deep learning model, and when the loss function reaches the minimum value, the pre-trained deep learning model is obtained.

[0081] In step 6, the stacked decoder further combines a replication mechanism to improve efficiency, and either satisfies the iterative use of a gradient descent algorithm to update the parameters in the model and speed up the calculation of the minimization loss function to finally obtain the generated code summary result and evaluate the quality. The loss function is expressed as:

[0082]

[0083] Where N is the total number of training data, n is the length of each target summary, is the jth word in the i-th sentence, is the probability of generating the jth word. Fig. 9As shown, the neural network model that uses both structural features and semantic features is able to generate high-quality code summary content based on the code text and generate prediction results that retrieves waypoint matching the given query from the database.

[0084] The present invention uses a Java public data set for training on a programming website with research value, and uses a feature joint representation method to learn the semantic features and AST structural features of the code text. First, in the AST representation stage, the AST of the code text is segmented using a method that aggregates local features and traces the global structure. Thereafter, a corresponding adjacency matrix is ​​constructed based on the AST features, and then the structural feature representation is optimized based on the graph structure. Then, a joint decoding model is used to process semantic embedding and structural embedding, and a code summary text is generated with the help of a feature-improved attention mechanism. The present invention solves the problems of text redundancy and poor character interpretability in the automatic generation of code summaries in software development and maintenance. At the same time, it alleviates the current situation that the maintenance summary text automatically generated during the software maintenance process has incomplete functional information description and ambiguous understanding of text information. It has good generalization ability in actual scenarios, and has high code summary generation quality and good convergence performance.

[0085] Obviously, those skilled in the art should understand that the various steps of the code summary generation method for semantic structure feature joint enhancement of the above-mentioned embodiment of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

Claims

1. A code summary generation method for joint enhancement of semantic structure features, characterized in that: Obtain the code text for which summary is to be generated based on the public data set of programming websites. Extract the structural feature embedding vector and semantic feature embedding vector from the code to be summarized; input the structural feature embedding vector and semantic feature embedding vector into the pre-trained deep learning model to output the corresponding code.

2. The method according to claim 1, characterized in that Extract the structural embedding vector, aggregate the syntax representation vectors of all nodes, and obtain the embedding vector of the improved abstract syntax tree (AST). The steps of extracting the embedding vector of the improved AST include: Step 1: Perform AST tree modeling on the code for which summary needs to be generated, model the nodes in the tree modeling, and obtain the modeling representation of AST and corresponding nodes; Step 2: Use depth-first traversal to process the entire AST tree modeled in step 1. Based on the tree splitting and aggregation rules, process the subtree information separately and retain the position structure information of the subtree in the original AST; Step 3: The AST processing process in step 2 is to insert a placeholder node that acts as an encapsulation when encountering a composite structure that generates a new branch (such as a for loop structure or an if conditional judgment structure). Using the encapsulation node as a marker, the composite structure is traversed internally to form a subtree. The external total tree adds the processed subtree semantics. After being processed by the recursive subtree method, the complete AST structure is divided into several local interval subtrees and a complete encapsulation semantic total tree that retains local structure information; Step 4: Vectorize the modeled partitioned AST tree to obtain the node embedding vector of each node, including: vectorize node A, input the node vector representation in the improved AST tree into the recursive neural network, perform forward propagation of the subtree encoding, and calculate and obtain the hidden state representation of the node as follows: Where W A and W B is a learnable weight matrix, Gh(v i ) are the hidden state, token embedding and node v respectively i For a special leaf node v i , equal Step 5: Aggregate the hidden states of all nodes in step 4 through pooling operations, use the maximum pooling as the aggregation criterion, and generate the embedding vector of the aggregated subtree; Step 6: Add global structural embedding to the embedding vector of the aggregated subtree in step 5. The specific process is: perform a second recursive neural network processing on the subtree structural relationship of different partition intervals in the partition AST total tree embedding to obtain the optimized tree structure embedding vector of the global structure of each subtree interval in the AST total tree.

3. The method according to claim 2, characterized in that: The subtree sequence information and the total tree structure sequence information are extracted as the reconstructed adjacency matrix, and the AST structure weight is enhanced with the help of the adjacency matrix. The corresponding graph embedding vector is obtained after being processed by the graph convolutional neural network (GCN). The specific process is: the embedding vector of the partitioned subtree and the embedding vector of the total tree are respectively input into a GCN module with a three-layer structure. The propagation method between layers in one GCN is: In the formula, I is the identity matrix, is the sequence feature of the current layer, σ is the nonlinear activation function, W (l) as a weight matrix.

4. The method according to claim 3, characterized in that: The graph embedding vectors of the subtree and the total tree are input into a convolution-based self-attention model. The model calculation result is an embedding vector containing the weights of the subtree and the total tree enhanced AST structure, expressed as: In the formula, Q T Represents the convolution result query of the total tree, K T Represents the convolution result key of the total tree, V T Represents the convolution result value of the subtree.

5. The method according to claim 1, characterized in that: The enhanced AST structure embedding vector and the optimized tree structure embedding vector are subjected to a block-level self-attention and normalized to obtain the improved AST embedding vector as the structural feature embedding vector. The calculation result F T It is expressed as:

6. The method according to claim 1, characterized in that: Use One-Hot to generate the initial vector representation of the characters in the code text: The corresponding token sequence is o=(o1, ..., o n ). The text is then fed into a Transformer with multi-head attention to get the structural feature vector embedding. Through the Transformer’s multi-head attention mechanism, the code token sequence o is processed into an output vector u = (u1, ..., u n ). The calculation process of one of the Transformer heads is: in W q , W k , W v is a trainable matrix for query, key, and value; d k is the dimension of query and key; and is the relative position of position n and position m.

7. The method according to claim 1, characterized in that The code text embedding, structural feature embedding vector, and semantic feature embedding vector are input into a joint decoder using a double cross attention mechanism. The structural feature embedding vector and the semantic feature embedding vector can be contrasted and expressed as: in and is a trainable projection matrix for query, key, and value. i is the number of features in the encoding layer. The cross entropy loss function is used to input the training set into the deep learning model to train the deep learning model. When the loss function reaches the minimum value or meets the number of iterations, the pre-trained deep learning model is obtained.

8. According to the method of claim 1, the stacked decoders are further combined with a replication mechanism to improve efficiency, and a gradient descent algorithm is used to update the parameters in the model and speed up the calculation of the minimum loss function to finally obtain the generated code summary, and the loss function is expressed as: in, N is the total number of training data, n is the length of each target summary, is the jth word in the i-th sentence, is the probability of generating the jth word.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the code summary generation method for joint enhancement of semantic structure features according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a code summary generation method for semantic structure feature joint enhancement according to any one of claims 1 to 7.

Images