A code digest generation method and apparatus

Abstract

The application discloses the technical field of code summary generation, and particularly relates to a code summary generation method and device, which comprises the following specific steps: S1: preprocessing function source code and its corresponding code summary to obtain a multi-modal function code summary dataset; S2: transforming FS-AST in any triple to obtain a first node feature embedding matrix and an adjacency matrix; S3: designing a two-stage fine-tuning method for a code summary generation task, taking X0 and E as inputs, fine-tuning an original CodePTMs model to obtain a fine-tuned prefix vector code summary model CodePTMs' model, and outputting a code summary; and S4: dividing the multi-modal function code summary dataset and using the same to train the CodePTMs' model. The two-stage fine-tuning method for the code summary generation task can improve the deep semantic modeling capability of the model for the code, thereby improving the accuracy of the model in generating the code summary.

Description

Technical Field

[0001] This invention relates to the field of code digest generation technology, specifically to a code digest generation method and apparatus. Background Technology

[0002] Code summarization is a task that automatically generates natural language text describing the functionality of a given function's source code. In code summarization, the code is often converted into an Abstract Syntax Tree (AST) as an intermediate representation. However, for complex program fragments, the resulting AST is typically large and deep. Directly using such an AST for model training can easily lead to the loss of contextual information during node transfer; moreover, the sparse tree structure of the AST makes it difficult for the model to fully capture the semantic relationships between nodes, resulting in inefficient use of code structure information during training. This makes it difficult for the model to comprehensively learn the syntactic and semantic features of the code, reducing the quality and accuracy of the generated summary.

[0003] Large-scale pre-trained code models (CodePTMs) are a class of models specifically designed for programming language tasks. They accumulate rich understanding capabilities through massive amounts of code data and specific training tasks. In the following discussion, large-scale pre-trained code models (CodePTMs) will be referred to simply as CodePTMs models. For example, the CodeBert model is one type of CodePTMs model. Existing CodePTMs models have not been specifically trained for AST-related tasks and cannot understand the tree-structured code representation of ASTs. This limitation restricts the ability of CodePTMs models to model deep semantics of code, especially when dealing with complex code scenarios.

[0004] Fine-tuning large models refers to adjusting some parameters in a pre-trained model or introducing a set of learnable parameters to improve the model's performance in specific scenarios by adjusting some parameters in the pre-trained model for a specific task. However, when only one-stage fine-tuning introduces prior knowledge of tree-structured code representation into CodePTMs models that have not undergone training tasks related to tree-structured code representation, it may disrupt the general knowledge system already possessed by the CodePTMs model. Especially when task data is limited, the model may overfit to specific task data, leading to unstable performance in other scenarios. At the same time, one-stage fine-tuning may also make it difficult to achieve a reasonable weight allocation between the tree-structured code modality and the code text sequence modality, resulting in poor information fusion between the two modalities and further reducing the overall performance of the model.

[0005] Based on the above, the current method has the following drawbacks:

[0006] 1. The source code of functions used for code summarization is usually quite complex, and the resulting AST structure is also complex. If it is used directly for model training, it is easy to lose node context information during the training process. In addition, the sparse distribution of nodes on the AST makes it difficult for the model to effectively utilize node information.

[0007] 2. Existing CodePTMs models, lacking training related to a certain tree-structured code representation method, cannot understand this tree-structured code representation method, resulting in poor performance in code summarization generation tasks;

[0008] 3. Introducing prior knowledge about tree-structured code representation by fine-tuning the CodePTMs model may not only disrupt the existing knowledge of the CodePTMs model, but also easily lead to overfitting of the model to a specific task. Furthermore, it is difficult to balance the weight distribution between the tree-structured code modality and the code text sequence modality representation, resulting in poor fusion of these two modalities and reduced model performance.

[0009] In summary, a code digest generation method and apparatus are invented. Summary of the Invention

[0010] To address the aforementioned technical problems, according to one aspect of the present invention, the present invention provides the following technical solution:

[0011] A code digest generation method includes the following specific steps:

[0012] S1: Preprocess the function source code and its corresponding code digest to obtain a multimodal function code digest dataset.

[0013] S2: For any triplet The FS-AST in the triplet is transformed to obtain the first node feature embedding matrix. and adjacency matrix

[0014] S3: Design a two-stage fine-tuning method for code digest generation tasks. With X0 and E as input, fine-tune the original CodePTMs model to obtain the fine-tuned prefix vector code digest model CodePTMs' model, and output the code digest.

[0015] S4: Code Summary Dataset for Multimodal Functions The data is divided and used to train CodePTMs' models;

[0016] S5: Apply the trained CodePTMs' model to the code summarization generation task, and generate a code summary corresponding to the given function source code code.

[0017] As a preferred embodiment of the code digest generation method described in this invention, the specific steps of step S1 are as follows:

[0018] S11: Collect publicly available function source code and its corresponding code summary docstring, and use an abstract syntax tree parsing tool to parse the code into an abstract syntax tree (AST);

[0019] S12: Perform a post-order traversal of the AST to construct a complete syntax tree structure FS-AST;

[0020] S13: Combine the source code of any function (code), the corresponding code digest (docstring), and the processed FS-AST into a triple.<code,docstring,FS-AST> All triples constitute a multimodal function code digest dataset.

[0021] As a preferred embodiment of the code digest generation method of the present invention, the specific steps of S12 are as follows:

[0022] S121: Use the Embedding method in the PyTorch framework to vectorize the text information of the node, and obtain the initial vector corresponding to the node. Where m is the size of the embedding vector dimension of the original CodePTMs model embedding layer;

[0023] S122: Calculate the hidden state of the node in Representing a 1×m dimensional vector space:

[0024] When node is a leaf node, the formula for calculating h is:

[0025] h=W1γ

[0026] in, It is the learnable parameter matrix in the Hidden State Encoder (HSE). Represents an m×m dimensional vector space;

[0027] When node is a non-leaf node, the formula for calculating h is:

[0028]

[0029] Where tanh represents the nonlinear activation function; Ch(node) is the set of nodes in the subtree rooted at node; W2 is the learnable parameter matrix of RvNN in HSE; f∈Ch(node) is any node in Ch(node); h f This represents the hidden state of f;

[0030] S123: If the type of node is not in the statement node type set Type, then node is replaced with h; otherwise, the subtree T rooted at node is vectorized to form a subtree vector η, and then η is used to replace the subtree T in the AST; after performing the above replacement operation on all nodes in the AST, a complete structure syntax tree FS-AST is obtained; where η is calculated as follows:

[0031] Perform max pooling on the hidden states h of all nodes in T to form the subtree vectors of T. The formula for calculating η is:

[0032]

[0033] Where CH(T) represents the set of all nodes in T; g∈CH(T) represents any node in T; h g Represents the hidden state of g; ∪ g∈CH(T) h g CH(T) represents the set of hidden states of all nodes in CH(T); the maxpooling operation means selecting the maximum value along each dimension of all hidden states in the set to form a vector.

[0034] As a preferred embodiment of the code digest generation method described in this invention, the specific steps of step S2 are as follows:

[0035] S21: Traverse the FS-AST. For each visited current node c, embed the hidden state h or subtree vector η of c as node features to form a first node feature embedding matrix. The h or subtree vector η of each node is used as a row of X0;

[0036] S22: Form an adjacency matrix based on the connection relationships between nodes in the FS-AST. For any c i ,c j ∈FS-AST, the connection relationships between nodes are stored in e i,j ∈E:

[0037]

[0038] As a preferred embodiment of the code digest generation method described in this invention, the specific steps of step S3 are as follows:

[0039] S31: In the first fine-tuning stage, a cue proxy module is constructed. X0 and E are input into the cue proxy module to obtain a cue embedding matrix. Then X0 and X1 are concatenated to obtain the fused embedding matrix X2;

[0040] S32: In the second fine-tuning stage, the parameters of the original CodePTMs model are expanded using the prefix vector ΔW to obtain the fine-tuned prefix vector code digest model CodePTMs' model.

[0041] S33: Input X2 into the CodePTMs' model, and after passing through L Transformer Layers, extract semantic features to obtain the summary representation H. L Then H L The input is fed into the Prediction Head to generate a code summary; where H L It is calculated from the following iterative equation:

[0042] H 0 =X2

[0043] H l =TransformerLayer(H l-1 ), 1≤l≤L

[0044] Among them, H l H represents the output of the l-th Transformer Layer. l-1 This represents the output of the (l-1)th TransformerLayer, where TransformerLayer represents the computation performed in the Transformer Layer.

[0045] As a preferred embodiment of the code digest generation method of the present invention, the specific steps of S31 are as follows:

[0046] S311: Construct a prompting proxy module. Input the first node feature embedding matrix X0 and the adjacency matrix E into the prompting proxy module to obtain the prompting embedding matrix.

[0047] S312: Concatenate X0 and X1 to obtain the fused embedding matrix X2:

[0048] X2 = Concat(X1, X0)

[0049] X2 contains the text information of each node on the FS-AST and the structural information between the nodes;

[0050] The specific steps of S311 are as follows:

[0051] S3111: Input X0 and E into the graph attention encoder to obtain the second node feature embedding matrix.

[0052] S3112: Flatten X' to obtain the first feature representation vector.

[0053] β1 = flatten(X')

[0054] in, Represents a (n×m)×1 dimensional vector space;

[0055] S3113: Input β1 into the first multilayer perceptron MLP1 for calculation to obtain the third feature representation vector.

[0056] S3114: Perform a reshape operation on β3 to obtain the hint embedding matrix.

[0057] X1 = reshape(β3,p,m)

[0058] in, Representing a p×m dimensional vector space, the calculated X1 also contains the FS-AST structural semantics, that is, the structural information between nodes on the FS-AST;

[0059] The specific steps of S3111 are as follows:

[0060] S31111: Map X0 to the query space, key space, and value space respectively to obtain the query vector matrix Q, key vector matrix K, and value vector matrix V. The calculation formula is as follows:

[0061] Q = X0W Q1 K = X0W K1 V = X0W V1

[0062] Among them, the query matrix Key matrix Value matrix These are all projected matrices for the learnable attention mechanism in GAT Coder;

[0063] S31112: Calculate the attention score matrix among all nodes constituting X0.

[0064]

[0065] S31113: Mask the attention scores between non-adjacent nodes in S using -∞ to obtain the masked attention score matrix S'. For any s' a,r ∈S', we have:

[0066]

[0067] Where nodes a, r ∈ D(X0), and D(X0) represents the set of all node feature embeddings that constitute X0; s a,r ∈S represents the scores of a and r in S; s' a,r ∈S' represents the scores of a and r in S'; edge e a,r ∈E, e a,r =1 indicates that a and r are adjacent, e a,r =0 indicates that a and r are not adjacent, and -∞ indicates a very small value;

[0068] S31114: Perform a softmax normalization operation on S' to obtain the attention coefficient matrix. α is calculated as follows:

[0069] α = softmax(S')

[0070] S31115: Combine the value vector matrix V to calculate the feature embedding matrix X' of the second node:

[0071] X'=αV

[0072] Among them, the calculated X' contains the FS-AST structural semantics, that is, the structural information between nodes on the FS-AST;

[0073] The specific steps of S3113 are as follows:

[0074] S31131: Input β1 into the first layer of the first multilayer perceptron MLP1, perform a linear transformation, and activate it using the ReLU activation function to obtain the second feature representation vector.

[0075] β2 = ReLU(W4β1 + b4)

[0076] S31132: Input β2 into the second layer of the first multilayer perceptron MLP1, perform a linear transformation, and activate it using the same ReLU activation function as the first layer to obtain the third feature representation vector.

[0077] β3 = ReLU(W5β2 + b5)

[0078] in, All are learnable parameter matrices; All are bias terms; Let this represent a vector space of (n×m)×(n×m) dimensions. Describe a vector space of (p×m)×(n×m) dimensions. Represents a (n×m)×1 dimensional vector space. Let represent a (p×m)×1 dimensional vector space.

[0079] As a preferred embodiment of the code digest generation method described in this invention, the specific steps of S32 are as follows:

[0080] S321: Calculate ΔW using low-rank matrices A and B:

[0081] ΔW=AB

[0082] The low-rank matrix A is an m×o-dimensional learnable parameter matrix; the low-rank matrix B is an o×Δu-dimensional learnable parameter matrix. Compared to directly updating ΔW, the method of calculating ΔW by updating the low-rank matrices A and B can improve the generalization ability and training stability of the CodePTMs' model.

[0083] S322: Compare ΔW with the query matrix W in each Transformer Layer. Q2 Key matrix W K2 To splice together, expand W Q2 and W K2 By considering the dimension of prefix vectors, a fine-tuned prefix vector code digest model, CodePTMs, is obtained; the calculation formula is as follows:

[0084] W Q2 =Concat(ΔW,W) Q2 (axis=1)

[0085] W K2 =Concat(ΔW,W) K2 (axis=1)

[0086] Concat is the concatenation operation, and the parameter axis=1 indicates that the concatenation operation is performed along the horizontal direction of the matrix.

[0087] As a preferred embodiment of the code digest generation method described in this invention, the specific steps of step S4 are as follows:

[0088] S41: Will It is divided into training dataset, test dataset, and validation dataset;

[0089] S42: Randomly initialize the set of learnable parameter matrices TD; and randomly initialize the low-rank matrices A and B in the CodePTMs' model; where TD = {W Q1 W K1 W V1 {b4,b5,b4,b5} is the set of learnable parameter matrices in the prompting agent module;

[0090] S43: In the first fine-tuning stage of training, all learnable parameter matrices in TD are updated by performing edge prediction on FS-AST; the task of performing edge prediction on FS-AST refers to using the obtained cue embedding matrix X1 to determine whether there is a direct connection between any two nodes on FS-AST.

[0091] S44: In the second fine-tuning stage of training, the hyperparameters set in the first fine-tuning stage are used. A suitable loss function and optimization algorithm are selected to update the parameters of the low-rank matrices A and B. The performance of the prefix vector ΔW is continuously evaluated during the training process, and the hyperparameters are dynamically adjusted to obtain a well-trained CodePTMs' model.

[0092] As a preferred embodiment of the code digest generation method described in this invention, the specific steps of S43 are as follows:

[0093] S431: For any node q, w∈D(X0), and The vector is concatenated with the hint embedding matrix X1, and then flattened to obtain the first structural representation vector.

[0094]

[0095] Where D(X0) represents the set of all node feature embeddings that constitute X0. This represents the node feature embedding corresponding to node q in X0. This represents the node feature embedding corresponding to node w in X0. Represent a vector space of ((p+2)×m)×1 dimensions;

[0096] S432: Input z1 into the second multilayer perceptron MLP2 to obtain the classification result.

[0097] S433: Set the hyperparameters required during training, including batch size, training epochs, gradient clipping threshold, learning rate, and weight decay coefficient. Then, select a suitable loss function and optimization algorithm to update the parameters of all learnable parameter matrices in TD. During training, continuously evaluate the performance of the prompting agent module and the second multilayer perceptron MLP2 to facilitate dynamic adjustment of hyperparameters, improve training efficiency, and finally obtain a well-trained prompting agent module.

[0098] The specific steps of S432 are as follows:

[0099] S4321: Input z1 into the first layer of the second multilayer perceptron MLP2, perform a linear transformation, and activate it using the ReLU activation function to obtain the second structural representation vector.

[0100] z2 = ReLU(W6z1 + b6)

[0101] S4322: Input z2 into the second layer of the second multilayer perceptron MLP2, perform a linear transformation, and apply softmax for normalization to obtain the classification result.

[0102]

[0103] in, All are learnable parameter matrices. All are bias terms. The structure of the second multilayer perceptron MLP2 is exactly the same as that of the first multilayer perceptron MLP1. The role of the second multilayer perceptron MLP2 is to evaluate the ability of X1 to contain structural information between nodes on the FS-AST. Represents a 2×1 dimensional vector space. Represents an m×1 dimensional vector space. Let m be a vector space of dimension m×((p+2)×m). This represents a 2×m dimensional vector space.

[0104] A code digest generation apparatus, comprising:

[0105] The preprocessing module is used to preprocess the function source code and its corresponding code digest to obtain a multimodal function code digest dataset.

[0106] The transformation module is used to transform any triplet. The FS-AST in the triplet is transformed to obtain the first node feature embedding matrix. and adjacency matrix

[0107] The two-stage fine-tuning design module is used to design a two-stage fine-tuning method for code digest generation tasks. Taking X0 and E as inputs, the original CodePTMs model is fine-tuned to obtain the fine-tuned prefix vector code digest model CodePTMs' model, and the code digest is output.

[0108] Partitioning Module: Used for processing multimodal function code summary datasets The data is divided and used to train CodePTMs' models;

[0109] The application module is used to apply the trained CodePTMs' model to the code summarization generation task, generating a code summary corresponding to the given function source code code.

[0110] Compared with existing technologies:

[0111] This invention constructs a simple FS-AST using an AST, avoiding the problem of losing contextual information at nodes during training caused by directly training complex ASTs. Through a two-stage fine-tuning method for code summarization tasks, the first stage introduces FS-AST structural semantic knowledge into the original CodePTMs model, and the second stage balances the weight allocation of the original CodePTMs model for FS-AST modal and code text sequence modal representation. This improves the model's ability to model deep semantics of code, thereby increasing the accuracy of code summarization. Furthermore, this invention can adapt to different types of code and code-related downstream tasks, improving its versatility. Attached Figure Description

[0112] Figure 1 This is a schematic diagram of the process of the present invention;

[0113] Figure 2 This is a flowchart of the two-stage fine-tuning method of the present invention;

[0114] Figure 3 This is a schematic diagram of the proxy module structure for this invention;

[0115] Figure 4 This is a structural diagram of the original CodePTMs model of this invention;

[0116] Figure 5 This is a structural diagram of the original CodeBert model of this invention;

[0117] Figure 6 This is a structural diagram of the CodeBert model of the present invention. Detailed Implementation

[0118] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0119] This invention provides a code digest generation method; please refer to [link / reference]. Figures 1-6 The specific steps are as follows:

[0120] S1: Preprocess the function source code and its corresponding code digest to obtain a multimodal function code digest dataset.

[0121] First, publicly available function source code and its corresponding code digest are collected. For each set of function source code (code) and its corresponding code digest (docstring), the code is transformed into an Abstract Syntax Tree (AST), and then a Complete Syntax Tree Structure (FS-AST) is constructed. Finally, a triple is formed.<code,docstring,FS-AST> All the triples constitute a multimodal function code summary dataset.

[0122] The specific steps of S1 are as follows:

[0123] S11: Collect publicly available function source code and its corresponding code summary docstring, and use an abstract syntax tree parsing tool to parse the code into an abstract syntax tree (AST).

[0124] in:

[0125] (1) In the implementation of this invention, publicly available function source code summary datasets, such as CodeSearchNet, CodeXGLUE, Funcom, etc., are downloaded from online sources such as Github and Kaggle. Each dataset contains several publicly available function source codes and their corresponding code summary docstrings.

[0126] (2) Abstract Syntax Tree (AST) parsing tools: These are tools used to generate abstract syntax trees (ASTs) for function source code, and are commonly used in scenarios such as static code analysis, refactoring, optimization, and security testing. Commonly used tools of this type include: 1) Tree-sitter, an efficient parsing library written in C that supports AST generation for multiple languages; and 2) javalang, an AST generation tool written in Python that only supports Java code.

[0127] (3) Abstract Syntax Tree (AST): A tree-structured code representation method where nodes represent syntactic units in the code, and the relationships between nodes reflect the syntactic structure of the code. All nodes in an AST are directly parsed from the source code and are called code nodes.

[0128] S12: Perform a post-order traversal of the AST to construct a complete syntax tree structure FS-AST.

[0129] During the post-order traversal of the AST, the initial vector of the currently visited node is first calculated. Then calculate the hidden state of the node. If the type of node does not belong to the statement node type set Type, then node is replaced with h; otherwise, the subtree T rooted at node is vectorized to form a subtree vector. Finally, replace T with η. Once all nodes in the AST have been visited, a complete structure syntax tree (FS-AST) is obtained.

[0130] The statement node type set Type = {IfStatement, WhileStatement, ForStatement} includes: IfStatement, which represents the if conditional statement for selecting the execution flow branch; WhileStatement, which represents the while loop structure statement that repeatedly executes a certain code segment under certain conditions; and ForStatement, which represents the for counting loop statement used for iteration control and has initialization, condition judgment, and iteration process.

[0131] The beneficial result of S12 is that when the size of the AST is large, the distribution of nodes in the AST will be relatively sparse, making it difficult to utilize effectively. By replacing nodes or subtrees on the AST, a FS-AST can be constructed, which can preserve the textual information of the nodes and the structured information between the nodes as much as possible, while reducing its complexity, thus making it more conducive to the model capturing effective features.

[0132] The specific steps of S12 are as follows:

[0133] S121: Use the Embedding method in the PyTorch framework to vectorize the text information of the node, and obtain the initial vector corresponding to the node. Where m is the size of the embedding vector dimension of the original CodePTMs model embedding layer;

[0134] in:

[0135] (1) Embedding method: The PyTorch framework provides an implementation of this method, which uses embedding vectors to map discrete input sequences into a continuous, low-dimensional vector space. These embedding vectors are learned through training and can capture semantic or class relationships between inputs.

[0136] (2) Embedding Layer: A crucial component in the original CodePTMs model for processing discrete symbolic inputs. It consists of a series of embedding vectors, whose main function is to map discrete input sequences (such as words, subwords, identifiers, etc.) into low-dimensional, continuous vector representations. Compared to high-dimensional sparse one-hot encoding, the embedding layer can capture the semantic and structural relationships between inputs in a lower-dimensional space. The embedding vectors are continuously optimized through backpropagation during training, thereby providing the model with more expressive input features and serving as a key bridge connecting the symbolic world and the semantic space of the neural network.

[0137] S122: Calculate the hidden state of the node in Representing a 1×m dimensional vector space:

[0138] When node is a leaf node, the formula for calculating h is:

[0139] h=W1γ

[0140] in It is the learnable parameter matrix in the Hidden State Encoder (HSE). Represents an m×m dimensional vector space;

[0141] The Hidden State Encoder (HSE) consists of a learnable parameter matrix W1 and a recursive neural network (RvNN) used to compute the hidden state h of each node in the tree.

[0142] When node is a non-leaf node, the formula for calculating h is:

[0143]

[0144] Where tanh represents the nonlinear activation function; Ch(node) is the set of nodes in the subtree rooted at node; W2 is the learnable parameter matrix of RvNN in HSE; f∈Ch(node) is any node in Ch(node); h f This represents the hidden state of f;

[0145] Tanh is a commonly used non-linear activation function that can compress the input to the interval (-1,1). Tanh has the characteristic of zero center, which helps to speed up the convergence speed of neural networks. Therefore, it is often used in hidden layers to enhance the expressive power of the model.

[0146] S123: If the type of node is not in the statement node type set Type, then node is replaced with h; otherwise, the subtree T rooted at node is vectorized to form a subtree vector η, and then η is used to replace the subtree T in the AST; after performing the above replacement operation on all nodes in the AST, a complete structure syntax tree FS-AST is obtained; FS-AST refers to a complete structure syntax tree formed by vectorizing the nodes or subtrees on the AST; where η is calculated as follows:

[0147] Perform max pooling on the hidden states h of all nodes in T to form the subtree vectors of T. The formula for calculating η is:

[0148]

[0149] Where CH(T) represents the set of all nodes in T; g∈CH(T) represents any node in T; h g U represents the hidden state of g; g∈CH(T) h g CH(T) represents the set of hidden states of all nodes in CH(T); the maxpooling operation means selecting the maximum value along each dimension of all hidden states in the set to form a vector;

[0150] Maxpooling is a common operation used to extract the maximum value of a local region from a set of input data. Specifically, it involves sliding a window along the two-dimensional or multi-dimensional space of the input data, selecting the maximum value in each window as the representative of that region. This operation can extract the most important information from a set of data, while reducing the dimensionality of the data, accelerating subsequent computation, and improving the robustness of the model.

[0151] S13: Combine the source code of any function (code), the corresponding code digest (docstring), and the processed FS-AST into a triple.<code,docstring,FS-AST> All triples constitute a multimodal function code digest dataset.

[0152] S2: For any triplet The FS-AST in the triplet is transformed to obtain the first node feature embedding matrix. and adjacency matrix

[0153] The specific steps of S2 are as follows:

[0154] S21: Traverse the FS-AST. For each visited current node c, embed the hidden state h or subtree vector η of c as node features to form a first node feature embedding matrix. The h or subtree vector η of each node is used as a row of X0;

[0155] Where n represents the total number of nodes on FS-AST, m is the size of the embedding vector dimension of the original CodePTMs model embedding layer, and the FS-AST can be traversed in any of the following ways: preorder traversal, inorder traversal, and postorder traversal. Represents an n×m dimensional vector space. Let represent an n×n dimensional vector space.

[0156] S22: Form an adjacency matrix based on the connection relationships between nodes in the FS-AST. For any c i ,c j ∈FS-AST, the connection relationships between nodes are stored in e i,j ∈E:

[0157]

[0158] S3: Design a two-stage fine-tuning method for code digest generation tasks. Using X0 and E as inputs, fine-tune the original CodePTMs model to obtain the fine-tuned prefix vector code digest model CodePTMs' model, and output the code digest.

[0159] In the two-stage fine-tuning method for code digest generation tasks, the original CodePTMs model is fine-tuned in two stages: In the first stage, a hint proxy module is constructed, and X0 and E are input into the hint proxy module to generate a hint embedding matrix. Then X0 and X1 are concatenated to form a fusion embedding matrix. In the second fine-tuning stage, prefix vectors are introduced. Expanding the parameters of the original CodePTMs model yields a fine-tuned prefix vector code digest model, CodePTMs'. Finally, inputting X2 into the CodePTMs' model generates a code digest.

[0160] The beneficial result of S3 is that the original CodePTMs model, which has not been pre-trained on FS-AST-related tasks, cannot understand the tree-structured code representation of FS-AST. To address this issue, this method first forms a fusion embedding matrix X2 in the first fine-tuning stage, introducing semantic knowledge related to the FS-AST structure into the original CodePTMs model. Then, in the second fine-tuning stage, the original CodePTMs model is fine-tuned using prefix vectors through a code summarization task, which further improves the model's performance on the code summarization task.

[0161] The specific steps of S3 are as follows:

[0162] S31: In the first fine-tuning stage, a cue proxy module is constructed. X0 and E are input into the cue proxy module to obtain a cue embedding matrix. Then X0 and X1 are concatenated (i.e. Figure 2 The concatenation operation (Concat) in the middle is used to obtain the fused embedding matrix X2.

[0163] The specific steps of S31 are as follows:

[0164] S311: Construct a prompting agent module, the structure of which is as follows: Figure 3 As shown, the first node feature embedding matrix X0 and the adjacency matrix E are input into the above-mentioned prompt proxy module to obtain the prompt embedding matrix.

[0165] The GraphAttention Network Coder (GAT Coder) is a publicly available encoder that utilizes an attention mechanism to encode graph-structured data. This encoder improves its expressive power when processing unstructured and heterogeneous graph data by introducing an attention mechanism to adaptively weight and aggregate information from neighboring nodes. In the attention mechanism, the input features of each node are first mapped to a query space, a key space, and a value space through a set of learnable attention mechanism projection matrices, constructing query vectors, key vectors, and value vectors, respectively. The query space describes the information requirements of the central node, the key space describes the index features of neighboring nodes, and the value space represents the actual information that can be aggregated. Attention weights are generated by calculating the correlation between the query vector and the corresponding neighbor's key vector, and then normalized using softmax. Finally, the attention weights are used to weight the value vectors of neighboring nodes, updating the node features.

[0166] A fully connected neural network (FCNN) is a type of neural network consisting of multiple fully connected layers. The neurons in each layer are connected to all neurons in the previous layer. It is commonly used for feature extraction and pattern recognition. It processes information through layer-by-layer computation via the input, hidden, and output layers, and is trained using backpropagation and gradient descent algorithms. It is widely used in image recognition, speech processing, and predictive modeling.

[0167] The specific steps of S311 are as follows:

[0168] S3111: Input X0 and E into the graph attention encoder to obtain the second node feature embedding matrix.

[0169] The specific steps of S3111 are as follows:

[0170] S31111: Map X0 to the query space, key space, and value space respectively to obtain the query vector matrix Q, key vector matrix K, and value vector matrix V. The calculation formula is as follows:

[0171] Q = X0W Q1 K = X0W K1 V = X0W V1

[0172] Among them, the query matrix Key matrix Value matrix These are all projected matrices for the learnable attention mechanism in GAT Coder;

[0173] S31112: Calculate the attention score matrix among all nodes constituting X0.

[0174]

[0175] LeakyReLU is an improved ReLU (Rectified Linear Unit) activation function that introduces a small negative slope parameter to the classic ReLU function. This allows it to handle inputs less than 0 and mitigates the "dead neuron problem" that may be caused by the ReLU function.

[0176] S31113: Mask the attention scores between non-adjacent nodes in S using -∞ to obtain the masked attention score matrix S'. For any s' a,r ∈S', we have:

[0177]

[0178] Where nodes a, r ∈ D(X0), and D(X0) represents the set of all node feature embeddings that constitute X0; s a,r ∈S represents the scores of a and r in S; s' a,r ∈S' represents the scores of a and r in S'; edge e a,r ∈E, e a,r =1 indicates that a and r are adjacent, e a,r =0 indicates that a and r are not adjacent, and -∞ indicates a very small value;

[0179] Example: In practical engineering applications, the value of -∞ can be 1.0 × 2. -126 .

[0180] S31114: Perform a softmax normalization operation on S' to obtain the attention coefficient matrix. α is calculated as follows:

[0181] α = softmax(S')

[0182] Softmax is a normalization operation commonly used in classification tasks. Its main function is to transform a set of real numbers into a probability distribution. It works by taking the exponent of each input value and then dividing by the sum of the exponents of all input values, so that the output is between (0,1) and the sum is 1. This can intuitively represent the probability of each class being selected.

[0183] S31115: Combine the value vector matrix V to calculate the feature embedding matrix X' of the second node:

[0184] X'=αV

[0185] The calculated X' contains the FS-AST structural semantics, that is, the structural information between nodes on the FS-AST.

[0186] S3112: Flatten X' to obtain the first feature representation vector.

[0187] β1 = flatten(X')

[0188] in, Represents a (n×m)×1 dimensional vector space;

[0189] The flatten operation is used to convert a multidimensional array into a one-dimensional array. Specifically, it extracts the elements from the original data structure in order and puts them into a one-dimensional structure.

[0190] S3113: Input β1 into the first multilayer perceptron MLP1 for calculation to obtain the third feature representation vector.

[0191] The first multilayer perceptron, MLP1, is a multilayer perceptron consisting of two fully connected neural networks (FCNN). The first layer contains a learnable parameter matrix W4 and a bias b4, which is used to map the input to the hidden space, extract features, and introduce non-linear expressive power. The second layer contains a learnable parameter matrix W5 and a bias b5, which is used to output the results.

[0192] The specific steps of S3113 are as follows:

[0193] S31131: Input β1 into the first layer of the first multilayer perceptron MLP1, perform a linear transformation, and activate it using the ReLU activation function to obtain the second feature representation vector.

[0194] β2 = ReLU(W4β1 + b4)

[0195] S31132: Input β2 into the second layer of the first multilayer perceptron MLP1, perform a linear transformation, and activate it using the same ReLU activation function as the first layer to obtain the third feature representation vector.

[0196] β3 = ReLU(W5β2 + b5)

[0197] in, All are learnable parameter matrices; All are bias terms; Let this represent a vector space of (n×m)×(n×m) dimensions. Describe a vector space of (p×m)×(n×m) dimensions. Represents a (n×m)×1 dimensional vector space. Let represent a (p×m)×1 dimensional vector space.

[0198] ReLU is a widely used non-linear activation function in deep learning. It outputs zero when the input is negative and proportional to the input when the input is positive; that is, it outputs zero for negative values ​​and remains linear for positive values. ReLU is simple to compute and effectively alleviates the vanishing gradient problem, accelerating the training speed of neural networks. Therefore, it has become the mainstream activation function for hidden layers in modern neural networks.

[0199] S3114: Perform a reshape operation on β3 to obtain the hint embedding matrix.

[0200] X1 = reshape(β3,p,m)

[0201] in, The vector space represented by p×m is calculated, and the calculated X1 also contains the semantics of the FS-AST structure, that is, the structural information between nodes on the FS-AST.

[0202] The reshape operation is an operation used to transform data from one shape to another.

[0203] S312: Concatenate X0 and X1 to obtain the fused embedding matrix X2:

[0204] X2 = Concat(X1, X0)

[0205] X2 contains the text information of each node on the FS-AST and the structural information between the nodes.

[0206] S32: In the second fine-tuning stage, the parameters of the original CodePTMs model are expanded using the prefix vector ΔW to obtain the fine-tuned prefix vector code digest model CodePTMs'.

[0207] Here, the prefix vector ΔW is an m×Δu dimensional vector; the original CodePTMs model refers to a large pre-trained code model that maintains its original structure and parameter settings and has not undergone any downstream task adaptation or structural modification. The structure of the original CodePTMs model is as follows: Figure 4 As shown, it consists of L stacked Transformer Layers. Each Transformer Layer applies a multi-head attention mechanism. After the Lth Transformer Layer, a prediction head is connected to generate code summaries.

[0208] The specific steps of S32 are as follows:

[0209] S321: Calculate ΔW using low-rank matrices A and B:

[0210] ΔW=AB

[0211] The low-rank matrix A is an m×o-dimensional learnable parameter matrix; the low-rank matrix B is an o×Δu-dimensional learnable parameter matrix. Compared to directly updating ΔW, the method of calculating ΔW by updating the low-rank matrices A and B can improve the generalization ability and training stability of the CodePTMs' model.

[0212] S322: Compare ΔW with the query matrix W in each Transformer Layer. Q2 Key matrix W K2 To splice together, expand W Q2 and W K2 By considering the dimension of prefix vectors, a fine-tuned prefix vector code digest model, CodePTMs, is obtained; the calculation formula is as follows:

[0213] W Q2 =Concat(ΔW,W) Q2 (axis=1)

[0214] W K2 =Concat(ΔW,W) K2 (axis=1)

[0215] Concat is the concatenation operation, and the parameter axis=1 indicates that the concatenation operation is performed along the horizontal direction of the matrix.

[0216] S33: Input X2 into the CodePTMs' model, and after passing through L Transformer Layers, extract semantic features to obtain the summary representation H. L Then H L The input is fed into the Prediction Head to generate a code summary; where H L It is calculated from the following iterative equation:

[0217] H 0 =X2

[0218] H l =TransformerLayer(H l-1 ), 1≤l≤L

[0219] Among them, H l H represents the output of the l-th Transformer Layer. l-1 This represents the output of the (l-1)th TransformerLayer, where TransformerLayer represents the computation performed in the Transformer Layer.

[0220] Example:

[0221] The CodeBert model is a specific CodePTMs model. In this embodiment, the original CodeBert model is fine-tuned using the prefix vector ΔW.

[0222] The untuned CodeBert model is referred to as the original CodeBert model. The original CodeBert model is a large pre-trained code model that maintains the original structure and parameter settings and has not undergone any downstream task adaptation or structural modifications. It consists of 12 stacked Transformer Layers. Each Transformer Layer has a standard encoder structure formed by a multi-head attention mechanism, a feedforward neural network, residual connections, and layer normalization (Add & LayerNorm) modules. The structure of the original CodeBert model is as follows: Figure 5 As shown, the parameters of the original CodeBert model are expanded using the prefix vector ΔW to obtain the CodeBert' model. The structure of the CodeBert' model is as follows. Figure 6 As shown.

[0223] Inputting X2 into the CodeBert model, and passing it through 12 Transformer Layers, extracts semantic features to obtain the summary representation H. 12 Then H 12 The input is fed into the Prediction Head, which generates a code summary. H... 12 It is calculated from the following iterative equation:

[0224] H 0 =X2

[0225]

[0226] in, This refers to the specific TransformerLayer calculation method in the CodeBert model, H. l H represents the output of the l-th Transformer Layer. l-1 L represents the output of the (l-1)th Transformer Layer, and LN represents the layer normalization computation (i.e., ...). Figure 6 In LayerNorm), MH represents the multi-head attention mechanism in the Transformer Layer (i.e., ... Figure 6 The computation of Multi-HeadAttention in the Transformer Layer (FFN) represents the computation of the feedforward neural network in the Transformer Layer (i.e., ... Figure 6 The calculation of FeedForward in the middle, This indicates performing a residual join (i.e.) Figure 6 Add in (the middle part).

[0227] S4: Code Summary Dataset for Multimodal Functions The data is divided and used to train the CodePTMs' model.

[0228] The specific steps of S4 are as follows:

[0229] S41: Will It is divided into training dataset, test dataset and validation dataset.

[0230] S42: Randomly initialize the set of learnable parameter matrices TD; and randomly initialize the low-rank matrices A and B in the CodePTMs' model; where TD = {W Q1 W K1 W V1 ,W4,W5,b4,b5} is the set of learnable parameter matrices in the prompting agent module.

[0231] S43: In the first fine-tuning phase of training, all learnable parameter matrices in TD are updated by performing edge prediction on FS-AST; where the task of performing edge prediction on FS-AST refers to using the obtained cue embedding matrix X1 to determine whether there is a direct connection between any two nodes on FS-AST.

[0232] The specific steps of S43 are as follows:

[0233] S431: For any node q, w∈D(X0), and The vector is concatenated with the hint embedding matrix X1, and then flattened to obtain the first structural representation vector.

[0234]

[0235] Where D(X0) represents the set of all node feature embeddings that constitute X0. This represents the node feature embedding corresponding to node q in X0. This represents the node feature embedding corresponding to node w in X0. Let represent a vector space of ((p+2)×m)×1 dimensions.

[0236] S432: Input z1 into the second multilayer perceptron MLP2 to obtain the classification result.

[0237] The specific steps of S432 are as follows:

[0238] S4321: Input z1 into the first layer of the second multilayer perceptron MLP2, perform a linear transformation, and activate it using the ReLU activation function to obtain the second structural representation vector.

[0239] z2 = ReLU(W6z1 + b6)

[0240] S4322: Input z2 into the second layer of the second multilayer perceptron MLP2, perform a linear transformation, and apply softmax for normalization to obtain the classification result.

[0241]

[0242] in, All are learnable parameter matrices. All are bias terms. The structure of the second multilayer perceptron MLP2 is exactly the same as that of the first multilayer perceptron MLP1. The role of the second multilayer perceptron MLP2 is to evaluate the ability of X1 to contain structural information between nodes on the FS-AST. Represents a 2×1 dimensional vector space. Represents an m×1 dimensional vector space. Let m be a vector space of dimension m×((p+2)×m). This represents a 2×m dimensional vector space.

[0243] In this patented solution, the first multilayer perceptron (MLP1) is used to perform a further nonlinear transformation on the second node feature embedding matrix X', which contains structural information between nodes, mapping X' to a dimension of a specified size to form a cue embedding matrix X1 with FS-AST structural semantics. The second multilayer perceptron (MLP2) is used during the training process in the first fine-tuning phase to evaluate the ability of X1 to contain structural information between nodes on the FS-AST.

[0244] S433: Set the hyperparameters required during training, including batch size, training epochs, gradient pruning threshold, learning rate, and weight decay coefficient. Then, select a suitable loss function and optimization algorithm to update the parameters of all learnable parameter matrices in TD. During training, continuously evaluate the performance of the prompting agent module and the second multilayer perceptron MLP2 to facilitate dynamic adjustment of hyperparameters, improve training efficiency, and finally obtain a well-trained prompting agent module.

[0245] S44: In the second fine-tuning stage of training, the hyperparameters set in the first fine-tuning stage are used. A suitable loss function and optimization algorithm are selected to update the parameters of the low-rank matrices A and B. The performance of the prefix vector ΔW is continuously evaluated during the training process, and the hyperparameters are dynamically adjusted to obtain a well-trained CodePTMs' model.

[0246] Example: [The sentence is incomplete and requires more context to be translated accurately.] The dataset is divided into training, testing, and validation datasets in an 8:1:1 ratio. In the actual implementation of this invention, the batch size is set to 32, the training epochs to 100, the gradient clipping threshold to 2.5, the learning rate to 1e-3, and the weight decay coefficient to 1e-5. In the first fine-tuning training phase, the selected loss function is the binary cross-entropy loss function, a commonly used loss function in binary classification tasks, which measures the difference between the model's predicted results and the true labels. The selected optimization algorithm is the Adam optimization algorithm, an optimization method that combines adaptive learning rate and momentum mechanism, which can dynamically adjust the learning rate of each parameter during training, thereby accelerating convergence and improving training efficiency. In the second fine-tuning training phase, the selected loss function is the cross-entropy loss function, a commonly used loss function in multi-class classification or sequence generation tasks, which measures the model's prediction accuracy when generating each target word by calculating the difference between the probability distribution predicted by the model and the true label distribution. The selected optimization algorithm is also the Adam optimization algorithm.

[0247] The specific formula for the binary cross-entropy loss function is:

[0248]

[0249] Where BS represents the total number of samples used in one training session, and y bs This represents the true label value of the b-th sample. This represents the prediction result for the b-th sample. The logarithmic function is used to amplify the deviation between the prediction and the true value.

[0250] The specific formula for the cross-entropy loss function is as follows:

[0251]

[0252] Where BC represents the total number of tokens predicted in this training, and VS represents the total number of predicted categories, i.e., the vocabulary size. This represents the true label value of the bc-th sample in class vs. This represents the predicted probability distribution of the bc-th sample belonging to the vs-th class. The logarithmic function is used to amplify the deviation between the prediction and the true value.

[0253] S5: Apply the trained CodePTMs' model to the code summarization generation task, and generate a code summary corresponding to the given function source code code.

[0254] A code digest generation apparatus, comprising:

[0255] The preprocessing module is used to preprocess the function source code and its corresponding code digest to obtain a multimodal function code digest dataset.

[0256] The transformation module is used to transform any triplet. The FS-AST in the triplet is transformed to obtain the first node feature embedding matrix. and adjacency matrix

[0257] The two-stage fine-tuning design module is used to design a two-stage fine-tuning method for code digest generation tasks. Taking X0 and E as inputs, the original CodePTMs model is fine-tuned to obtain the fine-tuned prefix vector code digest model CodePTMs' model, and the code digest is output.

[0258] Partitioning Module: Used for processing multimodal function code summary datasets The data is divided and used to train CodePTMs' models;

[0259] The application module is used to apply the trained CodePTMs' model to the code summarization generation task, generating a code summary corresponding to the given function source code code.

[0260] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A code digest generation method characterized by, The specific steps include the following: S1: preprocessing the function source code and its corresponding code summary, and the specific steps are as follows: S11: collect the public function source code code and its corresponding code summary docstring, and use the abstract syntax tree parsing tool to parse code into an abstract syntax tree AST; S12: post-order traversal is performed on the AST to construct a complete syntax structure tree FS-AST; S13: any function source code code, the corresponding code summary docstring and the corresponding FS-AST generated after processing form a triple <code, docstring, FS-AST>, and all triples constitute a multi-modal function code summary dataset C; S2: transform the FS-AST in the triplets to get a first node feature embedding matrix and an adjacency matrix ; the specific steps are as follows:​ S21 : traversing the FS-AST, for each visited current node c, taking the hidden state h or the subtree vector η of c as node feature embedding, forming a first matrix of node feature embeddings where h or the subtree vector η of each node is taken as a row of . S22: Form an adjacency matrix according to the connection relationship between each node in the FS-AST ; for any , the connection relationship between each node is stored in : ； S3: design a two-stage fine-tuning method for code summary generation tasks, to and For input, fine-tune the original CodePTMs model to obtain the fine-tuned prefix vector code summary model CodePTMs' model, and output the code summary; the specific steps are as follows: S31: In the first fine-tuning stage, a prompt agent module is constructed, which is used to and input the prompt agent module to obtain a prompt embedding matrix ; then and are spliced to obtain a fusion embedding matrix ; S32: In the second fine-tuning stage, use the prefix vector The parameters of the original CodePTMs model are expanded to obtain a fine-tuned prefix vector code summary model CodePTMs' model. S33: the input CodePTMs' model, sequentially through L layers of Transformer Layer, and extract semantic features to obtain an abstract representation ; then input to the prediction head Prediction Head to generate a code abstract; wherein is calculated by the following iterative equation: wherein, denotes the output of the layer Transformer Layer, output of the layer Transformer Layer, TransformerLayer denotes the computation performed in the Transformer Layer; S4: On multi-modal function code summary dataset partitioning and used to train the CodePTMs' model; S5: apply the trained CodePTMs' model to the code summary generation task, and generate the code corresponding to the code summary for the given function source code code.

2. The method of claim 1, wherein The specific steps of S12 are as follows: S121: Vectorize the text information of the node node using the Embedding method in the Pytorch framework to obtain the initial vector corresponding to the node node where m is the embedding vector dimension size of the original CodePTMs model embedding layer. S122: Compute the hidden state of the node where denotes a 1 x m-dimensional vector space: When node is a leaf node, The calculation formula is: wherein is a matrix of learnable parameters in the hidden state encoder HSE, denotes a vector space of dimension m x m; When node is a non-leaf node, The calculation formula is: wherein tanh denotes a non-linear activation function; Ch(node) is a set of nodes on a sub-tree rooted at node; is a learnable parameter matrix of RvNN in HSE; is any node in Ch(node); denotes a hidden state of S123: if the type of node node does not belong to the set of statement node types Type, replace node with h; otherwise, vectorize the subtree T with node as the root to form a subtree vector η, and then replace the subtree T in the AST with η; after performing the above replacement operation on all nodes on the AST, a complete structure syntax tree FS-AST is obtained; wherein the calculation method of η is as follows: maxpooling the hidden states h of all nodes in T to form a subtree vector for T The formula for calculating η is: where CH(T) denotes the set of all nodes in T; g G CH(T) denotes an arbitrary node in T; denotes the hidden state of g; denotes the set consisting of the hidden states of all nodes in CH(T); the maxpooling operation denotes selecting the maximum value along each dimension of all hidden states in the set to form a vector.

3. The method of claim 1, wherein The specific steps of S31 are as follows: S311: constructing a prompt agent module, embedding the first node features into a matrix and an adjacency matrix inputting the prompt agent module to obtain a prompt embedding matrix ; S312: Concatenate and to obtain the fusion embedding matrix : wherein, contains the text information of each node on the FS-AST and the structural information between nodes; The specific steps of S311 are as follows: S3111: inputting the first node feature embedding matrix and the second node feature embedding matrix into a graph attention encoder to obtain a third node feature embedding matrix and wherein n represents the total number of nodes on the FS-AST, and m is the embedding vector dimension size of the embedding layer of the original CodePTMs model.​ S3112: performing a flattening operation flatten on the first feature representation vector to obtain a second feature representation vector : wherein, denotes an (n x m) x 1 dimensional vector space; S3113: obtaining the third feature representation vector by inputting the first multi-layer perception (MLP1) and performing calculation inputting the first multi-layer perception (MLP1) to perform calculation, and obtaining the third feature representation vector ; S3114: reshape to get the hint embedding matrix :​ wherein, denotes a p x m dimensional vector space, computed as Also included in the FS-AST structure semantics, i.e. structural information between nodes on the FS-AST; The specific steps of S3111 are as follows: S31111: respectively Mapping to the query space, key space and value space, get the query vector matrix Q, key vector matrix K, value vector matrix V, the calculation formula is as follows: where the query matrix , the key matrix , and the value matrix are all learnable attention mechanism projection matrices in GAT Coder; S31112: compute attention score matrix between all nodes of the graph :​ S31113: use the attention scores between non-adjacent nodes in S masking, obtaining a mask attention score matrix , for any , we have: where nodes , represent the set of all node feature embeddings corresponding to nodes ; represent the scores of a and r corresponding in S; represent the scores of a and r corresponding in S; represent the scores of a and r corresponding in S; , represent that a and r are adjacent, represent that a and r are not adjacent, represent a very small value; S31114: to The normalization operation softmax is performed to obtain the attention coefficient matrix The calculation method is as follows:​ S31115: In combination with the value vector matrix V, a second node feature embedding matrix is calculated : wherein the calculated The FS-AST structure semantics, i.e. the structural information between nodes on the FS-AST, are contained in the The specific steps of S3113 are as follows: S31131: will be input to the first layer of the first multi-layer perceptron, MLP1, is linearly transformed and activated with a ReLU activation function to obtain a second feature representation vector : S31132: to input to the second layer of the first multi-layer perceptron MLP1, a linear transformation is performed and the same ReLU activation function as the first layer is applied for activation, obtaining a third feature representation vector : wherein, are both learnable parameter matrices; are both bias terms; denotes a (n x m) x (n x m) dimensional vector space, denotes a (p x m) x (n x m) dimensional vector space, denotes a (n x m) x 1 dimensional vector space, denotes a (p x m) x 1 dimensional vector space.

4. The method of claim 1, wherein The specific steps of S32 are as follows: S321: Compute by low-rank matrices A and B : where the low-rank matrix A is an m x o learnable parameter matrix; The low-rank matrix B is a learnable parameter matrix of o x ∆u dimensions; AW represents the prefix vector, compared to direct update , the generalization ability and the stability of the CodePTMs' model can be improved by updating the low-rank matrices A and B in the manner of ​ S322: Concatenate the query matrix , the key matrix , and the value matrix in each layer of the Transformer Layer, respectively, to expand the dimensions of and , to obtain a fine-tuned prefix vector code summary model CodePTMs' model; the calculation formula is as follows: wherein is a concatenation operation, and the parameter axis=1 indicates that the concatenation operation is specified along the horizontal direction of the matrix.

5. The method of claim 1, wherein The specific steps of S4 are as follows: S41: Will It is divided into training dataset, test dataset, and validation dataset; S42: randomly initialize the set of learnable parameter matrices TD; and randomly initialize the low-rank matrices A and B in the CodePTMs' model; where TD= {W Q1 , W K1 , W V1 , W4, W5, b4, b5} is a set of learnable parameter matrices in the prompting agent module; S43: In the training of the first fine-tuning stage, all the learnable parameter matrices in the TD are updated by performing the task of side prediction on the FS-AST; wherein the task of side prediction on the FS-AST refers to using the obtained prompt embedding matrix determining whether there is a direct connection relationship between any two nodes on the FS-AST; S44: In the training of the second fine-tuning stage, the hyperparameters set in the first fine-tuning stage are used, and appropriate loss functions and optimization algorithms are selected to update the parameters of the low-rank matrices A and B, and the performance of the prefix vector is continuously evaluated during the training process, and the hyperparameters in the training process are dynamically adjusted to obtain a trained CodePTMs' model. S44: In the training of the second fine-tuning stage, the hyperparameters set in the first fine-tuning stage are used, and appropriate loss functions and optimization algorithms are selected to update the parameters of the low-rank matrices A and B, and the performance of the prefix vector is continuously evaluated during the training process, and the hyperparameters in the training process are dynamically adjusted to obtain a trained CodePTMs' model.

6. The method of claim 5, wherein, The specific steps of S43 are as follows: S431: For any node , the following is performed : and the prompt embedding matrix are concatenated, and a flatten operation is performed to obtain a first structure representation vector : ； wherein, denotes the set of all node feature embeddings corresponding to nodes of denotes the node feature embedding corresponding to node q in denotes the node feature embedding corresponding to node w in denotes a ((p+2) x m) x 1 dimensional vector space;​​​ S432: obtaining the classification result by inputting the feature vector into a second multi-layer perceptron (MLP2) ;​ S433: set the hyperparameters required in the training process, including batch size, training rounds, gradient clipping threshold, learning rate, weight decay coefficient, then select appropriate loss function and optimization algorithm to update the parameters in TD, and continuously evaluate the performance of the prompt agent module and the second multi-layer perception MLP2 during the training process, so as to dynamically adjust the hyperparameters, improve the training efficiency, and finally obtain a trained prompt agent module; The specific steps of S432 are as follows: S4321 : the input to the first layer of the second multi-layer perceptron MLP2, a linear transformation is performed and activated using a ReLU activation function, resulting in a second structural representation vector : ； S4322: the input to the second layer of the second multi-layer perceptron MLP2, a linear transformation is performed and a softmax is applied for normalization operation to obtain a classification result : ； wherein, , are both learnable parameter matrices, , are both bias terms, the structure of the second multi-layer perceptron MLP2 is identical to the first multi-layer perceptron MLP1, and the role of the second multi-layer perceptron MLP2 is to evaluate the ability to include structural information between nodes on the FS-AST, denotes a 2 x 1 dimensional vector space, denotes an m x 1 dimensional vector space, denotes an m x ((p+2) x m) dimensional vector space, denotes a 2 x m dimensional vector space.

7. An apparatus for generating a code digest for implementing the method of claim 1, characterized by It includes: A preprocessing module is configured to preprocess function source code and corresponding code summaries to obtain a multi-modal function code summary dataset ; a transformation module for transforming any triplets transforming the FS-AST in the triplets to obtain a first node feature embedding matrix and an adjacency matrix ; A two-stage fine-tuning design module is used to design a two-stage fine-tuning method for a code summary generation task, so as to and For the input, the original CodePTMs model is fine-tuned to obtain a fine-tuned prefix vector code summary model CodePTMs' model, and the output is a code summary. Partition module: for partitioning a multi-modal function code summary dataset is partitioned and used to train a CodePTMs' model; The application module is used for applying the trained CodePTMs' model to the code summary generation task, and generating the code corresponding to the code summary for the given function source code code.

Images