Method and device for generating class-level code summary

Abstract

The present invention provides a method and device for generating a class-level code summary, including: obtaining a class-level source code sequence and a method code snippet of each member method from a class-level source code file, respectively generating a representation vector of the class-level source code sequence and a representation vector of a summary of the method code snippet, inputting the representation vector and the representation vector into a generation model of the class-level code summary, and predicting a class-level code summary corresponding to the class-level source code file, wherein the basic model of the generation model is a Transformer model, the generation model includes a shared encoder and a joint decoder, the shared encoder is trained by learning encoding capabilities in a two-stage shared parameter manner, and the joint decoder is trained by learning decoding capabilities by taking the outputs of the two stages of the shared encoder as inputs. The efficiency and accuracy of generating class-level code summaries are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a method and device for generating a class-level code summary. Background Art

[0002] Code summary is a natural language explanation of source code. High-quality code summary helps improve the efficiency of software developers in understanding and maintaining programs and better reuse code. Code summary generation refers to automatically understanding program source code and generating natural language explanations for it. It is a common code comprehension task in software engineering. Traditional code summary generation methods are mainly based on rules and pattern matching, and their adaptability and accuracy are limited. Moreover, most existing code summary generation technologies focus on the automatic generation of method or function level code summaries, but few focus on class level code summary generation.

[0003] As the basic building block of object-oriented programs, classes are more abstract and often encapsulate important functions and logic. Applying function or method-level summary generation methods directly to class-level code will produce poor results. However, in the actual software development process, class code understanding and reuse are often encountered, and a breakthrough in class code summary generation technology is more urgent.

[0004] The content of the background technology section is only the information known to the inventor personally, and does not mean that the above information has entered the public domain before the application date of this disclosure, nor does it mean that it can become the prior art of the present disclosure. Summary of the invention

[0005] The present disclosure provides a method and device for generating a class-level code summary to solve the above technical problems.

[0006] In a first aspect, the present disclosure provides a method for generating a class-level code summary, comprising:

[0007] Get the class-level source code file;

[0008] From the class-level source code file, respectively obtain a class-level source code sequence and a method code snippet of each member method;

[0009] Generate representation vectors of the class-level source code sequences respectively , and a representation vector of the summary of the method code snippet ;

[0010] The representation vector and the representation vector Inputting into a pre-trained class-level code summary generation model, predicting and obtaining a class-level code summary corresponding to the class-level source code file;

[0011] The base model of the generative model is a Transformer model, and the generative model includes a shared encoder and a joint decoder. The shared encoder is trained by learning the encoding ability by sharing parameters in two stages, and the joint decoder is trained by learning the decoding ability by taking the outputs of the two stages of the shared encoder as inputs. The two stages include learning to obtain the representation vector based on the representation vector. Encode to get class-level source code feature vector The encoding capability of the stage, and the representation vector based on Encoding method summary feature vector stage of coding ability.

[0012] In some embodiments, the generating of the representation vectors of the class-level source code sequences respectively , and a representation vector of the summary of the method code snippet ,include:

[0013] The class-level source code sequence is simplified based on the class-level code text simplification algorithm, and the simplified class-level source code sequence is word-embedded to generate the representation vector ;

[0014] Generate a summary of the method code snippet based on CodeT5, and use word embedding to generate the representation vector .

[0015] In some embodiments, the shared encoder includes multiple identical encoder layers, and the multiple identical encoder layers are superimposed; each encoder layer includes multi-head attention, residual connection, and feedforward neural network.

[0016] In some embodiments, the input sequence of the multi-head attention in the first layer encoder includes the representation vector and represents vector ;

[0017] Each head of the multi-head attention in the first layer encoder focuses on different positions of the input sequence with different weights, and the output corresponding to each head is fused to obtain a self-attention output through concatenation and linear transformation;

[0018] The residual connection in the first layer encoder adds the input sequence to the self-attention output and performs normalization;

[0019] The input of the feedforward neural network is the output of the residual connection. The feedforward neural network includes two linear transformations and a nonlinear activation function. The output of the feedforward neural network in the last layer of the encoder includes the feature vector and the eigenvector .

[0020] In some embodiments, the generative model is obtained by training the Transformer model based on class-level source code text samples;

[0021] The two-stage shared parameter learning method includes:

[0022] The first stage: based on the representation vector of the class-level source code sequence of the class-level source code text sample, updating the parameters of the shared encoder in the Transformer model;

[0023] The second stage: based on the representation vector of the summary of the method code snippet of the class-level source code text sample, updating the updated parameters of the shared encoder in the Transformer model of the first stage.

[0024] In some embodiments, the joint decoder includes a plurality of identical decoder layers, and the plurality of identical decoder layers are stacked; each decoder layer includes, in sequence: a class summary self-attention layer, a source code attention layer, a method summary attention layer, and a feed-forward neural network layer;

[0025] The class summary self-attention layer is used to provide context information for generating the class-level code summary, and the source code attention layer is used to fuse the feature vector , the method summarizes that the attention layer is used to fuse the feature vector .

[0026] In some embodiments, the input of the class summary self-attention layer in the first layer decoder includes the class summary representation vector of the class-level source code file ;

[0027] The input of the source code attention layer in the first layer decoder includes the feature vector and the output of the class summary self-attention layer in the first layer decoder;

[0028] The input of the attention layer in the first layer decoder method summary includes the feature vector and the output of the source code attention layer in the first layer decoder;

[0029] The output of the feed-forward neural network layer in the last encoder layer includes the class-level code summary.

[0030] In a second aspect, the present disclosure provides a device for generating a class-level code summary, comprising:

[0031] An acquisition unit, used to obtain a class-level source code file;

[0032] The acquisition unit is also used to obtain, from the class-level source code file, a class-level source code sequence and a method code fragment of each member method;

[0033] A generation unit, used to generate representation vectors of the class-level source code sequences respectively , and a representation vector of the summary of the method code snippet ;

[0034] A prediction unit is used to transform the representation vector and the representation vector Inputting into a pre-trained class-level code summary generation model, predicting and obtaining a class-level code summary corresponding to the class-level source code file;

[0035] The base model of the generative model is a Transformer model, and the generative model includes a shared encoder and a joint decoder. The shared encoder is trained by learning the encoding ability by sharing parameters in two stages, and the joint decoder is trained by learning the decoding ability by taking the outputs of the two stages of the shared encoder as inputs. The two stages include learning to obtain the representation vector based on the representation vector. Encode to get class-level source code feature vector The encoding capability of the stage, and the representation vector based on Encoding method summary feature vector stage of coding ability.

[0036] In some embodiments, the generating unit is used to simplify the class-level source code sequence based on a class-level code text simplification algorithm, and embed the simplification into words of the class-level source code sequence to generate the representation vector , and generate a summary of the method code snippet based on CodeT5, and use word embedding to generate the representation vector .

[0037] In some embodiments, the shared encoder includes multiple identical encoder layers, and the multiple identical encoder layers are superimposed; each encoder layer includes multi-head attention, residual connection, and feedforward neural network.

[0038] In some embodiments, the input sequence of the multi-head attention in the first layer encoder includes the representation vector and represents vector ;

[0039] Each head of the multi-head attention in the first layer encoder focuses on different positions of the input sequence with different weights, and the output corresponding to each head is fused to obtain a self-attention output through concatenation and linear transformation;

[0040] The residual connection in the first layer encoder adds the input sequence to the self-attention output and performs normalization;

[0041] The input of the feedforward neural network is the output of the residual connection. The feedforward neural network includes two linear transformations and a nonlinear activation function. The output of the feedforward neural network in the last layer of the encoder includes the feature vector and the eigenvector .

[0042] In some embodiments, the generative model is obtained by training the Transformer model based on class-level source code text samples;

[0043] The two-stage shared parameter learning method includes:

[0044] The first stage: based on the representation vector of the class-level source code sequence of the class-level source code text sample, updating the parameters of the shared encoder in the Transformer model;

[0045] The second stage: based on the representation vector of the summary of the method code snippet of the class-level source code text sample, updating the updated parameters of the shared encoder in the Transformer model of the first stage.

[0046] In some embodiments, the joint decoder includes a plurality of identical decoder layers, and the plurality of identical decoder layers are stacked; each decoder layer includes, in sequence: a class summary self-attention layer, a source code attention layer, a method summary attention layer, and a feed-forward neural network layer;

[0047] The class summary self-attention layer is used to provide context information for generating the class-level code summary, and the source code attention layer is used to fuse the feature vector , the method summarizes that the attention layer is used to fuse the feature vector .

[0048] In some embodiments, the input of the class summary self-attention layer in the first layer decoder includes the class summary representation vector of the class-level source code file ;

[0049] The input of the source code attention layer in the first layer decoder includes the feature vector and the output of the class summary self-attention layer in the first layer decoder;

[0050] The input of the attention layer in the first layer decoder method summary includes the feature vector and the output of the source code attention layer in the first layer decoder;

[0051] The output of the feed-forward neural network layer in the last encoder layer includes the class-level code summary.

[0052] In a third aspect, the present disclosure provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, wherein the computer program is used to enable the processor to execute the method described in the first aspect above.

[0053] In a fourth aspect, the present disclosure provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0054] The method and device for generating a class-level code summary provided by the present disclosure include: obtaining a class-level source code file, obtaining a class-level source code sequence and a method code fragment of each member method from the class-level source code file, and generating a representation vector of the class-level source code sequence. , and the representation vector of the summary of the method code snippet , which will represent the vector and represents vector The input is to the pre-trained class-level code summary generation model, and the class-level code summary corresponding to the class-level source code file is predicted. The basic model of the generation model is the Transformer model. The generation model includes a shared encoder and a joint decoder. The shared encoder is trained by learning the encoding ability by sharing parameters in two stages. The joint decoder is trained by taking the output of the two stages of the shared encoder as input to learn the decoding ability. The two stages include learning to obtain the representation vector based on the representation vector. Encode to get class-level source code feature vector The encoding capability of the stage, and the representation vector Encoding method summary feature vector stage of coding ability.

[0055] That is to say, in this embodiment, the generation model is learned in two stages using shared parameters. By generating a class-level code summary from the code sequence dimension and the method summary dimension through the generation model, it not only exerts excellent expressiveness in the grammatical structure of the source code, but also can make full use of the semantic information provided by the method summary. Relatively speaking, this two-stage shared parameter training and learning method helps to improve the efficiency and generalization ability of the model, better capture the semantics and functions between the code and the summary, and at the same time reduce the number of parameters of the model to support efficient and high-quality class code summary generation. Therefore, this embodiment determines the class-level code summary corresponding to the class-level source code file based on the generation model, which can improve the accuracy and reliability of the determined class-level code summary. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0057] Figure 1 A flowchart of a method for generating a class-level code summary provided by the present disclosure;

[0058] Figure 2 A schematic diagram of the principle of the method for generating a class-level code summary provided by the present disclosure;

[0059] Figure 3 The structural intention of the shared encoder provided by the present disclosure;

[0060] Figure 4 The structural intention of the joint decoder provided by the present disclosure;

[0061] Figure 5 A schematic diagram of the structure of a device for generating a class-level code summary provided by an embodiment of the present disclosure;

[0062] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.

[0063] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0064] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0065] It should be understood that the terms "including" and "having" and any variations thereof in the embodiments of the present disclosure are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components is not necessarily limited to those components explicitly listed, but may include other components not explicitly listed or inherent to these products or devices.

[0066] In the embodiments of the present disclosure, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0067] The term "plurality" in the embodiments of the present disclosure refers to two or more than two, and other quantifiers are similar thereto.

[0068] The terms "first", "second", "third", etc. in the present disclosure are used to distinguish similar or similar objects or entities, and are not necessarily meant to limit a specific order or sequence, unless otherwise indicated. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented in an order other than those given in the diagrams or descriptions of the embodiments of the present disclosure.

[0069] The term "unit / module" as used in this disclosure refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or a combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0070] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0071] See also Figure 1 , Figure 1 A flow chart of a method for generating a class-level code summary provided by the present disclosure. Figure 1 As shown, the method includes:

[0072] S101: Obtain a class-level source code file.

[0073] Exemplarily, the execution subject of the method for generating a class-level code summary (hereinafter referred to as the generation method) provided by the present disclosure may be a device for generating a class-level code summary (hereinafter referred to as the generation device). The generation device may be a server, a processor, a chip, etc., which are not listed here one by one. The generation device is described elsewhere in this article.

[0074] The following example can be used to obtain the class-level source code file:

[0075] In one example, the generating device may be connected to the collecting device and receive the class-level source code file sent by the collecting device.

[0076] In another example, the generating device may provide a tool for loading a class-level source code file, and the user may transfer the class-level source code file to the generating device through the tool for loading a class-level source code file.

[0077] Among them, the tool for loading class-level source code files can be an interface for connecting to an external device, such as an interface for connecting to other storage devices, through which the class-level source code files transmitted by the external device are obtained; the tool for loading class-level source code files can also be a display device, such as a generating device that can output an interface for loading class-level source code files on a display device, and a user can import the class-level source code files into the generating device through this interface.

[0078] S102: Obtaining a class-level source code sequence and a method code snippet of each member method from the class-level source code file.

[0079] For example, Figure 2 As shown, the generating device reads the class-level source code sequence from the obtained class-level source code file, and uses regular expressions to extract each member method of the class to obtain the method code snippet corresponding to each member method.

[0080] S103: Generate representation vectors of class-level source code sequences respectively , and the representation vector of the summary of the method code snippet .

[0081] Among them, represents the vector is a vector that represents the class-level source code file from the code sequence dimension. A vector that represents the class-level source code file from the dimension of method code snippets.

[0082] In some embodiments, S103 may include:

[0083] S1031: Simplify the class-level source code sequence based on the class-level code text simplification algorithm, and embed the simplified class-level source code sequence into words to generate a representation vector .

[0084] For example, Figure 2 As shown, for the class-level source code sequence, the generating device can use the class-level code text simplification algorithm to simplify it, and then use word embedding to generate the class-level source code sequence representation vector .like Figure 2 The code sequence shown represents a vector.

[0085] S1032: Generate a summary of the code snippet based on CodeT5 and use word embedding to generate a representation vector .

[0086] For example, for each method code snippet of a member method of a class, the generating device may use CodeT5 to generate a summary for each method code snippet, merge the summaries of each method code snippet, and then use word embedding to generate a representation vector of the method-level code summary. .like Figure 2 The method summary shown represents a vector.

[0087] Among them, CodeT5 is a pre-trained encoder-decoder model based on the T5 architecture. It uses identifiers in the code to perceive the pre-training target, distinguish and restore masked identifiers; it uses a large amount of code and comment bimodal data for bidirectional generation training, learns the mapping relationship between code and comments, and has strong code understanding and generation capabilities.

[0088] In this embodiment, the representation vector is generated by the technical solution described in S1031 and S1032. and represents vector It does not increase the input data source, but can automatically discover and extract richer feature information from the data source (i.e., class-level source code files) to help generate high-quality class-level code summaries.

[0089] S104: Representing vector and represents vector The class-level code summary is input into the pre-trained generation model to predict the class-level code summary corresponding to the class-level source code file.

[0090] The basic model of the generative model is the Transformer model. The generative model includes a shared encoder and a joint decoder. The shared encoder is trained by learning the encoding ability by sharing parameters in two stages. The joint decoder is trained by taking the output of the two stages of the shared encoder as input to learn the decoding ability. The two stages include learning to obtain the representation vector based on the shared encoder. Encode to get class-level source code feature vector The encoding capability of the stage, and the representation vector Encoding method summary feature vector stage of coding ability.

[0091] For example, Figure 2 As shown in Figure 1, the basic model of the generative model is the Transformer model. The generative model mainly includes a shared encoder and a joint decoder.

[0092] The input of the shared encoder includes the code sequence representation vector (i.e., the representation vector ) and method summary represents vectors (i.e. represents vectors ). That is, the input of the shared encoder includes representation vectors of two dimensions: the code sequence dimension and the method summary dimension.

[0093] The shared encoder can determine the representation vector The corresponding class-level source code feature vector (i.e., feature vector ), and the vector The corresponding method summary feature vector (i.e., the feature vector ). That is, the output of the shared encoder includes a feature vector of two dimensions: the code sequence dimension and the method summary dimension. The feature vector It is an encoding feature that characterizes the class-level source code file from the code sequence dimension. Feature vector Encoding features for characterizing class-level source code files from the dimension of method code snippets.

[0094] The input of the joint decoder is the output of the shared encoder. That is, the input of the joint decoder includes a feature vector of two dimensions: the code sequence dimension and the method summary dimension. The output of the joint decoder includes the class-level code summary.

[0095] It is worth noting that, in this embodiment, the generation model is learned in two stages using shared parameters. By generating a class-level code summary from the code sequence dimension and the method summary dimension through the generation model, it not only exerts excellent expressiveness in the grammatical structure of the source code, but also can make full use of the semantic information provided by the method summary. Relatively speaking, this two-stage shared parameter training and learning method helps to improve the efficiency and generalization ability of the model, better capture the semantics and functions between the code and the summary, and at the same time reduce the number of parameters of the model to support efficient and high-quality class code summary generation. Therefore, this embodiment determines the class-level code summary corresponding to the class-level source code file based on the generation model, which can improve the accuracy and reliability of the determined class-level code summary.

[0096] In some embodiments, the shared encoder includes multiple identical encoder layers, and multiple identical encoder layers are stacked; each encoder layer includes multi-head attention, residual connection, and feedforward neural network.

[0097] For example, Figure 3 As shown, the shared encoder is based on the Transformer architecture and uses a parameter sharing strategy to process the representation vector and represents vector The shared encoder is formed by stacking multiple identical encoder layers. The multi-head attention can be eight-head automatic attention. This embodiment is mainly described by taking eight-head automatic attention as an example.

[0098] In this embodiment, by stacking multiple identical encoder layers, the shared encoder can gradually extract higher-level semantic information of the input sequence. In addition, the self-attention mechanism helps to capture global dependencies, while the feedforward neural network can better process local features. Therefore, the feature vector output by the shared encoder can characterize the class-level source code file from semantic information, global features, and local features. That is, the effectiveness, accuracy, reliability, and comprehensiveness of the encoding can be improved. Thereby, the accuracy and reliability of generating class-level code summaries can be improved.

[0099] In some embodiments, the input sequence of the multi-head attention in the first layer encoder includes the representation vector and represents vector .

[0100] For example, the input sequence With three different weight matrices , , Multiply to get the query ,key Sum .in, ; ; .

[0101] If the shared encoder processes the representation vector , then the input sequence of the first layer encoder is the representation vector ; If the shared encoder processes the representation vector , then the input sequence of the first layer encoder is the representation vector .

[0102] Each head of the multi-head attention in the first layer encoder focuses on different positions of the input sequence with different weights, and the output corresponding to each head is fused through splicing and linear transformation to obtain the self-attention output.

[0103] For example, by calculating the eight attention heads in the first layer encoder, each head pays attention to different positions with different weights, and the output of the eight-head self-attention mechanism is obtained. Finally, the results of the eight-head attention are fused through splicing and linear transformation to obtain the final self-attention output.

[0104] In some embodiments, the final self-attention output It can be expressed by formula 1:

[0105]

[0106] in, For transposition, is the dimension of the query vector.

[0107] The residual connection in the first encoder layer adds the input sequence to the self-attention output and normalizes it.

[0108] For example, after the eight-head self-attention mechanism, a residual connection can be used to convert the input sequence And the final self-attention output Then the result of the addition is normalized to obtain the output of the residual connection.

[0109] For input The normalization of can be expressed by formula 2:

[0110]

[0111] in, and are the learned scaling and translation parameters, and The input The mean and standard deviation along a particular dimension.

[0112] Accordingly, in this embodiment, the output of the residual connection It can be expressed by formula 3:

[0113]

[0114] The input of the feedforward neural network is the output of the residual connection. The feedforward neural network includes two linear transformations and a nonlinear activation function. The output of the feedforward neural network in the last layer of the encoder includes the feature vector and the eigenvector .

[0115] For example, for the input , the output of the feedforward neural network can be expressed by equation 4:

[0116]

[0117] Accordingly, in this embodiment, the output of the feedforward neural network can be expressed by Formula 5:

[0118]

[0119] in, This is the output of the shared encoder. If the input sequence is represented by the vector , then the output of the shared encoder is the feature vector ; If the input sequence is a representation vector , then the output of the shared encoder is the feature vector .

[0120] In some embodiments, the generative model is obtained by training a Transformer model based on class-level source code text samples.

[0121] The two-stage shared parameter learning methods include:

[0122] The first stage: based on the representation vectors of the class-level source code sequences of the class-level source code text samples, the parameters of the shared encoder in the Transformer model are updated.

[0123] The second stage: based on the representation vector of the summary of the method code snippet of the class-level source code text sample, update the updated parameters of the shared encoder in the first stage Transformer model.

[0124] That is to say, training the Transformer model can be divided into two stages: the first stage and the second stage. In the first stage, the Transformer model learns to obtain the representation vector Encode to get class-level source code feature vector The second stage is mainly to learn the Transformer model to obtain the encoding ability based on the representation vector Encoding method summary feature vector Coding ability.

[0125] In order to distinguish the Transformer model and the generated model before and after training, it can be said that the Transformer model before training mainly includes an encoder and a decoder.

[0126] For example, in the first stage, the representation vector of the class-level source code sequence of the class-level source code text sample is input to the encoder, that is, the query vector of the first stage ,key Sum The vectors of the class-level source code sequences of the class-level source code text samples are calculated. The encoder gradually learns the semantic representation through multiple layers and updates the parameters during the learning process. The output of this stage will be passed to the decoder for subsequent decoding.

[0127] The second stage uses the parameters learned in the first stage, that is, the encoder in this stage is the encoder whose parameters have been updated in the first stage. In the second stage, the representation vector of the summary of the method code snippet of the class-level source code text sample is input into the encoder trained in the first stage, that is, the query vector of the second stage ,key Sum They are all calculated from the representation vector of the summary of the method code snippet of the class-level source code text sample. In the second stage, the parameters of the encoder updated in the first stage are continued to be updated to achieve two-stage shared parameter learning, that is, the two-stage parameter sharing of the encoder. Similarly, the output of the encoder in this stage will also be passed to the decoder for further processing.

[0128] In this embodiment, through two-stage training and learning, the relevant information of the source code and method summary is learned in the encoder with shared parameters to better capture the relationship between them. Compared with the traditional method using two independent encoders, the shared parameter approach facilitates the information interaction between the source code and the method summary, so that the feature representation learned by the encoder is more capable of converting between different inputs and better capturing the semantic relationship between the two, thereby improving the generalization ability of the model. In addition, the two independent encoders need to be trained independently, which requires more training time and computing resources. This model has only one encoder for two-stage learning, and the model structure is relatively simple, with a small number of parameters, which can be easier to train and tune.

[0129] In some embodiments, the joint decoder includes a plurality of identical decoder layers, and the plurality of identical decoder layers are stacked; each decoder layer includes, in sequence: a class summary self-attention layer, a source code attention layer, a method summary attention layer, and a feed-forward neural network layer;

[0130] The class summary self-attention layer is used to provide context information for generating the class-level code summary, and the source code attention layer is used to fuse the feature vector , Method Summary The attention layer is used to fuse feature vectors .

[0131] Exemplarily, the joint decoder is formed by stacking multiple identical decoder layers. Figure 4 As shown, each decoder layer consists of 4 parts: class summary self-attention layer, source code attention layer, method summary attention layer and feed-forward neural network layer.

[0132] In some embodiments, the input of the class summary self-attention layer in the first decoder layer includes the class summary representation vector of the class-level source code file . Where, represents the vector The joint decoder uses the previously generated output as the input this time based on autoregression.

[0133] Exemplary, combined Figure 4 It can be seen that the class summary self-attention layer includes masked multi-head attention and residual connection. The input of the residual connection includes the output of the masked multi-head attention and the class summary representation vector .

[0134] Class Summary The self-attention layer uses a masked multi-head (e.g., eight-head) self-attention mechanism to process the representation vector , to ensure that the generative model can only rely on the generated data and will not use future data. The role of this attention mechanism is to provide summary-level contextual information for the generation of class-level code summaries. The output of the class summary self-attention layer It can be expressed by formula 6:

[0135]

[0136] in, is layer normalization, It is a self-attention mechanism with mask.

[0137] The input of the source code attention layer in the first decoder layer consists of the feature vector and the output of the class summary self-attention layer in the first decoder layer.

[0138] Continue reading Figure 4 , the source code attention layer includes multi-head attention and residual connection. The input of the residual connection includes the feature vector after multi-head attention The output of the self-attention layer and the class summary .

[0139] The source code attention layer can use multiple (e.g. eight) heads to focus on the feature vector The output of the self-attention layer and the class summary Perform attention calculation. The function of the source code attention layer is to convert the feature vector Fuse into the Transformer model to get the output vector The output vector It can be expressed by formula 7:

[0140]

[0141] Method Summary The input of the attention layer in the first layer decoder consists of the feature vector and the output of the source code attention layer in the first decoder layer.

[0142] Continue reading Figure 4 ,Method Summary The attention layer includes multi-head attention and residual connection. The input of the residual connection includes the feature vector after multi-head attention and the output of the source code attention layer .

[0143] Method Summary The attention layer can use a multi-head (e.g., eight-head) attention mechanism to the output of the source code attention layer. and the eigenvector Do attention calculation and get the output vector The output vector It can be expressed by formula 8:

[0144]

[0145] The output of the feed-forward neural network layer in the last encoder layer includes the class-level code summary.

[0146] Continue reading Figure 4 , the feedforward neural network layer includes a feedforward neural network and a residual connection. The input of the feedforward neural network is the output of the method summary attention layer. The input of the residual connection includes the output of the method summary attention layer and the output of the feedforward neural network.

[0147] The feed-forward neural network layer can take the output of the method summary attention layer Put it into the feedforward neural network for nonlinear transformation to get the final output of the joint decoder Output It can be expressed by formula 9:

[0148]

[0149] Based on the above analysis, the joint decoder can perform feature selection from multiple angles, so that the generative model can obtain information more comprehensively, thereby improving the accuracy and reliability of generating class-level code summaries. In addition, each layer of the decoder introduces corresponding residual connections and normalization operations after each operation part, which can prevent the gradient from disappearing, thereby improving the stability of the generative model and ensuring its effectiveness when processing complex data. Moreover, by superimposing multiple layers of decoders, the joint decoder can understand the input data more deeply, thereby improving the overall performance.

[0150] It is understandable that, if in the training phase, when the summary is obtained based on the softmax function prediction of the Transformer model, the parameters of the Transformer model can be updated by minimizing the cross entropy as the loss function to obtain the generative model. It can be expressed by formula 10:

[0151]

[0152] in, is the predicted probability of each position in the predicted summary by the Transformer model, is the true label of the corresponding position.

[0153] According to an embodiment of the present disclosure, the present disclosure also provides a device for generating a class-level code summary.

[0154] See also Figure 5 , Figure 5 Schematic diagram of a device for generating a class-level code summary provided by an embodiment of the present disclosure. Figure 5 As shown, the class-level code summary generation device 500 includes:

[0155] An acquisition unit 501 is used to obtain a class-level source code file;

[0156] The acquisition unit 501 is also used to obtain the class-level source code sequence and the method code fragment of each member method from the class-level source code file;

[0157] A generating unit 502, used to generate representation vectors of the class-level source code sequence respectively , and a representation vector of the summary of the method code snippet ;

[0158] Prediction unit 503, used to transform the representation vector and the representation vector Inputting into a pre-trained class-level code summary generation model, predicting and obtaining a class-level code summary corresponding to the class-level source code file;

[0159] The base model of the generative model is a Transformer model, and the generative model includes a shared encoder and a joint decoder. The shared encoder is trained by learning the encoding ability by sharing parameters in two stages, and the joint decoder is trained by learning the decoding ability by taking the outputs of the two stages of the shared encoder as inputs. The two stages include learning to obtain the representation vector based on the representation vector. Encode to get class-level source code feature vector The encoding capability of the stage, and the representation vector based on Encoding method summary feature vector stage of coding ability.

[0160] In some embodiments, the generating unit 502 is used to simplify the class-level source code sequence based on the class-level code text simplification algorithm, and perform word embedding on the simplified class-level source code sequence to generate the representation vector , and generate a summary of the method code snippet based on CodeT5, and use word embedding to generate the representation vector .

[0161] In some embodiments, the shared encoder includes multiple identical encoder layers, and the multiple identical encoder layers are superimposed; each encoder layer includes multi-head attention, residual connection, and feedforward neural network.

[0162] In some embodiments, the input sequence of the multi-head attention in the first layer encoder includes the representation vector and represents vector ;

[0163] Each head of the multi-head attention in the first layer encoder focuses on different positions of the input sequence with different weights, and the output corresponding to each head is fused to obtain a self-attention output through concatenation and linear transformation;

[0164] The residual connection in the first layer encoder adds the input sequence to the self-attention output and performs normalization;

[0165] The input of the feedforward neural network is the output of the residual connection. The feedforward neural network includes two linear transformations and a nonlinear activation function. The output of the feedforward neural network in the last layer of the encoder includes the feature vector and the eigenvector .

[0166] In some embodiments, the generative model is obtained by training the Transformer model based on class-level source code text samples;

[0167] The two-stage shared parameter learning method includes:

[0168] The first stage: based on the representation vector of the class-level source code sequence of the class-level source code text sample, updating the parameters of the shared encoder in the Transformer model;

[0169] The second stage: based on the representation vector of the summary of the method code snippet of the class-level source code text sample, updating the updated parameters of the shared encoder in the Transformer model of the first stage.

[0170] In some embodiments, the joint decoder includes a plurality of identical decoder layers, and the plurality of identical decoder layers are stacked; each decoder layer includes, in sequence: a class summary self-attention layer, a source code attention layer, a method summary attention layer, and a feed-forward neural network layer;

[0171] The class summary self-attention layer is used to provide context information for generating the class-level code summary, and the source code attention layer is used to fuse the feature vector , the method summarizes that the attention layer is used to fuse the feature vector .

[0172] In some embodiments, the input of the class summary self-attention layer in the first layer decoder includes the class summary representation vector of the class-level source code file ;

[0173] The input of the source code attention layer in the first layer decoder includes the feature vector and the output of the class summary self-attention layer in the first layer decoder;

[0174] The input of the attention layer in the first layer decoder method summary includes the feature vector and the output of the source code attention layer in the first layer decoder;

[0175] The output of the feed-forward neural network layer in the last encoder layer includes the class-level code summary.

[0176] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0177] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0178] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0179] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0180] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0181] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the generation method of the class-level code summary. For example, in some embodiments, the generation method of the class-level code summary may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the generation method of the class-level code summary described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the method for generating a class-level code summary in any other appropriate manner (for example, by means of firmware).

[0182] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0183] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0184] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0185] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0186] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0187] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0188] It should be understood by those skilled in the art that the embodiments of the present disclosure may be provided as methods, devices, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0189] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, apparatuses (devices), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0190] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0191] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0192] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for generating a class-level code summary, characterized in that: The method comprises: Get the class-level source code file; From the class-level source code file, respectively obtain a class-level source code sequence and a method code snippet of each member method; Generate representation vectors of the class-level source code sequences respectively , and a representation vector of the summary of the method code snippet ; The representation vector and the representation vector Inputting into a pre-trained class-level code summary generation model, predicting and obtaining a class-level code summary corresponding to the class-level source code file; The base model of the generative model is a Transformer model, and the generative model includes a shared encoder and a joint decoder. The shared encoder is trained by learning the encoding ability by sharing parameters in two stages, and the joint decoder is trained by learning the decoding ability by taking the outputs of the two stages of the shared encoder as inputs. The two stages include learning to obtain the representation vector based on the representation vector. Encode to get class-level source code feature vector The encoding capability of the stage, and the representation vector based on Encoding method summary feature vector The shared encoder comprises a plurality of identical encoder layers, and the plurality of identical encoder layers are superimposed; each encoder layer comprises multi-head attention, residual connection, and feedforward neural network, and the input sequence of the multi-head attention in the first layer encoder comprises the representation vector and represents vector , each head in the multi-head attention in the first layer encoder focuses on different positions of the input sequence with different weights, and the output corresponding to each head is fused to obtain the self-attention output through splicing and linear transformation. The residual connection in the first layer encoder adds the input sequence to the self-attention output and performs normalization. The input of the feedforward neural network is the output of the residual connection. The output of the feedforward neural network in the last layer encoder includes the feature vector and the eigenvector .

2. The method according to claim 1, characterized in that The representation vectors of the class-level source code sequences are generated respectively , and a representation vector of the summary of the method code snippet ,include: The class-level source code sequence is simplified based on the class-level code text simplification algorithm, and the simplified class-level source code sequence is word-embedded to generate the representation vector ; Generate a summary of the method code snippet based on CodeT5, and use word embedding to generate the representation vector .

3. The method according to claim 1, characterized in that The feedforward neural network includes two linear transformations and a nonlinear activation function.

4. The method according to claim 1, characterized in that: The generation model is obtained by training the Transformer model based on class-level source code text samples; The two-stage shared parameter learning method includes: The first stage: based on the representation vector of the class-level source code sequence of the class-level source code text sample, updating the parameters of the shared encoder in the Transformer model; The second stage: based on the representation vector of the summary of the method code snippet of the class-level source code text sample, updating the updated parameters of the shared encoder in the Transformer model of the first stage.

5. The method according to claim 1, characterized in that The joint decoder comprises a plurality of identical decoder layers, and the plurality of identical decoder layers are stacked; each decoder layer comprises in sequence: a class summary self-attention layer, a source code attention layer, a method summary attention layer and a feedforward neural network layer; The class summary self-attention layer is used to provide context information for generating the class-level code summary, and the source code attention layer is used to fuse the feature vector , the method summarizes that the attention layer is used to fuse the feature vector .

6. The method according to claim 5, characterized in that The input of the class summary self-attention layer in the first decoder layer includes the class summary representation vector of the class-level source code file ; The input of the source code attention layer in the first layer decoder includes the feature vector and the output of the class summary self-attention layer in the first layer decoder; The input of the attention layer in the first layer decoder method summary includes the feature vector and the output of the source code attention layer in the first layer decoder; The output of the feed-forward neural network layer in the last encoder layer includes the class-level code summary.

7. A device for generating a class-level code summary, characterized in that: include: An acquisition unit, used to obtain a class-level source code file; The acquisition unit is also used to obtain, from the class-level source code file, a class-level source code sequence and a method code fragment of each member method; A generation unit, used to generate representation vectors of the class-level source code sequences respectively , and a representation vector of the summary of the method code snippet ; A prediction unit is used to transform the representation vector and the representation vector Inputting into a pre-trained class-level code summary generation model, predicting and obtaining a class-level code summary corresponding to the class-level source code file; The base model of the generative model is a Transformer model, and the generative model includes a shared encoder and a joint decoder. The shared encoder is trained by learning the encoding ability by sharing parameters in two stages, and the joint decoder is trained by learning the decoding ability by taking the outputs of the two stages of the shared encoder as inputs. The two stages include learning to obtain the representation vector based on the representation vector. Encode to get class-level source code feature vector The encoding capability of the stage, and the representation vector based on Encoding method summary feature vector The shared encoder comprises a plurality of identical encoder layers, and the plurality of identical encoder layers are superimposed; each encoder layer comprises multi-head attention, residual connection, and feedforward neural network, and the input sequence of the multi-head attention in the first layer encoder comprises the representation vector and represents vector , each head in the multi-head attention in the first layer encoder focuses on different positions of the input sequence with different weights, and the output corresponding to each head is fused to obtain the self-attention output through splicing and linear transformation. The residual connection in the first layer encoder adds the input sequence to the self-attention output and performs normalization. The input of the feedforward neural network is the output of the residual connection. The output of the feedforward neural network in the last layer encoder includes the feature vector and the eigenvector .

8. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, and the computer program is used to cause the processor to execute the method according to any one of claims 1 to 6.

9. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images