A method and device for generating pull request description

Abstract

The present invention relates to a method and device for generating a pull request description, which belongs to the technical field of pull requests and solves the problem that only the information of a new pull request itself is considered when generating a pull request description, and the historical data of several pull requests are not fully utilized. The method includes: obtaining multiple pull requests and corresponding descriptions from an open source community; generating a code syntax change tree based on each pull request; generating a change vector based on the code syntax change tree; calculating the similarity of the pull requests based on multiple change vectors; obtaining a description generation model using an input pull request and similar pull requests; and using the description generation model to calculate the description of the input pull request using the descriptions of multiple similar pull requests and the text information contained in the input pull request. The change vector effectively extracts the grammatical structure information of the code change, so as to automatically generate a pull request description using a pull request similar to the new pull request.

Description

Technical Field

[0001] The present invention belongs to the technical field of pull requests, and in particular, relates to a method and device for generating a pull request description. Background Art

[0002] The open source community, also known as the open source community, is a platform for developers to release software source code in accordance with the corresponding open source software license agreement. GitHub is currently the world's largest open source community and the world's largest hosting platform for open source and private software projects. In GitHub, developers conduct distributed collaborative development on their respective branches through the pull request mechanism, complete the modification and improvement of the project, and package the detailed code modifications to generate a pull request (the pull request includes text content and code changes, and optionally includes a text description part). However, since it is time-consuming and laborious for developers to manually fill in the text description, some pull requests do not have a description filled in. For pull requests without descriptions, in order to decide whether to accept the pull request, the project management team needs to understand the developer's intention by reading the pull request source code, which increases the workload of the management team. At the same time, due to the lack of correspondence between software functions and codes, it is not conducive to future code maintenance work.

[0003] With the popularity and rapid development of the open source community, the growth rate of pull requests far exceeds the expansion rate of the project management team. This inconsistency in the number of growth has led to an increasingly heavy workload for the project management team. At this time, pull requests without descriptions may be ignored or directly rejected by the project management team because they are relatively difficult to understand, which will dampen the developer's enthusiasm for contribution and is not conducive to the long-term development of the open source community. Therefore, a method for automatically generating pull request descriptions is needed. When developers submit pull requests, the description of the pull request can be automatically generated to reduce the adverse effects caused by the lack of descriptions and improve the efficiency of software iteration.

[0004] To realize the automatic generation of pull request descriptions, it is necessary to utilize the text content and code change information contained in the pull request. Most of the existing technologies use this information of past pull requests to train the model. After training, these historical data are no longer used directly, and only indirectly affect the generation of new pull request descriptions through the parameters learned by the model. Although this processing is simple and convenient, it cannot fully utilize the historical data of past pull requests, so the generation effect of the description is relatively limited. Second, the existing technology mostly utilizes the code change part of the pull request in the form of symbol sequence, which cannot make good use of the hierarchical information of the source code. Third, the text content of the pull request may contain some parts that are not usually written into the pull request description. The existing technology does not clean up these parts enough and relies too much on model learning, which increases the difficulty of model training and also affects the generation effect of the description.

[0005] After summarizing, existing research shows that a long code review cycle will dampen the enthusiasm of developers, and providing pull request descriptions can help shorten the code review cycle and increase the probability of pull requests being accepted. Therefore, there are currently the following shortcomings:

[0006] 1. There is a lack of an automated pull request description generation method. When a developer submits a pull request, the pull request description can be automatically generated to reduce the adverse effects caused by the lack of description, improve software iteration efficiency, and improve the open source ecological environment.

[0007] 2. Existing technologies do not make sufficient use of historical data of pull requests. These data are only used to train the model and are not used again after training. The generated effect is limited by the parameters learned by the model and is greatly affected by the model design and training process.

[0008] 3. The existing technology does not accurately utilize the code modification part of the pull request, and cannot make good use of the hierarchical information of the source code in the form of symbol sequences.

[0009] 4. The existing technology does not adequately clean up the parts of the text content of the pull request that are usually not written into the pull request description, and relies too much on model learning, which increases the difficulty of model training and also affects the generation effect of the description. Summary of the invention

[0010] In view of the above analysis, the present invention aims to provide a method and device for generating a pull request description, so as to solve the problem that only the information of the new pull request itself is considered when generating the pull request description, the historical data of the pull request is not fully utilized, and the code modification part of the pull request is not used accurately, resulting in inaccurate automatic generation of the pull request description.

[0011] In one aspect, the present invention provides a method for generating a pull request description, which specifically comprises the following steps:

[0012] Obtain multiple merged pull requests from the open source community, and obtain the text vector, code syntax change tree, and description vector of each pull request;

[0013] Establishing a tree structure autoencoder model, and training the tree structure autoencoder model based on the multiple code grammar modification trees to obtain a modification coding model;

[0014] Based on the modified coding model, calculating the similarity between the pull requests, and for each pull request, searching for a number of pull requests with the highest similarity;

[0015] Establishing an encoder-decoder model, training the encoder-decoder model based on the text vector of each pull request, the description vectors of several pull requests with the highest similarity to the pull request, and the description of the pull request to obtain a description generation model;

[0016] Obtain a pull request for a description to be generated, and based on the pull request for the description to be generated, use the modified coding model to search for several pull requests with the highest similarity to the pull request for the description to be generated; based on the text vector of the pull request for the description to be generated and the description vectors of several pull requests with the highest similarity to the pull request for the description to be generated, use the description generation model to generate a description of the pull request for the description to be generated.

[0017] Furthermore, each of the pull requests includes: a text portion, a code change portion, and a description portion; wherein the pull request text portion includes: multiple submissions included in the pull request, submission instructions for each submission, and comments included in the pull request.

[0018] Further, obtaining the pull request text vector includes:

[0019] performing text preprocessing on the text portion of the pull request using a natural language toolkit;

[0020] The preprocessed pull request text is converted into the text vector using OneHot encoding.

[0021] Further, obtaining the pull request description vector includes:

[0022] Performing text preprocessing on the description part of the pull request by using a natural language toolkit;

[0023] Obtaining a pull request template of the software project to which the pull request belongs, and using the Smith-Waterman algorithm to calculate a matching ratio between the preprocessed pull request description and the template;

[0024] When the matching ratio reaches or exceeds a threshold, the portion of the preprocessed pull request description that matches the template is removed;

[0025] The pull request description after removing the portion matching the template is converted into the description vector using OneHot encoding.

[0026] Furthermore, the performing text preprocessing on the text portion or the description of the pull request by using a natural language toolkit includes:

[0027] Delete HTML tags, comments and Markdown formatting control characters;

[0028] Delete the checklist;

[0029] Delete sentences containing URLs, email addresses, internal links, user mentions, or user signatures;

[0030] Replace numbers and version numbers in text;

[0031] Remove the merge commit;

[0032] Remove commits added after the pull request was created;

[0033] Remove the commit tag.

[0034] Further, the obtaining of the code syntax change tree includes:

[0035] Parse the code before and after the change into abstract syntax trees respectively;

[0036] Use the GumTree algorithm to calculate the correspondence between the abstract syntax tree nodes before and after the change;

[0037] Based on the corresponding relationship, the change type of the abstract syntax tree is determined, and a change tree node is generated based on the change type. The change tree nodes are combined into a tree shape according to the structure of the abstract syntax tree before the change to form a code syntax change tree; wherein the change types include update, delete, insert, move, and no change.

[0038] Furthermore, the step of establishing a tree structure autoencoder model and training the tree structure autoencoder model based on the multiple code grammar modification trees to obtain a modified coding model includes:

[0039] The encoder uses a tree-structured long short-term memory network Tree-LSTM to aggregate the information of each node of the grammatical change tree into an encoding vector from bottom to top;

[0040] The decoder uses two linear long short-term memory networks (LSTMs) to decode the encoded vector from top to bottom into a tree structure.

[0041] Based on the multiple code grammar change trees, a self-supervised training method is used so that the tree structure can restore the input code grammar change tree.

[0042] Furthermore, the calculating of the similarity between the pull requests comprises: encoding two pull requests using the tree structure autoencoder model to obtain two encoding vectors corresponding to the two pull requests respectively, calculating the cosine of the angle between the two encoding vectors, and judging the similarity between the two pull requests based on the size of the cosine of the angle.

[0043] Furthermore, the encoder-decoder model includes:

[0044] The encoder uses two bidirectional long short-term memory networks Bi-LSTM to encode the text vector of each pull request and the description vectors of several pull requests with the highest similarity to the pull request into vectors respectively;

[0045] The context vectors obtained by encoding the description vectors of several pull requests with the highest similarity to each of the pull requests are weighted by the corresponding similarity, and the context vectors obtained by encoding the text vector of the pull request are merged into one vector using the LogSumExp function;

[0046] The decoder uses a beam search method and a unidirectional long short-term memory network with an attention mechanism and a pointer generation network to decode the fused vector into a description of the pull request to be generated.

[0047] On the other hand, the present invention also provides a pull request description generating device, comprising:

[0048] A data acquisition module, used to obtain pull requests from the open source community, and based on each pull request, obtain a text vector, a code syntax change tree, and a description vector of each pull request;

[0049] A similarity pull request calculation module is used to establish a tree structure autoencoder model, train the tree structure autoencoder model based on a plurality of code syntax change trees to obtain a change coding model; calculate the similarity between the pull requests based on the change coding model, and for each pull request, find a number of pull requests with the highest similarity;

[0050] A description generation model training module is used to establish an encoder-decoder model, and train the encoder-decoder model based on the text vector of each pull request, the description vectors of several pull requests with the highest similarity to the pull request, and the description of the pull request to obtain a description generation model;

[0051] A pull request description generation module is used to obtain a pull request for which a description is to be generated, and based on the pull request for which a description is to be generated, use the modified coding model to search for several pull requests with the highest similarity to the pull request for which a description is to be generated; based on the text vector of the pull request for which a description is to be generated and the description vectors of several pull requests with the highest similarity to the pull request for which a description is to be generated, use the description generation model to generate a description of the pull request for which a description is to be generated.

[0052] The present invention can achieve at least one of the following beneficial effects:

[0053] 1. By extracting the abstract syntax trees before and after the code change part of the pull request, calculating the code syntax change tree, effectively extracting the code change structure in the pull request, generating the change vector, and fully extracting the change features contained in the pull request.

[0054] 2. Based on the code syntax change tree, a tree structure autoencoder is designed, and a more complete code change encoding method is proposed. Compared with using the code change part of the pull request in the form of symbol sequence, this method can more effectively utilize the hierarchical structure information of the source code and provide technical support for the automatic generation of pull request descriptions.

[0055] 3. A clear and reasonable pull request dataset is built based on the pull request text part, description part and descriptions of similar pull requests. The writing method of pull request description is trained based on the encoder-decoder neural network model, which improves the accuracy of model training and the generation effect of pull description.

[0056] 4. For a newly created pull request, select several pull requests with the closest code changes to it to make more effective use of historical data. Refer to the descriptions of several pull requests with high similarity to summarize the text part of the newly created pull request, which can assist or replace developers in writing the description of the newly created pull request, reduce the adverse effects caused by the lack of description, improve software iteration efficiency, and improve the open source ecosystem.

[0057] Other features and advantages of the present invention will be described in the following description, and some advantages may become apparent from the description, or may be understood by practicing the present invention. The purpose and other advantages of the present invention may be realized and obtained through the contents particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0059] Figure 1 A flowchart of a method for generating a pull request is described in accordance with an embodiment of the present invention.

[0060] Figure 2 A diagram of a pull request description generating apparatus according to an embodiment of the present invention.

[0061] Figure 3 Schematic diagram of a tree structured autoencoder model according to an embodiment of the present invention.

[0062] Figure 4 Schematic diagram of an encoder-decoder model according to an embodiment of the present invention.

[0063] Figure 5 A diagram of a pull request description generation model PRSim according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0065] Method Embodiment

[0066] Example 1

[0067] A specific embodiment of the present invention discloses a method for generating a pull request description, which specifically includes the following steps:

[0068] Step S102: Obtain multiple merged pull requests from the open source community, and obtain the text vector, code syntax change tree, and description vector of each pull request. Merged pull requests refer to pull requests that have passed the review and been approved by the administrator; the open source community also includes rejected pull requests and those that are in the review process.

[0069] Preferably, the research object of this embodiment is the pull request data of real projects on the GitHub platform, and the pull request data is crawled using the GitHub API. Pull requests that have been merged in projects with a high number of pull requests on GitHub are preferentially selected.

[0070] You can also get the pull request submitted to the open source project through the URL "https: / / api.github.com / repos / {owner} / {repo} / pulls", where {owner} represents the project owner and {repo} represents the project name. You can get the number of all pull requests in the project, whether they have been merged, the creation and merge time, description text and other information. Then you can get the commits contained in the pull request through the URL "https: / / api.github.com / repos / {owner} / {repo} / pulls / {pull_number} / commits", where {pull_number} represents the pull request number, and you can get the commit instructions and corresponding commit hashes contained in the pull request. You can get the code changes of the pull request through the URL "https: / / github.com / {owner} / {repo} / pull / {pull_number}.diff", which are stored in the form of diff.

[0071] In one embodiment, the text vector, code syntax change tree and description vector of each pull request are directly obtained according to the obtained multiple pull requests, that is, the step 102 includes the following steps S102-2 to S102-4;

[0072] To improve the quality of pull requests, in a preferred embodiment, after obtaining multiple pull requests, the pull requests are filtered using code submission, and the text vector, code syntax change tree and description vector of each pull request are obtained according to the filtered pull requests, that is, step 102 includes the following steps S102-1 to S102-4:

[0073] Step S102-1: Filter code submission for each pull request, specifically including:

[0074] Delete merge commits: Optionally, delete commit messages that start with "Merge branch", "Merge remote-tracking branch", etc. Such commits are generally used in pull requests to resolve merge conflicts with upstream branches, and the commit message format is fixed, and generally does not involve the specific content of the pull request.

[0075] Delete the submissions added after the pull request is created: Some pull requests require multiple communications and modifications with the project management team before they can be accepted, during which new submissions will be generated. However, this embodiment is aimed at generating descriptions when creating pull requests. Only submissions created before the pull request is created are used to generate descriptions, and submissions created after the pull request is deleted.

[0076] Delete submission tag: Optional. In the submission description "feat(api):add payload drop down event", "feat" indicates that this submission adds new features, "(api)" indicates that the scope of this submission is the program interface, and the part after the colon is a summary of the actual changes. This example focuses on specific changes and does not particularly care about the type and scope of the changes, so the colon and the part before it are deleted.

[0077] To improve the generation of pull descriptions, it is preferred to delete pull requests that contain less than 2 or more than 20 commits. If a pull request contains only one commit, the commit description of this commit can be used directly as the pull request description without generating one; while pull requests containing more than 20 commits are usually used to synchronize two code repositories, rather than developers making contributions.

[0078] Step S102 - 2 : obtaining a text vector of each pull request, including performing text preprocessing on the text portion by using a natural language toolkit and converting the preprocessed pull request text into the text vector by using OneHot encoding.

[0079] The pull request text includes multiple commits contained in the pull request, the commit message of each commit, and the comments contained in the pull request. Each pull request has several commit messages and comments, where each commit message is for a commit and generally summarizes the role of this commit; each comment is for a piece of code and generally describes the function or implementation of this code.

[0080] The text portion of the pull request is preprocessed using the Natural Language Toolkit.

[0081] First, clean up the text portion of the pull request to include some things that are not usually included in pull request descriptions, specifically:

[0082] Delete HTML tags, comments, and Markdown format control characters. These contents are used to control the text format. Since the present invention focuses on the text content rather than the format, these format characters are deleted. For example, the HTML tag " … " and the Markdown format control characters for italics "*...*" both delete the symbols and retain the text inside.

[0083] Remove Markdown checklists. Checklists are usually general tips on how to complete a pull request, such as "pass all tests", which are not closely related to the changes in a specific pull request, so remove the checklists. For example, lines starting with "-[]" or "-[X]" are removed.

[0084] Remove sentences containing URLs, email addresses, internal links, user mentions, or user signatures. These usually do not describe the changes made by the pull request and will introduce a large number of low-frequency words, affecting model training.

[0085] Replace numbers and version numbers in text. Replace numbers with "0", version numbers (such as "3.4.2") with the fixed word "version", and git version hashes (such as "a9cb0d7") with the fixed word "sha". These contents are usually project-specific and are only used for unique identification, and do not have much meaning in themselves.

[0086] In order to clean up some content that may not be usually written into the pull request description in the text part of the pull request, so as to reduce the difficulty of model training and improve the description generation effect, preferably, if the submission instructions and newly added comments after text preprocessing have less than 5 words in total, such pull request is deleted.

[0087] Secondly, for the text part of the pull request after text preprocessing, specify the dictionary size, that is, the number of different words to be retained, and give priority to retaining words with high frequency. Low-frequency words that exceed the dictionary size are specially marked " <unk>". Convert the pull request text part into a vector using OneHot encoding. OneHot encoding means that the first word in the dictionary is encoded as "1000...0", the second word is encoded as "0100...0", and so on, and the last word is encoded as "0000...1".

[0088] Step S102-3: Obtain a pull request description vector;

[0089] Specifically, this step includes the following contents:

[0090] a. Perform text preprocessing on the description part of the pull request using a natural language toolkit. The text preprocessing process is the same as the text preprocessing process in step S102 - 1 .

[0091] b. Obtain a pull request template of the software project to which the pull request belongs, and use the Smith-Waterman algorithm to calculate a matching ratio between the preprocessed pull request text and the template.

[0092] Specifically, some projects will provide templates for pull request descriptions, and developers can fill in the templates with the content that needs to be supplemented to complete a pull request description. At this time, the pull request description consists of two parts, the content copied from the template and the content filled in by the developer. On the one hand, the template is specific to the project, which is not conducive to the generalization performance of the model; on the other hand, in actual application scenarios, the template is visible and can be identified by rules, so this embodiment only focuses on the part filled in by the developer. According to GitHub's convention, the pull request template is stored in the project repository in a file with the path ".github / PULL_REQUEST_TEMPLATE.md". The Smith-Waterman algorithm is used to calculate the matching part of the pull request description and the template, where the minimum matching unit is a word, and the ratio of the number of words in the preprocessed pull request description to the total number of words in the template is calculated.

[0093] c. when the matching ratio reaches or exceeds a threshold, removing the portion of the preprocessed pull request text that matches the template;

[0094] Specifically, if the matching ratio reaches a preset threshold, it is considered that the description is filled in based on the template, and the matching part is deleted. The threshold can be set according to actual needs.

[0095] d. Use OneHot encoding to convert the pull request description after removing the part matching the template into a description vector.

[0096] Step S102-4: Obtain a code syntax change tree. Specifically, parse the code before and after the change into abstract syntax trees respectively; use the GumTree algorithm to calculate the correspondence between the abstract syntax tree nodes before and after the change; based on the correspondence, determine the change type of the abstract syntax tree, generate change tree nodes based on the change type, and combine the change tree nodes into a tree shape according to the structure of the abstract syntax tree before the change to form a code syntax change tree; wherein the change types include update, delete, insert, move, and no change.

[0097] Among them, the GumTree algorithm is a heuristic algorithm for establishing node correspondence between two trees. The goal of the algorithm is to find a matching method that can pair as many nodes as possible with each other, where only abstract syntax tree nodes with the same grammatical type can be paired with each other.

[0098] After using the GumTree algorithm to calculate the correspondence between the abstract syntax tree nodes before and after the change, the abstract syntax tree is traversed, and the change type is marked according to the matching relationship to generate a change tree node. Among them, the change types include: update, delete, insert, move, and no change. Among them, update means that the parent nodes of a pair of matching nodes match each other, but the labels of the pair of nodes are different; deletion means that a node in the abstract syntax tree before the change cannot find a node matching it in the abstract syntax tree after the change; insertion means that a node in the abstract syntax tree after the change cannot find a node matching it in the abstract syntax tree before the change; move means that the parent nodes of a pair of matching nodes do not match, or although the parent nodes match, the positions of the pair of nodes between their respective sibling nodes are different; no change means that the labels of a pair of matching nodes are the same, and their respective parent nodes also match each other.

[0099] Then, the modified tree nodes are combined into a tree according to the structure of the abstract syntax tree before the modification to form a syntax modification tree. If a subtree is composed entirely of unchanged nodes, this subtree is deleted; if an unchanged node has only one direct child node, and the subtree with it as the root is not all unchanged, this unchanged node is replaced with the direct child node and the subtree with it as the root.

[0100] Step S104: Establish a tree structure autoencoder model, and train the tree structure autoencoder model based on multiple code syntax modification trees to obtain a modification coding model.

[0101] The grammar change tree is converted into a Left-Child Right-Sibling representation; the encoder part uses a tree-structured long short-term memory network Tree-LSTM to collect the information of each node of the grammar change tree from bottom to top into a coding vector; the decoder part uses two linear long short-term memory networks LSTM to decode the coding vector from top to bottom into a tree structure; and the self-supervised training method uses the obtained multiple grammar change trees to train the tree structure autoencoder so that the decoded tree structure can restore the input grammar change tree, so that the coding vector can be used as a representative of the grammar change tree. The framework of the tree structure autoencoder model is as follows Figure 3 shown.

[0102] Specifically, the mathematical model of the encoder part tree structure long short-term memory network Tree-LSTM is as follows:

[0103] (i j ,o j ,u j ) T =(σ,σ,tanh) T (W (i,o,u) d j +b (i,o,u) +U (i,o,u) [h jl ;h jr ])

[0104]

[0105]

[0106] c j =i j ⊙u j +f jl ⊙c ji +f jr ⊙c jr

[0107] h j =o j ⊙tanhc j

[0108] Among them, d j represents the label embedding vector of the jth node. i, o, u, and f are the input gate, output gate, memory data, and forget gate of the LSTM structure respectively. W (i,o,u,f) , U (i,o,u,f) , b (i,o,u,f) are the corresponding learnable parameters, namely the node input weight matrix, the child node hidden state weight matrix, and the bias vector; h is the hidden state, and c is the unit state; the subscripts jl and jr represent the left and right child nodes of the jth node, respectively. If there is no such child node, the corresponding hidden state and unit state vectors are zero vectors; σ represents the sigmoid activation function, [;] represents the concatenation of two vectors, and ⊙ represents the element-by-element multiplication of two vectors.

[0109] Specifically, the two linear long short-term memory networks (LSTMs) in the decoder part accept the state vector of each decoding node and generate the state vectors of its left and right child nodes respectively. Subsequently, a fully connected output layer is used to predict the grammatical component category and modification category of the corresponding child node from the state vector. The decoding process starts from the root node and proceeds from top to bottom, where the hidden state and unit state of the root node are copied from the output of the encoder, and the hidden state and unit state of the remaining nodes are generated by decoding its parent node.

[0110] Step S106: Based on the modified coding model, the similarity between the pull requests is calculated, and for each pull request, a plurality of pull requests with the highest similarity are searched.

[0111] The similarity between two pull requests is calculated by using the tree structure autoencoder model to encode the two pull requests and then obtaining the cosine of the angle between the two encoded vectors. The cosine of the angle ranges from -1 to 1, where 1 represents the same direction of the vectors and -1 represents the opposite direction of the vectors. The larger the cosine of the angle, the smaller the angle between the two encoded vectors, that is, the closer they are, indicating that the two pull requests are more similar.

[0112] For each pull request, calculate the cosine of the angle between it and the change encoding vector of other pull requests, and select several other pull requests with the largest cosine of the angle as similar pull requests.

[0113] Step S108: Establish an encoder-decoder model, and train the encoder-decoder model based on the text vector of each pull request, the description vectors of several pull requests with the highest similarity to the pull request, and the description of the pull request to obtain a description generation model. Specifically, the encoder uses two bidirectional long short-term memory networks Bi-LSTM to encode the text vector of each pull request and the description vectors of several pull requests with the highest similarity to the pull request into context vectors respectively;

[0114] The context vectors obtained by encoding several pull request description vectors with the highest similarity to each of the pull requests are weighted using the corresponding similarity, and are fused with the context vectors obtained by encoding the text vector of the pull request into one vector using the LogSumExp function; the decoder uses a beam search method and a unidirectional long short-term memory network with an attention mechanism and a pointer generation network to decode the fused vector into a description of the pull request.

[0115] Specifically, the steps include:

[0116] Step S108-1: Establish an encoder, where the encoder part includes a unified embedding layer, a pull request text vector encoder, and a similar pull request description vector encoder.

[0117] The embedding layer is used to reduce the dimension of the text vector and description vector generated by OneHot encoding, ensuring that the same words are represented by the same vector in the model. Since the information density of OneHot encoding is too low, it wastes computing resources and is not conducive to capturing the relationship between related words. Matrix product dimension reduction ensures that the same words are represented by the same vector in the model, which helps the model capture the association between the two parts of the input. The matrix is ​​a trainable parameter.

[0118] The pull request text vector encoder is a bidirectional long short-term memory network Bi-LSTM, which is used to encode the text vector of the pull request to be generated into a context vector. The similar pull request description vector encoder is a Bi-LSTM independent of the pull request text vector encoder, which is used to encode the description vectors of similar pull requests into context vectors. Multiple similar pull request description vectors are uniformly encoded by the similar pull request description vector encoder without distinguishing each other.

[0119] Step S108-2: Context vectors obtained by encoding several pull request descriptions with the highest similarity to each of the pull requests are weighted using the corresponding similarity degree, and are merged with the context vector obtained by encoding the text vector of the pull request into one vector using the LogSumExp function.

[0120] The similarity is expressed by the cosine of the angle between the change vectors of the pull request and the corresponding similar pull request, and the weighting operation is vector multiplication. The fusion part formula is as follows:

[0121]

[0122] In the formula, is the attention-weighted context vector of the pull request text portion at the t-th decoding step, is the attention-weighted context vector of the i-th similar pull request description at the t-th decoding step, m i is the similarity of the ith similar pull request, i.e., the cosine of the change vector angle between it and the pull request to be generated, c t is the fused context vector.

[0123] Step S108-3: Establish a decoder, which specifically includes a unidirectional LSTM network, an attention layer, an output layer, and a pointer generation network. The decoder uses a beam search method and a unidirectional long short-term memory network with an attention mechanism and a pointer generation network to decode the fused vector into a description of the pull request to be generated.

[0124] Among them, the unidirectional LSTM network is used to generate the current state vector based on the previous word and state vector. The first decoding step inputs the fused vector output by the encoder and the special start token" <start>", each subsequent decoding step inputs the state vector generated by the previous decoding step and the word at the previous position. The formula is:

[0125] v,s t = Lstm(W ctx [u t-1 ;c t-1 ]+b ctx ,s t-1 )

[0126] In the formula, [;] represents the concatenation of two vectors, u t-1 is the embedding vector of the previous word, s t-1 and c t-1 are the LSTM state vector and attention-weighted context vector of the previous decoding step, s t is the state vector of the current decoding step, v is the output of LSTM, and W ctx and b ctx are trainable weight matrices and bias vectors.

[0127] Among them, the attention layer is used to effectively utilize the limited attention resources of the model and filter out information with high value for the current decoding step from the input information. The formula is:

[0128]

[0129]

[0130] In the formula, a t is the attention distribution vector, c t is the attention weighted context vector after this update, V is the matrix composed of the context vectors of each input position, and W v , W s and b s are trainable weight matrices and bias vectors. Quantities with subscripts 0 and i represent the vector corresponding to the text portion of the pull request and the vector corresponding to the i-th similar pull request, respectively.

[0131] The output layer is used to predict the word generated in the current decoding step based on the decoding state vector. It consists of two fully connected layers and one softmax layer. The formula is:

[0132] P vocab =softmax(W2(W1[v;c t ]+b1)+b2)

[0133] In the formula, P vocab It is a one-dimensional vector with a length equal to the dictionary size, representing the probability of each word generated by decoding. W1, W2, b1, b2 are trainable weight matrices and bias vectors. The subscripts 1 and 2 represent two fully connected layers, respectively.

[0134] Among them, the pointer generation network is used to copy words from input to output, so as to generate out-of-dictionary words, so as to be able to use out-of-dictionary words such as identifiers in the input to improve the generation effect. In particular, a pull request may be selected as a sample input as long as the code change structure is similar to the pull request to be generated. There is no necessary connection between the out-of-dictionary words contained in the two, so the pointer generation network in the present invention only considers the words contained in the pull request to be generated itself. The formula is:

[0135] p=σ(W g [c t ;s t ;u t-1 ]+b g )

[0136]

[0137] P=pP vocab +(1-p)P copy

[0138] In the formula, P copy is a one-dimensional vector of length equal to the size of the dictionary, representing the probability of each word being copied from the input, where input(j) is the jth word in the pull request text portion, is the attention weight of the jth word in the pull request text part in the tth decoding step, and p is the generation probability P vocab The weight of σ represents the sigmoid activation function, W g and b g is the trainable weight matrix and bias value, and P is the final predicted probability distribution.

[0139] Preferably, the decoding strategy of this embodiment uses a beam search method when generating a description. This search method can greatly save search time and space overhead compared to a complete breadth-first search or a best-first search, and can avoid falling into a local optimal solution compared to a greedy search.

[0140] Specifically, a positive integer B greater than 1 is pre-specified, which is called the cluster width. From the special start mark " <start>"Start by predicting the probability distribution of the first word through the above decoder, and take the B words with the highest probability as candidates; then, for each candidate, use the decoder to predict the next word, and take the B sequences with the highest probability from all the results as new candidates. If any of the candidates generates a special termination flag" <stop>", then move it from the candidate list to the candidate result list; repeat the above steps until B items are collected in the candidate result list, and finally select the one with the highest probability from the candidate result list as the final generated result.

[0141] Step S108-4: Train the encoder-decoder model based on the text vector of each pull request, the description vectors of several pull requests with the highest similarity to the pull request, and the description of the pull request to obtain a description generation model. Preferably, the training strategy is: in the training phase, specify a teacher forcing ratio, randomly select samples of this ratio from all training samples and set them in the forcing teaching mode, and the remaining samples are in the autoregressive mode. For each group of training samples, enter a special start marker " <start>", the subsequent steps input the previous word in sequence, where the sample input of the forced teaching mode is the word with the expected result at the previous position, and the sample input of the autoregressive mode is the output generated by the model of the previous decoding step. Each decoding step obtains the generation probability distribution of a position, and calculates the sum of the negative logarithms of the corresponding probabilities of the correct words at all positions, that is, the cross entropy loss function. The gradient descent of this loss function is performed to iterate the model.

[0142] Preferably, a pull request generation model of this embodiment is PRSim, and the model framework is as follows Figure 5 shown.

[0143] Step S110: obtaining a pull request for a description to be generated, and based on the pull request for the description to be generated, using a modified coding model to search for several pull requests with the highest similarity to the pull request for the description to be generated; based on the text vector of the pull request for the description to be generated and the description vectors of several pull requests with the highest similarity to the pull request for the description to be generated, using a description generation model to generate a description of the pull request for the description to be generated.

[0144] This embodiment discloses a method for generating a pull request description. Based on the code syntax change tree, a tree structure autoencoder is designed. Compared with using the code change part of the pull request in the form of a symbol sequence, this method fully extracts the change features contained in the pull request, can more effectively use the hierarchical information of the source code, and provide technical support for the automatic generation of pull request descriptions. For a newly created pull request, select several pull requests with the closest code changes to it to make more full use of historical data. With reference to the descriptions of several pull requests with high similarity, the text part of the newly created pull request is summarized, which can assist or replace developers in writing the description of the newly created pull request.

[0145] In this embodiment, real data obtained by crawling is used to train a tree structure autoencoder, so that the model can effectively extract the change structure information of the pull request, provide technical support for finding similar pull requests, and use the description generation model PRSim to automatically generate descriptions of newly created pull requests, simplifying the work of developers and project management teams and improving the pull request creation process.

[0146] System Example

[0147] A pull request description generation device comprises a data acquisition module, a similarity pull request calculation module, a description generation model training module and a pull request description generation module.

[0148] A data acquisition module, used to obtain pull requests from the open source community, and based on each pull request, obtain a text vector, a code syntax change tree, and a description vector of each pull request;

[0149] A similarity pull request calculation module is used to establish a tree structure autoencoder model, train the tree structure autoencoder model based on a plurality of code syntax change trees to obtain a change coding model; calculate the similarity between the pull requests based on the change coding model, and for each pull request, find a number of pull requests with the highest similarity;

[0150] A description generation model training module is used to establish an encoder-decoder model, and train the encoder-decoder model based on the text vector of each pull request, the description vectors of several pull requests with the highest similarity to the pull request, and the description of the pull request to obtain a description generation model;

[0151] A pull request description generation module is used to obtain a pull request for which a description is to be generated, and based on the pull request for which a description is to be generated, use the modified coding model to search for several pull requests with the highest similarity to the pull request for which a description is to be generated; based on the text vector of the pull request for which a description is to be generated and the description vectors of several pull requests with the highest similarity to the pull request for which a description is to be generated, use the description generation model to generate a description of the pull request for which a description is to be generated.

[0152] Since the fetch request description generating device and the pull request description generating method are based on the same inventive concept, the relevant parts can be borrowed from each other, and thus the same technical effect can be achieved.

[0153] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.< / start> < / stop> < / start> < / start> < / unk>

Claims

1. A method for generating a pull request description, characterized in that: include: Obtain multiple merged pull requests from the open source community, and obtain the text vector, code syntax change tree, and description vector of each pull request; Each of the pull requests includes: a text part, a code change part, and a description part; wherein the pull request text part includes: multiple submissions contained in the pull request, submission instructions for each submission, and comments contained in the pull request; obtaining the text vector of each pull request includes: performing text preprocessing on the text part of the pull request through a natural language toolkit; converting the preprocessed pull request text into the text vector using OneHot encoding; the text preprocessing on the text part or the description of the pull request through a natural language toolkit includes: deleting HTML tags, comments, and Markdown format control characters; deleting checklists; deleting sentences containing URLs, email addresses, internal links, user mentions, or user signatures; replacing numbers and version numbers in the text; deleting merged submissions; deleting submissions added after the pull request was created; and deleting submission tags; Establishing a tree structure autoencoder model, training the tree structure autoencoder model based on the multiple code grammar change trees to obtain a change coding model; training the tree structure autoencoder model based on the multiple code grammar change trees includes: converting the grammar change tree into a left child-right sibling representation; the encoder part collects the information of each node of the grammar change tree from bottom to top into a coding vector; the decoder part decodes the coding vector from top to bottom into a tree structure; and using the coding vector as a representative of the grammar change tree; Based on the modified coding model, calculating the similarity between the pull requests, and for each pull request, searching for a number of pull requests with the highest similarity; Establishing an encoder-decoder model, training the encoder-decoder model based on the text vector of each pull request, the description vectors of several pull requests with the highest similarity to the pull request, and the description of the pull request to obtain a description generation model; Obtain a pull request for a description to be generated, and based on the pull request for the description to be generated, use the modified coding model to search for several pull requests with the highest similarity to the pull request for the description to be generated; based on the text vector of the pull request for the description to be generated and the description vectors of several pull requests with the highest similarity to the pull request for the description to be generated, use the description generation model to generate a description of the pull request for the description to be generated.

2. The method for generating a pull request description according to claim 1, characterized in that: The obtaining of the pull request description vector includes: Performing text preprocessing on the description part of the pull request by using a natural language toolkit; Obtaining a pull request template of the software project to which the pull request belongs, and using the Smith-Waterman algorithm to calculate a matching ratio between the preprocessed pull request description and the template; When the matching ratio reaches or exceeds a threshold, the portion of the preprocessed pull request description that matches the template is removed; The pull request description after removing the portion matching the template is converted into the description vector using OneHot encoding.

3. The method for generating a pull request description according to claim 1, wherein: The acquisition of the code syntax change tree includes: Parse the code before and after the change into abstract syntax trees respectively; Use the GumTree algorithm to calculate the correspondence between the abstract syntax tree nodes before and after the change; Based on the corresponding relationship, the change type of the abstract syntax tree is determined, and a change tree node is generated based on the change type. The change tree nodes are combined into a tree shape according to the structure of the abstract syntax tree before the change to form a code syntax change tree; wherein the change types include update, delete, insert, move, and no change.

4. The method for generating a pull request description according to claim 1, wherein: The step of establishing a tree structure autoencoder model and training the tree structure autoencoder model based on the multiple code grammar modification trees to obtain a modified coding model includes: The encoder uses a tree-structured long short-term memory network Tree-LSTM to aggregate the information of each node of the grammatical change tree into an encoding vector from bottom to top; The decoder uses two linear long short-term memory networks (LSTMs) to decode the encoding vector from top to bottom into a tree structure. Based on the multiple code grammar change trees, a self-supervised training method is used so that the tree structure can restore the input code grammar change tree.

5. The method for generating a pull request description according to claim 4, characterized in that: The calculating the similarity between the pull requests comprises: encoding two pull requests using the tree structure autoencoder model to obtain two encoding vectors corresponding to the two pull requests respectively, calculating the cosine of the angle between the two encoding vectors, and judging the similarity between the two pull requests based on the size of the cosine of the angle.

6. The method for generating a pull request description according to claim 1, wherein: The encoder-decoder model includes: The encoder uses two bidirectional long short-term memory networks Bi-LSTM to encode the text vector of each pull request and the description vectors of several pull requests with the highest similarity to the pull request into vectors respectively; The context vectors obtained by encoding the description vectors of several pull requests with the highest similarity to each of the pull requests are weighted by the corresponding similarity, and the context vectors obtained by encoding the text vector of the pull request are merged into one vector using the LogSumExp function; The decoder uses a beam search method and a unidirectional long short-term memory network with an attention mechanism and a pointer generation network to decode the fused vector into a description of the pull request to be generated.

7. A pull request description generating device, characterized in that: include: A data acquisition module, used to obtain pull requests from the open source community, and based on each pull request, obtain a text vector, a code syntax change tree, and a description vector of each pull request; Each of the pull requests includes: a text part, a code change part, and a description part; wherein the pull request text part includes: multiple submissions contained in the pull request, submission instructions for each submission, and comments contained in the pull request; obtaining the text vector of each pull request includes: performing text preprocessing on the text part of the pull request through a natural language toolkit; converting the preprocessed pull request text into the text vector using OneHot encoding; the text preprocessing on the text part or the description of the pull request through a natural language toolkit includes: deleting HTML tags, comments, and Markdown format control characters; deleting checklists; deleting sentences containing URLs, email addresses, internal links, user mentions, or user signatures; replacing numbers and version numbers in the text; deleting merged submissions; deleting submissions added after the pull request was created; and deleting submission tags; A similarity pull request calculation module is used to establish a tree structure autoencoder model, and train the tree structure autoencoder model based on multiple code grammar change trees to obtain a change coding model; training the tree structure autoencoder model based on the multiple code grammar change trees includes: converting the grammar change tree into a left child-right brother representation; the encoder part collects the information of each node of the grammar change tree from bottom to top into a coding vector; the decoder part decodes the coding vector from top to bottom into a tree structure; the coding vector is used as a representative of the grammar change tree; based on the change coding model, the similarity between the pull requests is calculated, and for each pull request, a number of pull requests with the highest similarity are found; A description generation model training module is used to establish an encoder-decoder model, and train the encoder-decoder model based on the text vector of each pull request, the description vectors of several pull requests with the highest similarity to the pull request, and the description of the pull request to obtain a description generation model; A pull request description generation module is used to obtain a pull request for which a description is to be generated, and based on the pull request for which a description is to be generated, use the modified coding model to search for several pull requests with the highest similarity to the pull request for which a description is to be generated; based on the text vector of the pull request for which a description is to be generated and the description vectors of several pull requests with the highest similarity to the pull request for which a description is to be generated, use the description generation model to generate a description of the pull request for which a description is to be generated.

Images